System Design Road

Part 8 · Lesson 8.3

And What If…?

Common follow-up questions10 steps · 10 min

Where we are: lesson 8.1 said the deep dive is where most of an interview is decided, and that the interviewer's questions tell you what they want to see. This lesson collects the follow-up questions that come up most often, grouped by theme, with what each is really testing and a short, strong answer.

What you’ll learn

  • The follow-up questions that come up in almost every design interview
  • What each one is really testing
  • A short, strong answer for each, and the lesson that goes deeper
  • How to answer a question you’ve never seen
Lesson trace

Lesson 8.3 And What If…?

Step 1 of 10 · 1 min · The story

During Sana's practice runs, Meera kept a list of every follow-up question she asked. After six sessions, the same thirty or so questions had come up again and again, in different forms, whatever the original prompt was. "Design a chat app" and "design a ride-sharing app" sound different. "What happens when this server dies?" is the same question in both.

That's this lesson: the recurring questions, and how to answer them. Read the question, try answering it out loud in about a minute, then open the answer. Each answer is deliberately short; the linked lesson has the full reasoning, and in an interview, you'd expand on whichever part the interviewer seems most interested in.

Step 2 of 10 · 1 min

When things fail

"What happens if this server dies?" Tests: whether your design has single points of failure, and stateless servers.

App servers are stateless behind a load balancer, across zones, so a dead one is removed by health checks and the others carry on (lessons 2.1, 2.2). State lives in replicated stores. Name anything that isn't replicated, and fix it. Mention how in-flight requests are handled: retried by the client with idempotency keys (lesson 1.5).

"What if the database leader fails?" Tests: replication, failover, and data-loss awareness.

A replica is promoted (automatically, by a majority-based failover system so there can't be two leaders), and writes are redirected (lessons 2.5, 5.4, 6.5). With asynchronous replication, the last moments of writes can be lost. Wait for at least one replica in another zone if that's unacceptable. Old leaders are fenced so they can't accept stale writes.

"What if a dependency is slow?" Tests: timeouts, cascading failure.

Every call has a timeout and a share of the request's overall deadline. A circuit breaker stops calling a failing dependency, bulkheads stop it using up all workers, and a fallback keeps the page useful without it (lesson 6.1). A slow dependency is more dangerous than a dead one, because it ties up resources.

"What if a whole region goes down?" Tests: multi-region thinking, honesty about trade-offs.

Traffic moves to other regions (DNS with a short TTL, or anycast); replicas there are promoted for the affected data. Cross-region replication is usually asynchronous, so a few seconds of writes may be lost. Surviving regions need spare capacity, and nothing must depend on a single region, especially things like authentication (lesson 5.5).

"What if the cache goes down or restarts empty?" Tests: understanding that caches become load-bearing.

Every read now hits the database, possibly many times its usual load. Plan for it: replicated cache nodes, rolling restarts, warming before switching traffic, limiting concurrent database queries, and serving stale data where acceptable (lesson 2.4).

Step 3 of 10 · 1 min

Scale

"Traffic grows ten times. What breaks first?" Tests: whether you can find the bottleneck with numbers.

Redo the estimates at 10× (lesson 1.3) and compare with each component's limits (lesson 8.2). Usually it's database writes, then a hot key or partition, then bandwidth. Say which, and the next step: add replicas and caching for reads, partition for writes, a CDN for bandwidth.

"How would you shard this?" Tests: choosing a partition key from access patterns.

List the main queries, pick the key that keeps the most frequent ones on one shard, and name what becomes cross-shard (lesson 5.1). Use many logical shards mapped to fewer machines so you can add capacity later without moving everything (lesson 5.2). Mention global uniqueness (a separate table) and hot shards.

"One item is wildly more popular than the rest. Now what?" Tests: hot keys and skew.

On reads: short-lived local caches on app servers, replicated copies of the hot key, CDN for anything public. On writes: separate the action from the total (one row per like), shard or batch counters, and choose partition keys that spread the hot item's events (lesson 4.5). For feeds, the celebrity is pulled at read time rather than pushed (lesson 4.4).

"How do you handle a sudden spike, like the end of a big event?" Tests: bursts, queues, autoscaling limits.

Queues absorb bursts so work happens at a sustainable pace (lesson 4.1). Autoscaling is too slow for sudden spikes, so pre-scale for known events (lesson 2.1). Load shedding and rate limits protect the core when demand exceeds capacity (lessons 6.1, 6.3), and features degrade gracefully.

Step 4 of 10 · 2 min

Data and consistency

"SQL or NoSQL?" Tests: choosing by access pattern rather than reputation.

Write down the access patterns first (lesson 3.1). Relational when you need joins, constraints and transactions; wide-column for huge append-heavy writes read by key and time range; documents for self-contained objects; a search engine for text. It's often both: relational for core entities, plus a specialized store for one pattern.

"A user posts, refreshes, and their post is gone. Why?" Tests: replication lag and read-your-writes.

The write went to the leader; the read went to a replica that hadn't applied it yet. Fix with read-your-writes: read from the leader for recent writers, or track the write's log position and only use replicas that have reached it (lessons 2.5, 5.3).

"Two people buy the last ticket at the same moment. What happens?" Tests: race conditions and atomic operations.

If you check availability and then write, both can succeed. Make the check and the claim one atomic operation on the single place that owns the data: a conditional update or a unique constraint, on the leader (lessons 1.4, 5.3). For high contention, split the inventory into pools or queue buyers.

"How do you keep the cache (or search index) in sync with the database?" Tests: dual writes, invalidation, change data capture.

For caches: delete on write after the transaction commits, keep a TTL as a safety net, and watch for the stale-set race (lesson 2.4). For search and other copies: avoid dual writes from the app. Use change data capture from the database's log, with idempotent, versioned updates (lesson 3.4).

"What consistency do you need here?" Tests: choosing per feature.

Go feature by feature and pick the weakest guarantee each can live with: linearizable for uniqueness and money, read-your-writes for a user's own content, causal for conversations, eventual for counts and feeds (lesson 5.3). Weaker guarantees are cheaper and stay up during failures.

"Isn't this a CP or an AP system?" Tests: whether you understand CAP precisely.

CAP is about what to do during a network partition: refuse some requests or risk inconsistency. Choose per operation, not per system: username claims refuse; likes are accepted and reconciled later (lesson 5.4). In normal operation, the everyday trade-off is latency versus consistency (PACELC).

Step 5 of 10 · 1 min

Reliability of processing

"What if this message is processed twice?" Tests: at-least-once delivery and idempotency.

Queues deliver at least once, so consumers must be idempotent: record message IDs in the same transaction as the work, or design operations that are naturally safe to repeat (lessons 4.1, 4.2). For external effects, pass an idempotency key to the other system.

"How do you make sure the event is published if the database write succeeds?" Tests: the dual-write problem.

Write the event into an outbox table in the same transaction as the change, and have a relay publish it to the queue or log (lesson 4.2). Consumers de-duplicate by event ID.

"The client times out. Does it retry? What if the first request worked?" Tests: idempotency keys.

A timeout means the outcome is unknown. Clients send an idempotency key, generated before the first attempt and reused on every retry, so the server returns the first result instead of acting twice (lesson 1.5). For payments, also query the status rather than guessing (lesson 7.3).

"How do you make sure only one instance runs this job?" Tests: locks, leases, fencing, idempotency.

Prefer making the job idempotent, so two runs don't matter. If you need a single runner, use a lease from a consensus-backed system (etcd, ZooKeeper) with fencing tokens checked by whatever the job writes to, because a paused holder can't tell its lease expired (lessons 2.1, 6.5).

"How do you generate unique IDs?" Tests: ID schemes and their trade-offs.

State the requirements (size, sortable, generated where, guessable or not), then compare auto-increment, UUIDv4, UUIDv7 and Snowflake (lesson 4.3). Mention unique machine IDs and clocks going backwards for Snowflake, and random keys' effect on index inserts.

Step 6 of 10 · 1 min

Performance

"How do you make this faster?" Tests: whether you measure before optimizing.

Find where the time goes first: traces and percentiles, not averages (lessons 6.2, 6.4). The usual wins: fewer round trips and batching (lesson 1.2), a cache with a good hit ratio (lesson 2.4), an index (lesson 1.4), a CDN or a nearer region for distant users (lessons 2.3, 5.5), and doing slow work asynchronously (lesson 4.1).

"Your p99 is bad but p50 is fine. Why, and what do you do?" Tests: tail latency.

Fan-out multiplies rare slowness, and garbage-collection pauses, queueing, background work and cache misses all add to the tail. Reduce the number of calls, hedge requests after the p95 wait (for safe-to-repeat reads), keep slow machines out of rotation, and return partial results within a deadline (lesson 6.2).

"How would you design the feed?" Tests: fan-out on write vs read.

Push for normal accounts (precomputed feeds), pull for very large accounts, merged at read time: a hybrid. Mention the numbers behind it, the celebrity problem, feed storage limits, and that deletes and blocks are enforced at read time (lesson 4.4).

Step 7 of 10 · 1 min

Abuse, limits and operations

"How do you stop abuse of this endpoint?" Tests: rate limiting design.

Rate limits by account, API key, device and target (not just IP), using a token bucket in a shared store, with 429 and Retry-After (lesson 6.3). Add challenges or review for sign-ups, and decide whether each limit fails open or closed if the limiter is down.

"How do you know the system is working?" Tests: observability and SLOs.

Define SLIs from the user's side (the share of requests that succeed quickly), set SLOs with error budgets, and alert on how fast the budget is burning, not on CPU (lesson 6.4). Use metrics for detection, traces to find where, logs to find why.

"How do you deploy this without downtime?" Tests: rolling deploys and graceful shutdown.

Rolling deploys behind the load balancer: mark a server not ready, drain its connections, shut down gracefully, start the new version, wait until ready, move on (lesson 2.2). Changes to APIs and data are backward compatible (lesson 1.5), and long-lived connection servers are restarted gradually with jittered reconnects (lesson 7.1).

"How do you change the database schema on a huge table?" Tests: online migrations.

Make additive, backward-compatible changes first; backfill in small batches while watching replication lag (lesson 2.5); switch reads and writes in steps; remove the old column last. Use tools or database features that avoid long locks.

Step 8 of 10 · 1 min

From the real world: where follow-ups come from

Many of the follow-up questions in this lesson aren't invented by interviewers. They're the questions that real outages taught the industry to ask. If you know the incidents behind them, you can answer with both the principle and the evidence.

  • "What does your health check actually test?" Stack Overflow's health check loaded the home page; one post with 20,000 spaces made the home page slow, and the load balancer took every server out of rotation (lesson 2.2).
  • "What happens when a dependency gets slow and everyone retries?" DynamoDB's storage servers retried their metadata requests, which kept the metadata service overloaded for hours after the trigger had passed (lesson 6.1).
  • "What happens to in-flight writes when you fail over?" GitHub's 43-second network partition led to a cross-country failover and a day of reconciling writes that existed on only one side (lesson 5.4).
  • "How do you know your backups work?" GitLab's backups had been failing silently, and its alerts were being rejected, until the night it needed them (lesson 2.5).
  • "What if your configuration push is wrong?" Cloudflare's regex rule reached every server in seconds and pinned every CPU (lesson 1.2); Facebook's DNS servers all followed the same correct-looking rule at once (lesson 1.1).
  • "What does your monitoring depend on?" Roblox's monitoring depended on the very system that had failed (lesson 6.4); AWS's internal monitoring shared a network with the problem (lesson 5.5).

Mentioning an incident works best briefly and accurately: one sentence on what happened, one on the mechanism, then back to your design. "This is how Stack Overflow went down in 2016: the health check rendered the home page. So my readiness check only tests local dependencies, and the load balancer fails open if every server looks unhealthy." That shows you've learned from the industry's mistakes, not just its diagrams.

Step 9 of 10 · 1 min

When you haven't seen the question before

You will get questions that aren't on this list. A reliable way through:

  1. Restate it to make sure you understand what's being asked.
  2. Ask what could go wrong: what fails, what's slow, what's contended, what's duplicated or lost.
  3. Connect it to a pattern you know: most "new" problems are a familiar one in disguise. A shared to-do list is lesson 5.3's conflicts; a leaderboard is lesson 4.5's hot counters; a "who's watching this match" count is lesson 4.5's HyperLogLog.
  4. Give a simple answer first, then refine it with numbers and trade-offs.
  5. Say what you'd measure to know whether it worked.

It's also fine to say "I'm not sure, but here's how I'd reason about it." Interviewers value clear reasoning under uncertainty much more than a confident guess.

Step 10 of 10 · 1 min

Further reading

  • The "If you're here for an interview" sections at the end of every lesson in this course, which go deeper on each theme.
  • Martin Kleppmann, Designing Data-Intensive Applications, for most of the reasoning behind these answers.