System Design Road

Medium Interview walkthrough

News feed: the interview, played out

The home feed is the screen you see the most and think about the least. Behind it is one big decision: build each feed when people post, or when people read? We’ll start with the simple answer, watch it break, and end up where real social apps do.

10 min read6 interactive diagrams12 sections

This gives away every stage. For a clean attempt, practise it first, then come back.

The short version

  • Reads beat writes 25 to 1, so do the work when someone posts, and make opening the feed a single lookup.
  • Copying every post into every follower’s feed is affordable, except for celebrities.
  • Celebrities’ posts are pulled at read time and merged in. That’s the hybrid design.
  • Store only post IDs in capped lists, and page with a cursor, not page numbers.
  • Deleted posts and unfollows are filtered out when reading, not removed from millions of lists.

The 45 minutes

Before we start

Open Instagram, X or LinkedIn and the first thing you see is your feed: recent posts from people you follow. It loads almost instantly, every time, for hundreds of millions of people.

The naive way to build it is obvious: when you open the app, find everyone you follow, grab their recent posts, sort them by time. That works for a small app. The interview is about what happens at scale, and especially what happens when someone with ten million followers posts.

Same order as always: requirements, quick maths, data and API, a design that grows, then deep dives.

Minutes 0–5Requirements

Interviewer

Interviewer: Design the home feed for Chirp. Users follow other users, and when they open the app they see recent posts from the people they follow.

You

You: How many users, and how often do they open the feed?

Interviewer

Interviewer: 50 million people a day, each opening it about 10 times.

You

You: How many posts are written a day, and how many people does a typical user follow?

Interviewer

Interviewer: About 20 million posts a day. People follow around 200 accounts on average.

You

You: Are there accounts with huge followings?

Interviewer

Interviewer: Yes, some have over 10 million followers.

You

You: Is the feed newest first, or ranked by what’s interesting?

Interviewer

Interviewer: Newest first for now. Ranking if there’s time.

You

You: How quickly must a new post show up for followers?

Interviewer

Interviewer: Within a few seconds is fine.

You

You: And if someone deletes a post, how fast must it disappear?

Interviewer

Interviewer: Within seconds too.

You

You: Got it. Let me write this down.

Functional requirements

  1. Users can create posts with text and images.
  2. Users can follow and unfollow other users.
  3. Users see a feed of recent posts from people they follow, newest first.
  4. Users can keep scrolling to see older posts.

Out of scope today: comments, likes and direct messages. They’re separate features.

Non-functional requirements

  1. Fast: the feed loads in under about 200 ms.
  2. Fresh enough: new posts appear within a few seconds. It’s fine if two followers see a post a second apart.
  3. Celebrity-proof: accounts with millions of followers work too.
  4. Scale: 50 million daily users, 500 million feed loads and 20 million posts a day.
  5. Reading stays up: the feed keeps loading even if posting has problems.

Minutes 5–10Back-of-the-envelope estimates

You

You: 50 million people opening the feed 10 times is 500 million loads a day, about 5,800 a second, maybe 17,000 at peak.

You: 20 million posts a day is only about 230 a second. So reads beat writes roughly 25 to 1.

You: Now the key question: what if every post is copied into the feed of every follower? 20 million posts times 200 followers is 4 billion copies a day, about 46,000 a second. That’s a lot, but a cluster of Redis servers can handle it.

You: If each feed keeps just the IDs of the latest 800 posts, at 8 bytes per ID, all 50 million feeds take about 320 GB. That fits in memory on a handful of machines.

You: But one celebrity post with 10 million followers means 10 million copies at once. That’s the problem to solve.

WhatWorkingResult
Feed loads50M users × 10 opens ÷ 86,400 s≈ 5,800 a second (busiest ~17,000)
New posts20M ÷ 86,400 s≈ 230 a second
Copying every post20M posts × 200 followers ÷ 86,400 s≈ 46,000 copies a second
Feed memory50M users × 800 IDs × 8 bytes≈ 320 GB
One celebrity post10 million followers10 million copies at once

The numbers tell a clear story. Reads are 25 times more common, so it’s worth doing extra work when someone posts to make reading cheap. That works for normal accounts. It breaks for celebrities.

Minutes 10–15Core entities and the API

Core entities

  • User: who they are.
  • Post: author, text, image, time. Its ID sorts by time (we’ll see why).
  • Follow: who follows whom. We’ll need to ask it both ways: “who do I follow?” and “who follows this author?”
  • Feed: for each user, a short list of post IDs, newest first. Not the posts themselves, just their IDs.

The API

GET/v1/feed?limit=20&cursor=7280

Your home feed, newest first. The cursor says where the last page ended.

Responses
  • 200{ "posts": [ { "id": 7301, "author": "priya", "text": "Match day!", "image": "https://cdn.chirp.app/i/55.jpg" } ], "next_cursor": "7280" }
POST/v1/posts

Create a post. It reaches followers’ feeds in the background.

Body
{ "text": "Match day!", "image_id": "img_55" }
Responses
  • 201{ "id": 7301 }
POST/v1/users/{id}/follow

Follow someone. Send DELETE to the same address to unfollow.

Responses
  • 204Done.

Notice the feed uses a cursor, the ID of the last post you saw, instead of a page number. We’ll see why in the deep dives. (Lesson 1.5 compares the two.)

Minutes 15–25The design, built up

Stage 1Build the feed when it’s opened

AppApp serverPostgresposts + followsGET /feedposts by people I followOne query joins follows and posts.Fine for thousands of users. Slow for millions.
91%
You

You: The simplest version: one app server and one Postgres database. When you open the feed, one query finds everyone you follow and fetches their recent posts, sorted by time.

This is called pull, or building the feed on read. It meets every requirement for a small app, and posting is as cheap as it gets: one insert. But every feed load does work for every account you follow. Follow 200 people, and each of 17,000 feed loads a second touches 200 people’s posts. The database spends all day doing the same sorting over and over.

Stage 2Precompute feeds: copy each post to followers

AppLoad balancerPost serviceFeed servicereads + mergesPosts storeNew-post eventsNEWFan-out workerscopy ID to every followerNEWFollow graphNEWFeed cache800 IDs per userNEWpostread feedsavewho follows?push ID1. my feed3. post details
61%
Green: new in this stage
You

You: Since reads are 25 times more common than writes, let’s do the work at write time instead. When someone posts, we save the post and drop a “new post” event on a queue. Fan-out workers pick it up, look up the author’s followers, and add the post’s ID to the front of each follower’s feed list in Redis, keeping the newest 800.

You: Now opening the feed is one quick read of your own list, plus fetching those posts’ details in one batch. That’s how we hit 200 ms.

This is called push, or building the feed on write. Posting costs more, but posting is rare, and it happens in the background, so the person posting doesn’t wait. Lesson 4.4 shows the pipeline step by step:

The write path
Post servicesaves the post
Posts DB + outboxpost and event in one transaction
Outbox relaypublishes the event
Fan-out queuedelivers at least once
Splitterbreaks followers into chunks of 5K
Two lanesnormal accounts / big accounts
Fan-out workerspipelined ZADD + trim
Feeds in Rediscapped sorted sets

One transaction saves the post and its outbox row. Everything after that runs in the background and is safe to retry.

Stage 3Celebrities: pull their posts when the feed is read

AppLoad balancerPost serviceFeed servicereads + mergesPosts storeNew-post eventsFan-out workersskip celebritiesFollow graphFeed cache800 IDs per userCelebrity postsrecent, per authorNEWpostread feedsavewho follows?push ID1. my feed2. celebrities I follow3. post details
57%
Green: new in this stage
You

You: Push breaks for celebrities. One post from an account with 10 million followers means 10 million copies, which takes minutes and delays every ordinary post queued behind it. And many of those followers won’t open the app today anyway.

You: So, a hybrid. Accounts over a follower threshold, say 100,000, are skipped by the fan-out workers. Their recent posts go into a small list per author instead. When you open your feed, the feed service reads your precomputed list, then the recent posts of the few celebrities you follow, and merges them by time.

Try both designs:

Opening the feed: precomputed list + celebrities, merged
read
Celebrity posts10M followersYouopen the appFan-out workers10,000,000 insertsYour feed liststill waiting for itFeed serviceCelebrity listrecent posts

Reading is still one lookup, which is the upside. The cost moved entirely to posting.

Design

Reads stay fast, because you only follow a handful of celebrities. And no single post creates millions of writes. This hybrid is where large social apps end up.

Stage 4Images from a CDN

AppLoad balancerPost serviceFeed servicereads + mergesPosts storeNew-post eventsFan-out workersskip celebritiesFollow graphFeed cache800 IDs per userCelebrity postsrecent, per authorCDNimagesNEWpostread feedsavewho follows?push ID1. my feed2. celebrities I follow3. post detailsimages
57%
Green: new in this stage
You

You: Last piece: images. The feed returns image URLs, and the app loads them straight from a CDN, close to the user. The feed service never sends image bytes. (Lesson 2.3 covers CDNs.)

Checking the design against the requirements

RequirementWhere the design handles it
Create posts with text and imagesPost service saves the post; images go to storage behind a CDN
Follow and unfollowFollow graph, stored both ways
See followed users’ posts, newest firstPrecomputed list plus celebrity lists, merged by time
Keep scrollingCursor paging with time-sortable post IDs
FastOne list read plus a handful of celebrity reads, all in memory
Fresh within secondsFan-out runs in the background within seconds of posting
Celebrity-proofAccounts over a threshold are pulled at read time, not copied
Scale46,000 copies a second across Redis; 320 GB of IDs in memory
Reading stays upSeparate feed and post services; the feed reads from its own cache

Deep divePush, pull, or both?

Interviewer

Interviewer: Explain why you didn’t just pick push or pull.

Bad solutionPull for everyone

Build every feed when it’s opened. Simple, and posting is cheap. But reads are 25 times more common, and each one does work for every account the user follows. You pay the biggest cost on the most common action.

Good solutionPush for everyone

Copy every post into every follower’s feed. Reads become one quick lookup. But a celebrity post becomes 10 million writes, delaying everyone else’s posts for minutes, mostly for followers who won’t even look.

Great solutionPush for most, pull for celebrities

Push for ordinary accounts, which are almost everyone. Pull for the few huge ones, merged in when the feed is read. Two refinements interviewers like: decide “celebrity” by active followers rather than total, since many followers of big accounts rarely open the app; and avoid accounts flipping back and forth when they hover around the threshold, for example by switching only when they stay above it for a day. Lesson 4.4 shows how lopsided follower counts really are:

Where fan-out writes come from (power-law followers)
50M accounts, Zipf
0%25%50%75%100%1K3K10K32K100K316K1M3.2M10M31.6Mcelebrity threshold: followers (log scale)share of all fan-out writes from accounts above it100K+ followers: 565 accounts → 32% of writes10K+: 7K accounts → 41%

Follower counts follow a power law: a tiny number of accounts have most of the followers. In this model of 50 million accounts, about 500 accounts with over 100,000 followers cause about a third of all fan-out writes. Pulling just those at read time removes a big share of write work, while almost every read stays a single cache lookup plus a merge of a few celebrity lists. Measure your own distribution before picking the threshold.

Deep diveScrolling with new posts arriving

Interviewer

Interviewer: How does paging work when new posts keep arriving at the top?

Bad solutionPage numbers

“Give me page 2” means “skip the first 20, give me the next 20”. But if 5 new posts arrive while you read page 1, everything shifts down by 5. Page 2 now starts with 5 posts you’ve already seen.

Good solutionA timestamp cursor

“Give me 20 posts older than 10:42:07.” New posts at the top don’t matter any more. But two posts can share the same timestamp, and then one of them can be skipped or repeated at the page boundary.

Great solutionAn ID cursor, with IDs that sort by time

Give every post an ID that grows with time, like the Snowflake-style IDs from lesson 4.3. Then “give me 20 posts with IDs smaller than 7280” is exact, unique and fast: just a range read on the feed list.

Loading page 2 while new posts arrive
#50Monsoon’s here
#49New job!
#48Chai or coffee?
#47Train views
#46Weekend plans
#45Look at this cat
#44Finally finished it
#43Sunset again
#42Recipe thread
#41Match tonight
#40Road trip
#39Hello Chirp
#38Rainy Mumbai
#37Book club
Page 1
Page 2
GET /posts?page=1
GET /posts?page=2 → OFFSET 5 LIMIT 5

Pages hold five posts. Newest posts are at the top.

Page 2 skips the first five rows, but they’re different rows now. #45 and #44 show up again.

Paging

Deep diveDeletes and unfollows

Interviewer

Interviewer: A post is deleted, but its ID is already in millions of feeds. Now what?

You

You: I wouldn’t try to remove it from millions of lists right away. When the feed service fetches post details, a deleted post simply comes back missing, so it’s skipped. Same for unfollows: when building your page, drop posts from people you no longer follow. A background job can tidy the lists later.

You: That way, what you see is correct immediately, and the expensive clean-up happens slowly in the background.

This trick, filtering at read time instead of fixing every copy, comes up again and again in systems that store copies of data.

Deep diveKeeping memory under control

Interviewer

Interviewer: You said 320 GB for feeds. How do you keep that in check?

You

You: Three things.

You: Store only post IDs in the lists, 8 bytes each, never the post itself. The post is fetched once, from the post store, when it’s actually shown.

You: Cap each list at about 800 IDs. Very few people scroll that far. If someone does, we fall back to building older pages on read.

You: Only keep feeds for people who’ve used the app recently. When someone comes back after months, build their feed once on read, then keep it warm from then on.

Twitter’s timeline service worked this way in its early years: lists of tweet IDs, capped at a few hundred, kept in memory. Lesson 4.4 tells that story:

One tweet to 31 million timelines (Twitter, ≈ 2012)
hybrid
A pop star≈ 31M followersFan-out serviceskips huge accountsTimeline cachesRedis, replicated ×3Followersmerge at read time

The answer was the hybrid this lesson describes: don’t fan out for the largest accounts; merge their recent tweets into the timeline when it’s read.

Deep diveAdding ranking

Interviewer

Interviewer: Product wants the feed ranked by what’s interesting, not just newest first.

You

You: The fan-out design stays the same. Ranking sits on top of it in three steps.

You: First, gather candidates: your precomputed list plus celebrity posts, a few hundred posts. Second, score each one, using things like how often you interact with the author, how recent the post is, and how many likes it’s getting. Third, filter: no duplicates, not too many posts from one person in a row. Then return the top 20.

You: Scoring a few hundred posts on every refresh is real work, so I’d cache your ranked page for a minute or so.

The complete architecture

Here’s everything together: the full design from the last stage, with every part we added along the way. If you only remember one picture from this question, make it this one.

AppLoad balancerPost serviceFeed servicereads + mergesPosts storeNew-post eventsFan-out workersskip celebritiesFollow graphFeed cache800 IDs per userCelebrity postsrecent, per authorCDNimagespostread feedsavewho follows?push ID1. my feed2. celebrities I follow3. post detailsimages
57%

How one request travels through it:

  1. Someone posts. The post service saves it and drops a ‘new post’ event on a queue.
  2. Fan-out workers look up the author’s followers and add the post’s ID to the front of each follower’s feed list in Redis, trimming it to 800.
  3. If the author is a celebrity (over a follower threshold), workers skip this step. Their posts are kept in a small ‘recent posts’ list per author instead.
  4. When you open the app, the feed service reads your precomputed list, then fetches recent posts from the few celebrities you follow, and merges them by time.
  5. It fetches the post details in one batch, drops anything deleted or from people you’ve since unfollowed, and returns 20 with a cursor.
  6. Images load straight from the CDN.

How this answer gets scored

Interviewers don’t tick boxes, but their notes on this question tend to land on the same few moments. Here’s what each one sounds like from the other side of the table.

MomentWhat earns creditWhat costs you
The first five minutesAsking about celebrities and how often people readJumping straight to “we’ll use Redis”
The numbersComparing reads with fan-out writes, and noticing the celebrity spikeComputing storage and nothing else
Push or pullChoosing a hybrid, with a reason for the thresholdPicking push for everyone and stopping
Feed storageStoring capped lists of IDs, and sizing the memoryCopying whole posts into every feed
PagingA cursor with time-sortable IDsPage numbers
DeletesFiltering at read timeDeleting from millions of lists right away

What the left column has in common: every choice points back to a requirement or a number from earlier. If you can say where each box came from, the interviewer will notice.