Step 2 of 14 · 1 min

The basics, briefly

An API is the set of requests a client (Chirp's mobile app, its website) can send to the server, and what it gets back. Chirp, like most web services, uses a style called REST, where everything is a resource with a URL, and HTTP methods say what to do with it:

GET    /v1/posts/981          read a post
POST   /v1/posts              create a post
PATCH  /v1/posts/981          change part of a post
DELETE /v1/posts/981          delete a post
GET    /v1/users/4821/posts   list a user's posts

The server answers with a status code that says how it went:

That last column is the important one, and it leads straight to Priya's double post.

Step 3 of 14 · 3 min

Problem 1: retries that do things twice

Safe and idempotent

HTTP methods come with two promises that are easy to skim past and hard to live without:

• A safe method doesn't change anything. GET is safe. You can call it as often as you want.
• An idempotent method has the same effect whether you call it once or five times. PUT and DELETE are idempotent: setting a post's text to "hello" five times leaves it as "hello," and deleting a post five times leaves it deleted.

POST is neither. Every POST /v1/posts creates a new post. Send it twice and you get two.

Now look at a timeout from the app's point of view. It sent a request and never heard back. There are three possibilities:

1. The request never reached the server.
2. The server got it, but crashed or failed before finishing.
3. The server finished successfully, and the response got lost on the way back.

The app can't tell which one happened. For a GET, it doesn't matter: just try again. For a POST, case 3 means retrying creates a duplicate, and case 1 means not retrying loses the post. There's no safe choice, unless the server can recognize a retry when it sees one.

Idempotency keys

That's what an idempotency key does. Before sending the request the first time, the app generates a random unique ID for this particular attempt to post, and sends it in a header:

POST /v1/posts
Idempotency-Key: 6f1c2a9e-3b7d-4f0a-9c51-8e2d7a4b1f03
Content-Type: application/json

{ "text": "Sunset from the train", "photo_id": "ph_77812" }

If the app retries, it sends the same key. The server keeps track of keys it has seen:

• New key: process the request normally, then save the key along with the response.
• Key it has seen and finished: don't do anything. Just send back the saved response from the first time.

From Priya's point of view, the retry "succeeded," and it returned the same post the first request had created. One sunset.

Two details are easy to get wrong.

Two copies of the request can arrive at the same moment. If the app retries while the first request is still being processed, both see "new key" and both create a post. The fix is to claim the key first, using the database's uniqueness guarantee from lesson 1.4:

CREATE TABLE idempotency_keys (
    user_id     bigint,
    key         text,
    status      text,         -- 'in_progress' or 'done'
    response    jsonb,
    created_at  timestamptz DEFAULT now(),
    PRIMARY KEY (user_id, key)
);

The server tries to insert the key with status in_progress before doing any work. Only one insert can succeed, because of the primary key. The request that wins does the work, saves the post, and stores the response. A request that loses the race finds the key already there: if it's done, it returns the stored response; if it's still in_progress, it returns 409 Conflict and the app tries again shortly.

Even better, save the post and mark the key done in the same transaction, so there's never a moment where the post exists but the key says it doesn't.

Keys shouldn't live forever. Retries happen within seconds or minutes, so keeping keys for a day is plenty, and a nightly job can delete older ones.

This pattern is common for anything where doing it twice is bad: payments are the classic example. Stripe's API popularized it, and there's an IETF draft standardizing the Idempotency-Key header. You'll see the same idea again in Part 4, applied to messages between servers instead of requests from phones.

Retrying politely

While they were at it, the team fixed how the app retried. The first version retried failed requests immediately, up to five times. During a short outage, every phone with a failed request retried at once, then again, then again, which hit the server with several times its normal traffic just as it was trying to recover.

The standard fix is exponential backoff with jitter: wait a little before the first retry, roughly double the wait each time, and add some randomness so phones don't all retry at the same instant.

retry 1: wait ~0.5s (random between 0 and 1s)
retry 2: wait ~1s   (random between 0 and 2s)
retry 3: wait ~2s   (random between 0 and 4s)
then give up and show an error

And only retry when it can help: timeouts, 429, and 5xx errors. Never retry a 400, because the request is wrong and will be wrong again. Lesson 6.1 covers retries in much more depth.

Step 4 of 14 · 2 min

Problem 2: page 2 showed posts from page 1

The second complaint arrived the week after launch. Scrolling down a busy profile, people kept seeing the same posts twice.

The first version of the API paginated like this:

GET /v1/users/4821/posts?page=2     →  posts 21–40

Behind it was:

SELECT * FROM posts WHERE author_id = 4821
ORDER BY created_at DESC
LIMIT 20 OFFSET 20;

Picture someone reading page 1 of a profile. While they're reading, the user posts twice more. When they ask for page 2, "skip the first 20" now skips the 2 new posts plus 18 old ones, so posts 19 and 20 from page 1 appear again at the top of page 2. With deletes, it goes the other way and posts get skipped entirely.

There's a second problem that only shows up on big tables. OFFSET 10000 doesn't jump to row 10,000. The database has to find and step over 10,000 rows before returning anything. The deeper someone scrolls, the slower every page gets.

Cursors

The fix is to page by position instead of by count. Each response includes a cursor: a marker for where the page ended.

GET /v1/users/4821/posts?limit=20

{
  "posts": [ ... ],
  "next_cursor": "eyJ0IjoiMjAyNi0wNi0xNFQxODo0MjowMVoiLCJpZCI6OTgxfQ"
}

The app passes it back to get the next page:

GET /v1/users/4821/posts?limit=20&cursor=eyJ0IjoiMjAyNi0wNi0xNFQxODo0MjowMVoiLCJpZCI6OTgxfQ

Inside, that cursor is just the created_at and id of the last post on the page, encoded. The server turns it into this query:

SELECT * FROM posts
WHERE author_id = 4821
  AND (created_at, id) < ('2026-06-14 18:42:01', 981)
ORDER BY created_at DESC, id DESC
LIMIT 20;

"Give me the 20 posts that come after this one." New posts arriving at the top don't change what comes after post 981, so nothing repeats or goes missing. And with the (author_id, created_at) index from lesson 1.4 (extended to include id), the database jumps straight to that spot in the B-tree, so page 500 is as fast as page 1.

Two details:

• Include a tiebreaker. Two posts can have the same created_at. Without id in the cursor, a page boundary between them could skip one.
• Make the cursor opaque. Encoding it (here as base64) instead of exposing ?after_time=...&after_id=... means apps just pass it back without understanding it. Then you can change what's inside it later without breaking any app.

What you give up: you can't jump to "page 37" directly. For feeds and timelines, nobody wants to. For things like search results where people do jump around, offset paging is sometimes still the right choice, as long as the data doesn't change much while people browse.

Step 5 of 14 · 1 min

Problem 3: the home screen made seven requests

Lesson 1.1 ended with Chirp folding several calls into one, and this is where that idea comes from.

The first API was designed around Chirp's database tables: an endpoint for posts, one for users, one for notifications, one for settings. That's tidy, but opening the app meant the home screen called seven of them, several of which needed the result of an earlier one before they could start:

GET /v1/me                          → who am I?
GET /v1/feed                        → post IDs
GET /v1/posts?ids=...               → post contents
GET /v1/users?ids=...               → authors of those posts
GET /v1/notifications/unread_count
GET /v1/settings
GET /v1/feature_flags

From Mumbai, every one of those steps that had to wait for a previous one added about 200ms. An API that's convenient for the server can be slow for the app.

The team added one endpoint shaped around what the home screen actually needs:

GET /v1/home

{
  "me": { ... },
  "feed": [ { "post": {...}, "author": {...} }, ... ],
  "unread_notifications": 3,
  "next_cursor": "..."
}

One round trip instead of a chain of them. The server makes the same database calls as before, but they're half a millisecond away from the server instead of 200ms away from the phone (lesson 1.2).

The general rule is chunky, not chatty, especially for mobile apps on slow networks. Design the main screens' endpoints around what the screen needs, rather than making the app assemble it from small pieces. Some teams formalize this with a separate API layer per client (often called a backend for frontend), or use GraphQL, which lets the app describe exactly the data it needs in one query.

Step 6 of 14 · 2 min

Problem 4: you can't take an API back

Three months after launch, the team wanted to rename the text field in posts to body, to make room for rich formatting. On the website, which the team controls, they could change both sides at once. The mobile app was another story.

A new app version reaches people only when they update, and many people don't. Months later, a meaningful share of Chirp's users were still on the very first version. If the API stopped sending text, every one of those phones would show blank posts.

So an API, once real apps use it, is a contract. The rules that keep it working:

Adding is safe; removing and renaming are not. New fields, new endpoints, and new optional parameters don't break anyone, as long as apps ignore fields they don't recognize. That rule needs to be in the app from version one, because you can't add it to old versions later. So instead of renaming text, Chirp added body alongside it and kept sending both.

Breaking changes get a new version. When a change truly can't be made compatibly, it goes into a new version (/v2/...), and /v1 keeps working, unchanged, for as long as old apps use it.

Know who's still on old versions. Every request from the app includes its version in a header, so the team can see exactly how much traffic still uses an old field or endpoint. You can't retire something safely without knowing who still needs it.

Keep a way to force an update. For the rare cases when an old version is truly broken or unsafe, the app checks a minimum supported version at startup and asks the user to update. It's a last resort, because it's a bad experience, but you want it built into version one so it's available when you need it.

Errors are part of the contract too

The first API returned errors however each piece of code happened to produce them: sometimes plain text, sometimes JSON with a message, sometimes an HTML error page from the proxy. The app couldn't reliably tell what had gone wrong. Chirp settled on one shape for every error:

{
  "error": {
    "code": "post_too_long",
    "message": "Posts can be at most 500 characters.",
    "retryable": false
  }
}

The code is for the app to act on and never changes. The message is for people and can be reworded freely. And retryable tells the app directly whether trying again could help, instead of leaving it to guess from status codes.

Step 7 of 14 · 1 min

REST, GraphQL, or gRPC?

You'll be asked about this, so briefly:

• REST over HTTP with JSON is the default for public and mobile APIs: simple, readable, works everywhere, and plays well with caches and tooling.
• GraphQL lets the client ask for exactly the fields it needs, across related objects, in one request. That's great for apps with many screens needing different data, and it solves the chatty-API problem from another direction. In return, the server is more complex, caching is harder, and you have to protect against very expensive queries.
• gRPC uses a compact binary format and a strict schema, and runs over HTTP/2. It's fast and well suited to services talking to each other inside your own system. It's less convenient for browsers and public APIs.

Chirp uses REST for its apps. If it ever splits into several internal services, gRPC between them would be a reasonable choice. None of this changes the four problems above, which every style has to solve in some form.

Step 8 of 14 · 1 min

From the real world: one flag, two meanings

Problem 4 said you can't take an API back. The most expensive example of that isn't a public API at all. It's a flag passed between two pieces of one company's software.

On 1 August 2012, Knight Capital, then one of the largest traders on US stock exchanges, deployed new code for a program to handle retail orders. The code ran on eight order-routing servers, and was copied to them by hand. A technician missed one. Server 8 kept running the old code.

The new code turned itself on with a flag in incoming orders. That flag had a history: years earlier it had switched on "Power Peg," old test logic that buys and sells to move prices. Power Peg had been unused since 2003, but its code was still in the program, and a later refactor had moved away the part that told it to stop once an order was filled.

At 9:30 the market opened. On seven servers, the flag meant the new retail program. On server 8, it meant Power Peg, and Power Peg kept sending orders without ever noticing they'd been filled. Engineers tried to fix it by rolling back the new code on the other seven servers, which made it worse: now all eight treated the flag as Power Peg. In 45 minutes, Knight's systems made about 4 million trades in 154 stocks. The SEC later put the loss at more than $460 million. Knight needed emergency funding and was acquired within months.

There are several lessons (manual deploys, dead code, no kill switch), but the one for API design is this: a field whose meaning changes is a breaking change, even if its name and type stay the same. Old readers don't fail. They do the wrong thing, confidently.

Step 9 of 14 · 2 min

Going deeper: what really breaks old readers

Chirp's apps will run old versions for years, so every response the server sends must make sense to every app version still out there. Which changes are actually safe depends on what goes over the wire.

In protobuf (and Thrift, and Avro with some differences), fields are identified by numbers, not names. The message is a series of "field 3 = 42" entries. That makes some changes free and others deadly:

• Adding a field with a new number is safe. An old reader sees field 5, doesn't know it, and skips it. Protobuf even keeps unknown fields so that a message passing through an old service isn't stripped.
• Renaming a field is safe, because names never go over the wire.
• Reusing a number for something new is never safe. Old readers will decode the new value with the old meaning, and nothing will complain. That's Knight Capital's bug. Protobuf has a reserved keyword so the compiler stops anyone from reusing a retired number or name.
• Changing a field's type is mostly unsafe. Some changes happen to decode (an int32 read as int64), but most produce garbage.

JSON is the opposite: names are the contract. Renaming is a breaking change, and there are no numbers to reuse. The equivalent rules are:

• Readers ignore fields they don't know (the "tolerant reader"). If your app crashes on an unexpected field, every server change becomes risky.
• Never change what an existing field means, even if you keep its name and type. If price was in rupees, it stays in rupees; add price_paise instead.
• Enums grow. An app that handles "text" and "photo" will one day receive "video". Write clients so an unknown value degrades gracefully (show a placeholder) instead of crashing.

There's a law about all this, named after Hyrum Wright at Google: with enough users, every observable behaviour of your API will be depended on by somebody, whether you promised it or not. The order of items in a list, the exact wording of an error, how long a request takes. Go's designers saw programs come to depend on the order in which maps were iterated, which was never guaranteed, so they made iteration order deliberately random to stop it. When you design Chirp's API, assume that anything you let people see is something you'll have to keep.

Step 10 of 14 · 1 min

What Chirp changed

• Added idempotency keys to every request that creates something, with retries using exponential backoff and jitter.
• Replaced page numbers with cursors everywhere a list can change while someone's reading it.
• Added a /v1/home endpoint shaped around the home screen.
• Adopted firm rules: only additive changes within a version, apps ignore unknown fields, every request reports its app version, and a minimum-version check exists for emergencies.
• Standardized the error format.

Step 11 of 14 · 1 min

In the simulation

Stage 0's request log shows each API call the app makes, its status, and whether it was a retry. Watch it on a simulated slow mobile network: you'll see timeouts and retries happen on their own, and whether your design turns them into duplicates. The cursor and idempotency ideas come back in Part 4, where servers talk to each other through queues and every message might be delivered twice.

Open the simulation →

Step 12 of 14 · 1 min

If you're here for an interview

Most design interviews include a few minutes of "define the API." A list of endpoints is fine, but it's an opportunity to show judgment:

• For anything that creates data or moves money, mention idempotency keys and why: the client can't tell a lost request from a lost response.
• For any list, use cursor pagination and say why offsets break when new data arrives.
• For mobile clients, mention round trips, and that the main screen should load in one or two calls.

A sentence on each is usually enough, and together they show you've thought about how real clients behave, which is exactly what the interviewer is checking.

Step 13 of 14 · 2 min

Try these yourself

Step 14 of 14 · 1 min

Further reading

• RFC 9110, HTTP Semantics, sections on safe and idempotent methods and on status codes.
• Stripe's API documentation on idempotent requests, a clear real-world description of the pattern.
• The IETF draft "The Idempotency-Key HTTP Header Field."
• Marc Brooker, "Exponential Backoff And Jitter," on the AWS Architecture Blog.
• Markus Winand, "We need tool support for keyset pagination" and the pagination chapter of Use The Index, Luke!
• US Securities and Exchange Commission, order in the matter of Knight Capital Americas LLC (2013). The official account of the deploy, the reused flag and the losses.
• Protocol Buffers documentation, "Updating A Message Type" and "Reserved Fields."
• Hyrum Wright, "Hyrum's Law" (hyrumslaw.com), and Titus Winters et al., Software Engineering at Google, chapter 1.