Step 2 of 12 · 1 min

The napkin first

2M clips uploaded a day, average 45 seconds
200M clip views a day, average 30 seconds watched

As uploaded, at ~8 Mbps:
  45 s × 8 Mbps = 45 MB per clip
  2M × 45 MB = 90 TB of uploads a day

If every view streamed at 8 Mbps:
  30 s × 8 Mbps = 30 MB per view
  200M × 30 MB = 6 PB a day of delivery

Six petabytes a day is far beyond anything lesson 1.3 considered. At even a few cents per gigabyte, that's hundreds of thousands of dollars a day. Two facts make video affordable:

• Most viewers don't need 8 Mbps. A 6-inch phone screen on a mobile network looks fine at a fraction of that bitrate, if the video is encoded well for that size.
• Views are wildly uneven. A small share of clips gets most of the views (lesson 2.4's skew), and those can be cached at the edge, as photos were.

Everything in this lesson is about turning those two facts into a system.

Step 3 of 12 · 2 min

Processing uploads

Uploading reliably

A 45 MB upload over a mobile network will often be interrupted. Lesson 2.3's tools apply directly: the phone uploads straight to object storage with a signed URL, in parts (multipart upload), so an interrupted upload resumes from the last finished part rather than starting over. When the upload completes, storage emits an event, and the processing pipeline begins.

Transcoding into a ladder

The uploaded file is the original. Nobody watches it. Instead, it's transcoded: decoded and re-encoded into a set of versions, each at a different resolution and bitrate. This set is called a bitrate ladder:

Each rung is encoded with a codec (the compression method). The choice is a trade-off between compression and compatibility:

• H.264 plays on virtually every device made in the last decade. Chirp encodes every clip in H.264, so everyone can watch.
• Newer codecs (HEVC, VP9, AV1) give the same visual quality at noticeably lower bitrates, often 30–50% lower, but not every device can decode them, and they take far more computing power to encode. Chirp additionally encodes popular clips in AV1, for devices that support it. Spending extra encoding effort only pays off for videos many people will watch.

That "spend more effort on popular videos" idea goes further: some large services analyse each video and choose a different ladder per video (a static talking head needs far fewer bits than a fast-moving cricket match). Netflix described this as per-title encoding.

Doing it in parallel

Encoding a 3-minute 1080p clip into five rungs can take several minutes on one machine, longer for newer codecs. Users expect their clip to be watchable within seconds of posting.

The trick is to split the video into chunks, a few seconds each (at points where the video can be cut cleanly), encode the chunks in parallel on many workers, and join the results. A clip that would take five minutes on one machine takes about ten seconds on thirty. The work is described as a small pipeline of tasks: split; encode each chunk at each rung (in parallel); join each rung; make thumbnails; extract audio; run safety checks. Tasks are placed on a queue (lesson 4.1), and the pipeline is tracked as a state machine so a failed chunk is retried alone, idempotently, without redoing the rest.

Chirp also encoded the lowest rungs first, and published the clip as soon as they were ready. Viewers in the first minute might only get 480p; higher rungs appear as they finish.

Step 4 of 12 · 2 min

Playing smoothly: segments and adaptive bitrate

Priya's buffering came from the prototype serving one big file at one fixed quality. When her train's connection dropped from 6 Mbps to 1 Mbps, an 8 Mbps video couldn't keep up, so the player kept running out of video to show.

Modern video delivery fixes this with two ideas that work together.

Segments and a manifest

Each rung is cut into segments, short files of a few seconds each (2 to 6 seconds is common), aligned so that segment 7 of the 480p rung covers exactly the same moment as segment 7 of the 1080p rung. Alongside them is a small text file, the manifest (or playlist), listing the available rungs and where to find each one's segments.

The two widely used formats for this are HLS (HTTP Live Streaming, from Apple) and MPEG-DASH. Both deliver video as ordinary files over ordinary HTTP, which means any CDN can cache them like photos (lesson 2.3), and no special video servers are needed.

Adaptive bitrate

The player downloads the manifest, then fetches segments one at a time, and chooses the rung for each segment based on how things are going:

• It measures how fast recent segments downloaded.
• It watches its buffer: how many seconds of video it has downloaded but not yet shown.
• If downloads are fast and the buffer is healthy, it moves up a rung for the next segment. If downloads slow down or the buffer is draining, it moves down.

Because segments line up across rungs, switching is seamless: segment 7 at 720p is followed by segment 8 at 360p, and the picture just gets a little softer for a few seconds instead of stopping. This is adaptive bitrate streaming (ABR).

Players tune this carefully. Switching too eagerly makes quality jump up and down; switching too cautiously wastes a good connection. Starting low makes videos begin almost instantly (the first segment at 360p is small), then the player climbs as it learns the connection's speed. That fixed the slow-start complaint too. For a feed of short clips, players also preload the first segment or two of the next clip while the current one plays.

Step 5 of 12 · 2 min

Delivering it: the CDN, at video scale

Segments are cached at CDN edges exactly like lesson 2.3's photos, and the same ideas carry over, with bigger numbers:

• Popular clips are served almost entirely from edge caches. A viral clip watched 5 million times in a day might be fetched from Chirp's storage only a few hundred times, once per edge per rung.
• The long tail is the hard part. Most clips are watched a handful of times, usually soon after posting, then rarely. Those mostly miss at the edge. Tiered caching (lesson 2.3's regional mid-tier) helps; so does keeping recent clips in a warm regional cache, because views cluster in the first days.
• Pre-positioning: for content you know will be popular (a big match highlight posted by the cricket board), push its segments to edge caches before the traffic arrives, instead of waiting for the first viewers in each city to miss.

The bill, revisited

With an adaptive ladder, the average delivered bitrate on phones came out around 1.2 Mbps rather than 8:

200M views × 30 s × 1.2 Mbps ≈ 0.9 PB a day of delivery

Still enormous, but a seventh of the naive number. AV1 for popular clips cut it by another fifth. A high CDN hit ratio means most of that is served from the edge at CDN prices rather than cloud egress prices (lesson 2.3). And Chirp, at this scale, negotiated with CDNs and placed caches inside some large Indian internet providers' networks, which large video services commonly do.

Storage

Each clip is stored several times over: the original, five rungs in H.264, and sometimes AV1, plus thumbnails. Uploads were 90 TB a day before transcoding. Storage grows by petabytes a year. Two habits keep it under control:

• Storage classes (lesson 2.3): rungs for clips nobody has watched in months move to cheaper, slower storage. If someone does watch one, the first segments are fetched from the slower tier a little more slowly.
• Deleting what isn't needed: originals of old, unpopular clips can sometimes be dropped once the rungs exist, or the high rungs of rarely watched clips removed and re-created on demand. Each of these is a trade-off between storage cost and the ability to re-encode later with a better codec.

Step 6 of 12 · 1 min

Everything around the video

The rest of Clips reused the course:

• View counts are lesson 4.5: events into a log, counted in batches, HyperLogLog for unique viewers, rounded on display.
• The Clips feed is lesson 4.4's feed, with videos instead of posts.
• Metadata (title, author, durations, which rungs exist, processing state) lives in Chirp's sharded Postgres (lesson 5.1); the video bytes never touch the database.
• Safety checks (scanning for prohibited content) run as steps in the processing pipeline, and a clip isn't published until they pass.

Live video, briefly

Live streams (a creator broadcasting a match reaction) add a time constraint: segments are created continuously as the stream happens, and viewers want to be only a few seconds behind. With normal 6-second segments, viewers end up 15 to 30 seconds behind the live moment. Low-latency variants of HLS and DASH use much smaller pieces of segments, published as they're encoded, to get that down to a few seconds. For two-way, real-time video (calls), a different family of technology (WebRTC) is used instead, with latency under a second. Chirp's first version of Clips didn't include live video.

Step 7 of 12 · 1 min

From the real world: 25 million people, one match

Video-on-demand traffic is large but smooth. Live events are different, and India has some of the most extreme examples. In July 2019, Hotstar (now JioHotstar) reported 25.3 million concurrent viewers for the India–New Zealand semi-final of the Cricket World Cup, a world record at the time, and its engineers have spoken publicly about how they prepare for such matches.

The traffic shape is the problem. Viewers don't arrive gradually. They arrive in waves: when a star batsman walks out, when a wicket falls, when a push notification goes out to millions of phones at once. A few million extra viewers can arrive within a minute or two. Hotstar's engineers have explained that ordinary autoscaling can't react that fast: new servers take minutes to boot and warm up, and the cloud provider's capacity has to be there too.

So they work the other way round:

• Pre-scale to a forecast. Before each big match, they estimate peak concurrency (lesson 1.3, done seriously, with past matches as data) and bring the whole stack up to that level ahead of time.
• Load-test the forecast, with generated traffic at full scale.
• Plan graceful degradation. "Panic" switches turn off non-essential features (recommendations, some personalization, heavy API calls) so that the one thing that matters, the video, keeps playing. That's lesson 6.1's load shedding, decided in advance.
• Shape your own spikes. A push notification to 100 million phones is a self-inflicted traffic wave; send it in batches.

The video itself is mostly the CDN's problem, which is why Hotstar spreads traffic across several CDNs. The harder part is everything around it: logins, entitlement checks, the scorecard, ads, and chat.

Step 8 of 12 · 1 min

Going deeper: why live is half a minute behind

If you've watched a match on your phone while a neighbour watched on TV, you've heard them cheer before you saw why. Most of that delay is built into how segmented streaming works.

Add up the stages for classic HLS with 6-second segments:

1. Encoding: a second or two to encode the incoming feed into several bitrates.
2. The segment itself: a segment can't be published until it's complete, so the newest segment is up to 6 seconds old when it appears.
3. Delivery: a little more to reach the CDN edge and the player.
4. The player's buffer: players traditionally wait until they have about three segments before starting, so they can ride out network hiccups. That's another 18 seconds.

Total: around 25 seconds behind live. The levers:

• Shorter segments (2 seconds) bring it to about 10 seconds, at a cost: more requests per viewer to the CDN, slightly worse compression (each segment must start with a full keyframe), and less buffer to absorb bad networks.
• Low-latency HLS and low-latency DASH publish small parts of a segment (a few hundred milliseconds each) as they're encoded, and let players ask for the next part before it exists (the CDN holds the request until it does). That brings delay down to a few seconds while still using ordinary CDNs.
• WebRTC gets under a second, but needs very different infrastructure and scales less easily to millions of viewers.

Every option trades delay against smoothness: less buffer means more stalls when a phone's connection wobbles. Good players adjust the target delay per viewer, and for sport, many services offer lower latency only to viewers on good connections.

Step 9 of 12 · 1 min

What Chirp built

• Resumable multipart uploads directly to object storage.
• A processing pipeline on a queue: split, encode chunks in parallel, join, thumbnails, safety checks, with retries per chunk.
• A five-rung H.264 ladder for every clip, low rungs first; AV1 added for popular clips.
• HLS segments and manifests, served through the CDN, with adaptive bitrate players that start low and preload the next clip.
• Tiered caching, warm regional caches for recent clips, and pre-positioning for expected hits.
• Storage classes for old clips.
• View counts, feeds and metadata built from Parts 2 to 5.

Priya watched clips on the Mumbai–Pune train for a week during testing. She reported that the picture "sometimes gets blurry for a bit, but it doesn't stop," which is exactly what adaptive bitrate is designed to do.

Step 10 of 12 · 1 min

If you're here for an interview

"Design YouTube" or "design TikTok" is a staple. Strong answers:

• Start from bandwidth and storage numbers: video is mostly a delivery-cost problem.
• Describe the upload pipeline: resumable direct uploads, then transcoding into a ladder, in parallel chunks, driven by a queue.
• Explain segments, manifests and adaptive bitrate (HLS or DASH), and why switching rungs prevents buffering.
• Use the CDN well: hit ratio for popular videos, tiered caching for the long tail, pre-positioning for expected hits.
• Keep the database for metadata, and reuse earlier patterns for counts and feeds.

Step 11 of 12 · 2 min

Try these yourself

Step 12 of 12 · 1 min

Further reading

• RFC 8216, HTTP Live Streaming, and the MPEG-DASH overview from the DASH Industry Forum.
• Netflix Technology Blog, on per-title encoding and on chunk-based parallel encoding.
• Ilya Grigorik, High Performance Browser Networking, for how video delivery interacts with TCP and CDNs.
• Any introduction to adaptive bitrate algorithms (buffer-based and throughput-based), such as the BOLA paper by Spiteri, Urgaonkar and Sitaraman (2016).
• Hotstar's announcement of the 25.3 million concurrent-viewer record (July 2019), and the talk "Scaling Hotstar.com for 25 million concurrent viewers" (AWS re:Invent 2019).
• Apple, "Enabling Low-Latency HLS" (developer documentation); RFC 8216, HTTP Live Streaming.