Planet-Scale Video Streaming — a system design walkthrough

01Functional requirements

Six verbs, and only six.

Write the contract down before you draw a single box. Everything on the board exists to serve one of these, and anything that serves none of them is scope creep.

Verb	What it actually means
Upload	Accept a mezzanine file up to ~1 TB, resumable, over a lossy consumer link. Durability from the first byte.
Transcode	Turn one master into every rendition, codec and container the fleet of client devices can decode.
Stream	Deliver segments over plain HTTP so every cache, proxy and CDN on earth can help.
Seek	Jump to an arbitrary timestamp and show a frame fast. This is why segments are independently decodable.
Resume	Continue on a different device at the same offset. A tiny write path, a huge product feature.
Multi-device	One title, many decoders: an eight-year-old TV, a flagship phone, a browser, a set-top box.

Deliberately out of scope Recommendations, search, billing, social. They are large systems, but they are not this system; folding them in is the classic way to run out of interview time with nothing built.

The board on the right is one drafting table with three working areas: the route chart, the transcode bench, and the ABR strip chart. Press I at any time for a keyboard index of every object on it.

02Non-functional targets

Four numbers you will be judged on.

Quality of experience for video is not one metric. It is a small set, and they trade against each other in ways you have to say out loud.

SLO	Target	Why that number
Video start time	p50 ≤ 400 ms p99 ≤ 1.2 s	Abandonment climbs measurably past about two seconds. p99 is the one that hurts: it is the cold cache, the far PoP, the bad link.
Rebuffer ratio	≤ 0.20%	Stall seconds ÷ play seconds. One stall in a 90-minute film is already 0.02%. Past ~0.5% viewers quit sessions.
Playback availability	99.99%	~52 min/yr of the playback path. Upload and transcode can be slower and less available; nobody is watching them.
Delivered cost	≤ $0.001 / GB	The one that shapes the architecture. At 20 EB per month, a tenth of a cent per gigabyte is $20 M.

The honest tension Startup time wants short segments and a low starting rendition. Rebuffer ratio wants a deep buffer. Cost wants the lowest bitrate that still looks good, served from the cheapest tier. You cannot maximise all three; you pick a point and defend it.

03Back of the envelope

Do the arithmetic first. It picks the architecture for you.

Every figure below is live: move the sliders in the dock and the numbers, the board and the readout all move together.

Storage: one source hour, across the whole ladder

Sum of active rung bitrates34.4 Mbps

× 3600 s ÷ 8 bits15.5 GB per codec family

× codec families (H.264 / HEVC / AV1)3

+ audio, subtitles, trick-play sprites~0.4 GB

Encoded footprint per source hour46.8 GB

Ingest: the daily firehose

500 h/min × 1440 min720,000 source hours / day

× encoded footprint33.7 PB / day of new encoded video

Mezzanine masters kept at ~120 GB/h+86 PB / day, cold tier

Year one, encoded only12.3 EB

Egress: the bill

Peak concurrent streams40.0M

× average delivered bitrate4.5 Mbps

Peak egress180 Tbps

Monthly bytes at a 0.35 average-to-peak ratio20.5 EB

Edge-served 64% at $0.0004/GB$5.3M

Regional and shield served at $0.0012/GB$8.6M

Reached origin, at $0.0200/GB$5.3M

Cache fill at $0.0020/GB — 5.0 Tbps, always on$3.2M

Delivery, per month$22.4M

The floor nobody budgets for Every byte in every cache got there from origin exactly once, so the encode rate itself is a hard floor under origin egress — roughly 5 Tbps flowing outward forever, before a single viewer has asked for anything. Caching changes how many times a byte leaves origin; it cannot change that it leaves at least once.

Transcode: the other bill

CPU per source hour, full ladder, 3 codecs~58 core-hours

× 720,000 source hours / day ÷ 24 h1.74M cores, steady state

What that number is telling you Roughly two million cores running flat out, forever, just to encode. This is precisely why YouTube designed and deployed its own transcoding ASIC (Argos VCU) and why every large platform obsesses over encoder efficiency: a 10% bitrate saving is worth tens of millions a year on both bills at once.

04Why you cannot just serve the file

The master is a 1.4 Gbps file. Your viewer has 3 Mbps.

A camera master is not a delivery format and was never meant to be one. ProRes 422 HQ at 4K/30 runs about 880 Mbps; ProRes 4444 XQ passes 1.4 Gbps. Even a lightly compressed 1080p mezzanine sits around 50 Mbps. Serving that file is not "expensive", it is arithmetically impossible for almost every viewer on earth.

Axis of variation	Range you must actually cover
Throughput	0.4 Mbps rural cellular → 900 Mbps fibre. Three orders of magnitude, and it changes mid-session.
Screen	360×640 phone → 3840×2160 television. Sending 4K to a phone burns bytes nobody can see.
Codec support	H.264 everywhere; HEVC on most TVs and Apple devices; AV1 on recent silicon only; VP9 on Android and Chrome.
Container / DRM	HLS wants fMP4 or TS; DASH wants fMP4. Widevine, PlayReady and FairPlay each want their own key delivery.
Power	A phone decoding AV1 in software drains battery fast enough that shipping H.264 to it is the kinder choice.

The consequence One upload fans out into dozens of encoded outputs. That fan-out is the transcode bench on the board, and it is the single largest compute workload the company runs.

The dock below sets the viewer's link. Watch which rungs of the ladder are reachable at all.

05Chunked parallel transcoding

Cut it up, encode it everywhere at once, glue it back.

Encoding is embarrassingly parallel along the time axis, and that is the whole trick. Split the master at keyframe boundaries, hand each chunk to a different worker, and the wall-clock time stops depending on the length of the film.

Title length2 h = 7,200 s

÷ chunk length5 s → 1,440 chunks

One chunk, one rung, on one core~20 s

Workers assigned1,440

Waves needed1

Wall clock, encode + stitch27 s (serial: 8.0 h)

The three things that go wrong

GOP alignment. Every chunk must begin with an IDR frame, and every rendition must place its IDR frames at identical presentation timestamps. If they drift, a player switching rendition mid-stream either stalls or shows a glitch, because there is no common point to splice at. Force a fixed GOP, or drive all encodes from one shared keyframe list.
Scene-change boundaries. A per-chunk encoder cannot see past its own edges, so a cut landing 200 ms into a chunk gets a wasted keyframe right after the forced one, and the encoder's lookahead is starved at both ends. Detect scene changes globally first, then cut chunks on those cuts rather than on a fixed grid.
Encoder state does not cross the seam. Rate control, VBV fill and two-pass statistics are per-process. Independent chunks drift to different quality levels and you get visible pumping every few seconds. The fix is a cheap global analysis pass whose statistics are distributed to every worker, so all of them target the same quality rather than the same bitrate.

Stitching Concatenation is metadata work, not pixel work: rewrite timestamps, splice the segment index, verify no gaps or overlaps, then run an automated QC pass (PSNR/VMAF against the master, black-frame and audio-drift checks) before the title is publishable.

06The bitrate ladder

A staircase the player can walk up and down.

Each rung is the same content at a different resolution and bitrate, cut into segments at the same timestamps. The player never needs to know anything about your encoders; it only needs a list of rungs and their bitrates.

Rung	Bitrate	Where it earns its keep
240p	0.30 Mbps	The floor. Bad cellular, and the rung you start on to hit the startup SLO.
360p	0.60 Mbps	Congested mobile, background playback, data-saver modes.
480p	1.0 Mbps	The largest rung most of the world's median connection sustains reliably.
720p	2.5 Mbps	The volume rung. Most delivered bytes live here or at 1080p.
1080p	5.0 Mbps	The quality expectation on a laptop or a mid-size TV.
1440p	9.0 Mbps	Only worth encoding if a meaningful share of viewers can pull it.
2160p	16 Mbps	Small audience, large bill. Its storage and CPU cost is a third of the whole ladder.

Per-title, then per-shot A fixed ladder is wrong for almost every title. A talking-head interview is visually simple and looks flawless at 1080p/2.4 Mbps; a confetti-and-explosions action sequence is starved at 5. Netflix's per-title encoding runs a bitrate/quality sweep per title and picks the ladder that hits a target VMAF, then per-shot encoding does it again for every shot within the title. Reported savings: roughly 20–30% of bytes at equal measured quality. Toggle it in the dock and watch both bills fall.

07Packaging: HLS and DASH

Plain files on plain HTTP. That is the entire protocol.

There is no streaming server. There is a text manifest listing segment URLs, and there are segments. Every CDN, proxy and browser cache on earth already knows how to serve that, which is exactly why it won.

Piece	HLS	DASH
Manifest	`.m3u8`, plain text, a master playlist of rungs plus one media playlist each	`.mpd`, XML, with `SegmentTemplate` so URLs are derived, not listed
Segments	fMP4 (was MPEG-TS)	fMP4
Encryption	SAMPLE-AES / cbcs → FairPlay	CENC cenc or cbcs → Widevine, PlayReady

Why 2 to 6 seconds per segment

Shorter means faster startup, faster reaction to a bandwidth change, and less wasted download when the player switches rung. It also means more requests, more manifest bytes, and worse compression: every segment starts with an expensive IDR frame, so 1-second segments can cost 10–15% more bitrate for the same quality.
Longer compresses better and cuts request overhead, but the player commits to a rendition for the whole segment. A 10-second segment means up to 10 seconds of stuttering before ABR can react.
Four seconds is where most large services land. Live is different: low-latency CMAF streams 200 ms parts inside an open segment via chunked transfer, which is how you get sub-3-second glass-to-glass latency without giving up HTTP.

CMAF is the free win Before CMAF you stored one set of segments for HLS and another for DASH: double the storage, double the origin fill, and half the cache hit ratio because each edge held two copies of the same pictures. CMAF is one fMP4 segment set with two thin manifests over it. cbcs common encryption then lets a single encrypted copy satisfy FairPlay, Widevine and PlayReady. Turn it off in the dock to see both bills jump.

2 h title, 4 s segments1,800 segments per rung

× active rungs × codecs × container sets37,800 objects for one film

08Adaptive bitrate

The whole control loop lives on the client.

The server is dumb on purpose. The player measures, decides, and requests; the origin never knows what rendition anyone chose. Two signals feed the decision, and using only one of them is the classic mistake.

Throughput estimate. Bytes ÷ time for recent segments, combined with a harmonic mean over the last few, not an arithmetic one. Rates average harmonically; an arithmetic mean lets one fast segment mask three slow ones.
Buffer occupancy. How many seconds of decoded video you are holding. This is the only signal that tells you how much time you have to be wrong in.

The rule, stated exactly

// runs once per segment boundary
est   = harmonicMean(throughput[-5..])      // Mbps
safe  = 0.85 * est                          // safety factor
target = highestRung(bitrate <= safe)

if (buffer < 10s)                   switchTo(target)   // panic: obey throughput
else if (target < current)          switchTo(target)   // drop instantly, never wait
else if (target > current
      && buffer > 20s
      && next.bitrate <= 0.90 * est)  stepUp(1)         // climb one rung at a time
else                               hold()

The rule above has a hole in it It only runs at segment boundaries. If the link collapses halfway through a 1080p segment, the player is blind until that download finishes — and it will not finish, because the link collapsed. Production players therefore also watch the in-flight request: if the projected completion time is later than the buffer can cover, they abandon the download and re-issue at a rung that fits. Drop the slider to 0.4 Mbps and watch the log; without abandonment that single stuck segment is the stall.

Why the asymmetry Dropping is free and immediate; climbing is a bet. A drop costs a few seconds of softer picture, a wrong climb costs a stall, and the metric you are judged on counts stalls. So: fall fast, climb slowly, and never climb without buffer to absorb being wrong. Buffer-based schemes (BOLA) go further and derive the rung directly from buffer level, which is provably near-optimal and much more stable on cellular.

Drag the bandwidth slider in the dock. The strip chart plots the link, the gauge shows the buffer draining and refilling, the marker walks the ladder, and if you starve it long enough the buffer empties and you get a genuine rebuffer event in the log.

09The CDN hierarchy

Four tiers, and each one exists to protect the next.

Tier	Where	What it holds
Edge PoP	Inside the ISP, or at the local exchange. Hundreds to thousands of sites.	Tens to a few hundred TB. Today's popular titles, in the rungs that region actually watches.
Regional cache	A handful of large IX sites per continent.	Petabyte class. The whole active catalogue for that continent.
Origin shield	One logical tier in front of origin.	Little storage. Its real job is deduplication: collapse many edge misses for the same object into one origin fetch.
Origin	Object storage, two or more regions.	Everything, forever. Should serve a low single-digit percentage of bytes.

Getting the viewer to the right edge

Anycast BGP. One address announced from every PoP; the internet's own routing picks the nearest. Simple, instantly reconvergent when a site withdraws, but you get whatever BGP thinks is near, not what is actually fast, and a route flap can break an in-flight TCP connection.
DNS steering. Resolve per client, hand back the address of a chosen PoP. Full control over the choice, but you are steering the resolver, not the user, and TTLs mean 30–60 seconds of stale answers during a failover.
Client-side steering. Hand the player a ranked list of candidate URLs at session start, derived from real measured throughput for that ISP, that ASN, that time of day. The player can fail over to the next entry in milliseconds without any DNS or routing change. This is what Netflix's Open Connect actually does, and it is strictly better if you control the client.

Fill before anyone asks Popularity is predictable a day ahead. Every night, during the local trough, each edge pulls tomorrow's predicted top-N for its own region over links that are otherwise idle. By the time the peak arrives the bytes are already local, so peak-hour origin traffic is near zero and the expensive fill happened at the cheapest hour. Proactive fill is the difference between a CDN and a cache.

taut cord = tier carrying load slack cord = headroom long cord = distance, and therefore latency

10Cache economics

Hit ratio is a logarithm, and the bill is a hyperbola.

Video popularity follows a Zipf-like law: rank a title i and its request share is proportional to i^−α, with α usually measured between 0.7 and 1.0 for video-on-demand. With a cache holding the top C of N titles and perfect popularity ordering, the hit ratio is simply the ratio of two generalised harmonic numbers:

hit(C) = H(C, α) ÷ H(N, α)H(n,α) = Σ i^−α

Catalogue200,000 titles, 10.4 GB of hot renditions each

Edge cache180 TB → 17,346 titles

Skewα = 0.85

Edge hit ratio64.0%

The hot footprint is not the whole ladder: an edge keeps the rungs its own region watches, in the one or two codec families its devices decode. That is why the number moves when you change the ladder, the codecs or CMAF.

The shape is what matters. The first terabyte of cache buys enormous hit ratio; the hundredth buys almost none. The curve printed on the board is that function, with a marker at your current operating point — drag the sliders and watch where you sit on the knee.

The long tail is real, and it is expensive Chasing the last few percent of hit ratio means storing titles that are requested once a week per PoP. That is why the tail is deliberately not cached at the edge: it is cheaper to pay the regional cache's backhaul for it than to buy the disks to hold it everywhere. Every tier exists because there is a popularity band where it is the cheapest place to keep a byte.

Second-order effects worth naming Zipf assumes independent requests; in reality they arrive in correlated bursts, which helps (temporal locality beats the static model). Against that, cache admission matters as much as eviction: admitting a one-hit-wonder evicts something valuable, which is why production caches gate admission behind a frequency filter such as TinyLFU rather than admitting on first touch.

11The thundering herd

Midnight. A season drops. Every cache on earth misses at once.

The cold-start case is not a gentle ramp. At 00:00 local, a few million players all request segment 1 of the same title within a few seconds. No edge has it. Every edge forwards. Every regional forwards. Origin, sized for a low single-digit percentage of traffic, receives essentially all of it.

Edges in the fleet~1,600

Concurrent misses per edge, first 10 s~3,000

Naive origin requests for one object4.8 M

The three fixes, in the order you should reach for them

Request coalescing (single-flight). Within one cache node, all concurrent misses for the same key wait on one in-flight fetch. This is a dozen lines of code around a map of pending requests and it collapses 3,000 requests to 1. Nginx calls it proxy_cache_lock; Varnish calls it request coalescing.
Origin shield. Point every edge at one shield tier instead of at origin. The shield coalesces across edges, so the fleet's 1,600 remaining requests become one. Two levels of coalescing turn 4.8 M into 1.
Pre-positioning. The best request is the one that never happens. A scheduled drop is known days ahead; push it to every edge before midnight and the herd has nothing to stampede toward.

The failure that gets you paged Without coalescing, origin saturates, latency climbs, edges time out, and the timeouts trigger retries — which are new requests. The load goes up because the system is failing. Any retry path here needs a cap, exponential backoff and jitter, or you have built a self-sustaining outage.

Press Release a blockbuster in the dock, watch origin buckle, then switch coalescing on and watch it recover.

12Failure modes

Three failures worth designing for, and what each one costs.

Failure	Blast radius	Mitigation and its cost
PoP outage	Every viewer steered to that site. Sessions in flight stall within seconds.	Re-steer to the next nearest PoP. Client-side URL fallback is instant; BGP withdrawal takes seconds; DNS takes a TTL. Latency rises for everyone re-homed, and the neighbouring PoP absorbs load it was not provisioned for — so you must run every PoP under N−1.
Origin loss	No new fills. Cached content keeps playing; anything cold is unservable.	Multi-region origin with async replication, and a shield that keeps serving its own cache. Because the CDN absorbs 95%+ of reads, origin can be down for a while before viewers notice — which is exactly why it is often the least-hardened tier, and why that is a mistake.
Partial ladder	One rung's encode failed or its objects are missing. Players requesting it stall or error.	Publish the manifest from the rungs that actually exist. A title available at 240–720p is a degraded experience; a title whose manifest lists a 1080p rung that 404s is an outage. Validate object presence before publish, and treat manifest generation as the commit point.

The general principle Every degradation should have a shape the player can live with. Fewer rungs, a further PoP, a longer startup — all survivable. What kills a session is an inconsistency: a manifest that promises something the storage layer cannot produce.

13Your turn

Write the ABR controller yourself.

Everything in chapter 08 is now your problem. Your choose() runs once per segment against a fixed four-minute session: a link that sits around 5.5 Mbps, collapses to half a megabit for about thirty seconds, then recovers to 24. Unlike the production player above, this one cannot abandon an in-flight download — whatever you request is fetched to the end, so a greedy pick at the wrong moment becomes the stall. The graded run replays on the strip chart: the trace, the rendition marker, the buffer gauge and every alarm tick are your code's doing.

The two levers, and why hysteresis Throughput headroom decides which rung is safe: never buy at 100% of the estimate, because it is a smoothed guess about a link that changes mid-segment — leave ~20%. Buffer occupancy decides how much risk you can carry: seconds in hand are the only time you have to be wrong in. And switch with hysteresis — drop as far as needed at once, but climb one rung at a time and only with buffer to absorb a mistake, or the marker saws up and down and every reversal is wasted bytes and visible churn.

14Recap

Every decision, what it bought, what it cost.

Decision	Bought	Cost
Segmented HTTP, not a streaming protocol	Every cache on earth becomes infrastructure you did not build. Firewalls and proxies already allow it.	Latency floor of roughly one segment. Live needs chunked CMAF to claw it back.
Client-side ABR	Stateless servers, trivially scalable; the client has the only signals that matter.	You cannot fix a bad player from the server. Every client platform reimplements the loop, and they disagree.
Chunked parallel transcode	Wall time decoupled from title length. A 2-hour film in under a minute.	GOP alignment, scene-change handling and cross-chunk rate control all become your problem.
Per-title / per-shot encoding	20–30% fewer bytes at equal measured quality. Saves on both bills at once.	An analysis pass per title, a much more complex pipeline, and ladders that differ per title.
Deep bitrate ladder	Every device gets a rung it can actually play; ABR has room to manoeuvre.	Storage and CPU scale with rung count. 2160p alone is about a third of both.
CMAF + cbcs	One segment set for HLS and DASH, one encrypted copy for three DRMs. Storage halves, hit ratio rises.	Drops the oldest devices that only speak TS or cenc-only DRM.
Owned edge appliances	Delivered cost falls roughly 10× versus commercial CDN egress. At 20 EB/month that is the difference between viable and not.	Capital, logistics, hardware refresh, and ISP relationships in every market you serve.
Origin shield + coalescing	A 4.8 M-request herd becomes one origin fetch. Origin is sized for steady state, not for launch night.	One more tier to operate, and one more hop of latency on a true cold miss.
Proactive nightly fill	Peak-hour origin traffic near zero; expensive fill moved to the cheapest hour.	Needs a popularity prediction, and it wastes bandwidth whenever the prediction is wrong.
Buffer-guarded ABR (fall fast, climb slow)	Rebuffer ratio under 0.2%, which is the metric that actually correlates with viewers leaving.	Average delivered quality is measurably lower than a throughput-greedy player would achieve.

If you remember one thing At this scale the architecture is not chosen by latency or by availability. It is chosen by cost per delivered gigabyte, and every tier, every cache, every encoder improvement is a lever on that one number.