How Netflix Actually Works

In August 2008, Netflix suffered a major database corruption issue that took down DVD shipping for three days. At the time, Netflix ran on a single relational database a fairly conventional setup for the era. That one outage became the seed of almost everything covered in this article: the move to the cloud, the shift to microservices, and eventually, a practice of deliberately breaking their own production systems on purpose, specifically so a real failure would never catch them off guard again.

This article looks at how Netflix is actually built today the architecture that emerged from that outage, the custom-built CDN that now delivers the overwhelming majority of the world's Netflix traffic from inside individual internet providers' own networks, and the deliberately paranoid engineering culture that treats failure as a certainty to design around rather than a rare event to merely hope against. Netflix runs one of the most detailed public engineering blogs of any company in this series, so a good amount of this draws directly from Netflix's own published writing, alongside independent technical breakdowns filling in the surrounding context.

The Outage That Started Everything

A write-up on BNXT.ai makes a point that's easy to miss when looking at Netflix's architecture today as some kind of grand master plan: none of it was designed all at once. It was built incrementally, usually as a direct response to a specific failure. The 2008 database outage forced the move to AWS. Scaling limits on the resulting monolith eventually forced the split into microservices. Rising bandwidth costs eventually forced the build-out of Open Connect, Netflix's own CDN.

A house that got its fire escape added only after an actual close call, its reinforced foundation added only after a real crack appeared, and its backup generator installed only after a real blackout each individual addition looks perfectly sensible in isolation, but the full picture only really makes sense once you understand it was assembled piece by piece, in direct response to real pain, rather than blueprinted from a single, complete plan on day one.

From One Database to Hundreds of Independent Services

A case study on Clustox describes the resulting architectural shift plainly: Netflix migrated from a monolithic architecture to a microservices-based model specifically to isolate services, reduce interdependencies between them, and enable much faster, more independent deployments. Failure, the same piece notes, is now treated as inevitable by default, and Netflix's engineering culture is built around that assumption rather than around trying to prevent every possible failure outright.

A breakdown on BunksAllowed explains a specific design discipline that makes this isolation actually work in practice: every new microservice Netflix builds is stateless by design, meaning any single instance of that service can handle any incoming request, with no request tied to one specific server that happens to be holding some unique piece of memory only it knows about. The same piece notes this is a decision that's far easier to build in from the very start than to retrofit onto an existing system later, once services have already grown comfortable quietly depending on their own local state.

A large restaurant kitchen where every station is fully stocked and every cook is trained to handle any order that comes through, rather than one specific cook being the only person who happens to know how to make a particular dish if any single cook has to step away, literally any other cook nearby can pick up exactly where they left off, because nothing critical was ever trapped inside one specific person's head.

Stateful service (harder to scale, harder to recover):
  Request → must go to the ONE server holding the relevant state
  That server goes down → those in-flight requests are in trouble
 
Stateless service (Netflix's default):
  Request → any available instance can handle it
  One instance goes down → traffic simply shifts to the others

Separating "Finding a Show" From "Streaming a Show"

Here's a genuinely important architectural split that a lot of the independent breakdowns converge on. A guide on GeeksforGeeks draws a clear line between two very different kinds of work Netflix has to do: the backend, mostly running on AWS, handles things like content onboarding, video processing, personalization, and account management everything that isn't the actual video bytes streaming into your living room. Open Connect, Netflix's own CDN, handles the video delivery itself, from whichever server happens to be nearest to you.

A summary on BNXT.ai frames why this split matters architecturally, not just organizationally: separating the control plane (the decision-making layer what to recommend, what's in your queue, whether your account is valid) from the data plane (the actual video delivery) lets each one scale, get optimized, and be changed independently of the other. The same piece is candid that this split looks obvious in hindsight, but a lot of architectures never quite get around to actually separating business logic this cleanly from data delivery.

A company that keeps its customer service call center and its physical shipping warehouse as two entirely separate operations, even though both exist to serve the exact same customer order you can add more warehouse capacity during a shipping surge without needing to hire a single additional phone operator, and you can staff up the call center during a support-heavy period without touching warehouse operations at all. That's the practical value of Netflix keeping personalization and account logic completely separate from raw video delivery.

Open Connect: Putting Servers Inside Your Own Internet Provider

This is genuinely one of the more unusual infrastructure decisions in this entire series, and it's worth understanding exactly why Netflix chose it. A case study on Clustox explains the core idea directly: rather than relying entirely on public, third-party CDNs the way most streaming platforms do, Netflix deploys its own physical servers, called Open Connect Appliances, directly inside internet service providers' own networks around the world minimizing the physical distance video has to travel, and meaningfully reducing bandwidth costs in the process.

The payoff shows up in genuinely striking numbers. The same GeeksforGeeks breakdown reports that Open Connect achieves roughly a 98% cache hit rate, meaning the overwhelming majority of what you watch is served directly from a nearby cached copy rather than reaching all the way back to Netflix's own origin servers with edge traffic measured at roughly 73 terabits per second, while origin traffic sits at only about 1.46 terabits per second as a result. A separate write-up on the Medium blog by Vishal Lokam, summarizing Netflix's own engineering writing, notes Open Connect delivers 100% of Netflix's video traffic, currently accounting for over 125 million hours of viewing every single day.

A grocery chain that doesn't just build a handful of giant central warehouses, but instead pays to place a small, well-stocked mini-fridge directly inside your own apartment building's lobby the vast majority of what you actually need never has to travel from some distant warehouse at all, because it's already sitting practically at your own front door. That's essentially Netflix's bet with Open Connect: instead of just leaning on general-purpose CDN infrastructure, physically embed the content as close to the end viewer's own network as it's possible to get.

Without Open Connect (typical CDN model):
  You hit play → request travels to a regional CDN node
    → possibly further back to an origin server
 
With Open Connect:
  You hit play → request served from an Open Connect Appliance
    → physically sitting inside your own ISP's network
  ~98% of requests never need to travel any further than that

Chaos Engineering: Breaking Things on Purpose, Before Something Else Does

This is the philosophy Netflix is arguably most famous for in engineering circles, and it flows directly from the lesson of that original 2008 outage. An academic paper on arXiv describing Netflix's architecture explains the underlying reasoning: Netflix's infrastructure is built around two main components, Open Connect for content delivery and a Control Plane for everything else and given how many interdependent microservices make up that Control Plane, the team needed a way to genuinely trust that the system could survive individual pieces failing, rather than just hoping it would.

A breakdown on VDOCipher's blog describes how this trust actually gets built: Netflix created what became known as the Simian Army a set of tools, starting with the famous Chaos Monkey, that deliberately and randomly disable production instances, extended over time with more precise tools like FIT for targeted failure injection and ChAP for automated, carefully guarded chaos experiments run directly against live production traffic.

A fire department that doesn't just wait around for a real fire to eventually happen, but instead runs regular, deliberate fire drills sometimes even setting a small, carefully controlled practice fire specifically to test whether the sprinkler system and the evacuation plan actually work the way everyone assumes they do. Chaos Monkey plays that same role for Netflix's software: randomly, deliberately killing a production instance isn't recklessness, it's a scheduled drill, run specifically so that the real, unplanned version of that same failure is already something the system has proven it can shrug off.

Traditional approach:
  Hope failures don't happen
  → When one eventually does, scramble to understand the blast radius
 
Chaos Engineering approach:
  Deliberately, randomly kill production instances (Chaos Monkey)
  Deliberately inject specific, targeted failures (FIT)
  Run automated, guarded chaos experiments on live traffic (ChAP)
  → Confirm the system survives, on a schedule, before a real failure ever tests it

The same Medium piece by Vishal Lokam explains what actually validates whether a chaos experiment succeeded: Netflix collects event data and logs from both member devices and its own first-tier services, converts that data into counters, and then directly compares the resulting metrics between a canary cluster (the one currently absorbing the deliberate failure) and a healthy baseline cluster turning "did the system survive that?" from a gut feeling into an actual, measured comparison.

Caching So the Real Databases Rarely Get Bothered

A guide on GeeksforGeeks makes a recommendation that reflects how Netflix's own systems are actually built: use a caching layer like EVCache, Netflix's own in-memory caching system, aiming for a 95 to 99% cache hit rate, specifically to protect the underlying databases from being hit by the sheer volume of everyday requests things like checking what's in your watchlist, or loading personalized recommendations for the home screen that don't actually need to touch a database fresh every single time.

A popular tourist attraction that keeps a large stack of pre-printed maps and brochures right at the entrance, so the vast majority of visitors never need to individually ask a staff member behind the counter for basic directions the staff member (the database) is still there for genuinely unique questions, but almost everyone's needs get satisfied by something already sitting close by, pre-prepared, and instantly available.

Turning "Someone Paused a Video" Into Something Netflix Can Learn From

A breakdown on Medium by Yadav Padiyar sketches out the practical path a single playback event takes through Netflix's systems: a service called Eureka helps locate the right backend microservices, those services talk to Cassandra, MySQL, and EVCache as needed, and hitting play on a title triggers a request to whichever Open Connect Appliance is nearest to you falling back to Netflix's own AWS-hosted origin servers only if that nearby cache doesn't already have what you're asking for. Smaller signals, like pausing, skipping ahead, or changing video quality mid-stream, get sent onward to Kafka and processed through Spark, feeding back into the recommendation and personalization systems that shape what you see the next time you open the app.

Putting It Together

None of the individual pieces here are exotic inventions entirely on their own. Microservices are a common architectural pattern. CDNs, even Netflix's uniquely aggressive version of one, build on a well-understood idea. Deliberately injecting failures into a live system is a documented, increasingly mainstream engineering discipline. What makes Netflix's story genuinely compelling isn't any single piece of technology it's the honesty in how it actually came together: not a grand architecture diagram drawn up in advance, but a long series of deliberate responses to real, painful failures, each one hardening the system a little further against the specific kind of pain that had just been felt.

What This Means Going Forward

As Netflix keeps expanding into live sports, gaming, and increasingly interactive content layered on top of the same underlying infrastructure, the same lessons from this article keep resurfacing: separating what changes quickly (personalization, recommendations) from what needs to stay rock-solid (actual video delivery), caching aggressively enough that the core systems rarely even notice everyday traffic, and continuing to test failure deliberately rather than simply hoping a new kind of load never breaks something nobody thought to check. The lesson underneath Netflix's story isn't really about microservices, Open Connect, or Chaos Monkey specifically. It's the same one running through this entire series, maybe stated most plainly of all here: the most resilient systems aren't the ones that never fail. They're the ones that have already failed, on purpose, hundreds of times, long before a real failure ever gets the chance to matter.