Netflix is a streaming giant that serves its content to its 300 million users worldwide.

It has thousands of series and movies on its platform, and users can watch them on demand. Netflix serves them smoothly.

It serves Petabytes of Data daily and handles 15 to 20 percent of Internet traffic in peak hours.

It invented the Chaos Monkey tool, which intentionally kills a server to test systems’ scalability

Does Netflix use AWS for hosting (compute and other services)?

Every tech enthusiast in the world came across this question that does Netflix, a streaming giant, a beast of the internet that serves content on the Netflix app and web, even on TV, use AWS for its hosting and compute and streaming? If yes, and how, and when did they start using AWS? And before that, did they use their own infra?

This story has some twists and turns like before, AWS Netflix faces some major problems, then moved to AWS but faces billing problems and lately found some solution and created some of their own solution which does not involve AWS.

Let’s see the history of the AWS and Netflix relationship

Before AWS.

Netflix streaming was launched in 2007.

Initially, Netflix has their own data centers and physical rack servers.

Mark Randolph, one of the co-founders of Netflix, shared crucial things about their initial infrastructure. He said they were focusing mostly on speed rather than perfection in the system in it;s early days.

There were no microservices, Basic Servers, and Basic Database were used

Mark also shared an instance where they had to buy some extra servers and add them manually in the data center because, after launch, streaming requests kept growing and became a nightmare for them.

 

Migrating to Amazon Web Services

Netflix co-founders did not think that in just one year, Netflix would grow so big that their traditional data centers would start failing and in 2008, their infra became a nightmare for them.

2008 outage was a nightmare event for Netflix, and even today, they treat this outage as a lesson; for many users, streaming stopped, and business loss occurred.

AWS was a promising tool to solve their problems; it had robust autoscaling and a strong ecosystem of services for streaming, and a pay-as-you-go model could save money for them as well.

Netflix eventually decided that they are a content company, not an infrastructure company, and infrastructure worries should be transferred to a specialist company. For a company with such scale, AWS was the no 1 choice, and they were confident that AWS would satisfy their users’ streaming needs.

In 2009, Netflix started its migration, but since it was a very big system even back then, they could fully migrate to AWS only in 2016, and in 2010 to 15 hybrid phase existed, it means some parts of Netflix were operational on AWS.

We must remember in 2009, when Netflix started migrating, AWS had not gained that much trust, which it has it today. So Netflix moved non-critical parts to AWS first, and when trust is gained then started moving critical parts to AWS. Streaming was moved as the last critical part of the migration.

How much of Netflix’s code is on AWS, and what services it uses

Today, Netflix is fully hosted on AWS and has gained popularity as one of the biggest clients of AWS.

The backbone of Netflix is AWS EC2 and Autoscaling.

EC2 runs their microservices, their main logic

S3 (Simple Storage Service) is also one of the main components, because it stores their videos.

Streaming: Here is the twist, for streaming, Netflix use their own CDN, which we will see later, this one is very interesting.

DynamoDB: This is no SQL database for internal tools

Elastic Load Balancer (ELB): Traffic serving is done by this service

SQS / SNS: This is a messaging tool used for messaging

Netflix has built its own tools on top of AWS

Erueka and Zuul, these tools keep track of running services and handle security

Chaos Monkey: Intentionally kills a server to test their scalibility

 

Open Connect-The twist

Open Connect is Netflix’s own CDN built on top of AWS

Netflix was using AWS for its streaming before Open Connect came into the picture, and created huge problems for them.

As we know, Netflix has to serve petabytes of Data every day to its users, and it was generating an unimaginably high bill for Netflix.

Netflix has to pay for every GB of transfer

Netflix was becoming financially unstable because of this huge streaming bill.

Apart from the huge AWS Bill, latency problems were arising, and video quality was dropping

As a result, Netflix created Open Connect, its own private CDN.

In this way, Netflix started saving money, which they needed to pay for per GB

So this was the journey of Netlfix and AWS relationship.

 

Keep visiting DeepSystemStuff, we write deep system articles and tech strories frequently