When Google Seraching was growing they had a risk of Cost.
Imagine evey internet user using your site to find best results on the Internet, this site has lot of complex calculations, require huge amout of CPU Power and RAM, and Google started having nightmare that their company will be buried under the Infrasturcture cost.
Larry abd Sergey know that regular Infra won’t work from them, from day 1 they started using custom hardware, but now after several years that hardware was also not scaling, they needed a Genius and they wanted that genius to deliver something that world has never imagined, Infra that saves the cost when entire world is using that.
Image Credit: Naoise Culhane / SPORTSFILE / Web Summit via Flickr (CC BY 2.0)
Urs Hölzle, was founder of a startup acquired by sun microsystem and a professor at University of California, he earned his PHD from Stanford University.
When Larry and Sergey met Urs, they immidiately believed that he can save the Google and make it cost cost-effective Search Engine from Infra point of view
He led the design of Google data centers, which use half of the power of a regular data center. This was a huge achievement.
He did not lead only hardware but also software developments as well, he led the development of GWS, their own proprietary, highly scalable web server. Experts believe it’s one of the most scalable Web Servers in the world
He also suggested changes in Linux Google use
Urs is Employee number 8 in the Google, he was few of those people who made Google Serach Engine number 1 in the world
Why did Google need so much computing power?
Google was not a conventional search engine; Larry always said their aim was to collect the world’s information and present that to the user in the most organized way
But when Google was launched, they had 2 problems
- Collect every website in the world that wants to register itself in the search engine
- Keep searching for new websites and keep adding. In the entire world, people used to keep deploying thousands of websites every day, even at that time period, the scale of the internet was unimaginable
Apart from this, Google has to index and rank each site according to its algorithm
Let’s see what area of the Google Search Engine requires huge computing power
Crawling:
Each search engine has to crawl for the website to store its information on its Server
Google Crawler is a distributed system. To make crawling faster, Google has to deploy it in many data centers, also Google has to index each website in every part of the world
Google has to crawl billions of pages, for each page, Google does
Sending an HTTP request
Downloading HTMl
Parsing data
After this page, Google renders the page where Google has to execute JavaScript and build DOM
After this phase last phase comes, which is Ranking
Ranking requires huge computing power,even more than the computing power required for previous phases
Google also has to detect spam and execute AI functions to understand semantics
Showing Results
When a user submits a search query, it is passed to the nearest data center, and then it runs load balancer. After that, Google splits the query into hundreds of shards so that it can be executed on a distributed system.
Google divides queries into multiple queries to show results as quickly as possible. By using this strategy, Google became the world’s fastest search engine
Now that we understand the scale of computing power required by Google, we can see now how Urs created an infrastructure that can not only survive this scale but also surpass user expectations
Those days comapnies like IBM relied on expensive hardware and servers, google could not afford this because they required massive CPU power
Urs decided to assemble server himself using of the self PC parts, Motherboards, Hard disk and CPUs.
Early Google computers were mounted on corkboards, this design let them eliminate non necessary parts
This was a radical DIY approach and it reduced infrastructure cost by 50 to 70%
The next challenge was the data center, it was the real asset where a huge amount of money was required because of its high rental fees, and it also consumeda huge amount of energy
Urs suggested that Google should make its own Data Centers; he not only suggested but also guided the team to make their own data centers
The main target of Urs was not only to save money that could have been spent in third-party rental data centers but also to save energy saving.
Now Google have full control on their data center Urs could make efforts to save energy cost
Google used custom power supply that optimized the energy and saved the cost.
Urs not only saved cost using only hardware but also saved cost using software
Google was using a distributed system, so for data replication Urs and team created GFS, Google File System which distributed data across all data centers efficiently without requiring expensive RAID hardware
If any hardware fails, routing was smooth; this made Google’s fault tolerance world-class.
As we saw, Google breaks queries into multiple pieces and runs them simultaneously to achieve maximum speed, this concept was called MapReduce and Urs was the leader of implementing this model.
MapReduce was a complex system under the hood, but it made google a giant, users loved Google’s speed of delivering results
With radical approach ,Urs made Google an example of how a company can save a cost with talent in assembling hardware and scaling further using software, algorithms, and programming models
Infra made by Urs was so scalable that Google Cloud runs on it.
Google Cloud is growing rapidly because of its efficiency and fault tolerance.
Client across globe trust on Google Cloud because the infrastructure that runs Google runs their own applications
Today ,Google Cloud provides 150+ services and products of Google Cloud
This services include core services like Computer, AI ML, Storage, Networking, Analytics, Developers Tools and many more
Urs also approved request of making Kunbernates, open source, now Kubernates is used by thousands of companies with 0 dollar fee, and they could also scale it according to their needs.
The impact of Urs is not limited to Google, but it motivates Engineers and Enterprenuers all over the world, where Larry and Sergey saw a dream which could have required billions of dollars but Urs helped him by saving massive amount of cost that not only let Google survive but become a Search Giant and one of the most important Cloud Service provider in the World.