How Systems Design is Similar to a Cookie Restaurant.
Systems design explained simply — thinking about technology at scale.
Grandma’s Secret Chocolate Chip Cookie Recipe
Imagine that your grandma gifts you her renowned recipe for chocolate chip cookies. This recipe has been passed down in your family for generations and are the most delectable cookies that you’ve ever tasted. Now that you have it, you decide to make some for your friends. They salivate nonstop after eating them. Every week they beg and plead for you to bake more and soon start inviting their other friends to your house. Your cookies are so appetizing that people are actually willing to pay you for them. You’re suddenly the talk of the town.
After a few weeks of strangers coming in and out of your house, you start to grow discontent. You enjoy selling cookies, but don’t want strangers in your personal space. Thus, you decide to open up a restaurant.
This is the beginning of basic systems design: the process of defining the architecture, interfaces, and data for a system. It all starts with an algorithm, a piece of code that takes in some input and generates some output. In our case, your cookie recipe. People are willing to pay you for this algorithm, but right now it only exists on your computer. For them to use it, they have to come to your house. In order to solve this problem, you decide to host your algorithm on the internet. For us, this translates to opening a restaurant.
But what does it actually mean to “host your algorithm on the internet?”
This means that you pay a cloud computing platform, such as Amazon Web Services (AWS), to store your code on a virtual computer, otherwise known as a server. This provides several advantages:
(1) Anybody can use your algorithm. Now, instead of people having to come to your house and use your computer, you can provide a website link that will redirect to your virtual computer.
(2) Security and Encryption. AWS provides incredible safety so that you don’t have to worry as much about hackers stealing your precious algorithm (keep in mind that nothing is ever 100% secure).
(3) Accessibility and reliability. No need to worry about the power going out in your house and crashing the algorithm. AWS is extremely reliable and accessible. If you want to make any changes, you can remote login from anywhere.
In order to host your code on the internet, you’ll need to follow something called a protocol, which is a set of rules that govern how you receive and send data over a network. For the internet, we will use the hypertext transfer protocol (HTTP), which I’m sure you already have mild familiarity with. For our restaurant, these are going to be the laws, rules, and regulations that we have to follow when serving cookies to our customers.
Within the first week of hiring staff and opening the restaurant, the place is packed. People are obsessed over these cookies. Based on increasing demand from customers, you intend on offering a larger variety of cookie flavors. In addition to chocolate chip, you now offer snickerdoodle, sugar, macadamia nut, and oatmeal raisin. At all hours of the day, the store is filled with a line wrapped around the block. You can no longer manage all of the traffic; it’s time to start thinking about expanding.
Basic Systems Design
Before we get too far, let’s examine how your cookie restaurant resembles the most basic architecture of a system. In basic systems, we have a client, a server, and a database (Figure 1). The client sends requests to the server through an Application Programming Interface (API). This is basically a messenger that passes information back and forth between the client and the server. Once the server receives the request from the client, it will process it and send back a response through an API. This response will be either a failure or a success message along with some data. In our cookie example, the client represents our customers. When our customers look at the menu and decide to order food, they will send a “request” through the waiter, who represents an API. The waiter will then bring this request to the kitchen, which represents the server. The kitchen then knows exactly what to prepare based on the request. In order to complete the order, the kitchen will have to pull some ingredients (data) from the food storage cabinet, which represents the database. When the kitchen finishes the order, they will send a “success” response back to the customer using the waiter as their messenger (an API). However, if the customer requests something from the kitchen that is not available, then the kitchen will respond with a “failure” message.
Scaling the Server
Now that we have an understanding of the basic infrastructure of systems, let’s talk about expanding our restaurant (also known as scaling our server). Why would ever want to scale our server? To handle more requests, of course. Ever since we added a variety of cookie flavors, our popularity has gone through the roof and we can no longer make cookies fast enough to handle all of the requests. This is what would happen if tens of thousands of users tried visiting your website at the same time (remember when you visit a website, you’re sending a request to a server for the information that you’re seeing). In order to solve this problem, we can either decrease the amount of time it takes for our customers to get their orders (latency), or increase the maximum number of cookies that we can produce in a given period of time (bandwidth). Both of these solutions have to do with making changes to the server. There are two ways to scale: Vertically and Horizontally.
“Why would ever want to scale our server? To handle more requests, of course.”
Vertical Scaling
Vertical scaling is adding more resources to the existing server (computing power, memory, etc.) so that you can process more requests. For our restaurant, this would look like hiring more manpower (another baker) and buying more ovens. This is the quickest solution but has prudent limitations: a single point of failure and hardware limit.
Single point of failure: If you decide to keep everything under one roof and add more resources, you’re consolidating the risk of your entire business on one server. This can be problematic for our restaurant if a fire burned down our kitchen.
Hardware Limit: There’s a limited number of new bakers and ovens that we can buy before we run out of space in our restaurant and will be forced to buy another kitchen.
Horizontal Scaling
Horizontal scaling is adding more servers to balance out the amount of requests that any given server has to handle. This would translate to building another physical location for our restaurant.
Overall, horizontal scaling is the better solution for growing with customers. However, this method has limitations of it’s own: load balancing and potential data inconsistency (the food might not taste the same at the new restaurant).
Diving into Load Balancing
If you decided to build a new physical location, you would ideally need some sort of application that directed customers to different locations in order to manage the traffic (you’d want to avoid a situation where one location is empty and another is packed). A load balancer is an algorithm that takes in requests from the client and routes them to different backend servers to balance network traffic (Figure 2). This can take a significant amount of time to get running because it involves building out an entire network of cloned servers that are running the same code and have access to the same data. There are several different types of load balancers that we could use to direct traffic. For example, if we moved all orders from in-person to online, we could track the I.P. address of where all the orders are coming from. We could then redirect customers to the restaurants closest to them to pick up their orders. This is called a Networking Load Balancer (Layer 4). Additionally, we could redirect customers based on what types of cookies they ordered. For example, we could send all the people who ordered chocolate chip to one location, and snickerdoodle to another. This is called an Application Load Balancer (Layer 7).
Of course, Amazon Web Services could take care of all of this for you.
Caching
Ultimately, you decide that building a new restaurant, or horizontal scaling, is the best approach for the future. You’ve also decided to move all orders to an online website that uses a simple load balancer to direct traffic based on I.P. addresses. To further increase the customer experience, you’ve decided that you’re going to keep tabs on the last order that each customer made so that you’ll remember for the next time they return. This is the idea behind caching. Caching data is an extremely important and persistent concept in systems design. This is typically a key-value pairing that sits in between the server and the database. Here’s an example of a simple cache for recent orders:
{
"Dave": "1 snickerdoodle",
"Lisa": "4 chocolate chip",
"Michael": "3 oatmeal raisin"
}The purpose of a cache is to store temporary data, which increases the speed at which it is read (reading information from a large database can be a slow operation).
Other Design Considerations
As you continue to expand, there are several other systems design decisions that you’ll need to make that will be crucial to your success. Each of these decisions carry unique tradeoffs. For example, as the amount of menu items increase, you may start to think about only offering certain flavors of cookies in specific locations. This is known as database partitioning, or sharding, and could significantly reduce the amount of ingredients you’d need to hold at any given store. In order to implement this, your load balancer would need to have a lookup table to be aware of exactly which cookies each of your restaurants was selling. Another important decision could be how to efficiently setup your food storage cabinet, or database, to prioritize either organization or speed. To prioritize speed, you could use a strategy called database denormalization: grouping ingredients in the cabinet by recipe as opposed to by category (e.g. all the ingredients to make macadamia nut cookies could be found next to each other). However, as with all other systems design decisions, this would have its own tradeoffs: eggs, milk, and sugar would appear in 5 different places within the cabinet.
