15 System Design Concepts Every Senior Engineer Should Know

So, those are the 15 most important concepts that you need to know in order to excel at system design. So, we're going to talk about things like load balancing, caching, rate limiting, the CAP theorem, database replication, database sharding, and architecture styles like microservices and event-driven. And then we'll also dive deeper into more complex topics for the senior level like consistent hashing and rate limiting. So, let's start with number one, load balancing. And load balancing is very useful whenever there's a lot of pressure on one of your servers. So, you got all these users making requests and this server right here is becoming overloaded. Now, of course, we could just buy a bigger hardware, but that would be number one expensive and it's not very flexible.

Whenever you stop your server and then you buy bigger hardware and then you restart your server, number one, well, you have downtime. And the second thing is traffic, it's usually not constant. So, in an average web application, there's always certain peaks and valleys. And the problem is if we buy a big hardware to satisfy this peak, when we are here at the valley, we have a lot of resources that we're not using and it's becoming very expensive. So, vertically scaling our server by increasing resources is not a good long-term solution. Instead, we can use load balancing. And in order to load balance, we need a load balancer, which is an infrastructure piece that will receive our request and then distribute them evenly between all our servers. In this case, we use the round robin load balancing algorithm, which means the first request go here, second one goes here, the third one goes here, and the

fourth goes here, and then we start over. The problem with this kind of algorithm is that those requests are not born equal. So, maybe by any chance we send a lot of heavy requests to the server and we still have one server that's overloaded. So, there are different algorithms like the least frequency, the least connection, or monitoring the CPU and memory of the server and sending the next request to the server that has the least load. Now, two things to remember that will really show seniority. Number one, those servers have to be stateless. What does it mean? If I get a request from this client and it goes to and it gets redirected to server four, but then the next request goes redirected to number two, I should be able to fulfill all of the requests so there cannot be any state, any data here on those servers.

All state should live in shared resources like a database or a shared cache. We'll talk about it later on. And your servers should be reproducible because whenever you have more users, you want to add a fifth instance. So, you need all the DevOps tooling around it to be able to spin up a resource and link it up to the load balancer so it knows that now it has five targets instead of four. System design interview concept number two is caching and caching is based on a very old programming pattern, which is memorization. Basically, being able to return the result of a function if it was pre-calculated before instead of computing it again. We get some input, we do some logic, and we get some output, and the next time people call us with the same arguments, we actually reuse that first result. We don't compute it again. Now, you can do the same in a system at every single layer.

Now, let's dive deep into caching. So, people think caching it's all about CDNs, but in reality it's a lot more than that. And when I add caches to my system, I like to go from left to right in this direction starting at the traffic. The first cache we can add and the cheapest one is to make sure that HTTP caching it's well configured. This is basically done with HTTP request and responses and the headers they have and it is to ask the browser to keep something on the local machine of the user not to have to request it again if it didn't change. Now, this is a combination of the caching header and the ETag header and the last modified I mean last modified is legacy. I don't want to go too deep, but this is the first thing you want to look at. Do we have good decent HTTP caching? Is this well configured? Once this is done and it's well configured pretty good routes

and assets, you can move on to a CDN. And CDN it's where you can place fonts, images, and CSS. It's distributed across the globe, so this user will also receive the copy that it's hosted closest to them, so we remove latency. And you can invalidate your cache by publishing new assets. They'll get replaced and propagated. CDNs are one of the simplest and cheapest way to scale nowadays because there's a lot of, you know, out-of-the-box solutions like Cloudflare or CloudFront from AWS. Once you're done with that, we talk about data. So, normally a CDN is really good for static assets, but when it comes to data, you only need a server cache. And the easiest way to do so is by using a tool like Redis. You can think about Redis, if you've never used it, as a JavaScript object, key-value pairs

hosted somewhere in the cloud, very close to your server. So, the latency is minimal. Sometimes you have the Redis instance in our server or also, but remember, if we want to load balance this, it's better if this is also a shared state. So, you might have several instances of those, and they can reuse the same Redis instances. How does this work? Well, whenever you try to get something from the database, it will first try to get it from Redis and see if it's in there. If it's not in there, what happens is a cache miss. So, it will go to Postgres, get it from Postgres, and then populate the cache, right? Send the response, persist to cache. The next request that's asking for the exact same thing will check Redis and find something in there. And it will return it from there without hitting the database. This can protect

you also from spikes in traffic, so you don't need to necessarily scale your database. You can add a caching layer, especially if you have a system where you get a lot of reads. Now, the hardest thing about cache is cache invalidation, and I went for a TTL in this case, which basically, you can say everything in the cache will expire after 60 seconds. That's quite aggressive, and it means if you are requesting the same thing, but if than 1 minute later, you have to go to the database. But the way it's set up right now, it does put the experience spikes in read traffic. Concept number three, API design. So, whenever your

client requests something from the server, we have an API. That's a standardized way for you to communicate with the server, and usually client is not aware of what's inside the server. They just see the API. So, what are the different APIs out there? Well, when it comes to request and response, we have basically three API styles. We have our REST APIs, or GraphQL, and gRPC. And in a second, we'll go over all of them really quickly and see pros and cons. So, REST is the oldest one, and it's basically just an organization layer for your HTTP request. It usually defines methods with HTTP verbs like get, and

you make a request to an URL that contains the plural noun, in this case jobs. And then we might have an ID. And because we selected one resource, or REST it's all about resources, we get that resource back as JSON in this case. The advantage of REST is that it's very simple. It's cacheable, and it's universal. If you deploy a REST API, there's so many integrations that you have out of the box, and there's even advanced stuff like testing and generating types based on your REST APIs. So, it's pretty advanced. Now, there are some problems that we'll see in a second, which are over-fetching and under-fetching. Basically, imagine you have a web page that's quite complex. You're going to have thousands of requests for every single resource. And if you start aggregating those in a single request, well, you're not doing RESTful API design. You're just freestyling. So, you'll lose all the advantages that a REST has. To solve

that problem, people came up with GraphQL around 2015. So, more of a recent technology. Basically, in GraphQL, you query your server just like you query your database. You write a query. And here we ask for a job posting, but we could also ask for users, or applications, or resumes, or whatever we want in the same query. Now, of course, this query is written against a server schema. So, the server has to define all the different nodes that we can query. But the advantage here is that I make one request and I get everything back. So, in this case, we made a very simple request, but it could be nested, and that's how I avoid that too many request problem that REST had. So, the other advantage is I have a lot of control as a client. You know, from building the front end, I can actually select which fields I want, which fields I don't. Whereas in REST, those

interfaces are defined on the back end side. So, the back end says, "Hey, for this request, you're going to get those three things." You always get them. This is super important when you're building for mobile, where you don't want to waste the user data. So, you don't want to request things that we're not going to use. Also, because of a memory problem, but also for a data budget problem. So, you have client control. It's all typed. You can generate types on the front end very easily, and it's flexible on the client. The problem with GraphQL is that caching is very hard. Why is caching hard? Because by default, the GraphQL request is a POST request,

basically. This is a POST. This is a request that traditionally in REST APIs would create a resource. So, they are by default not cacheable. And what I mean is whenever your request would go through an API gateway or a caching layer, you'll have to work around it to really make it cacheable, because by default, they don't really support it. Now, there's many ways to solve GraphQL caching, so it's not an impossible problem, but it definitely makes things harder. And from my experience, also, GraphQL is a lot harder to build on the back end. Like, designing the GraphQL resolvers and the schema, it's much more complex than just building a bunch of

REST endpoints. So, I'll say back end complexity is a problem with GraphQL. The final one is TRPC. This one is really useful when you cover machine-to-machine like service-to-service communication. It's end-to-end typed, and basically, the difference is that with TRPC, you are actually running a function on the other machine. So, you're actually executing something and then getting the result right back, rather than just sending a resource and getting something back. Again, I'm not going to dive deep. I think API design is a very complex topic, including pagination and caching. And if you want me to make a video specifically about this one, let me know

in the comments. And by the way, if you're an engineer and you want to find out what your gaps are or get to the senior level, check out our free technical assessment. There's a lot of real interview questions out there that can help you understand where are you right now. Link it in the comments. I'll see you there. System design concept number four, rate limiting. We talked about APIs before and one of the thing with APIs is that you need to protect them against malicious users. In this case, I do the example of a bot. Normally, it could be an AI bot or any kind of crawler that starts hitting your API from whatever reason with 50,000 requests per second. Even if these are not really users, your server doesn't really see it and it might go down and so will your database. So, we need a way to protect ourselves against this kind of behavior. And the way to protect

ourselves against bots hitting our API is by using rate limiting. And rate limiting is something you can implement on your server or you can have a web server component like an API gateway that implements it. And it's basically a way to track how many requests have a specific client made in a certain period of time. And you can do this their IP address or geography. But basically, whenever they request to you, you'll kind of store the request and figure out if they still can should still be able to call you or if they're abusing the service and you should just send them back a 429 and have them waiting. Now, there's a couple of rate limiter algorithms and I'm going to explain two of them. Number one, it's fixed window. And in a fixed window, you're basically giving every client number of requests per minute. So, in this case, it's 18, right? If they make more than 18

requests between 12 and 12:01, then those four will get a 429 back. And you keep track of this in a external, you know, Redis kind of memory that is shared between all your server instances. Now, this window will reset every minute. So, if the next minute they do not make more than 18 requests, they'll get green. And if next minute they start sending too many requests, you again block them. Now, this works fine, but there is a problem with it. Imagine we're giving them 2,000 requests per hour. What can happen is that the bot will wait until the exact minute when we give them those requests and hit 2,000 at the same time. And so, it will still bring down our servers because they will concentrate all their budget in the first very second of the window. So, another way to do rate limiting is

by implementing a token bucket. Basically, every client gets a bucket of tokens. Let's say in this case we have four tokens per client, and they get regenerated in our case per minute. So, every 15 seconds they get a new one, but there is a maximum of four. They can never have more than four. So, let's imagine we start with four tokens. They make three requests very fast, and they still have one token left. Those three get a 200 okay. Now, I mentioned that they get four per minute. So, as time passes by, 15 seconds later we regenerate one. So, now they have one, the one they had left over, and a new one. So, now we have two. Now, let's imagine they hit us with three new requests. Because they had two, we kind of satisfy those two and send the 429 after. As time passes by and they keep

hitting our API, there is no more new tokens for them, so they'll all get rejected. But then, 15 seconds later they get a new token. So, if they start requesting now, it will pass by. So, basically it's a way to number one limit the maximum amount they get per time window because we all literally have a maximum of four per minute, but also this token bucket kind of gets regenerated as time goes by. So, we also kind of throttle them. And that's how rate limiting with a token bucket works. And in order to keep track of this, you need some application logic on your server and a external storage like Redis. System design concept number five, and probably one of the most important, is the CAP theorem. And basically, CAP theorem tells us that you can only give two guarantees out of three in a distributed system. And the

three guarantees are availability, that means you're always available, the system is always up, you answer fast. Consistency, that means that every read is the exact last write, so you're never reading stale data. And partition tolerance, which means your system is distributed. Now, we've seen a lot of what we do in order to scale, it's with distributed always system. And so, because your system is usually distributed over the network, the biggest trade-offs you'll talk about in a system design interview, it's between consistency and availability. And in the next concept, we'll touch on both availability and consistency because there's different scalability patterns that will tend towards one or the other. Now, depending on which problem you're trying to solve, you probably want a system that is more consistent, or you want one that's more available. If we're

talking about anything that it's sensitive, like financial data, you usually want a strong consistency. That means everybody that's reading from your system will really receive the latest thing that was written, which usually involves them waiting a bit more until you propagate all the nodes. But, if you're building a social network, usually availability is more important, so you might return every now and then stale data. If somebody loads an Instagram post and they see a comment that's not there yet after 30 seconds, nobody will die. But, if the same happens in a bank account or in a stock exchange, then the consequences can be very bad. Now, based on those guarantees, you can also classify all the different kind of solutions and databases. For example, Elasticsearch, it's partition tolerant and available, but it has what we call eventual consistency. We'll talk about that. So,

it's not super consistent. On the other hand, Postgres, it's a single server that's both consistent and available until you decide to scale it out. So, technologies might default to one or the other, but then there's ways in which you can move them around and make them either more consistent or more available. System design concept number six, SQL versus non-SQL. This is basically related to choosing a database. In general, you don't want to spend too much time talking about this in a system design interview because you really have to move forward fast, but for the most cases, you will have to choose between something like a document database that's non-SQL and a SQL database. The advantage of a SQL database is that it's relational, so you can actually query different relationships, you can combine data, you can enforce strong relationships in a

schema, whereas in non-SQL, it's a bit more of a freestyle. The advantage of NoSQL is, on the other hand, availability. It's extremely easy to distribute this database and to scale it out almost by default. Now, in general, there's different constraints and ways in which you could actually have your non-SQL behave more like a SQL database, like a relational database, and you can have Postgres acting like a non-relational database. A simple example is if you just go ahead and in your data storage, you just store JSONB, so you put plain JSON in your database. That's a very bad way to use Postgres, but in that case, you'll become a document database. So, you'll have pretty much the disadvantages of the document database and the disadvantages of Postgres. All this is to say that

senior developers know that those things are a spectrum and that those applications can be moved around. What I just gave you is a very quick explanation. I am planning to make a database video that's more in-depth. Do let me know which one do you want me to make first. Is the API design or database or both? I'll see what I can do. And by the way, if you're a developer and you're looking to get to the next level, check out our website theseniordev.com, where we help developers become engineers that AI can't replace. System design concept number seven, database indexing. So, we talked a lot about scalability on the server, but what happens with the database? Well, as our database grows, one of the first problem you'll see is that our read queries become slower. That's because if we have, for example, 10 million rows and we're looking for a specific user with a specific email, in

the worst-case scenario, we'd have to do what we call a full table scan. That means we have to go through the whole table and look up all those 10 million operations before we find our user. And so, this kind of query, even with a very good server, a very good Postgres server, will take around 2 seconds. And that's only the database query. It doesn't include all the HTTP round trips. And by any performance standard, this is very slow. So, how do we solve this? Well, we can use database indexing, and this relies on all our knowledge about data structures and algorithms. Basically, what we do is we create an index on that table on a specific field or on several fields. Now, what is an index? Well, the easiest way to explain it is that under the hood, Postgres will create a B-tree for

us. That's a tree that is a lot easier to to traverse and find the right answer than doing linear search. So, basically, we move from linear to logarithmically. Now, what is the difference between those two? Well, if we have 10 million records, if we want to find a record by linear search, the worst-case time complexity is 10 million, literally looking through every record if your record is the last one. But if it's logarithmically, it's only 23. Now, I don't want to go too deep into how trees work. This is more of a data structure and algorithms problem, but I love this example because it shows us that your low-level knowledge, your knowledge of the fundamentals, it's extremely important even when you design systems. And so, a big seniority signal in system design interviews is your ability to relate this fundamental logarithmic

knowledge with the wider system decisions. System design question number eight is replication, and we talk specifically about database replication. So, we've seen with database indexing we could solve the reads problem, but as we scale out our server and we have more and more instances putting pressure on our Postgres database, queries and writes here will become slow even if we have database indexing and we have a clean schema just because there's too much pressure on the database server. So, how can we scale our database? Well, the easiest way to scale our database is by creating a read-only replicas. So, basically, we'll still have our primary database, and this is where we write. So, any write request from any of the server goes to the primary database. However, when we need to read data, we

can read it from a read-only a The advantage here is that you can also have those replicas in different geographical positions. So, it works pretty much like a CDN, where you're actually distributing your data close to the user. So, you're solving both the availability, but also the latency problem. Now, part of being a senior developer, it's understanding trade-offs. So, what is the problem with these read-only replicas? Well, this is when our knowledge of the CAP theorem that we mentioned before comes in handy, because the way we can do this thing is to either choose to be consistent or available. Remember, we already have partition tolerance, because we distributed our database. So, now there's only one guarantee from those two we can choose. And to understand this, let's look at some sequence diagrams. And so, basically we have our

server, we have our primary database, and we have our replica. And in the first scenario, we'll go for the synchronous scenario. Basically, whenever the server changes some data, we change it in the primary, so it's done, and then we also propagate this to all the replicas, and we wait. And when it's done, we finally tell the server that we are done. Disadvantage here is that writing takes a lot of time. But the advantage is that reading from any of the replicas gives you the latest data. So, we have consistency, but we have less availability. We are less available because writes take longer. On the other side, what we could do is to have the server writing to the primary database, and then immediately return this as okay, the write finished, and do the propagation asynchronous. Depending on how many replicas you have, that could take 200 milliseconds or

1 millisecond, and for a short while you will be inconsistent. That means in this case we do have availability, cuz we're very fast at writing, but there's no consistency. Now, depending on the solution you're building, you might choose one or the other. But remember, whenever you distribute your system, you do have trade-offs. And those trade-offs always follow the CAP theorem. Now that we talked about the replication and the CAP theorem, let's go deeper into the consistency models. System design interview concept number nine, consistency models. So, this basically dictates how consistent your data would be. The first and easiest one to understand is the strong consistency model. In this model, the users would always see the latest data. That's why consistency is strong. So, let's imagine the use case for something like LinkedIn or job search platform. An employer

might change a job posting and they'll go to the server and write a new salary for that job posting. And in the case of strong consistency, we will properly adapt from primary node to all the nodes. And once that's done, we tell them, "Okay, that's written." Now, the disadvantage here, as I mentioned before, is that the employer will have to wait. The writes will be slower. But the advantage is that the job seeker will always see the latest number, no matter which node they are redirected to. The next consistency model will be eventual consistency, which means the system would be consistent at some point in time, but not immediately. And this

is exactly the use case we talked before. The employer will call the server, they update the job, and we immediately tell them, "Okay, you're finished." As soon as they modified it in the primary copy. Now, in the background, we also update our replicas, and that might take 200 milliseconds or even more. So, there might be a point in time where a job seeker making a request reaches a node where the salary data for that job posting it's the old one. And that's why we say that at some point in time, after the 200 millisecond, the system will be consistent. So, it will be eventually consistent. But there will be a small time window when the system will be inconsistent. This is a very good model if the feature you're building or the system you're building prioritizes availability over consistency. You can think of pretty much everything that's non-critical,

like a social media comments or post. Nobody really cares if they see it 30 seconds later. So, it's okay. It's a trade-off you can live with. But in the financial industry or the insurance industry, whenever something is critical and having the latest information it's very important for the user, you do not want to use eventual consistency. You want to go for the strong consistency model instead. Now, the latest consistency model I want to talk about it's read your rights and this model is kind of a mix in the sense of the employer will change the job posting and will update a salary and they will see their latest read. Even if in the background we are still propagating it, the read they see is the latest. And so, in this case for them, we do have a case of strong consistency, but for the other user, for the job seeker, it's eventual

consistency. So, as you see, this is a bit of a mix and this is important when for example this person, you know, they need to see their update so they didn't keep updating or they keep sending you know in the case of the financial industry they didn't keep making the same transaction over and over, but this job seeker they are looking at the information is not super important for their decision making. So, for example, you can do that with analytics. Let's say you're building a banking application. Whenever people make transactions, you want strong consistencies, but for the analytics you can update those asynchronously. So, in that case the model for the analytics

would be eventual consistency. System design concept number 10, it's database sharding. So, we mentioned how we scale our database by replication, but replication still means that all data is still in that primary database. And there might be a use case where you have so much data that even on the biggest database server it just doesn't fit. Again, sharding is very complex to set up and it's something I would delay as much as possible. There's many ways to scale your system and to be honest, if you avoid sharding as a scaling strategy, you probably don't worry senior in an interview. But let's take a look at sharding. So, sharding is very similar to load balancing in terms of the mental model because you basically put subsets of your data in different locations and you have a router. So, depending which record you access, that

record might live in the first shard or the second shard or the third shard and there's usually a way to figure out where is that record. In this case, I added this directory sharding which is basically having an extra memory server. It could be a Redis for example and so, here you basically store the relationship between the record ID and which side does it live in. This is one of the simplest way to do sharding, but you would need more complex application code and you need to maintain this component and you have extra latency because the server will request the alter, then have to go to this lookup table, come up with the shard and then finally do the query in the shard. The other problem with sharding it's compound queries where imagine you want to make a select with a different filter and some of the records you're looking for live in this shard and this other shard. Now there's a lot more that can

be said about sharding, but in most interviews they'll just ask you for a high-level explanation. So this should be enough. If you are a software engineer and you are looking for that next position, you're really on the market right now, we do have a fast track program where we help you get in a job offer in 90 days or less. Check it out in the comments. System interview concept number 11, consistent hashing. We talked about database sharding and how we split our data between different shards, but how do we decide where does a specific record has to go? Well, we can do that by having a hashing strategy. So we basically pick a hashing key like a job ID and in this case would be probably a UUID and we have some logic that will transform that into a specific number that goes from zero to

three. How does this really work under the hood? Well, basically we take that UUID, we hash it in a number and finally we make the mod by N, N being the number of shards. So basically let's imagine we have this UUID, we basically hash that in a number, it's this kind of very ugly number, and then we make the mod operation with four, which is the number of shards, and that will give us three. For a different UUID, we'll get a different number, we make the mod, it gives us zero. So that will go into the shard zero and so on. And so for every record ID, we can decide in which shard is it supposed to belong. Now what is the problem with our current hashing strategy? Well, in this case if we were to remove a shard, we basically need to number one reallocate all of this data into the specific shards, but also because N it's now three, we'll have to recompute the hashing key in a lot of

those records. And when we divide this number by three instead of four, we'll get a different shard, and that means we'll have to reallocate a lot of data because the way we compute our shard changed. And in this case, we'll have to reallocate around 60% of the live data. That means record 15 that was in shard three, now it's in shard zero, but record 10 that used to live in shard two is now in shard one. That's because we went from doing a mod four to a mod three, and the results all change. And the problem with this is that whenever you eliminate a shard, you need to make this expensive operation of reallocating all that data, but also you lose all your cache. So, you used to have cached data in a caching layer on the shards, but now you'll lose all that. So, basically cache busting, which is a bad thing for a performance. And pretty much the same happens when you want to add a

shard. And all this makes horizontal scalability harder because adding a shard and another one means reshuffling all of your data, which tends to be very expensive. So, how can we solve this? Well, there's a very smart way to change the way you compute your shard. So, basically, instead of computing that mod, we will place our shards around the circle. And what our hashing key computes, it's what is the different degree, where should this record be? And all the records, they will move clockwise. So, pretty much every record will look clockwise and sit in that shard. Now, let's see the exact function signature. So, basically, we get that UID, we hash it, it gives us a big number, but then we divide by 2 ^ 32 and multiply by 360, and the end result would be a degree. And so, basically, what do we do? We go back to our

clockwise circle. And so, let's say we got the degree 157. So, this is 0°, this is 90, and this is 180. So, 152 would be around here, and we know that each one of them gets pushed to the closest shard clockwise. So, it will end up in C. Now, of course, this is a bit more sophisticated, but there's a huge advantage here. If, just like in our previous example, we move D, the only thing we need to do is to take those records, those ones right here, and reshuffle them to A. And so, the good thing about consistent hashing is that all these other records we don't need to reallocate. They stay exactly where they were. So, we need to do a lot less operations where we are moving those shards around. Now, there's also thing called virtual nodes that I'm not going to go too deep into, but if you want me to make a video where I go much deeper

in consistent hashing, let me know because it's the kind of topic that really deserves a deep dive. Okay, now let's talk about real-time communication. So, we talked a lot about APIs and the request and response model, but sometimes we want to have live data in our application. We want LLMs to keep sending tokens. We want financial data to flow. And so, we need to choose an API that supports real-time communication. Now, at a very high level, there's three ways to support real-time communication. Number one, it's polling. So, basically the client keeps calling the server and gets updates from the server as they arrive. The disadvantage here is that there's a lot of requests and you put a lot of pressure on your server. Because you keep calling it all the time and you have many clients, you basically multiply the number of requests that each client sends. The second one, it's

server-sent events. And here we invert the model. Basically, the client will open a persistent connection with the server, and the server will be publishing updates. So, the advantage here is that the client doesn't need to make all those requests. And this is the way most LLM-powered applications work. You're basically connecting, you send a chat to ChatGPT, and then the server of ChatGPT will keep sending you all the updates until you get the full answer, right? So, that's why they feel like they're typing. What they're doing is append the next chunk of text. And the final one is WebSockets. Now, the

advantage of WebSockets is that it's really bidirectional. That means, if you have a client and a server, they'll open a channel that goes both ways, and you might have another client that opens another bidirectional channel. So, all of a sudden, these two clients, they can use an application that is for example chatting. So, in chatting, one of your friends will send you a message, it will go to the server, and the server will publish it back to you. So, if you're building any peer-to-peer application, social media, anything that involves chatting between different users, then both the server and the clients are

sending messages back and forth. So, that's when you go for web sockets. System design interview concept number 13 microservices. So, we talked a lot about scalability, but we haven't really touched a web server. And as we add more and more functionality, what happens to our server is that it becomes bigger and bigger. Now, we may structure this with different modules and create boundaries, but it still gets to a point where it's just too much to host in the same place. The other problem is that all these different modules are used by our users differently. So, in this use case, imagine we're building something like linkedin.com. We have a lot of job postings and applications, but we also have a companies module. Well, in terms of traffic, this is where most traffic will concentrate, whereas this service will not be used so much. So, it's a lot

easier for us to scale if we could break those two apart. The other thing that might motivate you to split this into microservices is different requirements. For example, the resume module might need Python to parse modules. There might be a better library that you can use for this specific problem, which is written in a different language, but the runtime here is Node.js, so you can't really do that. And so, whenever you have any kind of asymmetry, be it resources they need, traffic they receive, or even technology requirements on these different modules, it starts becoming obvious that you need to break those apart. The other problem is the tight coupling, which means everything right now is in this Postgres database, and we cannot really take advantage of having different database technologies hosting different kind of data. So, as you scale, you start thinking about microservices. And so, microservices are

basically a way to get all that monolith and split it into individual microservices. Now, they all might have a load balancer, they might expose a API, it be REST or GraphQL, and they might even have a different data storage. And this is a beautiful thing, it is that you get a lot of flexibility. You can really solve each individual problem with the best technology available. The disadvantage and the trade-off is that now you need to maintain and deploy all this infrastructure. So, all of the sudden you're not maintaining one server, but 15. Now, there's a lot of modern DevOps solutions like infrastructure as code and Docker and containers that can help you really streamline and standardize deployment, but you also all of the sudden need HTTPS between those services, authentication, and there's a big tax that you pay, which is the network tax. Basically, if the job posting

microservice now needs some information about the company that posted a job, it has to go through the network, which adds more latency. Back in the days, they all lived in the same database, so it was just a database query, which is simpler, but also much faster. All those problems can be mitigated, but they'll always be there, and the CAP theorem becomes more important than ever because right now we basically partitioned all our systems, so when it comes to the communication between the services, you'll have to choose between consistency and availability at an individual level. And this is where you can see who's a mid-level, who's a senior, and who is a staff engineer

candidate because staff engineer candidates, for example, are very senior candidates. They talk about consistency and availability at the service and feature level. So, they can be very granular on how they handle the different system requirements. System designing interview concept number 14, event-driven architectures. As you see before, with microservices, we have all these services talking to each other, and that looks very simple in the beginning, but you start having all these compound calls. Somebody might call the search microservice, but that one needs a bit of data from the job applications, which needs data from the

company microservices, which will need one field from the profiles one, and finally, it will need to pull something from payments, let's say. And so, the problem with this kind of setup is that it's extremely hard to debug, but it's also extremely hard to develop because whenever you have a feature, like a software feature, it's usually spread out between all these different services. So, a team might have to talk to the job service microservice and talk to those folks and those folks and make three different pull requests in three different repos just to publish one feature. You also have to coordinate

three different rollouts. They all happen at the same time. They all have to have the right database schema. And to be honest, the communication overhead is basically a disaster. And this is because the architecture right now enforces strong coupling between all these other teams. And the solution is to change the way we becoming more event-driven. So, we'll have a central event hub. And whenever an action happens, we will publish that into different streams inside our event hub. And all the services that might need to do something when that event happens, they just listen in. And so, this approach is

completely decoupled. That means if we need an analytics service and we want to add it, we can just go in and connect to the event stream where everything is published and add our new service. We don't have to talk to all the other teams. We can deploy independently. And we could even emit our own events in the stream if somebody else needs them. So, we move from a direct call from service to service to broadcasting. Now, you've seen I added the Kafka logo over here. It says Kafka. And there's a specific reason why I went with Kafka, which is the level of support and reliability this service has. But again,

it's very expensive. Now, event-driven architectures play really well hand-in-hand with the real-time APIs that we just learned about. And again, it's such a wide topic that I really think it deserves a specific video. Let me know in the comments. Now, congratulations. You made it to the last concept. System design concept number 15. Idempotency. Idempotency it's a way to avoid that retries cause a double charge. So, let's explain the problem first. We might have a user and they try to make a payment. So, they make a POST request to payments. That payment will result into an insert in a database of payments and we create a new payment. We return that. But it turns out that maybe their data connection was low and they somehow do not receive the 201. They might think

the payment didn't happen, so they click again and that sends another post request. And in REST APIs, post is not idempotent. That means whenever you send a post request to /payments, you create a new payment. So, what will happen is we create a new payment and now they finally get the answer back. So, they think they created one payment, but in our database, we created two. And this is the problem we solve with idempotency. So, what we do with idempotency is that we have an idempotency key. This is a secret UUID is generated by the user on the client, but you can consider that it's unique in the whole system. That is, the user will never send the same idempotency key. It will pretty much be unique to every request they make. So, we add this to the request header when we make a post. It will be added here and we have the amount. And so, what happens is that our

service, our payment service, will actually receive that. It will insert the payment and if there's a conflict, it will do nothing. So, this is very, very important. But right now, it will just persist that to the database. It will return that the payment was inserted and just like before, we lost the connection. So, what the user does is to click again on payment. They do attach the same idempotency key because this is client stored. Now, the client knows this idempotency key, this payment was not yet completed. They haven't received the 200. So, they send that through. The server gets it, but it

knows that with this idempotency key, on a conflict, it should do nothing. So, it's like, "Hey, I'm not going to update nothing because this operation already happens." It returns that there was a conflict. It has nothing back, but it does return a 200 created. It says, "You know what? This was already there, so I'm not going to do anything." So, it doesn't change the state of the database, but our users do get notified that they paid. And this is the way most payment providers like Stripe make sure that you do not make the same payment twice when you rage click. And this is pretty much it. Congratulations, you've made it. This video was very fun to make, but also very intense. I would

love to hear your thoughts and if there's anything else you would like to see on the channel, let us know. And as for me, I'll see you in the next one.