Why use Message Brokers in your System Design Interview?
date
Apr 3, 2024
slug
why-use-message-brokers-in-your-system-design-interview
status
Published
tags
System Design Interview
summary
Message Brokers are powerful in decoupling distributed systems. Learn the core concepts to use them in your next system design interview!
type
Post
probability
In my experience, nailing the system design interview is not about knowing the real-world architecture of youtube, google search, dropbox, Netflix, Twitter, Facebook, etc. by heart. No, it's all about knowing the concepts and technologies to draft those systems on the spot. If you approach learning those systematically, the whole interview preparation comes down to learning and exercising a very manageable amount of components and concepts.
At its core, you only need to learn how to answer four questions to design any system.
- How to transport data?
- How to store data?
- How to transform data?
- How to display data?
In this article, I introduce you to a widely-used solution to the problem of how to transport data within a distributed system, which you can use to draft your answer for almost any system interview design question.
Using Messaging to Scale Systems
When you start to design a system without scalability in min, the most trivial way to transfer data is to directly send messages to other parts of the system.
Once you reach the point where your system consists of a couple of service and asynchronous operations, which include some of your services - it gets very messy. If you stick to your original approach, it won't take long that your distributed system to become so tightly coupled that you struggle to move forward. I refer to this anti-pattern as the "distributed monolith".
If you choose to implement distributed messaging to facilitate message communication, your system architecture is set up for scalability!
Core Concepts
Distributed messaging is the foundation for a healthy and microservice architecture. The fundamental idea is to implement a component called the message broker. It handles the transfer of data between different services without them knowing anything about each other.
A broker allows loosely coupled communication, meaning that the message sender doesn't know who will receive the message. The broker is responsible for routing the message to the proper destination. Distributing the message system allows increasing the subscribers as needed to handle more messages. It also adds desired properties like reliability and persistence.
This results in a more loosely coupled application through the composition of multiple services, which can be deployed and scaled independently without the rest of the system knowing it.
How Distributed Messaging Works
Most messaging solutions support two methods of distribution: queue-based and topic-based.
Queues (Point-to-Point Messaging)
Point-to-Point communication allows a publisher to send a message to a single service from a pool of subscribers. The broker selects the subscriber that will receive the published message for processing. Only one subscriber will receive a message published to the queue unless the subscriber fails to process the message within a given timeout period.
Topics (Publish/Subscribe Messaging)
Topic-based messaging allows to publish every message to a topic that distributes it to every subscriber currently registered. The broker doesn’t care if the message was processed by all subscribers or just a subset of them.
Applied to System Design Interview Questions
In the interview process of big tech companies (FAANG especially), it is very common to be challenged with system design questions. Typically you are asked to create a high-level design for famous systems like Uber, Google, Airbnb, or Netflix. Message brokers are an excellent choice to ensure reliable and scalable communication between your services.