Messaging in Distributed Systems - Part I

A deep dive into message queues: core components, messaging protocols, and communication styles.

Hi Friends,

Welcome to the 87th issue of the Polymathic Engineer newsletter.

I feel very excited since this week we will start discussing a core component of distributed systems: message queues.

This is the outline:

• Why using message queues

• Components of a message queue

• Message Routing and Communication Styles

• Messaging Protocols

Why using message queues

Let's start by explaining message queues and how they work through a practical example.

Imagine we have an online store application that lets people order different products.

When a customer makes an order, our API gateway takes care of the request by storing the order details in a database and interacting with several services, such as processing payment, updating the inventory, etc.

These operations can take several seconds or even minutes to complete, especially during high-traffic periods.

A naïve implementation might have the API gateway calling the services and waiting for their responses. However, this approach could lead to long response times for the user since the tasks are time-consuming and can't be completed instantly.

It also wouldn't work to use a "fire-and-forget" pattern where the API gateway sends requests to each service and moves on without waiting for responses wouldn't work.

Suppose any downstream services (like payment processing or inventory updating) fail to handle the request. In that case, the order might be partially processed or even lost, leading to inconsistencies in the system.

A more robust solution is introducing a message queue between the API gateway and the various services responsible for processing the order. Message queues enable indirect communication where a producer (the API gateway) writes a message to a channel or message broker, and one or more consumers (the inventory, payment, and shipping services) read the message from the channel.

Each message in the queue has a well-defined format with a header and a body. The header includes metadata like a unique identifier for the message, while the body contains the actual message.

Here's how our application would work after introducing a messaging layer:

1. The API gateway receives an order and creates a message containing the order details.

2. The API gateway sends this message to the message queue and immediately responds to the client with a 202 Accepted status, indicating that the order has been received and is being processed.

3. The inventory, payment, and shipping services independently read messages from the queue and process them at their own pace.

4. If any service fails to process a message, the message remains in the queue and can be retried later.

This approach offers several advantages:

• The API gateway doesn't need to know about or directly communicate with the other services.

• If a service is temporarily unavailable, requests aren't lost but can be processed when the service comes back online.

• Services can be scaled independently based on their specific load, and the queue can distribute messages across multiple service instances.

• The queue acts as a buffer during traffic spikes, preventing the backend services from becoming overwhelmed.

Components of a message queue

Our example shows that a message queue system consists of three main components: producers, message brokers, and consumers. Let's discuss all of them and their responsibilities in more detail.