Slack Real-Time Messaging

An analysis of the Slack architecture explaining how they send millions of messages in real time.

Hi Friends,

Welcome to the 56th issue of the Polymathic Engineer newsletter.

Last week, I read some interesting blog posts on the Slack engineering blog, and it would be great to discuss it with you.

The outline will be as follows:

• The Slack architecture

• Service discovery

• How Slack clients boot up and send messages

• System Design Considerations

• Posts that make you think

Slack Architecture

Slack is a popular communication platform that handles millions of messages daily across numerous channels. Teams create channels for various projects or topics to collaborate on in real time.

Several services make the backbone of Slack's system, all developed in Java.

The more relevant services are:

• Channel Servers. They are stateful services that store information about channels in memory. This setup allows quick message delivery to clients, avoiding the need to access external storage like databases frequently. However, it presents scalability challenges, such as the need for more memory as the number of users grows and the risk of data loss if a server fails. Slack addresses these issues by using replication with consistent hashing, which helps efficiently distribute data across the cluster, minimizing rebalancing when servers are added or removed. Every channel has a numeric ID, and consistent hashing is used to know which requests belong to which channels.

• Gateway Servers. They are intermediary servers between Slack clients and Channel Servers strategically placed in various cloud regions to ensure fast connections. These servers are also stateful and retain user information and WebSocket channel subscriptions.

• Presence Servers. These servers track which users are online, powering the green presence dots seen in the Slack interface. They use hashing to assign users to specific servers. Slack clients can make queries to the Presence Servers through the Gateway Servers.

• Admin Servers. These servers bridge Slack's web application backend and the Channel Servers. Unlike the others, Admin Servers are stateless.

Service Discovery

Slack uses Consul as a service discovery tool. Service discovery is critical to every microservice architecture since it ensures seamless communication even if servers dynamically change.

All Channel Servers are registered in Consul, and the service discovery tool automates tracking each server's location and status. This makes it easier to manage the servers during auto-scaling, failures, and service upgrades. 

How Slack clients boot up and send messages

Before being able to send and receive messages, a Slack client has to follow a specific boot up procedure:

1. Get the user token and WebSocket connection setup from the Webapp backend. This hosts APIs called by Slack clients and includes JavaScript code for rendering the Slack clients.

2. Initiate a WebSocket connection to the closest proxy service

3. The request is forwarded to the Gateway Server.

4. The Gateway Server retrieves information about all the users' channels from the Webapp.

5. The Gateway Server subscribes asynchronously to all the Channel Servers that hold those channels and sends the first message to the client

Once the client is up and running, it can send messages as follows:

1. The client sends a message using the Webapp API.

2. The message reaches the Admin Server (AS), which uses the channel ID to find the appropriate Channel Server.

3. The Channel Server broadcasts the message to every Gateway Server subscribed to that channel.

4. Every Gateway Server receiving the message forwards it to all connected client.

System Design Considerations

Many design considerations can be made about the Slack architecture:

• Persistent Connections with Web Sockets. Each Slack client maintains a continuous connection with the backend via web sockets. This technology is ideal for real-time applications like Slack. It allows low-latency, bidirectional communication without repeatedly establishing connections, thus saving bandwidth and resources.

• Why a proxy service? Slack uses Envoy as a proxy and load balancer. Envoy, which supports web sockets, performs roles like TLS termination and integrates with Consul for service discovery. So, it balances the load across servers and provides additional features like security and observability.

• Webapp Location. The Webapp backend isn't deployed at the edge because it's not a performance-critical component. Most real-time interactions occur through the Gateway Servers. Moreover, placing the Web app at the edge could complicate interactions with other core services and increase network latency.

• Real-Time Broadcast: Once a client is connected, messages in a channel are instantly broadcast to all online clients. The message travels from the Webapp to the Admin Server, then to the appropriate Channel Server based on the channel ID. The Channel Server distributes the message to all subscribed Gateway Servers, delivering it to the clients.

Interesting posts

Writing readable code makes it unnecessary to write many comments. However, sometimes, they're necessary to document hacks, decisions, or workarounds. Comments telling what code does should only be for the sporadic cases where you are forced to use code that is hard to understand.

Unfortunately many tech companies started the year with layoffs: Discord, Google, Amazon Prime Video, X, Twitch, and so on. I want to express closeness to all the affected engineers there. It is not the first time and it won’t be the last: the managers increase the company headcount too much, and then employers pay the price.

The CEOs always say they feel responsible, but the employees are the ones being fired. The thing is that many executives are only going to be in that position for a short time and want to make sure their margins look good for the term. That’s how incentives work.

The ability to communicate and deal with people is way more important than having outstanding technical skills.