Service Mesh vs API Gateway (Part III)
Two patterns that look similar, solve different problems, and work best together.
Hi Friends,
Welcome to the 178th issue of the Polymathic Engineer newsletter.
This week, we are wrapping up our series of articles on service mesh. In the first post, we talked about what a service mesh is and how its architecture works. In the second, we looked at what it actually gives you in practice and at the trade-offs that come with it.
There is one more topic that feels at home in this series, and often confuses folks: the difference between a service mesh and an API gateway. Both manage traffic, both can do things like authentication, routing, and observability, and sit somewhere between a client and a service. It’s easy to think of both as the same kind of tool, or to reduce the difference to a simple rule of thumb such as “API gateways are for north-south traffic, service meshes are for east-west,” and go on.
That shortcut is a one-liner, but it masks what each tool is actually good at. They were built to tackle different problems, and treating them as interchangeable usually leads to one of two architectural mistakes: forcing one tool to do the other’s job, or layering them together without a clear understanding of why.
In this issue, we look at API gateways and service meshes side by side. We will discuss what an API gateway does, how it differs from a service mesh, when to use which, and how they work together when used correctly.
The outline is as follows:
What an API gateway does
Side-by-side differences
When to use which
Using them together
Wrapping up the series
Project-based learning is the best way to develop technical skills. CodeCrafters is an excellent platform for tackling exciting projects, such as building your own Redis, Kafka, a DNS server, SQLite, an HTTP server, or Git from scratch using your favorite programming language. Now you can also try to build your own Claude Code (for free, still in beta).
Sign up, and become a better software engineer.
What an API gateway does
An API gateway lives at the edge of your system and is the front door for everything coming in from outside. Mobile apps, web browsers, partner systems, and any other external client send their requests to the gateway, and the gateway is responsible for handing them off to the right backend service. It is a centralized component, which means that a single instance, or a small cluster, takes care of all the traffic entering the system.
This central position is what makes the gateway so useful. Each external request flows through it, so you can put a lot of cross-cutting work in one place instead of spreading it across many services. The gateway works at the application layer of the OSI model. This means that it understands HTTP, HTTPS, gRPC, and WebSocket, and it can read URLs, headers, methods, and payloads to decide what to do with each request.
The most basic job of an API gateway is routing. It looks at the path or the method of an incoming request and forwards it to the right backend service. It can also translate between protocols: a common example is accepting an HTTP call from a mobile client and turning it into a gRPC call to an internal service.
Another big responsibility is authentication and authorization. The API gateway does validation once at the edge, instead of each backend service having to include its own logic. It can validate OAuth2 tokens, JWTs, API keys, and integrate with an identity provider. The requests that do not pass these checks are rejected before they reach the internal services.
The gateway also takes care of rate limiting and request shaping. It is able to throttle abusive clients, block requests that aren’t valid, and stop a single noisy customer from taking over the whole system. This keeps bad behavior on the outside from affecting services in the backend.
Some other common tasks are spreading the load across multiple backend instances, caching frequent responses to speed up the system, or gathering logs and metrics for the traffic that goes through. Since the gateway is the single point through which all external traffic passes, it is a natural place to gather observability data on how clients are using the APIs.
Many API gateways also handle API versioning. Backend services do change, but the contract that external clients rely on cannot break overnight. The gateway can give a stable interface to customers while sending requests to newer versions of the services behind the scenes. It can also change the response so that older clients get what they expect.
Besides traffic, API gateways today are typically the key to what is sometimes referred to as API productization. If you think about APIs as products which are consumed by internal teams, partners, or third-party developers, routing and auth are not enough. A lot of gateways are integrated with large API management platforms with developer portals, self-service onboarding, monetization features, sandbox environments, and access control dashboards. This transforms the gateway from a network infrastructure element into the front office of your APIs.
For a more detailed overview of API gateways, you can refer to our previous article.
Side-by-side differences
Now that we have looked at both an API gateway and a service mesh in isolation, let’s put them side by side. There are 5 main differences worth knowing about.
The first one is the deployment model. A gateway is a centralized component. You can have a single instance or a small cluster at the edge that handles all the traffic coming from the outside. That makes it easier to apply global policies and observe all external interactions from a single place.
On the other hand, a service mesh is decentralized. It uses a sidecar proxy adjacent to each service replica instead of a single component. This shifts traffic control to each service, giving you fine-grained control, but adding moving parts to the system.