Welcome to the 180th issue of the Polymathic Engineer newsletter.

In a recent article, we talked about the DRY principle and why removing duplication can lead to fewer bugs and more reusable code. As you know, DRY is one of the most well-known rules in software engineering, and most of us learn it early in our careers.

But here’s the thing: DRY is not free. Every time we remove duplication, we pay for it somewhere else. Sometimes it is the coordination between teams. Sometimes it is the tight coupling between components. Sometimes a new business requirement arrives, our common abstraction is not flexible enough to support it and starts to push back.

In a small codebase owned by a single team, this cost is small, and DRY almost always wins. In distributed systems with many services and many teams, the cost can become larger than the duplication it is trying to remove.

There is also a second question that engineers tend to skip. Two sections of code that appear the same may not mean the same thing. Sometimes what looks like duplication is just two things that look alike today and may evolve differently tomorrow,

In this issue, we will see when keeping duplication is a reasonable tradeoff, and when it is worth the effort to remove it. The outline is as follows:

• The coordination tax

• Is it really duplication?

• Sharing code across services: from library to microservice

• The same tradeoff inside a codebase

• Closing thoughts

Quantify and Eliminate the “Friction Tax” on Your Codebase

Technical debt isn’t just an abstract problem. As AI accelerates code generation, it also compounds debt, acting as a silent tax on your engineering velocity and exponentially increasing developer cognitive load.

SonarQube gives tech leaders and architects the exact visibility they need to stop software entropy in its tracks. By analyzing your codebase against thousands of language-specific rules, SonarQube acts as an automated, multi-layered guardrail that keeps your code predictable, adaptable, and architecturally sound.

Discover how to systematically measure and manage code-level debt with SonarQube.

Thanks to the SonarQube Team for collaborating with me on this article.

The coordination tax

To see why duplication is sometimes the better tradeoff, let’s suppose that two teams are building two services that need a similar piece of functionality. Team A writes its version in its own service, and Team B does the same. The two sections of code look almost identical, but they are in different repos and are owned by different teams.

This is the classic scenario where most engineers reach for DRY and say: it should be a shared component. On the surface, that sounds right. Two teams writing the same thing twice is exactly what DRY warns us about.

However, there is a hidden benefit to leaving the duplication in place. Both teams can move at full speed. There are no meetings to align on the design, no shared roadmap, no review process across team boundaries, and no merge conflicts on someone else’s repository. Each team owns its copy and can change it whenever it wants.

The parts of a project that need everybody in the room don’t get faster when you add more people. They slow down. The cost of sharing is in the meetings, design reviews, sign-offs, and merge negotiations. When you remove duplication by adding a shared component, you also introduce coordination among everyone who uses it.

Of course, the other side of the coin is real too. When each team owns its own copy, a bug fixed in one area doesn’t make it to the other. Knowledge doesn’t spread. The two implementations slowly drift apart, and after a few months, they may not even behave the same way for the same input. This is particularly risky for security-sensitive code because the likelihood of two independent teams getting authentication or encryption right twice is quite low.

So the question is not if we should remove duplication, but what is more expensive in our situation. Which one creates more trouble: the coordination cost of sharing, or the drift cost of duplicating? The answer is different for each team and depends on how often the code changes and how different the two versions are going to be.

Is it really duplication?

Before we think about how to share code, there is another crucial question to answer first: is what we are looking at really duplication?

The fact that two sections of code look very similar does not mean they solve the same problem. Two validation functions that both check “value greater than zero” can look the same on the screen, but they might have different purposes. For example, the first might be validating the user’s age and the second a product price. The validation rules are the same now, but they may evolve differently tomorrow.

This is what we call incidental duplication. It is code that looks the same but does not encode the same knowledge. It is different from inherent duplication, where the same business rule is represented in multiple places. Inherent duplication is something we usually want to remove. Incidental duplication is something we usually want to leave alone.

The tricky thing is that it can be hard to distinguish the two scenarios, especially early in a project. What appears to be duplicated code today could turn out to belong to two different domains tomorrow. Here are some hints to help:

• The two call sites are owed by different teams or driven by different stakeholders

• The reason for the logic is different on each side, even though the code looks identical

• You can imagine a future where one side changes and the other does not

When any of these signals are present, it is usually better to let the duplication live for a while. If there is a pattern, it will become clear over time, and the proper abstraction will emerge from real evidence rather than a guess. But if the pattern never solidifies, you have lost nothing.

There is also an asymmetry that makes this more than a suggestion. Once you create a shared abstraction, every caller is coupled to it. As new callers are added, the coupling becomes stronger since any change needs to satisfy all of them. Splitting it back apart once there are a dozen consumers is hard, sometimes harder than the duplication you were trying to remove. Going the other way is much easier. Two similar things can be merged into one whenever you decide they really are the same.

When you are in doubt, it is better to keep the duplication. The price of being patient is small. Premature abstraction is expensive, paid over many years by many engineers in code that no one fully knows how to disentangle.

Sharing code across services: from library to microservice

Let’s make this concrete with a real-world example. Suppose we have two services in the system: payments and notifications. Both of them expose a public API, and both need to throttle clients to protect themselves from abuse. Team payments writes its own rate limiter inside the payments service. Team notifications does the same.

After a few months, the duplication becomes a pain. A bug in the token bucket logic gets fixed on the payments side, but it is not applied on the notifications side. At this point, someone suggests the obvious next step: let’s put the rate-limiting code into a shared component. The question is: what shape should that component take?

Option one is a library. Both services pull it in as a dependency. The rate limiter itself is in its own repo. This is the cheapest path. The fixed cost of setting up the library is small: a build, a versioning scheme, an artifact repository, and some documentation. Once it is in place, bug fixes are available to all clients at the next release. Both teams work on the same codebase, and knowledge spreads naturally.

However, a library is not a black box. When you import it, its code is included in your service, with all the consequences that follow. The first is the programming language. If payments is written in Go and notifications in Java, a shared library is not going to work. The second one is transitive dependencies. Your rate-limiting library pulls in a Redis client at version 4. The payments service uses another library that requires the Redis client to be version 6. Now everyone fights about which version wins, and the resolution rules of your build tool tell you which one to use. Debugging such conflicts can be tricky and may even turn off other teams from adopting your library in the first place. The right defensive move is to keep direct dependencies to a minimum, but the issue never really goes away.

In the end, libraries exchange duplication for coupling on the dependency level. The two services cannot evolve independently, but must agree on the language stack and tolerate each other’s transitive dependencies.

The second option is to extract rate limiting as its own microservice. Both clients now invoke an HTTP (or a gRPC) endpoint rather than importing a package. The boundary between services is an API, which is a cleaner contract than a binary dependency. The rate-limiting service can be deployed and scaled on its own. The team that owns it is free to change the implementation, as long as the API stays compatible. API evolution is also easier because you can measure endpoint usage and deprecate the ones nobody calls anymore.

However, this approach is more expensive. Each request for payments or notifications now makes an extra network call to the rate-limiting service. Latency budgets become tight, and you may need caching, retries, or speculative execution to meet your service-level agreements. If the dependency breaks, well, that is also your problem now. What should payments do if the rate-limiting service is down? Remain safe and decline all incoming requests, or accept them all and stay available?

Both decisions have implications, and you need to choose thoughtfully. On top of that, running a service has operational costs such as monitoring, on-call rotations, alerting, and capacity planning. The first one in particular is a serious investment.

To wrap up the comparison, a library is cheap and tightly coupled at the dependency level. A microservice is expensive and loosely coupled at the API level. There isn’t a universal answer. The choice depends on how complex the shared logic is, whether it really has its own business domain, what your latency requirements are, and whether your organization can afford another running service. The simpler the logic, the more likely a library is to be enough. The more independent and complex the domain is, the more a microservice starts to make sense.

The same tradeoff inside a codebase

The same tradeoff applies one step down. The standard mechanism for sharing code inside a single codebase is inheritance. You extract the common functionalities into a base class and let subclasses provide the specialized behavior. This is clean in theory, but has the same coupling cost as a shared library or shared service, just on a smaller scale.

Let’s stick with the notifications service from the previous section. Suppose it has to send messages over two different channels: email and SMS. Both channels need the same infrastructure: a buffer for outgoing messages, logic for retrying messages that fail temporarily, and batching for when traffic is high. The call that sends the message to the provider is the only real difference between them.

The textbook solution is to extract a BaseNotificationSender class with the buffering, retry, and batching logic, and let EmailSender and SmsSender extend it by adding only the send method. DRY is satisfied. Both classes look small and easy to read. Everyone is happy until the next change arrives.

Say marketing wants to run a large campaign and asks that SMS switch to unbounded buffering for the duration. Now we have an issue. The buffering logic is in the parent class, and the parent class is shared with email. We can’t change the behavior for SMS without changing it for email too, and email has good reasons to preserve its bounded buffer.

The instinct is to treat it as an exceptional situation. You could add an if statement in the parent class to check whether it is an instance of SmsSender, or pass a flag, or add a hook method. All these ways work, in the sense that they make the code compile and the campaign ship. But they also leak SMS-specific knowledge into the parent class, and the parent class no longer represents a clean abstraction. It is now a place where everyone’s special cases live. The next time a divergent change comes in for email, we do the same thing again. After a few iterations of this, the parent class gets harder to understand than the two duplicated subclasses would have been in the first place.

The takeaway is that inheritance is shared code with extra coupling. The moment one child needs to diverge, the abstraction starts to push back.

However, there is another approach to deal with this, which is composition. Instead of one inheritance hierarchy, you split the responsibilities into distinct pieces: a buffer, a retry policy, and a transport. Then you plumb them together. The email sender uses a bounded buffer with HTTP transport. The SMS sender uses an unbounded buffer and a different transport. The buffer is no longer coupled to the transport. The flexibility is real, and so is the cost: there are more moving parts to understand, and the reader has to hold more pieces in their head to follow what happens when a message is sent.

There is no free lunch. Inheritance is cheap and rigid. Composition is flexible but has more moving parts. The best choice really depends on the divergence you expect.

Closing thoughts

DRY is not wrong. It is just not free. If we remove duplication, the risk is that we pay for it somewhere else. These costs can be in the form of coordination issues between teams, dependency problems, extra network calls, or abstractions that get in the way when a new change comes in.

The skill is being able to tell when the coordination cost of sharing is higher than the maintenance cost of duplicating. And that calculation shifts as you cross boundaries.

Inside a small codebase owned by one team, DRY almost always wins. But across team boundaries, service boundaries, or tech stacks, the math gets less straightforward, and duplication can be the better answer.

There is one more thing worth keeping in mind. It’s much easier to merge two similar things later than to split a shared abstraction once it has a dozen callers depending on it. Going from duplicated to shared is easy to revert. Going from shared to duplicated is hard, sometimes harder than living with the duplication in the first place.

So when in doubt, let the duplication live. The right abstraction will emerge from real evidence, or it will not. Either way, you are better off than committing to the wrong one too early.

Welcome to the 179th issue of the Polymathic Engineer newsletter.

Regardless of whether you are an experienced machine learning practitioner or just a data scientist who has started their journey, you have surely had the same frustrating experience. You built a model, trained it on your data, everything looked fine, but then it fell apart in production.

There are two issues to be aware of. The first one is called underfitting. A model that underfits is too simple for the data. It makes many mistakes, even on the training data itself, because it can't capture the patterns underneath.

The second problem is called overfitting. This is the opposite of the first problem. In this situation, the model memorizes the training set perfectly; therefore, it can’t make generalizations.

In this article, we explore both underfitting and overfitting, as well as the methods to detect and address them. The outline is as follows:

• What can go wrong when training a model?

• Underfitting and overfitting: two sides of the same coin

• Detecting the problem with testing

• The golden rule and the validation set

• The model complexity graph

• Solving the problem: regularization

• Measuring complexity: L1 and L2 norms

• The regularization parameter

• When to use L1 vs L2

Project-based learning is the best way to develop technical skills. CodeCrafters is an excellent platform for tackling exciting projects, such as building your own Redis, Kafka, a DNS server, SQLite, an HTTP server, or Git from scratch using your favorite programming language. Now you can also try to build your own Claude Code (for free, still in beta).

Sign up, and become a better software engineer.

What can go wrong when training a model?

Let’s try to answer this question starting with a simple analogy. Imagine you have to study for an exam. There are many things that could go wrong as you prepare.

You might not have studied enough. There is no way to fix that, and you are likely to perform poorly. But what if you studied a lot, just in the wrong way? For example, you choose to memorize the entire textbook word for word instead of trying to understand the material. Will you do well? Likely not, because you memorized everything without actually learning anything. You won’t know how to answer questions on the exam that you haven’t seen before.

The best approach is to study in a way that lets you answer questions you haven’t seen during your preparation.

The same things can go wrong with machine learning models. Not studying enough is like underfitting: the model is too simplistic to learn anything helpful from the input data. Memorizing the textbook is like overfitting: the model becomes too complicated and remembers the training data rather than learning the underlying patterns. A good model is one that learns the data properly and can make good predictions on new data it has never seen.

There is a fundamental trade-off in machine learning between finding the best model for the training data and ensuring it performs well with unseen data. Optimization is the process of tuning a model to work as well as possible on the training data. On the other hand, generalization is how well the model performs with data it has never seen before. The goal is always to generalize well, but there is a catch: you can only control the optimization. You adjust the model to fit the training data, and you hope it works well on other data. If you optimize too much, you can overfit, and generalization gets worse.

In the next section, we will look at underfitting and overfitting in more detail and see what they look like in practice.

Underfitting and overfitting: two sides of the same coin

A different way to look at underfitting and overfitting is through the lens of problem- solving. There are two mistakes you can make when you have a problem to solve. You can make the problem too simple and come up with an overly basic solution. Or you can overcomplicate the problem and come up with a solution that is too elaborate.

Imagine your task is to fix a leaking faucet. If you just put a bucket under it, you have oversimplified the problem. The faucet keeps dripping, and you are only catching the water, not fixing anything. This is underfitting: you are trying to model your data with something that is too simple to work effectively.

On the other hand, if you knock out all the plumbing in your house to fix a small drip, you have overcomplicated the solution. You may have fixed the leak, but this approach wasted resources and made the work more difficult than necessary. This is overfitting: your data might be simple, but you try to fit it with a model that is too complex. The model will fit the training data, but it will memorize it rather than learn from it.

Let’s look at a concrete example with polynomial regression. Suppose that you have a dataset that looks like a parabola, which is a curve that goes up and then down. You want to fit a polynomial to this data, but you don’t know what degree. Should you use a line, a quadratic, or something more complex?

If you fit a line (a polynomial of degree 1), the model is too simple. It is impossible for a straight line to capture the shape of a parabola, and the model underfits. If you fit a quadratic (a degree-2 polynomial), the model captures the essence of the data well. It neither underfits nor overfits. However, if you fit a polynomial of degree 10, the model becomes too flexible. It can bend and twist to reach each point in the dataset, but that is not the goal. Instead of learning that the shape looks like a parabola, it memorizes the exact positions of each point. The model overfits.

The real problem with overfitting shows up when you try to make predictions on new data. A model that memorized the training set will perform very poorly on data it has never seen before. It learned the noise and quirks in the training data, rather than the underlying pattern. Here is a simple way to tell what is happening with your model:

• Underfitting: The model performs poorly on the training data and poorly on new data. It is too simple to learn anything useful.

• Good fit: The model performs well on the training data and on new data. It has learned the underlying patterns.

• Overfitting: The model performs very well on the training data but poorly on new data. It has memorized rather than learned.

But how can you actually measure this? How can you know if your model will perform well on new data before you deploy it? This is where testing and validation come in.

Read more

Welcome to the 178th issue of the Polymathic Engineer newsletter.

This week, we are wrapping up our series of articles on service mesh. In the first post, we talked about what a service mesh is and how its architecture works. In the second, we looked at what it actually gives you in practice and at the trade-offs that come with it.

There is one more topic that feels at home in this series, and often confuses folks: the difference between a service mesh and an API gateway. Both manage traffic, both can do things like authentication, routing, and observability, and sit somewhere between a client and a service. It’s easy to think of both as the same kind of tool, or to reduce the difference to a simple rule of thumb such as “API gateways are for north-south traffic, service meshes are for east-west,” and go on.

That shortcut is a one-liner, but it masks what each tool is actually good at. They were built to tackle different problems, and treating them as interchangeable usually leads to one of two architectural mistakes: forcing one tool to do the other’s job, or layering them together without a clear understanding of why.

In this issue, we look at API gateways and service meshes side by side. We will discuss what an API gateway does, how it differs from a service mesh, when to use which, and how they work together when used correctly.

The outline is as follows:

• What an API gateway does

• Side-by-side differences

• When to use which

• Using them together

• Wrapping up the series

Project-based learning is the best way to develop technical skills. CodeCrafters is an excellent platform for tackling exciting projects, such as building your own Redis, Kafka, a DNS server, SQLite, an HTTP server, or Git from scratch using your favorite programming language. Now you can also try to build your own Claude Code (for free, still in beta).

Sign up, and become a better software engineer.

What an API gateway does

An API gateway lives at the edge of your system and is the front door for everything coming in from outside. Mobile apps, web browsers, partner systems, and any other external client send their requests to the gateway, and the gateway is responsible for handing them off to the right backend service. It is a centralized component, which means that a single instance, or a small cluster, takes care of all the traffic entering the system.

This central position is what makes the gateway so useful. Each external request flows through it, so you can put a lot of cross-cutting work in one place instead of spreading it across many services. The gateway works at the application layer of the OSI model. This means that it understands HTTP, HTTPS, gRPC, and WebSocket, and it can read URLs, headers, methods, and payloads to decide what to do with each request.

The most basic job of an API gateway is routing. It looks at the path or the method of an incoming request and forwards it to the right backend service. It can also translate between protocols: a common example is accepting an HTTP call from a mobile client and turning it into a gRPC call to an internal service.

Another big responsibility is authentication and authorization. The API gateway does validation once at the edge, instead of each backend service having to include its own logic. It can validate OAuth2 tokens, JWTs, API keys, and integrate with an identity provider. The requests that do not pass these checks are rejected before they reach the internal services.

The gateway also takes care of rate limiting and request shaping. It is able to throttle abusive clients, block requests that aren’t valid, and stop a single noisy customer from taking over the whole system. This keeps bad behavior on the outside from affecting services in the backend.

Some other common tasks are spreading the load across multiple backend instances, caching frequent responses to speed up the system, or gathering logs and metrics for the traffic that goes through. Since the gateway is the single point through which all external traffic passes, it is a natural place to gather observability data on how clients are using the APIs.

Many API gateways also handle API versioning. Backend services do change, but the contract that external clients rely on cannot break overnight. The gateway can give a stable interface to customers while sending requests to newer versions of the services behind the scenes. It can also change the response so that older clients get what they expect.

Besides traffic, API gateways today are typically the key to what is sometimes referred to as API productization. If you think about APIs as products which are consumed by internal teams, partners, or third-party developers, routing and auth are not enough. A lot of gateways are integrated with large API management platforms with developer portals, self-service onboarding, monetization features, sandbox environments, and access control dashboards. This transforms the gateway from a network infrastructure element into the front office of your APIs.

For a more detailed overview of API gateways, you can refer to our previous article.

Side-by-side differences

Now that we have looked at both an API gateway and a service mesh in isolation, let’s put them side by side. There are 5 main differences worth knowing about.

The first one is the deployment model. A gateway is a centralized component. You can have a single instance or a small cluster at the edge that handles all the traffic coming from the outside. That makes it easier to apply global policies and observe all external interactions from a single place.

On the other hand, a service mesh is decentralized. It uses a sidecar proxy adjacent to each service replica instead of a single component. This shifts traffic control to each service, giving you fine-grained control, but adding moving parts to the system.

Read more

Welcome to the 177th issue of the Polymathic Engineer newsletter.

This is the final article in our series on recursion. In the last article, we looked at the divide-and-conquer pattern, which breaks a problem into independent subproblems, solves each one, and combines the results. This time, we examine depth-first search, a pattern for traversing trees and graphs.

The main idea behind DFS is simple: start at a node, visit its children or neighbors recursively, and go as deep as possible before backtracking. When you reach a dead end, you return and try a different path. You keep doing this until you find what you are looking for or you run out of paths to explore.

DFS is the pattern you reach for when you need to find a path, check if something is reachable, or explore all possible paths through a structure. It works on trees, graphs, and many problems that can be represented as graphs, even if they don’t look like one at first glance.

The outline is as follows:

• The Core Pattern

• Finding Paths

• Handling Cycles

• Finding All Paths

• Matrices as Implicit Graphs

• When DFS Applies

Your AI shouldn’t grade its own homework

Claude Code writes beautiful code. So does Codex. But here’s the thing, they also think they write beautiful code. And when you ask an AI to review code it just wrote, you get the intellectual equivalent of a student grading their own exam. Shockingly, they always pass.

CodeRabbit CLI plugs into Claude Code and Codex as an external reviewer, different AI Agent, different architecture, 40+ static analyzers, and zero emotional attachment to the code it’s looking at. The agent writes, CodeRabbit reviews, and the agent fixes. Loop until clean.

You show up when there is actually something worth approving.

One command. Autonomous generate-review-iterate cycles. The AI still does the work. It just doesn’t get to decide if the work is good anymore.

Free tier available. Try CodeRabbit’s CLI.

Thanks to the CodeRabbit team for collaborating with me on this newsletter issue.

The Core Pattern

The simplest example of DFS is searching a binary tree to check whether it contains a particular value. We start at the root and recursively search the left and right subtrees. If we find the target value, we return true, and if we reach a null node, we return false. If either subtree contains the value, the whole tree contains it. Here is a possible way to implement this:

def contains(node, target):
    if node is None:
        return False
    
    if node.value == target:
        return True
    
    return contains(node.left, target) or contains(node.right, target)

Let’s do a quick walk-through on the following example.

Suppose we want to find 4. We start at 5. As it is not the target, we proceed on the left subtree rooted at 3. Node 3 is not the target either, so we proceed by checking its left subtree, rooted at 1. Node 1 is not the target, and it has no children, so we return false.

We backtrack to 3 and search its right subtree rooted at 4. Node 4 is the target, so we return true. This true value propagates back up through 3 and then 5, and the function returns true.

This is the essence of DFS: go down one path as far as possible, and if you do not find what you are looking for, backtrack and try another path.

The time complexity is O(n), where n is the number of nodes in the tree. In the worst case, we go through each node until we identify the target or determine it is not there. The space complexity is also O(n) because of the recursion stack. This happens when the tree is completely imbalanced and resembles a linked list. So the depth is n.

One thing to keep in mind when analyzing tree problems is to be clear about what n represents. Here, n is the total number of nodes. If the tree were balanced, the depth would be O(log n), and the space complexity would improve accordingly.

Finding Paths

Checking whether a tree contains a value is useful, but often we need more. We would like to find the actual path from the root to the target node. This is a slight difference in how we think about the problem.

There are two approaches to do this. The first is to pass a path variable down through the recursive calls, adding nodes as we go and removing them when backtracking. The second is to build up the path as we return from the recursive calls.

I prefer the second approach. This is because at each node, we have two options as we travel down the tree: go left or go right. We do not know which way will lead us to the goal. But when we find the target and start returning, there’s only one way up. We just follow the successful path and construct the result as we go.

Here is a possible way to implement this:

def path_to_node(node, target):
    if node is None:
        return None
    
    if node.value == target:
        return [node.value]
    
    left_path = path_to_node(node.left, target)
    if left_path is not None:
        return [node.value] + left_path
    
    right_path = path_to_node(node.right, target)
    if right_path is not None:
        return [node.value] + right_path
    
    return None

The function returns None if the target is not in the subtree. Otherwise, it returns the path from the current node to the target. When the target is found, the return value is a list with just that node. As we keep returning up, each node adds itself to the front of the path.

Let’s walk through an example using the same tree as before to search for 4. We start at 5 and search the left subtree. We reach 3 and search its left subtree. We then reach 1, which has no children and is not the target, so we return None.

Back at 3, we search the right subtree. We reach 4, the target, and return [4]. Back at 3, we see that right_path is not None, so we return [3, 4]. Back at 5, we see that left_path is not None, so we return [5, 3, 4]. The path is built from the root down to the target as we return from recursive calls.

Both the time and space complexities are O(n), as for the basic containment check. We still visit each node once, and the recursion depth is at most n in an imbalanced tree.

Handling Cycles

The DFS pattern we have seen so far works on trees. But what about when we move to graphs? The main distinction is that a graph may contain cycles. If we are not careful, we could end up visiting the same node repeatedly and entering an infinite loop.

The solution is to mark all the nodes that we have already visited. We verify whether a node is in our visited set before traversing it. If it is, we skip it. If not, we add it to the set and keep exploring.

Here is how we can check if a path exists between two nodes in a graph:

def has_path(graph, source, destination, visited=None):
    if visited is None:
        visited = set()
    
    if source == destination:
        return True
    
    if source in visited:
        return False
    
    visited.add(source)
    
    for neighbor in graph[source]:
        if has_path(graph, neighbor, destination, visited):
            return True
    
    return False

The structure of the code is similar to what we have seen with trees. The base case is when the current node is the target. The recursive step is to try each neighbor and see if any of them leads to the destination.

The major addition is the visited set. We check whether the current node has already been visited; if so, we return false immediately. This prevents us from going in circles. Once we visit a node, we add it to the set so we do not visit it twice.

Let’s go through a quick example. Suppose we have a graph in which node A connects to nodes B and C, node B connects to nodes A and D, and node C connects to node A. We want to check if there is a path from A to D.

We start at A and add it to the visited set. We try neighbor B. We add B to visited and try its neighbor A. But A is already visited, so we skip it. We try B’s other neighbor D. Node D is the destination, so we return true. Without the visited set, we would have gone from A to B to A to B to A, forever.

The time complexity of the algorithm is O(V + E), where V is the number of vertices and E is the number of edges. Indeed, we visit each vertex at most once and examine each edge at most once. The space complexity is instead O(V) for the visited set and the recursion stack.

Finding All Paths

In the previous section, we checked whether there is a path between two nodes. But sometimes we need to find all possible paths, not just one. This requires a subtle but important change to how we handle the visited set.

When we are searching for a single path, we add a node to the visited set and leave it there. Once we know a node does not lead to the destination, there is no purpose in visiting it again. But when we are looking for all paths, different paths may share the same node. If we leave a node in visited after exploring it, other paths will not be able to use it.

The solution is backtracking. We add the node to the visited set before looking at its neighbors, then remove it when we have finished. This makes the node available for other pathways again.

Here is a possible implementation to determine all pathways between two nodes in a graph:

def all_paths(graph, source, destination, visited=None, path=None, results=None):
    if visited is None:
        visited = set()
    if path is None:
        path = []
    if results is None:
        results = []
    
    if source in visited:
        return results
    
    visited.add(source)
    path.append(source)
    
    if source == destination:
        results.append(path[:])
    else:
        for neighbor in graph[source]:
            all_paths(graph, neighbor, destination, visited, path, results)
    
    path.pop()
    visited.remove(source)
    
    return results

The code structure is the same as before, except for two things. First, we keep a path variable that tracks the path we are exploring. Second, after visiting all its neighbors, we delete the current node from both the visited set and the path variable.

Let’s look at an example. Suppose node A connects to nodes B and C, node B connects to node D, and node C also connects to node D. We want to find all paths from A to D.

We start at A, add it to visited and path. We try neighbor B, add it to visited and path. We try B’s neighbor D, which is the destination, so we save the path [A, B, D]. We then backtrack: remove D, then remove B from the visited set and the path.

Back at A, we try neighbor C, add it to visited and path. We try C’s neighbor D, which is the destination, so we save the path [A, C, D]. We backtrack and return both paths.

The crucial point is that D is common to both paths. If we hadn’t removed B from the visited set once we had explored it, the second path through C would still work. But in a more complicated graph, failing to remove nodes would prevent us from discovering valid paths.

In the worst case, the time complexity is O(V! * V) because there could be a factorial number of paths, and each path can contain at most V nodes. The space complexity is O(V) for the recursion stack, visited set, and current path.

Matrices as Implicit Graphs

Many problems involve working with matrices instead of explicit graphs. The positive news is that we can use the same DFS pattern without explicitly converting the matrix into a graph. The graph is the matrix: nodes are cells, edges are neighboring cells.

The crucial finding is that it is straightforward to compute neighbors in a matrix. If we are at cell (i, j) and can move in four directions, the neighbors are (i+1, j), (i-1, j), (i, j+1), and (i, j-1). For some problems, you can only go right or down, so the neighbors are just (i+1, j) and (i, j+1).

Let’s now look at an example. Suppose we have a matrix of integers and want to find the path with the largest product from the top-left corner to the bottom-right corner. As the following implementation shows, we can only go down or right:

def greatest_product_path(matrix):
    rows = len(matrix)
    cols = len(matrix[0])
    result = [float('-inf')]
    
    def dfs(i, j, product):
        if i >= rows or j >= cols:
            return
        
        product *= matrix[i][j]
        
        if i == rows - 1 and j == cols - 1:
            result[0] = max(result[0], product)
            return
        
        dfs(i + 1, j, product)
        dfs(i, j + 1, product)
    
    dfs(0, 0, 1)
    return result[0]

The structure is the same as the DFS we have seen before. The base case checks if we have gone outside the matrix bounds. Every time we reach the bottom-right corner of the matrix, we compare the current product with the best product found so far. In the recursive step, we try both possible moves: down and right.

Notice that we do not need a visited set here. Since we can only move right or down, there is no way to revisit a cell. We will never form a cycle, so tracking visited nodes is unnecessary.

This technique can be applied to many matrix problems: path finding, path counting, determining whether a destination can be reached, etc. The matrix gives the structure, and DFS provides the exploration strategy. We just need to define what a neighbor is and what the target state is.

The time complexity is O(2^(n+m)) in the worst case, where the matrix has n rows and m columns. This is because, at each cell, we have two alternatives, and the path length is at most n + m. The space complexity is O(n + m) for the recursion stack.

When DFS Applies

DFS is the right choice when you need to explore paths across some kind of structure, whether that structure is a tree, a graph, or something that can be represented as one.

Here are some common signs that a problem calls for DFS.

The first sign is that you need to find a path or check if one exists. If the problem asks whether you can get from point A to point B, or which path you would take to arrive there, DFS is a logical fit. You explore one path at a time, moving as deep as possible before trying alternatives.

The second sign is that you need to find all paths or explore all possibilities. If there are multiple valid routes, and you need to list them, DFS combined with backtracking does the job. You explore each path fully, record it if it is valid, then backtrack and try the next one.

The third sign is that the problem involves a tree or graph structure. This covers not only explicit trees and graphs, but also implicit ones such as matrices, game states, and decision trees. If you can define what a node is and what its neighbors are, you can apply DFS.

A few more examples of classic DFS problems:

• Determining if a graph is connected: start from any node and see if you can reach all others.

• Finding connected components: run DFS from each unvisited node and mark all reachable nodes as part of the same component.

• Detecting cycles in a graph: track nodes currently in the recursion stack and check if you revisit one.

• Solving puzzles like mazes or Sudoku: explore one choice at a time, backtrack when you hit a dead end.

• Computing properties of trees: height, diameter, path sums, and similar metrics often use DFS traversal.

One last thing that is worth keeping in mind is the distinction between DFS and BFS (breadth-first search). DFS goes deep along a path before going wide to explore other paths, while BFS first explores all neighbors at the current level and only then moves deeper. DFS is easier to implement recursively and uses less memory for deep, narrow structures. BFS is better when you need the shortest path in an unweighted graph.

Conclusion

DFS is one of the most flexible patterns for recursion. The basic principle is straightforward. Take one path as far as you can, and if it doesn’t get to where you want, backtrack and try another.

When it comes to trees, the pattern is simple. You traverse nodes recursively, returning results as you come back up. When dealing with graphs, you have to keep track of visited nodes to avoid cycles. In matrix problems, cells are nodes and neighboring cells are neighbors. The logic is always the same.

This brings us to the end of our series on recursion. We have covered six patterns:

• Iteration: process items one at a time and collect results as you go.

• Subproblems: solve a smaller version of the problem and build on it.

• Selection: choose whether to include or exclude each item when creating combinations.

• Ordering: determine which item comes next to produce permutations.

• Divide and conquer: split the input into independent parts, solve each, and combine.

• Depth-first search: explore paths by going deep before going wide.

These patterns are not mutually exclusive. Many problems combine elements of several patterns. A backtracking problem might use selection with DFS. A tree problem might use divide and conquer with subproblems. The more comfortable you become with each pattern, the easier it is to recognize which ones apply and how to combine them.

Recursion can feel tricky at first, but it becomes natural with practice. The key is to focus on two things: the base case and the recursive step. Get those right, and the rest takes care of itself.

Welcome to the 176th issue of the Polymathic Engineer newsletter.

This week, we continue our series of articles on service mesh. In the previous article, we discussed what a service mesh is and how it works. We’ve seen that a service mesh is a separate layer of infrastructure that handles all the traffic between services. The key idea is the sidecar pattern, where a small proxy runs next to each service instance and quietly takes over the networking job.

Now that we understand what a service mesh is, the natural next question is what you really get out of it. The short answer is that it gives you 4 things: traffic management, reliability, security, and observability. All microservice systems have to deal with these issues, and a service mesh solves them all in one place.

Another benefit that runs through all of these is polyglot support. As we have seen in the previous article, the library approach breaks down when teams use more than one programming language. A service mesh avoids this issue because it does everything in the proxy layer. Regardless of whether a service is written in Go, Python, Java, or C#, it gets the same retries, the same mTLS, the same metrics, and the same routing rules. There is no library for each language to build and maintain.

However, not all is for free. A service mesh has a cost in terms of latency, complexity, and resources. First, we will talk about the beneficial things, then about the trade-offs, and finally, we’ll take a quick look at the most typical service mesh implementations.

The outline is as follows:

• Traffic management and routing

• Reliability

• Security

• Observability

• Trade-offs: when a mesh isn’t worth it

• A quick tour of popular meshes

Project-based learning is the best way to develop technical skills. CodeCrafters is an excellent platform for tackling exciting projects, such as building your own Redis, Kafka, a DNS server, SQLite, an HTTP server, or Git from scratch using your favorite programming language. Now you can also try to build your own Claude Code (for free, still in beta).

Sign up, and become a better software engineer.

Traffic management and routing

The first thing a service mesh gives you is control over how traffic flows between your services.

When a service wants to call another service, it needs to know its address. In the past, when data centers were on-premises, these addresses were usually hardcoded because machines rarely changed. This isn't true anymore in the cloud. Instances come and go, IP addresses change, and services often move to different nodes. Your code will break whenever the network layout shifts if you hardcode addresses.

A service mesh takes care of this for you through service discovery. The sidecar keeps track of where each service lives in real time. The proxy finds out the service that the application is calling by name and directs the request there. The application doesn’t need to know any IP addresses, and it doesn’t need to be redeployed when something moves.

On top of this, the mesh can normalize service names across environments. The code might, for example, call a service called sessions-service. Then, when the code is put into production, it talks to a production instance. When it is put into staging, it talks to a staging instance. The mesh knows which one to use based on where it is running, so the application doesn’t have to carry environment-specific configuration.

Another key feature is load balancing. When there are multiple instances of a service, the mesh spreads the traffic across them. Because the sidecar understands application-level protocols like HTTP/2 and gRPC, it can do request-level load balancing, not just connection-level. This is important since one gRPC connection can handle numerous requests. If there is no request-level balancing, one connection would pin all the work to a single instance.

Beyond simple balancing, a service mesh supports more advanced traffic control. We can split traffic between two versions of a service: for example, send 95% of the calls to v1 and 5% to v2 to roll out a new release gradually. This is how canary deployments work at the infrastructure layer, without having to touch the application. We can also mirror traffic, sending a copy of the requests to a test instance to see how it handles production load, without affecting the real responses.

Traffic shaping and traffic policing are other useful tools. Shaping is when you slow down some traffic to make it fit a certain pattern. For example, you might give paying customers priority over free-tier users. Policing is making sure a contract is followed. For example, you could block a service that makes too many requests to a downstream dependency. Both are ways to keep the system stable when some parts behave badly.

You can set all of this without changing the application code. The mesh configuration is stored in files that the platform or infrastructure teams can update, and the sidecars pick up the changes.

Reliability

Networks fail. Services can become slow or unresponsive. Instances can crash or get too loaded during traffic spikes. In microservice systems, any call between services is a potential point of failure. If you don’t manage these failures in a consistent way, the issues spread through the system very quickly.

A service mesh can apply the well-known reliability patterns to every call, without the applications having to know about them.

The first one is retries. If a call fails because of a temporary error, the sidecar retries it automatically, usually with exponential backoff. This means that the proxy waits a bit longer between each attempt to avoid making things worse when a service is already having trouble. The mesh can also distinguish between errors that are worth retrying, like a network timeout, and errors that are not, like a 404 response.

Timeouts work in the same way. The sidecar gives up instead of waiting forever when a call takes too long. This is important because slow calls can add up quickly.If there is no timeout, a slow downstream service can use up all the system’s resources while every call upstream waits for a response that never comes.

Read more

Welcome to the 175th issue of the Polymathic Engineer newsletter.

Building software has changed a lot in the last decade. We moved from big monolithic applications to systems made up of many smaller services that talk to each other over the network. This change helped teams in several ways, such as the possibility to pick the right language for each job, faster development, and independent deployments.

But there is something to consider. A call to a function inside a monolith is now a call between two services over the network. And network calls are a whole different story. They can be slow, break or become lost. We have to think about things like balancing the load across service instances, retrying failed requests, encrypting traffic between services, and deciding which service can call which.

These are the kinds of problems a service mesh is built to solve.

At a high level, a service mesh is a dedicated layer that sits between your services and handles all the network traffic between them. The mesh handles encryption, timeouts, retries, and service discovery, so the services don’t need to take care of them in their own code. The application code makes a standard HTTP or gRPC call, and the mesh does the heavy lifting behind the scenes.

In this issue, we break down the fundamentals of service mesh: why the pattern exists, what it actually is, and how its architecture works. This is the first of three articles in a series. In the next two, we will talk about what a service mesh gives you in terms of features and how it stacks up against an API gateway.

The outline is as follows:

• Why we need a service mesh

• What a service mesh is

• The sidecar pattern

• Data plane and control plane

• Where a service mesh is deployed

Project-based learning is the best way to develop technical skills. CodeCrafters is an excellent platform for tackling exciting projects, such as building your own Redis, Kafka, a DNS server, SQLite, an HTTP server, or Git from scratch using your favorite programming language. Now you can also try to build your own Claude Code (for free, still in beta).

Sign up, and become a better software engineer.

Why we need a service mesh

As we discussed, switching to microservices made it harder for services to talk to each other. These problems are not new. Anybody who has worked on distributed systems knows that network calls are slow and unreliable, and getting services to trust each other is hard. The size is what changed. With dozens of services talking to each other, every engineering team has to deal with these difficulties daily.

The first attempt to solve this was the library approach. Teams would create a shared library that managed things like retries, circuit breaking, load balancing, and service discovery, and every service would use it as a dependency. The Netflix stack from the early 2010s is a well-known example. It had Eureka for service discovery, Hystrix for circuit breakers, and Ribbon for client-side load balance. For a while, it worked well, and it became the standard way to build reliable microservice architectures. But the approach started to show its flaws as systems got bigger and teams used more than one programming language.

The biggest issue was that every programming language needed its own version of the library. If the team used C#, Python, and Go, they needed three implementations that behaved the same way. This is tougher than it sounds because even slight changes in how timeouts or retries are handled might make the system behave differently.

Another problem was version skew. Different services would use different versions of the library at any given time. To roll out a bugfix in the retry logic, every team had to redeploy their services. This slowed down changes that should have been easy to make.

Last but not least, the library model put a lot of work on the application teams to keep things running. Every developer had to know how to set up circuit breakers, discovery clients, and retries, although these have nothing to do with the business problem they were trying to solve. The code was harder to read, test, and change since application logic and networking logic were mixed together.

These limitations made engineering teams look for another approach. If networking concerns were the same across services, and if the library version of it was painful to maintain across languages, maybe the best thing to do was to move them out of the application altogether.

This is the idea behind a service mesh.

What a service mesh is

A service mesh is a separate layer of infrastructure that handles all the traffic between the services in a distributed system. It takes care of routing, load balancing, timeouts, retries, encryption, and observability, without the services having to be aware of it.

The last sentence is the main point. In the library approach we discussed before, each service had to consume a dependency and call specific APIs to get circuit breaking or retries. With a service mesh, the service just makes a standard call to another service. The mesh sits in the network path and quietly adds the extra behavior before the call reaches its destination.

This kind of traffic, where one internal service talks to another, is usually called east-west traffic. Traffic that comes from outside the system, such as a mobile app or a web browser calling a backend, is called north-south traffic. These 2 kinds of traffic have different needs and patterns. A user may only send a request to the backend once in a while, but with a microservice design, one request can lead to dozens of calls between services. This is why service mesh tools focus on east-west traffic and look different from the tools that manage traffic at the edge of the system. We will come back to this comparison in the third part of the series.

So how does a service mesh sit in the middle of every service call without the services having to know? The answer is the sidecar pattern.

The sidecar pattern

The sidecar pattern is the trick that makes a service mesh work. The key idea is to run a small proxy process next to each service instance, on the same machine or inside the same pod if we are on Kubernetes. The proxy is always deployed with the service as a single unit. This is why it is termed a “sidecar,” like the sidecar of a motorbike.

Read more

Welcome to the 174th issue of the Polymathic Engineer newsletter.

In the last article in our series on recursion, we looked at the ordering pattern, which lets us generate all permutations of a set of items. This time, we talk about a different method: divide and conquer.

Instead of working on items one at a time, the main idea is to partition the input into multiple independent chunks, solve each separately, then combine the results.

If you have worked with algorithms, you have surely already seen divide and conquer in action. Binary search divides an array in half and searches only on the proper side. Merge sort splits an array in half, sorts both halves, and merges them back together. Tree traversals split a tree into left and right subtrees and visit each one individually.

The important thing is that we are not just taking something out and solving a smaller problem. We are breaking the input into several non-trivial subproblems and merging their solutions. That is what makes divide and conquer so powerful for some types of problems.

The outline is as follows:

• The Core Pattern

• Fixed Splitting: Binary Search

• Trying All Splits: Unique Binary Search Trees

• Other Divide and Conquer Examples

• When Divide and Conquer Applies

Project-based learning is the best way to develop technical skills. CodeCrafters is an excellent platform for tackling exciting projects, such as building your own Redis, Kafka, a DNS server, SQLite, an HTTP server, or Git from scratch using your favorite programming language. Now you can also try to build your own Claude Code (for free, still in beta).

Sign up, and become a better software engineer.

The Core Pattern

The divide and conquer pattern follows three steps. First, we divide the input into two or more smaller pieces. Then, we conquer each piece by solving it recursively. Finally, we combine the results from each piece to get the solution for the original input.

This is different from what we did with the iteration and subproblems patterns. With iteration, we processed one item at a time and collected the result. With subproblems, we removed one element and solved a smaller version of the same problem. In both cases, we were reducing the input by one item.

Using divide and conquer, we partition the input into multiple nontrivial chunks. For example, if we have an array of 8 elements, we might split it into two arrays of 4. Each of those gets split into 2 arrays of 2. We keep splitting until we reach a base case, solve each base case, and then merge the results back up.

The main criterion is that the subproblems must be independent. The solution of one part should not depend on the solution of another part. This is what allows us to solve them separately and then combine the results.

There are two main variations of this pattern, depending on whether we know where to split.

In the first variation, we know exactly where to divide the input. Binary search always splits in the middle. Merge sort always divides in the middle. The split point is fixed, and the only decision is which half to look at in the case of binary search, or how to merge the halves in the case of merge sort.

In the second variation, we do not know where to split. We have to try every possible split point, solve both sides for each split, and see which one gives us the best result. This comes up in grouping problems, where we need to find all ways to partition an input or the optimal way to group elements together.

We will look at examples of both variations in the following sections.

Fixed Splitting: Binary Search

Binary search is the simplest example of a divide and conquer algorithm with a fixed split point. We always divide the array in the middle and decide which half to search based on a comparison.

The problem is straightforward: given a sorted array and a target value, check whether the target is present in the array. The brute-force approach would have to check every element, which takes O(n) time. Binary search does it in O(log n) by halving the search space at each step.

This is how the algorithm works. We examine the middle element of the array. If it is equal to the target, we are done. If the target is smaller, we search the left half. If the target is larger, we search the right half. We keep doing this until we find the target or run out of elements to check.

Here is a possible way to implement this:

def binary_search(arr, target, low, high):
    if low > high:
        return False
    
    mid = (low + high) // 2
    
    if arr[mid] == target:
        return True
    elif arr[mid] > target:
        return binary_search(arr, target, low, mid - 1)
    else:
        return binary_search(arr, target, mid + 1, high)

Let’s walk through an example to grasp how this works. Say we have the array [1, 2, 3, 5, 7, 9, 11, 13] and we are looking for 3. We start with low = 0 and high = 7. The middle index is 3, and the element at that index is 5. Since 3 is smaller than 5, we continue our search in the left half: low = 0, high = 2.

Now the middle index becomes 1, and the element there is 2. Since 3 is larger than 2, we search the right half: low = 2, high = 2. The middle index is 2, and the element at that index is the target, so we return true.

The time complexity is O(log n) because we cut the search space in half at each step. If we have 8 elements, we need at most 3 comparisons. If we have 1024 elements, at most 10 comparisons are needed. The space complexity is also O(log n) due to the recursion stack.

The crucial thing to observe is that we always know exactly where to split. No need to attempt different split points. The midpoint is always the appropriate choice, and the comparison tells us which half to search. This is why binary search is so efficient.

Trying All Splits: Unique Binary Search Trees

Now, let’s look at a problem where we do not know where to divide. Instead of always splitting in the middle, we try every possible split point and combine the results.

Read more

Welcome to the 173rd issue of the Polymathic Engineer newsletter.

In a previous article, we talked about the dictionary problem and discussed different types of data structures that solve it, from unsorted arrays to hash tables and binary search trees. We have seen that hash tables are the best choice when all you need is to check if something is in a collection, since you can look up, add, and remove items in O(1) time.

But we have also seen that hash tables come with a cost: memory. Storing 100K words in a hash table means keeping all those words in memory, plus the extra space that the hash table needs. What if you are working in a context where memory is limited, like a router processing millions of packets per second, or a distributed database that needs to quickly check if an entry exists before writing to disk?

This is where Bloom filters come in. A Bloom filter is a data structure that can tell you if an item is in a set using only a small amount of memory with respect to a hash table. The catch is that it is not always right. A Bloom filter can tell you with certainty that something is not in the set. However, when it indicates something is there, it could be inaccurate. These inaccurate answers are known as false positives, and as we will see, you get to control how often they happen.

In this article, we will look in detail at how Bloom filters work, why they can give false positives but never false negatives, how to tune them to fit your needs, and where they are used in practice. This is the outline:

• How Bloom filters differ from hash tables

• How Bloom filters work

• Why are there no false negatives

• Why are there false positives

• Tuning the filter

• Real-world use cases

Project-based learning is the best way to develop technical skills. CodeCrafters is an excellent platform for tackling exciting projects, such as building your own Redis, Kafka, a DNS server, SQLite, an HTTP server, or Git from scratch using your favorite programming language. Now you can also try to build your own Claude Code (for free, still in beta).

Sign up, and become a better software engineer.

How Bloom filters differ from hash tables

Before we go into the details, it is worth knowing what makes Bloom filters different from hash tables, since both use hashing under the hood. There are three main things to keep in mind.

First, Bloom filters don't store the actual data. A hash table stores the keys (and maybe their values) in memory. A Bloom filter doesn't do that. The sole question it answers is whether or not an item is in the set.

Second, Bloom filters use considerably less memory. This is the main reason why they exist. A hash table for 100,000 strings needs to store all those strings plus the internal structure of the table. A Bloom filter for the same 100K strings only needs a bit array, regardless of how long the strings are.

Third, Bloom filters can give wrong answers. When a Bloom filter says an item is not in the set, it is always right. But it could be inaccurate when it indicates an item is in the set. This is a false positive. There is a trade-off between the memory a Bloom filter employs and how often these false positives happen. The less memory, the more false positives.

There is also one more thing to be aware of: the basic version of Bloom filters doesn’t support deletion. Once you add an item, you can’t remove it. Some advanced variants have been developed to handle removal, but the classic version does not.

If you can live with these constraints, you get a data structure that is as fast as a hash table for lookups and insertions, but uses only a fraction of the memory.

How Bloom filters work

A Bloom filter is made of two things: a bit array with m elements, all initially set to 0, and a set of k hash functions. Each hash function takes a key and returns an index in the range between 0 and m-1.

The most important thing to remember is that there is no direct connection between the bits in the array and the keys you add to the filter. You specify the value of k when you create the filter. Each key is stored using k bits, one for each hash function. This means that every key you add takes up the same amount of memory, k bits, no matter how lengthy it is. A 5-character word and a 500-character URL both use k bits.

Inserting an element

When you add a key to the filter, you give it as an input to all the k hash functions. Each function gives you an index in the bit array, and you set the bits at those k positions to 1. For example, let’s suppose we have a Bloom filter with 15 bits and 3 hash functions, and we want to add the word “binary.” We compute:

• h0(”binary”) = 2

• h1(”binary”) = 6

• h2(”binary”) = 9

Read more

Welcome to the 172nd issue of the Polymathic Engineer newsletter.

In the last article in our series on recursion, we looked at the selection pattern, which lets us find all possible subsets of a set. The most important question was whether or not to include each item.

In this article, we shift our focus from which items to include to how to arrange them. This is the ordering pattern, and its purpose is to generate all permutations of a set of items.

The difference between combinations and permutations is easy. With combinations, [1, 2] and [2, 1] are the same because the order of the elements does not matter. With permutations, the order matters. A set of three items, such as [1, 2, 3], has 8 possible subsets but 6 possible permutations: [1, 2, 3], [1, 3, 2], [2, 1, 3], [2, 3, 1], [3, 1, 2], [3, 2, 1].

Ordering problems come up a lot in practice. Every time you need to find all possible ways to arrange, schedule, or sequence anything, you need to generate permutations. And, like selection, the ordering pattern employs backtracking to consider all options.

The outline is as follows:

• The Core Ordering Pattern

• The Swap-Based Approach

• Permutations of Specific Length

• Handling Duplicates in the Input

• Application: Binary Search Tree Orderings

• When Ordering Applies

Project-based learning is the best way to develop technical skills. CodeCrafters is an excellent platform for tackling exciting projects, such as building your own Redis, Kafka, a DNS server, SQLite, HTTP server or Git from scratch using your favourite programming language. Noe you can also try to build your own Claude Code (for free, still in beta).

Sign up, and become a better software engineer.

The Core Ordering Pattern

While selection goes through each item and asks whether to include it, ordering takes a different approach. Instead of making include/exclude decisions, it asks: which item comes next?

The underlying idea is to keep a set of available items. At each step, we pick one item from the set, add it to our current permutation, remove it from the set, and recursively find all permutations of the remaining items. When there are no remaining items, we have built one complete permutation.

Let’s start with a simple example, with three items: [1, 2, 3]. In the beginning, all items are available. We could choose 1 first, which leaves us with [2, 3]. From there, we could pick 2, which leaves us with only [3]. Then we can only choose 3, and now we have our first permutation: [1, 2, 3].

At this point, we backtrack. We put 3 back, put 2 back, and try picking 3 instead. This gives us [1, 3, 2]. We continue this process, trying every possible first item, then every possible second item, and so on.

The following code snippet shows how to implement this:

def permutations(items):
    result = []
    available = set(items)
    
    def helper(path):
        if not available:
            result.append(path[:])
            return
        
        for item in list(available):
            available.remove(item)
            path.append(item)
            helper(path)
            path.pop()
            available.add(item)
    
    helper([])
    return result

The structure is quite similar to what we have seen with selection. We have a helper function that builds the permutation step by step. The base case is when there are no items left to choose from, meaning we have used all of them and produced a complete permutation.

In the recursive step, we go through each available item. We take each one out of the set, add it to the path, and call the function again. After the call is over, we go back by taking the item off the path and putting it back in the set.

One thing to note is that we iterate over a copy of the list of available items instead of iterating over them directly. This is because we are modifying the set inside the loop, and most languages do not allow you to change a collection while iterating over it.

The time complexity is O(n! × n). The branching factor starts at n and decreases by one at each level, which gives us n × (n-1) × (n-2) × ... × 1 = n! paths. Each time we complete a permutation, we then need O(n) time to copy it into the result. The space complexity is O(n) for the recursion stack and the path we are building.

The Swap-Based Approach

The set-based approach we looked at is straightforward, but there is a more efficient way that you can also use in interviews. This method does not store a separate list of available items. Instead, it uses the input array itself to track what has been used and what remains available.

The underlying idea is to divide the array into two parts using an index. All the items before the index already belong to the permutation we are generating. The items from the index onward are the pool of items we can still choose from. At each step, we swap an item from the available portion into the current position and then move to the next index. Let’s go back to our simple example, [1, 2, 3], to grasp how the process works.

Initially, the index is equal to 0 because the entire array is available. We swap the first item with itself, which doesn’t change anything, then we move to index 1. We can now choose either 2 or 3. We choose 2 and swap the second item with itself. Then we go to index 2, pick 3, and get our first permutation: [1, 2, 3]. At this point, we backtrack.

We go back to index 1 and swap positions 1 and 2. This places 3 in position 1 and 2 in position 2, yielding [1, 3, 2]. We keep doing this until we have executed every potential swap at every slot.

The following code snippet shows how to implement this:

def permutations(items):
    result = []
    
    def helper(index):
        if index == len(items):
            result.append(items[:])
            return
        
        for j in range(index, len(items)):
            items[index], items[j] = items[j], items[index]
            helper(index + 1)
            items[index], items[j] = items[j], items[index]
    
    helper(0)
    return result

The structure is very similar to the set-based version. The base case is when our index reaches the end of the array. This means that we have made a choice for every position and produced a complete permutation.

In the recursive step, we go through each item from the current index to the end. For each item, we swap it into the current position, call the function again with the next index, and then swap it back to restore the original order.

The best thing about this method is that we don't have to make copies of the available items at every step. We are directly changing the order of the elements in the array. In practice, this makes it a bit more efficient, even though the worst-case analysis stays the same: O(n! × n) for time and O(n) for space.

Permutations of Specific Length

Sometimes we don’t need all the possible permutations of a set, but only those of a specific length. For example, if we have [1, 2, 3, 4], we could want all the two-length permutations: [1, 2], [1, 3], [1, 4], [2, 1], [2, 3], and so on.

Read more

Welcome to the 171st issue of the Polymathic Engineer newsletter.

In one of our previous articles, we looked at the most common root causes of failures in distributed systems and how to use redundancy and fault isolation to contain them. These are protections at the architectural level, which means they work based on how you design and deploy your system.

In this article, we are going to talk about something more tactical. We will discuss the patterns that stop faults from spreading from one service to another at runtime. These are techniques you can apply to existing systems with little effort.

The article is split into two parts. The first covers patterns that protect a service when one of its dependencies fails or slows down (downstream resiliency mechanisms). The second covers patterns that protect a service when its callers send more traffic than it can handle (upstream resiliency mechanisms).

The outline is as follows:

• Timeouts

• Retries

• Circuit breakers

• Load shedding

• Load leveling

• Rate limiting

• Constant work

Project-based learning is the best way to develop technical skills. CodeCrafters is an excellent platform for tackling exciting projects, such as building your own Redis, Kafka, a DNS server, SQLite, HTTP server or Git from scratch using your favourite programming language. Noe you can also try to build your own Claude Code (for free, still in beta).

Sign up, and become a better software engineer.

Timeouts

When one service makes a network call, it should always set a timeout. The call fails if no response comes back within a certain amount of time. Without a timeout, the call might never return, and as we already discussed in the previous article, network calls that don’t return lead to resource leaks. Timeouts are the first protection you can use to find connectivity faults and stop them from cascading from one service to another.

You might think that setting timeouts is such a simple task that all network libraries do it for you, but that's not the case. When JavaScript's fetch API came out, there was no way to specify a timeout. Go's HTTP package doesn't employ timeouts by default, and the default timeout for the most popular Python library is infinity. Modern HTTP clients for Java and .NET do a better job and often come with default timeouts, but it's still a good idea to set them manually every time.

There is a golden rule here: always set timeouts when making network calls, and keep a close eye on third-party libraries that make network calls but don’t let you configure a timeout. But how do you pick a good timeout value?

One method is to base it on how many false timeouts you are okay with. For example, if you don’t mind 0.1% of requests timing out even though they would have worked in the end, you can set the timeout according to the 99.9th percentile of the downstream service’s response time.

It is also important to have good monitoring in place to measure the full lifecycle of a network call. This allows you to check how long a network call took, what status code came back, and if it timed out. Without this kind of visibility at the points where your system is integrated, it's much harder to figure out what's wrong with production.

To avoid repeating the same timeout and monitoring logic everywhere, you can wrap the network call in a library that does both. Another option is a sidecar proxy running on the same machine. This proxy will intercept remote calls and take care of timeouts and monitoring for you.

Retries

When a network call fails or times out, the caller can either give up or try again. If the failure was caused by a short-lived connectivity issue, retrying after a short wait has a good chance of working. However, retrying immediately will make things worse if the downstream service is already too busy,

This is why retries should be slowed down. The most frequent strategy is exponential backoff, in which the delay between retries increases with each attempt. For example, if the initial wait is 2 seconds and you double it each time with an 8-second cap, the delays will be 2, 4, 8, 8, 8... seconds.

This is why retries need to be slowed down. The most common method is exponential backoff, in which the delay between retries increases with each attempt. For example, if the initial wait is 2 seconds and you double it each time with a cap at 8 seconds, the delays would be 2, 4, 8, 8, 8... seconds.

Exponential backoff is helpful, but it has a problem. When a downstream service goes down temporarily, many callers’ requests will fail at the same time. They all retry on a similar schedule, which will cause the downstream service to experience load spikes that make things worse. To fix this, you can add some random jitter to the delay. This spreads out the retries over time and reduces the load on the struggling service.

Actively retrying right away isn’t always the only option, either. In batch systems that don’t need an immediate response, a failed request can be put into a retry queue. The same process, or a different one, can pick it up later and try again.

Additionally, not every failure is worth trying again. If the error isn’t temporary, such as the caller not being allowed to access the destination, retrying will fail again. If that is the case, the service should stop retrying right away. Also, if the network call is not idempotent, retrying can cause problems that affect the correctness of the application.

A last problem that is easy to overlook is retries amplification. Let’s suppose you have a chain of three services: the user calls service A, A calls service B, and B calls another service C. If the call from B to C fails, service B retries. But in the meantime, A sees a longer response time. If A times out, it retries too, and this adds even more load to the chain. If the user’s client also has retries, the total number of requests goes up quickly.

This amplification effect puts more pressure on a service the further down the chain it is. If you have large dependence chains, it is safer to only retry at one level and fail fast at all the others.

Circuit Breakers

Timeouts help you find when a downstream service is slow or unreachable, and retries help you get past short-lived failures. But what if the failure doesn't go away quickly?

Read more

Today’s special issue features a guest post from Tim Varelmann, founder of Bluebird Optimization. Tim is a reader of our newsletter. He contacted me to ask if there was an opportunity to collaborate on the newsletter as a guest author for an issue, and I gladly agreed because I think what he has to say could be interesting to you all.

Tim develops optimization software that helps planners of core business operations make sound, data-driven decisions. He holds a PhD in optimization, with studies at RWTH Aachen, UT Austin, University of Queensland, and MIT, and has combined machine learning and optimization in innovative ways across multiple client projects.

He is also the author of the world’s first course on GAMSPy (“Effortless Modeling in Python with GAMSPy”). If you want a practical framework for combining ML and optimization from Tim, you can find it here. Now, over to Tim.

Your AI shouldn’t grade its own homework

Claude Code writes beautiful code. So does Codex. But here’s the thing, they also think they write beautiful code. And when you ask an AI to review code it just wrote, you get the intellectual equivalent of a student grading their own exam. Shockingly, they always pass.

CodeRabbit CLI plugs into Claude Code and Codex as an external reviewer, different AI Agent, different architecture, 40+ static analyzers, and zero emotional attachment to the code it’s looking at. The agent writes, CodeRabbit reviews, and the agent fixes. Loop until clean.

You show up when there is actually something worth approving.

One command. Autonomous generate-review-iterate cycles. The AI still does the work. It just doesn’t get to decide if the work is good anymore.

Free tier available. Try CodeRabbit’s CLI.

Thanks to the CodeRabbit team for collaborating with me on this newsletter issue.

Machine learning and optimization

Every day, you make a lot of decisions based on information that doesn’t actually tell you what to do. Your navigation app knows there will be heavy traffic at 8 am. That is a prediction. What you do about it is a decision that depends on things the app knows nothing about.

Can you move your first meeting? Is the alternative route through the school zone worth it? Is today the day you finally just work from home? The same forecast, ten different people, ten different choices.

A big part of our job as engineers is finding an appropriate solution to a problem.

Netflix works differently. It predicts how much you will enjoy each show in its catalog (typically using Machine Learning). And from that prediction, the decision about what to recommend to you is almost mechanical: sort by the probability that you will like a show, and fill the three recommendation slots with the top results. Here, knowing the forecast basically means knowing the answer.

Most real-world problems sit somewhere between these two. Predictions matter a lot. But constraints, trade-offs, and competing priorities sit between the forecast and the action. The gap between them is exactly where machine learning and mathematical optimization (or operations research) have to work together.

A forecast tells you what the world looks like. It takes something else to decide what to do in it.

In my experience, the combinations that actually work follow a small number of recurring patterns. Let me show you three of them.

Three Combinations of ML & Optimization

I keep seeing the same structural roles in different fields, even when applications look very different at first glance:

• Making good decisions in the face of uncertain futures

• Replacing math that is difficult to compute with clearly structured ML models

• Optimizing systems where no physical laws exist, only observed behavior

There is a fourth pattern where machine learning is used inside optimization solvers themselves, helping them make internal algorithmic choices. This requires deep solver-internal knowledge, which I do not want to presume here. The three patterns above already provide plenty of substance.

Let’s start with the most important one.

Pattern 1: Making good decisions when the future is uncertain

Inventory management makes this pattern very clear. Some products sell at a steady rate. Others barely move for weeks, then suddenly spike. Imagine two spare parts with the same average annual demand. On paper, they look identical. In reality, you should manage them very differently.

The danger of a single number

Here is the problem: rich information often dies somewhere between the ML forecast and the optimization model. A probabilistic forecast (one that shows you a whole range of possible futures) gets boiled down to one number, the expected value, before the optimizer ever sees it. This is poison for the combination of ML and optimization.

Modern ML models are naturally good at producing rich outputs. Distributions. Confidence intervals. Samples from possible futures. That is exactly the information the optimizer needs. Throwing it away and handing over just the average is like hiring a weather forecaster and only asking them whether it will rain - yes or no.

What optimization can do with the full picture?

With the full distribution, the optimizer can distinguish between the two spare parts, even though their averages match.

For the part with steady demand: order just-in-time, keep safety stock lean.

For the part with spiky demand: an inventory manager might consciously accept a 10% stockout risk during high-demand phases, if that means avoiding expensive external storage. Maybe that trade-off makes financial sense. Maybe a 15% risk of stockout does not, and the cost for external storage would be preferable. The optimizer finds that line. Without the probabilistic input, it cannot even see the question.

Where it works - and where it breaks

This pattern shines when:

• Decisions have asymmetric consequences. A missing safety-critical spare part that shuts down a production line is a very different problem from a surplus of optional accessories gathering dust.

• Different futures require fundamentally different responses. Two parts with the same average demand but different distributions need different inventory strategies.

• Performance must be robust, not just good on average. If a part is both spiky and safety-critical, you might want guaranteed coverage across two consecutive demand spikes - even if that means holding more stock most of the time.

It breaks when uncertainty gets collapsed too early. If the optimizer only sees one number, it will make one-size-fits-all decisions - and quietly leave money on the table.

Pattern 2: Replacing math that is difficult to compute with clearly structured ML models

Sometimes the core difficulty is not uncertainty, but complexity. As a researcher, I worked on optimization problems involving air separation units. These are industrial plants that cool air to extremely low temperatures. Oxygen liquefies slightly before nitrogen, so this liquefaction helps separate oxygen from nitrogen.

The physics involved is well known, but it is also very nonlinear and hard to calculate. Also, these equations are already approximations of the quantum-chemical truth of the world. So, in optimization, the real question wasn’t whether the equations were “true,” but whether they could be used to solve a big optimization problem.

In that case, my coworkers replaced the detailed thermodynamic relationships with a neural network trained on simulation data.

The neural network introduced a small additional approximation error, which was acceptable. In return, the computations became much easier. The optimizer now saw a clean, repeated structure it could work with efficiently.

The win was structure instead of chaos.

Where this pattern works - and where it breaks

This pattern is particularly powerful when:

• Some difficult equations are already approximations, so an additional approximation is not as problematic

• The true slowdown is computational difficulty, not physical accuracy

It breaks when surrogate models are forced to extrapolate beyond their training data, or when critical structural properties of the original equations are lost in the approximation.

Pattern 3: Optimizing systems where no physical laws exist, only observed behavior

Some systems simply have no rules in physics that can describe them well. Think of human attention, trust, or preferences. In such cases, behavior must be learned from data.

In website optimization, for example, ML models can learn how users react to layout changes, rankings, or pricing decisions. This learned behavior is then given to the optimizer, which adjusts the website while respecting multiple other constraints: revenue targets, fairness requirements, budgets, and contracts.

Where this pattern works - and where it breaks

This pattern is particularly powerful when:

• Human behavior must be observed

• Sufficient data is available to learn stable behavioral responses

• Decisions must satisfy non-negotiable constraints (otherwise, an occasional hallucination would be acceptable, but optimization guarantees no hallucinations)

It breaks when learned behavior is treated as static truth, and its drift over time is forgotten.

A Match made in Heaven

Machine learning is a powerful tool for describing reality from data. Optimization is a strong tool for choosing actions under constraints.

They work better together when each is used for what it is best at, and when both uncertainty and assumptions are dealt with honestly rather than ignored. Once you find the pattern that your problem fits into, implementation often becomes relatively straightforward.

If you want to take these ideas further, I have put together a practical framework for you here. Or just reach out, I am always happy to talk optimization.

Welcome to the 170th issue of the Polymathic Engineer newsletter.

Linear regression is a fundamental algorithm in machine learning. Even though it is simple and intuitive, many engineer treat it as a black box. They call a function from scikit-learn, get a result, and move on.

But understanding what happens under the hood is critical because linear regression is the basis for more complicated models like neural networks. In this issue, we break down linear regression from the ground up.

We start with a simple example, build an intuition for how it works, and then explore the math and algorithms that make it all possible. The outline is as follows:

• What is Linear Regression?

• Building an Intuition: Predicting Salaries

• Features, Weights, and Bias

• How to Find the Best Line?

• Measuring How Good a Model Is: Error Functions

• From One Feature to Many: Multivariate Linear Regression

• Polynomial Regression

• Parameters vs Hyperparameters

• Real-World Applications

• Putting Everything Into Practice

Regardless of whether you are an experienced engineer who wants to brush up on the basics or a younger engineer exploring machine learning for the first time, this article will give you a solid understanding of how linear regression works and why it matters.

Project-based learning is the best way to develop technical skills. CodeCrafters is an excellent platform for tackling exciting projects, such as building your own Redis, Kafka, a DNS server, SQLite, HTTP server or Git from scratch using your favourite programming language. Noe you can also try to build your own Claude Code (for free, still in beta).

Sign up, and become a better software engineer.

What is Linear Regression?

Let’s start with the basics. Say you have a set of data points that roughly look like they are forming a line. The goal of linear regression is to draw the line that passes as close as possible to all of them.

Think of the data points as houses in a town, and your goal is to build a road that goes through the town. Everyone wants to live near the road, and your job is to make them as happy as possible. The road should go as close as possible to all the homes.

Of course, this leads to some questions like

• What do we mean by “points that roughly form a line”?

• What do we mean by “a line that passes really close to the points”?

• How do we find such a line?

• Why is this useful in the real world?

We answer all these questions in the following sections.

Building an Intuition: Predicting Salaries

Let’s suppose that you want to guess a software engineer’s salary based on their years of experience. You get some information from a few engineers and get something like this:

If you plot the data on a coordinate system, with years of experience on the x-axis and salary on the y-axis, you see that the dots make a line that goes up. The more you are experienced, the more money you make (we all know it doesn’t work exactly like that, but let’s pretend for the sake of this example).

Now, let’s say that a new engineer starts working for your company. She has 4.5 years of experience and you need to make her an offer. There isn’t a data point for 4.5 years, but you can do it another way: you can find out where 4.5 falls on the line and read the salary. That is linear regression in a nutshell.

What the model actually learns is a simple equation:

Salary = weight × (years of experience) + bias

The weight shows you how much the salary goes up for each year of experience. The bias is the starting point: starting salary with zero experience. We will get into what these two terms mean in detail in the next section.

Features, Weights, and Bias

Let’s have a look at the building blocks of linear regression. There are four things you need to know:

• Features: they are the inputs we use to make a prediction. In our example, years of experience are the feature. But in a more complex model, we could have many features: programming languages known, location, company size, and so on.

• Label: they are what we are trying to guess. In our case, the label is the salary.

• Weights: they tell the model how much each feature matters. Going back to our equation, the weight might be something like 6,500, since for every extra year of experience, the salary increases by that amount. If the weight was negative, it would mean that more experience is associated with a lower salary, which would be strange. A weight close to zero means the feature barely matters. If you added a feature like the number of monitors on the desk and its weight turned out to be near zero, the model is telling you it doesn’t help guess the salary.

• Bias: it is the starting point. It is the salary the model would guess if the engineer had zero years of experience. In practice, the bias is just there to give the line the flexibility to sit at the right height on the graph.

In other words, the model finds the best weights and bias values that make the line fit the data. That’s the whole process of learning. We’ll talk about how it does that in the next part.

How To Find the Best Line?

You could draw an endless number of lines through a set of data points. The question is: which one is the best? The answer is the line that is as close as possible to all data points. But we need to be more precise. Here, “Close to all the points” means the total distance between the line and the data points is as small as possible.

In this context, the distance between what the model predicts and what the data point says is called an error or residual. There are two main ways to find this line.

Read more

Welcome to the 169th issue of the Polymathic Engineer newsletter. This week, we continue our recursion series with the third article.

In the previous two articles, we discussed the basics of recursion and looked at the first two patterns: Iteration and Subproblems. While these patterns are useful, they don’t go very deep. Now we move on to a stronger pattern: Selection.

Selection is arguably the most important recursive pattern you can learn. It frequently comes up in coding interviews, and it is the basis for dynamic programming. A lot of problems that seem hard at first become easy once you realize that they are selection problems in disguise.

The core concept is easy to understand. You look at a set of items and decide whether or not to include each one in your result. By exploring every possible combination of these choices, you generate every possible subset of the original set. From there, you can filter, count, or optimize to get the answer you need.

This article will explore how this pattern works and how to apply it to real problems.

The outline is as follows:

• The core selection pattern

• Counting combinations

• Generating Combinations

• Combinations of a specific length

• The 0/1 Knapsack problem

• When selection applies

Project-based learning is the best way to develop technical skills. CodeCrafters is an excellent platform for tackling exciting projects, such as building your own Redis, Kafka, a DNS server, SQLite, or Git from scratch. Sign up, and become a better software engineer.

The Core Selection Pattern

The selection pattern answers a specific type of question: “Can this problem be solved by finding all possible combinations?” If the answer is yes, then you can use the same approach every time.

The starting point is a set of items. For each item, you have two options: include it in the result or leave it out. You go through the items one by one, and at every step, you branch into two paths. One includes the current item, and the other path excludes it.

As a concrete example, consider the array [1, 2, 3]. You start at the first item. You can either include 1 or exclude it. If you include it, your partial result is [1]. If you exclude it, your partial result is [].

Now you move to the second item. For each of the two partial results, you again have two choices. Include or exclude 2. This gives you four partial results: [1, 2], [1], [2], and []. You continue this process for the third item. Each of the four results branches into two, and you end up with eight total combinations: [1, 2, 3], [1, 2], [1, 3], [1], [2, 3], [2], [3], and []. This is the complete set of subsets.

There are always 2^n subsets for an array of n elements, including the empty set. The selection pattern generates all of them by examining every possible path through the decision tree.

The word “selection” comes from the choice you make at each step. You are selecting which items to include in your result. People also call this the "combinations" pattern, but I prefer "selection" because it captures what you do: you either choose the item, or you don't.

Counting Combinations

Before looking at how to generate all the combinations, it helps to start with a simpler task: counting how many combinations exist. This is a good approach since it lets you focus on the recursive structure without worrying about building up results.

The logic is straightforward. You return 1 if you have reached the end of the array and discovered one valid combination. Otherwise, you make two recursive calls, one which includes the current item and one that excludes it. You add the results together.

def count_combinations(arr, i=0):
    if i == len(arr):
        return 1

    include = count_combinations(arr, i + 1)
    exclude = count_combinations(arr, i + 1)

    return include + exclude

Let’s trace what happens when we run this code with the array [1, 2, 3]. The initial call branches into two: one where we include 1 and one where we don't. Then we move to item 2, and each of those branches splits into two other branches. The same happens for item 3. When we reach the end of the array, we return 1. All those 1s add up as the calls return, and we get 8 at the top.

You might notice that the include and exclude calls are identical. That is because the number of remaining combinations stays the same regardless of whether you include an item. There are always the same number of paths forward. This will not hold when we add constraints.

The time complexity is O(2^n) because we make two recursive calls at each level, and the depth is n. The space complexity is O(n) because of the call stack. These numbers match what we expect: there are 2^n combinations, and we visit each one.

Generating Combinations

Counting is straightforward, but for most problems, you need to actually generate the combinations. This is where things get more interesting. There are two main ways to do this: either build up the results as you return them, or update a passed variable as you go.

Read more

Welcome to the 168th issue of the Polymathic Engineer newsletter.

Let’s say you are building a spell checker that runs directly in the browser. You have a dictionary of over 100,000 words, and every time the user types something, you need to quickly check if the word exists in your dictionary. If it doesn’t, you underline it in red.

The feature sounds easy, but there is a problem: how can you store and search through more than 100,000 words quickly? It would be too slow to send a request to the server for every single word. You need to maintain the dictionary locally and search through it fast, without consuming too much memory.

This is a classic problem in computer science, since it comes up all the time: checking if a username is already taken or if an email is in a contacts list, or verifying whether a URL exists in a cache are all good examples. The main point is always the same: what is the most efficient way to check if something is in a large collection of items?

In this article, we will explore several data structures that solve this problem, starting from the simplest approach and working our way up to more sophisticated solutions. Along the way, we will see how each one makes different trade-offs in terms of speed, memory, and flexibility. This is the outline:

• Abstract data types

• Unsorted arrays

• Sorted arrays and binary search

• Hash tables

• Binary search trees

• Comparing the options

Your AI shouldn't grade its own homework

Claude Code writes beautiful code. So does Codex. But here’s the thing, they also think they write beautiful code. And when you ask an AI to review code it just wrote, you get the intellectual equivalent of a student grading their own exam. Shockingly, they always pass.

CodeRabbit CLI plugs into Claude Code and Codex as an external reviewer, different AI Agent, different architecture, 40+ static analyzers and zero emotional attachment to the code it’s looking at. The agent writes, CodeRabbit reviews, and the agent fixes. Loop until clean.

You show up when there’s actually something worth approving.

One command. Autonomous generate-review-iterate cycles. The AI still does the work. It just doesn’t get to decide if the work is good anymore.

Free tier available. Try CodeRabbit's CLI.

Thanks to the CodeRabbit team for collaborating with me on this newsletter issue.

Abstract data types

Before jumping into specific solutions for the dictionary problem, it is worth taking a step back and talking about abstract data types.

An abstract data type (ADT) defines what operations a data structure should support, without saying anything about how those operations are really done. It is a contract: you know what you can do with it, but the internal details are hidden.

The reason why it’s important to distinguish between abstract data types and concrete data structures is that it forces you to figure out what you need before thinking about how to make it work.

The ADT we care about here is the dictionary, which is also known as a symbol table, associative array, or map. The ADT dictionary keeps a collection of (key, value) pairs and supports three operations:

• insert(key, value): adds a new pair to the collection

• remove(key): removes the pair associated with the given key

• contains(key): checks if a key exists and returns the associated value

The most crucial thing is that there is only one copy of each key in the collection, and you may get any value directly by using its key. The easiest way to think about this is to consider a regular array as a special case.

In an array like [”the”, ”lazy”, ”fox”], the keys are simply the indices 0, 1, and 2, and you can always get a value by providing its position. A dictionary takes this idea further by letting you use keys from almost any domain, such as strings, numbers, or objects.

For the above-mentioned spell checker problem, we actually need a simpler version of the dictionary called a set. A set only knows if a key is there or not; it doesn’t store any values with the keys. You can think of it as a dictionary where values are always either true or false.

Unsorted arrays

The simplest way to solve our spell checker problem is to dump all 100,000 words into an unsorted array. You don't have to do anything special. You simply store the words in the order you get them.

It is easy to add a new word since you can just put it at the end of the array. That takes O(1) time on average. If you don't care about keeping things in order, it is also quick to remove a word. You just swap it with the last element and make the array smaller.

However, things fall apart when it comes to search. If someone types "algorithm" and you need to see whether it is in the dictionary, the only option is go through the array word by word, starting at the beginning and proceeding until you find it or reach the end. In the worst case, that is O(n), which means that for 100,000 words, you may have to look at all the 100,000 entries before you can notify the user that the word is spelled correctly.

Now imagine doing this for every single word the user types. A short paragraph of 200 words would mean up to 20 million comparisons in the worst case. That is not going to work for a feature that needs to feel instant.

The one good thing about the unsorted array is that it is simple. No extra memory, no upfront cost, and insertion is as fast as it gets. But the O(n) lookup makes it useless for anything beyond very small collections. We need a faster way to search.

Sorted arrays and binary search

What if we sort the array first? We could take advantage of the ordering to find things much faster instead of scanning word by word.

You probably already do this without thinking about it. When you look up a word in a paper dictionary, you don’t read every entry in a paper dictionary from the beginning. You open it somewhere in the middle, look at the first word on that page, and choose whether to go ahead or backward. If you are looking for “binary” and you land on the page starting with “method,” you know you can ignore everything after that page and focus on the first half.

This is exactly how binary search works. You start in the middle of the array, compare the target word with the word at that position, and then discard half of the remaining elements based on the result. Then you repeat the process on the surviving half. Each step cuts the search space in two, so it takes at most O(log n) comparisons to find any word. For 100,000 words, that is circa 17 comparisons, which is a huge improvement with respect to the unsorted array.

The problem is that keeping the array sorted comes at a cost. It takes O(nlogn) time to sort it in the first place, which is acceptable if you only do it once. But you have to find the correct place for the new word and move everything that comes after it every time you want to add a new word. That is O(n) for each insertion. Removing a word has the same issue because you need to shift elements to fill the gap.

If the dictionary doesn't change, a sorted array could be a reasonable solution for the spell checker example. But if the dictionary changes a lot, the O(n) cost of adding and removing words will be a problem. We need something that can quickly look up and add things at the same time.

Hash tables

The idea behind hash tables is simple. What if instead of searching through the array, we could compute exactly where a word should be stored?

A hash table uses a hash function to do this. You give it a word, and it gives you back a number. This number is an index in an array. To insert the word “binary,” you run it through the hash function, get back something like 2,512, and store the word at that position. To check if “binary” is in the dictionary, you run the same function, look at position 4,712, and see if it is there. No scanning, no sorting, or comparing.

This makes both insertion and lookup O(1) on average, which is exactly what we were looking for. However, there is a catch. Since the hash function maps a very large set of possible words to a much smaller array, it is only natural that two different words will sometimes end up in the same position. This is what is called a collision. For example, both "binary" and "garden" might hash to index 2,512.

There are two common ways to handle collisions. With chaining, each position in the array has a short list, and you add colliding words to that list. With open addressing, you look for the next available slot in the array when a collision happens. Both ways work well as long as the array is large enough compared to the number of elements it holds.

One distinction to keep in mind is the one between hash maps and sets. You can use a hash map to link a value to a key, like putting a word and its meaning together. A hash set only keeps track of whether a key exists or not. We don't require definitions for the spell checkers; we just need to know if the word is there. All we need is a hash set.

With O(1) on average for all the operations that matter, hash tables seem like the clear winner. But they come with a cost: memory. A hash table needs an array that is larger than the number of elements you plan to store, plus extra space to deal with collisions.

Binary search trees

A binary search tree (BST) takes a different approach. Instead of using a hash function to find where a word goes, it sorts words based on how they compare to each other.

A BST is a tree where each node holds a key, and for every node, all the keys in its left subtree are smaller and all the keys in its right subtree are larger. To add a new word, you start at the root and compare it to the current node. If the word is smaller, you go left. If it is larger, you go right. You keep doing this until you reach an empty spot, and that is where the word goes.

Lookup works similarly. To check if "binary" is in the tree, you start at the root and go down the comparisons. If you get to the word, it is there. If you get to an empty space, it isn't. The number of comparisons is equal to the height of the tree because you get one level deeper each time.

Things start to get interesting here. If the tree is balanced, its height is O(logn), which means that all operations, such as insertion, removal, and lookup, take O(logn) time. It is the same as binary search on sorted arrays, but it doesn't cost O(n) to add or remove items.

The main benefit of BSTs over hash tables is ordering. Since the words are organized by comparison, a BST can give you things like finding all words between “apple” and “banana,” or getting the word that comes right before or after a given one. In a hash table, these operations take O(n), whereas in a BST, they take O(log n).

For the spell checker, we don’t need any of that. But if you are building something like an autocomplete feature, where you need to find all words within a range, a BST is the better choice.

Comparing the options

Now that we have gone through the main data structures, let’s put them side by side and see how they compare.

If all you need is to check whether an element is in a collection and don’t care about the order, hash tables are the clear winner. O(1) on average for lookup, insertion, and removal is hard to beat.

If order is important, BSTs are a better choice. A BST can let you find all the words in a range, get the next or previous element, or get the whole collection in sorted order in O(log n) time. Those operations would take O(n) or more time using a hash table.

Sorted arrays are a solid option when the data doesn’t change. You pay the O(n log n) cost once to sort, and then you get O(log n) lookups with no extra memory. But if you need to add or remove things often, the O(n) cost makes those operations impractical.

Unsorted arrays are only worth considering for very small collections where simplicity and cache speed up outweigh the slow lookup.

For the spell-checker problem, the hash table is the best approach. But something is worth considering: a hash table for 100,000 words still occupies a significant amount of memory.

What if we could trade a small amount of accuracy for a huge reduction in memory? What if we could have a data structure that is as fast as a hash table but uses only a fraction of the space, at the cost of occasionally saying a word is in the dictionary when it is not?

That is exactly what Bloom filters do, and we will explore them in a follow-up article.

Welcome to the 167th issue of the Polymathic Engineer.

A rule of thumb when working on a distributed system is to assume that anything can go wrong. The way engineers usually deal with this is by using scalability patterns like data partitioning, functional decomposition, and replication.

The problem with these patterns is that they all add more moving parts and make our systems more complicated. Since each part has a chance of failing, the more there are, the higher the chance that at least one of them will fail at any given moment. Software crashes, power outages, hardware faults, and memory leaks can happen all the time.

This isn't only a worry in theory. As we discussed in another article, a system can only be down for approximately 15 minutes a day to guarantee it is available 99.99% of the time. For three nines, that drops to 43 minutes per month. In general, the more nines you want, the faster your system needs to detect failures and recover from them when they occur. There is no time for a person to notice anything went wrong and fix it.

In this article, we will discuss the most common root causes of failures in distributed systems and see how to address them. The outline is as follows:

• Common root causes of failures

• Managing risk

• Redundancy

• Correlation

• Fault isolation

• Shuffle Sharding and Cellular Architecture

Project-based learning is the best way to develop technical skills. CodeCrafters is an excellent platform for tackling exciting projects, such as building your own Redis, Kafka, a DNS server, SQLite, or Git from scratch. Sign up, and become a better software engineer.

Common root causes of failures

To build fault-tolerant distributed systems, you first need to fully understand what can go wrong. Let's start looking at the most common failure modes you can encounter.

Infrastructure Faults

Hardware doesn’t last forever. Hard drives fail, memory modules go bad, and network cards stop working. Power outages happen, and fiber cuts or electrical storms can take down whole data centers. The good news is that redundancy makes it straightforward to remedy these infrastructural problems. The underlying idea is simple: if one of your servers goes down, the others may take over.

Incorrect Error Handling

A study of user-reported failures from well-known distributed data storage showed an unexpected result. Most major failures were not caused by infrastructure issues. They were caused by mistakes made when dealing with non-fatal faults. The bugs were very easy to fix. In some cases, error handlers didn’t even look at errors; in others, handlers were only partially set up, or caught too generic exceptions. The main point was that most bugs could have been found with simple unit tests.

Configuration Changes

Changes to configurations cause failures more often than most engineers think, and it is not always a simple mistake like a typo in a database connection string that makes them risky. One example is a valid configuration change turning on a feature flag that hasn’t been used in years. Even if there is a code path, and the value looks correct, the behavior is no longer what was meant.

Configuration changes are challenging because they may not take effect right away. A change that is not valid might not be seen for hours or days if an application only gets a configuration value when a certain code path runs. This is why configuration should be managed like code: version-controlled, reviewed, and released incrementally.

Single points of failure

A single point of failure is any component that, brings the entire system down when it fails. The tricky part is that they are not always obvious.

People are a surprisingly common one. If someone has to manually execute a sequence of operational steps in the right order without making a mistake, it is only a matter of time before something goes wrong. Computers, on the other hand, are much better at executing instructions. This is why you should automate whenever possible.

But systems have non-human SPOFs as well. DNS is a good example: if clients can’t resolve your domain, it doesn’t matter how healthy your servers are. TLS certificates are another one: if yours expires, every client connection will be refused.

When designing a system, a useful exercise is to go through each component and ask what happens if it disappears. Some SPOFs can be removed with redundancy. Others can’t, but you can at least work to reduce how much damage they cause when they do fail.

Network faults

When two processes talk to each other over the network, a lot can go wrong. Messages might get lost, arrive late, or be sent more than once. The sender rarely knows which of these things happened because they only see a timeout.

But timeouts are the easy case. A more difficult situation is when the network doesn’t completely fail; it just gets slow. A process sends a request and gets a response, but it takes 10 times longer than usual. There is no error to catch and no timeout to trigger. The caller just waits, hanging on to resources while they do. These partial failures are sometimes called gray failures and are considerably harder to find than a clean crash. A slow dependency can silently bring the entire system to a halt before anybody knows something is wrong.

Resource leaks

Every process has a limited number of resources it can use, such as memory, threads, file handles, and database connections. When a bug causes one of these resources not to be released properly, you get a leak. The hard part is that leaks don’t show up right away. A slow memory leak might take days to fill up the heap. A connection pool that loses one connection per hour won’t cause problems until there are none left.

This is what makes resource leaks so dangerous: they pass all the tests in staging but don't show up in production until the app has been running long enough. And when they do ultimately fix the problem, it often doesn't seem like the original defect at all. A leaky connection doesn't make a sound, but it just shows up as a strange timeout in another part of the system.

Load pressure

A system can perform perfectly fine under normal conditions, but it can break down when the load gets too high. The hard aspect is that the extra load doesn't necessarily come from the users. Sometimes the spike comes from places you don’t expect: a web scraper hitting your API thousands of times per second, or a denial-of-service attack flooding your servers with traffic.

What makes load-related failures difficult is that they can turn a healthy system into an unhealthy one very quickly. When the thread pool is full, a service that responds in 50 milliseconds under normal load can take 5 seconds. And that slowdown does not stay in one place, but extends to all the other services that are waiting for a response.

Cascading failures

Cascading failures happen when a fault in a component causes faults in some others, which then trigger more faults, and so on. A common example is when a server gets too many requests, starts to answer slowly, and callers start to time out. Those callers retry, adding even more load to the already struggling server. The situation gets worse and worse until the whole system grinds to a halt.

Metastable failures are the most dangerous kind of cascade failures. This happens when the system gets stuck in a bad state that doesn't go away even after the original cause has gone away. For example, a short spike in traffic creates a lot of retries. The retries themselves create enough load to produce more retries, and the system persists in this self-sustaining failure mode long after the original spike is over. When this happens, it is not possible to just wait for the problem to pass. The system needs an intervention to break the cycle.

Managing Risk

By now, the list of things that can go wrong may seem a bit too long. But just because a mistake could happen doesn't imply we have to fix it immediately.

Read more

Hi Friends,

Welcome to the 166th issue of the Polymathic Engineer newsletter.

When you think about becoming a better software engineer, your mind probably goes straight to technical skills. Learning another framework or mastering a programming language, or practicing system design patterns. And yes, these things matter because they are what get you throug…

Read more

Welcome to the 165th issue of the Polymathic Engineer. This week, we keep going our recursion series with the second article.

In the previous article, we covered the foundations: base cases, recursive steps, return strategies, tracing code, and complexity analysis. Now it’s time to put those basics to work.

This article covers the first two recursive patterns: Iteration and Subproblems. These are the most approachable patterns and a natural starting point before tackling all the others.

The outline is as follows:

• The Iteration pattern

• Iterating forward and backward over arrays

• Inserting at the bottom of a stack

• Nested iteration with recursion

• The Subproblems pattern

• Towers of Hanoi

• Checking if a string is a palindrome

• The stair step problem

Project-based learning is the best way to develop technical skills. CodeCrafters is an excellent platform for tackling exciting projects, such as building your own Redis, Kafka, a DNS server, SQLite, or Git from scratch. Sign up, and become a better software engineer.

The Iteration Pattern

The iteration pattern is the simplest of the six. The basic idea is to replace a for loop with a recursive function. Each recursive call handles a step of the iteration, and the base case is when there are no more elements to process.

At first, this may seem like a waste of time. Why change a for loop that already works to recursion? There are a few times when it makes sense.

The first is when you need to access data in reverse order. Consider printing a linked list backward. With iteration, you would have to push all the values onto a stack and then pop them off. With recursion, the call stack does this for you automatically. The code becomes shorter and easier to follow.

The second is when the recursion takes care of memory for you. Some problems need temporary storage that matches the structure of the input. You don’t have to explicitly create and manage that storage yourself; instead, you can let the recursive calls hold the data in their stack frames. We will see a concrete example of this when we insert an element at the bottom of a stack.

The third is when you use functional programming. Languages like Haskell or certain styles of JavaScript avoid mutable state entirely. Recursion takes the place of loops in these contexts because there is no loop that doesn’t change a counter variable.

There is one thing to keep in mind: the iteration pattern doesn’t usually improve time complexity. Most of the time, the recursive version does the same amount of work as the iterative one. The advantage is that the code is cleaner, not that it runs faster. And since recursion uses stack frames, it can actually use more memory than a simple loop.

When you use this pattern, there are two questions to ask yourself. The first is: what is the iterative step? In a regular for loop, this might be i + 1, or i + 2, or something more complex. Your recursive call will mirror this. The second is: which way do you want to go? Going forward or backward? The answer affects where you put the recursive call relative to the work you do in each step. We'll see both directions in the next section.

Iterating Forward and Backward

Let’s start with a simple example: print every element in an array. Here is the iterative version (I know the code is not pythonic but it is better for the sake of the examples):

def print_array(arr):

    for i in range(len(arr)):

        print(arr[i])

To make this code recursive, we need to think about what happens at each step of the iteration. In this case, we move from index i to index i + 1. The base case occurs when the index reaches the array's length, which means that there are no more elements to print. The recursive version does the same. It prints the current element, then moves to the next. When it runs out of elements, it stops.

def print_array(arr, i=0):
    if i == len(arr):
        return

    print(arr[i])
    print_array(arr, i + 1)

Now here is something interesting. If you swap the order of the last two lines, you get the array printed in reverse:

def print_array_reverse(arr, i=0):
    if i == len(arr):
        return

    print_array_reverse(arr, i + 1)
    print(arr[i])

The only change is that the print statement comes after the recursive call. This means we first go to the end of the array, then print the elements as the calls come back. The last element in the array is printed first, and so on.

This is a pattern you will see often: the order of the recursive call relative to the work determines the direction of processing. If you do the work first and then recurse, you process forward. If you recurse first and then do the work, you process backward.

You can use the same idea when you want to return a result rather than print it. As we discussed in the first article, you can either use a passed variable or build up the result as you return. Here is an example that sums all the elements in an array with the build-up approach:

def sum_array(arr, i=0):
    if i == len(arr):
        return 0

    return arr[i] + sum_array(arr, i + 1)

The function finds the sum from index i to the end. The base case says that the sum of an empty slice is 0. The recursive step states that the sum is the current element plus the sum of everything after it. As the calls return, the values add up.

If you prefer using a passed variable, the code looks like this:

def sum_array(arr, i=0, result=None):
    if result is None:
        result = [0]

    if i == len(arr):
        return result[0]

    result[0] += arr[i]
    return sum_array(arr, i + 1, result)

Both approaches lead to the same answer. The build-up version is usually cleaner for simple cases, while the passed-variable can be better as the logic gets more complex.

Inserting at the Bottom of a Stack

Now, let’s look at a problem where recursion does more than just take the place of a for loop. You have to put something at the bottom of a stack without using any extra data structures.

If you try to do this iteratively, you run into a problem. You can only access the top of a stack, so you have to pop everything off, insert your item, and push everything back on. This requires a temporary stack to hold the elements (In Python, we use a list as a stack, with append() to push and pop() to remove from the top):

def insert_at_bottom_iterative(stack, item):
    temp = []
    while stack:
        temp.append(stack.pop())

    stack.append(item)
    while temp:
        stack.append(temp.pop())

The code works, but you need to create and maintain the temporary stack. When you use recursion, the call stack takes care of this implicitly. The recursive approach pops elements off one at a time, makes a recursive call, and then pushes each element back on after the call comes back. The base case is when the stack is empty, at which point we insert the item. As the recursive calls return, each previously popped element gets pushed back on top:

def insert_at_bottom(stack, item):
    if not stack:
        stack.append(item)
        return

    top = stack.pop()
    insert_at_bottom(stack, item)
    stack.append(top)

Let’s trace through this with a concrete example. Suppose the stack is [1, 2, 3, 4] (with 4 on top) and we want to insert 5 at the bottom.

The first call pops 4 and saves it. The stack is now [1, 2, 3]. The second call pops 3. The stack is [1, 2]. The third call pops 2. The stack is [1]. The fourth call pops 1, leaving the stack empty.

Now we hit the base case. We append 5, so the stack is [5]. Then the calls start coming back. The fourth call appends 1, giving us [5, 1]. The third call appends 2, giving us [5, 1, 2]. The second call appends 3, giving us [5, 1, 2, 3]. The first call appends 4, and we end up with [5, 1, 2, 3, 4].

The time complexity is O(n) because we need to pop and push each element once. The space complexity is also O(n) since the call stack must hold all popped elements. This is the same as the iterative version, but the recursive code is shorter and easier to read.

Nested Iteration with Recursion

Read more

Welcome to the 164th issue of the Polymathic Engineer newsletter.

Recursion is one of those topics that many software engineers “understand” in theory but struggle with in practice. They know the textbook definition (a function that calls itself). They have seen the Fibonacci example multiple times. But when they are asked to solve a recursive problem in an interview, something doesn’t work.

The problem is not that they are not smart, but that they don’t have the correct mental models. It took me a while to figure this out. When someone explained it to me, I was able to follow recursive code, but writing it from scratch often felt like guessing. The turning point was when I stopped thinking of recursion as a single, isolated technique and began seeing it as a set of patterns, each with its own structure.

In this article, we will start building the basic mental models you need to actually use recursion. In future articles, we will apply these foundations to specific patterns and problems.

The outline is as follows:

• When to use recursion

• The two building blocks: base case and recursive step

• Three ways to return results from recursive functions

• How to trace any recursive code (even when you’re lost)

• Time and space complexity for recursion

• A preview of the six recursive patterns

Project-based learning is the best way to develop technical skills. CodeCrafters is an excellent platform for tackling exciting projects, such as building your own Redis, Kafka, a DNS server, SQLite, or Git from scratch. Sign up, and become a better software engineer.

When to Use Recursion

There is a fact about recursion that people often overlook: any problem that can be solved recursively can also be solved iteratively. There is no case where you need to use recursion.

So the right question isn’t “can I use recursion here?” but rather “is recursion easier here?” There are a few questions you can ask yourself to figure this out.

1. Does the problem break down into subproblems? This is mostly what recursion is about. If a problem breaks into smaller versions of itself, recursion is probably a good fit. For example, sorting an array can be decomposed into sorting two halves and merging them. Finding a path through a maze can be broken into taking one step and finding a path from there.

2. Could you solve the problem using an arbitrary number of for loops? This point gets people confused. Let’s say you are asked to print all possible 6-digit numbers. It sounds easy: you can just write six nested for loops, one for each digit. But what if you are told to print all possible n-digit numbers? You can't write n nested for loops anymore since the number of loops changes depending on the input. This is an excellent example of when to use recursion, because it effectively allows you to create arbitrary nesting.

3. Does the problem fit one of the common recursive patterns? For most recursive problems you see in interviews, there are six patterns that you can look for. We’ll preview them at the end of this article and cover them in depth in future ones. If you recognize that a problem fits one of these patterns, that is a strong hint to use recursion.

4. Is it easier to solve recursively than iteratively? Sometimes the recursive code is just cleaner. Consider the task of printing a linked list in reverse order. Without recursion, you would have to push all the values onto a stack, then pop and print them. Recursively, the process reduces to three lines of code:

def print_reverse(node):

  if node is None:
    return

  print_reverse(node.next)
  print(node.value)

The recursive version uses the call stack as your stack, so you don’t need to explicitly manage one yourself. The code is more concise and probably easier to grasp.

However, there is a trade-off. If the time complexity is the same, the iterative solution usually runs faster. Function calls, stack frames, and other things that recursion does all add up. In general, if you have the possibility to choose between two equally simple recursive and iterative solutions, the iterative one will perform better.

But in an interview, clarity often matters more than micro-optimizations, so it is wise to pick whatever approach lets you write correct, readable code faster.

Base Case and Recursive Step

Every recursive function has two essential building blocks: a base case and a recursive step. Getting these right is what separates working code from infinite loops and stack overflows.

The base case is the point at which the recursion stops. It is the simplest form of the problem, and it can be solved immediately without any further recursive calls.

A useful technique for identifying the base case is to start with small examples. What is the answer when the input size is zero or one? These edge cases are usually the base cases.

For example, if you are writing a function to compute the sum of an array of integers, an empty array has a sum of 0, and an array with one element has a sum equal to that element. Both can be solved without recursion, making them natural base cases.

There is one important rule to follow here: the base case must match the return type of your function. If your function returns a list, your base case must return a list (not None or 0). This sounds obvious, but it is a common source of bugs. People often get stuck on this because they try to guess what the base case is instead of using concrete examples to figure it out.

# Wrong: function returns a list, but base case returns None
def get_combinations(items):
    if len(items) == 0:
        return None  # This will break the calling code
    ...

# Correct: base case returns an empty list
def get_combinations(items):
    if len(items) == 0:
        return [[]]  # A list containing the empty combination
    ...

In the recursive step, you divide the problem into smaller subproblems and combine the results. There are two things to keep in mind here.

The first is that a subproblem should have the same shape as the original problem. If your function computes the sum of elements of an array from index i to the end, then your recursive call should compute the sum of elements from index i+1 to the end. The meaning is the same, just applied to a smaller input.

The second critical thing is that the recursive step must converge to the base case. If your base case is i == 0 and your recursive call passes i + 1, you are moving away from the base case, not toward it. This causes an infinite recursion and eventually a stack overflow.

# Wrong: moves away from base case
def countdown(n):
    if n == 0:
        return
    countdown(n + 1)  # n grows, never reaches 0

# Correct: moves toward base case
def countdown(n):
    if n == 0:
        return
    countdown(n - 1)  # n shrinks toward 0

When you are writing code in an interview, you don’t get to run your code and see the stack overflow error. You need to find this mistake by looking at the code you wrote. A quick check that your recursive step gets you closer to the base case can save you from an embarrassing bug.

Three Ways to Return Results

When you write a recursive function, you need to decide how to collect and return the results. There are three main ways you can do this: use a global variable, use a passed variable, or build up results as you return. Each has its trade-offs.

The simplest approach is to use a global variable. You set a variable outside the code and change it as the recursion proceeds. As an example, consider the following code snippet that counts all the even numbers in an array:

def count_evens(arr, i):
    global result
    if i == len(arr):
        return
    if arr[i] % 2 == 0:
        result += 1
    count_evens(arr, i + 1)

This works, but global variables are generally not a good idea. They make code harder to test and reason about. A better alternative is to use a passed variable.

Read more

Welcome to the 163rd issue of the Polymathic Engineer newsletter.

Concurrent programming involves breaking down applications into separate units of work. These units, known as tasks, set up the execution flow of an application. At their core, tasks are just abstractions. At some point, they must eventually be mapped to the physical devices that execute the code.

Thank goodness, we don’t have to worry about this. The operating system, which is another layer of abstraction, handles this for us. Its role is to make the best use of the available hardware as possible, but it is not a magical box. Still, programmers need to organize their work in a way that helps the operating system use hardware in the best way possible.

In this article, we will discuss processes and threads, which are the two main building blocks of concurrency, and understand how they can talk to each other (Demo code in this GitHub repo).

The outline is as follows:

• Processes

• Threads

• Processes vs Threads

• Interprocess Communication

• Message-Passing Mechanisms

• The Thread Pool Pattern

Project-based learning is the best way to develop technical skills. CodeCrafters is an excellent platform for tackling exciting projects, such as building your own Redis, Kafka, a DNS server, SQLite, or Git from scratch. Sign up, and become a better software engineer.

Processes

The informal definition of a process is simple: it is a running program. A program by itself is just a file sitting on a disk. It contains a set of instructions, but it does nothing on its own. When the operating system takes those instructions and starts executing them on hardware, it becomes a process.

Think of it like a recipe in a cookbook. The recipe itself is just words on paper listing ingredients, steps, and timings. It doesn’t make anything by itself. But if a chef reads it and starts cooking, it becomes a meal in progress. Source code works the same way.

It is just a passive sequence of instructions. Developers write code using abstractions like memory, files, and network connections, but the actual resources must be given at runtime. The OS wraps all of this into what we call a process.

A process is made of several parts that the OS keeps track of:

• Address space: the memory the process can see and access

• Executable: the file with the machine instructions

• Process ID (PID): a unique name to identify the process

• Process state: whether it is running, waiting, or finished

• Open files and connections: any resources the process is using

All of these together make up the execution context. Since so many things are packed into a process, starting a new one is a pretty heavy thing to do, and processes are often called heavyweight.

If you look at a process as a whole, its lifecycle is, its lifecycle is straightforward. First, it doesn’t exist. After that, the OS creates it and places it in memory (the Created state). From there, it moves to the Ready state: it can run at any moment, but the CPU hasn’t picked it up yet. When the OS scheduler selects it, the process goes into the Running state. Once it completes or fails, it reaches the Terminated state.

Creating and terminating a process is relatively expensive since so many resources are attached to it and must be assigned or released.

Processes can create their own processes, called child processes, using system calls like fork() or spawn(). This is called spawning. Each child process gets its own independent memory address space, which means it is completely isolated from the parent process and any other children.

This is where things get interesting for concurrency. By spawning multiple processes, we can split execution across them and run them simultaneously on parallel hardware.

However, the isolation is both a good and a bad thing. Since two processes don’t share much by default, communication between them requires special mechanisms, which are usually several orders of magnitude slower than direct memory access. We’ll talk about that more later in this article.

Threads

Most operating systems let you share memory between processes, but this takes extra effort. However, there is a different abstraction that lets you share memory and other kinds of resources in a simpler way: threads.

At the end of the day, a program is simply a list of machine instructions that must be carried out in the right order. The OS uses the concept of a thread to do this. A thread is an independent stream of instructions whose execution can be scheduled by the OS.

In the previous section, we said that a process is a running program plus resources. If we break that apart, a process is a container of resources (address space, files, network connections, and so on), while a thread is the dynamic part: the set of instructions that run inside that container. From the OS’s point of view, a process can be seen as a unit of resources, while a thread can be seen as a unit of execution.

The idea behind threads is that sharing a common address space is the most efficient way for processes to communicate. Threads in the same process can share resources (address space, files, connections, and data) with each other and their parent process, in an easy way.

Each thread also keeps its own state to allow for safe, local, independent execution of its instructions. Each thread is unaware of the other threads unless it is trying to mess with them on purpose. The OS manages threads and distributes them across available processor cores. This makes a multithreaded program an effective way to run multiple tasks concurrently.

Historically, hardware vendors implemented their own versions of threads, and these implementations were quite different. This made it challenging for programmers to write portable multithreaded applications. For UNIX systems, IEEE POSIX defined a standard programming interface. Implementations that follow this standard are called POSIX threads or Pthreads.

Every program we run causes the OS to create a process, and every process has at least one thread. A process without a thread cannot exist. When we start a program, a main execution thread is created. Any thread, even the main one, can create child threads at any time.

Threads have much less memory overhead than the standard fork() function. Since threads use the same process, nothing is copied. This is one reason why threads are sometimes called “lightweight processes.” It takes less time for the OS to assign and handle thread resources, plus starting and ending threads is faster than starting and ending processes.

However, there is a catch. The OS provides complete independence of processes from each other, so if one of them crashes, other processes are not harmed. This is not true for threads. All threads in a process use the same shared resources, so if one crashes or messes up something, it’s likely that the others will too. To avoid this, programmers need to make sure that shared resources are only used by one thread at a time and give them more control over how threads behave.

Processes vs Threads

To illustrate the difference between processes and threads, let’s go back to our recipe example. Imagine you run a restaurant and need to prepare three different dishes.

Read more

Welcome to the 162nd issue of the Polymathic Engineer newsletter.

As everybody knows, computer vision is everywhere. From the filters on your phone camera to self-driving cars, from medical imaging to security systems, the ability for machines to “see” and understand visual information has become integral to modern software.

If you want to get…

Read more