Welcome to the 171st issue of the Polymathic Engineer newsletter.

In one of our previous articles, we looked at the most common root causes of failures in distributed systems and how to use redundancy and fault isolation to contain them. These are protections at the architectural level, which means they work based on how you design and deploy your system.

In this article, we are going to talk about something more tactical. We will discuss the patterns that stop faults from spreading from one service to another at runtime. These are techniques you can apply to existing systems with little effort.

The article is split into two parts. The first covers patterns that protect a service when one of its dependencies fails or slows down (downstream resiliency mechanisms). The second covers patterns that protect a service when its callers send more traffic than it can handle (upstream resiliency mechanisms).

The outline is as follows:

• Timeouts

• Retries

• Circuit breakers

• Load shedding

• Load leveling

• Rate limiting

• Constant work

Project-based learning is the best way to develop technical skills. CodeCrafters is an excellent platform for tackling exciting projects, such as building your own Redis, Kafka, a DNS server, SQLite, HTTP server or Git from scratch using your favourite programming language. Noe you can also try to build your own Claude Code (for free, still in beta).

Sign up, and become a better software engineer.

Timeouts

When one service makes a network call, it should always set a timeout. The call fails if no response comes back within a certain amount of time. Without a timeout, the call might never return, and as we already discussed in the previous article, network calls that don’t return lead to resource leaks. Timeouts are the first protection you can use to find connectivity faults and stop them from cascading from one service to another.

You might think that setting timeouts is such a simple task that all network libraries do it for you, but that's not the case. When JavaScript's fetch API came out, there was no way to specify a timeout. Go's HTTP package doesn't employ timeouts by default, and the default timeout for the most popular Python library is infinity. Modern HTTP clients for Java and .NET do a better job and often come with default timeouts, but it's still a good idea to set them manually every time.

There is a golden rule here: always set timeouts when making network calls, and keep a close eye on third-party libraries that make network calls but don’t let you configure a timeout. But how do you pick a good timeout value?

One method is to base it on how many false timeouts you are okay with. For example, if you don’t mind 0.1% of requests timing out even though they would have worked in the end, you can set the timeout according to the 99.9th percentile of the downstream service’s response time.

It is also important to have good monitoring in place to measure the full lifecycle of a network call. This allows you to check how long a network call took, what status code came back, and if it timed out. Without this kind of visibility at the points where your system is integrated, it's much harder to figure out what's wrong with production.

To avoid repeating the same timeout and monitoring logic everywhere, you can wrap the network call in a library that does both. Another option is a sidecar proxy running on the same machine. This proxy will intercept remote calls and take care of timeouts and monitoring for you.

Retries

When a network call fails or times out, the caller can either give up or try again. If the failure was caused by a short-lived connectivity issue, retrying after a short wait has a good chance of working. However, retrying immediately will make things worse if the downstream service is already too busy,

This is why retries should be slowed down. The most frequent strategy is exponential backoff, in which the delay between retries increases with each attempt. For example, if the initial wait is 2 seconds and you double it each time with an 8-second cap, the delays will be 2, 4, 8, 8, 8... seconds.

This is why retries need to be slowed down. The most common method is exponential backoff, in which the delay between retries increases with each attempt. For example, if the initial wait is 2 seconds and you double it each time with a cap at 8 seconds, the delays would be 2, 4, 8, 8, 8... seconds.

Exponential backoff is helpful, but it has a problem. When a downstream service goes down temporarily, many callers’ requests will fail at the same time. They all retry on a similar schedule, which will cause the downstream service to experience load spikes that make things worse. To fix this, you can add some random jitter to the delay. This spreads out the retries over time and reduces the load on the struggling service.

Actively retrying right away isn’t always the only option, either. In batch systems that don’t need an immediate response, a failed request can be put into a retry queue. The same process, or a different one, can pick it up later and try again.

Additionally, not every failure is worth trying again. If the error isn’t temporary, such as the caller not being allowed to access the destination, retrying will fail again. If that is the case, the service should stop retrying right away. Also, if the network call is not idempotent, retrying can cause problems that affect the correctness of the application.

A last problem that is easy to overlook is retries amplification. Let’s suppose you have a chain of three services: the user calls service A, A calls service B, and B calls another service C. If the call from B to C fails, service B retries. But in the meantime, A sees a longer response time. If A times out, it retries too, and this adds even more load to the chain. If the user’s client also has retries, the total number of requests goes up quickly.

This amplification effect puts more pressure on a service the further down the chain it is. If you have large dependence chains, it is safer to only retry at one level and fail fast at all the others.

Circuit Breakers

Timeouts help you find when a downstream service is slow or unreachable, and retries help you get past short-lived failures. But what if the failure doesn't go away quickly?

Read more

Today’s special issue features a guest post from Tim Varelmann, founder of Bluebird Optimization. Tim is a reader of our newsletter. He contacted me to ask if there was an opportunity to collaborate on the newsletter as a guest author for an issue, and I gladly agreed because I think what he has to say could be interesting to you all.

Tim develops optimization software that helps planners of core business operations make sound, data-driven decisions. He holds a PhD in optimization, with studies at RWTH Aachen, UT Austin, University of Queensland, and MIT, and has combined machine learning and optimization in innovative ways across multiple client projects.

He is also the author of the world’s first course on GAMSPy (“Effortless Modeling in Python with GAMSPy”). If you want a practical framework for combining ML and optimization from Tim, you can find it here. Now, over to Tim.

Your AI shouldn’t grade its own homework

Claude Code writes beautiful code. So does Codex. But here’s the thing, they also think they write beautiful code. And when you ask an AI to review code it just wrote, you get the intellectual equivalent of a student grading their own exam. Shockingly, they always pass.

CodeRabbit CLI plugs into Claude Code and Codex as an external reviewer, different AI Agent, different architecture, 40+ static analyzers, and zero emotional attachment to the code it’s looking at. The agent writes, CodeRabbit reviews, and the agent fixes. Loop until clean.

You show up when there is actually something worth approving.

One command. Autonomous generate-review-iterate cycles. The AI still does the work. It just doesn’t get to decide if the work is good anymore.

Free tier available. Try CodeRabbit’s CLI.

Thanks to the CodeRabbit team for collaborating with me on this newsletter issue.

Machine learning and optimization

Every day, you make a lot of decisions based on information that doesn’t actually tell you what to do. Your navigation app knows there will be heavy traffic at 8 am. That is a prediction. What you do about it is a decision that depends on things the app knows nothing about.

Can you move your first meeting? Is the alternative route through the school zone worth it? Is today the day you finally just work from home? The same forecast, ten different people, ten different choices.

A big part of our job as engineers is finding an appropriate solution to a problem.

Netflix works differently. It predicts how much you will enjoy each show in its catalog (typically using Machine Learning). And from that prediction, the decision about what to recommend to you is almost mechanical: sort by the probability that you will like a show, and fill the three recommendation slots with the top results. Here, knowing the forecast basically means knowing the answer.

Most real-world problems sit somewhere between these two. Predictions matter a lot. But constraints, trade-offs, and competing priorities sit between the forecast and the action. The gap between them is exactly where machine learning and mathematical optimization (or operations research) have to work together.

A forecast tells you what the world looks like. It takes something else to decide what to do in it.

In my experience, the combinations that actually work follow a small number of recurring patterns. Let me show you three of them.

Three Combinations of ML & Optimization

I keep seeing the same structural roles in different fields, even when applications look very different at first glance:

• Making good decisions in the face of uncertain futures

• Replacing math that is difficult to compute with clearly structured ML models

• Optimizing systems where no physical laws exist, only observed behavior

There is a fourth pattern where machine learning is used inside optimization solvers themselves, helping them make internal algorithmic choices. This requires deep solver-internal knowledge, which I do not want to presume here. The three patterns above already provide plenty of substance.

Let’s start with the most important one.

Pattern 1: Making good decisions when the future is uncertain

Inventory management makes this pattern very clear. Some products sell at a steady rate. Others barely move for weeks, then suddenly spike. Imagine two spare parts with the same average annual demand. On paper, they look identical. In reality, you should manage them very differently.

The danger of a single number

Here is the problem: rich information often dies somewhere between the ML forecast and the optimization model. A probabilistic forecast (one that shows you a whole range of possible futures) gets boiled down to one number, the expected value, before the optimizer ever sees it. This is poison for the combination of ML and optimization.

Modern ML models are naturally good at producing rich outputs. Distributions. Confidence intervals. Samples from possible futures. That is exactly the information the optimizer needs. Throwing it away and handing over just the average is like hiring a weather forecaster and only asking them whether it will rain - yes or no.

What optimization can do with the full picture?

With the full distribution, the optimizer can distinguish between the two spare parts, even though their averages match.

For the part with steady demand: order just-in-time, keep safety stock lean.

For the part with spiky demand: an inventory manager might consciously accept a 10% stockout risk during high-demand phases, if that means avoiding expensive external storage. Maybe that trade-off makes financial sense. Maybe a 15% risk of stockout does not, and the cost for external storage would be preferable. The optimizer finds that line. Without the probabilistic input, it cannot even see the question.

Where it works - and where it breaks

This pattern shines when:

• Decisions have asymmetric consequences. A missing safety-critical spare part that shuts down a production line is a very different problem from a surplus of optional accessories gathering dust.

• Different futures require fundamentally different responses. Two parts with the same average demand but different distributions need different inventory strategies.

• Performance must be robust, not just good on average. If a part is both spiky and safety-critical, you might want guaranteed coverage across two consecutive demand spikes - even if that means holding more stock most of the time.

It breaks when uncertainty gets collapsed too early. If the optimizer only sees one number, it will make one-size-fits-all decisions - and quietly leave money on the table.

Pattern 2: Replacing math that is difficult to compute with clearly structured ML models

Sometimes the core difficulty is not uncertainty, but complexity. As a researcher, I worked on optimization problems involving air separation units. These are industrial plants that cool air to extremely low temperatures. Oxygen liquefies slightly before nitrogen, so this liquefaction helps separate oxygen from nitrogen.

The physics involved is well known, but it is also very nonlinear and hard to calculate. Also, these equations are already approximations of the quantum-chemical truth of the world. So, in optimization, the real question wasn’t whether the equations were “true,” but whether they could be used to solve a big optimization problem.

In that case, my coworkers replaced the detailed thermodynamic relationships with a neural network trained on simulation data.

The neural network introduced a small additional approximation error, which was acceptable. In return, the computations became much easier. The optimizer now saw a clean, repeated structure it could work with efficiently.

The win was structure instead of chaos.

Where this pattern works - and where it breaks

This pattern is particularly powerful when:

• Some difficult equations are already approximations, so an additional approximation is not as problematic

• The true slowdown is computational difficulty, not physical accuracy

It breaks when surrogate models are forced to extrapolate beyond their training data, or when critical structural properties of the original equations are lost in the approximation.

Pattern 3: Optimizing systems where no physical laws exist, only observed behavior

Some systems simply have no rules in physics that can describe them well. Think of human attention, trust, or preferences. In such cases, behavior must be learned from data.

In website optimization, for example, ML models can learn how users react to layout changes, rankings, or pricing decisions. This learned behavior is then given to the optimizer, which adjusts the website while respecting multiple other constraints: revenue targets, fairness requirements, budgets, and contracts.

Where this pattern works - and where it breaks

This pattern is particularly powerful when:

• Human behavior must be observed

• Sufficient data is available to learn stable behavioral responses

• Decisions must satisfy non-negotiable constraints (otherwise, an occasional hallucination would be acceptable, but optimization guarantees no hallucinations)

It breaks when learned behavior is treated as static truth, and its drift over time is forgotten.

A Match made in Heaven

Machine learning is a powerful tool for describing reality from data. Optimization is a strong tool for choosing actions under constraints.

They work better together when each is used for what it is best at, and when both uncertainty and assumptions are dealt with honestly rather than ignored. Once you find the pattern that your problem fits into, implementation often becomes relatively straightforward.

If you want to take these ideas further, I have put together a practical framework for you here. Or just reach out, I am always happy to talk optimization.

Welcome to the 170th issue of the Polymathic Engineer newsletter.

Linear regression is a fundamental algorithm in machine learning. Even though it is simple and intuitive, many engineer treat it as a black box. They call a function from scikit-learn, get a result, and move on.

But understanding what happens under the hood is critical because linear regression is the basis for more complicated models like neural networks. In this issue, we break down linear regression from the ground up.

We start with a simple example, build an intuition for how it works, and then explore the math and algorithms that make it all possible. The outline is as follows:

• What is Linear Regression?

• Building an Intuition: Predicting Salaries

• Features, Weights, and Bias

• How to Find the Best Line?

• Measuring How Good a Model Is: Error Functions

• From One Feature to Many: Multivariate Linear Regression

• Polynomial Regression

• Parameters vs Hyperparameters

• Real-World Applications

• Putting Everything Into Practice

Regardless of whether you are an experienced engineer who wants to brush up on the basics or a younger engineer exploring machine learning for the first time, this article will give you a solid understanding of how linear regression works and why it matters.

Project-based learning is the best way to develop technical skills. CodeCrafters is an excellent platform for tackling exciting projects, such as building your own Redis, Kafka, a DNS server, SQLite, HTTP server or Git from scratch using your favourite programming language. Noe you can also try to build your own Claude Code (for free, still in beta).

Sign up, and become a better software engineer.

What is Linear Regression?

Let’s start with the basics. Say you have a set of data points that roughly look like they are forming a line. The goal of linear regression is to draw the line that passes as close as possible to all of them.

Think of the data points as houses in a town, and your goal is to build a road that goes through the town. Everyone wants to live near the road, and your job is to make them as happy as possible. The road should go as close as possible to all the homes.

Of course, this leads to some questions like

• What do we mean by “points that roughly form a line”?

• What do we mean by “a line that passes really close to the points”?

• How do we find such a line?

• Why is this useful in the real world?

We answer all these questions in the following sections.

Building an Intuition: Predicting Salaries

Let’s suppose that you want to guess a software engineer’s salary based on their years of experience. You get some information from a few engineers and get something like this:

If you plot the data on a coordinate system, with years of experience on the x-axis and salary on the y-axis, you see that the dots make a line that goes up. The more you are experienced, the more money you make (we all know it doesn’t work exactly like that, but let’s pretend for the sake of this example).

Now, let’s say that a new engineer starts working for your company. She has 4.5 years of experience and you need to make her an offer. There isn’t a data point for 4.5 years, but you can do it another way: you can find out where 4.5 falls on the line and read the salary. That is linear regression in a nutshell.

What the model actually learns is a simple equation:

Salary = weight × (years of experience) + bias

The weight shows you how much the salary goes up for each year of experience. The bias is the starting point: starting salary with zero experience. We will get into what these two terms mean in detail in the next section.

Features, Weights, and Bias

Let’s have a look at the building blocks of linear regression. There are four things you need to know:

• Features: they are the inputs we use to make a prediction. In our example, years of experience are the feature. But in a more complex model, we could have many features: programming languages known, location, company size, and so on.

• Label: they are what we are trying to guess. In our case, the label is the salary.

• Weights: they tell the model how much each feature matters. Going back to our equation, the weight might be something like 6,500, since for every extra year of experience, the salary increases by that amount. If the weight was negative, it would mean that more experience is associated with a lower salary, which would be strange. A weight close to zero means the feature barely matters. If you added a feature like the number of monitors on the desk and its weight turned out to be near zero, the model is telling you it doesn’t help guess the salary.

• Bias: it is the starting point. It is the salary the model would guess if the engineer had zero years of experience. In practice, the bias is just there to give the line the flexibility to sit at the right height on the graph.

In other words, the model finds the best weights and bias values that make the line fit the data. That’s the whole process of learning. We’ll talk about how it does that in the next part.

How To Find the Best Line?

You could draw an endless number of lines through a set of data points. The question is: which one is the best? The answer is the line that is as close as possible to all data points. But we need to be more precise. Here, “Close to all the points” means the total distance between the line and the data points is as small as possible.

In this context, the distance between what the model predicts and what the data point says is called an error or residual. There are two main ways to find this line.

Read more

Welcome to the 169th issue of the Polymathic Engineer newsletter. This week, we continue our recursion series with the third article.

In the previous two articles, we discussed the basics of recursion and looked at the first two patterns: Iteration and Subproblems. While these patterns are useful, they don’t go very deep. Now we move on to a stronger pattern: Selection.

Selection is arguably the most important recursive pattern you can learn. It frequently comes up in coding interviews, and it is the basis for dynamic programming. A lot of problems that seem hard at first become easy once you realize that they are selection problems in disguise.

The core concept is easy to understand. You look at a set of items and decide whether or not to include each one in your result. By exploring every possible combination of these choices, you generate every possible subset of the original set. From there, you can filter, count, or optimize to get the answer you need.

This article will explore how this pattern works and how to apply it to real problems.

The outline is as follows:

• The core selection pattern

• Counting combinations

• Generating Combinations

• Combinations of a specific length

• The 0/1 Knapsack problem

• When selection applies

Project-based learning is the best way to develop technical skills. CodeCrafters is an excellent platform for tackling exciting projects, such as building your own Redis, Kafka, a DNS server, SQLite, or Git from scratch. Sign up, and become a better software engineer.

The Core Selection Pattern

The selection pattern answers a specific type of question: “Can this problem be solved by finding all possible combinations?” If the answer is yes, then you can use the same approach every time.

The starting point is a set of items. For each item, you have two options: include it in the result or leave it out. You go through the items one by one, and at every step, you branch into two paths. One includes the current item, and the other path excludes it.

As a concrete example, consider the array [1, 2, 3]. You start at the first item. You can either include 1 or exclude it. If you include it, your partial result is [1]. If you exclude it, your partial result is [].

Now you move to the second item. For each of the two partial results, you again have two choices. Include or exclude 2. This gives you four partial results: [1, 2], [1], [2], and []. You continue this process for the third item. Each of the four results branches into two, and you end up with eight total combinations: [1, 2, 3], [1, 2], [1, 3], [1], [2, 3], [2], [3], and []. This is the complete set of subsets.

There are always 2^n subsets for an array of n elements, including the empty set. The selection pattern generates all of them by examining every possible path through the decision tree.

The word “selection” comes from the choice you make at each step. You are selecting which items to include in your result. People also call this the "combinations" pattern, but I prefer "selection" because it captures what you do: you either choose the item, or you don't.

Counting Combinations

Before looking at how to generate all the combinations, it helps to start with a simpler task: counting how many combinations exist. This is a good approach since it lets you focus on the recursive structure without worrying about building up results.

The logic is straightforward. You return 1 if you have reached the end of the array and discovered one valid combination. Otherwise, you make two recursive calls, one which includes the current item and one that excludes it. You add the results together.

def count_combinations(arr, i=0):
    if i == len(arr):
        return 1

    include = count_combinations(arr, i + 1)
    exclude = count_combinations(arr, i + 1)

    return include + exclude

Let’s trace what happens when we run this code with the array [1, 2, 3]. The initial call branches into two: one where we include 1 and one where we don't. Then we move to item 2, and each of those branches splits into two other branches. The same happens for item 3. When we reach the end of the array, we return 1. All those 1s add up as the calls return, and we get 8 at the top.

You might notice that the include and exclude calls are identical. That is because the number of remaining combinations stays the same regardless of whether you include an item. There are always the same number of paths forward. This will not hold when we add constraints.

The time complexity is O(2^n) because we make two recursive calls at each level, and the depth is n. The space complexity is O(n) because of the call stack. These numbers match what we expect: there are 2^n combinations, and we visit each one.

Generating Combinations

Counting is straightforward, but for most problems, you need to actually generate the combinations. This is where things get more interesting. There are two main ways to do this: either build up the results as you return them, or update a passed variable as you go.

Read more

Welcome to the 168th issue of the Polymathic Engineer newsletter.

Let’s say you are building a spell checker that runs directly in the browser. You have a dictionary of over 100,000 words, and every time the user types something, you need to quickly check if the word exists in your dictionary. If it doesn’t, you underline it in red.

The feature sounds easy, but there is a problem: how can you store and search through more than 100,000 words quickly? It would be too slow to send a request to the server for every single word. You need to maintain the dictionary locally and search through it fast, without consuming too much memory.

This is a classic problem in computer science, since it comes up all the time: checking if a username is already taken or if an email is in a contacts list, or verifying whether a URL exists in a cache are all good examples. The main point is always the same: what is the most efficient way to check if something is in a large collection of items?

In this article, we will explore several data structures that solve this problem, starting from the simplest approach and working our way up to more sophisticated solutions. Along the way, we will see how each one makes different trade-offs in terms of speed, memory, and flexibility. This is the outline:

• Abstract data types

• Unsorted arrays

• Sorted arrays and binary search

• Hash tables

• Binary search trees

• Comparing the options

Your AI shouldn't grade its own homework

Claude Code writes beautiful code. So does Codex. But here’s the thing, they also think they write beautiful code. And when you ask an AI to review code it just wrote, you get the intellectual equivalent of a student grading their own exam. Shockingly, they always pass.

CodeRabbit CLI plugs into Claude Code and Codex as an external reviewer, different AI Agent, different architecture, 40+ static analyzers and zero emotional attachment to the code it’s looking at. The agent writes, CodeRabbit reviews, and the agent fixes. Loop until clean.

You show up when there’s actually something worth approving.

One command. Autonomous generate-review-iterate cycles. The AI still does the work. It just doesn’t get to decide if the work is good anymore.

Free tier available. Try CodeRabbit's CLI.

Thanks to the CodeRabbit team for collaborating with me on this newsletter issue.

Abstract data types

Before jumping into specific solutions for the dictionary problem, it is worth taking a step back and talking about abstract data types.

An abstract data type (ADT) defines what operations a data structure should support, without saying anything about how those operations are really done. It is a contract: you know what you can do with it, but the internal details are hidden.

The reason why it’s important to distinguish between abstract data types and concrete data structures is that it forces you to figure out what you need before thinking about how to make it work.

The ADT we care about here is the dictionary, which is also known as a symbol table, associative array, or map. The ADT dictionary keeps a collection of (key, value) pairs and supports three operations:

• insert(key, value): adds a new pair to the collection

• remove(key): removes the pair associated with the given key

• contains(key): checks if a key exists and returns the associated value

The most crucial thing is that there is only one copy of each key in the collection, and you may get any value directly by using its key. The easiest way to think about this is to consider a regular array as a special case.

In an array like [”the”, ”lazy”, ”fox”], the keys are simply the indices 0, 1, and 2, and you can always get a value by providing its position. A dictionary takes this idea further by letting you use keys from almost any domain, such as strings, numbers, or objects.

For the above-mentioned spell checker problem, we actually need a simpler version of the dictionary called a set. A set only knows if a key is there or not; it doesn’t store any values with the keys. You can think of it as a dictionary where values are always either true or false.

Unsorted arrays

The simplest way to solve our spell checker problem is to dump all 100,000 words into an unsorted array. You don't have to do anything special. You simply store the words in the order you get them.

It is easy to add a new word since you can just put it at the end of the array. That takes O(1) time on average. If you don't care about keeping things in order, it is also quick to remove a word. You just swap it with the last element and make the array smaller.

However, things fall apart when it comes to search. If someone types "algorithm" and you need to see whether it is in the dictionary, the only option is go through the array word by word, starting at the beginning and proceeding until you find it or reach the end. In the worst case, that is O(n), which means that for 100,000 words, you may have to look at all the 100,000 entries before you can notify the user that the word is spelled correctly.

Now imagine doing this for every single word the user types. A short paragraph of 200 words would mean up to 20 million comparisons in the worst case. That is not going to work for a feature that needs to feel instant.

The one good thing about the unsorted array is that it is simple. No extra memory, no upfront cost, and insertion is as fast as it gets. But the O(n) lookup makes it useless for anything beyond very small collections. We need a faster way to search.

Sorted arrays and binary search

What if we sort the array first? We could take advantage of the ordering to find things much faster instead of scanning word by word.

You probably already do this without thinking about it. When you look up a word in a paper dictionary, you don’t read every entry in a paper dictionary from the beginning. You open it somewhere in the middle, look at the first word on that page, and choose whether to go ahead or backward. If you are looking for “binary” and you land on the page starting with “method,” you know you can ignore everything after that page and focus on the first half.

This is exactly how binary search works. You start in the middle of the array, compare the target word with the word at that position, and then discard half of the remaining elements based on the result. Then you repeat the process on the surviving half. Each step cuts the search space in two, so it takes at most O(log n) comparisons to find any word. For 100,000 words, that is circa 17 comparisons, which is a huge improvement with respect to the unsorted array.

The problem is that keeping the array sorted comes at a cost. It takes O(nlogn) time to sort it in the first place, which is acceptable if you only do it once. But you have to find the correct place for the new word and move everything that comes after it every time you want to add a new word. That is O(n) for each insertion. Removing a word has the same issue because you need to shift elements to fill the gap.

If the dictionary doesn't change, a sorted array could be a reasonable solution for the spell checker example. But if the dictionary changes a lot, the O(n) cost of adding and removing words will be a problem. We need something that can quickly look up and add things at the same time.

Hash tables

The idea behind hash tables is simple. What if instead of searching through the array, we could compute exactly where a word should be stored?

A hash table uses a hash function to do this. You give it a word, and it gives you back a number. This number is an index in an array. To insert the word “binary,” you run it through the hash function, get back something like 2,512, and store the word at that position. To check if “binary” is in the dictionary, you run the same function, look at position 4,712, and see if it is there. No scanning, no sorting, or comparing.

This makes both insertion and lookup O(1) on average, which is exactly what we were looking for. However, there is a catch. Since the hash function maps a very large set of possible words to a much smaller array, it is only natural that two different words will sometimes end up in the same position. This is what is called a collision. For example, both "binary" and "garden" might hash to index 2,512.

There are two common ways to handle collisions. With chaining, each position in the array has a short list, and you add colliding words to that list. With open addressing, you look for the next available slot in the array when a collision happens. Both ways work well as long as the array is large enough compared to the number of elements it holds.

One distinction to keep in mind is the one between hash maps and sets. You can use a hash map to link a value to a key, like putting a word and its meaning together. A hash set only keeps track of whether a key exists or not. We don't require definitions for the spell checkers; we just need to know if the word is there. All we need is a hash set.

With O(1) on average for all the operations that matter, hash tables seem like the clear winner. But they come with a cost: memory. A hash table needs an array that is larger than the number of elements you plan to store, plus extra space to deal with collisions.

Binary search trees

A binary search tree (BST) takes a different approach. Instead of using a hash function to find where a word goes, it sorts words based on how they compare to each other.

A BST is a tree where each node holds a key, and for every node, all the keys in its left subtree are smaller and all the keys in its right subtree are larger. To add a new word, you start at the root and compare it to the current node. If the word is smaller, you go left. If it is larger, you go right. You keep doing this until you reach an empty spot, and that is where the word goes.

Lookup works similarly. To check if "binary" is in the tree, you start at the root and go down the comparisons. If you get to the word, it is there. If you get to an empty space, it isn't. The number of comparisons is equal to the height of the tree because you get one level deeper each time.

Things start to get interesting here. If the tree is balanced, its height is O(logn), which means that all operations, such as insertion, removal, and lookup, take O(logn) time. It is the same as binary search on sorted arrays, but it doesn't cost O(n) to add or remove items.

The main benefit of BSTs over hash tables is ordering. Since the words are organized by comparison, a BST can give you things like finding all words between “apple” and “banana,” or getting the word that comes right before or after a given one. In a hash table, these operations take O(n), whereas in a BST, they take O(log n).

For the spell checker, we don’t need any of that. But if you are building something like an autocomplete feature, where you need to find all words within a range, a BST is the better choice.

Comparing the options

Now that we have gone through the main data structures, let’s put them side by side and see how they compare.

If all you need is to check whether an element is in a collection and don’t care about the order, hash tables are the clear winner. O(1) on average for lookup, insertion, and removal is hard to beat.

If order is important, BSTs are a better choice. A BST can let you find all the words in a range, get the next or previous element, or get the whole collection in sorted order in O(log n) time. Those operations would take O(n) or more time using a hash table.

Sorted arrays are a solid option when the data doesn’t change. You pay the O(n log n) cost once to sort, and then you get O(log n) lookups with no extra memory. But if you need to add or remove things often, the O(n) cost makes those operations impractical.

Unsorted arrays are only worth considering for very small collections where simplicity and cache speed up outweigh the slow lookup.

For the spell-checker problem, the hash table is the best approach. But something is worth considering: a hash table for 100,000 words still occupies a significant amount of memory.

What if we could trade a small amount of accuracy for a huge reduction in memory? What if we could have a data structure that is as fast as a hash table but uses only a fraction of the space, at the cost of occasionally saying a word is in the dictionary when it is not?

That is exactly what Bloom filters do, and we will explore them in a follow-up article.

Welcome to the 167th issue of the Polymathic Engineer.

A rule of thumb when working on a distributed system is to assume that anything can go wrong. The way engineers usually deal with this is by using scalability patterns like data partitioning, functional decomposition, and replication.

The problem with these patterns is that they all add more moving parts and make our systems more complicated. Since each part has a chance of failing, the more there are, the higher the chance that at least one of them will fail at any given moment. Software crashes, power outages, hardware faults, and memory leaks can happen all the time.

This isn't only a worry in theory. As we discussed in another article, a system can only be down for approximately 15 minutes a day to guarantee it is available 99.99% of the time. For three nines, that drops to 43 minutes per month. In general, the more nines you want, the faster your system needs to detect failures and recover from them when they occur. There is no time for a person to notice anything went wrong and fix it.

In this article, we will discuss the most common root causes of failures in distributed systems and see how to address them. The outline is as follows:

• Common root causes of failures

• Managing risk

• Redundancy

• Correlation

• Fault isolation

• Shuffle Sharding and Cellular Architecture

Project-based learning is the best way to develop technical skills. CodeCrafters is an excellent platform for tackling exciting projects, such as building your own Redis, Kafka, a DNS server, SQLite, or Git from scratch. Sign up, and become a better software engineer.

Common root causes of failures

To build fault-tolerant distributed systems, you first need to fully understand what can go wrong. Let's start looking at the most common failure modes you can encounter.

Infrastructure Faults

Hardware doesn’t last forever. Hard drives fail, memory modules go bad, and network cards stop working. Power outages happen, and fiber cuts or electrical storms can take down whole data centers. The good news is that redundancy makes it straightforward to remedy these infrastructural problems. The underlying idea is simple: if one of your servers goes down, the others may take over.

Incorrect Error Handling

A study of user-reported failures from well-known distributed data storage showed an unexpected result. Most major failures were not caused by infrastructure issues. They were caused by mistakes made when dealing with non-fatal faults. The bugs were very easy to fix. In some cases, error handlers didn’t even look at errors; in others, handlers were only partially set up, or caught too generic exceptions. The main point was that most bugs could have been found with simple unit tests.

Configuration Changes

Changes to configurations cause failures more often than most engineers think, and it is not always a simple mistake like a typo in a database connection string that makes them risky. One example is a valid configuration change turning on a feature flag that hasn’t been used in years. Even if there is a code path, and the value looks correct, the behavior is no longer what was meant.

Configuration changes are challenging because they may not take effect right away. A change that is not valid might not be seen for hours or days if an application only gets a configuration value when a certain code path runs. This is why configuration should be managed like code: version-controlled, reviewed, and released incrementally.

Single points of failure

A single point of failure is any component that, brings the entire system down when it fails. The tricky part is that they are not always obvious.

People are a surprisingly common one. If someone has to manually execute a sequence of operational steps in the right order without making a mistake, it is only a matter of time before something goes wrong. Computers, on the other hand, are much better at executing instructions. This is why you should automate whenever possible.

But systems have non-human SPOFs as well. DNS is a good example: if clients can’t resolve your domain, it doesn’t matter how healthy your servers are. TLS certificates are another one: if yours expires, every client connection will be refused.

When designing a system, a useful exercise is to go through each component and ask what happens if it disappears. Some SPOFs can be removed with redundancy. Others can’t, but you can at least work to reduce how much damage they cause when they do fail.

Network faults

When two processes talk to each other over the network, a lot can go wrong. Messages might get lost, arrive late, or be sent more than once. The sender rarely knows which of these things happened because they only see a timeout.

But timeouts are the easy case. A more difficult situation is when the network doesn’t completely fail; it just gets slow. A process sends a request and gets a response, but it takes 10 times longer than usual. There is no error to catch and no timeout to trigger. The caller just waits, hanging on to resources while they do. These partial failures are sometimes called gray failures and are considerably harder to find than a clean crash. A slow dependency can silently bring the entire system to a halt before anybody knows something is wrong.

Resource leaks

Every process has a limited number of resources it can use, such as memory, threads, file handles, and database connections. When a bug causes one of these resources not to be released properly, you get a leak. The hard part is that leaks don’t show up right away. A slow memory leak might take days to fill up the heap. A connection pool that loses one connection per hour won’t cause problems until there are none left.

This is what makes resource leaks so dangerous: they pass all the tests in staging but don't show up in production until the app has been running long enough. And when they do ultimately fix the problem, it often doesn't seem like the original defect at all. A leaky connection doesn't make a sound, but it just shows up as a strange timeout in another part of the system.

Load pressure

A system can perform perfectly fine under normal conditions, but it can break down when the load gets too high. The hard aspect is that the extra load doesn't necessarily come from the users. Sometimes the spike comes from places you don’t expect: a web scraper hitting your API thousands of times per second, or a denial-of-service attack flooding your servers with traffic.

What makes load-related failures difficult is that they can turn a healthy system into an unhealthy one very quickly. When the thread pool is full, a service that responds in 50 milliseconds under normal load can take 5 seconds. And that slowdown does not stay in one place, but extends to all the other services that are waiting for a response.

Cascading failures

Cascading failures happen when a fault in a component causes faults in some others, which then trigger more faults, and so on. A common example is when a server gets too many requests, starts to answer slowly, and callers start to time out. Those callers retry, adding even more load to the already struggling server. The situation gets worse and worse until the whole system grinds to a halt.

Metastable failures are the most dangerous kind of cascade failures. This happens when the system gets stuck in a bad state that doesn't go away even after the original cause has gone away. For example, a short spike in traffic creates a lot of retries. The retries themselves create enough load to produce more retries, and the system persists in this self-sustaining failure mode long after the original spike is over. When this happens, it is not possible to just wait for the problem to pass. The system needs an intervention to break the cycle.

Managing Risk

By now, the list of things that can go wrong may seem a bit too long. But just because a mistake could happen doesn't imply we have to fix it immediately.

Read more

Hi Friends,

Welcome to the 166th issue of the Polymathic Engineer newsletter.

When you think about becoming a better software engineer, your mind probably goes straight to technical skills. Learning another framework or mastering a programming language, or practicing system design patterns. And yes, these things matter because they are what get you through the interview and into the job.

But here is the thing: technical skills are only half the equation. After almost 20 years in this industry, I have noticed that the engineers who truly excel share a set of skills that have little to do with code. Those are passive skills that work in the background, helping you through your daily routine without you even noticing. However, there are no online courses or bootcamps that teach them. You have to develop them yourself.

Any technical skill can be learned over time by reading documentation or watching a video. But the skills I am about to talk about are the cornerstones on which you grow as a developer. They are what help you through the learning process itself. If you skip them, it is like learning to run before you know how to walk.

The outline is as follows:

• Patience

• Determination

• An eternal student mindset

• Accept criticism and learn from it

• Know how to communicate

Stop debating architecture in pull requests.

If your team’s pull requests are filled with comments like:

• This architecture won’t scale

• We should approach this another way

You are suffering from high costs and wasting developers’ time.

Developers often realize what should have been built only after the PR discussion begins, leading to endless back-and-forth and complete rewrites.

Introducing the CodeRabbit Plan that makes planning a first-class workflow before you write a single line of code. CodeRabbit Plan lets teams design how a feature should be built before writing code, so PR reviews serve as validation rather than architecture debates.

• Old Way: build > PR > debate

• New Way with Plan: plan > align > build

Thanks to the CodeRabbit team for collaborating with me on this newsletter issue.

Get Started Today

Patience

Nothing says you are a software engineer like spending three hours debugging a piece of code, only to find the problem was a tiny detail forgotten somewhere in the middle.

If you think this is a sign of being a junior developer or of lacking experience, you are getting it wrong. I go through that same process every now and then, after almost 20 years in this profession. It never fully goes away.

The reality is that almost everything in software development requires time and effort. Understanding someone else’s code or researching how to solve a problem? Time and effort. Same for writing code and getting it actually to work.

Sure, you can take shortcuts. Copying and pasting code from an AI assistant feels like the quick path to a solution. But it only gets you halfway. The rest needs to come from you, and there is a lot of trial and error involved. You try something; it fails. Then you try something else, and it fails again. This is simply the process.

The engineers who burn out fastest are often the ones who expect everything to work on the first try. When it doesn’t, frustration builds up. Debugging for three hours isn’t so bad if you know that being patient is part of the job. It’s just another day at work.

Determination

Determination goes hand in hand with patience. Everyone who has done our job long enough knows it isn't easy, but setting the right expectations is the best way to avoid being discouraged when things go wrong.

The truth is, you will encounter roadblocks in your career. Problems that, once solved, turn into ten new ones. Bugs that take a long time to fix, like weeks or months. In any of these cases, quitting is a possibility.

There have been many occasions in my career when I wanted to quit programming. A lot of developers in our field would like to move to the middle of nowhere, away from technology, and become farmers. However, this is not a sign that something is wrong with software development. It only proves that our job can be frustrating at times.

That's why determination is so important. It is hard to know how determined you are until you face a struggle. But if you are already a determined person who doesn't give up after the first try, you will be a good developer.

The good news is that you can build determination over time. Every bug you fix, every roadblock you overcome, adds a little more to your reserves. You start to trust that you will figure it out eventually, because you have done it before.

A Student Mindset

Growth and learning are inseparable. You can't have one without the other. However, many developers forget this after they get a job or become more senior.

Our field never stops changing. Sometimes it moves forward with concepts that are really new. Other times, it goes back to previous concepts and presents them as the next big thing. Either way, if you don't pay attention, you will fall behind.

This doesn’t mean you need to chase every new framework or learn every trending language. That would be tiring and useless. What that means is that you should be open to the idea that what you know today won't always be relevant. The tools and practices you use now will change, and you will have to adjust to them.

I have seen developers who refuse to accept this. They learn one stack and think it will help them for the rest of their careers. It sometimes works for a few years. But eventually, the industry changes, and they are unable to move forward because they stopped learning too soon.

Technology drives technology. New advancements lead to new areas of research, new ways to work with data, and new ways of building products. When that happens, the tools we use have to adapt. If you adapt with them, you stay relevant. Otherwise, you get left behind. The key is not to learn everything. The key is to never close the door on learning.

Accept Criticism and Use It to Learn

Programming is not a solo activity. Even if you work as a freelancer, you will interact with other developers in one way or another. Giving and receiving feedback is part of that.

A good example is code reviews. It can be uncomfortable when other developers read your code to look for problems, missing standards, or flaws. However, if both sides do it well, code review can be one of the best methods to learn and grow.

The purpose of reviewing is to improve the code, and not to put down the person who wrote it. On the other side, if you are getting feedback, you need to keep in mind that it is not personal. The reviewers are sharing their experiences. Take their opinions to heart, and make sure you know why they were offered. You will learn a lot from this.

But feedback doesn't just show up during code reviews. You get feedback all the time, such as during performance reviews, in comments on open-source projects, or from a coworker pointing out an issue with your work. You should expect some feedback, but some will surprise you.

It is also crucial to make a distinction. Some feedback highlights a flaw in your work, like a bug you introduced. Other feedback is just negative noise: people venting their anger without giving any meaningful information. Learn how to tell them apart. Even a negative critique generally offers little insight. Don't worry about anything else; just focus on that.

If you only see feedback as something bad, you will start to doubt every decision you make. And that goes against the whole goal, which was to help you get better.

Know How to Communicate

This one is tough. Many developers, even experienced ones, struggle to communicate effectively with others. We spend so much of our time learning how to write logic for computers that we forget how to talk to people. Computers don't care about how well you speak or what words you use. They require clear, easy-to-understand instructions. But you also have to work with people, and people are all different.

When you need to ask a colleague for help, explain a problem you are facing, or help someone else with their issue, you have to switch from your machine syntax to your human syntax. This shift is not easy, but getting good at it gives you a big edge. If an interviewer sees that you can communicate well, you have already won half the battle.

How do you get better at this? One way is through writing. When I first started, I was bad at both writing and speaking. I spent hours reviewing emails that I believed were important, adding clarifications, and asking my peers to check that they made sense.

Things didn't change until I started making an effort to write online. Writing posts for my blog made me think of clearer ways to explain things. It didn't happen right away, but eventually something clicked. My oral communication also improved. I learnt how to talk to people better.

Communication is a skill you can practice right now, for free. Start a blog, write about what you are learning, or just explain a concept to a colleague. The more you do it, the better you get.

Conclusion

Getting the job is not the end of the line. It’s just the beginning. There are developers who think that once they get a job, they have made it. But it’s like thinking you’ve won a video game when you reach the highest level. In reality, you have simply unlocked a whole new set of challenges. Technical knowledge can get you through the door. But to get to the top, you need to be patient, determined, open to learning, good at talking to people, and able to take criticism. These are the building blocks for everything else you will learn along the way.

Welcome to the 165th issue of the Polymathic Engineer. This week, we keep going our recursion series with the second article.

In the previous article, we covered the foundations: base cases, recursive steps, return strategies, tracing code, and complexity analysis. Now it’s time to put those basics to work.

This article covers the first two recursive patterns: Iteration and Subproblems. These are the most approachable patterns and a natural starting point before tackling all the others.

The outline is as follows:

• The Iteration pattern

• Iterating forward and backward over arrays

• Inserting at the bottom of a stack

• Nested iteration with recursion

• The Subproblems pattern

• Towers of Hanoi

• Checking if a string is a palindrome

• The stair step problem

Project-based learning is the best way to develop technical skills. CodeCrafters is an excellent platform for tackling exciting projects, such as building your own Redis, Kafka, a DNS server, SQLite, or Git from scratch. Sign up, and become a better software engineer.

The Iteration Pattern

The iteration pattern is the simplest of the six. The basic idea is to replace a for loop with a recursive function. Each recursive call handles a step of the iteration, and the base case is when there are no more elements to process.

At first, this may seem like a waste of time. Why change a for loop that already works to recursion? There are a few times when it makes sense.

The first is when you need to access data in reverse order. Consider printing a linked list backward. With iteration, you would have to push all the values onto a stack and then pop them off. With recursion, the call stack does this for you automatically. The code becomes shorter and easier to follow.

The second is when the recursion takes care of memory for you. Some problems need temporary storage that matches the structure of the input. You don’t have to explicitly create and manage that storage yourself; instead, you can let the recursive calls hold the data in their stack frames. We will see a concrete example of this when we insert an element at the bottom of a stack.

The third is when you use functional programming. Languages like Haskell or certain styles of JavaScript avoid mutable state entirely. Recursion takes the place of loops in these contexts because there is no loop that doesn’t change a counter variable.

There is one thing to keep in mind: the iteration pattern doesn’t usually improve time complexity. Most of the time, the recursive version does the same amount of work as the iterative one. The advantage is that the code is cleaner, not that it runs faster. And since recursion uses stack frames, it can actually use more memory than a simple loop.

When you use this pattern, there are two questions to ask yourself. The first is: what is the iterative step? In a regular for loop, this might be i + 1, or i + 2, or something more complex. Your recursive call will mirror this. The second is: which way do you want to go? Going forward or backward? The answer affects where you put the recursive call relative to the work you do in each step. We'll see both directions in the next section.

Iterating Forward and Backward

Let’s start with a simple example: print every element in an array. Here is the iterative version (I know the code is not pythonic but it is better for the sake of the examples):

def print_array(arr):

    for i in range(len(arr)):

        print(arr[i])

To make this code recursive, we need to think about what happens at each step of the iteration. In this case, we move from index i to index i + 1. The base case occurs when the index reaches the array's length, which means that there are no more elements to print. The recursive version does the same. It prints the current element, then moves to the next. When it runs out of elements, it stops.

def print_array(arr, i=0):
    if i == len(arr):
        return

    print(arr[i])
    print_array(arr, i + 1)

Now here is something interesting. If you swap the order of the last two lines, you get the array printed in reverse:

def print_array_reverse(arr, i=0):
    if i == len(arr):
        return

    print_array_reverse(arr, i + 1)
    print(arr[i])

The only change is that the print statement comes after the recursive call. This means we first go to the end of the array, then print the elements as the calls come back. The last element in the array is printed first, and so on.

This is a pattern you will see often: the order of the recursive call relative to the work determines the direction of processing. If you do the work first and then recurse, you process forward. If you recurse first and then do the work, you process backward.

You can use the same idea when you want to return a result rather than print it. As we discussed in the first article, you can either use a passed variable or build up the result as you return. Here is an example that sums all the elements in an array with the build-up approach:

def sum_array(arr, i=0):
    if i == len(arr):
        return 0

    return arr[i] + sum_array(arr, i + 1)

The function finds the sum from index i to the end. The base case says that the sum of an empty slice is 0. The recursive step states that the sum is the current element plus the sum of everything after it. As the calls return, the values add up.

If you prefer using a passed variable, the code looks like this:

def sum_array(arr, i=0, result=None):
    if result is None:
        result = [0]

    if i == len(arr):
        return result[0]

    result[0] += arr[i]
    return sum_array(arr, i + 1, result)

Both approaches lead to the same answer. The build-up version is usually cleaner for simple cases, while the passed-variable can be better as the logic gets more complex.

Inserting at the Bottom of a Stack

Now, let’s look at a problem where recursion does more than just take the place of a for loop. You have to put something at the bottom of a stack without using any extra data structures.

If you try to do this iteratively, you run into a problem. You can only access the top of a stack, so you have to pop everything off, insert your item, and push everything back on. This requires a temporary stack to hold the elements (In Python, we use a list as a stack, with append() to push and pop() to remove from the top):

def insert_at_bottom_iterative(stack, item):
    temp = []
    while stack:
        temp.append(stack.pop())

    stack.append(item)
    while temp:
        stack.append(temp.pop())

The code works, but you need to create and maintain the temporary stack. When you use recursion, the call stack takes care of this implicitly. The recursive approach pops elements off one at a time, makes a recursive call, and then pushes each element back on after the call comes back. The base case is when the stack is empty, at which point we insert the item. As the recursive calls return, each previously popped element gets pushed back on top:

def insert_at_bottom(stack, item):
    if not stack:
        stack.append(item)
        return

    top = stack.pop()
    insert_at_bottom(stack, item)
    stack.append(top)

Let’s trace through this with a concrete example. Suppose the stack is [1, 2, 3, 4] (with 4 on top) and we want to insert 5 at the bottom.

The first call pops 4 and saves it. The stack is now [1, 2, 3]. The second call pops 3. The stack is [1, 2]. The third call pops 2. The stack is [1]. The fourth call pops 1, leaving the stack empty.

Now we hit the base case. We append 5, so the stack is [5]. Then the calls start coming back. The fourth call appends 1, giving us [5, 1]. The third call appends 2, giving us [5, 1, 2]. The second call appends 3, giving us [5, 1, 2, 3]. The first call appends 4, and we end up with [5, 1, 2, 3, 4].

The time complexity is O(n) because we need to pop and push each element once. The space complexity is also O(n) since the call stack must hold all popped elements. This is the same as the iterative version, but the recursive code is shorter and easier to read.

Nested Iteration with Recursion

Read more

Welcome to the 164th issue of the Polymathic Engineer newsletter.

Recursion is one of those topics that many software engineers “understand” in theory but struggle with in practice. They know the textbook definition (a function that calls itself). They have seen the Fibonacci example multiple times. But when they are asked to solve a recursive problem in an interview, something doesn’t work.

The problem is not that they are not smart, but that they don’t have the correct mental models. It took me a while to figure this out. When someone explained it to me, I was able to follow recursive code, but writing it from scratch often felt like guessing. The turning point was when I stopped thinking of recursion as a single, isolated technique and began seeing it as a set of patterns, each with its own structure.

In this article, we will start building the basic mental models you need to actually use recursion. In future articles, we will apply these foundations to specific patterns and problems.

The outline is as follows:

• When to use recursion

• The two building blocks: base case and recursive step

• Three ways to return results from recursive functions

• How to trace any recursive code (even when you’re lost)

• Time and space complexity for recursion

• A preview of the six recursive patterns

Project-based learning is the best way to develop technical skills. CodeCrafters is an excellent platform for tackling exciting projects, such as building your own Redis, Kafka, a DNS server, SQLite, or Git from scratch. Sign up, and become a better software engineer.

When to Use Recursion

There is a fact about recursion that people often overlook: any problem that can be solved recursively can also be solved iteratively. There is no case where you need to use recursion.

So the right question isn’t “can I use recursion here?” but rather “is recursion easier here?” There are a few questions you can ask yourself to figure this out.

1. Does the problem break down into subproblems? This is mostly what recursion is about. If a problem breaks into smaller versions of itself, recursion is probably a good fit. For example, sorting an array can be decomposed into sorting two halves and merging them. Finding a path through a maze can be broken into taking one step and finding a path from there.

2. Could you solve the problem using an arbitrary number of for loops? This point gets people confused. Let’s say you are asked to print all possible 6-digit numbers. It sounds easy: you can just write six nested for loops, one for each digit. But what if you are told to print all possible n-digit numbers? You can't write n nested for loops anymore since the number of loops changes depending on the input. This is an excellent example of when to use recursion, because it effectively allows you to create arbitrary nesting.

3. Does the problem fit one of the common recursive patterns? For most recursive problems you see in interviews, there are six patterns that you can look for. We’ll preview them at the end of this article and cover them in depth in future ones. If you recognize that a problem fits one of these patterns, that is a strong hint to use recursion.

4. Is it easier to solve recursively than iteratively? Sometimes the recursive code is just cleaner. Consider the task of printing a linked list in reverse order. Without recursion, you would have to push all the values onto a stack, then pop and print them. Recursively, the process reduces to three lines of code:

def print_reverse(node):

  if node is None:
    return

  print_reverse(node.next)
  print(node.value)

The recursive version uses the call stack as your stack, so you don’t need to explicitly manage one yourself. The code is more concise and probably easier to grasp.

However, there is a trade-off. If the time complexity is the same, the iterative solution usually runs faster. Function calls, stack frames, and other things that recursion does all add up. In general, if you have the possibility to choose between two equally simple recursive and iterative solutions, the iterative one will perform better.

But in an interview, clarity often matters more than micro-optimizations, so it is wise to pick whatever approach lets you write correct, readable code faster.

Base Case and Recursive Step

Every recursive function has two essential building blocks: a base case and a recursive step. Getting these right is what separates working code from infinite loops and stack overflows.

The base case is the point at which the recursion stops. It is the simplest form of the problem, and it can be solved immediately without any further recursive calls.

A useful technique for identifying the base case is to start with small examples. What is the answer when the input size is zero or one? These edge cases are usually the base cases.

For example, if you are writing a function to compute the sum of an array of integers, an empty array has a sum of 0, and an array with one element has a sum equal to that element. Both can be solved without recursion, making them natural base cases.

There is one important rule to follow here: the base case must match the return type of your function. If your function returns a list, your base case must return a list (not None or 0). This sounds obvious, but it is a common source of bugs. People often get stuck on this because they try to guess what the base case is instead of using concrete examples to figure it out.

# Wrong: function returns a list, but base case returns None
def get_combinations(items):
    if len(items) == 0:
        return None  # This will break the calling code
    ...

# Correct: base case returns an empty list
def get_combinations(items):
    if len(items) == 0:
        return [[]]  # A list containing the empty combination
    ...

In the recursive step, you divide the problem into smaller subproblems and combine the results. There are two things to keep in mind here.

The first is that a subproblem should have the same shape as the original problem. If your function computes the sum of elements of an array from index i to the end, then your recursive call should compute the sum of elements from index i+1 to the end. The meaning is the same, just applied to a smaller input.

The second critical thing is that the recursive step must converge to the base case. If your base case is i == 0 and your recursive call passes i + 1, you are moving away from the base case, not toward it. This causes an infinite recursion and eventually a stack overflow.

# Wrong: moves away from base case
def countdown(n):
    if n == 0:
        return
    countdown(n + 1)  # n grows, never reaches 0

# Correct: moves toward base case
def countdown(n):
    if n == 0:
        return
    countdown(n - 1)  # n shrinks toward 0

When you are writing code in an interview, you don’t get to run your code and see the stack overflow error. You need to find this mistake by looking at the code you wrote. A quick check that your recursive step gets you closer to the base case can save you from an embarrassing bug.

Three Ways to Return Results

When you write a recursive function, you need to decide how to collect and return the results. There are three main ways you can do this: use a global variable, use a passed variable, or build up results as you return. Each has its trade-offs.

The simplest approach is to use a global variable. You set a variable outside the code and change it as the recursion proceeds. As an example, consider the following code snippet that counts all the even numbers in an array:

def count_evens(arr, i):
    global result
    if i == len(arr):
        return
    if arr[i] % 2 == 0:
        result += 1
    count_evens(arr, i + 1)

This works, but global variables are generally not a good idea. They make code harder to test and reason about. A better alternative is to use a passed variable.

Read more

Welcome to the 163rd issue of the Polymathic Engineer newsletter.

Concurrent programming involves breaking down applications into separate units of work. These units, known as tasks, set up the execution flow of an application. At their core, tasks are just abstractions. At some point, they must eventually be mapped to the physical devices that execute the code.

Thank goodness, we don’t have to worry about this. The operating system, which is another layer of abstraction, handles this for us. Its role is to make the best use of the available hardware as possible, but it is not a magical box. Still, programmers need to organize their work in a way that helps the operating system use hardware in the best way possible.

In this article, we will discuss processes and threads, which are the two main building blocks of concurrency, and understand how they can talk to each other (Demo code in this GitHub repo).

The outline is as follows:

• Processes

• Threads

• Processes vs Threads

• Interprocess Communication

• Message-Passing Mechanisms

• The Thread Pool Pattern

Project-based learning is the best way to develop technical skills. CodeCrafters is an excellent platform for tackling exciting projects, such as building your own Redis, Kafka, a DNS server, SQLite, or Git from scratch. Sign up, and become a better software engineer.

Processes

The informal definition of a process is simple: it is a running program. A program by itself is just a file sitting on a disk. It contains a set of instructions, but it does nothing on its own. When the operating system takes those instructions and starts executing them on hardware, it becomes a process.

Think of it like a recipe in a cookbook. The recipe itself is just words on paper listing ingredients, steps, and timings. It doesn’t make anything by itself. But if a chef reads it and starts cooking, it becomes a meal in progress. Source code works the same way.

It is just a passive sequence of instructions. Developers write code using abstractions like memory, files, and network connections, but the actual resources must be given at runtime. The OS wraps all of this into what we call a process.

A process is made of several parts that the OS keeps track of:

• Address space: the memory the process can see and access

• Executable: the file with the machine instructions

• Process ID (PID): a unique name to identify the process

• Process state: whether it is running, waiting, or finished

• Open files and connections: any resources the process is using

All of these together make up the execution context. Since so many things are packed into a process, starting a new one is a pretty heavy thing to do, and processes are often called heavyweight.

If you look at a process as a whole, its lifecycle is, its lifecycle is straightforward. First, it doesn’t exist. After that, the OS creates it and places it in memory (the Created state). From there, it moves to the Ready state: it can run at any moment, but the CPU hasn’t picked it up yet. When the OS scheduler selects it, the process goes into the Running state. Once it completes or fails, it reaches the Terminated state.

Creating and terminating a process is relatively expensive since so many resources are attached to it and must be assigned or released.

Processes can create their own processes, called child processes, using system calls like fork() or spawn(). This is called spawning. Each child process gets its own independent memory address space, which means it is completely isolated from the parent process and any other children.

This is where things get interesting for concurrency. By spawning multiple processes, we can split execution across them and run them simultaneously on parallel hardware.

However, the isolation is both a good and a bad thing. Since two processes don’t share much by default, communication between them requires special mechanisms, which are usually several orders of magnitude slower than direct memory access. We’ll talk about that more later in this article.

Threads

Most operating systems let you share memory between processes, but this takes extra effort. However, there is a different abstraction that lets you share memory and other kinds of resources in a simpler way: threads.

At the end of the day, a program is simply a list of machine instructions that must be carried out in the right order. The OS uses the concept of a thread to do this. A thread is an independent stream of instructions whose execution can be scheduled by the OS.

In the previous section, we said that a process is a running program plus resources. If we break that apart, a process is a container of resources (address space, files, network connections, and so on), while a thread is the dynamic part: the set of instructions that run inside that container. From the OS’s point of view, a process can be seen as a unit of resources, while a thread can be seen as a unit of execution.

The idea behind threads is that sharing a common address space is the most efficient way for processes to communicate. Threads in the same process can share resources (address space, files, connections, and data) with each other and their parent process, in an easy way.

Each thread also keeps its own state to allow for safe, local, independent execution of its instructions. Each thread is unaware of the other threads unless it is trying to mess with them on purpose. The OS manages threads and distributes them across available processor cores. This makes a multithreaded program an effective way to run multiple tasks concurrently.

Historically, hardware vendors implemented their own versions of threads, and these implementations were quite different. This made it challenging for programmers to write portable multithreaded applications. For UNIX systems, IEEE POSIX defined a standard programming interface. Implementations that follow this standard are called POSIX threads or Pthreads.

Every program we run causes the OS to create a process, and every process has at least one thread. A process without a thread cannot exist. When we start a program, a main execution thread is created. Any thread, even the main one, can create child threads at any time.

Threads have much less memory overhead than the standard fork() function. Since threads use the same process, nothing is copied. This is one reason why threads are sometimes called “lightweight processes.” It takes less time for the OS to assign and handle thread resources, plus starting and ending threads is faster than starting and ending processes.

However, there is a catch. The OS provides complete independence of processes from each other, so if one of them crashes, other processes are not harmed. This is not true for threads. All threads in a process use the same shared resources, so if one crashes or messes up something, it’s likely that the others will too. To avoid this, programmers need to make sure that shared resources are only used by one thread at a time and give them more control over how threads behave.

Processes vs Threads

To illustrate the difference between processes and threads, let’s go back to our recipe example. Imagine you run a restaurant and need to prepare three different dishes.

Read more

Welcome to the 162nd issue of the Polymathic Engineer newsletter.

As everybody knows, computer vision is everywhere. From the filters on your phone camera to self-driving cars, from medical imaging to security systems, the ability for machines to “see” and understand visual information has become integral to modern software.

If you want to get started with computer vision, OpenCV is the tool you should look at. This open-source library has been around since 2000, and it is used by everyone from hobbyists to large companies like Google and Intel. You can build many cool applications with it, such as face detection, object tracking, and augmented reality. But before you can do anything interesting, you need to understand the basics: how images are represented in memory, how to read and write them, and how to display results to users.

In this article, we will build that basic know-how. We won’t just look at the code, but we will also explain why things work the way they do. If you get a solid understanding of these fundamentals, it will be much easier to grasp everything else you learn about computer vision.

The outline is as follows:

• How Images Are Represented in Memory

• Reading and Writing Images

• Working with Video and Camera Input

• Displaying Results and Handling User Input

Project-based learning is the best way to develop technical skills. CodeCrafters is an excellent platform for tackling exciting projects, such as building your own Redis, Kafka, a DNS server, SQLite, or Git from scratch. Sign up, and become a better software engineer.

How Images Are Represented in Memory

To work with images in OpenCV, you first need to understand what an image actually is at the code level. An image is a NumPy array: a grid of numbers where each number represents a pixel value.

Let’s begin with the simplest case: a grayscale image. Each pixel is a number between 0 and 255, where 0 is black and 255 is white. Any other number in between is a shade of gray. This is how you can make a 3x3 black picture from scratch:

import numpy as np
img = np.zeros((3, 3), dtype=np.uint8)
print(img)
"""Output: 
[[0 0 0]
[0 0 0]
[0 0 0]] 
"""

Color images work the same way. However, each pixel has three values instead of one, corresponding to the three color channels. One thing that confuses many newbies is that OpenCV uses the BGR (Blue, Green, Red) format, not RGB. When you change our previous picture to color, the NumPy array gets a third dimension:

import cv2
img = cv2.cvtColor(img, cv2.COLOR_GRAY2BGR)
print(img.shape) # Output: (3, 3, 3)

Now each pixel is a list of three values: [Blue, Green, Red]. A pure blue pixel would be [255, 0, 0], not [0, 0, 255] as you might expect from RGB. The middle image below has four colored quadrants: the blue quadrant (top-left corner) is created with [255, 0, 0], the green quadrant (top-right) is created with [0, 255, 0], the red quadrant (bottom-left) is created with [0, 0, 255], and the white quadrant is created with [255, 255, 255].

You can inspect any image’s structure using three properties:

• shape: Returns the dimensions. For a color image, this is (height, width, channels). For grayscale, it’s just (height, width).

• size: The total number of elements in the array. For a 100×100 BGR image, this would be 30,000 (100×100×3).

• dtype: The data type of the pixels. Most images use uint8 (unsigned 8-bit integers), which gives you the 0-255 range.

Getting familiar with this layout is critical to do everything else in OpenCV. When you manipulate images, you are really just doing math on NumPy arrays.

Reading and Writing Images

Now that you know how images are stored in memory, let’s have a look at how to load them from files and save them back. To do this, OpenCV has two key methods: imread and imwrite. The first loads an image, while the second saves an image. Their basic usage is straightforward:

import cv2
img = cv2.imread(’photo.png’)
cv2.imwrite(’photo.jpg’, img)

Even if the original file is grayscale, imread loads images by default in the BGR color format. If you want to load an image as grayscale, you need to specify it:

gray_img = cv2.imread(’photo.png’, cv2.IMREAD_GRAYSCALE)

Using the cv2.IMREAD_UNCHANGED option, you can also load the image as it was saved, including the alpha (transparency) channel if it exists.

The cool thing is that once you have an image loaded, you can modify it directly since it is just a NumPy array. For example, if you want to change single pixels to white, you can just assign new values:

img[0, 0] = [255, 255, 255] # Top-left pixel is now white

For bulk changes, NumPy’s array slicing is much faster than looping through pixels one by one. For example, this is how you can remove all green from an image or copy regions of an image using slicing:

img[:, :, 1] = 0 # Set all green channel values to zero

# Copy a 100x100 region from the top-left corner
roi = img[0:100, 0:100]

# Paste it somewhere else
img[200:300, 200:300] = roi

Of course, when copying regions, you need to ensure the areas are the same size. If the shapes don't match, NumPy will raise an error. The following image shows the result of manipulating the four-quadrant image we saw in the previous section: the green channel is removed from the top half, the bottom-left is set to blue, and a region is copied from the bottom-right to the top-left.

Working with Video and Camera Input

A video is simply a sequence of images, and OpenCV processes them the same way it processes images: frame by frame. The VideoCapture class lets you read videos, and the VideoWriter lets you save them.

The following code snippet creates a VideoCapture object and then loops through the frames to read from a video file:

video = cv2.VideoCapture('input.avi')

while True:
    success, frame = video.read()
    if not success:
        break
    # Do something with the frame

The read method returns two values: a boolean flag indicating whether a frame was successfully read, and the frame itself. When the video ends, success is set to False, and the loop is exited.

The amazing part is that using a webcam to capture works almost the same way. You just pass a device index instead of a filename. The default camera's index is usually 0. To save a video, you need a VideoWriter object. This requires a few more parameters: the output filename, a codec, the frame rate, and the frame size:

import cv2

camera = cv2.VideoCapture(0)
fps = 30
width = int(camera.get(cv2.CAP_PROP_FRAME_WIDTH))
height = int(camera.get(cv2.CAP_PROP_FRAME_HEIGHT))

writer = cv2.VideoWriter(
    'output.avi',
    cv2.VideoWriter_fourcc('X', 'V', 'I', 'D'),
    fps,
    (width, height)
)

while True:
    success, frame = camera.read()
    if not success:
        break
    writer.write(frame)

The codec determines how the video is compressed. Common options include XVID for AVI files and MP4V for MP4 files. Of course, the available codecs depend on your system, so you might need to experiment to find one that works.

One gotcha with cameras: the get method does not always return accurate frame rate values. It often returns 0. If you need precise timing, you are better off measuring the actual frame rate yourself or assuming a reasonable value, such as 30 fps.

Displaying Results and Handling User Input

To debug and build interactive applications, you need to be able to display images and videos. OpenCV provides a simple windowing system for this purpose. The imshow function displays an image in a window:

import cv2

img = cv2.imread('photo.png')
cv2.imshow('My Window', img)
cv2.waitKey()
cv2.destroyAllWindows()

The waitKey function is often confusing for beginners. Without such a function, the window would open and then close right away. It pauses the execution and waits for a key to be pressed. You can give it a timeout in milliseconds as an argument. Passing 0 will cause it to wait forever. Passing a positive number, it will cause it to wait for that many milliseconds before continuing.

If the user doesn't press any keys before the timeout, waitKey returns -1. Otherwise, it returns the ASCII code of the key that was pressed. You can use this mechanism to make your application respond to keyboard input:

while True:
    cv2.imshow('My Window', frame)
    key = cv2.waitKey(1)
    if key == 27:  # ESC key
        break

For mouse input, you need to register a callback function using setMouseCallback. The callback gets the kind of event, the x and y coordinates, and some extra flags:

import cv2

def on_mouse(event, x, y, flags, param):
    if event == cv2.EVENT_LBUTTONUP:
        print(f'Clicked at ({x}, {y})')

cv2.namedWindow('My Window')
cv2.setMouseCallback('My Window', on_mouse)

img = cv2.imread('photo.png')
while True:
    cv2.imshow('My Window', img)
    if cv2.waitKey(1) == 27:
        break

cv2.destroyAllWindows()

The event types are EVENT_MOUSEMOVE for movement, LBUTTONDOWN and LBUTTONUP for left-click, and similar events for right- and middle-click.

One important detail: OpenCV windows only update when waitKey is called. If you forget to call it inside your loop, you will see a frozen or blank window. In the same way, waitKey only captures keyboard input when an OpenCV window has focus.

Conclusion

We covered a lot of basic OpenCV ground in this article. You learned that images are just NumPy arrays, how to load and save them with imread and imwrite, how to work with video files and camera input using VideoCapture and VideoWriter, and how to display results and handle user input with OpenCV’s windowing system.

These are the building blocks for everything else you will do. If you want to go deeper, I suggest playing around with the code samples I used for this article, which I put in this GitHub repo. Try loading images having different formats, recording video from your webcam, or making a small app that responds to mouse clicks. The best way to get this concepts is to run code and see what happens.

Welcome to the 161st issue of the Polymethic Engineer newsletter.

Regardless of whether you are an experienced software engineer or are earlier in your career, performance reviews can be scary. There are so many ways they cause anxiety: you get graded as an employee, often compared directly to your peers, and in some big tech companies, the bottom 5-10% get cut every cycle.

But here’s the thing: performance reviews are a system. And like any system, once you understand how it works, you can learn to navigate it effectively.

Many engineers treat performance reviews as something that happens to them. They put in the hours, work hard, avoid negative feedback, and hope for the best. However, this is a dangerous mentality. It can work sometimes, but the more important thing to think about is when it doesn’t. And you want to minimize that as much as possible.

The engineers who consistently get strong ratings and promotions aren’t just better at coding. They understand how the game is played. They know how they are measured, they prepare strategically, and they make sure their work is visible to the right people.

In this issue, we break down the recipe for succeeding within a performance review:

• Understanding the Yardstick

• The three types of performance review systems

• Preparing for success before the cycle begins

• Executing during the cycle

• Writing feedback that matters (for yourself and others)

Project-based learning is the best way to develop technical skills. CodeCrafters is an excellent platform for tackling exciting projects, such as building your own Redis, Kafka, a DNS server, SQLite, or Git from scratch. Sign up, and become a better software engineer.

Understanding the Yardstick

The output of a performance review system is typically a rating. Think of it like school grades: you need to understand what the meaning of each rating is, what is considered average, and what you need to do to get promoted.

For example, at Meta, the rating system looks like this:

• Meets Some: You are doing quite poorly. Getting this rating usually means you will be put on a PIP or even fired.

• Meets Most: You are not meeting all expectations. Two or more Meets Most ratings typically lead to a PIP.

• Meets All: The default rating. If you just joined the company or are doing solid but unremarkable work, this is where you land.

• Greatly Exceeds: One level up from Meets All. This is quite hard to achieve.

• Redefines: The top rating, significantly harder than Greatly Exceeds.

Most other companies have a similar system. However, there is a part that surprises many engineers: the distribution of ratings is uneven. The bottom two ratings usually apply to 5-10% of employees. Around 40% get the middle rating, another 40% get the first “exceeds” tier, and only 10-15% reach the top rating.

This distribution also changes as you become more senior. At the junior level, it is common for most engineers to receive an “exceeds” rating because expectations are lower. But at senior levels, getting that same rating becomes much harder, and 60-70% of engineers end up with the equivalent of “Meets All.”

And here is the counterintuitive part: a Meets All rating does not mean you are doing well. It means you are effectively in the bottom half of your level. If you are aiming for promotion, Meets Al is not enough. In fact, it is almost a bad signal because it means you're not standing out from your peers.

So the first step is to understand your company’s rating system and ask yourself: What are the expectations for my level? Which rating is considered average? What is needed for promotion? The answers will be different for each company and each level.

The Three Types of Performance Review Systems

There are different kinds of performance review systems. Companies usually fit into one of three groups based on how they gather feedback and the power their managers have.

The ad hoc systems are the worst type. There is no regular framework, and feedback only happens when something major occurs. Maybe you made a significant mistake, or you told your boss you have a competing offer. Suddenly, they scramble to tell you how valuable you are and offer you a raise to stay.

This approach is common at small startups and consulting shops. The problem is that it rewards the squeaky wheel. Engineers who know how to complain and play politics make more money than those who just do their jobs.

In manager-dominated systems, performance assessments are conducted regularly, but your manager has full authority over your rating, compensation, and promotion. This is common in older companies, especially those where software engineers work in the IT department.

The problem here is that your career is based on what one individual thinks. You do well if your boss likes you. If they don't, or if they just don't speak out for you, your growth will stop, no matter how amazing your job is.

The 360-degree feedback system is what modern tech companies use. The idea is that feedback comes from all directions:

• From within: you write a self-review

• From above: your manager writes feedback about you

• From the side: peers you have worked with share their perspective

• From below: if you lead a team, your reports evaluate your leadership

The benefit of this approach is that it provides the clearest picture of how well you are doing. It also prevents one individual from having too much control over your career. Strong peer feedback might make up for your manager not valuing you enough. If you say you are a great team player, but your partners from other departments don't agree, that will show up too.

If your company does not have a 360-degree feedback system, think about introducing it yourself. You can ask your coworkers for feedback regularly via email or Slack. Ask them how you could improve on certain projects. If you can create a culture of organic feedback requests, that is a powerful thing for both you and your team.

Why Promotions Should Be Lagging

It took me a while to get this concept, but it changed how I think about career growth.

Read more

Welcome to the 160th issue of the Polymathic Engineer newsletter. This is the second article in our series on authentication and authorization.

In the last article, we talked about a major issue with API keys. They can be used to identify applications, but they can’t speak for users safely. You have to make a tough decision if a third-party app needs to do something for a user: either fully trust the third party or ask users to share their passwords.

Neither choice is good enough. As long as the third party has blind trust, they can pretend to be any user. If people share passwords, a third party can always get into user accounts. All user credentials are at risk if there is a single breach at the third party.

OAuth2 solves this problem. It is an authorization framework that lets users grant limited access to their data without sharing credentials. A user can say, “This app can read my email, but it can’t delete anything,” without giving away their password.

As of now, OAuth2 is the usual way to authorize APIs. It’s used by Google, Facebook, Microsoft, GitHub, and thousands of other big platforms. You need to know about OAuth2 if you are building APIs that third parties will use.

In this article, we'll cover:

• OAuth2 fundamentals and the four roles

• Authorization Code Grant for web applications

• Authorization Code Grant + PKCE for mobile apps and SPAs

• Client Credentials Grant for machine-to-machine communication

• JSON Web Tokens (JWT) as the standard token format

• Refresh tokens for seamless user experience

• PASETO as a modern alternative to JWT

Project-based learning is the best way to develop technical skills. CodeCrafters is an excellent platform for tackling exciting projects, such as building your own Redis, Kafka, a DNS server, SQLite, or Git from scratch. Sign up, and become a better software engineer.

OAuth2 Fundamentals

The first thing to keep in mind is that OAuth2 is not an authentication system, but a framework for authorizing access. This distinction is critical. OAuth2 tells you what a user has allowed an application to do. It doesn’t directly identify the user.

Here is a typical scenario that OAuth2 addresses. Let’s say a user wants to use a third-party scheduling tool to manage their social media post submissions. The scheduling tool needs to access the users’ social media profile data. Without using OAuth2, the user would need to give their social media password to the scheduling tool.

With OAuth2, the user can grant specific permissions to the scheduling tool without sharing credentials. The speaker might say, “You can read my profile and write posts, but you can’t access my payment information.” If the user later wants to revoke access, they can do so without changing their password.

OAuth2 defines four distinct roles in the authorization process:

• Resource Owner is typically an end user who owns the data. In our example, this is the user who has a profile in the social media app. The resource owner decides what access to grant to third-party applications.

• Authorization Server gives out access tokens after verifying the resource owner's identity and getting their permission. Most identity providers, such as Google, Microsoft, and Auth0, are OAuth 2.0 authorization servers. Servers have two endpoints: the authorization endpoint (where users give consent) and the token endpoint (where applications get access tokens).

• Client is the application requesting access to protected resources. In our example, this is the scheduling tool. There are two types of clients: confidential clients can maintain secrets (like web applications with server backends), and public clients cannot (like single-page applications and mobile apps).

• The Resource Server hosts protected resources and accepts requests using access tokens. In our example, this is the social media API. The resource server validates tokens and serves data if the token is valid and has appropriate permissions.

In real-world systems, the resource server is often an API gateway that sits in front of multiple backend services. Clients call the API gateway, which works as a single entry point. The gateway validates access tokens and directs requests to the right backend services. From the client’s perspective, the API gateway is the resource server, even though multiple services are behind it.

OAuth2 Grant Mechanisms

OAuth2 provides an abstract protocol that all authorization requests follow:

1. The client requests authorization from the resource owner.

2. The resource owner is redirected to a consent screen or another mechanism that grants or denies authorization.

3. The client presents the authorization grant to the authorization server and requests an access token.

4. The authorization server validates the grant and issues an access token if everything checks out.

5. The client makes a request to the resource server, including the access token in the request header.

6. The resource server validates the access token and returns the requested resource if the token is valid.

This flow is meant to be flexible. Different grant types follow these steps differently, depending on their security requirements and use cases. The most important thing to remember is that the resource owner is in charge, the client never sees the user’s credentials, and the resource server only needs to verify that tokens are valid.

In order to choose the right OAuth2 grant, you need to know the difference between confidential and public clients. Confidential clients can securely store secrets. A web application with server-side code is confidential because the server can store a client secret that users never see. Using this secret, the server can prove its identity to the authorization server.

Public clients cannot securely store secrets. A single-page application runs entirely in the browser. Users can view all JavaScript code, including secrets. The same problem exists for mobile apps since users can decompile apps.

This difference is essential because some OAuth2 grants need a client to authenticate with a secret. Confidential clients can use these grants. Clients who are open to the public need different grants that don’t depend on secrets.

In the next section, we’ll explore the Authorization Code Grant, the most secure and widely used grant for confidential clients.

Authorization Code Grant

The Authorization Code Grant is the most complete and secure OAuth2 grant type. It is made for confidential clients who can protect a secret.

The critical thing about this type of grant is that it uses an intermediate authorization code before issuing an access token. This adds a layer of security by dividing the user-facing authorization step from the server-to-server token exchange.

Read more

Welcome to the 159th issue of the Polymathic Engineer newsletter.

As you know, machine learning is everywhere. It powers the movie recommendations you see on Netflix, filters spam from your inbox, enables self-driving cars to navigate city streets, and can even help doctors diagnose diseases. However, behind all these applications lies a fundamental question: what type of machine learning should you use?

There are multiple ways to classify machine learning systems. The most common is based on the type of supervision they receive during training. However, you can also categorize them by whether they learn incrementally or all at once, or by how they generalize what they have learned from training data to new examples.

Understanding these distinctions is important because choosing the wrong approach wastes time and resources. You cannot build a spam detector system without labeled spam examples. In the same way, you can’t group customers into segments if you have already decided what those segments should be. Determining which machine learning method is right for the job is the first step to building something that works.

In this issue, we will have a look at the different types of machine learning and see the use cases they fit. The outline is as follows:

• Labeled vs Unlabeled Data: The Foundation

• Supervised Learning: Predicting Labels

• Unsupervised Learning: Finding Hidden Patterns

• Reinforcement Learning: Learning Through Interaction

• Beyond the Big Three: Other Ways to Classify ML Systems

Project-based learning is the best way to develop technical skills. CodeCrafters is an excellent platform for tackling exciting projects, such as building your own Redis, Kafka, a DNS server, SQLite, or Git from scratch. Sign up, and become a better software engineer.

Labeled vs Unlabeled Data: The Foundation

Before diving into the different types of machine learning, we need to understand the raw material that powers them all: data. In fact, machine learning is just the field that studies how to teach computers to learn from data. The examples that the system uses to learn are called the training set, while the part of the system that learn and generates predictions is known as the model.

Data is just information put together in a structured way. If you have a table with rows and columns, you have data. Each row represents a single data point, and each column represents a feature, which is a property or trait that describes that data point.

For example, in a dataset of houses, some features include size, number of bedrooms, distance to the nearest school, and the local area crime rate. In an email dataset, the features could be the sender, the subject, the body text, or the number of attachments.

But here is where things start to get interesting. Some features stand out to us, and we refer to them as “labels.” A label is a feature we want to predict. It’s the answer we are looking for based on all the other features. The important thing is that what counts as a label depends entirely on the problem we’re trying to solve. If we’re predicting house prices, the price is the label; if we want to detect spam emails, the label is spam/not-spam classification; if we’re estimating how long it will take for a patient to recover, that time is the label.

The goal of a machine learning model is to analyze the features and produce the most accurate possible prediction of a label. This brings us to a fundamental distinction:

• Labeled data comes with the answers already filled in. We know the house prices, we know which emails are spam, and we see the patient outcomes.

• Unlabeled data has no such answers. We have all the features, but no target to predict.

For example, a set of emails marked as “spam” or “not spam” is labeled data. A set of emails without such tags is unlabeled.

Why does this matter? Because the type of data you have determines which machine learning approach you can use. Labeled data enables supervised learning, where the model learns from examples with known correct answers. When you give a model unlabeled data, it has to learn on its own, without being told what to look for. This is called unsupervised learning. This distinction is the foundation for everything that follows.

Supervised Learning: Predicting Labels

Supervised learning is the most widely used type of machine learning. It’s what makes picture recognition, spam filters, recommendation systems, and a massive number of other applications we use every day possible. The defining characteristic is simple: the training data includes the answers we want the model to learn.

In supervised learning, the model follows a remember-formulate-predict framework. The first thing it does is load the training dataset and memorize the examples. Then it comes up with rules or patterns that link the labels to the features. Lastly, when a new piece of data comes in, the model uses those rules to guess what the label will be.

For example, a spam filtering system looks at thousands of emails that have already been marked as spam or not spam to figure out what patterns make them different. It then uses those patterns to classify new incoming emails.

Supervised learning problems fall into two categories, based on what we want to guess.

Regression models predict numerical values. The output is a number that can take on many possible values. An example of a regression problem is predicting how much a house will cost based on its size and location. Other examples might be estimating how long a customer will stay on a website or forecasting stock prices. The most used regression method is linear regression, which fits a line (or a higher-dimensional surface) to the data. Other popular approaches include decision trees and ensemble methods like random forests and gradient boosting.

Classification models predict categories. There are only a limited number of possible states for the output. Finding spam is like dividing emails into two groups: those that are spam and those that are not. Image recognition is also classification. Given an image, the model tells you whether it contains a cat, a dog, a car, or another category. Logic regression, decision trees, support vector machines, naive Bayes, and neural networks are all common classification methods.

The line between regression and classification is not always rigid. Some algorithms can do both things with only small changes. For example, logistic regression produces numeric values representing the probability of belonging to a given category, making it commonly used for classification.

Supervised learning can be very accurate when there are many well-labeled examples. Labels, on the other hand, can be pricey. Someone has to mark those emails as spam by hand, put those pictures into the right category, or write down the actual prices of the homes for sale. Supervised learning might not work in domains where marking is hard or expensive, at least not without help from other methods.

This is where semi-supervised learning comes in. There are algorithms that can work with data that is only partly labeled, meaning that many cases aren’t labeled and only a few are. Photo-hosting services such as Google Photos use this method. The system clusters similar faces together (unsupervised), so you only need to assign each person a single label. Semi-supervised learning methods combine the pattern-finding power of unsupervised methods with the precision of labeled examples.

Read more

Welcome to the 158th issue of the Polymathic Engineer newsletter. This week, we are starting a series of in-depth articles on two topics that every developer deals with, but few really understand: authentication and authorization.

Authentication asks “Who are you?” while authorization asks “What are you allowed to do?” The first checks identities, while the latter defines permissions.

Even if you are familiar with login forms, password validation, and user sessions, do you actually know what’s going on behind the scenes? Do you know when to utilize tokens instead of passwords, or why OAuth2 is now the norm in the industry?

The stakes are really high. An authentication system that doesn’t work properly can expose private user data, let unauthorized people in, and damage your application’s reputation.

In this series, we will look at many different ways to authenticate, from old-fashioned passwords to new solutions that don’t use passwords. We’ll look at the pros and cons of each, as well as how they can be used in practice. By the end, you will know how to pick the best method for your needs.

Here’s what we’ll cover in this first article:

• Password-based authentication

• HTTP Basic Access Authentication

• Session-cookie authentication

• Token-based authentication

• API Keys: A Simple Token Approach

• One-Time Passwords (OTP)

• Time-Based One-Time Password (TOTP)

• Federated identity and Single Sign-On (SSO)

Project-based learning is the best way to develop technical skills. CodeCrafters is an excellent platform for tackling exciting projects, such as building your own Redis, Kafka, a DNS server, SQLite, or Git from scratch. Sign up, and become a better software engineer.

Password-Based Authentication

Passwords are the most traditional form of authentication. The way they work is very straightforward: you check someone’s identity by verifying if they can type a secret phrase. You let them in if they know it. If they know it, you grant access.

This approach has been around since the first computers, and we use it for everything from email to banking apps. But making password-based authentication safe is harder than it seems. Even though passwords seem straightforward, they are challenging for both users and developers.

Most people still rely on memorization or handwritten notes for their passwords. This means they frequently forget or lose their passwords, so you have to have a password recovery flow in place. Users need a way to reset their password and then show who they are through email or text message. This makes your system more complicated and could leave it vulnerable to security holes.

Also, reusing passwords is very common. Most people use the same password across multiple sites if they don’t have a password manager. When there is a data breach at one service, that problem becomes yours. If people who use your site don’t change their passwords, anyone with access to credential files that have been leaked can access their accounts.

The 23AndMe breach demonstrated this perfectly. Hackers didn’t need to break into any system. They just tested leaked passwords from other breaches against 23AndMe accounts. 7 million accounts of users who had reused passwords were compromised.

But password management is complex for developers. You need to hash passwords securely, not store them in plain text. You must use strong hashing algorithms and add unique salts to each password to protect against rainbow table attacks. You need to implement password strength requirements, handle password resets securely, and protect against brute-force attacks.

Thankfully, third-party tools such as AWS Cognito and Firebase can handle much of this complexity. However, even with these tools, you still have to deal with the basic weaknesses of password-based authentication.

HTTP Basic Access Authentication

Let’s start with the simplest password authentication mechanism: HTTP Basic Access Authentication. When a client tries to access a protected resource, the server returns a 401 Unauthorized status code and includes the Authorization: Basic header. This lets the server inform the client that it needs to provide credentials.

The client then prompts the user for a username and password. These credentials are combined into a single Base64-encoded string in the format username: password and sent back to the server in the Authorization: Basic header.

The server then decodes the Base64 string, splits the username and password, and compares them against its list of users. If the credentials are correct, the user can get in. If not, the server returns a second 401 response.

Here is the key problem: Base64 encoding is not encryption. Anyone who intercepts the request can easily decode the credentials. It is like sharing your password in plain text with only a little cover.

Modern applications solve this by using HTTPS, which encrypts all data between the browser and server. But even with this approach, HTTP Basic Access Authentication still has limitations. The browser sends credentials with every request to protected resources within the same domain. This makes things easier for users, but it means that passwords are always being sent across the network.

This method of authentication isn’t beneficial for current websites. However, it is helpful to understand because it shows the main problem: how do we make sure people are who they say they are without sending passwords across the network all the time?

Session-Cookie Authentication

Session-cookie authentication has been introduced with the goal of minimizing the number of password transmissions across the network.

The core idea is simple: users enter their password once during login. The server then issues a session ID that represents their authenticated state. This session ID travels with subsequent requests instead of the password. Think of it like a hotel key card. You show your ID at check-in (authentication), and they give you a key card (session ID). You don’t need to show your ID every time you enter your room; you use the key card.

But what is the role of Cookies? When we discussed how to build stateless services, we saw that cookies are small pieces of data that web servers store in your browser.

When a server sends a cookie to your browser, the browser automatically includes it in every subsequent request to that domain. This allows the server to maintain state across multiple requests.

Cookies serve many purposes: tracking logged-in users, storing preferences, and recording user behavior. For authentication, cookies typically store session identifiers.

Here is the complete flow:

1. User Login. You submit your username and password to the server. The server validates these credentials against its user database. If they match, the server creates a session for you.

2. Session Creation. The server generates a unique session ID that serves as the key to your authenticated state. The server stores session data on the server side. This might include your user ID, roles, preferences, and when the session was created.

3. Session ID Delivery. The server sends the session ID to your browser in a Set-Cookie header. Your browser stores this cookie automatically.

Read more

Welcome to the 157th issue of the Polymathic Engineer.

Every software engineer has seen something like this happen: you write a feature, ship it to production, and everything works great. Then, a few weeks later, you get different requirements. You have to support a new use case, but you find out that your code is not ready.

You open the codeba…

Read more

Welcome to the 156th issue of the Polymathic Engineer newsletter.

This week, we will take a closer look at treaps, a hybrid data structure that beautifully combines the best properties of binary search trees and heaps.

If you have worked with binary search trees, you know they are great for maintaining sorted data and providing fast lookups. If you have used heaps, you know how well they track the highest- (or lowest) priority element. But what if you need both features in a single structure?

That’s precisely where treaps shine. The name itself comes from the words "tree" and "heap," and that's what they are: a hybrid data structure that keeps elements in sorted order while simultaneously tracking priorities.

The outline will be as follows:

• Multi-Dimensional Data Indexing

• Combining Trees and Heaps

• How Treaps Work

• The Balance Problem and Randomization

• Applications and Use Cases

Multi-Dimensional Data Indexing

Imagine you are building a task management system for a development team. Each task has a unique identifier (e.g., “bug-1247” or “feature-dark-mode”) and a priority score. The priority score is based on how important the task is to the business, how quickly it needs to be done, and how many other tasks it depends on.

Your system needs to support two critical operations:

• Search for a specific task by its identifier to update details like status, assignee, or description.

• Retrieve the highest-priority task that should be worked on next.

Both operations need to be fast, because developers use the system all day long.

The straightforward approach is to use a hash table for the tasks. Hash tables provide O(1) average-case lookups by identifier, which handles the first requirement perfectly. However, it becomes difficult to find the highest-priority job. You would need to scan through all the n tasks to find the one with the maximum priority, and that results in O(n) time complexity.

Another option is to use a min-heap ordered by priority. You can always get to the top priority task in a heap in O(1) time, but it takes O(log n) time to extract it. The trouble now moves to the search process. Heaps aren’t meant for quick, key-based searches. To find a specific task by its identifier, you have to scan the whole heap, which takes O(n) time.

At this point, you might consider using both structures simultaneously: a hash table for fast lookups and a heap for priority tracking. This approach works but comes with several challenges. First, you need twice the memory since each task is stored in both structures. Second, you need to keep both structures synchronized. When you update a task’s priority, you must update it in both places. When you delete a task, you must remove it from both structures.

The main problem is that each task has two dimensions: its identifier and its priority. We need a data structure that can efficiently manage operations on both dimensions without the overhead of maintaining two separate structures.

This is where treaps come in. They natively support fast key-based search and provide quick access to the element with the highest priority. In the following sections, we’ll see how treaps achieve this by combining the structural properties of trees and heaps in a single data structure.

Combining Trees and Heaps

Treaps solve the problem of multidimensional sorting by making sure that the same data structure satisfies two distinct constraints for ordering.

The first constraint comes from binary search trees. Every node in a BST is associated with a key. The keys in its left subtree are smaller, and the keys in its right subtree are larger. This property enables efficient search operations, as you can traverse the tree by comparing keys and selecting the appropriate branch at each node.

The second constraint comes from heaps. In a min-heap, the priority of any node is no greater than the priorities of its children. This property makes sure that the node with the lowest priority value is always at the root of the tree.

The key concept is that you can apply these two constraints to different attributes of the same data. In our task management example, the task identifiers follow the BST (horizontal) ordering constraint, while the priority scores follow the heap (vertical) ordering constraint.

Let’s look at a concrete example to better understand. If we have seven tasks with the following identifiers and priorities: (“bug-1247”, 77), (“feature-auth”, 10), (“refactor-db”, 55), (“docs-api”, 95), (“test-ui”, 32), (“clean backlog”, 76), (“erp”, 89), (“schedule call”, 56).

In a treap, these tasks might be organized as follows:

• The root contains “feature-auth” (priority 10) because it has the highest priority

• “bug-1247”, “docs-api”, “clean backlog”, and “erp“ are in the left subtree because they come before “feature-auth” alphabetically

• “refactor-db”, “test-ui”, and “schedule call” are in the right subtree because they come after “feature-auth.”

• Within each subtree, the heap property is maintained for priorities

This structure allows you to search for “clean-up repo” by traversing left from the root (following the BST property). At the same time, the root is always where you may find the highest-priority task (following the heap property).

The hard part is making sure that both constraints are met when adding or removing nodes. You can’t just use standard BST operations because they might not follow the heap property. In the same way, you can’t use standard heap operations because they might not follow the BST property.

The solution is to use a special operation called rotation, which allows you to change the tree structure so the heap property is restored without violating the BST ordering.

How Treaps Work

Now that we understand what treaps are, let’s see how they actually work. We’ll start with the data structure representation, then explore rotations, and finally look at the main operations.

Node Structure

A treap node needs to store more information than a regular BST node. Here is a basic implementation:

class Node:
    def __init__(self, key, priority):
        self.key = key
        self.priority = priority
        self.left = None
        self.right = None
        self.parent = None
    
    def is_leaf(self):
        return self.left is None and self.right is None
    
    def is_root(self):
        return self.parent is None
    
    def is_left_child(self):
        return self.parent is not None and self.parent.left == self
    
    def is_right_child(self):
        return self.parent is not None and self.parent.right == self

class Treap:
    def __init__(self):
        self.root = None
    
    def _set_left(self, parent, child):
        """Helper to set left child and update parent reference."""
        parent.left = child
        if child is not None:
            child.parent = parent
    
    def _set_right(self, parent, child):
        """Helper to set right child and update parent reference."""
        parent.right = child
        if child is not None:
            child.parent = parent

Each node stores its key, priority, and references to its left, right, and parent children. The parent reference is not strictly necessary, but it makes rotations much easier to implement.

The Role of Rotations

Read more

Welcome to the 155th issue of the Polymathic Engineer newsletter, the first in the new year.

The front-end layer of a web application is where the battle for scalability is won or lost. Every user interaction, every click, every API call flows through this layer. It is your first line of defense and the component receiving the most traffic.

The front-end layer has to deal with the most traffic and concurrent requests of any part of the system. One hundred users might be manageable. A thousand users start getting interesting. But what do you do when you need to serve 10,000 users at once? Or one hundred thousand?

The traditional approach of buying bigger servers hits a wall sooner or later. You can only scale vertically so far before you run out of RAM, CPU cores, or budget. Adding more servers to handle the load is the only way to keep going.

However, you can’t simply add more servers if your architecture won’t allow it. The reason is the state. State is any data that makes one server different from another. It could be session data stored in memory, files saved on local disk, or locks held by a specific process.

When servers hold state, users are tied to particular servers. This means that Requests can't be sent freely to different servers, servers cannot be cloned or replaced without downtime, and auto-scaling becomes a nightmare.

The solution seems simple: remove all state from your front-end servers. Make them completely interchangeable clones. But simple doesn’t mean easy. In this article, we will explore the most common types of state that sneak into web applications and the practical strategies to eliminate them.

The outline will be as follows:

• Stateless vs. Stateful

• Managing HTTP Sessions: Three Approaches

• Managing Files: User Content and System-Generated Data

• Managing Other Types of State

Project-based learning is the best way to develop technical skills. CodeCrafters is an excellent platform for tackling exciting projects, such as building your own Redis, Kafka, a DNS server, SQLite, or Git from scratch.

Sign up, and become a better software engineer.

Stateless vs. Stateful

Before talking about specific solutions, we need to understand what makes a service stateless or stateful. The difference is simple but very important.

A stateless service doesn’t hold any data between requests. It processes each request independently, without relying on information stored from previous requests. All the data needed to handle a request comes from either the request itself or from external storage.

A stateful service, on the other hand, keeps data between requests. Other instances of the same service can’t see this data because it’s stored locally, like in memory, on disk, or in local variables.

The key difference is interchangeability. From the point of view of the client, stateless service instances can be swapped out at any time. The client can send a request to any instance and get the same result. Stateful services instead force clients to stick to the same instance, because that’s where their data is stored.

Let’s use an analogy to make this concrete. Imagine you’re visiting a coffee shop chain with multiple locations across the city. In this analogy, the chain is your website, each location is a server, and ordering a coffee is making a web request.

If the chain coffee shop is stateless, you walk into any location, place your order, pay, and receive your coffee. You provide all the necessary information with every order: what you want, how you want it done, and how you want to pay. The barista doesn’t need to know anything about you. You can go somewhere else tomorrow and get the same service. Each transaction is complete and independent.

Things are different if each shop in the chain keeps a physical loyalty card for each customer that the barista needs to stamp after each purchase. Now you are tied to this specific location. If you visit a different branch of the same shop, they can’t honor your loyalty card because it is not in their system. You need to return to your original location to use the stamps you’ve accumulated. The chain coffee shop became stateful.

The state's implications for scalability are enormous. With stateless servers, you can add or remove servers whenever you want. Traffic can be distributed evenly across all instances. If a server crashes, you just route requests to another server without losing any data, since none was stored there in the first place.

With stateful servers, everything becomes complicated. There must be ways for users to always get to the right server. It is not easy to get rid of servers because users' data would be lost when they are removed. Users are locked into certain instances, so you can't easily distribute the load.

Managing HTTP Sessions

Most web applications need to keep track of users across multiple requests. You log in once, and the site needs to remember you as you browse across different pages. When you add items to a shopping cart, they need to stay there.

However, data exchange on the web relies on HTTP, a stateless protocol. Each request is independent, and HTTP doesn’t retain any information about previous requests. To work around this, web applications use cookies to create the illusion of a session.

Here is how it works. When a user first visits a website, the server sends back a small piece of data called a session cookie. The browser stores the cookie and sends it with every request to that site. The server then uses the session ID in the cookie to figure out which requests are from the same user and belong to the same session. Even when multiple users connect from the same IP address, cookies let the web server figure out which requests belong to which user.

Now, each session has associated data, such as the user ID, preferences, shopping cart contents, or the page the user was last viewing. This session-specific data needs to be available on every subsequent request so the application knows who is logged in and can personalize their experience.

But where does this session data actually live? If it lives in the memory of a specific web server, the user is stuck with that server. All the requests need to go to the same machine. You can’t freely distribute traffic and easily add or remove servers, and you have created a stateful system.

There are three practical approaches to avoid this, each with different tradeoffs:

1. Put Everything in Cookies. The most straightforward approach is to store all session data in the cookie itself. Instead of the cookie containing just a session ID, it includes the actual data, encrypted and encoded. When a request comes in, the server reads everything it needs from the cookie. When sending a response, it updates the cookie with any changes. This works beautifully when your session data is small, such as a user ID, a security token, and a few preferences. Overhead is minimal, and you avoid the complexity of managing external storage. The problem comes when you need to store more data. Browsers send cookies with every single request, and if your session data is 5KB, that’s 5KB added to every request and response. On mobile connections, this adds up quickly. To make things worse, consider that after encryption and encoding, your data grows by about a third.

   

Read more

Hi Friends,

Welcome to the 154th issue of the Polymathic Engineer newsletter.

This is the last article of 2025, and the outline will be as follows:

• End-of-year reflections

• The Polymathic Engineer in 2025

• Best articles of the year

Read more

Welcome to the 153rd issue of the Polymathic Engineer.

When you design a distributed system, one of the most important aspects is how you divide it into components. What should each part have? How should they talk to each other? It may seem easy to answer these questions correctly, but it is very important that you do. Changing the general ar…

Read more