Hi Friends,
Welcome to the 197th issue of the Polymathic Engineer newsletter.
Every software engineer who has written a library knows how critical it is to foresee a need for some features. If you ship a minimal set of features with a design that isn't extensible, you can easily get in trouble. If you get asked for knobs you didn’t think of, you end up refactoring the code and changing the APIs frequently.
It’s challenging to find the right balance. If you make the library as flexible as possible from the beginning, putting an extension point everywhere someone might ever need one, you end up with an over-engineered codebase. If you stay minimal and refactor every time a new requirement shows up, you end up with APIs that change every month.
Flexibility isn’t free. Every feature increases complexity at some level, in your code or in the code of your clients. Also, you need to pay attention to the thread execution model of your API extension points.
If your design allows synchronous client calls, you should assume some clients will block on them, impacting your component’s service-level agreement (SLA) and resource usage, such as threads. But introducing asynchronous logic that should work in parallel in your code adds additional complexity. You need to maintain a dedicated thread pool and monitor it. Complexity doesn’t disappear, and somebody has to pay for it.
Abstractions and extension points should emerge from evidence rather than from a guess. What you can foresee is not which features clients will ask for, but where in your component they will need to plug in.
This article starts from the most straightforward design of a small component without extensibility points and makes it more flexible one step at a time. The code examples are in C#, but the ideas apply to any language.
The outline is as follows:
- The straightforward version
- Letting clients bring their own metrics library
- Hooks: running client code between phases
- Listeners: notify, don’t wait
- Picking your spot on the axis
Your AI Agent Wrote the Code. What Happens Next?
The code still needs testing, failed checks need fixing, and someone has to figure out whether the PR is ready to merge. Multiply that across a backlog, and there’s a lot to keep track of.
SuperPlane 1.0 is an open source AI software factory that helps engineering teams automate that work. You define how routine tasks move through coding agents, automated checks, and feedback, so engineers can focus on reviewing the results and deciding what gets merged.
The straightforward version
Suppose that your team has to build a shared component that publishes a message to a message broker. Besides that, it needs to track metrics like how many messages were published and how many failed. The third functionality that our component should provide is the possibility to retry a failed publish. The caller of our code will specify the maximum number of retries.
Retries should be transparent to the caller. If the first attempt fails and the second one succeeds, the caller gets a successful result. We only increment the retries metric. If the retries are exhausted, we report a failure to the caller.
We will call our new component MessagePublisher. The constructor takes the metrics registry and the broker client as arguments. Both are classes from two third-party libraries. The metrics registry is the object that a metrics library gives you to create counters and timers and export them to a monitoring backend. Most metrics libraries have one, and each has its own API.
public class MessagePublisher
{
private readonly MetricsRegistry registry; // from the third-party metrics library
private readonly BrokerClient broker; // from the third-party broker SDK
private readonly int maxRetries;
public MessagePublisher(MetricsRegistry registry, BrokerClient broker, int maxRetries)
{
this.registry = registry;
this.broker = broker;
this.maxRetries = maxRetries;
}
public void PublishWithRetry(string topic, byte[] payload)
{
var message = BuildMessage(topic, payload);
Exception lastError = null;
for (int attempt = 0; attempt <= maxRetries; attempt++)
{
if (attempt > 0)
{
registry.Counter("messages.retry").Increment();
}
try
{
broker.Publish(message);
registry.Counter("messages.success").Increment();
return;
}
catch (BrokerException e)
{
registry.Counter("messages.failure").Increment();
lastError = e;
}
}
throw new PublishException(
$"Could not publish to {topic} after {maxRetries} retries", lastError);
}
}Let’s now focus on the implementation of the algorithm. To publish a message, the PublishWithRetry method takes two parameters: a string topic and the message payload. Every attempt after the first one increments the retry counter. A successful execution increments the success counter and returns. A failed execution increments the failure counter and proceeds to the next attempt. When the attempts are exhausted, the component raises a PublishException encapsulating the last error returned by the broker.
This code meets the requirements, but it doesn't provide a way to extend it. The only way the caller can alter its behavior is by passing the maximum number of retries in the constructor.
The metric registry is a black box, and we use only its public API. However, by using this class in our component, we are coupling our MessagePublisher to a specific metric library. Many metrics libraries are available, and if the client wants to choose a different one, our code will not allow it. The same coupling makes the component hard to test, since every unit test needs a real registry from the vendor library.
Letting clients bring their own metrics library
At this point, our component has a hard dependency on a third-party library that is responsible for gathering metrics. This isn’t a problem at first sight, but other engineers and systems will use our code. By using any class from this library in our codebase, we limit the future implementations of our component. We also impose the restriction that wherever our code is used, the same metrics library must be used.
The way out is to abstract away the metrics-related code. The pattern is quite simple. We define a generic interface that becomes the entry point to our component, and from this point on, our code integrates with any metrics implementation only through this interface. The interface defines the contract between our component and any third-party library, and it doesn’t need to import any class from a metrics library.
public interface IMetricsProvider
{
void IncrementSuccess();
void IncrementFailure();
void IncrementRetry();
}Notice that the interface doesn’t allow the caller to get the data back out of the component. This is a sound limitation because the caller injects their own metrics registry to track the metrics. They own the registry and can read the metrics directly, so there is no need to add accessor methods.
The constructor of MessagePublisher now takes an IMetricsProvider instead of a MetricsRegistry, and the loop calls IncrementRetry(), IncrementSuccess(), and IncrementFailure() instead of touching the counters by name. The client must provide the implementation details. Suppose that a client wants to keep using the metrics library from the previous section. All they need to do is implement the interface on top of their registry.
public class RegistryMetricsProvider : IMetricsProvider
{
private readonly MetricsRegistry registry;
public RegistryMetricsProvider(MetricsRegistry registry)
{
this.registry = registry;
}
public void IncrementSuccess() => registry.Counter("messages.success").Increment();
public void IncrementFailure() => registry.Counter("messages.failure").Increment();
public void IncrementRetry() => registry.Counter("messages.retry").Increment();
}How did foreseeing this feature impact the flexibility and complexity of our component? The implementation details were abstracted away, and the MessagePublisher logic got simpler. There is no longer a need to know anything about counters or registries. The complexity of our component decreased. And since clients can now provide any metrics implementation they want, flexibility increased.
It would seem that increasing flexibility and decreasing complexity is a win-win situation. However, the complexity we removed from our component lives somewhere. If multiple clients use our component, they all need to implement the new interface. We outsourced complexity to the various clients’ repositories. From the client’s perspective, a component that requires many extra steps before it can be used may be cumbersome, and clients may end up picking a different one.
A good middle ground is to extract the interface but ship a default implementation that most users will pick. RegistryMetricsProvider can be part of the component itself, and the constructor can use it when the client doesn’t provide anything else. This keeps the complexity in our system. We don’t outsource it to the clients. If they want a different implementation, they can implement the interface and give it to our component.
In real life, our systems rely on many external components, and abstracting all of them may not be feasible. Even in our simple example, the component still depends on the concrete broker client. A broker client exposes many more methods than a metrics registry, and they are hard to hide behind an abstraction. Trying to develop an abstraction that anticipates every use case for such a library would greatly increase our code's complexity. Besides that, an interface only lets clients swap one implementation for another. It doesn’t help when a client asks for something new, like signing the payload before it leaves the component. This is where hooks come in.
Hooks: running client code between phases
Every component has a life cycle with multiple steps. Ours is short: the caller invokes PublishWithRetry, the component builds the message, and then it publishes it to the broker, retrying if needed. Instead of trying to foresee every possible use case for our clients, we can let them provide their own behavior and inject it into our component between these steps. We no longer need to change our API and code for new feature requests. Clients provide their logic, and our code doesn’t need to know about it.
The mechanism that gives this kind of flexibility is a hook. A hook is a piece of client code that our component calls at a specific point in its life cycle. In our example, we want to let clients hook into the code after the message is built but before it is sent to the broker. This is the point where a client can sign the payload, add a tracing header, or validate the message against a schema. The client only needs to implement a hook interface with one method, and our component calls it at the right time, passing the message just built.
public interface IMessageHook
{
void OnMessage(Message message);
}Clients can inject more than one hook, so the constructor of MessagePublisher accepts a list of them. Between the two phases of the life cycle, the component iterates over every injected hook and passes it the message.
public void PublishWithRetry(string topic, byte[] payload)
{
var message = BuildMessage(topic, payload);
foreach (var hook in hooks)
{
hook.OnMessage(message);
}
// retry loop as before
}The hook receives the message object itself, so it can modify it before it leaves the component. The flexibility of this solution is high, and at first sight, we achieved it without increasing the complexity of our code. A few lines, and we removed the burden of foreseeing the exact features that our clients may need in the future.
Unfortunately, we don’t have any influence over the client’s code. Nothing in the hook interface says what the hook is allowed to do, and nothing stops it from throwing an exception. If a client’s hook throws, the exception propagates through our loop, the message is never published, and the caller gets an error that has nothing to do with our component. By providing flexibility to a client, we introduced a new failure mode in our code. To guard against it, we need to wrap the code we don’t own in a try-catch block. If the hook fails, we log the error to give feedback to the client and continue with our processing. Regardless of the logic injected by the caller, we don’t want our core processing to be impacted by problems in it.
Things get even more complicated if we pass internal state to the hooks. In our example, the hooks only get the message. Suppose that we passed the broker client too, since a client might want to publish an extra audit message. Now the hook can execute additional publishes through a connection we have tuned for our traffic, with our queue sizes and our timeouts. It may consume resources that are needed for the normal workflow, and in the worst case, it can make the broker client fail, taking our component down with it. You need to be careful when passing any internal state to code you don’t own. Documenting the assumptions of the hooks API is a good first step, but it won’t prevent unexpected usages in real life.
Another problem is performance. The client-provided logic may be blocking, and we have no control over how long a hook takes. Suppose that a hook signs the payload by calling a remote signing service. Every I/O call is unpredictable and may result in high latency. If publishing a message takes 300 ms end-to-end and the signing call takes 1,000 ms, the total time of one call becomes 1,300 ms. That is more than 4 times slower, and with two such hooks, the call takes 2,300 ms. The caller observes the high latency, and with the current design, we can't do anything about it.
The synchronous call inside the hook uses the same thread as our component. In a service with a minimal number of threads, a hook that waits for an external resource keeps one of them busy. If this happens a few times at once, there aren’t enough threads left to process the incoming requests. We could require clients to write nonblocking hooks, but even if we document it, we can’t enforce it. Detecting a blocking call on our thread is possible, but it complicates the design substantially.
The other solution is to assume that the hook code is never safe and may always block. With this assumption, we wrap every hook call into a task executed on a separate thread, as the following code snippet shows.
var tasks = hooks.Select(hook => Task.Run(() =>
{
try
{
hook.OnMessage(message);
}
catch (Exception e)
{
logger.LogWarning(e, "Hook failed, continuing without it");
}
}));
Task.WaitAll(tasks.ToArray());
Compared to the first version of our component, the code got complicated. We need to handle any failure inside the task, since an exception thrown on another thread doesn’t reach our loop and would otherwise be lost. And the blocking is still there, only on a pool thread instead of ours. If enough hooks block at the same time, they exhaust the shared thread pool of the runtime. In practice, the hooks need a dedicated pool with its own thread count and queue size, and someone has to monitor it so that pending hooks don’t pile up and eat the memory.
Also, running the hooks in parallel doesn’t solve all the performance issues. The hooks plug in between two phases of our life cycle, so there is a happens-before relationship between completing all the hooks and proceeding to the next phase. Even if the hooks run in parallel, we need to wait for all to finish. The latency we add is at least as high as the slowest hook. The flexibility that clients get with hooks isn’t free. We pay for it with a thread pool, and we can’t reduce the latency to zero.
The wait is inherent in hooks, because the next phase needs the message they may have modified. But when the client code only needs to be informed, and nothing downstream depends on it, we can drop the wait altogether. That is what listeners are for.
Listeners: notify, don’t wait
At first glance, a listener API looks similar to a hooks API, but there is a difference that makes it worth explaining separately. With hooks, our component waits for all the client code to complete before progressing to the next step. A listener API takes a different approach. Our component lets clients register listeners, and it notifies them when something happens inside it. The notification is fully asynchronous. There is no happens-before relationship between sending an event and progressing to the next phase, so as long as the listeners run on a separate thread, they don’t add latency to our component. The rules for handling failures are the same as for hooks. A listener that throws gets logged and ignored.
The tempting use of a listener API is to expose internal state, or to signal that some event occurred. Suppose we want to let clients know how many retries a publish needed, since so far only the retry metric tells us that. We can send a notification with the retry state when our component finishes executing. For that, we need a RetryStatus type to hold the information we want to propagate, a listener interface with one method, and a way for clients to register listeners.
public record RetryStatus(int RetryNumber);
public interface IRetryListener
{
void OnRetry(IList<RetryStatus> statuses);
}
public class MessagePublisher
{
private readonly List<IRetryListener> retryListeners = new();
public void RegisterRetryListener(IRetryListener listener)
{
retryListeners.Add(listener);
}
// remaining methods
}
For simplicity, the retry status contains only the retry number. Every retry in the loop adds a new RetryStatus to a list, and when the retry logic finishes, the component iterates over every listener and passes it the whole list.
foreach (var listener in retryListeners)
{
Task.Run(() => listener.OnRetry(statuses)); // same try-catch as for hooks
}Even if the logic looks correct, the list passed to OnRetry is a reference to the actual list, and nothing prevents a listener from calling Clear() on it or from adding and removing elements. Suppose that a client registers two listeners. If the first clears the list and the second expects to find one status in it, the second sees an empty list. This is a side effect introduced by the caller’s code, and it makes our API error-prone and nondeterministic. Which listener runs first depends on the registration order and on the thread pool, so the same publish can give different results on different runs.
You can prevent this by giving each listener a copy of the list. Even if a listener modifies its copy, it won’t impact any other listener or our own code. This has two drawbacks. If we have n listeners, we copy the data n times, and the memory consumption grows accordingly. And the problem in the listener’s code stays silent. The modification lands on a copy; nothing breaks, and nothing works either, so the client may never find out.
Another way you can prevent this is to wrap the list into a read-only view before passing it.
Task.Run(() => listener.OnRetry(statuses.AsReadOnly()));The view doesn’t copy the list's contents. It throws an exception on any attempt to modify the underlying list, while every read still goes to the original data. Besides being cheaper, this approach is explicit and fail-fast. If the listener code modifies the list by accident, the feedback is immediate, and there is no risk of failing silently.
If you propagate any state to code you don’t own, you should always make sure that the state is immutable. The cleanest way is to design it immutable from the beginning, with read-only fields and records, since an immutable object needs no defensive copy at all. In reality, most of the collections we use are mutable, so the safe choice is to wrap them into a read-only class that forbids modifying the underlying data.
The other problem with listeners is the traffic they receive. If we invoke n listeners every time something happens, and the caller’s code can’t keep up, the pending notifications pile up and block, impacting the main processing of our application. Adding backpressure, or buffering the events and sending them in batches, solves this but complicates the design further.
Even a simple use case, propagating the number of retries, makes our code more complex. We need to make sure that the data is immutable, and we need to make sure that the caller’s code can keep up with the traffic. Foreseeing a feature that looks simple at first sight adds a lot of complexity to our code.
Picking your spot on the axis
The fundamental point to grasp from these examples is that every new feature entails, at some level, an increase in complexity. Abstracting away the metrics library decreased our component's complexity, but increased it in the code of every client that had to implement the interface. In most cases, the best solution proved to be a hybrid approach: providing an abstraction while including a default implementation that meets the needs of the majority of users.
At first glance, hooks and listeners seemed even more compelling, as they are flexible and require only a few lines of additional code. But we paid for the extensibility. With hooks, it became necessary to handle potential exceptions, offload client code to a thread pool, and account for a latency at least equal to that of the slowest hook. With listeners, the waiting went away, but we had to make the propagated state immutable and worry about clients that can’t keep up.
The two families of solutions sit at different points on the same axis. An abstraction offers flexibility while maintaining an explicit contract that clients are required to follow. Hooks and listeners are far more flexible, but they expose internal events and state, and the client can do anything with them. We can no longer reason about the client’s code, its failures, or its concurrency model. Sometimes that much flexibility is more trouble than it is worth, and a smaller, well-defined extension point is the better design.
The same rule applies to any other pattern, such as decorators, factories, or proxies. When a solution gives you a lot of flexibility, look at the complexity that comes with it before adopting it. Complexity doesn’t disappear. The design decision is who pays for it.