Synchronization
An intro to processes and thread synchronization. Why it is critical and how to do it: hardware-based solutions, mutexes, and semaphores.
Hi Friends,
Welcome to the 96th issue of the Polymathic Engineer.
Processes and thread synchronization are two of the most challenging topics in programming, not only for beginners but also for experienced developers.
This week, we start a deep dive into such topics to help you build a strong knowledge about the most fundamental concepts.
The outline will be as follows:
What is synchronization
The producer-consumer problem
Critical sections
Hardware-based solutions
Mutexes
Semaphores
Binary Mutexes vs Semaphores
Synchronization
Synchronization is an important concept in programming that has a broader meaning with respect to how we intend it in everyday life.
In real life, we usually think of synchronization as making two things happen at the same time. But in computer systems, synchronization means more than that. Here, it refers to any possible relationship among events in terms of time.
Almost all the synchronization constraints you encounter are requirements about the order of events. To understand such constraints, you need a model of how computer programs run.
In the simplest model, computers execute instructions sequentially, one after another.
In this case, synchronization is easy because the structure of the program lets us see the order of events. But things are often more complicated.
First, computers can have more than one processor running at the same time, which makes it difficult to tell whether a statement on one processor runs before a statement on another.
Second, a single processor can run more than one thread of execution. Since the operating system's scheduler decides when each thread runs, it's hard to say the order in which they run.
The challenges with synchronization are similar for both parallel and multithreaded models. We can determine the order of execution within a single processor or thread, but between processors or threads, it's impossible to ensure the order of operations without using specific synchronization methods.
In the following sections, we’ll look at different synchronization problems and the techniques used to solve them, so that you can understand which tools you to tackle complex synchronization challenges.
The producer-consumer problem
Let's start with a classic example of a synchronization problem like the producer-consumer.
In this scenario, some threads create items (the producers), and others process those items (the consumers). Producers put items in a shared buffer where consumers can retrieve them.
There are several synchronization constraints that we need to enforce to make this system work correctly.
While an item is being added to or removed from the buffer, the buffer is in an inconsistent state. Therefore, threads must have exclusive access to the buffer.
Moreover, if a consumer thread arrives while the buffer is empty or a producer thread arrives when the buffer is full, it must block until a producer adds a new item or a consumer removes an old item.
To make things easier to understand, let's look at a simple version that uses an integer variable as a counter to keep track of how many items are in the buffer. Producers increase the counter after adding an item, and producers decrease it after removing an item.
Suppose a producer and a consumer simultaneously try to update the counter. In that case, we might end up with an incorrect count that could lead to overwritten data or trying to consume non-existent items.