Remote procedures

An introduction to Remote Procedure Calls (RPCs) and gRPC. Plus a step by step guide to setup gRPC.

In distributed systems, Remote Procedure Calls (RPCs) are the primary way to make inter-machine communication possible. These can be different backend microservices or a mobile application (client) interacting with the backend (server).

In a nutshell, RPC is a protocol that one application uses to request a service from an application located in another computer on a network without understanding network details.

RPCs can be implemented using various methods:

• RPC frameworks

• WAMP protocol over WebSockets

• asynchronous message enqueuing with RPC instructions

• custom protocols

Despite this, the term RPC is mostly used to refer to RPC frameworks like Thrift, gRPC, JSON-RPC, etc. These frameworks make remote communication as easy as making a local function call.

gRPC

gRPC is a framework for Remote Procedure Calls initially developed by Google and then open-sourced in 2015. Since then has become prevalent outside of Google, becoming the default way of implementing RPCs in an application. Two main features characterize gRPC:

1. It's built on HTTP/2, allowing data multiplexing and streaming; so clients and servers can send and receive data simultaneously

2. It uses Google's Protobuf (Protocol Buffers) to serialize data; so different services can store and transfer data in a language/platform-neutral way

The first step to using gRPC is defining the data schema with protobuf. In the schema, you must specify the remote messages (services), the data to transfer, their types (uint32, int64, string, arrays, and more), and identifiers there.

Now, whenever you want to use gRPC on any of the supported languages, you can give that proto file to the client/server library and have access to all the services and types. This way, accepting and responding to the RPCs becomes as easy as handling local function calls.

gRPC advantages

There are reasons why people prefer to use frameworks like gRPC over REST APIs.

1. Scalability. RESTful services aren't designed to be highly scalable. They use human-readable text formats like JSON, which is inefficient when sending thousands of messages per second. Instead, gRPC uses a binary message format like Protobuf, which is much more efficient when dealing with a high volume of messages.

2. Strongly Types. Working with REST APIs is more probable to have compatibility issues and runtime errors caused by the lack of enforced types. gRPC uses Protobuf, which forces you to define the transferred data's shape (structure) and style. The usage of a schema reduces the chance of surprises.

3. Bidirectional communication. gRPC supports not only the typical request-response pattern but also bidirectional streaming. This means you can have two-way communication between your applications so that both can talk and listen simultaneously. This feature is beneficial for communication between backend microservices, which need to exchange data dynamically and interactively.

4. Features. gRPC provides valuable features that can save time and effort, like load balancing, service discovery, and timeouts.

gRPC disadvantages

While gRPC offers several powerful features, it also has its limitations:

1. gRPC requires defining a schema. While this can be a plus for ensuring data consistency, it could also be a drawback if your application doesn't need strict rules or needs more flexibility.

2. Right now, calling a gRPC service from a web browser is impossible. To get around this, you must use a gRPC-web library, which acts as the adapter.

3. gRPC is popular, but its community is still much smaller than REST APIs. You may have fewer resources or people to help with your specific needs, especially if you are not using one of the supported programming languages.

Comparison with Thrift

Apache Thrift is a technology developed by Facebook in 2007 that can be considered a competitor of gRPC as an RPC framework. However, gRPC is often favored over Apache Thrift these days.

This preference is mainly because gRPC is built on HTTP/2, and it provides better support for streaming data, which means it's excellent for sending data continuously in both directions between client and server.

Also, gRPC has a more active and vibrant community of developers, which means more frequent updates, more resources for learning, and faster solutions to problems.

For example, Dropbox and Reddit used Apache Thrift but then decided to switch to gRPC through a gradual migration.

How to setup gRPC

Let’s consider a server hosting a calculator application and show how a client can use gRPC to communicate with the server. In the whole example I will store the files in two folders: a protobuf folder and a calculator folder.

The first step is to define a protobuf file in the protobuf folder. The file specifies two remote methods to add and subtract integers. The methods receive an AddRequest and a SubRequest messages and return AddResponse and SubResponse messages.

The second step is to generate some boilerplate code in the calculator folder that interact with the protobuf file. To this purpose, it’s necessary to install install grpcio-tools:

1. Create a file requirements.txt file with the following line: grpcio-tools ~= 1.30

2. Install the dependencies into a virtual environment with the following commands

   

3. Generate Python code from the protobuf file, with the following command

   

The above procedure generates two Python files: calculator_pb2.py and calculator_pb2_grpc.py. These files include types and functions to interact with the API both client and server side.

The third step is to write the code of the server. The server implement the add and sub functions mentioned in the protobuf file with code that looks like they are local function calls. The gRPC framework handles all the underlying logic.

The fourth step is to run the server opening a new terminal and run the command python calculator.py

The last step is to test the client on some data opening another terminal and creating a channel stub with the commands

Finally it is possible to make requests and get answers from the Calculator server using gRPC.

References

• gRPC website

• Python microservices with gRPC

Interesting Tweets

A manager who is personally monitoring and assigning tasks to team members is doing micro-management. I would never have a manager like this. Link

I always find room for improvements when reviewing my code in the browser before submitting a PR. It's incredible how a simple switch from the IDE to the browser changes the level of my attention.

Since I joined my current company 3 years ago, I started working in a less structured way without daily standups and regular meetings. I have to say that it worked pretty well and I’ve never been so productive.