Developing the Guidelines for Migration from Restful Microservices to Grpc

Masaryk University Faculty of Informatics

Developing the guidelines for migration from RESTful microservices to gRPC

Master’s Thesis

Michal Štefanič

Brno, Spring 2021 Masaryk University Faculty of Informatics

Developing the guidelines for migration from RESTful microservices to gRPC

Master’s Thesis

Michal Štefanič

Brno, Spring 2021 This is where a copy of the official signed thesis assignment and a copy ofthe Statement of an Author is located in the printed version of the document. Declaration

Hereby I declare that this paper is my original authorial work, which I have worked out on my own. All sources, references, and literature used or excerpted during elaboration of this work are properly cited and listed in complete reference to the due source.

Michal Štefanič

Advisor: RNDr. Josef Spurný

i Acknowledgements

I would like to thank my advisors, RNDr. Josef Spurný for his time and valuable consultations, and doc. Mouzhi Ge, Ph.D. who left the university, but still found a time for thesis consultations. I would also like to thank the colleagues from Kiwi.com, especially Matouš Dziviak with whom I consulted a lot of technical questions related to the thesis. And Tomáš Gerich for proofreading the thesis.

ii Abstract

With the rising popularity of microservice architecture, Google decided to revive the "Remote procedure call" (RPC) concept to facili- tate effective communication between single services and developed their own framework (gRPC) for RPC service communication. gRPC became quickly popular and many famous companies as Netflix, Mi- crosoft, Slack integrated it into their systems. However, there is not a clear, step-by-step guide on when and how to migrate your services to gRPC architecture. Therefore, the goal of this thesis is to explain differences between RESTful and gRPC concepts, detect when itis effective to migrate to gRPC, and describe migration procedure in single steps without affecting business logic. For validation of the proposed guidelines, the thesis applies the gRPC architecture to one well-known online travel agency Kiwi.com, by migrating some of their microservices to gRPC with the proposed guidelines.

iii Keywords gRPC, microservices, REST, API, migration, guidelines, ...

iv Contents

1 Introduction 1

Introduction 1

2 Background 4 2.1 Microservices ...... 4 2.1.1 Monolith vs microservices ...... 4 2.1.2 Core microservices components ...... 6 2.2 Containerization ...... 7 2.2.1 Docker ...... 8 2.2.2 Kubernetes ...... 10 2.3 HTTP ...... 10 2.3.1 HTTP vs HTTPs ...... 11 2.3.2 HTTP/1.1 vs HTTP/2 ...... 12 2.4 REST API ...... 13 2.4.1 Usage of REST APIs ...... 13 2.4.2 Stateless character of the REST architecture . . . 14 2.4.3 The client-server architecture ...... 14 2.4.4 REST vs SOAP ...... 15 2.4.5 REST API challenges ...... 17 2.5 gRPC as RPC framework ...... 19 2.5.1 Remote Procedure call ...... 19 2.5.2 gRPC ...... 20 2.5.3 Protocol buffers ...... 21 2.5.4 gRPC communication ...... 23 2.5.5 RPC types in gRPC ...... 23 2.6 Technical migrations ...... 24 2.6.1 Migration from the monolith to the microservices 25 2.6.2 Migration from SOAP to REST ...... 26 2.6.3 Key concepts of these migrations ...... 28

3 Decision process and migration guidelines 30 3.1 Decision process ...... 30 3.1.1 Benefits of gRPC ...... 30 3.1.2 Downsides of the gRPC ...... 34 3.1.3 gRPC vs REST ...... 37

v 3.1.4 Optimal architecture ...... 39 3.1.5 When to migrate ...... 41 3.2 How to migrate ...... 44 3.2.1 Key concepts ...... 44 3.2.2 Protocol buffers management ...... 47 3.2.3 Protocol buffers best practices ...... 49 3.2.4 HTTP/JSON gateway ...... 52 3.2.5 Define API endpoints ...... 53 3.2.6 Interceptors ...... 54 3.2.7 Error handling ...... 54 3.2.8 Authentication ...... 55 3.2.9 Testing ...... 55 3.2.10 Deadlines and retries ...... 56 3.2.11 Load balancing ...... 56 3.2.12 Monitoring ...... 59 3.2.13 Summary ...... 59

4 Case study in Kiwi.com 60 4.1 Kiwi.com ...... 60 4.2 Architecture ...... 60 4.3 Decision process ...... 61 4.3.1 Optimal architecture ...... 61 4.3.2 API contract ...... 62 4.3.3 Performance boost ...... 62 4.3.4 Drawbacks ...... 63 4.3.5 Decision ...... 63 4.4 Migration process ...... 64 4.4.1 Proto buffers structure ...... 64 4.4.2 Proto buffers linter ...... 66 4.4.3 gRPC gateway ...... 66 4.4.4 Error handling ...... 67 4.4.5 Authentication ...... 67 4.4.6 Load balancing ...... 67 4.4.7 Monitoring ...... 68 4.5 Experience report ...... 68 4.5.1 What went well ...... 69 4.5.2 What went wrong ...... 70 4.5.3 What is next ...... 70

vi 5 Conclusion 72

Bibliography 74

vii List of Tables

2.1 Table of references linked to concepts they describe. 29 3.1 gRPC and REST comparison 38

viii List of Figures

2.1 Monolithic vs microservices architecture. 5 2.2 Docker components. 9 2.3 HTTP vs HTTPs 11 2.4 REST vs SOAP. 16 2.5 RPC mechanism. 19 2.6 Simple gRPC message defined in proto language 21 2.7 Simple gRPC service defined in proto language 22 2.8 Steps for migration from SOAP to REST. 27 2.9 Mapping between SOAP-based operations and resources to HTTP methods. 28 3.1 Microservices architecture with gRPC. 40 3.2 Enum example in proto language 50 3.3 Deprecation of the field in proto language 50 3.4 grpc-gateway demonstration. 52 3.5 grpc-gateway proto example 53 3.6 Look-aside load balancing. 58 4.1 Proto repository structure. 65 4.2 gRPC metrics of one anonymized service 68 4.3 gRPC(blue) vs HTTP(red) execution time comparison 69

ix 1 Introduction

Microservices architecture pattern is becoming one of the leading architectural styles for modern applications, and there is a fair as- sumption it will be the first choice for the design of many applications in upcoming years. There are several reasons for it, such as support for agile development, the architecture that allows businesses to develop and deploy new services and features faster, in a small iteration with short lead time, but at the same time ensure high system’s scalability, reliability, and security. These advantages are being more and more important especially in the upcoming post-covid engineering world, when location-agnostic remote work and team’s decentralization are being accelerated.

The best approach for running microservices architecture is the containerization of applications. Containers encapsulate the whole application together with its lightweight runtime environment and present it as a consistent software environment that allows you to run the application consistently everywhere - from the developer’s machine, through the testing environment, to production deployment. Containers can run on on-premise physical machines and at the same time virtual cloud machines, while using their full potential without wasting resources, which give us both - economic benefits for companies and environmental and ecology benefits for the world.

While containerized microservices are becoming a very popular architectural pattern, the standard of how single services communicate and share information with each other did not change much and is usually handled with REST APIs over HTTP/JSON messages. Despite the fact that REST endpoints are a perfect solution for communication and data transfer, there are several issues with them. One of the main problems is that the REST does not have strictly required standards. Standardization is an optional, defined via best practices agreed bythe community, but at the same time, implementation and maintenance of REST API endpoints is usually non-trivial task. In practice, many APIs that are referred to as REST, do not meet RESTful standards. And even when API follows RESTful standards at the beginning, they often being

1 1. Introduction

abandoned after some time of production iterative development. There is another alternative way how to handle communication between services.

Alternative for the REST and HTTP/JSON communication are Re- mote Procedure Calls (RPC). RPC is not a new mechanism and was described many years ago, but the previous RPC solutions as SOAP or CORBA disappeared from modern application and were replaced by HTTP/JSON. However, in 2005 Google introduced open-source RPC framework called gRPC [1] that resolves most of the issues of its predecessors. gRPC provides high-performance RPC solution that can run in a variety of environments and is supported by the majority of popular programming languages. gRPC uses protocol buffers for data and RPC service definition. Messages are transferred over HTTP/2 protocol only. It secures faster data serialization, more strict API contract, and overall performance boost while maintaining steady and persistent connections between services. In addition, gRPC supports multiple types of streaming mechanisms.

The goal of this thesis is to address the downsides of REST APIs, introduce the gRPC technology, and compare the advantages and disadvantages of both options for microservices communication. The next goal is to define a decision process that helps engineers to decide when it is efficient to think about gRPC as an alternative to REST and propose high-level guidelines on how to migrate REST microservices to gRPC. The proposed decision process and guidelines will be used for a case study to integrate gRPC in a popular online travel agency based in Brno, called Kiwi.com.

The thesis begins with a chapter describing all necessary methodol- ogy and theory needed for understanding a given topic. It describes concepts of microservices and containerization and why it become so popular, goes over REST and problems of REST development, introduces gRPC and its basic concepts, and mentions two examples of technical migrations. The next chapter goes over the advantages and disadvantages of gRPC and compares it with REST, introduces a decision process, and proposes high-level guidelines on how to migrate

2 1. Introduction services into gRPC. The last chapter describes how Kiwi.com applied these methodologies in decision process to start adopting gRPC, following by the migration of first services into gRPC. The last chapter contains also the results of the migration and developers’ experience report.

3 2 Background

This chapter describes technologies, terms and principles necessary for understanding of the thesis. First, it introduces us to microservices, describes differences between monolithic and microservices architectures and explains the term containerization and the technologies such as Docker and Kubernetes. Then the chapter goes over HTTP and REST principles, a quick introduction into gRPC world and the last part contains two examples of technical migrations as an illustration of some core migration concepts.

2.1 Microservices

Over the last few years, the term microservices has been growing in popularity and keeps growing every day. What exactly is meant by the term microservices? Dragoni et al. define a microservice as following: "A microservice is a cohesive, independent process interacting via messages" [2, p.2]. They further describe microservices as an architectural style inspired by service-oriented computing, which aims to split software system into smaller, separately maintainable packages. Microservices is a relatively new architectural style and is closely connected with other popular concepts such as the containerisation or the continuous delivery. These patterns define a new software development standards. Many renowned companies like Google or Netflix are using microservices, while significantly contributing to the further development of these technologies.

2.1.1 Monolith vs microservices

Many applications consist of a one single monolithic repository and a deployment. Every line of code, all functionality and configuration are maintained and stored within a single package. A monolithic architecture is characteristic for on-premises software. On-premises software is a software model, which is installed and running on user’s machine (server), utilizing his infrastructure and computing resources [3]. An example of an on-premises software are applications like Microsoft Office or internet browsers. Such kind of a software is considered to

4 2. Background

be more secured than on-demand software. On the other hand, they are more expensive as they may require initial investment in a form of capital (on-premises software usually requires the purchase of a license) and later in form of own computing resources to make it work. The opposite to on-premises software is on-demand software, which runs on its own servers and infrastructure, hidden from user. Many on-demand software applications work with the monolithic architecture as well. However, for on-demand applications it is becoming more and more popular to adopt microservices architectural style. Especially for the cloud-native software, where adopting microservices architecture unlocks the true power of the cloud computing with all its advantages [4]. Microservices architectural style can be applied not only in the development of a new software, but also for already existing monolithic applications, by migrating monolithic architecture to microservices.

Figure 2.1: Monolithic vs microservices architecture. Source: https://www.mccourier.com/microservices-architec ture-market-indoor-applications-projected-to-be-the-most- attractive-segment-during-2020-202

While thinking when and how to use microservices architecture, it is important to think about a system as a stack of different func- tionalities it provides. This is achieved by diving a system into logical blocks, where a single block is handling common functionality and

5 2. Background

subsequently putting it into a diagram. In a case where it is possible to create such a diagram, it makes sense to start thinking about adapting microservices architecture [5]. The approach with a decomposition diagram can be applied not only for existing monolithic system, but also while designing a new system.

Microservices architecture model can be compared to an anthill. There are thousands of ants working on different tasks. Some of them gather food, others build anthills, and the rest are taking care of their larvae. Multiple ants do the same job, so we can think about it as multiple instances of a single service. They communicate with each other, share food and other resources, live in the same colony. Together they represent workings environment – system. In a similar way, it is possible to describe the application system. Instead of ants, there is a number of smaller services, each responsible for certain functionality. Running on multiple instances and communicating with each other through APIs and sharing some common resources as a database. These services are usually running on the same cluster and together provide a working system that delivers desired value to users. Neither monolithic nor microservices architecture is the ultimate architectural solution for every system. Both have some advantages and disadvantages and it is all dependent on the particular use case in question [6].

2.1.2 Core microservices components

To achieve maximum potential of the microservices architecture, New- man [7] describes important components, common for systems using microservices. Few of them are described below:

1. Configuration Server - is a mechanism for separation of all the application’s configuration from the source code. With such approach, it is much easier to apply any changes in application setup without the necessity to re-deploy whole service stack.

2. Service Discovery - is used altogether with Load balancer. The microservices architecture runs multiple instances of the same service, which are added and deleted dynamically when needed

6 2. Background

(scaling). The service discovery component stores data about all addresses and ports in service registry to provide an available instance for every service.

3. Load Balancer - is a component responsible for the traffic distribution. There are two types of load balancers - the client–side and the service–side. While the client–side is leaving the responsibility of finding available service instance on the client by querying service registry, server–side balancer provides address and a port of available instance to the client automatically, leaving less responsibility on the client side. This component also effectively distributes traffic among multiple instances, which provides a scalable service. In a case where all instances are overloaded, load balancer will start a new instance to decrease the overloading. The load balancer is a crucial component in provision of an highly-scalable effective services.

4. Circuit breaker - in the microservices architecture some tasks need to call multiple services. With an increased amount of different connections, there is a higher risk that error will occur in one of the services. A failure of one the services can cause a cascade effect and lead to a failure of all services. To prevent this, there is a design pattern called circuit breaker, which attempts to mitigate potential loss to the lowest level.

2.2 Containerization

Microservices architecture is often associated with the containerization. The containerization is a process of packing of an application and its configuration files, libraries and frameworks necessary torun the application into an isolated package. This process secures that application can be easily started in different environments and operating systems. It provides a similar level of isolation and security as virtual machines, while keeping it more simple to work with. It is possible to run multiple containers (as isolated processes) on the same machine. The containers consist of their own Linux-based kernel core and simple file systems. This allows to run the applications even on operating systems, which are by default not supported by that

7 2. Background

certain application. For example Redis is officially not supported on Windows OS, but it is possible to run the Redis container on Windows. Therefore, containerization can be described as an encapsulation of an application and its requirements into stand-alone unit [8], which supports easier development, deployment and maintenance of the microservices.

Compared with virtual machines, it is easier to setup and run an application encapsulated into a container. Virtual machines emulate the whole operating system, required by application as a running environment, inside the operating host system. This allocates fixed amount of resources as memory and CPU to run all processes of an operating system, which leads to the waste of resources and computing power. Another benefit of the containerized application is that the whole configuration is already part of the container. It provides comfortable solution of how to run a new instance of an application. All what is needed is to fetch a container source image and run the instance of an application in a new environment with couple of simple commands. Setting up a new virtual machine tends to be time con- suming, and so does setting up a new instance of the same application on every single virtual machine separately [8]. These are the reasons why containerization is perfect solution how to run highly scalable, reliable microservices, while using all resources at maximum.

2.2.1 Docker

Docker [9] is an open-source software platform providing solutions for containerization. Docker is a daemon which provides developers and software engineers a simple way of building, deploying, securing and testing their applications by using containers. Docker allows developers to create their own containerized applications and share them with others developers, which helps to save the development time and accelerate the software deployment [10]. One of the main benefits of Docker technology is its simplicity and amount oftime spared - and developer’s time equals money. Docker consists of three core elements: Docker image, Docker container and Docker registry [11].

8 2. Background

• Docker image is the building stone of every container. It is a package that consists of all requirements such as dependencies, libraries, programs, environment configuration and the application itself, which are needed to create a container. Docker platform hosts a public warehouse called Docker hub, which contains a huge amount of official trusted Docker images. Users can fetch and run containers on their machines for free. • By running an instance of the docker image, all dependencies and libraries are automatically fetched and installed and whole configuration of application is setup altogether. This building process creates a Docker container. Container is an instance of the desired application that is ready to be run on the Docker host. • In addition to the public warehouse called Docker hub, it is possible to have a private warehouse of application images. Both warehouses are called Docker registry. Docker registry can have multiple repositories. The repository holds information about complete version history of a specific Docker image. One Docker image can have multiple versions, which are stored in the Docker repository.

Figure 2.2: Docker components. Source: https://dev.to/pavanbelagatti/getting-started-with- docker-for-developers-3apo

9 2. Background 2.2.2 Kubernetes

In a company with a couple of containers, it is not that hard to manage the deployment, releases, distribution of containers on different servers, and track status of all containers. However, in a case that a company has to manage hundreds of containers and distribute them properly to not waste too many resources, it is a bit more complicated task. Developers at Google (which is a company with a lot of containers) were facing a similar problem. In order to not have to manually track a list of all their containers, the programmers decided to develop an automated way of dealing with this issue. The effort resulted in the development of a an open source platform called Kubernetes. [12]. Kubernetes is a platform for the container orchestration. It inspects the whole infrastructure, analyzes available resources, measures CPU and memory usage for the each machine involved. When a new container is deployed, Kubernetes identifies resources and decides what is the best place for a new container. Kubernetes creates an abstract layer from the data center above the infrastructure, so it is unnecessary to manually decide where to deploy a new container [13]. The main advantage of Kubernetes is money saving. Resources (either hardware or developers) are not cheap, so keeping an effective usage of it can save significant amount of resources.

2.3 HTTP

Hypertext Transfer Protocol (HTTP) [14] is a globally known network protocol which allows data transfer over the internet or other networks. Without the HTTP protocol the WWW 1 as we know it today, would probably not exist. HTTP is the most popular transfer protocol of the Transfer layer (L7) in networking. It defines a way in which data from web pages are transferred further over a network. All web pages and their data are stored on servers and are available to the clients via HTTP protocol. Majority of the content seen on the web is issued over HTTP. It is used to render web pages, accept and process user data

1. World Wide Web commonly known as the Web, is an information system where documents and other web resources are identified by Uniform Resource Locators (URLs) and are accessible over the Internet

10 2. Background

and provide new data from the server to the client. The client is not necessarily a browser, it can be another application.

2.3.1 HTTP vs HTTPs

The HTTP protocol is the plain-text based protocol and its format consists of text/ascii 1 symbols. Therefore it is easily readable by a human user . To be more specific, HTTP/1.1 is the plaint-text protocol (as there is also the HTTP/2 described below). The data transferred over HTTP (unsecured) connection are not encrypted, therefore there is a risk of data being stolen by third-party attackers. As data are not private, all information submitted by the user, such as credit card information, photos or other sensitive information, could be leaked.

Figure 2.3: HTTP vs HTTPs Source: https://www.semrush.com/blog/what-is-https

Hypertext Transfer Protocol Secure (HTTPs) is a solution for pri- vacy issue described above [15]. Before data are shared, a connection is first secured by a third-party vendor to provide a transparent data transmission. The most popular way to achieve this, is by using SSL

1. abbreviated from American Standard Code for Information Interchange, is a character encoding standard for electronic communication

11 2. Background

certificate1 or the TLS2. The SSL certificate verifies the identity of the both parties and provides encrypted connection with additional level of security for sensitive data transfer. It is especially important in the sectors of internet banking or trading brokers. TLS also provides a secured connection with cryptographic techniques for authentication, which ensures that data comes from a reliable source and are not modified in the middle of the process by a third-party services. The communication is initiated with the TLS handshake - an authentication with keys.

2.3.2 HTTP/1.1 vs HTTP/2

HTTP/2 is a newer version of its predecessor HTTP/1.1, and it has introduced several improvements [16]. It was published as RFC 7540 on May 14, 2015. The previous section mentioned that HTTP/1.1 is text-based protocol. The biggest difference between the older and the newer version is that the HTTP/2 is a binary-based protocol, which makes it harder to see what is going on with it at the protocol level. Thanks to the binary coding, the framing is much easier. In the HTTP/1.1 it is a bit difficult to indicate the start and the end ofthe frames. Without mandatory white spaces included in the HTTP/1.1, implementation is simple and it is easier to recognize all protocol parts. Another benefit of the HTTP/2 is a reduced number of network round-trips, the possibility to use a large amount of parallel streams for communication or prioritization of the stream flow, which allows transfer of the most crucial data first. All combined, provides faster page loading and more secure data transfer. According to W3techs [17], as of January 2021, 49.9% of all websites use the HTTP/2 and the majority of browsers 98% supports the HTTP/2. It took 5 years till half of the internet adopted the HTTP/2 protocol. In the meantime, the HTTP/3 is already on the way with more improvements.

1. acronym for Secure Sockets Layer, is the internet standard security technology used to establish an encrypted link between a web server (website) and a client 2. Transport Layer Security, the successor of the now deprecated SSL, is a cryptographic protocol designed to provide communications security over a computer network

12 2. Background 2.4 REST API

"REST" stands for the Representative State Transfer, which indicates that it has something to do with the data transfer and its representation. REST is a well known architectural style for an application programming interface (API), operating around a classic request-response approach. It uses the HTTP protocol to access or modify resources with HTTP methods such as GET, POST, PUT or PATCH and DELETE. The REST API architectural style is used especially for the Web services development and communication on the internet. A lot of applications adopted the REST as an alternative to the XML-based Simple Object Access Protocol (SOAP) communication protocol.

2.4.1 Usage of REST APIs

Every large tech company like Google, Facebook, or Amazon uses REST APIs in their services in some capacity. REST has become a very popular solution for the cloud computing, providing a way to expose services and communicate with clients. The majority of the popular programming languages such as Go, Python, JavaScript allow developers to build REST APIs. The objects and components exposed and modified via REST APIs are called the resources. The resources can be retrieved and modified with the existing HTTP methods [18]. The following are the most common methods used in REST:

• POST to create a new resource

• GET to retrieve an existing resource

• PUT to update an existing resource

• DELETE to remove an existing resource

There are couple more methods used by the REST APIs, such as HEAD, OPTION or PATCH, but they are not used as frequently as the four methods described above. REST supports multiple data formats

13 2. Background

for resource representation (JSON1, XML2, form-data). However, the most common one is JSON. The main reason for its popularity is its simplicity and light weight structure. Compared to the XML format, the JSON is easier to read and understand, as it does not contain nu- merous constructors and repeatable components. As the data payload is much smaller, it uses less bandwidth, making the communication over REST APIs faster and more effective as e.g. over the SOAP, which is based exclusively on the XML format [19].

2.4.2 Stateless character of the REST architecture

REST API operations are stateless. The state of the resource at any point in time is its representation. A resource cannot be modified during the execution of a particular operation by any other concurrent operation from a different session. All server operations are stateless, therefore any necessary state management should be done on the client’s side. In this way the server is always considered stateless. Once the server has returned a resource to the client, it does not care about any state of that resource on the client side anymore [20]. The stateless services are used on a daily basis all over the internet. When user opens a Facebook application on his phone and loads recent messages, an application (client) loads messages from the server and the received response becomes completely independent from any state on the server side. The whole response is now cached on the client side and there is no active connection between the client and the server to exchange the state of the resource anymore.

2.4.3 The client-server architecture

The RESTful architecture is client-server based, which means it should always be clear who is the provider of the service and who is using the service. Server is responsible for the data accessibility, load balancing, security, and other restrictions. The client can access and modify re-

1. JavaScript Object Notation is an open standard file format and data interchange format that uses human-readable text to store and transmit data objects consisting of attribute–value pairs, arrays and other serializable values 2. Extensible Markup Language is a markup language that defines a set of rules for encoding documents in a format that is both human-readable and machine-readable

14 2. Background

sources through the unique URL1 + a certain HTTP protocol method. The URL can be the same for multiple operations. For the REST APIs it is characteristic to have the same URL path for GET, PUT, DELETE operations of a certain resource, while expecting some unique resource ID as a query parameter to identify the desired resource instance. This indicates the possibility to pass input parameters as a part of the URL path, which can complicate the understanding of the resource spaces, especially in poorly designed APIs. Therefore, API endpoints design should follow the common Web service standards and preserve con- sistency across the whole service to make it easier for the clients to determinate all data input locations and the overall understanding of the resource space structure [20].

2.4.4 REST vs SOAP

Why SOAP as RPC alternative for data transfer did not survive compe- tition with the REST? SOAP uses Web Service Description Language (WSDL) to describe Web services interfaces, messages, operations, and endpoints. WSDL is based on XML format. A SOAP XML message, which is equivalent to a REST API payload, consists of an envelope object which contains two parts - a header and a body. The header includes application-specific the context information and is an optional part of the message, while the body is mandatory and contains information intended for the message recipient. [21]. While the REST API is an architectural style, the SOAP is a messaging protocol, therefore it may be hard to compare these two methodologies. High-level comparison could be interpreted as following. SOAP is more about restrictions on the message structure, which is always XML-based with strict structure (envelope, header, body), while REST is more focused on the correct usage of HTTP transfer protocol and its features (requests methods, headers, response, status codes, etc) [21].

When using the SOAP, the client has to be familiar with the communication protocol mechanisms involved in the message transfer - such as proper usage of headers, available commands or knowledge

1. Uniform Resource Locator, or web address, is a reference to a web resource that specifies its location on a computer network and a mechanism for retrieving it

15 2. Background

of error codes. SOAP as a messaging protocol is independent of the HTTP protocol, ignoring semantic of HTTP operations like GET, POST, PUT, DELETE. Even if the HTTP protocol is used to transfer the SOAP messages, it typically uses the POST method for every single operation. Before the REST became popular, the SOAP was often used for API integration. Software engineers wrote XML messages and defined Remove Procedure Call (RPC) to execute the specific operation on other machines without the necessity to know the implementation details of the operation. While the XML is the only option for message format in SOAP, there is no strict request format required in REST. It is possible to choose between XML, JSON, or any other format. However, the most popular one is the JSON that is easier to parse and process than the XML.

Figure 2.4: REST vs SOAP. Source: http://www.differencebetween.net/technology/interne t/difference-between-rest-and-soap

To make manipulation with resources more intuitive and straightforward for the clients, the REST defines a set of design constraints. The correct choice of the HTTP method, well designed URLs, self- descriptive messages, and usage of unique identification of the resource should make services scalable, reliable, and reusable. Figure

16 2. Background

2.4 summarizes main differences between the REST and the SOAP, and shows why the SOAP as RPC protocol did not survive.

2.4.5 REST API challenges

There is no doubt that the REST API architecture is a great way of how to develop and expose APIs. However, the REST also has a couple of developing challenges. Designing a good REST API is not a trivial task. A many APIs that are called a REST APIs, in fact do not follow the REST standards.

• One of the most common mistakes is the wrong usage of the GET and the POST methods. Even though the GET method allows to include payload body in a request, it is a common practice to not use it. The GET method should be idempotent, which means that it can repeatedly send the same request, but the response should remain the same. The GET method is designed to retrieve resources, not to create new ones or modify the excising resource. The worst scenario is when the API itself is designed to expect the GET payload [18]. Similar to the wrong interpretation of the GET method, it is a frequent mistake to use the POST endpoint to retrieve a resource. Despite the fact that this is not as critical as the previous misusing of GET, using POST for GET purposes is more typical for the SOAP architecture. Imagine there is a requirement to implement a new endpoint to create a certain resource, but the combination of POST method + URL is already taken for the GET purpose.

• Proper error handling is an another critical issue of the REST APIs. Many APIs use 200 status code basically for everything. No matter whether the request was really successful or there was some validation error or a connection issue. It is strongly recommended to adopt proper usage of the HTTP status codes. The majority of developers are familiar with these status codes and know how to work with them. When they receive 500 status code, they expect some technical issues on the server-side (problem with an internet provider, servers host, etc.). On the

17 2. Background

other hand, if the response returned an error message + 200 status code, developers are forced to parse the body and figure out what exactly is wrong. It is a good practice to introduce custom-made error messages, appropriate specifically for our API [20]. • Issues could also appear due to a lack of documentation, wrong validation, or dynamic format of the API responses. When it is unknown what to expect in the API response, every known API parameter needs to be considered as being optional. The API response should be consistent, defined in a clear format, and with proper typing. Quality documentation is also desirable in order to make the integration of the API easier for the clients. All of these can be achieved with tools like the OpenAPI and the Swagger. A response should be as lightweight as possible. In a case where a bigger load of data is requested, using of pagination and sorting allows clients to fetch the data gradually. Too much of data in a response requires a lot of computation power, data fetching, and response building, which slows the API. • Many API clients ignore the API caching, despite it might significantly speed up the response time. Especially when the payload is bigger and caching will save a lot of unnecessary bandwidth, as the payload message itself is not included in the response once it was cached (and in case it has not been changed in the meantime). The client receives just a flag saying the resource is unchanged, using a cached response. • Usage of wrong and long parameter names makes the URI unappealing and hard to read. There is no need to provide a long precise parameter name as order_item_name. Ideally, the description of a resource is a part of the URI path e.g. "/api/order/{id}/item/{name}", so there is no need to explic- itly describe it in parameter again. gRPC technology described below solves a some of these REST API challenges. gRPC does not use HTTP methods, allows streaming to avoid huge payloads or pagination, and forces the strong API contract and documentation.

18 2. Background 2.5 gRPC as RPC framework

This section describes concepts of the RPC and one of the currently most popular RPC framework – the gRPC.

2.5.1 Remote Procedure call

Thurlow describes the RPC in his RFC [22, p.4] as following: "RPC consists of two processes, the caller’s process and a server’s process. The caller first sends a call message to the server process and waits (blocks) for a reply message. The call message includes the procedure’s parameters, and the reply message includes the procedure’s results. Once the reply message is received, the results of the procedure are extracted, and the caller’s execution is resumed."

Figure 2.5: RPC mechanism. Source: https://www.geeksforgeeks.org/remote-procedure- call-rpc-in-operating-system

19 2. Background

Process on the server side is dormant and awaits the arrival of a caller message. Once it arrives, the process on the server side starts, procedure’s parameters are extracted and the process computes the result, sending response message to the caller and then waits for the next call. Figure 2.5 illustrates an implementation of the RPC process. The client application invokes the RPC method provided by the server in the client stub, with all the necessary parameters. Stub parses the parameters and packs the method name and parameters into the request message and sends it to the run-time library. The run-time library handles networking and sends the client’s message to the server, where the server’s run-time library accepts it and reads it. The message is unwrapped by the server stub and send to the server application for processing with all input parameters from the request. The server processes a request and prepares a response message, which is sent back to the client in the same way as the request message has arrived.

Idea of the Remote Procedure call (RPC) has been around for many years, ever since the in-memory object messaging had started to be replaced by distributed cross-network computing and object messaging, using in object oriented applications [23]. Before the rise of the REST architecture, the RPC concept was often used for Web service architecture. SOAP, WS-Addressing, WS-ReliableMessaging are all RPC-based styles used in the Web service development. The SOAP protocol is a typical example of the RPC-style architecture, where clients communicate with the server over XML-based messages using typically the HTTP protocol for a data transfer. One of the most popular RPC frameworks nowadays is the gRPC.

2.5.2 gRPC

gRPC [1] is a relatively new open source RPC framework, whose core functionality has been developed by the developers from Google and can be run in any environment. The gRPC provides a high performance and secured RPC communication between services, as well as various types of data streaming. The high performance and transfer speed is secured mainly by the protocol buffers (protobuf) and usage of the HTTP/2 protocol for data transfer. The gRPC is often

20 2. Background

considered to be another generation of RPC-based technologies, such as SOAP, WSDL or CORBA. It shares some common concepts, and may look familiar. Unlike the REST, all of these technologies are not an architectural style with a lot of design best practices, guidelines and principles on how to build services. The gRPC, like the SOAP or WSDL, is more like a technical specification which provides a way of calling the remote procedure on a different machine, while the client and the server do not have to be written using the same language or run in the same environment. The gRPC does not follow REST standards. Therefore, the documentation and knowledge of procedures provided by the server is important. gRPC uses a special format for request and response messages called the protocol buffers.

2.5.3 Protocol buffers

Protocol buffer or protobuffer is an open source mechanism developed by Google, used for serializing structured data. The gRPC uses the protobuffers as a default choice, but it also supports other data formats as JSON. All data structures used for request and response messages, as well as RPC procedures provided by the gRPC server must have its definition written in proto files. Proto files are a text files witha .proto extension and are a backbone of every gRPC service. Language used for the proto file definition is either proto2 or never version proto3 [24]. Below is an example of a simple message:

Figure 2.6: Simple gRPC message defined in proto language

The message is a logical record containing all the information of a data object in form of name-value pairs, which are called fields [25]. In an example above, there is a message called Student with three fields: name, id and is_active. Message defines the name of the fields, as well

21 2. Background

as the type (in Student message every field has different data type) and also the ID of the field. This ID is a unique number which is used for field identification in a binary format. Once a message was usedin production, this number should not be changed. The allowed numbers which can be used as ID are between 1 and 229 − 1 (536,870,911).

Once all messages in proto files are defined, the protocol buffer compiler called protoc generates data access classes and procedures in the desired programming languages. The classic getter and setter methods are generated for each field, as well as functions to de/serialize messages from/to raw bytes. As gRPC is RPC-like system, messages are bound to remote procedures as input and output objects. Here is a definition of simple remote procedure using Student message as the response object.

Figure 2.7: Simple gRPC service defined in proto language

This example defines a new service called SimpleService, which has one procedure GetStudent, accepting student ID as an input parameter and returning Student as the response message.

22 2. Background 2.5.4 gRPC communication

The gRPC protocol buffer compiler generates all the necessary code not only on server-side, but also for client-side. Server implements the procedures, declared by the input and output parameters defined by messages in the proto file, which are remotely called by the clients. The gRPC server decodes incoming calls and executes the remote procedure - code on server side. After processing is done, the server encodes a response message and sends it to the client.

The gRPC clients need local files (which are called stubs) generated by the gRPC server to open the RPC communication. Stub provides a programming language-specific interface, which adds procedures provided by the server into the code. These local files allow calling remote procedures directly from the code while looking like just another local functions. Everything that the clients need to do is to prepare input parameters for the remote call. Stub encodes parameters from language-specific objects into the corresponding protobuf message. The server receives the message, processes the request, and sends back a response in form of protocol buffer messages, which are then decoded by the client’s stub into appropriate language-specific object [26].

2.5.5 RPC types in gRPC

The gRPC supports 4 different types of RPC communication - unary RPC, server streaming RPC, client streaming RPC and bidirectional streaming RPC.

• Unary RPC is the simplest one. The client sends one request and receives back a single response. The communication between the client and the server starts when the client calls a stub method. The server is notified about the client’s desire to establish communication, by receiving the client’s metadata, name of the method, and optionally a deadline. The server can either send its own specific metadata before processing begins or await till the client sends a request message. The flow is application-specific and defined in advance. After the client’s request message was received by the server, the appropriate

23 2. Background

server code is executed and if processing was successful, the response message and status code (status message) is sent to the client. When the client receives a message, unary RPC call is completed. • Server streaming RPC is the second type of the RPC, which is similar to the unary one. The main difference is that instead ofa one response message, a server is streaming multiple response messages to the client. When all the messages are processed, the server sends the status code and final message (optionally with metadata) to the client. This ends processing of the RPC request on the server side. The client considers the request as completed once it has received all messages from the server. • Client streaming RPC is very similar to the server streaming RPC. In this scenario, the client is streaming multiple request messages to the server, while the server responds with a single message and specific status code – usually once it has received and processed all messages. • Bidirectional streaming RPC is the last type of RPC streaming, which combines the last two streaming mechanisms. The client initializes the streaming and the server starts accepting the client’s messages. The server processes the requests and can start streaming response messages back to the client. Order of a request and response messages is application-specific. The server can either wait until all client’s messages have been received before streaming back response messages, or the client and the server could process messages gradually - the client sends a message, the server processes it and replies.

More details about the gRPC and its advantages, disadvantages, and when to use this technology are described in the next chapter.

2.6 Technical migrations

This section contains examples of two technical migrations, that high- lights general migration strategies, best practices, and the overall ra- tionale behind the migration process.

24 2. Background 2.6.1 Migration from the monolith to the microservices

The monolith is currently often considered to be a legacy pattern and something which should be avoided. However, for many companies the monolithic architecture is a good choice. The microservices are great, but both patterns has advantages and disadvantages. The monoliths have different shapes and structures and not all of them are suitable for migration to microservices. The main characteristic of the monolith is that it is considered to be a one unit of deployment. All of the code related to a specific part of company business (sometimes even a whole codebase) is wrapped together into a one deployable package. No matter which part of the codebase is changed, the whole unit has to be re-deployed to apply any changes to the production. This monolith type is called the classical monolith architecture [6]. Such a structure can be broken into several modules. With proper module boundaries, each module can work separately. Even though it is still a one deployable unit, it is has at least some hierarchical structure, which makes the development and maintenance easier. This is called a modular monolith [6]. The modular monolith brings module boundaries, allows developers to work independently on each module, and splits responsibilities. It is often a recommended architecture for startups and presents a good alternative to the microservices. If the modular monolith is not a good solution, a company may decide for migration to the microservices. The golden rule of every migration is to minimize the service outage on the production. When companies decide to migrate into the microservices, their system is usually too big and the goal is to break it into smaller independent services. The first task is to detect and define these potential services. Domain-driven design (DDD) is a helpful tool for this task. After applying the DDD concept, logical parts of the monolith, like invoicing, booking, etc. should be detected and reveal the potential microservices.

After a logical decomposition of a monolithic system, it is important to set up priorities. What is the goal of the migration and what should be migrated first. It can be either the part of the code which isthe easiest to migrate and will help to test the migration process or the part of the code which is used by many other parts of the system,

25 2. Background

so it would be beneficial to extract it into a separately deployable unit. It can be overwhelmingly difficult to migrate everything at once. Migration of services in an incremental way is a lot easier and strongly recommended. Imagine that 50 new microservices are deployed at once and something is not working as expected. All changes need to be reverted so that the debugging of 50 new microservices can begin. Therefore, migrating should be gradual – service by service. Once it was decided which part of the code will be migrated first, it is necessary to extract and move its functionality from an old system. In case the functionality is in wrapped in a single package, the task is as trivial as to copy-paste the code and prepare the APIs to replace methods with API calls. If the code is distributed, it could require a bit of pre-refactoring, wrapping logic together, and then moving it. However, it should not be deleted from the old codebase yet. When the new microservice is ready and tested, it is time to replace the methods with API calls. A good practice is to test service on a production with limited traffic first, to detect any unexpected errors. If all is good, switching the whole traffic to the new microservice can proceed. After some safety period, it should be safe to finally clean up the legacy code in the old system. The whole functionality of the new microservice and the old monolithic code should be the same. While still in the testing period, is it best to avoid adding a new functionality. Once the first microservice works, the migration of another one can start.

2.6.2 Migration from SOAP to REST

Another example is a migration from the SOAP to the REST architecture. The first step of every migration is to make clear why it is even needed to adopt a different technology, what are the expectations and what will be achieved. The migration of services from the SOAP to the REST architecture makes services more pervasive, faster, and suitable for thin clients. The REST services are closely interacting with the HTTP methods and using its full potential. As the HTTP is a popular communication protocol, integration of such services is easier even for less skilled developers and development of such services is much faster.

26 2. Background

SOAP uses the WSDL to describe service resources and corresponding HTTP methods. Upadhyaya et al. [21] developed a framework for how to automatically analyze existing SOAP-based service and generates configuration files needed for the service build on top ofthe REST architecture and providing the same functionality.

Figure 2.8: Steps for migration from SOAP to REST. Source: [21]

Figure 2.8 shows steps for migration from SOAP-based services to the REST architecture. First things first, the WSDL files are analyzed to build a dependency graph which describes how resources, input and output parameters, and operations are connected. The operations related to the same resource are clustered together. Cluster analysis is applied. Afterwards, the Natural language processing (NPL) is used to map the SOAP operations (verbs) to the corresponding HTTP methods (GET, POST, PUT, DELETE). For example, addCity operation should be mapped to the POST HTTP method to create a new city resource. A couple of operations may require manual action, as the NPL technique is not 100 percent accurate. And sometimes the original implementation of the WSDL files just does not follow the best practices. After applying various analysis, clustering, NLP techniques and manual refinement of resources and HTTP methods, output mapping as we can see in 2.9 should be created. After further validation and verification by the user, it is time to prepare the new REST configuration. The solution proposed by Upad- hyaya et al. [21] is capable of automatically generating all the wrappers and configuration files for the REST service, which should be ready

27 2. Background

for deployment and testing. Their case study reported 0.74 percent accuracy of automatic analysis and mapping on the 61 analyzed WSDL files and over 400 hundred operations. Despite the fact that someman- ual refinement and adjustments are needed, the migrated services proved to be faster and more reliable.

Figure 2.9: Mapping between SOAP-based operations and resources to HTTP methods. Source: [21]

2.6.3 Key concepts of these migrations

1. A detailed decision process is very important. Consider all pros and cons of migration and set proper expectations of migration, before rushing right into it.

2. Gradual migration is usually a better way how to do significant changes to the system. It is less error-prone and easier to revert and fix if something goes wrong or migration proves tobe inefficient.

3. Define common standards, best practices, and processes associated with the migration. If every team handles migration differently, it may result in more problems than benefits.

28 2. Background

Table 2.1: Table of references linked to concepts they describe.

Reference Concept Micro- Container- HTTP REST RPC services isation [2] [3] [4] [5] [6] [13] [7] [8] [9] [10] [11] [12] [14] [15] [16] [17] [18] [20] [21] [22] [23] [1] [25] [26]

29 3 Decision process and migration guidelines

In this chapter we propose a decision process on when to integrate the gRPC into an architecture and migration guidelines and best practices which should help to achieve desired result in a case company decided to adopt the gRPC instead or in parallel with the REST.

3.1 Decision process

Should the company start thinking about the gRPC technology as an alternative to the REST? Is it really more beneficial and better than the REST architecture? What benefit it can bring? Is the company even able to adopt the gRPC with its current architecture and tech stack? This section tries to answer all these questions, describe the benefits and drawbacks of the gRPC technology, comparing it with the REST, formulating the optimal architecture, and providing the use cases in which it is beneficial to consider the gRPC integration.

3.1.1 Benefits of gRPC

Below are the main benefits of the gRPC integration.

1. Strong API contract The gRPC is using protocol buffers for data transfer and serialization. The proto language allows to define the strict structure of API messages with strong data typing. This ensures that both the client and the server work with the same data. The protocol buffers allow to define simple messages consisting ofthe basic data types like integer, string, or boolean, but also more complex messages with multiple nested structures inside. It is possible to define own enums or messages and use them asa reference in other messages. Keywords as required, optional, and repeated allow defining the shape of data structure even more precisely, defining whether fields are required or just optional. The proto buffer library automatically raises an exception ifthe message is missing the required field. With good knowledge of what proto language allows, users can define fairly complex

30 3. Decision process and migration guidelines

messages with proper data typing, ensuring that the server and clients share the same data structures.

2. HTTP/2 The second chapter described what are the differences between the HTTP/1.1 and the HTTP/2. The HTTP/2 is an improved version of the HTTP/1.1 transfer protocol, and the biggest change is that it is binary based. This means frames transferred over the TCP connection are binary encoded, which is more efficient and compact for the transfer. The HTTP/2 uses the Huffman coding to compress data. In addition, the server and client maintain a list of frequently used fields and their compressed values. When these fields are repeated, they include the reference to the compression values. The HTTP/1.1 supports pipelining of the requests. While the client can send multiple requests at once without waiting for a response, the server is still responding on requests one by one in the same order. This is defined as FIFO (First in first out) queue, therefore some requests can be delayed due to the processing of other costly requests that require more computing time. The HTTP/2 on the other hand supports fully asynchronous request processing - multiplexing of requests by streaming. Both server and client can initiate multiple streams under a single TCP connection [14].

3. Security There is no need to point out, that the securing applications and services are essential. The gRPC supports a variety of authentication and security mechanisms to ensure easy and safe communication with other systems [26]. The following are supported built-in authentication methods in the gRPC:

• SSL/TLS: the gRPC promotes usage of the SSL/TLS for the server authentication and encryption of all the data exchanged via gRPC channel between the client and the server. While the SSL/TLS is primarily recommended for the server authentication, there is also a possibility for the optional mutual TLS authentication. Every client has a generated public key signed by the CA (certificate au-

31 3. Decision process and migration guidelines

thority) so the server and the client can exchange and authenticate private keys during communication. • Token based authentication via Google: this generic token based mechanism should be used only to connect to Google services, otherwise there is a threat of the token being stolen by third parties and used for attacks. Google tokens (OAuth2 tokens) are attached as metadata credentials to requests and responses as additional authen- tications and are allowed to be used only in combination with the SSL/TLS. So Google will not allow to use Google tokens without the SSL/TLS enabled (on unencrypted channels). • ALTS: The gRPC supports also ATLS (Application Layer Transport Security) in case services are running on Google cloud platform (GCP).

In addition to the authentication mechanisms mentioned above, the gRPC allows developers to integrate their own type of credentials by using the Credentials plugin API. There are two levels of the gRPC authentication and authorization:

• Call-level authentication/authorization: is an authentication on the every single request and such authentication is usually handled with tokens that are added into metadata when a call is constructed. • Channel-level authentication: is an authentication on a channel/connection level, established with the client certificate, typically using the SSL/TLS authentication.

At least a channel-level authentication is recommended to establish a secured connection for the data transfer and to prevent data leaks and impersonation. However, while using the gRPC over the HTTP/2 TLS connection, the whole traffic between the clients and servers is encrypted, even if the channel-level authentication is not involved.

32 3. Decision process and migration guidelines

4. Data compression The gRPC supports both, the request and the response payload compressions. The compression is used to narrow the bandwidth between peers. The implementation supports multiple compression algorithms. It is possible to control compression settings on a message level by defining it in the header of every call, or it can be specified during the channel dialing and used as the default algorithm during the whole session. It is also possible for the client and the server to use different compression algorithms withing the same connection. Such asymmetry is handled by the grpc-accept-encoding response header send by the server, so the client is aware which algorithms are supported by the server. In the case an unsupported algorithm is set, it will result in an UNIMPLEMENTED error status from the server. In the similar way, clients can define the supported algorithms with the Message-Accept-Encoding header. The data compression is optional and flexible.

5. Performance Thanks to the protocol buffers, HTTP/2, data compression and various streaming possibilities, the gRPC provides the high performance RPC processing in many programming languages. In addition, the HTTP/2 also compresses headers, which also slightly speeds up the communication. The gRPC performs very well with a large payloads in comparison with the REST. In the benchmark testing performed by Ruwan Fernando [27] the gRPC was 5-7 times faster than the REST in acquiring data, and 10 times faster in sending data. Tests were performed on multiple operating systems, while Linux performed twice as fast as Windows.

6. Client and server streaming As it was already mentioned in section with the HTTP/2, the gRPC supports the client as well as the server streaming. This means the client is not limited to one request - one response message at a time, as the gRPC allows to send multiple requests in the client streaming or to receive multiple response messages

33 3. Decision process and migration guidelines

in the server streaming scenario. All is handled within one TCP connection. In addition to these two streaming options, the gRPC supports the duplex (bi-directional) streaming as well, which allows client and server streaming messages simultaneously. This provides a better alternative to HTTP/JSON pagination and a way how to effectively share a large volume of data.

7. Auto-generated code With the gRPC, developers do not have to write custom client libraries to use the server APIs. All what is required is to define messages and services in proto files and the rest is handled by the gRPC and protoc compiler. The gRPC has built-in code generation supporting different programming languages, includ- ing Java, C++, Python, Go, Dart, Objective-C, Ruby, and more. Code generation is defined on top of the protobuf files. The generation mechanism is able to produce both, the server-side handlers as well as client-side stubs which simplifies integration of new APIs by the clients. The auto-generated code serves as language-specific interface to use the RPC methods with strict message formats. It provides functionality to access and set each message field, methods for marshaling/unmarshaling of messages from/to raw bits, opening connection with server, etc. What at first glance may look like just another function in an application code, is in fact a remote procedure of a different application, running on another machine.

3.1.2 Downsides of the gRPC

The gRPC technology has a lot of useful and inventive features to offer. However, like in every other technology, there are also downsides connected with the gRPC. So what are gRPC disadvantages and weaknesses?

1. New technology The gRPC is still a relatively new technology. It may seem a bit unfair, that such thing is considered as a disadvantage, but every new library, framework, protocol, or architecture has

34 3. Decision process and migration guidelines

the same disadvantage at the beginning. Even the best technologies that are new are initially questioned, as they are unknown for the vast majority of the developer community. Es- pecially, if there are working alternatives like the REST, which are battle-tested by the community over the years. It is more comfortable to choose a technology that other developers know well and work with on regular basis. It may take some time until the new technology is tested by brave developers to make others think about trying it. It is possible, that future development of the gRPC will lead to its fall and it will not be accepted by the community, but so far it looks like it is on a good path to be a solid competitor to the REST architecture. At least in some use cases.

2. Weak browser support The gRPC is not directly supported by browsers. The main problem is that browsers cannot force usage of the HTTP/2 and as was mentioned in the previous chapter, half of the internet browser traffic is still using the HTTP/1. There are already some initiatives and projects like the gRPC-Web, which should help with gRPC integration into the frontend development, but the implementation just introduces additional hop to communication between the server and the client in a form of a proxy, which should handle the transition from the HTTP/2 into the HTTP/1.1 if needed, as well as messages transcoding from proto buffers to the desired format. In other words, the gRPC is not a very tempting alternative for the REST in the browsers.

3. Status codes and error messages When it comes to error handling, things might start to get complicated. The HTTP status codes are a perfect tool and a well known concept for error handling in the developer community. The vast majority of developers know the basic HTTP status codes and their meaning. In the gRPC it is a bit different. The gRPC does not have codes like 200, 404 or 503. The status codes are defined of course, but they just look different. The gRPC defines enum of approx. 20 strings, which represent the listof

35 3. Decision process and migration guidelines

the most common status codes (errors) like OK, NOT_FOUND, UNIMPLEMENTED or PERMISSION_DENIED. OK is returned on success. In the case that something goes wrong, proper error status code and message should be sent to the client. However, there is no clear way how properly build and catch the errors across multiple programming languages. Error instance building is language specific. It is crucial to realize that instead of expected response messages (procedure responses), error message may be returned. Therefore it needs to be properly caught by the client and handled accordingly. Fortunately, there are interceptors (custom code executed before API call on server-side or response acquiring on the client-side), which are very helpful for the error handling. The concept of interceptors is described a bit more in the next section.

4. No URLs (harder testing) The gRPC is an implementation of the RPC technology. Instead of the URL and HTTP requests, methods are called directly from the codebase. It is very comfortable during the implementation, but when it is desired to test the requests directly, it is not that simple. While in the REST architecture it is possible to use curl or tools like Postman to call the endpoints, the gRPC procedures cannot be tested that easily. The protobuf files are unlike the JSON non-human readable, therefore special com- mand line tools or applications are required to help developers and testers to prepare messages and communicate with the server.

5. Lack of developer support As was mentioned in the first point, the gRPC is still a relatively new technology. This is a reason why developers are facing a lack of gRPC developer tools. A lot of tools are built around the HTTP/1.1 so they are unusable for the gRPC. However, it is not just about the tools, but also about the lack of proper documentation, tutorials and plenty of unanswered questions and unresolved issues. The developers will need to get used to the gRPC as technology first and afterwards the community

36 3. Decision process and migration guidelines

will start focusing on how to make usage of the gRPC easier. So while popularity of the gRPC will be rising, number of new helpful tools will grow as well.

3.1.3 gRPC vs REST

The gRPC and the REST provide a way for services to communicate with each other and share the data. Even though both technologies have the same purpose, the manner in which they implement it differs. The REST is a standard, where everything is optional. It is up to developers which features of the REST they want to implement. There is more than a one way of how to "do" the REST, as long as the final result follows the definition of the REST. Developers have plenty of frameworks, libraries and protocols to choose from, as well as programming languages. It is possible to implement the REST API even in assembly. The gRPC is more strict. It is not a standard as the REST, but more like a protocol, and there is usually just a one approach of how to do something. The REST is a resource oriented standard. It provides a way to create, get, update or delete a resource. For example to create a new bank account, show a deposit, add funds to the account or delete the account. It is typically implemented as a combination of the URL path representing the resource location and the appropriate HTTP method (GET, POST, PUT, DELETE). The gRPC is focused on procedures, rather than resources. Not every time the services communicate with each other need to share any resource. Sometimes one service just needs to inform the other service about some specific action.

The API schema in the REST is completely optional. Unfortunately, in the IT community it is very common that the APIs are missing a schema, as well as documentation. It often leads to an unexpected behaviour and integration of such APIs may be a difficult task for the developers. They need to figure out what type of response toex- pect, consider all fields as optional, and include a lot of unnecessary parsing logic around the API client to prevent errors. Thanks to the protocol buffers, the API schema in the gRPC is clear, unambiguous and strict. The request and response messages are defined in proto files and without these definitions it is not possible to call thepro-

37 3. Decision process and migration guidelines

Table 3.1: gRPC and REST comparison gRPC REST API contract Strict, required Optional (OpenAPI) Payload Protobuf (binary) JSON (text) Protocol HTTP/2 HTTP/1.1 Streaming Bidirectional Client-server request Code generation Built-in protoc Third-party tools Performance faster than REST slower than gRPC Tools Little Many Maturity New technology Well-known standard Browser support Limited Yes

cedures. The client knows how exactly the request message should looks like, as well as what to expect in the response. Additionally, protoc compiler generates all the necessary language-specific code for the client, so there is no need to implement any additional logic and adapters around these procedure calls. Therefore development is faster and less error-prone. It is possible to force strict API schema in the REST as well. There are popular technologies such as the OpenAPI, in which the developers can define the schema and light documentation. However, it is completely optional and not a required part of the REST implementation.

The strong API contract is not the only advantage of the protocol buffers. While the REST is typically using the JSON format, whichis text-based, protocol buffers transfers data in a binary format. Ituses less bandwidth, so speed of data transfer is much faster in the gRPC. It uses HTTP/2 protocol, which benefits were described in second chapter. The REST can use the HTTP/2 as well, but usually it works over the HTTP/1.1 and there is no easy way of how to force the usage of HTTP/2, especially in APIs exposed to outer world and browsers. Currently, the browser support in the gRPC is very limited and is not recommended to use it for now. The REST is much better for this use case. The REST allows client-server communication, per one request only. There is not any native support for streaming. The gRPC on the

38 3. Decision process and migration guidelines

other hand, supports 4 types of streaming, as described above, which may be beneficial as an alternative to pagination.

Popularity of the REST is unquestionable and the gRPC is in a hard position to attract the users to consider it as an option. It will take at least couple more years to see, whether the gRPC will become a standard at least in the microservices architecture. Thanks to the popularity of REST, there is plenty of tools, frameworks and libraries helping with performance, documentation or just making the development easier. Tools available for the gRPC are currently very limited. While comparing these two technologies, the use cases for both are becoming more clear. Summarized comparison of both technologies in available in table 3.1.

3.1.4 Optimal architecture

This section describes an optimal architecture, for which it is good to start thinking about the gRPC as an alternative to the REST. As previous sections indicated, the gRPC is perfectly suitable to the microservices architecture. It is possible to adopt the gRPC in scenarios where two monoliths communicate with each other, but it will not use the full potential of the gRPC itself. However, if a company is considering to split the monolithic architecture into the microservices, or is already in process of migration into the microservices, it is an ideal time to consider the gRPC as an option for the internal service communication. While the whole code is situated in one place, one repository, all processes and business logic is implemented by calling methods within the same monolithic repository. There are not API calls, just direct method calls. Very similar functionality can be achieved with the gRPC integration. The gRPC auto-generates all necessary code needed for client-server communication. It helps to minimize internal changes in the codebase, as the RPC procedures look very similar to these direct method calls used in the monolith. Despite it is possible to run the microservices on virtual machines in the traditional on-premises architecture, it is not a very popular solution to implement it in such a way. As described at the begging of the thesis, the microservices architecture uses its true potential while

39 3. Decision process and migration guidelines

Figure 3.1: Microservices architecture with gRPC. Source: https://docs.microsoft.com/en-us/dotnet/architectu re/cloud-native/grpc

running inside the cloud-native environment, using containerization with technologies like the Docker, and platforms like the Kubernetes or Rancher for the management and orchestration of containerized applications. It helps with automation, scalability, self-healing mechanism, monitoring and much more, while securing more time to focus on the development and functional requirements of a system.

In figure 3.1 is an example of the microservices architecture running with the gRPC. The microservices run their own gRPC server that listens for the incoming HTTP/2 traffic. When a device (web client, mobile device, or another backend) requests the data over the API gateway via HTTP/JSON call, the message is sent to aggregator microservice to process the request. The aggregator can distribute processing between multiple different microservices. This concept is also called BFF (backend for frontend) and it helps the clients to avoid having unnecessary aggregation logic on their side (which would also mean separate implementation on every single client). Internal communication is handled with gRPC over HTTP/2. Such architecture

40 3. Decision process and migration guidelines

design defines the modern development pattern in cloud computing, which uses the full potential of every technology.

Even though it is not a requirement, systems running on the Google cloud platform (GCP) have an advantage while adopting the gRPC architecture. The reason is obvious, gRPC is a technology developed by Google. The GCP provides several features and improvements for the gRPC services like the Google endpoints, Google token authentication or Cloud run, which makes integration of the gRPC a bit easier than outside of the GCP environment.

3.1.5 When to migrate

So far thesis has described what are the advantages and disadvantages of the gRPC, differences between the REST and the gRPC and optimal system architecture to use the gRPC effectively. So what are the use cases when company should consider the gRPC as an alternative to the REST? As mentioned previously, the microservices are probably the main requirement. Either already existing system distributed in microservices communicating over the REST, or systems which are in process of the migration from monolith to microservices or just starting with the migration. It is worth thinking about the purpose of microservices, whether they will be completely internal (the gRPC is a good option) or will be exposed to the outer world (the REST or a combination of the REST and the gRPC is probably a better option than pure gRPC service). Another good opportunity to adopt the gRPC is the service refactoring or development of a new service version. The new service version can be either provided as a gRPC server only, or support both – the gRPC altogether with the REST (HTTP/JSON) alternative, to let the clients adopt the gRPC voluntarily.

Below are another scenarios, when it might be beneficial to use the gRPC:

• For system where the microservices are implemented in multiple programming languages, the gRPC is a good choice to secure effective communication between services. As the gRPC

41 3. Decision process and migration guidelines

supports automatic code generation for multiple programming languages, there is no need to write a separate API handler for every client, which saves a lot of developers’ time. The server can automatically generate client stubs to multiple programming languages, which the clients just add into the codebase.

• If performance and effective communication is crucial for the service, every saved bit counts. The gRPC can provide a desired boost in comparison to the REST communication. All thanks to the protocol buffers, binary data transfer, fast marshaling and unmarshaling of messages and forced usage of HTTP/2 transfer protocol.

• It is a well-known problem that a lot of the REST APIs are missing strict schemes and documentation, which complicates the work with these APIs. In case REST services have poor or none API schemes, and strict API contract is desired, protocol buffers used in the gRPC are the perfect tool how to achieve this. The clients know how exactly to call the procedures as well as what to expect as a response. Both sides share the same objects.

• Serializing and deserializing of huge JSON payloads could be really costly on devices which do not have that much computing power or memory, like mobile devices. The gRPC and HTTP/2 improves both latency and network utilization on mobile devices.

• The main feature of the gRPC, which is not available in the REST is streaming. If services require real-time communication with streaming, or necessity to effectively transfer large volume of data, the gRPC is a clear option.

It is important to realize that migration to the gRPC should not be forced or careless. The REST architecture may be either very bad or very good, it all depends on how it is used and implemented. When the APIs work well, all endpoints have documentation, schemes are defined with tools like the OpenAPI or Swagger, and there isnore- quirement for streaming or any other strong need for one of the points

42 3. Decision process and migration guidelines

described above, then the REST APIs exposed via the HTTP/JSON are absolutely sufficient. Careless migration could damage already properly working architecture. Potential migration can be also highly dependent on the company culture and teams’ willingness to learn and adopt a new technology. Developers would not be happy to invol- untarily migrate services to the gRPC, especially when they provide high quality APIs, following RESTful standards, with descriptive documentation and schemes, where everything works well.

The microservices architecture is "must-have" prerequisite for the gRPC. Either already existing microservices or process of migration into the microservices. The following two tables should help to identify, whether the company has a suitable architecture for gRPC adoption. There are four evaluated parameters (which differ in severity).

Severity Value Microservices Must 8 Only for internal Strongly 3 communication recommended Containerised Strongly 3 services recommended Multi programming languages Preferred 1

Adding up values of those parameters which the company fulfills, reveals the score, which can be then evaluated in the second table to get the result, whether the company has a suitable architecture for the gRPC integration.

Score Result <= 7 Not suitable for gRPC 7 < x < 12 Acceptable for gRPC >= 12 Optimal for gRPC

In case the first evaluation has a positive result (acceptable or optimal architecture), proceed to the second evaluation. This second evaluation table checks what is expected from the potential migration

43 3. Decision process and migration guidelines

and whether desired improvements and advantages the gRPC could provide are beneficial enough to proceed with the migration.

Value Streaming 5 Strict API contract 3 Faster API integration 3 Performance boost 1

Again, check the table and calculate the final score. Compare the score in the table below to get the final result.

Score Result <= 3 Not worth to integrate gRPC 3 < x < 9 Consider gRPC integration >= 9 Beneficial to integrate gRPC

This process should help the company decide, whether they have a suitable architecture and use case strong enough to proceed with the migration. Even if both checks have positive results, the final decision is individual and may differ in every company, as there are also other company-specific factors as the availability of skilled developers who can handle the migration, the current company priorities, or the willingness of teams to learn the new technology. However, if these checks revealed positive results, gRPC migration should at least be considered, as it could provide a lot of benefits.

3.2 How to migrate

In the previous chapter, the thesis described two migrations. Migration from a monolith into microservices, and migration from the SOAP to REST. Proposed guidelines for the gRPC adoption are inspired by both of these migrations.

3.2.1 Key concepts

Firstly, there are few general concepts to keep in mind during the migration.

44 3. Decision process and migration guidelines

1. Gradual migration This concepts is common for many technical migration in the IT world. Gradual migration from the REST to gRPC is safer, less error-prone, and requires less effort. It helps to mitigate the negative effects of a migration. Lessons learn from adopting the gRPC in one service, can be used in a migration of another one. Such approach is less invasive to the system. Every team can individually schedule and properly plan the migration of their services. One team could use the migration to introduce the new version of their service, another one could see the opportunity to deprecate old stuff from the HTTP/JSON endpoints. Prioritize the migration of services where adoption of gRPC brings the most benefits. For example, services with a high load where the performance is important, services with the poor or missing API schemes, or where any type of streaming can improve the communication. In a case in which all services are migrated together, the complexity of migration grows significantly. The more changes are required at once, the higher the possibility something unexpected will happen.This leads to the second point.

2. Minimal changes Unless it is not an intention, the amount of changes in the migrated service should be kept to a minimal. It is important to avoid doing changes, which could affect the functionality of the service. When introducing the new service version, it has be be clear what will be changed. Introduction of any unexpected and unannounced changes is not desired. After the new version is developed, further development in the old version should be stopped (perhaps except bug fixes).

3. Do not break the production The ideal migration is when the clients of the service are not negatively affected at all. Avoiding any outages in a production environment is crucial. When the gRPC service is tested and ready for production traffic, there are more options onhow to handle the switch. One option is to temporally maintain the support of the gRPC and the REST simultaneously. Old

45 3. Decision process and migration guidelines

service deployment accepting the HTTP/JSON traffic and the new service deployment handling gRPC traffic. Once all clients switched to the gRPC version of the service, the old deployment can be shut down. This approach is good for cases when the gRPC service provides a changed functionality (new service version). The development of the old service version can be stopped (which motivates the clients to switch into gRPC), while development in the new version continues. Another option is to accept both (HTTP/JSON and gRPC requests) within the same deployment. Once the service accepting both types of requests is deployed, the switch of traffic from the old deployment to a new one can be done using the temporary proxy and DNS redirecting. The clients should not even notice that their requests are being processed on a new service. Technology like grpc-gateway [28] is able to accept the HTTP/JSON requests and translate them into the gRPC requests. It allows clients to continue accessing APIs via HTTP/JSON. It adds an additional proxy layer to the communication but provides the possibility to switch into the gRPC without the downtime and once the clients are ready.

4. Voluntary migration Enabling the REST and gRPC simultaneously is an important step in a gradual migration. Existing clients can keep calling the REST endpoints until they decide to migrate. Migration should not be forced. Existing clients may have more critical problems to handle, and adopting the new technology is the last thing they need at the moment. Inform clients above possibility to switch into the gRPC, but do not forbid them to access the service in the old way, from day to day. Voluntary migration does not apply only to the clients. Some services may work without any problems using the REST architecture, and it is not their priority to migrate into the gRPC. When migrating microservice from the REST into the gRPC it is not mandatory to migrate the whole system. For the services exposing their APIs to the outer world, it may be difficult to migrate endpoints into the gRPC. It is one thing to force

46 3. Decision process and migration guidelines

internal clients within the same company to migrate into the gRPC, it is another thing to force external clients and vendors to adopt a completely new technology, which is probably not even included in their tech-stack. Be patient and provide enough time to the clients to switch to the gRPC version. However, migration may become a nuisance, which can last months and often even years. If the clients are aware of the requirement to switch their requests from the HTTP/JSON into the gRPC for quite a long time, it is okay to define a clear deadline when the support of the old version will be stopped. Maintaining both versions of the application simultaneously can be annoying, especially when it is necessary to keep adding new features and bug fixes into both versions.

These are the 4 general concepts to keep in mind during the migration. The next part includes other guidelines, best practices, and recommendations to follow, which should help with a successful migration to the gRPC.

3.2.2 Protocol buffers management

The protocol buffers files (with .proto extension) are the building stone of every gRPC service. While working in a team or with multiple teams in a company, it is important to define some conventions and best practices as to not create complete chaos and a cumbersome code. As the gRPC works on the principle of sharing the necessary proto files to the clients, whenever a new client is built, the developers have to copy the generated files out of the server into the new project. The more microservices are migrated into the gRPC, the more proto files are generated. There are multiple ways of how to share proto files [26].

1. Separate proto repositories One approach is to let every team be the maintainers of all their own proto repositories. They are responsible for all the changes, API definitions and service implementation. All service clients include the proto files into their service that can be achieved in form of a package/library/module import or

47 3. Decision process and migration guidelines

directly as git submodules. This means every team maintains two repositories, one with their proto files and one with the service implementation + other protos integrated as the clients. 2. Common proto repository Another approach is to have a one common proto repository shared by multiple teams (or whole organization). Such repository should not contain anything else, just the teams’ proto files and utility tools. Again, every team includes this repository as a git submodule or import client code in form of package/library/module [26]. All changes are discussed at one place and all proto files share the same structure and conventions. Ev- ery new merge request can be validated by the CI to ensure it follows common standards. 3. Hybrid approach There is also possibility to combine these two options together into a third hybrid option. Every team owns their service specific proto files, while some common proto used by manyteams can live in a shared repository. It can includes protos related to logging, error handling, status API, etc. Whichever approach is chosen, it is very important to define an organizational structure for the proto files. Having the proto files distributed across multiple repositories, in an inconsistent structure can unnecessarily complicate the development and possible automation. The common structure should be defined and followed by everybody involved.

Below is an example of a common protocol repository. With such a structure it is easy to see the company structure and all teams. It is fairly simple to jump right into the proto files of a particular team, and it also supports the versioning of the protos. Every team can manage the changes in their module and discuss them with other teams in one place. The common proto repository looks like the best solution, providing one source of truth for company proto files and a place where to discuss all API changes, deprecation, share the useful common definitions, or automate processes via CI. It also provides a place for documentation of all gRPC services.

48 3. Decision process and migration guidelines

organization finance invoicing v1 invoicing.proto v1beta1 invoicing.proto payment v1 payment.proto

booking v1 booking_api.proto booking.proto v2 booking_api.proto booking.proto

3.2.3 Protocol buffers best practices

As the proto files are the crucial part of every gRPC service, itisa good practice to follow common conventions to make protocol buffer definitions consistent, easy to read and maintain. Google defines a really lightweight guide [29] with some basic concepts. However, developers in Uber did a great job and created a much better and more descriptive styled guide [30] with a lot of do’s and dont’s. Here are a few of them: • Naming conventions – use short and descriptive names, using basic characters [a-z0-9] – avoid abbreviations if possible – files should be names with lower_snake_case.proto – messages and services/RPC should always be in PascalCase – field names in messages should always bein lower_snake_case

49 3. Decision process and migration guidelines

– request messages should end with Request prefix, and response messages with Response prefix

• Enum names should always be in PascalCase, and enum values in UPPER_SNAKE_CASE. It is recommended to use enum name as prefix for valuesenum_name_value ( ). Define also invalid value, which represent unknown or invalid values

Figure 3.2: Enum example in proto language

• Do not use the reserved keyword in messages or enums. Using the reserved keyword can lead to breaking changes while delet- ing or changing the value from the source code. Use deprecated instead.

Figure 3.3: Deprecation of the field in proto language

50 3. Decision process and migration guidelines

• Due to collisions with programming language constructors and built-in functions, it is not recommend to use some keywords, like json_name, file_name, file_path, so always make sure the fields you are using are not in collision with targeted programming language.

• The new versions of the packages should be properly marked to easily recognize a new version. Use vMAJOR or vMAJOR- betaBETA formats (v1, v2beta1), where MAJOR and BETA are greater than 0. Major versions should not be used for every change. Increment the package version only if necessary, for code clean-up, when introducing bigger refactoring, etc. Rather than creating a new package for every breaking change, using deprecation keyword instead is more effective. Breaking changes are changes which can negatively affect the client and potentially cause an outage. Example of such changes are delet- ing or renaming an enum, enum value, message or message field, changing the field type, changing the function signature, changing the label of a message field, i.e. from optional to repeated or required, etc.

• Use (two) spaces instead tabs in formatting, and one line dis- tance between objects in file.

• Use comments. Every message should contain at least one com- ment describing what it represents.

To enforce all these proto buffers conventions and to prevent merging the code which does not follow these conventions into a master, use the linters. The linters are tools for a static code check, which helps to improve the code quality. Example of such linters is the one provided by the Uber developers [31]. It is effective to use the linter as part of the automatic CI builds, which forbid merging a new code if pipelines fail.

51 3. Decision process and migration guidelines

3.2.4 HTTP/JSON gateway

As described previously, the migration to the gRPC should be gradual. There may be multiple clients, which will require an option to call service via the REST endpoints for a while. Ability of service to accept both types of the API calls simultaneously, is especially helpful during the migration period. As mentioned above, grpc-gateway [28] is a technology which provides such functionality. The gRPC to JSON reverse proxy maps incoming JSON requests into particular gRPC endpoints.

Figure 3.4: grpc-gateway demonstration. Source: https://programmaticponderings.com/2019/04/17/istio- observability-with-go-grpc-and-protocol-buffers-based- microservices

In the figure 3.4, proto file with service definition is used togen- erate the gRPC service, as well as the reverse proxy, which maps the incoming REST API request into a gRPC procedure. Below 3.5 is an example, how such mapping from the HTTP/JSON into gRPC looks like in a proto file. Thanks to this proxy pattern, the service accepts both types of requests and provides time for the clients to switch into the gRPC.

52 3. Decision process and migration guidelines

Figure 3.5: grpc-gateway proto example

3.2.5 Define API endpoints

To provide API endpoints via gRPC, it is necessary to detect which REST endpoints should be migrated. Every endpoint which should be migrated, has to be defined in its proto alternative. In a case in which the REST API endpoints are defined with the OpenAPI schema, itis possible to use available tools to auto-generate a proto definition. It might help to build a proto definitions, but these tools are often not 100% accurate. The more complex OpenAPI schema is, the greater the chance generated code is incorrect. Therefore, it is better to define the proto files carefully, as they are the key part of the gRPC API definition. The gRPC is more strict than the REST API, and every endpoint requires request and response message definition in proto language. Follow the naming conventions described in the previous part. All fields must have a type defined, which should not differ fromits REST counterpart. If multiple endpoints are using the same message, reuse the already defined message by importing it into the target file.

53 3. Decision process and migration guidelines

Keyword options as optional, required, repeated should be defined for every field.

The HTTP methods from the REST API can help to define appropriate name for the RPC procedure. For example: • POST /accounts == CreateAccount • GET /accounts/{username} == GetAccount • PUT /accounts/{username} == UpdateAccount • DELETE /accounts/{username} == DeleteAccount

3.2.6 Interceptors

The interceptors are a piece of code which are used to modify behaviour of the called request. It is similar to the middleware code used for modification of handler on a HTTP server. The gRPC interceptors allow to modify the gRPC request before passing it further for processing, change the behaviour of the called handler, or modify the response before it is returned to the client. For example, interceptor could be used to check whether the same request is not already cached in memory, and if so, return cached response instead of calling the method again. The interceptors can be used for the authentication, custom error handling, logging, monitoring or basically anything which might be somehow useful for the developers. It is possible to use interceptor also on the client side (e.g. to automatically add authentication or header parameters). It is a very useful tool, which can help with plenty of different things during the gRPC integration.

3.2.7 Error handling

Error handling is not the same as in the REST and may be a bit con- fusing. The gRPC does not work with HTTP status codes. But the gRPC has its own status codes and format of error messages. In case the REST API handles known exception (like 404 Resource not found), the gRPC server should try to mitigate this flow and return the most accurate gRPC exception with appropriate status code and error message. The unspecified error exception should not be returned tothe

54 3. Decision process and migration guidelines

client, instead always build a proper error message with status code and descriptive error message, so the clients can check what exactly is wrong.

3.2.8 Authentication

Depending on the incoming traffic, the services should be secured with the proper authentication method. As describe previously, the gRPC supports multiple types of authentication. In a case where traffic is just internal, and all services are called within the same network, authentication may be omitted. However, it is always better to include at least some type of authentication, like via tokens. If the service is available also for the external clients outside the internal network, authentication is important. Either client or a server can use the mentioned interceptors to add token authentication before every requests is called/processed, or use the SSL/TLS certificates to encrypt the server-client communication.

3.2.9 Testing

Testing of the new code should not be omitted. Mocking and simu- lating the gRPC traffic is also possible with the interceptors. Manual testing by calling the RPC methods directly is a bit more difficult than testing in the REST, where endpoints can be easily accessed via CURL or tools like the Postman. However, there are alternative manners of achieving a similar testing environment in the gRPC. To allow browsing of the gRPC schema, show available services, and messages, or even calling them, the user needs to either read the proto files or the server has to allow the server reflection. When server reflection option is enabled, the server can on-demand send a description of all its methods and messages, which can be transformed into a human-readable format with CLI tools like grpcurl [32] or Evans [33]. These tools allow user to define messages, send gRPC requests to the server and receive a result. The main purpose of the server reflection is to help with debugging and allows the users (developers and testers) to work with the gRPC without the necessity to have all the proto files definition stored on a local machine.

55 3. Decision process and migration guidelines

3.2.10 Deadlines and retries

gRPC deadline is basically equivalent to the timeout setting used in HTTP/JSON computing. The deadline allows the gRPC clients to specify how long they want to wait for a call to complete before it is terminated with the error status code DEADLINE_EXCEEDED [26]. The deadlines are typically very large numbers, but how they are defined is language-specific, so it may differ in every language. When the deadlines are not set, there is a risk the server resources will be held for too long and with more incoming requests the server runs out of resources as memory or CPU. Therefore, the server should set up a deadline for a maximum processing time, and the clients should wait for the response till it is still reasonable and useful for them. Usually, there are timeouts settings included in the REST API calls, which have the exactly same purpose. For the initial setup, it is okay to use the same deadlines as timeouts, but it is important to set at least some deadlines.

Sometimes it may happen that the request fails. This can be caused by connectivity issues, reached deadline on server-side, or issues on the client-side, like invalid request. The gRPC allows setting retry policy. The retry policy config consists of a list of retriable status codes, which typically contain UNAVAILABLE, but it is up to the client, which status is added to the list. There are also other parameters as maximal retries or delay between the retry calls [34]. Setting both, retries and deadlines, helps to increase the API success rate and sustains reliable communication.

3.2.11 Load balancing

The gRPC load balancing is a tricky thing. It is different from the typical HTTP/JSON load balancing. The HTTP/JSON load balancer operates on the L4 (Transport layer) working over the TCP protocol. When a new request comes in, the L4 load balancer chooses a healthy service instance and the connection is routed to that instance till it is closed. That is how the classic network load balancers work. The gRPC uses the HTTP/2 to multiplex many requests within the same connection and holds persistent and steady connection for a longer

56 3. Decision process and migration guidelines time, to avoid resources spent on connection handshakes [35]. There- fore, the classic L4 load balancers are not a good option for the gRPC load balancing, as they throw away one of the main advantages of the gRPC. There is several approaches how to handle the gRPC load balancing:

1. Server-side load balancing One option when load balancing is handled on the server-side was described above. This option is not ideal and is only recommended for the communication with external clients, as developers do not have control over the implementation on the client side. Another option is to periodically force disconnection with the clients. After the reconnection, a healthier instance of the service is automatically chosen for the new connection. The next option is to use the application load balancers, operating on L7 layer. However, it may have the same problem as the network load balancers. If there is one client with a persistent connection, which keeps overloading the service with large volume of requests, it can overload the instance. There is also the DNS service discovery option. When request for connection comes, the DNS will return IP addresses of healthy instances in a random order, from which one is chosen. Again, there is the same problem with one client and heavy load. 2. Client-side load balancing Another option is to let the clients solve the load balancing, by implementing decision logic on their side. The client queries the IP addresses for a healthy instance and chooses one. How- ever, this option causes the clients to have more logic on their side and is only viable if we have full control over clients. It is rarely used and for publicly accessible applications also very unsecured. 3. Look-aside load balancing There is also a hybrid solution called the look-aside load balancer, shown in 3.6. The client calls an external service, which returns a list of available instances, and all other necessary information. Server instances periodically report their status

57 3. Decision process and migration guidelines

to that external service. There is some logic necessary on the client-side as well, but not as much as in the pure client-side load balancing described above. The load balancing is happening per-call not per-connection, so the problem with one client overloading one instance should be solved by this. However, there is a necessity to implement and deploy another dedicated service for this and include some logic on the client side.

Figure 3.6: Look-aside load balancing. Source: https://grpc.io/blog/grpc-load-balancing

To sum it up, there are multiple different options of how to handle load balancing in the gRPC, and none of them is a bulletproof solution, because all have some drawbacks. If the company has full control over all the clients, client-side load balancing seems to be in theory the best option, but practically it may be very annoying for the clients. The server-side load balancing is a convenient solution, but gRPC is losing some core benefits. Therefore, the hybrid option with look-aside load balancing is usually a preferred solution. Typically,

58 3. Decision process and migration guidelines

every infrastructure platform has its own tool for the gRPC load balancing, like Linkerd [36] for Kubernetes or Application Load Balancer [37] on AWS, which is probably the best solution compared to the implementation of a custom solution.

3.2.12 Monitoring

Once the gRPC service is implemented, deployed, tested, and ready for the clients’ requests, it is advised not to forget setting of alerting and monitoring of the service. It is good to mimic the already existing HTTP/JSON monitors and alerts. Prepare dashboards showing basic metrics like a number of incoming requests, connections, latency, and especially error rates. In the case of the error rate monitors ex- ceeding a meaningful threshold, alerts should be triggered, so that potential issues are not overlooked. It is recommended to also monitor resources usage, as the gRPC implementation differs from the HTTP/JSON services. It is crucial to monitor it at least in the early days after migration.

3.2.13 Summary

Following all these guidelines, concepts, and best practices should guide the potential gRPC adopters to realize successful and safe migration of services from the REST architecture to the gRPC. These guidelines do not provide any implementation-specific tutorials or strict rules on how to DO the gRPC. The thesis describes the high-level guidelines and recommended concepts to follow, in order to make the migration easier and smoother. Every migration is specific and may differ from company to company. However, the thesis describes rules, which should be applicable for the majority of the migrations. The crucial part of the thesis is also the decision process. This part helps companies to decide whether the gRPC technology is a suitable alternative for their architecture and if it will provide the desired benefits like performance boost, better API contract or easier development. To make a right decision is very important to avoid worthless and expensive investment into a new technology.

59 4 Case study in Kiwi.com

In this chapter, the thesis introduces company called Kiwi.com, its architecture and describes how the proposed decision process and guidelines were applied during adoption of the gRPC technology in Kiwi.com.

4.1 Kiwi.com

The Kiwi.com s.r.o (further referred as company or just Kiwi.com) is a Czech online travel agency (OTA), founded in 2012 (formerly known as Skypicker.com). The company primarily provides a flight search engine which aggregates and combines many different carriers and other content providers. Kiwi.com business is oriented mainly on airline tickets, but also includes ground transport. Bookings created on Kiwi.com connect carriers which do not usually cooperate with each other, what allows to create enormous number of unique itinerary combinations. The company has the vision to become Virtual Global Supercar- rier(VGS) and provide transport options from any point A to any point B on the Earth, using one single application.

4.2 Architecture

A couple of years ago the company started to adopt the microservices architecture and migrate from huge monolithic repositories handling everything into smaller independent microservices covering specific functionality. Altogether with code distribution into smaller services, the monolithic databases were divided into smaller ones. Currently, most of the services are containerized and run as Docker containers. It was a very similar migration process as described by Newman [6] in chapter 2.

Services were hosted on the AWS service platform, but after microservices adoption and containerization, the company decided to migrate services under Google Cloud Platform and Kubernetes or-

60 4. Case study in Kiwi.com

chestration. Altogether, databases were migrated into CloudSQL. The microservices are written in many different languages (Python, PHP, Golang, C++, Javascript, ...) so the tech stack is very varied. A couple of services were rewritten from Python or C++ to Golang. As we can see, the company has a lot of experience with different kinds of migrations and is not afraid to adopt new technology if it can provide benefits and improvements to the current architecture. The whole internal communication between the microservices is handled primarily over the HTTP/JSON requests (occasionally over native packages). Most of the services use the OpenAPI to describe the REST API schema, so there is at least some API contract between services. There are no strict rules and best practices on how to write and maintain the OpenAPI schemes, and every team does it independently.

4.3 Decision process

What led the company to consider the gRPC adoption and why in the end we decided to go for it? The gRPC is a new and interesting technology, and we like to try a new things. But this would be very weak reason. Below are several arguments for the adoption.

4.3.1 Optimal architecture

Kiwi.com architecture fulfills a lot of parameters of optimal architecture suitable for the gRPC. The business logic and all supportive services run as containerized microservices, hosted in Google Cloud Platform, which could provide additional gRPC support. Services are written in multiple programming languages, so the integration of every new REST API into a specific service typically requires additional adapter code to integrated a new API. This means a lot of time is spent writing additional code just to make two services communicate with each other. And with a number of microservices at Kiwi.com (lower hundreds), there is a need to integrate a new API call into one of the services pretty often. This process could be sped up with the gRPC integration, as all that is needed to integrate the new gRPC API, is to include auto-generated files into the desired service repository to handle the communication over the proto buffers.

61 4. Case study in Kiwi.com

By evaluating Kiwi.com architecture with the proposed technique, we got a very positive result with a score of 15, fulfilling all of the parameters of optional architecture for the gRPC.

Severity Value Microservices Must 8 Only for internal Strongly 3 communication recommended Containerised Strongly 3 services recommended Multi programming languages Preferred 1

Kiwi.com also has the Backend for Frontend (BFF) design pattern implemented, similar to one shown in the figure 3.1. However, all request calls are sent into microservices through the REST APIs, which is not as effective as it could be over the gRPC.

4.3.2 API contract

A lot of teams at Kiwi.com provide the OpenAPI definition of their services. However, there is also a number of services that are missing proper API schemes. Due to iterative development and the constant need for new requirements, REST APIs have a tendency to slowly abandon REST standards and become messy. Another problem is that the same objects are often represented with different schemes across teams. This creates the need for additional validation and formatting that has to be done while sending or receiving responses from others. Concepts of common repository of the proto buffer definitions can enforce the usage of the same objects across multiple teams and provide unified handling of the common objects. This can indirectly improve the overall data quality in the company and provide better data for the warehouse databases and data analysis.

4.3.3 Performance boost

Most of the requests do not require extremely fast and efficient processing, but there are also API calls where performance matters. Good

62 4. Case study in Kiwi.com performance is required especially during the exploration and shop- ping experience, when customers or partners search for flights and require results quickly, followed by the booking process while the cus- tomer selects preferred itineraries and proceeding to payment. There are plenty of on-demand and periodic requests happening during this flow in the background, like validation of selected content, availability of additional services, allowed baggage, and overall price and availability of the chosen itinerary. During this process, multiple microservices are communicating with each other, and a number of HTTP/JSON connections have to be established every few milliseconds. A lot of marshaling and unmarshaling of responses is happening as well. The gRPC with its steady HTTP/2 connection and protocol buffers binary format should provide the desired performance boost.

4.3.4 Drawbacks

While looking at all these advantages, the company also considered all the drawbacks. The biggest concern is the immaturity of the gRPC technology. While the REST API architecture is a battle-tested technology, well-known basically by every developer, the gRPC is relatively unknown. Therefore, it requires a lot of learning and time investment at the beginning. Another concern is how individual teams will be willing to adopt new technology. Every team has the responsibility for their services, and there are no strict company-wide rules on how to manage these services.

4.3.5 Decision

When evaluating another proposed table, to reveal whether our use case for the gRPC migration is strong enough, we got a score of 7, which means it is good to consider the gRPC as a REST alternative. We were missing just streaming for now, but it is very likely that in the near future also streaming will be the desired benefit of gRPC, but for now, it is not a key factor that should decide about the gRPC integration.

63 4. Case study in Kiwi.com

Value Streaming 5 Better API contract 3 Faster API integration 3 Performance boost 1

With positive evaluation results of both tables and consideration of all pros and cons, the company has decided to proceed with migration and try to integrate the gRPC in a non-invasive way, as a proof of concept in first services. As described in the previous chapter, there is no point to massively migrate all services into the gRPC, especially into unknown waters. The next section describes the process of migration, which service and why it was chosen as the first service to be migrated, and finally the results of the migration.

4.4 Migration process

The company used the development of a new version of a certain service as an opportunity to integrate a gRPC technology. While refactoring the service and preparing a new version of the service called conveyance written in Golang, the gRPC was integrated and new API calls of this service are provided also in form of gRPC procedures. As mentioned above, as the first candidate for the gRPC adoption, the company chose a service written in Golang. It was not a coinci- dence, as it is the most characteristic language for the gRPC, because both the gRPC as well as Golang, were developed by Google.

4.4.1 Proto buffers structure

As the very first thing, developers Matouš Dziviak and Ivan Boyarkin wrote an RFC document to propose unified handling of the protocol buffer files and their usage for service communications. After technical discussions, we decided to adopt an idea of one shared common proto repository. Every team should include their proto files in the common repository, as is shown in figure 4.1. The repository hasa descriptive README file, which serves as a guide on how to add proto

64 4. Case study in Kiwi.com

definitions of a new team into the repository, how to generate and publish generated code and provide links for the best practices.

Figure 4.1: Proto repository structure. Source: Kiwi.com by Matouš Dziviak

The auto-generated client code is available in form of packages in all the supported programming languages. Currently, three programming languages are supported – Golang, C++, and Python. Flow is automated in a way, that clients do not have to generate the code locally, all they need to do is to import the relevant files from particular package. Here is an example of import in Python: from kiwi.platform.protos.v1 import EchoApiStub

65 4. Case study in Kiwi.com

Once the merge request with the service specification is reviewed, approved and all pipelines succeeded, the merge request can be merged to the master branch, which afterward generates and publishes all the new proto files. Additionally, the Swagger schema and the gRPC gateway are generated as well, so it is easily possible to add the HTTP/JSON API on top of the gRPC. In case there are some critical or breaking changes, a new package version is created to not break the client’s functionality. Otherwise, there is a risk of a production outage. Smaller changes, like adding a new parameter, changing doc, or adding a new enum value can be handled within the same package, as they are not breaking anything.

4.4.2 Proto buffers linter

In Kiwi.com we have decided to follow the Uber protocol buffer best practices [30]. All new files in the common proto repository have to follow agreed standards. To secure it, we use the linter to check every new change in a repository. The linter can be run locally during the development (make lint), but it is also a part of the CI pipelines. In case the pipeline fails, it is not possible to merge new changes into the master branch.

4.4.3 gRPC gateway

As the gRPC integration into our services is still at the very beginning, migrated services by default support the HTTP/JSON requests altogether with the gRPC request. This is handled with the gRPC gateway, and as described above, the proto repository automatically adds the gRPC gateway option into the new service. We have decided to support both options, as we do not want to force all the clients to integrate the gRPC immediately. We prefer gradual adoption so that the teams can get used to it and learn new technology properly, not in rush and under pressure. Once all clients of a particular service switch to the gRPC, the support for the HTTP/JSON can be dropped.

66 4. Case study in Kiwi.com

4.4.4 Error handling

For error handling and logging of errors, we use the interceptors. The interceptor catches errors returned by the handler and tries to set up the proper gRPC status, before returning the error response to the client. For unknown errors, it returns the unexpected error. Another interceptor logs errors, so they can be examined later if needed. Some errors which are not relevant for us are skipped from logging (e.g. StatusBadRequest).

4.4.5 Authentication

Currently, there are only few services running the gRPC server and they are not exposed to the outer world, therefore no special security restrictions are needed. Services are running on an internal network, and it is not possible to access them from any external IP addresses. However, we want to protect our endpoints at least in some way, therefore we use the simple authorization tokens and user agents to distinguish the single clients. For testing and staging servers (outside internal IP addresses) we use TLS authentication in combination with VPN.

4.4.6 Load balancing

Incoming traffic on the gRPC services is not very high yet. Some ofthe services provide only new versions of APIs via gRPC and HTTP/JSON communication is still supported. There are only a few clients which already switched to the gRPC APIs. Therefore, traffic is currently handled over server-side load balancer with Nginx, which does not differ much from our HTTP/JSON traffic load balancer. However, when more clients will switch to the gRPC and start to generate bigger gRPC traffic, most probably the Linkerd by Kubernetes [36] will be our choice of how to handle gRPC load balancing. It is a safe and comfortable option, as our services are running on Kubernetes. Another option will be considered only in case the Linkerd will not be sufficient, which is very unlikely.

67 4. Case study in Kiwi.com

4.4.7 Monitoring

The interceptors are a really powerful tool and we use them also to add various data metrics into our monitoring service, which is the Datadog. The gRPC allows to create a connection with various options and one of them is to add a tracker to log metrics like the number of new connections, incoming and outgoing messages, current connection count, and many more. We used those metrics to prepare dashboards to monitor the gRPC activity of service. The dashboard of one of our services is shown in figure 4.2.

Figure 4.2: gRPC metrics of one anonymized service

4.5 Experience report

We have managed to successfully integrate the gRPC into first services, which are already running on production and have a couple of internal clients. Even though the theory and guidelines are pretty straightforward and descriptive, practical execution is a bit more complicated. The guidelines describe all the key concepts, best practices, and general recommendations, and helped us establish a working

68 4. Case study in Kiwi.com

migration process. However, during the actual implementation, we also discovered a couple of pain points of the gRPC.

4.5.1 What went well

The first challenging part was to learn and understand the concept of the protocol buffers and their structure. We spent some time brain- storming about the best way how to share the proto files and make generated code easily available for the clients. In the end, the final solution is very promising and automates a lot of processes, using the custom scripts and our own CI setup. This concept of a common proto repository looks very promising and should slowly start to unify common objects between all the teams and in doing so improve the overall data quality. Another positive the gRPC brought, is the performance boost. The most significant improvement was detected in the data marshaling/unmarshaling, which also reduced the data volume transferred between the services. This shows that all the main benefits we were expecting from gRPC integration were achieved - stronger API contract, performance boost and easier client integration (which now works in form of package and library imports).

Figure 4.3: gRPC(blue) vs HTTP(red) execution time comparison

While a literature describes that gRPC is roughly 7-10 faster than REST, we did not achieve such a significant improvement. The gRPC outscales the REST while sending large payloads, but with a smaller payload, there is not such a big difference. In one of the most called

69 4. Case study in Kiwi.com

endpoints of the migrated service, the gRPC execution time is twice as fast as the REST (mainly due to HTTP/2 and faster marshaling/unmarshaling of the payload messages), as shown in figure 4.3. Even though it is not 7 times faster, we are happy with such an improvement.

4.5.2 What went wrong

The unpleasant surprise was the bad quality of the auto-generated code. Even though the auto-generated code does not have to be modified or change anyhow, it looks a bit ugly and chaotic. Also, theIDEs have a problem to recognized this imported code, and also whispering is not working properly, which makes the development a bit more complicated. Another annoying thing is not unified handling of the proto com- pilers. While Golang and C++ use protoc compiler, Python uses python specific solution, by importing compiler from grpc_tools package. It makes it complicated to write custom scripts for automation, as it is necessary to handle the generation of code in multiple ways. Despite the gRPC is a framework and not an architectural style as the REST, it still contains a huge amount of different setups and options on how to do things. It would not be so bad if half of them were not deprecated and the second half experimental. They are constantly changing and it is difficult to follow what is the actual best option atany given moment. It may appear that Google’s developers making things according to their current use case, and do not care that much about the rest of the community. This theory is unfortunately supported by the fact, that there are plenty of opened issues on GitHub, which are several years old.

4.5.3 What is next

By integrating the gRPC into first services we have discovered a few pain points and the price for adopting new technology. There is still a lack of developer support and a lot of issues are not resolved. It will probably take a year or two till the gRPC will gain more popularity and proper support. However, we have also discovered the potential which gRPC provides. It is definitely an interesting technology

70 4. Case study in Kiwi.com

and a good way how to handle the internal communication between microservices. We do not plan to force the migration from the REST into the gRPC in all our services. There are services where the REST is a better solution, especially the one which are exposed also to the outer world and are integrated by our vendors and partners. For fully internal services, where performance matters or there is an option to support streaming communication, we will continue with the gRPC adoption. The next plans are to improve the way how our common proto repository works. We will try to automate the flow even more and provide better support for other teams, to motivate them for the gRPC adoption. This can possibly be achieved by providing more predefined code for common functionality, like interceptors for logging, error handling, or monitoring.

Hopefully, the whole gRPC ecosystem will improve, especially from the Google side, and gRPC will become a standard for internal communication in the microservices architecture. Otherwise, it is possible that more alternative technologies such as the dRPC [38] will be developed in the future. dPRC is basically an open-source lightweight copy of gRPC, which was developed in opposition to Google’s wild development of gRPC, focused primarily on their use case. The goal of dRPC was to take only the good aspects of the gRPC and create their own copy. It will be exciting to see what will be the future evolution of the gRPC.

71 5 Conclusion

The thesis introduces the gRPC technology, necessary theory, architecture and technology ecosystem, describes gRPC use case in the production environment, and compares advantages and differences with REST API design. The first chapter contains definitions of important methodologies, like microservices, containerization, REST API, its usage, and development challenges. This chapter also describes gRPC technology itself, its mechanism, and basic functionality. And in last part mentioning two examples of technical migrations. The second chapter listing all advantages and disadvantages of gRPC and compare it with the REST, following by a description of the optimal architecture for gRPC. The main part of the second chapter is a decision process that should help companies to decide when to consider gRPC as an alternative for internal communication between microservices. The chapter ends with a proposal of high-level guidelines on how to manage the gRPC integration. The last chapter describes how the decision and the migration have been handled in technology travel company Kiwi.com. The decision process helped to convince engineers in Kiwi.com to integrate gRPC technology into their system, and following the described guidelines company managed to integrate the gRPC into the first services, without negative effects on the production environment.

The last chapter contains also an experience report, describing what went well, what went wrong, and Kiwi.com’s next steps regarding gRPC technology. Migration was successful and the first services already communicate via gRPC. It brings several improvements: performance boost, communication over gRPC is faster than REST, services have strict API contract with each other, so there are no unexpected data inconsistencies, and integration of new services on client-side is pretty easy and quick. Despite the fact that in theory, everything looks good, the actual implementation discovered a few hidden and unpleasant aspects of gRPC technology. The whole gRPC environment is still a bit chaotic which is partially caused by the immaturity of the technology and

72 5. Conclusion partially by weaker support from Google as a main gRPC contributor. There is plenty of unresolved issues, a lot of options in gRPC are experimental or deprecated and Google develops this technology to suit primarily their use cases. This observation does not have to be obvious at first glance, without going deeper into production implementation. gRPC is a technology that is not suitable for every architecture and environment, and there is no point to heedlessly reject REST and migrate everything into gRPC. REST architecture is, and for upcoming years definitely will be the number one solution for service communication for many. The main reason is its popularity, support and simplicity. Therefore, it is important to pay attention especially to a decision process, and consider all pros and cons of both technologies before the migration. There is no doubt that gRPC has a huge potential and its adoption brings a lot of advantages, but at the end, it would be good to consider waiting another year or two, until the gRPC community will grow bigger, which should as result provide a better developers’ support and easier integration of gRPC.

Kiwi.com will continue with the gradual adoption of gRPC, where it makes sense. The company will try to automate and improve already existing solutions and motivate other teams to try this promising technology.

73 Bibliography

1. GOOGLE. gRPC. 2021. Available also from: https://grpc.io/. 2. DRAGONI, Nicola; GIALLORENZO, Saverio; LLUCH- LAFUENTE, Alberto; MAZZARA, Manuel; MONTESI, Fabrizio; MUSTAFIN, Ruslan; SAFINA, Larisa. Microservices: yesterday, today, and tomorrow. CoRR. 2016, vol. abs/1606.04036. Available from arXiv: 1606.04036. 3. BALALAIE, Armin; HEYDARNOORI, Abbas; JAMSHIDI, Pooyan. Migrating to Cloud-Native Architectures Us- ing Microservices: An Experience Report. CoRR. 2015, vol. abs/1507.08217. Available from arXiv: 1507.08217. 4. LINTHICUM, David S. Cloud-Native Applications and Cloud Migration: The Good, the Bad, and the Points Between. IEEE Cloud Computing. 2017, vol. 4, no. 5, pp. 12–14. Available from doi: 10.1109/MCC.2017.4250932. 5. NEWMAN, S. Monolith to Microservices: Evolutionary Pat- terns to Transform Your Monolith. O’Reilly Media, 2019. isbn 9781492047810. Available also from: https://books.google. cz/books?id=nNa%5C_DwAAQBAJ. 6. NEWMAN, Sam. Migrating Monoliths to Microservices with Decom- position and Incremental Changes [online]. infoq, 2021-02-09 [visited on 2021-02-24]. Available from: https://www.infoq.com/ articles / migrating - monoliths - to - microservices - with - decomposition/. 7. NEWMAN, S. Building Microservices: Designing Fine-Grained Sys- tems. O’Reilly Media, 2015. isbn 9781491950333. Available also from: https://books.google.cz/books?id=jjl4BgAAQBAJ. 8. DUA, Rajdeep; RAJA, A Reddy; KAKADIA, Dharmesh. Virtu- alization vs Containerization to Support PaaS. In: 2014 IEEE In- ternational Conference on Cloud Engineering. 2014, pp. 610–614. Available from doi: 10.1109/IC2E.2014.41. 9. DOCKER, INC. Docker. 2021. Available also from: https://www. docker.com/.

74 BIBLIOGRAPHY

10. GIARETTA, Alberto; DRAGONI, Nicola; MAZZARA, Manuel. Joining Jolie to Docker - Orchestration of Microservices on a Containers-as-a-Service Layer. CoRR. 2017, vol. abs/1709.05635. Available from arXiv: 1709.05635. 11. WANG, Suping; ZHU, Ligu; CHENG, Mengke. Docker-based Web Server Instructional System. In: 18th IEEE/ACIS International Conference on Computer and Information Science, ICIS 2019, Beijing, China, June 17-19, 2019. IEEE, 2019, pp. 285–289. Available from doi: 10.1109/ICIS46139.2019.8940219. 12. GOOGLE. Kubernetes. 2021. Available also from: https : //kubernetes.io/. 13. MAO, Ying; FU, Yuqi; GU, Suwen; VHADURI, Sudip; CHENG, Long; LIU, Qingzhi. Resource Management Schemes for Cloud- Native Platforms with Computing Containers of Docker and Ku- bernetes. CoRR. 2020, vol. abs/2010.10350. Available from arXiv: 2010.10350. 14. FIELDING, R. Hypertext Transfer Protocol – HTTP/1.1. June 1999, 1999-06. RFC, 2616. RFC Editor. Available also from: https:// tools.ietf.org/html/rfc2616. 15. HARNISH, Brian. What Is HTTPS: The Definitive Guide to How HTTPS Works [online]. Boston: SEMRUSH, 2020 [visited on 2021- 01-04]. Available from: https://www.semrush.com/blog/what- is-https/. 16. STENBERG, Daniel. HTTP2 explained. Comput. Commun. Rev. 2014, vol. 44, no. 3, pp. 120–128. Available from doi: 10.1145/ 2656877.2656896. 17. Usage statistics of HTTP/2 for websites [https://w3techs.com/technologies/details/ce- http2]. [N.d.]. Accessed: 2021-01-04. 18. BÜLTHOFF, Frederik; MALESHKOVA, Maria. RESTful or RESTless - Current State of Today’s Top Web APIs. CoRR. 2019, vol. abs/1902.10514. Available from arXiv: 1902.10514. 19. FENG, Xinyang; SHEN, Jianjing; FAN, Ying. REST: An alternative to RPC for Web services architecture. In: 2009 First International Conference on Future Information Networks. 2009, pp. 7–10. Avail- able from doi: 10.1109/ICFIN.2009.5339611.

75 BIBLIOGRAPHY

20. DAIGNEAU, R. Service Design Patterns: Fundamental Design Solu- tions for SOAP/WSDL and RESTful Web Services. Addison-Wesley, 2012. Addison-Wesley signature series Service design patterns. isbn 9780321544209. Available also from: https://books.google. cz/books?id=tK3%5C_vB304bEC. 21. UPADHYAYA, Bipin; ZOU, Ying; XIAO, Hua; NG, Joanna; LAU, Alex. Migration of SOAP-based services to RESTful services. 2011, pp. 105–114. Available from doi: 10.1109/WSE.2011.6081828. 22. THURLOW, R. RPC: Remote Procedure Call Protocol Specification Version 2. May 2009, 2009-05. RFC, 5531. RFC Editor. Available also from: https://www.rfc-editor.org/rfc/rfc5531. 23. BIRRELL, Andrew D.; NELSON, Bruce Jay. Implementing Re- mote Procedure Calls. ACM Trans. Comput. Syst. 1984, vol. 2, no. 1. issn 0734-2071. Available from doi: 10.1145/2080.357392. 24. GOOGLE. Language guide proto3. 2021. Available also from: https: //developers.google.com/protocol-buffers/docs/proto3. 25. KAUR, Gurpreet; FUAD, Mohammad Muztaba. An evaluation of Protocol Buffer. In: Proceedings of the IEEE SoutheastCon 2010 (SoutheastCon). 2010, pp. 459–462. Available from doi: 10.1109/ SECON.2010.5453828. 26. INDRASIRI, K.; KURUPPU, D. gRPC: Up and Running: Building Cloud Native Applications with Go and Java for Docker and Kubernetes. O’Reilly Media, 2020. isbn 9781492058281. Available also from: https://books.google.cz/books?id=8c3LDwAAQBAJ. 27. FERNANDO, Ruwan. Evaluating Performance of REST vs. gRPC [online]. 2019-04-03 [visited on 2021-03-24]. Available from: https://medium.com/@EmperorRXF/evaluating-performance- of-rest-vs-grpc-1b8bdf0b22da. 28. GOOGLE. Grpc gateway. 2021. Available also from: https:// github.com/grpc-ecosystem/grpc-gateway. 29. GOOGLE. Style Guide. 2021. Available also from: https : //developers.google.com/protocol-buffers/docs/style. 30. UBER. Uber Protobuf Style Guide V2. 2021. Available also from: https : / / github . com / uber / prototool / blob / dev / style / README.md.

76 BIBLIOGRAPHY

31. UBER. Prototool linter. 2021. Available also from: https://github. com/uber/prototool. 32. HUMPHRIES, Joshua. grpcurl. 2021. Available also from: https: //github.com/fullstorydev/grpcurl. 33. OPEN SOURCE COMMUNITY. Evans. 2021. Available also from: https://github.com/ktr0731/evans. 34. GENCO, Alex. Executing gRPC Client Retries in Ruby [online]. 2021-03-30 [visited on 2021-04-05]. Available from: https:// onesignal.com/blog/executing-grpc-client-retries-in- ruby. 35. GOOGLE. gRPC Load Balancing [online]. 2017-06-15 [visited on 2021-04-06]. Available from: https : / / grpc . io / blog / grpc - load-balancing. 36. BUOYANT INC. Linkerd. 2021. Available also from: https:// linkerd.io/2.10/overview. 37. AWS. Application Load Balancer. 2021. Available also from: https: / / aws . amazon . com / blogs / aws / new - application - load - balancer-support-for-end-to-end-http-2-and-grpc. 38. STORJ LAB. dRPC. 2021. Available also from: https : / / www . storj.io/blog/introducing- drpc- our- replacement- for- grpc.