BLUE SKIES Open Issues in Scheduling Microservices in the Cloud Maria Fazio and Antonio Celesti University of Messina he adoption of container-based microservice architec- tures is revolutionizing application design. By adopting Rajiv Ranjan a microservice architecture, developers can engineer Newcastle University applications that are composed of multiple lightweight, Lydia Chen self-contained, and portable runtime components deployed across IBM Research a large number of geodistributed servers. Chang Liu A microservices-based cloud application in- not fully anticipate functional- Newcastle University volves the interoperation of multiple micro services, ities in advance (for example, each developed separately, that can be deployed, up- the types of devices that might Massimo Villari dated, and redeployed independently without com- one day access the applica- University of Messina promising the application’s ecosystem’s integrity. tion). Microservice architec- The ability to independently update and redeploy the tures are a part of a larger shift code base of one or more microservices increases ap- in IT departments towards a plications’ scalability, portability, updatability, and DevOps culture, in which development and opera- availability, but at the cost of expensive remote calls tions teams work closely together to support an ap- (instead of in-process calls) and increased overhead plication over its lifecycle, and go through a rapid or for cross-component synchronization. even continuous release cycle. The microservices approach is in contrast to the Microservices act as standalone application traditional “monolithic” development of applications, subunits or components, implementing specific where each application is a single, autonomous unit. communication protocols for sending and receiv- For example, in a client-server application, the server ing messages. In microservices, data flows through is a monolithic entity that handles HTTP requests, smart endpoints, which also process incoming in- executes logic, and retrieves or updates its data. The formation. Using well-defined interfaces and pro- problem with such monolithic architectures is that tocols, application developers can deploy different even a small modification of the application’s logic microservices on heterogeneous infrastructures requires the deployment of a new running version of without a specific integration framework. Gener- the entire code base. A microservice architecture is ally, microservice communication uses a REST ap- lightweight and easily shipped and updated. Hence, proach based on HTTP and TCP protocols, XMPP, it’s ideal for engineering applications where we can- or JavaScript Object Notation (JSON). However, 2325-6095/16/$33.00 © 2016 IEEE SEPTEMBER/OCTOBER 2016 IEEE CLOUD CoMPUTING 81 BLUE SKIES Guest Guest Guest Guest Guest Guest microservice microservice processes processes microservice microservice Runtime Libs Runtime Libs Runtime Libs Runtime Libs Runtime Libs Runtime Libs Container Container Guest OS Guest OS Container Container Container engine VM VM Container engine Host operating system Hypervisor Host operating system Hypervisor Physical cloud hardware Physical cloud hardware Physical cloud hardware (a) (b) (c) FIGURE 1. Comparison of cloud architectures: (a) hypervisor-based application deployment, (b) hypervisor-free containerized microservice, and (c) containerized microservice within a hypervisor-managed physical host. currently, there are no widely adopted standardized tualization leads to weaker isolation and introduces protocols or data formats for microservice com- greater security vulnerabilities than hypervisor-based munication.1 Microservice deployment and execu- virtualization.4 tion also leads to various networking issues. To this From the user viewpoint, each container looks end, application developers currently adopt various and executes exactly like a standalone operating sys- software-defined networking (SDN) and network tem. Additionally, in a cloud computing scenario, function virtualization (NFV) solutions for network- developers can deploy a higher density of contain- ing microservices. ers (compared to VM density in hypervisor-managed datacenters) on the same physical hardware. Linux Overview of Virtualization Technologies container virtualization (LCV) is the most well- Hypervisor-based resource virtualization (such as known container-based virtualization technology. that used by Xen and VMware) is a key concept in Popular LCV solutions include Docker, LXC, lmct- cloud computing. Hypervisor-based virtualization fy, and OpenVZ. enables cloud providers to create unique virtual ma- Figure 1 shows the key architectural differenc- chines (VMs) that share a set of physical hardware re- es between hypervisor-based and container-based sources (CPU, memory, network, and disk). Each VM virtualization. Figure 1a shows application compo- executes distinct operating system instances (rang- nents deployed within a hypervisor-based VM that ing from proprietary to open source), which supports provides abstraction for full guest operating sys- fault-tolerant and isolated security context behavior. tems (one per VM). Figure 1b shows microservice Container-based virtualization can be used to deployment within a hypervisor-free containerized create microservices.2 A container is a collection environment. Finally, Figure 1c shows microservice of operating system kernel utilities configured to deployment within a containerized environment on manage the physical hardware resources used by a a physical hardware managed by a hypervisor-based particular application component.3 Containeriza- VM. After physical hardware (for example, a server tion allows cloud providers to instantiate, relocate, or appliance), a downward-facing hypervisor is more and optimize hardware resources in a more flexible suitable for managing infrastructure-as-a-service way while providing near-native performance (if de- (IaaS) clouds, whereas containers are more suited for ployed in “hypervisor-free” mode). Because the con- managing platform-as-a-service (PaaS) clouds. Hav- tainers share a single operating system kernel, they ing said that, hypervisor-free containerization isn’t a incur lower overhead.3 However, container-based vir- replacement for traditional hypervisor technologies; 82 IEEE CLOUD CoMPUTING WWW.COMPUTER.ORG/CLOUDCOMPUTING the two technologies complement each other and age core networking functions via software instead must be carefully analyzed during the application of relying on hardware to handle these functions. architecture design phase in terms of performance Creating NFVs using Open Virtual Network isolation, overhead, and security requirements. (OVN) technology guarantees an efficient and se- cure use of the network. OVN complements existing Container Engines for Microservices SDN capabilities, adding native support for virtual Scheduling and Management network abstractions, such as virtual L1 and L2 Several tools can instantiate and manage containers overlays and security groups. OVN also supports the in clouds. Docker Swarm, for example, provides na- security inspection of data transfer inside virtual tive clustering for Docker containers. It turns a pool networks (for example, packet inspection); hence it of Docker hosts into a single virtual Docker host. provides extra features useful for increasing custom- Because Docker Swarm serves the standard Dock- er security and privacy er API, any tool that already communicates with a Docker daemon can use Swarm to transparently Open Issues in Scheduling and Resource scale to multiple hosts. A Docker container manager Management represents the basic container-oriented technology. Despite the clear technological advances in con- Kubernetes is an open-source technology for tainer and hypervisor-based virtualization technol- automating deployment, operations, and scaling of ogies, we are yet to realize a standard large-scale, containerized applications. It groups the containers performance-optimized scheduling platform for making up an application into logical units for easy managing an ecosystem of microservices networked management and discovery—for example, based on together to create a specialized application stack, their resource requirements and other constraints. such as a multitier Web application and Internet of Kubernetes also provides horizontal scaling of ap- Things (IoT) application. Future efforts will focus plications, which can be performed manually or on solving the following research challenges. automatically based on CPU load. Finally, it pro- vides automated rollouts and rollbacks and self- healing features. Configuration Selection and Management Magnum is the OpenStack API service that A cloud application (for example, a multitier Web makes container orchestration engines such as application) must typically combine multiple inter- Docker Swarm and Kubernetes available as first-class dependent microservices that provide diverse func- resources in the OpenStack managed datacenter. tionalities—for example, load balancer, webserver, Magnum uses the Heat service to schedule an operat- and database server. Moreover, these microservices ing system image, which contains Docker and Kuber- have both control and dataflow dependencies. The netes, and runs this image on either VMs or a bare challenges exist in dealing with heterogeneous con- metal cluster. figurations of microservices and cloud datacenter The Google Container Engine provides a com- resources driven by heterogeneous