Container Management

Total Page:16

File Type:pdf, Size:1020Kb

Container Management ALMA MATER STUDIORUM - UNIVERSITÀ DI BOLOGNA SCUOLA DI INGEGNERIA E ARCHITETTURA DIPARTIMENTO DI INFORMATICA – SCIENZA E INGEGNERIA CORSO DI LAUREA MAGISTRALE IN INGEGNERIA INFORMATICA TESI DI LAUREA In COMPUTER NETWORKS M Containerization in Cloud Computing: performance analysis of virtualization architectures CANDIDATO RELATORE: Amedeo Palopoli Chiar.mo Prof. Ing. Antonio Corradi CORRELATORI: Prof. Ing. Luca Foschini Ing. Filippo Bosi Dott. Ing. Stefano Monti Anno Accademico 2016/17 Sessione II Acknowledgements • First and foremost, I would like to thank my supervisors at University of Bologna, Prof. Antonio Corradi and Prof. Luca Foschini, for this fantastic opportunity to work with them and develop this interesting thesis. • I would also like to thank everybody affiliated with Imola Informatica S.p.A who directly or indirectly contributed in this work. In particular, I would like to thank Ing. Filippo Bosi who followed me with dedication. Surely, this experience has been very fundamental for my professional and educational career and so, I cannot forget everyone that gave me the possibility to work in this interesting project. • Moreover, I have to thank everyone who allowed me to realize this little dream. In particular, my parents Carmine and Virginia. Without their support, I could never have completed this important step of my life. • I consider myself a lucky person, because I am surrounded of special people. Of course, one of them is Maria, my girlfriend. You have always been present in every moment, and maybe thanking you is not enough… • Lastly, but not the least, I would like to thank my brother Francesco and my grandmother Annuzza. I am grateful for the various pieces of good advice which have been given that, of course, helped me to grow up easier. Abstract La crescente adozione del cloud è fortemente influenzata dall’emergere di tecnologie che mirano a migliorare i processi di sviluppo e deployment di applicazioni di livello enterprise. L’obiettivo di questa tesi è analizzare una di queste soluzioni, chiamata “containerization” e di valutare nel dettaglio come questa tecnologia possa essere adottata in infrastrutture cloud in alternativa a soluzioni complementari come le macchine virtuali. Fino ad oggi, il modello tradizionale “virtual machine” è stata la soluzione predominante nel mercato. L’importante differenza architetturale che i container offrono ha portato questa tecnologia ad una rapida adozione poichè migliora di molto la gestione delle risorse, la loro condivisione e garantisce significativi miglioramenti in termini di provisioning delle singole istanze. Nella tesi, verrà esaminata la “containerization” sia dal punto di vista infrastrutturale che applicativo. Per quanto riguarda il primo aspetto, verranno analizzate le performances confrontando LXD, Docker e KVM, come hypervisor dell’infrastruttura cloud OpenStack, mentre il secondo punto concerne lo sviluppo di applicazioni di livello enterprise che devono essere installate su un insieme di server distribuiti. In tal caso, abbiamo bisogno di servizi di alto livello, come l’orchestrazione. Pertanto, verranno confrontate le performances delle seguenti soluzioni: Kubernetes, Docker Swarm, Apache Mesos e Cattle. Abstract The increasing adoption of cloud computing is strongly influenced by emerging of technologies whose aim is to improve the development and deployment processes of enterprise applications. The goal of this thesis is to investigate one of these solutions, called “containerization”, and deeply analyze how this solution can be included in cloud infrastructures as an alternative to complementary solutions like virtual machines. So far, the virtual machine model has been the predominant solution. The key differentiator nature that containers offer has stimulated an increasing adoption of this technology because improves resource management, resource sharing, and it guarantees substantial improvements regarding provisioning time of single instances. In this work, we will analyze the containerization paradigm from both infrastructure and application point of views. For the first one, we will investigate the performances by comparing LXD, Docker, and KVM, as hypervisor of OpenStack cloud infrastructure, while the second one concerns the development of enterprise applications that are distributed over a set of server hosts. In this case, we need to exploit high-level services such as orchestration. Therefore, we will compare the performances of the following container orchestrators: Kubernetes, Docker Swarm, Apache Mesos, and Cattle. Summary Summary Introduction ......................................................................................................... 12 1 Virtualization and Containerization .............................................................. 14 1.1 Overview ................................................................................................. 14 1.2 Virtualization in cloud computing .......................................................... 15 1.2.1 Storage virtualization ....................................................................... 15 1.2.2 Network virtualization ...................................................................... 16 1.2.3 Server virtualization ......................................................................... 18 1.3 Types of virtualization with virtual machine .......................................... 19 1.3.1 Full virtualization ............................................................................. 20 1.3.2 Hardware-assisted virtualization ...................................................... 21 1.3.3 Paravirtualization .............................................................................. 22 1.4 Container-based virtualization ................................................................ 23 1.4.1 Linux-VServer .................................................................................. 24 1.4.2 OpenVZ ............................................................................................ 26 1.4.3 LXC .................................................................................................. 28 1.5 Closing remarks ...................................................................................... 30 2 Container Management ................................................................................. 34 2.1 Overview ................................................................................................. 34 2.2 Docker ..................................................................................................... 35 2.2.1 Architecture ...................................................................................... 37 2.2.2 Docker Images .................................................................................. 39 2.2.3 Docker containers ............................................................................. 40 2.2.4 Network ............................................................................................ 43 2.2.5 Storage .............................................................................................. 46 2.2.6 Docker Compose .............................................................................. 47 2.3 RKT ........................................................................................................ 48 2.3.1 Architecture ...................................................................................... 49 5 Summary 2.3.2 Process Model .................................................................................. 50 2.3.3 Network ............................................................................................ 51 2.3.4 Storage .............................................................................................. 52 2.4 LXD ........................................................................................................ 52 2.4.1 Architecture ...................................................................................... 53 2.4.2 Containers ......................................................................................... 54 2.4.3 Snapshots .......................................................................................... 55 2.4.4 Images ............................................................................................... 55 2.4.5 Profiles .............................................................................................. 56 2.4.6 Network ............................................................................................ 56 2.4.7 Storage .............................................................................................. 56 2.4.8 Closing remarks ................................................................................ 57 3 Container Orchestration Engine .................................................................... 60 3.1 Overview ................................................................................................. 60 3.2 The need of Orchestration ...................................................................... 60 3.3 Docker Swarm ........................................................................................ 62 3.3.1 Docker Clustering ............................................................................. 62 3.3.2 Architecture ...................................................................................... 63 3.3.3 Docker Swarm API .........................................................................
Recommended publications
  • Final Report CS 5604: Information Storage and Retrieval
    Final Report CS 5604: Information Storage and Retrieval Integration and Implementation (INT) Team: Alex Hicks, Cherie Poland, Suraj Gupta, Xingyu Long, Yash Mahajan, Mohit Thazhath, Hsinhan Hsieh December 18, 2020 Instructed by Professor Edward A. Fox Virginia Polytechnic Institute and State University Blacksburg, VA 24061 Abstract The first major goal of this project is to build a state-of-the-art information storage, retrieval, and analysis system that utilizes the latest technology and industry methods. This system is leveraged to accomplish another major goal, supporting modern search and browse capabilities for a large collection of tweets from the Twitter social media platform, web pages, and electronic theses and dissertations (ETDs). The backbone of the information system is a Docker container cluster running with Rancher and Kubernetes. Information retrieval and visualization is accomplished with containers in a pipelined fashion, whether in the cluster or on virtual machines, for Elasticsearch and Kibana, respectively. In addition to traditional searching and browsing, the system supports full-text and metadata searching. Search results include facets as a modern means of browsing among related documents. The system supports text analysis and machine learning to reveal new properties of collection data. These new properties assist in the generation of available facets. Recommendations are also presented with search results based on associations among documents and with logged user activity. The information system is co-designed by five teams of Virginia Tech graduate students, all members of the same computer science class, CS 5604. Although the project is an academic exercise, it is the practice of the teams to work and interact as though they are groups within a company developing a product.
    [Show full text]
  • Despliegue Automático De Servicios De Datos Climáticos Con Kubernetes (Automatic Deployment of Climate Data Services with Kubernetes)
    ESCUELA TÉCNICA SUPERIOR DE INGENIEROS INDUSTRIALES Y DE TELECOMUNICACIÓN UNIVERSIDAD DE CANTABRIA Trabajo Fin de Grado Despliegue automático de servicios de datos climáticos con Kubernetes (Automatic deployment of climate data services with Kubernetes) Para acceder al Título de Graduado en Ingeniería de Tecnologías de Telecomunicación Autor: Pablo Celaya Crespo Septiembre - 2019 E.T.S. DE INGENIEROS INDUSTRIALES Y DE TELECOMUNICACION GRADUADO EN INGENIERÍA DE TECNOLOGÍAS DE TELECOMUNICACIÓN CALIFICACIÓN DEL TRABAJO FIN DE GRADO Realizado por: Pablo Celaya Crespo Director del TFG: Antonio S. Cofiño González Título: “Despliegue automático de servicios de datos climáticos con Kubernetes” Title: “Automatic deployment of climatic data services with Kubernetes” Presentado a examen el día: 30 de septiembre de 2019 para acceder al Título de GRADUADO EN INGENIERÍA DE TECNOLOGÍAS DE TELECOMUNICACIÓN Composición del Tribunal: Presidente (Apellidos, Nombre): Luis Muñoz Gutiérrez Secretario (Apellidos, Nombre): Alberto Eloy García Gutiérrez Vocal (Apellidos, Nombre): Antonio S. Cofiño González Este Tribunal ha resuelto otorgar la calificación de: ...................................... Fdo.: El Presidente Fdo.: El Secretario Fdo.: El Vocal Fdo.: El Director del TFG (sólo si es distinto del Secretario) Vº Bº del Subdirector Trabajo Fin de Grado Nº (a asignar por Secretaría) Agradecimientos Al Grupo de Meteorología y Computación de Santander, y en particular al director de este trabajo Antonio Cofiño, por darme la oportunidad de colaborar con ellos todo este tiempo y aprender tanto durante el proceso. A todos los miembros del CEDA, especialmente Philip Kershaw y Matt Pryor, por permitirme trabajar con ellos y ayudarme en todo lo necesario. A todos mis profesores del Grado estos últimos tres años, por su trabajo y esfuerzo que me han dado los conocimientos para desarrollar las competencias que me han llevado hasta aquí.
    [Show full text]
  • Analyse Der Sicherheit Und
    Hochschule Bremen Zentrum für Informatik und Medientechnologien Fakultät 4 – Elektrotechnik und Informatik Analyse der Sicherheit und der automatisierten Bereitstellung eines On-Premises-Clusters auf der Grundlage der Container-basierten Virtualisierung: Kubernetes im Wissenschaftsbetrieb Masterarbeit im Studiengang Informatik (KSS) in Kooperation mit dem Alfred-Wegener-Institut, Helmholtz-Zentrum für Polar- und Meeresforschung Autor: Fabian Mangels <[email protected]> Matrikel-Nr.: 5074215 Ort / Datum der Abgabe: Bremen, den 15.09.2020 Erstgutachter: Prof. Dr. Lars Braubach Zweitgutachter: Dipl.-Inf. (FH) Jörg Matthes Betrieblicher Betreuer: Angelo Steinbach Analyse der Sicherheit und der automatisierten Bereitstellung eines On-Premises-Clusters auf der Grundlage der Container- Inhaltsverzeichnis basierten Virtualisierung: Kubernetes im Wissenschaftsbetrieb Inhaltsverzeichnis Danksagung I Zusammenfassung II Abstract III AbkürzungsverzeichnisIV AbbildungsverzeichnisV TabellenverzeichnisVI Auflistungsverzeichnis VII 1 Einleitung 1 2 Grundlagen 4 2.1 Software-Architektur . .4 2.1.1 Monolithen . .5 2.1.2 Microservices ..................................5 2.2 Software-Bereitstellung . .7 2.2.1 Traditionelle Bereitstellung . .9 2.2.2 Virtualisierte Bereitstellung . .9 2.2.3 Container-Bereitstellung . 11 2.3 Dienstkomposition . 12 2.3.1 Orchestrierung . 13 2.3.2 Choreographie . 15 2.4 DevOps ......................................... 15 2.5 Cloud Native ...................................... 16 2.6 Zusammenfassung . 18 3 Komponenten der Infrastruktur
    [Show full text]
  • Container Orchestration: Which Conductor? Containercon Europe, Berlin, Oct 2016
    Container Orchestration: Which Conductor? ContainerCon Europe, Berlin, Oct 2016 Mike Bright, @mjbright Haikel Guemar, @hguemar Mario Loriedo, @mariolet First ... A little bit of history S Unix containers, Unikernels o l e . t . ' s 2 Linux containers (LXC), Mesos f 0 i r 0 s 9 t l o o k LXC ( CloudFoundry, DotCloud PaaS ) a t r e c Docker e 2 n 0 t 1 c 3 Micro-OSes, Swarm, Rkt, LXD o n t a i 2 Container Orchestration Options n 0 e 1 r 4 h @ i s t h o g r u PaaS adoptions of Docker y e . m . a r @ m 2 j 0 b 1 Docker buys Unikernel Systems r 5 i g h t @ 2 Docker "Swarm mode", OCID ... 0 m 1 a 6 r i . o . l e t History μ-OSes Many vendors are developing μ-OSes, small OS (mainly Linux-based) to be the basis for container engine hosts whether they be bare-metal or virtual host machines. They're small, with fast startup, use few resources and have a small attack surface and often "atomic" software updates. OS Vendor CoreOS - (CoreOS) Project Atomic - (RedHat) RancherOS - (Rancher Labs) Photon - (VMWare) Nano Server OS - (Microsoft) Ubuntu Snappy Core - (Canonical) ...Unikernels μ-Services μ- From monoliths to μ- services services Remember when high availability meant this ...? Active Standby App App Servers running monolithic applications in Active- Standby modes, as 1+1, N+1, or N+M or split across 3 tiers. Scaling meant to "scale up" by adding CPU, RAM, disk.
    [Show full text]
  • Elastic Microservice Environment Setup
    Elastic Microservice Environment Setup Julian Hanhart Table of Contents Management Summary . 1 Initial Situation . 2 Netstream AG . 2 TV as a Service Platform . 2 Problem Description . 3 Objectives . 4 Approach . 5 Evaluation of Orchestration Solution . 6 Candidates . 6 Unsuitable Solutions. 6 Evaluation Criteria . 7 Weights . 7 Criteria . 7 Risks . 9 Ratings . 10 Evaluation . 11 Docker Swarm . 11 Kubernetes . 15 Apache Mesos + Marathon . 19 HashiCorp Nomad. 23 Decision Matrix . 27 Ranking . 27 Proposal . 28 Decision . 28 Defining the Base Image. 29 Candidate Distributions . 29 Red Hat Project Atomic . 30 The Canonical Distribution of Kubernetes . 30 CoreOS Container Linux . 30 VMware Photon OS. 30 Tested Distributions . 31 Automated Environment Setup . 32 Target Infrastructure . 32 Kubernetes Basics . 32 Kubernetes Components . 32 Networking . 33 Tools & Technologies . 33 Provisioning . 34 Kubernetes Setup . 34 Implementation . 35 Atomic Host + Ansible Playbook . 35 Kubernetes Anywhere . 37 Photon OS Provisioning . 38 Kubeadm . 41 Manual Kubernetes Setup. 43 Canonical Kubernetes . 44 Next Steps . 45 Service Deployment. 46 Next Steps . ..
    [Show full text]
  • Container Orchestration
    KUBERNETES IS THE STANDARD FOR CONTAINER ORCHESTRATION No matter the data source, Kubernetes is the clear leader in cloud-native orchestration platforms among CIOs, operations and developers alike. As has been learned from past vendor efforts to do so, no one can buy the network effect. All the data points to Kubernetes and containers (Docker, rkt) as having this network effect. Kubernetes’ momentum, market maturity and usage is not matched by any comparable solution as the data of its first year in general availability (captured below) makes clear. In One Year 779 Kubernetes Jobs 7,110 Kubernetes News Articles 4,635 Kubernetes Professionals GITHUB 32,113 140 825 COMMITS RELEASES CONTRIBUTORS TOP 100 TOP 2 TOP .01% FORKED GITHUB PROJECT STARRED GO PROJECT STARRED GITHUB PROJECT WEB SEARCH INTEREST JUN 2015 SEP 2015 DEC 2015 MAR 2016 JUN 2016 KUBERNETES CLOUDFOUNDRY APACHE MESOS DOCKER SWARM How Organizations Leverage Kubernetes Component for self-service distributed cloud platform Container hosting and orchestration that is managed - i.e. a DIY enterprise PaaS by operations (not self-service) Hosting of off-the-shelf applications and frameworks Multicloud orchestration supported by Kubernetes KUBERNETES MESOS/MARATHON DOCKER SWARM CLOUD FOUNDRY Cloud Native Computing Apache None Cloud Foundry Foundation FOUNDATION Foundation (part of Linux Foundation) Apprenda (Kismatic), CoreOS Mesosphere (DCOS) Docker Inc (Swarm) Pivotal (PCF), HPE (Helion) SOFTWARE (Tectonic), Engine Yard (Deis), Red VENDORS Hat (OpenShift)*, Mesosphere (PRODUCT (DCOS)*, Rancher Labs (Rancher)* NAME) *Kubernetes is a component of a larger product PUBLIC CLOUD Google Container Engine, Red Hat A number of public cloud providers, Azure IBM, Predix (GE's IoT PaaS) SERVICE OpenShift, and many more currently including Azure, use Mesos as part of PROVIDERS in development.
    [Show full text]
  • Source Container Orchestration Frameworks
    Technical report A comprehensive feature comparison study of open- source container orchestration frameworks Eddy Truyen *, Dimitri Van Landuyt, Davy Preuveneers, Bert Lagaisse and Wouter Joosen 1 imec-DistriNet, KU Leuven; [email protected] (D.V.L); [email protected] (D.P.); [email protected] (B.L); [email protected] (W.J.) * Correspondence: [email protected]; Tel.: +32-163-735-85 Featured Application: Practitioners and industry adopters can use the descriptive feature comparison as a decision structure for identifying the most suited container orchestration framework for a particular application with respect to different quality attributes such as genericity, maturity and stability. Researchers and entrepreneurs can use it to check if their ideas for innovative products or future research are not already covered in the overall technological domain. Abstract: 1) Background: Container orchestration frameworks provide support for management of complex distributed applications. Different frameworks have emerged only recently, and they have been in constant evolution as new features are being introduced. This reality makes it difficult for practitioners and researchers to maintain a clear view on the technology space. 2) Methods: we present a descriptive feature comparison study of the three most prominent orchestration frameworks: Docker Swarm, Kubernetes and Mesos that can be combined with Marathon, Aurora or DC/OS. This study aims at (i) identifying the common and unique features of all frameworks, (ii) comparing these frameworks qualitatively ánd quantitatively with respect to genericity in terms of supported features, and (iii) investigating the maturity and stability of the frameworks as well as the pioneering nature of each framework by studying the historical evolution of the frameworks on GitHub.
    [Show full text]
  • Inventx 75% 100%
    CASE STUDY Inventx Accelerating Kubernetes Adoption in Finance with Rancher 75% 100% 80% reduction in deployment time increase in deployment frequency reduction in management time Inventx is the Swiss IT partner of choice for leading financial and Introducing insurance service providers. It is the pioneer of the ix.AgileFactory, a full stack infrastructure DevOps platform designed to meet the Inventx unique needs of the Swiss financial community. Founded in 2010, the company is headquartered in Chur, Switzerland, and has more than 260 employees. The ix.AgileFactory portfolio of products allows financial institutions to digitize their technology infrastructure to become more cloud native and microservices-centric. Underpinned by Docker, Kubernetes, Rancher and Red Hat, ix.AgileFactory allows financial institutions to implement microservices and consider hybrid cloud and on-premise deployments to reduce cost. [email protected] | 1 The platform allows the decoupling of core applications from the central infrastructure, enabling organizations to better manage and innovate in applications in safety, without affecting mission- critical systems. Most importantly, ix.AgileFactory is completely technology and environment agnostic; financial customers can choose any variety of software and tooling and host them in any environment — in the data center, in the cloud and at the edge. Like most companies working in the financial space, Inventx The Journey to has a secure, on-premise architecture that, until four years ago, comprised a mix of VM-based IBM architecture and Linux (Red Containers Hat) servers. Due to obvious customer sensitivities, security and compliance have always been major priorities. Consequently, the team has invested heavily in transforming the infrastructure, while maintaining security posture and regulatory compliance.
    [Show full text]
  • Kubernetes Evolves Into an Enterprise Platform
    Market Landscape Kubernetes Evolves Into An Enterprise Platform Author: Tom Petrocelli Market Landscape: Kubernetes Evolves Into An Enterprise Platform October 2020 1 Market Landscape Kubernetes Evolves Into An Enterprise Platform EXECUTIVE SUMMARY Key Stakeholders: CIO, CTO, Vice President/Director/Manager of Platform Engineering, Vice President/Director/Manager of Operations, System Architect, Product Management, Product Marketing Why It Matters: Kubernetes is quickly becoming the base for next generation enterprise microservicesi platforms. Understanding what is happening in the Kubernetes market now and in the near future is vital for planning, training, and design of new enterprise software platforms. Top Takeaway: As Kubernetes-based platforms gain traction, larger clusters that span datacenters, clouds, and geographies are becoming a feature of new software platforms. Much of the focus in the Kubernetes community is about solving problems of scale, especially management of large diverse clusters. This is why the industry is seeing technology and practices that enable the federationii of Kubernetes and related services such as service mesh. Vendors and Organizations Listed in This Report Amazon Lightstep Canonical Microsoft Cloud Foundry Foundation Mirantis Cloud Native Computing Foundation (CNCF) Oracle Datadog Platform9 Google Rancher Labs Hashicorp Red Hat HPE Solo.io Humio Spectro Cloud Layer5 VMware Market Landscape: Kubernetes Evolves Into An Enterprise Platform 2 MARKET CONTEXT The past several years has shown a radical shift in the platforms, tools, and practices that IT employs to develop software. Driven by the need to change and update software more quickly, realize higher uptime and resiliency, increase portability, and bring new levels of scalability to critical systems, new architectures have begun to emerge.
    [Show full text]
  • List of Leading Kubernetes Distributions by Dmitry Netis & Jordan Rupar / 5.21.2020
    List of Leading Kubernetes Distributions By Dmitry Netis & Jordan Rupar / 5.21.2020 Q Advisors assembled a list of prominent Kubernetes software distributions leading the container revolution. A "distribution" is a software bundle that incorporates Kubernetes, as well as container orchestration and management tools. Because Kubernetes is a complex piece of software that is difficult to set up, configure, run and maintain, these distributions offer complete solutions for application hosting and life-cycle management that can be run locally or as a cloud-hosted option. Company Description Apache Mesos is a cluster management tool which can manage container orchestration originally created by Twitter and released as an open source. It is being used by companies like eBay, Airbnb, etc. Mesos is a competitor to Kubernetes. It can be used to manage physical servers and virtual machines (VMs) running workloads, as well as containers. It has an efficient framework called Marathon for deploying and managing Kubernetes containers on a Mesos cluster. Canonical has its own Kubernetes distribution, which hails from the makers of Ubuntu Linux, which also runs underneath. The distribution supports both CPU and GPU workloads. Canonical partnered with Rancher Labs, which is a container management platform that manages multiple Kubernetes distribution clusters. CoreOS Tectonic is one of the most prominent Kubernetes distributions advancing container revolution. The CoreOS operating system (formerly Container Linux) enables updates to OS that can be slipped into product environments without taking down other running applications. CoreOS was acquired by IBM Red Hat in early 2019. Docker is the embodiment of containers. Since 2014, Docker has had its own clustering and orchestration system, Docker Swarm, which, until recently, was a main competitor to Kubernetes.
    [Show full text]
  • Comparing Rancher Orchestration Engine Options
    Comparing Orchestration Engine Options in Rancher October 2016 Usman Ismail ©Rancher Labs 2016. All rights reserved. 1 COMPARING RANCHER ORCHESTRATION ENGINE OPTIONS Contents Introduction ...................................................................................... 3 Docker Native Orchestration .................................................................. 3 Basic Architecture ............................................................................ 3 Usability ....................................................................................... 4 Feature Set .................................................................................... 4 Kubernetes ....................................................................................... 6 Basic Architecture ............................................................................ 6 Usability ........................................................................................ 7 Feature Set .................................................................................... 8 Marathon ....................................................................................... 11 Basic Architecture .......................................................................... 11 Usability ...................................................................................... 11 Feature Set .................................................................................. 12 Final Verdict ...................................................................................
    [Show full text]