DOCSLIB.ORG

Singularity Containers for HPC

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Singularity Containers for HPC Singularity containers for HPC Westgrid Webinar, Feb. 5, 2020 Dr. Grigory Shamov, HPC Specialist, Site Lead UManitoba/WG 1 How to deal with software complexity? ● Controlling software dependencies ● Configuring software packages ● Distributing software (in a portable way). ● High Performance Computing: ○ Reproducibility, Mobility of computing? ● One of the approaches is to isolate/containerize software/apps together with their dependencies 2 Outline ● What are containers (In Linux) ● Containers in Enterprise and High-Performance Computing ● Singularity basics, new capabilities in Singularity 3.x ○ Building containers from recipes ○ Pulling from DockerHub ○ Singularity libraries and builders (SHub, Sylabs) ○ NVidia GPUs and Singularity ○ MPI parallel codes with Singularity ● Singularity use cases, best practices 3 Glossary Operating system - All the software that let you interact with a computer, run applications, UI, etc. ; consists of the “kernel” and “userland” parts. Kernel - Central piece of software that manages hardware and provides resources (CPU, IO, memory, filesystems ) to the processes it is running. Users - Linux separates access for end users, system service accounts and root. Fine grained control (sudo, capabilities). Program and Process - process is an instance of a running program with resources allocated by kernel . Processes associated with users. Daemon process - a process that runs long time, “in the background” 4 Virtual Machines or Containers? What is the difference, and which of the m to use when? 1. VMs offer true isolation , maximal flexibility at performance and space cost (improving a lot recently) a. Can run any combination of host and guest OS b. You can improve performance at cost of losing flexibiity and isolation 2. Containers are an OS-level mechanism of isolating userland parts of OS together with a given application a. Chroot on steroids; namespaces for processes b. Security issues of sharing the same kernel, privilege escalation c. No performance overhead source: Greg Kurtzer keynote at HPC Advisory Council 2017 @ Stanford 5 Containers: Docker use cases Developed as Containers platform for services/daemons. ● Enterprise computing, Microservices approach ○ Load balancers, oversubscription ○ Orchestration tools (Google Kubernetes) ○ Many containers per node, oversubscription ● Runs as root or a service user; cgroups for RM ● Uses commodity, Internet network stack extensively Very popular with software developers; thanks to Recipes and the huge registry at DockerHub (https://hub.docker.com) . Made its way to Research Computing software dev.! 6 Containers: HPC use case ● In HPC world jobs are ran in the batch mode: queue, start, run, end cycle. ● Often long running compute, large state (memory, data on disk). ● Often a whole node or many nodes per job, statically allocated. ○ Worst case scenarios, no CPU oversubscription possible. ● Close access to hardware for speed; specialized interconnects, RDMA, direct GPU access, zero-copy IPC ● Shared systems, pretend to be a single large machine with RM Scheduler, shared network FS, users connecting directly on the machine. 7 Can I run Docker on HPC? Short answer is often no. ● Security model: no root or sudo is given on shared machines ● Cgroups resource management will conflict with HPCs RM But users really want it? Containers for HPC: Shifter; Singularity; CharlieCloud -- either based on Docker or inter-operable with it; running containers in user space, as a user. ● Zero performance overhead for either of them (an SC19 paper). 8 Singularity Created first at LBNL, now developed by a company (SyLabs) https://sylabs.io/ Mobility of compute, reproducible research with Containers. Developed for HPC use case: runs as a regular user, can access shared filesystems. Interoperable with Docker; can run services as well. Supported on ComputeCanada’s HPC machines. Many updates with version 3. (switch to Golang, new container formats, new build services, overlays, etc. -- will talk today!) 9 Singularity : getting started If you have a) Singularity container image, and b) Singularity runtime installed, you can run your app in the container in thes. ./my-app [options] my-input.dat [singularity command] [singularity-options] ./container.sif [options] input.dat For example, singularity run ./lolcow.sif singularity exec -B /scratch:/scratch ./R-INLA.sif R Benchmark.R 10 Singularity : getting the runtime On ComputeCanada HPC machines: module spider singularity; ml singularity/3.5 With OS package manager: apt-get install singularity-container (old versions on Ubuntu 18.04+) or yum install singularity (EPEL, has a recent version). Build from sources: see documentation at SyLabs. (https://sylabs.io/guides/3.5/admin-guide/installation.html) For CentOS 7: install dependencies; sudo yum groupinstall -y 'Development Tools' && sudo yum install -y \ openssl-devel libuuid-devel libseccomp-devel wget squashfs-tools cryptsetup Download the sources: wget \ https://github.com/sylabs/singularity/releases/download/v3.5.2/singularity-3.5.2.tar.gz rpmbuild -ta singularity-3.5.2.tar.gz; find the RPMS and install: sudo yum localinstall singularity-*.rpm 11 How to get Singularity container images? Build your own; pull others’ images; build on top of othersimages Lets start with pulling a small image. (Things to consider on CC systems: network and FS performance; memory and threads to pack/unpack the container). singularity pull is the command. https://hub.docker.org is the Registry. singularity pull lolcow1.sif docker://godlovedc/lolcow Or running it right away: singularity run docker://godlovec/lolcow:latest ; or opening a shell in the container! ● Images are cached , under $HOME/singularity ● Docker layers are cached too. singularity pull docker://jfloff/alpine-python:2.7 ● We can control cache location with SINGULARITY_CACHE environment (and SINGULARITY_TMP) ● singulatity cache {list|clean} commands to manage cache. ● singularity inspect shows the image’s metadata ; --runscript, --deffile opttions for SIF images are useful. 12 Where to get Singularity container images? Pulling containers Public containers repositories from where to “pull” or build containers. ● DockerHub public registry. docker:// ● SyLabs public library (new). library:// ● SingularityHub. shub:// Private repositories ● Require authentication. singularity pull --docker-login docker://your-private-repo/container:tag CVMFS distributions! They distribute container images in an unpacked directory format. CVMFS handling various optimizations and caching of this format. ● OpenScienceGrid: https://opensciencegrid.org/docs/worker-node/install-singularity/ 13 Building Singularity container images Building might require sudo, because some OS parts belong to root Container formats in Singularity 3.x ● SIF: the default, read-only, compressed SquashFS image. Can be signed. Can be encrypted (new in 3.x) ● Sandbox: a directory tree. Writable. Use --sandbox option to commands that build containers. ● Overlay: read-only SIF formats can be enhanced by a writable ext3 layed (new , experimental, requires kernel 3.18+) Building from recipes and/or other containers with the singularity build command ● singularity build {target} {source} . ● singularity build alpine.sif alpine-dir/ ; singularity build alpine.sif Singularity ; singularity build alpine.sif docker://jfloff/alpine-python:2.7 , etc. ● Interactive container development: the singularity shell command. But finalize with a recipe! 14 Bootstrap: docker From: rocker/r-ver:latest Singularity recipes %post apt-get update apt-get install -y libssl-dev libsasl2-dev jags 1. Specifies from where to Bootstrap autoconf automake apt-get install -y curl wget libudunits2-dev bash from something (OS repo, docker, libicu-dev libeigen3-dev etc.) apt-get install -y gcc-multilib g++-multilib # generic R packages R -e "install.packages('ggplot2')" 2. Modifies the container in %post, # skipped a few packsges # copies %files R -e "install.packages('R2jags')" #R2OpenBUGS wget 3. Sets the %environment http://pj.freefaculty.org/Ubuntu/15.04/amd64/openbugs/openbugs _3.2.3.orig.tar.gz 4. Defines entry point in %runscript tar xzf openbugs_3.2.3.orig.tar.gz cd openbugs-3.2.3 ./configure More examples at Github make && make check && make install R -e "install.packages('R2OpenBUGS')" (https://github.com/sylabs/singularity/blob/master/examples) 15 Remote building services / libraries What if you do not have sudo anywhere? How do you distribute your Sing. images? The original build service: V. Sochat’s SingularityHUB ● Link your Github repo with recipe to https://singularity-hub.org ; wait for it to build; setup auto rebuild hooks. ● Pull the container from anywhere like so (putting your URI of course): singularity pull shub://vsoch/hello-world ● Right now locked down due to an abuse by a malicious user; limits are set to the number of pulls. The new SyLabs Cloud service. ● Register with an identity provider (Google, FB, MS, Github) at https://cloud.sylabs.io ; Get an access token. ● Either use --remote build option from a local Singularity installation or deposit a recipe ● Pull the container from anywhere with singulatity pull library://your-name/aproject/your-image:tag 16 Running containers on HPC machines Access to filesystems using --bind | -B options. ● Bind-mount is a Linux kernel mechanism. -B outside:inside ● Some paths are mounted by default. /home , /tmp, … ● Sysadmin can configure more/less paths by default ● You can prevent mounting paths by --containall (other?) options singularity exec -B /sbb/scratch:/global/scratch
Load more

Recommended publications
  • Integrating On-Premises Core Infrastructure with Microsoft Azure
    Course 10992 • Microsoft Azure Integrating On-Premises Core Infrastructure with Microsoft Azure Length This 3-day, instructor-led workshop covers a range • 3 days of components, including Azure Compute, Azure Audience Storage, and network services that customers can • IT professionals who have used on- benefit from when deploying hybrid solutions. In premises virtualization technologies, including both this context, the term hybrid means integrating Hyper-V and VMware platforms, but who want to deploy, configure, infrastructure technologies that customers host in and administer services and virtual on-premises datacenters with Azure IaaS and PaaS machines in Azure • IT professionals who have used services. This course offers an overview of these Microsoft System Center to services, providing the knowledge necessary to manage and orchestrate an on- premises server infrastructure design hybrid solutions properly. It also includes • Windows and Linux administrators who are looking to evaluate and several demonstrations and labs that enable migrate on-premises workloads students to develop hands-on skills that are and services to the cloud • IT professionals who need to necessary when implementing such solutions. implement network connectivity between on-premises environments and services that Workshop Outline Azure or Microsoft Office 365 hosts • IT professionals who want to use Module 1: Introduction to Microsoft Azure Azure to increase the resiliency and • Overview of cloud computing and Azure agility of their on-premises • Overview of
    [Show full text]
  • Industrial Control Via Application Containers: Migrating from Bare-Metal to IAAS
    Industrial Control via Application Containers: Migrating from Bare-Metal to IAAS Florian Hofer, Student Member, IEEE Martin A. Sehr Antonio Iannopollo, Member, IEEE Faculty of Computer Science Corporate Technology EECS Department Free University of Bolzano-Bozen Siemens Corporation University of California Bolzano, Italy Berkeley, CA 94704, USA Berkeley, CA 94720, USA fl[email protected] [email protected] [email protected] Ines Ugalde Alberto Sangiovanni-Vincentelli, Fellow, IEEE Barbara Russo Corporate Technology EECS Department Faculty of Computer Science Siemens Corporation University of California Free University of Bolzano-Bozen Berkeley, CA 94704, USA Berkeley, CA 94720, USA Bolzano, Italy [email protected] [email protected] [email protected] Abstract—We explore the challenges and opportunities of control design full authority over the environment in which shifting industrial control software from dedicated hardware to its software will run, it is not straightforward to determine bare-metal servers or cloud computing platforms using off the under what conditions the software can be executed on cloud shelf technologies. In particular, we demonstrate that executing time-critical applications on cloud platforms is viable based on computing platforms due to resource virtualization. Yet, we a series of dedicated latency tests targeting relevant real-time believe that the principles of Industry 4.0 present a unique configurations. opportunity to explore complementing traditional automation Index Terms—Industrial Control Systems, Real-Time, IAAS, components with a novel control architecture [3]. Containers, Determinism We believe that modern virtualization techniques such as application containerization [3]–[5] are essential for adequate I. INTRODUCTION utilization of cloud computing resources in industrial con- Emerging technologies such as the Internet of Things and trol systems.
    [Show full text]
  • Kubernetes Security Guide Contents
    Kubernetes Security Guide Contents Intro 4 CHAPTER 1 Securing your container images and CI/CD pipeline 6 Image scanning 6 What is image scanning 7 Docker image scanning open source tools 7 Open source Docker scanning tool: Anchore Engine 8 Securing your CI/CD pipeline 9 Image scanning in CI/CD 10 CHAPTER 2 Securing Kubernetes Control Plane 14 Kubelet security 14 Access to the kubelet API 15 Kubelet access to Kubernetes API 16 RBAC example, accessing the kubelet API with curl 16 Kubernetes API audit and security log 17 Audit log policies configuration 19 Extending the Kubernetes API using security admission controllers 20 Securing Kubernetes etcd 23 PKI-based authentication for etcd 23 etcd peer-to-peer TLS 23 Kubernetes API to etcd cluster TLS 24 Using a trusted Docker registry 24 Kubernetes trusted image collections: Banning non trusted registry 26 Kubernetes TLS certificates rotation and expiration 26 Kubernetes kubelet TLS certificate rotation 27 Kubernetes serviceAccount token rotation 28 Kubernetes user TLS certificate rotation 29 Securing Kubernetes hosts 29 Kubernetes 2 Security Guide Using a minimal host OS 30 Update system patches 30 Node recycling 30 Running CIS benchmark security tests 31 CHAPTER 3 Understanding Kubernetes RBAC 32 Kubernetes role-based access control (RBAC) 32 RBAC configuration: API server flags 34 How to create Kubernetes users and serviceAccounts 34 How to create a Kubernetes serviceAccount step by step 35 How to create a Kubernetes user step by step 37 Using an external user directory 40 CHAPTER 4 Security
    [Show full text]
  • Portability: Containers, Cloud
    JEDI Portability Across Platforms Containers, Cloud Computing, and HPC Mark Miesch, Rahul Mahajan, Xin Zhang, David Hahn, Francois Vandenberg, Jim Rosinski, Dan Holdaway, Yannick Tremolet, Maryam Abdioskouei, Steve Herbener, Mark Olah, Benjamin Menetrier, Anna Shlyaeva, Clementine Gas Academy website http://academy.jcsda.org/june2019 ‣ Instructions for accessing AWS ‣ Activity instructions ‣ Presentation slides ‣ Doxygen documentation for fv3-bundle We will add further content throughout the week Outline I) JEDI Portability Overview ✦ Unified vision for software development and distribution II) Container Fundamentals ✦ What are they? How do they work? ✦ Docker, Charliecloud, and Singularity III) Using the JEDI Containers ✦ How they are built and deployed ✦ Mac and Windows (Vagrant) IV) HPC and Cloud Computing ✦ Environment modules ✦ Containers in HPC? V) Summary and Outlook JEDI Software Dependencies ‣ Essential ✦ Compilers, MPI ✦ CMake Common versions among users ✦ SZIP, ZLIB and developers minimize ✦ LAPACK / MKL, Eigen 3 stack-related debugging ✦ NetCDF4, HDF5 ✦ udunits ✦ Boost (headers only) ✦ ecbuild, eckit, fckit ‣ Useful ✦ ODB-API, eccodes ✦ PNETCDF ✦ Parallel IO ✦ nccmp, NCO ✦ Python tools (py-ncepbufr, netcdf4, matplotlib…) ✦ NCEP libs ✦ Debuggers & Profilers (ddt/TotalView, kdbg, valgrind, TAU…) The JEDI Portability Vision I want to run JEDI on… Development ‣ My Laptop/Workstation/PC ✦ We provide software containers ✦ Mac & Windows system need to first establish a linux environment (e.g. a Vagrant/VirtualBox virtual machine) Development
    [Show full text]
  • Running Legacy VM's Along with Containers in Kubernetes!
    Running Legacy VM’s along with containers in Kubernetes Delusion or Reality? Kunal Kushwaha NTT Open Source Software Center Copyright©2019 NTT Corp. All Rights Reserved. About me • Work @ NTT Open Source Software Center • Collaborator (Core developer) for libpod (podman) • Contributor KubeVirt, buildkit and other related projects • Docker Community Leader @ Tokyo Chapter Copyright©2019 NTT Corp. All Rights Reserved. 2 Growth of Containers in Companies Adoption of containers in production has significantly increased Credits: CNCF website Copyright©2019 NTT Corp. All Rights Reserved. 3 Growth of Container Orchestration usage Adoption of container orchestrator like Kubernetes have also increased significantly on public as well private clouds. Credits: CNCF website Copyright©2019 NTT Corp. All Rights Reserved. 4 Infrastructure landscape app-2 app-2 app-M app-1 app-2 app-N app-1 app-1 app-N VM VM VM kernel VM Platform VM Platform Existing Products New Products • The application infrastructure is fragmented as most of old application still running on traditional infrastructure. • Fragmentation means more work & increase in cost Copyright©2019 NTT Corp. All Rights Reserved. 5 What keeps applications away from Containers • Lack of knowledge / Too complex to migrate in containers. • Dependency on custom kernel parameters. • Application designed for a custom kernel. • Application towards the end of life. Companies prefer to re-write application, rather than directly migrating them to containers. https://dzone.com/guides/containers-orchestration-and-beyond Copyright©2019 NTT Corp. All Rights Reserved. 6 Ideal World app-2 app-2 app-M app-1 app-2 app-N app-1 app-1 app-N VM VM VM kernel VM Platform • Applications in VM and containers can be managed with same control plane • Management/ Governance Policies like RBAC, Network etc.
    [Show full text]
  • Ovirt and Docker Integration
    oVirt and Docker Integration October 2014 Federico Simoncelli Principal Software Engineer – Red Hat oVirt and Docker Integration, Oct 2014 1 Agenda ● Deploying an Application (Old-Fashion and Docker) ● Ecosystem: Kubernetes and Project Atomic ● Current Status of Integration ● oVirt Docker User-Interface Plugin ● “Dockerized” oVirt Engine ● Docker on Virtualization ● Possible Future Integration ● Managing Containers as VMs ● Future Multi-Purpose Data Center oVirt and Docker Integration, Oct 2014 2 Deploying an Application (Old-Fashion) ● Deploying an instance of Etherpad # yum search etherpad Warning: No matches found for: etherpad No matches found $ unzip etherpad-lite-1.4.1.zip $ cd etherpad-lite-1.4.1 $ vim README.md ... ## GNU/Linux and other UNIX-like systems You'll need gzip, git, curl, libssl develop libraries, python and gcc. *For Debian/Ubuntu*: `apt-get install gzip git-core curl python libssl-dev pkg- config build-essential` *For Fedora/CentOS*: `yum install gzip git-core curl python openssl-devel && yum groupinstall "Development Tools"` *For FreeBSD*: `portinstall node, npm, git (optional)` Additionally, you'll need [node.js](http://nodejs.org) installed, Ideally the latest stable version, be careful of installing nodejs from apt. ... oVirt and Docker Integration, Oct 2014 3 Installing Dependencies (Old-Fashion) ● 134 new packages required $ yum install gzip git-core curl python openssl-devel Transaction Summary ================================================================================ Install 2 Packages (+14 Dependent
    [Show full text]
  • Container and Kernel-Based Virtual Machine (KVM) Virtualization for Network Function Virtualization (NFV)
    Container and Kernel-Based Virtual Machine (KVM) Virtualization for Network Function Virtualization (NFV) White Paper August 2015 Order Number: 332860-001US YouLegal Lines andmay Disclaimers not use or facilitate the use of this document in connection with any infringement or other legal analysis concerning Intel products described herein. You agree to grant Intel a non-exclusive, royalty-free license to any patent claim thereafter drafted which includes subject matter disclosed herein. No license (express or implied, by estoppel or otherwise) to any intellectual property rights is granted by this document. All information provided here is subject to change without notice. Contact your Intel representative to obtain the latest Intel product specifications and roadmaps. The products described may contain design defects or errors known as errata which may cause the product to deviate from published specifications. Current characterized errata are available on request. Copies of documents which have an order number and are referenced in this document may be obtained by calling 1-800-548-4725 or by visiting: http://www.intel.com/ design/literature.htm. Intel technologies’ features and benefits depend on system configuration and may require enabled hardware, software or service activation. Learn more at http:// www.intel.com/ or from the OEM or retailer. Results have been estimated or simulated using internal Intel analysis or architecture simulation or modeling, and provided to you for informational purposes. Any differences in your system hardware, software or configuration may affect your actual performance. For more complete information about performance and benchmark results, visit www.intel.com/benchmarks. Tests document performance of components on a particular test, in specific systems.
    [Show full text]
  • USING the ZERTO RED HAT CLUSTER MANAGER ZVR-RHC-5.5U3 Rev 01 Dec2017
    USING THE ZERTO RED HAT CLUSTER MANAGER ZVR-RHC-5.5U3 Rev 01 Dec2017 Using the Zerto Red Hat Cluster Manager, you can protect a Red Hat Cluster that is comprised of two virtual machines sharing a disk and is managed by Red Hat Clustering Services. Using Jenkins infrastructure hosted by Docker on a Linux virtual machine, Zerto Red Hat Cluster Manager periodically checks the status of the active and passive hosts. If they have changed, the Zerto Cluster Manager: ■ Pauses the VPG that contains the newly passive virtual machine, also known as the passive node. ■ Resumes the VPG that contains the newly active virtual machine, also known as the active node. ■ Forces a sync to ensure that the newly active virtual machine, the active node, is fully synchronized with the recovery site. You can also manually perform the tasks that the Zerto Red Hat Cluster Manager performs. This document describes the following: ■ “Zerto Red Hat Cluster Manager Requirements”, on page 1 ■ “Installing and Configuring the Zerto Red Hat Cluster Manager”, on page 2 ■ “Upgrading a Docker Container”, on page 3 ■ “Protecting a Red Hat Cluster with the Zerto Orchestrator”, on page 4 Zerto Red Hat Cluster Manager Requirements ■ The Zerto Red Hat Cluster Manager works with: ■ Zerto Virtual Replication 4.5 U1 and higher. ■ vCenter Server only. ■ Docker 1.10 and higher. ■ The shared disk in the cluster in the recovery site must be defined as an eager zeroed thick disk. Use this disk for preseeding after the VPG is created. ■ One task in the Zerto Red Hat Cluster Manager can work with a maximum of two nodes.
    [Show full text]
  • Kubernetes As an Availability Manager for Microservice Based Applications Leila Abdollahi Vayghan
    Kubernetes as an Availability Manager for Microservice Based Applications Leila Abdollahi Vayghan A Thesis in the Department of Computer Science and Software Engineering Presented in Partial Fulfillment of the Requirements for the Degree of Master of Computer Science at Concordia University Montreal, Quebec, Canada August 2019 © Leila Abdollahi Vayghan, 2019 CONCORDIA UNIVERSITY SCHOOL OF GRADUATE STUDIES This is to certify that the thesis prepared By: Leila Abdollahi Vayghan Entitled: Kubernetes as an Availability Manager for Microservice Based Applications and submitted in partial fulfillment of the requirements for the degree of Master in Computer Science complies with the regulations of the University and meets the accepted standards with respect to originality and quality. Signed by the final examining committee: ________________________________________________ Chair Dr. P. Rigby ________________________________________________ Internal Examiner Dr. D. Goswami ________________________________________________ Internal Examiner Dr. J. Rilling ________________________________________________ Co-Supervisor Dr. F. Khendek ________________________________________________ Co-Supervisor Dr. M. Toeroe Approved by: ___________________________________ Dr. L. Narayanan, Chair Department of Computer Science and Software Engineering _______________ 2019___ __________________________________ Dr. Amir Asif, Dean, Faculty of Engineering and Computer Science ii ABSTRACT Kubernetes as an Availability Manager for Microservice Based Applications Leila
    [Show full text]
  • Immutable Infrastructure, Containers, & the Future of Microservices
    Immutable infrastructure, containers, & the future of microservices Adam Miller Senior Software Engineer, Red Hat 2015-07-25 What we'll cover in this session ● Define “microservices” ● Define “containers” in the context of Linux systems ● Container Implementations in Linux ● What Immutable Infrastructure is – Example of what Immutable Infrastructure deployment workflow looks like ● Red Hat Enterprise Linux Atomic Host – How RHEL Atomic enables and enhances these concepts ● Kubernetes – Orchestrating the Immutable Infrastructure ● OpenShift – Enabling the development and container building pipeline Microservices Microservices are not entirely new. ● The vocabulary term is “new-ish” (2012 – James Lewis and Martin Fowler) ● The idea is very old – Microkernels have existed since the 1980s – Could argue that system admins have been doing this with shell scripts and pipes for years ● Applying this concept to services higher in Monolithic Kernel Microkernel the stack is a newer trend based Operating System based Operating System – Application Heavily influenced by popular technologies System Call such as web microframeworks and containers. user mode VFS IPC, File System Application UNIX Device File IPC Server Driver Server Scheduler, Virtual Memory kernel mode Device Drivers, Dispatcher, ... Basic IPC, Virtual Memory, Scheduling Hardware Hardware What are Microservices? ● Services, “the UNIX Way” – Do one thing, do it well. – Decouple tightly coupled services, make the architecture more modular. ● Loosely coupled services using programming language agnostic APIs for communication – Example: REST APIs The mythical cloud The mythical cloud Micro services Containers What are containers? ● Operating-system-level Virtualization – We (the greater Linux community) like to call them “containers” ● OK, so what is Operating-system-level Virtualization? – The multitenant isolation of multiple user Traditional OS Containers space instances or namespaces.
    [Show full text]
  • Kubernetes As an Availability Manager for Microservice Applications
    Kubernetes as an Availability Manager for Microservice Applications Leila Abdollahi Vayghan Mohamed Aymen Saied Maria Toeroe Ferhat Khendek Engineering and Computer Engineering and Computer Ericsson Inc. Engineering and Computer Science Science Montreal, Canada Science Concordia University Concordia University [email protected] Concordia University Montreal, Canada Montreal, Canada Montreal, Canada [email protected] [email protected] [email protected] Abstract— The move towards the microservice based services that can be deployed and scaled independently by fully architecture is well underway. In this architectural style, small and automated deployment machinery, with minimum centralized loosely coupled modules are developed, deployed, and scaled management [2]. Microservices are built around separate independently to compose cloud-native applications. However, for business functionalities. Each microservice runs in its own carrier-grade service providers to migrate to the microservices process and communicates through lightweight mechanisms, architectural style, availability remains a concern. Kubernetes is often using APIs [3]. Microservices address the drawbacks of an open source platform that defines a set of building blocks which monolithic applications. They are small and can restart faster at collectively provide mechanisms for deploying, maintaining, the time of upgrade or failure recovery. Microservices are scaling, and healing containerized microservices. Thus, loosely coupled, and failure of one microservice will not affect Kubernetes hides the complexity of microservice orchestration while managing their availability. In a preliminary work we other microservices of the system. The fine granularity of this evaluated Kubernetes, using its default configuration, from the architectural style makes the scaling more flexible and more availability perspective in a private cloud settings.
    [Show full text]
  • Docker and Kubernetes: Changing the Opentext Documentum Deployment Model
    White paper Docker and Kubernetes: Changing the OpenText Documentum deployment model Containerization with Docker and Kubernetes’ cloud-first technology is not only a game changer for effectively managing on-premises ™ ™ OpenText Documentum solutions, it also paves the way for deploying EIM solutions in the cloud. Contents New deployment models 3 Customer case study—Part I 3 What is containerization? 4 What are Docker containers? 4 Docker container advantages 5 Available containers 6 What is Kubernetes? 6 Kubernetes advantages 6 Customer case study—Part II 7 EIM in the cloud 7 What is the cloud? 7 Cloud EIM 8 Customer case study—Part III 9 ™ OpenText Managed Services 9 Summary 9 Docker and Kubernetes: Changing the OpenText Documentum deployment model 2/10 New deployment models ™ ™ OpenText Documentum administrators can face two challenges: 1. Effectively managing complex Documentum deployments. Highly customized, mission-critical applications consume disproportionate administrative cycles, budgets and resources to install, upgrade, maintain and enhance. Upgrading these applications requires significant investments in change management. As a result, applications are often not upgraded in a timely fashion and do not leverage the latest technology. 2. Developing a cloud strategy for Enterprise Information Management (EIM) applications. Corporate IT is under intense pressure to produce an enterprise cloud strategy. Leveraging cloud technology for Enterprise Information Management (EIM) applications can be a big win, as long as it does not impact adoption, productivity and governance. Containerization enables new deployment models to help organizations meet these challenges, effectively managing on-premises solutions and paving the way for deploying EIM solutions in the cloud. Customer case study—Part I This real-world customer case study illustrates how containerization can benefit existing Documentum customers.
    [Show full text]