KVM, OpenStack, and the Open Cloud Adam Jollans, IBM & Mike Kadera, Intel LinuxCon North America – August 2015
17Aug15 Open Virtualiza on Alliance 1 Agenda
• A Brief History of Virtualiza on • KVM Architecture • OpenStack Architecture • KVM and OpenStack • Case Studies – NTT Com – CERN – Intel IT • Addi onal Resources
17Aug15 Open Virtualiza on Alliance 2 A Brief History of Virtualiza on LXC / Docker
KVM hypervisor
x86 hardware virtualization
Xen hypervisor for x86
VMware hypervisor for x86
Virtualization on Unix systems
Virtualization on mainframes
1960s 1980s 1990s 2000s 2010s 2015 17Aug15 Open Virtualiza on Alliance 3 Conceptual Framework User Interface Applications
Management Tools
Storage Compute Networking
17Aug15 Open Virtualiza on Alliance 4 Introduc on to KVM User Interface Applications
Management Tools oVirt Kimchi libvirt KVM
Storage Compute Networking
17Aug15 Open Virtualiza on Alliance 5 KVM Architecture Open source hypervisor based on Linux
Virtual Virtual KVM Machine Machine • Kernel module that turns Linux into a Virtual Machine Monitor Linux Other Applications Applications • Merged into the Linux kernel
Linux Other QEMU Guest OS Guest OS Linux • Emulator used for I/O device virtualization QEMU QEMU Applications Processors supported KVM Linux • x86 with virtualization extensions • Intel VT-x • AMD (AMD-V) x86, POWER, z Systems, ARM • POWER8 • IBM z Systems • ARM64 17Aug15 Open Virtualiza on Alliance 6 KVM Performance
Source: SPECvirt_2013 Published Results - http://www.spec.org/virt_sc2013/results/specvirt_sc2013_perf.html
17Aug15 Open Virtualiza on Alliance 7 KVM Security
SELinux Virtual Virtual • Mandatory Access Control (MAC) Machine Machine integrated into Linux • Provides “need to know” security Linux Other between processes Applications Applications
sVirt Linux Other Guest OS Guest OS • Combines SELinux and KVM Linux • Delivers “need to know” security QEMU Applications QEMU between virtual machines
KVM Linux Certifications • EAL4+ certification for KVM in RHEL 6 x86, POWER, z Systems, ARM and SLES 11 SP 2 on various x86 64-bit Intel and AMD64-based hardware from Dell, HP, IBM and SGI
17Aug15 Open Virtualiza on Alliance 8 KVM Management - libvirt User Interface
Remote Library Management • Open Source project Command Line • Manages multiple hypervisors
Command Line • Powerful Network libvirt Library • Complex to use Daemon
Network Daemon KVM • Enables remote management Xen LXC Base for other management tools …. • virt-manager, Kimchi, oVirt • OpenStack Compute
17Aug15 Open Virtualiza on Alliance 9 KVM Management - Kimchi User Interface Kimchi • Open Source project HTML5 • Manages KVM on x86, Power Web Browser User Interface • Easy to use Daemon • Access from HTML5 web browser Kimchi libvirt Servers managed • Single digits KVM
Compute
17Aug15 Open Virtualiza on Alliance 10 KVM Futures • Heterogeneous processor support – ARM – POWER – System z – GPUs • Network Func on Virtualiza on • Addi onal Performance Improvements – Minimizing locks – Mul -threaded device model • Nested Virtualiza on • Containers with Virtualiza on
17Aug15 Open Virtualiza on Alliance 11 Building Open Clouds • Security • Resilience • Performance • Scalability – thousands of nodes • Heterogeneity • Interoperability
17Aug15 Open Virtualiza on Alliance 12 Introduc on to OpenStack User Interface Applications
Command Horizon Line Management Tools
Open Ceilometer Keystone Heat Sahara Stack Cinder Swift Glance Nova Neutron Trove
Choice Choice Choice of of of storage hyper- network visor
Storage Compute Networking
17Aug15 Open Virtualiza on Alliance 13 OpenStack Design Principles
• Open – Open Development Model – Open Design Process – Open Community • General Purpose – Balancing Compute, Storage, Network • Massively Scalable • Mul -site • Resilient and recoverable
17Aug15 Open Virtualiza on Alliance 14 Nova – Compute Service Manages VM lifecycle • Starting and stopping VMs • Scheduling and monitoring VMs Command Horizon Line Key Components • API • Database Keystone • Scheduler Swift Glance Nova • Compute node and plug-ins
Authentication VM Choice Images of • Keystone hyper- visor Access to VM images • Glance Storage Compute • Swift
17Aug15 Open Virtualiza on Alliance 15
OpenStack and Hypervisor Usage
Source: OpenStack User Survey May 2015 - http://superuser.openstack.org/articles/openstack-users-share-how-their-deployments-stack-up
17Aug15 Open Virtualiza on Alliance 16 Keystone – Authen ca on Service Manages security • Service for all other modules • Authentication Command • Authorization Horizon Line
Key components • API Keystone • Backends • Token • Catalog • Policy • Identity
17Aug15 Open Virtualiza on Alliance 17 Cinder – Block Storage Service Manages persistent block storage • Provides volumes to running instances • Pluggable driver architecture Command • High Availability Horizon Line
Key components • API Keystone • Queue Cinder • Database • Scheduler • Storage plug-ins Choice of Block Storage Authentication • Keystone
Storage
17Aug15 Open Virtualiza on Alliance 18
Neutron – Networking Service Manages networking connectivity • Provides volumes to running instances • Pluggable driver architecture Command • Support for range of networking technologies Horizon Line
Key components • API Keystone • Queue • Database Neutron • Scheduler • Agent Choice of • Networking plug-ins Network Authentication • Keystone Networking
17Aug15 Open Virtualiza on Alliance 19
Glance – Image Service Manages VM images • Catalog of images • Search and registration Command • Fetch and delivery Horizon Line
Key components • API Keystone • Registry Swift Glance • Database Authentication VM Images • Keystone
Storage of VM images • Swift Storage • Local file system
17Aug15 Open Virtualiza on Alliance 20
Swi – Object Storage Service Manages unstructured object storage • Highly scalable • Durable – three times replication Command • Distributed Horizon Line
Key components • Proxy / API Keystone • Rings Swift • Accounts • Containers • Objects Object Storage • Data stores
Authentication • Keystone Storage
17Aug15 Open Virtualiza on Alliance 21
Provisioning a VM User Interface Applications
Command Horizon Line Management Tools 1 2, 10 Keystone 4 Cinder Swift Glance Nova Neutron 9 8 7 6 3 5
Storage Compute Networking
17Aug15 Open Virtualiza on Alliance 22 OpenStack New Features – Kilo
• Horizon – Updated user interface • Glance – Addi onal ar facts beyond just images • Ironic – Bare Metal Provisioning • Zaqar – Messaging and Queuing System
17Aug15 Open Virtualiza on Alliance 23 KVM and OpenStack
• KVM excels at choice criteria for Hypervisor – Cost – Scale & Performance – Security – Interoperability • Development Affinity – Both open source projects – KVM is default hypervisor for OpenStack development • Deployment Affinity – KVM is best supported, easiest to deploy, with most full-featured driver
17Aug15 Open Virtualiza on Alliance 24 NTT Com’s OpenStack Deployment • NTT Com – Leading global carrier headquartered in Japan – Early adopter of both KVM and OpenStack – Basing one of its public cloud offerings on OpenStack and KVM • NTT involvement – Ac vely involved with the OpenStack and KVM communi es – Con nues to contribute to the development of both projects, with an emphasis on the cloud service provider use case • Use of OpenStack – Flexible plug-in infrastructure used as a unified orchestrator of both compu ng and networking resources – Integrate so ware-defined-networking (SDN)-powered enterprise VPN service, allowing customers to create virtual datacenters that can span two or more physical ones – GUI portal for its cloud services using OpenStack na ve APIs, le ng customers provision and manage virtual machines, networks, and storage without having to know the OpenStack APIs Source: IDC white paper – “KVM – Open Source Virtualization for the Enterprise and OpenStack Clouds” on OVA website 17Aug15 Open Virtualiza on Alliance 25 CERN Private Cloud • CERN – Fundamental research into par cle physics – Large Hadron Collider seeking to find new par cles – Massive need for scalable compu ng resource on demand • CERN Private Cloud – Produc on since July 2013 with OpenStack using KVM, MySQL and RabbitMQ – Currently 3,200 hypervisors with 83,000 cores – Expected to reach over 100,000 cores by 2Q 2015 • Key Requirements – Scale – Technology and Developer ecosystem – Interac on with exis ng IT services
Source: CERN OpenStack public reference on www.openstack.org
17Aug15 Open Virtualiza on Alliance 26 Intel IT’s Cloud Goals
• Pervasive virtualization (> 75%) 80% Effective • > 90% new land in cloud • Enterprise app virtualization Utilization • Secure virtualization Efficiency through federation • Larger pools in fewer data centers
• On-demand self-service the norm Velocity Increase • Provision VMs within minutes Agility through automation & self • Innovative idea to production < day service • External cloud for burst demand
• Reduce MTTR • App design for failure Zero Business Impact • Increase availability
17Aug15 Open Virtualiza on Alliance 27 Intel IT & OpenStack*/KVM Deployment History
Public Public
Initial Deployment – 2012 Today
• OpenStack Essex • OpenStack Havana (Juno • ~1000 virtual instances for upgrade soon) external services • ~4000 instances for mul ple • qemu-system-x86_64 1.0 services (~70:1, ~100 vCPU) • qemu-system-x86_64 1.4.2
17Aug15 Open Virtualiza on Alliance 28 Intel IT & OpenStack*/KVM KVM Benefits Performance Stability • 2012 Study on ‘standard’ cloud workloads (database) • Open Source, ght OpenStack and Linux kernel • Par or be er vs. marketplace integra on • HV realm is seemingly near-stable on straight • Hypervisor efficiency performance • Drinking our own champagne - we’ve got a few KVM devs :-) KVM Lessons Learned Performance Stability • Check flags – lots of features/op ons • Oversubscribing & big mul -vCPU instances • Windows guest updates • Windows guest can be sensi ve IO interrup ons • Keep your images current • Its not good enough to have a cloud environment, applica ons need to evolve to become more cloud aware
17Aug15 Open Virtualiza on Alliance 29 OpenStack* intelligent workload scheduling OpenStack* VMs have a greater awareness of the capabilities of the hardware platforms
FILTERS
Capabilities
Location HOST HOST SERVERS SERVERS Power & Thermals
Security
17Aug15 Open Virtualiza on Alliance 30 OpenStack* intelligent workload scheduling Intelligent VM placement based on monitoring of resource utilization
FILTERS WEIGHTING
Capabilities Performance SELECTED Location NODE HOST HOST SERVERS SERVERS Power & Thermals Utilization Security
17Aug15 Open Virtualiza on Alliance 31 OpenStack* intelligent workload scheduling OpenStack* VMs have a greater awareness of the capabilities of the hardware platforms
FILTERS WEIGHTING
Capabilities Performance SELECTED Location NODE HOST HOST SERVERS SERVERS Power & Thermals Utilization Security
Intelligent VM scheduling based on monitoring of resource utilization, power and thermals
17Aug15 Open Virtualiza on Alliance 32 Easy to implement 1. Create a generic flavor named m1.avx – The grammar is: nova flavor-create
$ nova flavor-create m1.avx auto 1024 10 1
2. Add the extra proper es to the flavor we created – The grammar is: nova flavor-key
$ nova flavor-key
17Aug15 Open Virtualiza on Alliance 33 Intel & OpenStack/KVM Future Direc on IT – It’s not just the hypervisor… it’s how they are managed within the stack – Choice in managing the cloud • OpenStack enabled Single Control Plane to simplify hos ng mul ple hypervisor environments
Intel in the community – Expose op mized hardware features to KVM and OpenStack schedulers – EG: Cache QoS monitoring, chipset features (AVX2, Intel® AES-NI, etc.), VMCS Shadowing, APIC virtualiza on
17Aug15 Open Virtualiza on Alliance Clear Linux Project For Intel® Architecture
WITH MEMORY OVERHEAD OF 18-20 MEGABYTES
(PER EACH INCREMENTAL CONTAINER)
RUN 3,500+ LAUNCH A SECURED CONTAINER CONTAINERIZED APPS WITHIN MULTI-TENANT ON A SERVER USING JUST ENVIRONMENTS 128 GB OF RAM IN UNDER 150 MILLISECONDS
Deployment speed Security & isolation Works with Docker of containerized Intel® of traditional VMs images apps Clear
17Aug15 Open Virtualiza on Alliance Containers A end the Clear Linux session Tuesday 11:30 AM and see it in ac on at our booth (#321) Addi onal Resources
• Open Virtualiza on Alliance – h ps://openvirtualiza onalliance.org • IDC White Paper – “KVM – Open Source Virtualiza on for the Enterprise and Open Stack Clouds” • Linux Founda on Training Course – LFS540 – “Linux KVM Virtualiza on” • KVM Forum – August 19-21 – h p://events.linuxfounda on.org/events/kvm-forum • OpenStack Founda on – h p://www.openstack.org • OpenStack Nova Filter Scheduler – h p://docs.openstack.org/developer/nova/devref/ filter_scheduler.html
17Aug15 Open Virtualiza on Alliance 36