®
Xen Community Update
Ian Pratt
Chairman of Xen.org and SVP Products at Bromium 1 Outline ®
• Year Review • Secure Isolation • Xen Differentiators • Reference Architecture Proposal
2 Xen.Org Changes ®
• Welcome Lars Kurth as new Community Manager! – Thanks to Stephen Spector for a great job done • Lars’ Mission: Encourage more vendor engagement and co-ordination and co- operation in the community; Foster closer links with related OSS communities
3 Development Activity ®
Xen-Devel Mailing List Activity 8000
7000
6000
5000
4000
3000
2000
1000
0
4 Xen.Org Blog Activity ®
5 Calendar Review ®
• Aug 2010: XenDirections in Boston, USA • Sep 2010: XenDirections in Sao Paulo, Brazil • Postponed from Nov 2011: XenSummit Seoul, South Korea • March 2011: Xen Hackathon, Cambridge UK • July 2011: OSCON, Portland, USA • Summer 2011: 6 Google Summer of Code
students working on Xen 6
Xen 4.1 Release – 21 March 2011 ®
• Key Features – New “XL” lightweight control stack – Memory Introspection API – CPU Pools for partitioning – Very large system support (>255 CPUs) – Experimental: credit2 scheduler; Remus FT; Emergency swap
7 Community Interactions ®
• Linux – Privileged domain support upstream in Linux 3.0 – Guest optimizations: use the optimal combination of h/w and s/w virtualization • QEMU – Xen qemu target now upstream • OpenStack – XCP integration with OpenStack 8 Secure Isolation ®
• Maintaining isolation between VMs is priority #1 – Essential for Cloud, and for Client – Spatial and Temporal isolation • Use good software engineering practice – Thin hypervisor: minimize code running with privilege – Disaggregate and de-privilege functionality into dedicated Service VMs – Narrow interfaces between components – Hypervisors are simpler than OSes, simpler than OS kernels – Use modern high-level languages where possible • New hardware technologies help – VT-x, VT-d, EPT: reduce software complexity, enhanced protection – TPM/TXT: Enable Dynamic Root of Trust 9
XenClient XT / Qubes OS ®
• First products configured to take advantage of the security benefits of Xen’s architecture • Isolated Driver Domains • QEMU Emulation Domains • Service VMs (global and per-guest) • Xen Security Modules / SElinux • Measured Launch (TXT)
10 Typical Xen Configuration ®
VM0 VM1 VM2 VM3 Device Manager & Applications Applications Applications Control s/w
GuestOS GuestOS GuestOS GuestOS
Back-End Native Device Front-End Front-End Driver Device Drivers Device Drivers Device Emulation
Control IF Safe HW IF Event Channel Virtual CPU Virtual MMU Xen Virtual Machine Monitor Hardware (SMP, MMU, physical memory, Ethernet, SCSI/IDE) Xen Driver Domains ®
VM0 VM1 VM2 VM3 Device Manager & Applications Applications Control s/w GuestOS GuestOS GuestOS GuestOS
Back-End Back-End Native Native Device Device Front-End Driver Driver Device Drivers Device Emulation
Control IF IOMMU Event Channel Virtual CPU Virtual MMU Xen Virtual Machine Monitor Hardware (SMP, MMU, physical memory, Ethernet, SCSI/IDE) XenClient XT Architecture ®
Service VMs
User VM User VM
Control VPN VPN
Domain for XC
Network Network
Isolation Isolation Receiver Receiver
SELinux Policy Granularity Policy Granularity
Xen Xen Security Modules
VT-d TXT Intel vPro Hardware VT-x AES-NI
13 Disaggregation ®
• Unique benefit of the Xen architecture: • Security – Minimum privilege; Narrow interfaces • Performance – Lightweight e.g. minios directly on hypervisor – Exploit locality – service VMs see a subset of the machine, run close to resources with which they interact • Reliability – Able to be safely restarted
14 Isolated Driver VMs for High Availability ®
• First implemented in 2004 350
• Detect failure e.g. 300 – Illegal access 250 – Timeout 200 150 • Kill domain, restart 100 – E.g. Just 275ms outage from 50 failed Ethernet driver 0 0 5 10 15 20 25 30 35 40 • New work uses restarts to time (s) enhance security Proposal ®
• We should strive to get all Xen products and deployments to take full advantage of the Xen architecture • We need to make this much easier! • Proposal: define and maintain a reference architecture and implementation that embodies best practice recommendations
16 Reference Architecture ®
• Define using new technologies – Latest stable Xen – Linux 3.x pvops • Optimization effort required – Libxl control stack • For easy consumption by other vendor tool stacks
17 Target Features ®
• Network restart-able driver domains – Integrated OpenFlow vswitch • Storage restart-able driver domains – Also allows easier deployment of new storage options e.g. vastsky, ZFS • Qemu emulation domains • Xen Security Modules • Measured Launch • Roadmap for enhanced security and performance features – E.g. the SR-IOV network plugin / vswitch architecture 18
Implementation ®
• Need an initial reference implementation – Easily consumable by users • XCP could fulfil this role – Showcase latest Xen technologies – Optimized for OpenStack • Aim to be as kernel/toolstack etc agnostic to allow easy adoption by all vendors
19 Summary ®
• Xen project continues to thrive! – Great success in Cloud and Client • Key architectural security, reliability and performance benefits that are unique to Xen – We need to do a better job of getting the message out! – We need to do a better job of actually taking advantage of the benefits 20 ®
21 Xen Today ®
• ~20% enterprise server market share • >80% of the Public Cloud is Xen based – World's largest virtualization deployments are Xen based • Development Community: over 50 Companies, 25 Universities, from 25 Countries, ~250 developers – More than 20,000 code submissions • Used in Severs, Desktops, Laptops, Storage Appliances, Network Appliances and Smart Phones – x86, IA64, ARM support Xen Powers the World’s Infrastructure Clouds ®
Xen is great. It’s powerful “ and easy to use. But most important is the very active community around it.
That was a very big reason for us in selecting Xen. Werner Vogels CTO, Amazon.com ” Xen Tops Performance Comparisons ®
“Xen is the Porsche of hypervisors”
Keith Ward, Virtualization Review
“Xen outperforms VMware ESX 3.5 by 41% in user scalability tests.”
The Tolly Group ®
Pioneers of Xen Hypervisor OS Para-virtualization
First and Best to support new CPU, chipset, and Smart IO Technologies ®
Xen 4.0
26 Xen 4.0 ®
• Released 12 Apr 2010 • Reliability, Availability, Scalability – Enhanced MCA support, blktap2, netchannel2 • Memory optimizations • pvops privileged domain support • Fault tolerance for VMs
27 Hardware Fault Tolerance ®
Restart-HA monitors hosts and VMs to keep apps running
Hardware Fault Tolerance with deterministic replay or checkpointing
Xen’s Software-Implemented Hardware Fault Tolerance enables true High Availability for unmodified applications and operating systems Hardware Fault Tolerance ®
• University of British Columbia’s “Remus” project is now in xen 4.0 • Smart checkpointing approach yields excellent performance – VM executes in parallel with checkpoint transmission, with all externally visible state changes suppressed until checkpoint receipt acknowledged – Checkpoints delta compressed • Checkpointing possible across wide-area, even for multi- vCPU guests
29 SR-IOV ®
• SR-IOV: Single Root IO Virtualization – Virtualization friendly IO devices • High performance, high efficiency, low latency • Enables even the most demanding applications to now be virtualized • Compatible with live relocation via hotplug • World First, demonstrated at Intel Developer Forum in September!
30
SR-IOV NIC Demonstration ®
Dell 10G Switch
Dell R710 Server Dell R710 Server XenServer and Intel 10G SR-IOV NIC XenServer and Intel 10G SR-IOV NIC
NFS Common Storage w/OpenFiler Dell R710 Server XenServer and Intel 10G SR-IOV NIC
• Full 20Gb/s bi-directional throughput to VMs • Low latency, High CPU efficiency • Live relocation between hosts - Even hosts with different NICs 31
Network Performance ®
35 30 usercopy 25 kern 20 201% Type-0 xen1 15 CPU (%) 123% 103% grantcopy 10 100% kern0 5 xen0 0 s/w only basic smart SR-IOV native NIC NIC • New Smart NICs reduce CPU overhead substantially • Care must be taken with SR-IOV NICs to ensure benefits of VM portability and live relocation are not lost • Need for an industry standard for “driver plugins”
®
Xen Cloud Platform
33 Xen Cloud Platform (XCP) ®
• XCP Expands Xen.org’s scope beyond the core hypervisor, to create a full virtual infrastructure layer for Cloud deployments – Simplify and streamline use of Xen by Cloud providers and vendors – Promote greater standardisation of components between vendors • Advanced virtual infrastructure to enable Virtual Private Datacenters rather than just Virtual Private Servers – Multi-tenant hosts, networking, storage, etc – Promote interoperability between xen-based clouds and other clouds – Drive standards activities via DMTF
34 August 2009 XCP Announcement ®
35 Where Xen Cloud Platform Fits ®
Management API & OVF Format VM VM VM
Mgt VM Mgt State VM VM VM State Mgt Resource Pool State Mgt VM State VM XCP 0.2 ®
• Xen 3.4; Linux 2.6.27; optimized dom0 file system • xapi toolstack – Resource Pools; VM, host, networking and storage management; snapshots and checkpoints; live and persistent performance statistics; status alerting; role-based access control; OVF/CIM support • Windows PV Drivers; Full installer etc. • Open vSwitch
37 New Open vSwitch ® Isolation · Resource control · Multi-tenancy · Visibility · Security
VM VM VM VM VM VM VM VM VM VM VM
Hypervisor Hypervisor Hypervisor
• Open Source Virtual Switch maintained at www.openvswitch.org • Rich layer 2 feature set Distributed vSwitch ® Built-in policy-based ACLs move with VMs
VM VM VM VM VM VM VM VM VM VM VM
Hypervisor Virtual InterfaceHypervisor (VIF) {MAC, IP} ACLs Hypervisor
permit tcptcp 10.0.0.0 0.0.0.255 10.20.0.0 0.0.0.255 eq domain permit tcptcp 192.168.0.0 0.0.0.255 10.20.0.0 0.0.0.255 eq domain permit tcptcp 172.16.0.0 0.0.0.255 10.20.0.0 0.0.0.255 eq domain permit udp 10.0.0.0 0.0.0.255 10.20.0.0 0.0.0.255 eq domain permit udp 192.168.0.0 0.0.0.255 10.20.0.0 0.0.0.255 eq domain permit udp 172.16.0.0 0.0.0.255 10.20.0.0 0.0.0.255 eq domain permit tcptcp 10.0.0.0 0.0.0.255 10.20.0.0 0.0.0.255 eq 123
Distributed Virtual Switch Distributed vSwitch ® Isolation · Resource control · Multi-tenancy · Visibility · Security
VM VM VM VM VM VM VM VM VM VM VM
Hypervisor Hypervisor Hypervisor
Distributed Virtual Switch Distributed Virtual Switch
Tenant A Tenant B XCP 1.0 Plans ®
• New Storage Repository plug-ins – For cloud-optimized storage models • libxenlight integration • Enhanced vswitch capabilities • pvops domain0 • Better integration of OVF support • Secure boot and attestation • Cloud orchestration and management APIs • Easier complete build environment 41 ®
Xen Client Initiative
42 The Xen Client Initiative ®
• Formed in 2007 to develop Xen for desktop and laptop • Develop enhanced power management, USB, WiFi, WWAN, 3D Graphics, fingerprint reader, multi-touch, etc • Support for latest hardware technologies • Tiny footprint hypervisor, Embeddable in Flash memory or small disk partition • Aiming to make virtualization ubiquitous on client devices...
43 Client Hypervisor Benefits ®
• Security, Manageability, Supportability, Auditability • Building Multi-Level Secure systems – Run multiple VMs with policy controlled information flow • E.g. Personal VM; Corporate VM; VM for web browsing; VM for banking – Trusted hypervisor provides secure isolation • Enables “out-of-band” management and policy enforcement via Service VMs – Malware detection, remote access, image update, backup, VPN, etc. Requires a true type-1 hypervisor architecture Xen is ideally suited to this! 44 Xen Client Architecture ®
User VM1 User VM2
Control Service Domain VM
Xen Hypervisor
Audio USB GPU TXT x86 Hardware Disk ACPI NIC TPM “Business” & “Personal” Environments ®
Business Personal
• Locked Down • Allows Local App Installs • No Local App Installs • Minimal Management – Virus Scanner • Tightly Managed – Security Patches • Self-Service Corporate • No SLA App Installs – Self-Service Wipe Conclusions ®
• The Xen Community continues to grow from strength to strength • Xen’s architecture makes it #1 in security, with great performance – From Cloud to Client • Xen.org’s role is broadening to develop whole reference platforms, promote standards, interopability 47
®
48