®

Xen Community Update

Ian Pratt

Chairman of .org and SVP Products at Bromium 1 Outline ®

• Year Review • Secure Isolation • Xen Differentiators • Reference Architecture Proposal

2 Xen.Org Changes ®

• Welcome Lars Kurth as new Community Manager! – Thanks to Stephen Spector for a great job done • Lars’ Mission: Encourage more vendor engagement and co-ordination and co- operation in the community; Foster closer links with related OSS communities

3 Development Activity ®

Xen-Devel Mailing List Activity 8000

7000

6000

5000

4000

3000

2000

1000

0

4 Xen.Org Blog Activity ®

5 Calendar Review ®

• Aug 2010: XenDirections in Boston, USA • Sep 2010: XenDirections in Sao Paulo, Brazil • Postponed from Nov 2011: XenSummit Seoul, South Korea • March 2011: Xen Hackathon, Cambridge UK • July 2011: OSCON, Portland, USA • Summer 2011: 6 Google Summer of Code

students working on Xen 6

Xen 4.1 Release – 21 March 2011 ®

• Key Features – New “XL” lightweight control stack – Memory Introspection API – CPU Pools for partitioning – Very large system support (>255 CPUs) – Experimental: credit2 scheduler; Remus FT; Emergency swap

7 Community Interactions ®

• Linux – Privileged domain support upstream in Linux 3.0 – Guest optimizations: use the optimal combination of h/w and s/w virtualization • QEMU – Xen qemu target now upstream • OpenStack – XCP integration with OpenStack 8 Secure Isolation ®

• Maintaining isolation between VMs is priority #1 – Essential for Cloud, and for Client – Spatial and Temporal isolation • Use good engineering practice – Thin : minimize code running with privilege – Disaggregate and de-privilege functionality into dedicated Service VMs – Narrow interfaces between components – are simpler than OSes, simpler than OS kernels – Use modern high-level languages where possible • New hardware technologies help – VT-x, VT-d, EPT: reduce software complexity, enhanced protection – TPM/TXT: Enable Dynamic Root of Trust 9

XenClient XT / Qubes OS ®

• First products configured to take advantage of the security benefits of Xen’s architecture • Isolated Driver Domains • QEMU Emulation Domains • Service VMs (global and per-guest) • Xen Security Modules / SElinux • Measured Launch (TXT)

10 Typical Xen Configuration ®

VM0 VM1 VM2 VM3 Device Manager & Applications Applications Applications Control s/w

GuestOS GuestOS GuestOS GuestOS

Back-End Native Device Front-End Front-End Driver Device Drivers Device Drivers Device Emulation

Control IF Safe HW IF Event Channel Virtual CPU Virtual MMU Xen Virtual Machine Monitor Hardware (SMP, MMU, physical memory, Ethernet, SCSI/IDE) Xen Driver Domains ®

VM0 VM1 VM2 VM3 Device Manager & Applications Applications Control s/w GuestOS GuestOS GuestOS GuestOS

Back-End Back-End Native Native Device Device Front-End Driver Driver Device Drivers Device Emulation

Control IF IOMMU Event Channel Virtual CPU Virtual MMU Xen Virtual Machine Monitor Hardware (SMP, MMU, physical memory, Ethernet, SCSI/IDE) XenClient XT Architecture ®

Service VMs

User VM User VM

Control VPN VPN

Domain for XC

Network Network

Isolation Isolation Receiver Receiver

SELinux Policy Granularity Policy Granularity

Xen Xen Security Modules

VT-d TXT vPro Hardware VT-x AES-NI

13 Disaggregation ®

• Unique benefit of the Xen architecture: • Security – Minimum privilege; Narrow interfaces • Performance – Lightweight e.g. minios directly on hypervisor – Exploit locality – service VMs see a subset of the machine, run close to resources with which they interact • Reliability – Able to be safely restarted

14 Isolated Driver VMs for High Availability ®

• First implemented in 2004 350

• Detect failure e.g. 300 – Illegal access 250 – Timeout 200 150 • Kill domain, restart 100 – E.g. Just 275ms outage from 50 failed Ethernet driver 0 0 5 10 15 20 25 30 35 40 • New work uses restarts to time (s) enhance security Proposal ®

• We should strive to get all Xen products and deployments to take full advantage of the Xen architecture • We need to make this much easier! • Proposal: define and maintain a reference architecture and implementation that embodies best practice recommendations

16 Reference Architecture ®

• Define using new technologies – Latest stable Xen – Linux 3.x pvops • Optimization effort required – Libxl control stack • For easy consumption by other vendor tool stacks

17 Target Features ®

• Network restart-able driver domains – Integrated OpenFlow vswitch • Storage restart-able driver domains – Also allows easier deployment of new storage options e.g. vastsky, ZFS • Qemu emulation domains • Xen Security Modules • Measured Launch • Roadmap for enhanced security and performance features – E.g. the SR-IOV network plugin / vswitch architecture 18

Implementation ®

• Need an initial reference implementation – Easily consumable by users • XCP could fulfil this role – Showcase latest Xen technologies – Optimized for OpenStack • Aim to be as kernel/toolstack etc agnostic to allow easy adoption by all vendors

19 Summary ®

• Xen project continues to thrive! – Great success in Cloud and Client • Key architectural security, reliability and performance benefits that are unique to Xen – We need to do a better job of getting the message out! – We need to do a better job of actually taking advantage of the benefits 20 ®

21 Xen Today ®

• ~20% enterprise server market share • >80% of the Public Cloud is Xen based – World's largest virtualization deployments are Xen based • Development Community: over 50 Companies, 25 Universities, from 25 Countries, ~250 developers – More than 20,000 code submissions • Used in Severs, Desktops, Laptops, Storage Appliances, Network Appliances and Smart Phones – x86, IA64, ARM support Xen Powers the World’s Infrastructure Clouds ®

Xen is great. It’s powerful “ and easy to use. But most important is the very active community around it.

That was a very big reason for us in selecting Xen. Werner Vogels CTO, Amazon.com ” Xen Tops Performance Comparisons ®

“Xen is the Porsche of hypervisors”

Keith Ward, Virtualization Review

“Xen outperforms VMware ESX 3.5 by 41% in user scalability tests.”

The Tolly Group ®

Pioneers of Xen Hypervisor OS Para-virtualization

First and Best to support new CPU, chipset, and Smart IO Technologies ®

Xen 4.0

26 Xen 4.0 ®

• Released 12 Apr 2010 • Reliability, Availability, Scalability – Enhanced MCA support, blktap2, netchannel2 • Memory optimizations • pvops privileged domain support • Fault tolerance for VMs

27 Hardware Fault Tolerance ®

Restart-HA monitors hosts and VMs to keep apps running

Hardware Fault Tolerance with deterministic replay or checkpointing

Xen’s Software-Implemented Hardware Fault Tolerance enables true High Availability for unmodified applications and operating systems Hardware Fault Tolerance ®

• University of British Columbia’s “Remus” project is now in xen 4.0 • Smart checkpointing approach yields excellent performance – VM executes in parallel with checkpoint transmission, with all externally visible state changes suppressed until checkpoint receipt acknowledged – Checkpoints delta compressed • Checkpointing possible across wide-area, even for multi- vCPU guests

29 SR-IOV ®

• SR-IOV: Single Root IO Virtualization – Virtualization friendly IO devices • High performance, high efficiency, low latency • Enables even the most demanding applications to now be virtualized • Compatible with live relocation via hotplug • World First, demonstrated at Intel Developer Forum in September!

30

SR-IOV NIC Demonstration ®

Dell 10G Switch

Dell R710 Server Dell R710 Server XenServer and Intel 10G SR-IOV NIC XenServer and Intel 10G SR-IOV NIC

NFS Common Storage w/OpenFiler Dell R710 Server XenServer and Intel 10G SR-IOV NIC

• Full 20Gb/s bi-directional throughput to VMs • Low latency, High CPU efficiency • Live relocation between hosts - Even hosts with different NICs 31

Network Performance ®

35 30 usercopy 25 kern 20 201% Type-0 xen1 15 CPU (%) 123% 103% grantcopy 10 100% kern0 5 xen0 0 s/w only basic smart SR-IOV native NIC NIC • New Smart NICs reduce CPU overhead substantially • Care must be taken with SR-IOV NICs to ensure benefits of VM portability and live relocation are not lost • Need for an industry standard for “driver plugins”

®

Xen Cloud Platform

33 Xen Cloud Platform (XCP) ®

• XCP Expands Xen.org’s scope beyond the core hypervisor, to create a full virtual infrastructure layer for Cloud deployments – Simplify and streamline use of Xen by Cloud providers and vendors – Promote greater standardisation of components between vendors • Advanced virtual infrastructure to enable Virtual Private Datacenters rather than just Virtual Private Servers – Multi-tenant hosts, networking, storage, etc – Promote interoperability between xen-based clouds and other clouds – Drive standards activities via DMTF

34 August 2009 XCP Announcement ®

35 Where Xen Cloud Platform Fits ®

Management API & OVF Format VM VM VM

Mgt VM Mgt State VM VM VM State Mgt Resource Pool State Mgt VM State VM XCP 0.2 ®

• Xen 3.4; Linux 2.6.27; optimized dom0 file system • xapi toolstack – Resource Pools; VM, host, networking and storage management; snapshots and checkpoints; live and persistent performance statistics; status alerting; role-based access control; OVF/CIM support • Windows PV Drivers; Full installer etc. • Open vSwitch

37 New Open vSwitch ® Isolation · Resource control · Multi-tenancy · Visibility · Security

VM VM VM VM VM VM VM VM VM VM VM

Hypervisor Hypervisor Hypervisor

• Open Source Virtual Switch maintained at www.openvswitch.org • Rich layer 2 feature set Distributed vSwitch ® Built-in policy-based ACLs move with VMs

VM VM VM VM VM VM VM VM VM VM VM

Hypervisor Virtual InterfaceHypervisor (VIF) {MAC, IP} ACLs Hypervisor

permit tcptcp 10.0.0.0 0.0.0.255 10.20.0.0 0.0.0.255 eq domain permit tcptcp 192.168.0.0 0.0.0.255 10.20.0.0 0.0.0.255 eq domain permit tcptcp 172.16.0.0 0.0.0.255 10.20.0.0 0.0.0.255 eq domain permit udp 10.0.0.0 0.0.0.255 10.20.0.0 0.0.0.255 eq domain permit udp 192.168.0.0 0.0.0.255 10.20.0.0 0.0.0.255 eq domain permit udp 172.16.0.0 0.0.0.255 10.20.0.0 0.0.0.255 eq domain permit tcptcp 10.0.0.0 0.0.0.255 10.20.0.0 0.0.0.255 eq 123

Distributed Virtual Switch Distributed vSwitch ® Isolation · Resource control · Multi-tenancy · Visibility · Security

VM VM VM VM VM VM VM VM VM VM VM

Hypervisor Hypervisor Hypervisor

Distributed Virtual Switch Distributed Virtual Switch

Tenant A Tenant B XCP 1.0 Plans ®

• New Storage Repository plug-ins – For cloud-optimized storage models • libxenlight integration • Enhanced vswitch capabilities • pvops domain0 • Better integration of OVF support • Secure boot and attestation • Cloud orchestration and management APIs • Easier complete build environment 41 ®

Xen Client Initiative

42 The Xen Client Initiative ®

• Formed in 2007 to develop Xen for desktop and laptop • Develop enhanced power management, USB, WiFi, WWAN, 3D Graphics, fingerprint reader, multi-touch, etc • Support for latest hardware technologies • Tiny footprint hypervisor, Embeddable in Flash memory or small disk partition • Aiming to make virtualization ubiquitous on client devices...

43 Client Hypervisor Benefits ®

• Security, Manageability, Supportability, Auditability • Building Multi-Level Secure systems – Run multiple VMs with policy controlled information flow • E.g. Personal VM; Corporate VM; VM for web browsing; VM for banking – Trusted hypervisor provides secure isolation • Enables “out-of-band” management and policy enforcement via Service VMs – detection, remote access, image update, backup, VPN, etc.  Requires a true type-1 hypervisor architecture  Xen is ideally suited to this! 44 Xen Client Architecture ®

User VM1 User VM2

Control Service Domain VM

Xen Hypervisor

Audio USB GPU TXT x86 Hardware Disk ACPI NIC TPM “Business” & “Personal” Environments ®

Business Personal

• Locked Down • Allows Local App Installs • No Local App Installs • Minimal Management – Virus Scanner • Tightly Managed – Security Patches • Self-Service Corporate • No SLA App Installs – Self-Service Wipe Conclusions ®

• The Xen Community continues to grow from strength to strength • Xen’s architecture makes it #1 in security, with great performance – From Cloud to Client • Xen.org’s role is broadening to develop whole reference platforms, promote standards, interopability 47

®

48