Virtualization Xen Features Escalabilidade Performance QoS Implementation Future Virtualizing servers with Xen Evaldo Gardenali VI International Conference of Unix at UNINET Virtualization Xen Features Escalabilidade Performance QoS Implementation Future Outline Virtualization Xen Features Scalability Performance Quality of Service Implementation Future of Xen Virtualization Xen Features Escalabilidade Performance QoS Implementation Future Overview Why? Support heterogeneous environments: Linux r 2.4 e 2.6, NetBSD r , Plan9 r FreeBSD r , OpenSolaris r Consolidate work Legacy Systems Gradual Upgrade Service Isolation Quality of Service Isolated testing and development Ease of administration Ease of relocation and migration Virtualization Xen Features Escalabilidade Performance QoS Implementation Future Virtualization Techniques Single System Image: Ensim r , Vservers, CKRM, VirtuozzoTM, BSD r jail(), Solaris r Zones √ Groups processes in “resource containers” Hard to get isolation × Emulation: QEMU, Bochs √ Portable Extremely slow × Virtualization: VMware r , VirtualPC r √ Runs unmodified Operating Systems Virtualizing x86 is inefficient × User Mode Kernel: User Mode Linux, CoLinux Guest runs as a process on the host OS × Low performance (I/O, context switches) × Paravirtualization: Xen r , Denali √ Excellent performance Requires port to special architecture × Virtualization Xen Features Escalabilidade Performance QoS Implementation Future Virtualization Techniques Single System Image: Ensim r , Vservers, CKRM, VirtuozzoTM, BSD r jail(), Solaris r Zones √ Groups processes in “resource containers” Hard to get isolation × Emulation: QEMU, Bochs √ Portable Extremely slow × Virtualization: VMware r , VirtualPC r √ Runs unmodified Operating Systems Virtualizing x86 is inefficient × User Mode Kernel: User Mode Linux, CoLinux Guest runs as a process on the host OS × Low performance (I/O, context switches) × Paravirtualization: Xen r , Denali √ Excellent performance Requires port to special architecture × Virtualization Xen Features Escalabilidade Performance QoS Implementation Future Virtualization Techniques Single System Image: Ensim r , Vservers, CKRM, VirtuozzoTM, BSD r jail(), Solaris r Zones √ Groups processes in “resource containers” Hard to get isolation × Emulation: QEMU, Bochs √ Portable Extremely slow × Virtualization: VMware r , VirtualPC r √ Runs unmodified Operating Systems Virtualizing x86 is inefficient × User Mode Kernel: User Mode Linux, CoLinux Guest runs as a process on the host OS × Low performance (I/O, context switches) × Paravirtualization: Xen r , Denali √ Excellent performance Requires port to special architecture × Virtualization Xen Features Escalabilidade Performance QoS Implementation Future Virtualization Techniques Single System Image: Ensim r , Vservers, CKRM, VirtuozzoTM, BSD r jail(), Solaris r Zones √ Groups processes in “resource containers” Hard to get isolation × Emulation: QEMU, Bochs √ Portable Extremely slow × Virtualization: VMware r , VirtualPC r √ Runs unmodified Operating Systems Virtualizing x86 is inefficient × User Mode Kernel: User Mode Linux, CoLinux Guest runs as a process on the host OS × Low performance (I/O, context switches) × Paravirtualization: Xen r , Denali √ Excellent performance Requires port to special architecture × Virtualization Xen Features Escalabilidade Performance QoS Implementation Future Virtualization Techniques Single System Image: Ensim r , Vservers, CKRM, VirtuozzoTM, BSD r jail(), Solaris r Zones √ Groups processes in “resource containers” Hard to get isolation × Emulation: QEMU, Bochs √ Portable Extremely slow × Virtualization: VMware r , VirtualPC r √ Runs unmodified Operating Systems Virtualizing x86 is inefficient × User Mode Kernel: User Mode Linux, CoLinux Guest runs as a process on the host OS × Low performance (I/O, context switches) × Paravirtualization: Xen r , Denali √ Excellent performance Requires port to special architecture × Virtualization Xen Features Escalabilidade Performance QoS Implementation Future Advantages of Virtualization Advantages For Administrators Service Isolation, minimizing damages Failure Isolation Ease of Administration Quality of Service enforcement For Hosting providers and datacenters Offer “Virtual Private Server” services Raise Aggregated Value Virtualization Xen Features Escalabilidade Performance QoS Implementation Future Advantages of Virtualization Advantages For Administrators Service Isolation, minimizing damages Failure Isolation Ease of Administration Quality of Service enforcement For Hosting providers and datacenters Offer “Virtual Private Server” services Raise Aggregated Value Virtualization Xen Features Escalabilidade Performance QoS Implementation Future Advantages of Virtualization Costs! Purchase or rent of equipments Rack Space Colocation costs Energy Consumption Downtime Virtualization Xen Features Escalabilidade Performance QoS Implementation Future Overview Xen architecture Virtualization Xen Features Escalabilidade Performance QoS Implementation Future Adopted Technique Paravirtualization X86 has 4 operation modes (rings) Traditional OSes run on 2 rings: 0 and 3 OS/2 uses/used 4 rings Hypervisor runs in ring 0 Operating System kernels: ring 1 ou 2 Privileged operations done via hypercalls Needs to be ported to ring 1 or 2 User processes: ring 3 Runs without any modification* Virtualization Xen Features Escalabilidade Performance QoS Implementation Future Xen Architecture Xen Architecture characteristics Kernel runs in ring 1 or 2 Userland runs unmodified in ring 3 Privileged operations through hypercalls Device access done through hypercalls Linux 2.4 Port: less than 3000 lines of code Linux 2.6 Port did not modify any “core” files. Virtualization Xen Features Escalabilidade Performance QoS Implementation Future Xen Architecture Xen 3.0 roadmap AGP in Domain 0 ACPI in Domain 0 SMP Guests Architectures: x86 64, IA64, IBM POWER r Intel VT-x (Vanderpool) and AMD Pacifica Better management tools Network structure optimization Virtualization Xen Features Escalabilidade Performance QoS Implementation Future Hardware Access Hardware access in Xen systems Domain 0 accesses devices with “native” drivers, through hypercalls Domain Us access virtual devices exported by Domain 0 Safe asynchronous access through shared memory “Zero-copy” Implementation Network: Use of regular bridging and routing techniques Block Devices: Domain 0 exports any block device (sda4,loop0,vg3,md2,...) Access by Privileged Domain Us Native access through hypercalls Virtualization Xen Features Escalabilidade Performance QoS Implementation Future Hardware Access Hardware access in Xen systems Domain 0 accesses devices with “native” drivers, through hypercalls Domain Us access virtual devices exported by Domain 0 Safe asynchronous access through shared memory “Zero-copy” Implementation Network: Use of regular bridging and routing techniques Block Devices: Domain 0 exports any block device (sda4,loop0,vg3,md2,...) Access by Privileged Domain Us Native access through hypercalls Virtualization Xen Features Escalabilidade Performance QoS Implementation Future Hardware Access Hardware access in Xen systems Domain 0 accesses devices with “native” drivers, through hypercalls Domain Us access virtual devices exported by Domain 0 Safe asynchronous access through shared memory “Zero-copy” Implementation Network: Use of regular bridging and routing techniques Block Devices: Domain 0 exports any block device (sda4,loop0,vg3,md2,...) Access by Privileged Domain Us Native access through hypercalls Virtualization Xen Features Escalabilidade Performance QoS Implementation Future Isolation and Contention Device Isolation “Virtual” Virtual PCI Configuration Space Virtual Interrupts Failures don’t affect other domains It is safe to reboot a domain without affecting others example root@julia:˜# lspci 00:0a.0 Multimedia audio controller: Ensoniq 5880 AudioPCI root@julia:˜# Virtualization Xen Features Escalabilidade Performance QoS Implementation Future Supported OSes Xen 2.0 Supported Operating Systems Systems ported to Xen x86 Operating System Domain 0 Domain U Linux r 2.4 √ √ Linux r 2.6 √ √ NetBSD r 3.0 √ √ Plan9 r √ FreeBSD r √ OpenSolaris r √ Windows r √ Virtualization Xen Features Escalabilidade Performance QoS Implementation Future Balloon Dynamic memory management Idle Linux r domain can consume as low as 4MB RAM Maximum memory footprint configurable at run-time from Domain 0 Better use of memory, avoiding Swap Control under Domain 0 (xm) or Domain U (/proc/xen/balloon) Balloon Auto-Control Virtualization Xen Features Escalabilidade Performance QoS Implementation Future Pausa de dom´ınios Pause: temporary interruption Interrupts domain execution Stays ready to resume xm pause dom´ınio xm unpause dom´ınio Virtualization Xen Features Escalabilidade Performance QoS Implementation Future Pausa de dom´ınios Save: domain suspension Interrupts domain execution Saves machine state (RAM, registers) to a file Destroys the running domain Can be used when upgrading domain0 xm save dom´ınio arquivo xm restore arquivo Virtualization Xen Features Escalabilidade Performance QoS Implementation Future Live Migration Live Migration Transfers a running OS to another host “Downtime” of a few milliseconds! Obeys bandwidth limits Needs shared devices between source and target machines Simple! xm migrate –live –resource 70 DominioA OutroHostXen 70Mbit Limit Virtualization Xen Features Escalabilidade Performance QoS Implementation Future Live Migration Live Migration: How It Works Virtualization