Distributed System: Lecture 4 Virtualizations Box Leangsuksun SWECO Endowed Professor, Computer Science Louisiana Tech University [email protected] CTO, PB Tech International Inc. [email protected] Introduction to Virtualization • System virtualization studied since the 70's (Goldberg, Popek) • Fundamental – Run multiple virtual machines (OSes) simultaneously – Isolating between virtual machines. – Controlling Resources sharing between VMs – Increase resources utilization – One of the hottest technologies since 2006 Virtualization: Key concepts • Virtual Machine (VM), guest OS: complete operating system running in a virtual environment • Host OS: operating system running on top the hardware, interface between the user and the VMM and VMs • Virtual Machine Monitor (VMM):, Hypervisor: manage VMs (scheduling, hardware access) Virtualization: Usage Ø Server consolidation (cloud) Ø Software testing Ø Security, Isolation (cloud) Ø Lower cost of ownership of server. (cloud) Ø Increase manageability (cloud) Ø Enhance server reliability Major Fields of Virtualization • Storage Virtualization • Network Virtualization • Server Virtualization Architecture & Interfaces • Architecture: formal specification of a system’s interface and the logical behavior of its visible resources. API Applications Libraries System Calls ABI Operating System ISA System ISA User ISA Hardware n API – application binary interface n ABI – application binary interface n ISA – instruction set architecture Credit: CS5204 – Operating Systems from vtech u Sample of API vs ABI 4/22/14 Towards survivable architecture 7 VMM Types • System ¨ Provides ABI interface ¨ Efficient execution ¨ Can add OS-independent services (e.g., migration, intrusion detection) n Process ¨ Provdes API interface ¨ Easier installation ¨ Leverage OS services (e.g., device drivers) ¨ Execution overhead (possibly mitigated by just- in-time compilation) CS5204 – Operating Credit: CS5204 – Operating Systems from vtech u Systems System-level Design Approaches • Full virtualization (direct execution) – Exact hardware exposed to OS – Efficient execution – OS runs unchanged – Requires a “virtualizable” architecture – Example: VMWare n Paravirtualization ¨ OS modified to execute under VMM ¨ Requires porting OS code ¨ Execution overhead ¨ Necessary for some (popular) architectures (e.g., x86) ¨ Examples: Xen, Denali CS5204 – Operating Credit: CS5204 – Operating Systems from vtech u Systems Design Space (level vs. ISA) API interface ABI interface • Variety of techniques and approaches available • Critical technology space highlighted CS5204 – Operating Credit: CS5204 – Operating Systems from vtech u Systems System VMMs Type 1 • Structure – Type 1: runs directly on host hardware – Type 2: runs on HostOS • Primary goals – Type 1: High performance – Type 2: Ease of construction/installation/acceptability • Examples – Type 1: VMWare ESX Server, Xen, OS/370 – Type 2: User-mode Linux Type 2 CS5204 – Operating Credit: CS5204 – Operating Systems from vtech u Systems Hosted VMMs • Structure – Hybrid between Type1 and Type2 – Core VMM executes directly on hardware – I/O services provided by code running on HostOS • Goals – Improve performance overall – leverages I/O device support on the HostOS • Disadvantages – Incurs overhead on I/O operations – Lacks performance isolation and performance guarantees • Example: VMWare (Workstation) CS5204 – Operating Credit: CS5204 – Operating Systems from vtech u Systems Whole-system VMMs n Challenge: GuestOS ISA differs from HostOS ISA n Requires full emulation of GuestOS and its applications n Example: VirtualPC CS5204 – Operating Credit: CS5204 – Operating Systems from vtech u Systems Strategies GuestOS • De-privileging – VMM emulates the effect on system/hardware resources of privileged instructions whose execution traps into the VMM – aka trap-and-emulate privileged instruction – Typically achieved by running GuestOS at a lower hardware priority level than the VMM – Problematic on some architectures where privileged instructions do not trap when executed at deprivileged priority trap resource • Primary/shadow structures emulate change – VMM maintains “shadow” copies of critical structures whose “primary” versions are manipulated by the GuestOS vmm – e.g., page tables change – Primary copies needed to insure correct environment visible to GuestOS resource • Memory traces – Controlling access to memory so that the shadow and primary structure remain coherent – Common strategy: write-protect primary copies so that update operations cause page faults which can be caught, interpreted, and emulated. CS5204 – Operating Credit: CS5204 – Operating Systems from vtech u Systems Different Virtualization Concepts • Full-virtualization: full virtual machine, from the boot sequence to the virtualized hardware • Para-virtualization: the guest OS has to be modify for performance optimization • Emulation: the guest OS architecture is different from the architecture of the host OS (translation on the fly). Ex: PPC VM on top of a x86 host OS. Classification • Two kinds of system virtualization – Type-I: the virtual machine monitor and the virtual machine run directly on top of the hardware, – Type-II: the virtual machine monitor and the virtual machine run on top of the host OS VM VM Host OS VM VM VMM VMM Host OS Hardware Hardware Type I Virtualization (Bare-metal) Type II Virtualization (hosted) VMware Workstation, Microsoft Virtual PC, VMware ESX, Microsoft Hyper-V, Xen Sun VirtualBox, QEMU, KVM Bare-metal or hosted? • Bare-metal – Has complete control over hardware – Doesn’t have to “fight” an OS • Hosted – Avoid code duplication: need not code a process scheduler, memory management system – the OS already does that – Can run native processes alongside VMs – Familiar environment – how much CPU and memory does a VM take? Use top! How big is the virtual disk? ls –l – Easy management – stop a VM? Sure, just kill it! • A combination – Mostly hosted, but some parts are inside the OS kernel for performance reasons 17 – E.g., KVM Available Solutions • Example of Virtualization Projects – Type I: Xen, L4, VMware ESX, Microsoft Hyper- V • Type II: VMware Workstation, Microsoft Virtual PC, Sun VirtualBox, QEMU, KVM • Different Benefits – Type I: performances • direct access to the hardware simple to implement • para-virtualization possible – Type II: development • no limitation of para-virtualization • emulation possible How to run a VM? Emulate! • Do whatever the CPU does but in software • Fetch the next instruction • Decode – is it an ADD, a XOR, a MOV? • Execute – using the emulated registers and memory Example: addl %ebx, %eax is emulated as: enum {EAX=0, EBX=1, ECX=2, EDX=3, …}; unsigned long regs[8]; regs[EAX] += regs[EBX]; 19 How to run a VM? Emulate! • Pro: – Simple! • Con: – Slooooooooow • Example hypervisor: BOCHS 20 How to run a VM? Trap and emulate! • Run the VM directly on the CPU – no emulation! • Most of the code can execute just fine – E.g., addl %ebx, %eax • Some code needs hypervisor intervention – int $0x80 – movl something, %cr3 – I/O • Trap and emulate it! – E.g., if guest runs int $0x80, trap it and execute guest’s interrupt 0x80 handler 21 How to run a VM? Trap and emulate! • Pro: – Performance! • Cons: – Harder to implement – Need hardware support • Not all “sensitive” instructions cause a trap when executed in usermode • E.g., POPF, that may be used to clear IF • This instruction does not trap, but value of IF does not 22 change! – This hardware support is called VMX (Intel) or SVM (AMD) – Exists in modern CPUs • Example hypervisor: KVM How to run a VM? Dynamic (binary) translation! • Take a block of binary VM code that is about to be executed • Translate it on the fly to “safe” code (like JIT – just in time compilation) • Execute the new “safe” code directly on the CPU • Translation rules? – Most code translates identically (e.g., movl %eax, %ebx translates to itself) – “Sensitive” operations are translated into hypercalls • Hypercall – call into the hypervisor to ask for service • Implemented as trapping instructions (unlike POPF) • Similar to syscall – call into the OS to request service 23 How to run a VM? Dynamic (binary) translation! • Pros: – No hardware support required – Performance – better than emulation • Cons: – Performance – worse than trap and emulate – Hard to implement – hypervisor needs on-the-fly x86- to-x86 binary compiler 24• Example hypervisors: VMware, QEMU How to run a VM? Paravirtualization! • Does not run unmodified guest OSes • Requires guest OS to “know” it is running on top of a hypervisor • E.g., instead of doing cli to turn off interrupts, guest OS should do hypercall(DISABLE_INTERRUPTS) 25 How to run a VM? Paravirtualization! • Pros: – No hardware support required – Performance – better than emulation • Con: – Requires specifically modified guest – Same guest OS cannot run in the VM and bare-metal • Example hypervisor: Xen 26 Industry trends • Trap and emulate • With hardware support • VMX, SVM 27 Linux-related virtualization projects Project Type License Bochs Emulation LGPL QEMU Emulation LGPL/GPL VMware Full virtualization Proprietary z/VM Full virtualization Proprietary Xen Paravirtualization GPL UML Paravirtualization GPL Linux-VServer Operating system- GPL level virtualization OpenVZ Operating system- GPL level virtualization Hardware support for full virtualization and paravirtualization • Recall that the IA-32 (x86) architecture creates some issues when it comes to virtualization. Certain privileged-mode instructions do not trap, and