DOCSLIB.ORG

VM Virtualization Tasks Problems of X86 Categories of X86 Virtualization

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
VM Virtualization Tasks Problems of X86 Categories of X86 Virtualization VM Starts from IBM VM/370 Old purpose: share the resource of the expensive mainframe machine New purpose: fault tolerance and security; run multiple different OSes on the same desktop; server consolidation and performance isolation, etc. Virtualization tasks (isolate, secure, fast, transparent …) 1. cpu virtualization How to make sure privileged instructions (e.g., enable/disable interrupts; change critical registers) will not be executed directly 2. memory virtualization OS thought they can control the whole physical memory. No! 3. I/O virtualization OS thought I/O devices are all under its own control. No! Problems of X86 1. Some privileged instructions do not trap; they just fail/change-semantic silently. • IRET (interrupt return) increment stack pointer by 3 words when the instruction does not change privilege level, by 5 when the instruction does. • PUSHF stores registers (including sensitive information like IF (interrupt enable/disable)) onto stack (so that os can see it) 2. segment is a problem Categories of X86 virtualization Full-virtualization (VMWare) No change to guest OS Use binary translation to change some instructions (privilege instructions or all OS code) on the fly Para-virtualization (Xen) Change guest OS to improve performance Rewrite OS; replace privileged instructions with hypercalls Many other changes Hardware support (Intel VT, AMD …) 1st generation, add one level (root level) beneath ring 0 (non-root level); the execution of all privileged instructions will trap to the root level. Support for `world-change’ (change between root-level and upper level 0—3) Currently, performance is not better than VMware, because world-change has huge overhead and rigid programming mode makes it hard to reduce #world-change 2nd generation, will add support to help shadow page table … • VMWare, of course, is against the Xen trend. Xen is criticized for not portable, hard to maintain, paralyzed os cannot directly run on hardware. VMWare proposes VMI (virtual machine interface) Make paravirtualized OS standardized and more portable Para-virtualize os by VMI interface. VMI interface is backed up by two versions of implementation: (1) implementation to run on raw machine; (2) call underlying hypercall of supporting vmm Disco On MIPS DISCO directly runs on multi-processor machine; Oses run upon DISCO (w/ slight modification --- relinking OS to put it on mapping address range; option for more OS code change for better performance) Goal: use commodity OSes to leverage many-core cluster VMM runs in kernel mode VM OS runs in supervisor mode VM user runs in user mode Privileged instructions will trap to VMM and be emulated CPU virtualization No special challenge compared with CPU (seemed that all privilege instructions will trap) DISCO provide option for OS to be para-virtualized ☺. Disable/enable interrupt and accessing privileged registers can be done through special memory location, which will avoid trap emulation. (*this os change is not have-to, different from Xen) VMM maintains a process-table-entry-like data structure for each VM (virtual PC) (including registers, states, and TLB content (used for emulation …) ). Memory Shadow page table Leverage shadow page table to ease different OSes can communicate with each other and page sharing (especially share buffer-cache) through memory mapping DISCO also uses mapping to do page migration, duplication, etc., for good performance under ccNUMA. A problem is that in MIPS user address space will go through TLB, BUT OS’ address space does NOT go through TLB. This requires to relink OS (change its code address). I/O Virtualized device … DISCO proposes changes to OS (HAL) for better performance (note: this is different from Xen, where changes are definitely necessary for correct virtualization) 1. replace some privileged instructions to simply accessing certain memory locations 2. have OS to call-monitor (like Xen’s hypercall) to tell resource-management related information to monitor (e.g., certain memory pages can be reclaimed now) VMWare (patent) Two major things: 0. memory trace utility through MMU (page protection, I think) to register and trace certain page’s access/write. 1. binary translation and direct execution and the logic to decide when to do which (1) binary translation translator: translate privilege to non-privilege or callout function (to special monitor functions) translation cache: a hash table index starting physical instruction to translated version in the translation cache; the translation is discarded when memory trace indicates the source code change. Special problem: in original code sequence, exception only happens at instruction boundary, which is not the case in the translated code. Need special attention (2) decision logic binary translation if it is in CPL=0/1, if interrupt flag is cleared, if I/O privilege level is .., if it is in non-reversible segmenting, Actually, x86 has 4 modes (protected, real, v-…, …) making things even harder :S 2. handle segmenting and non-reversible segmenting • VMM and VM are in the same linear address space, so special attention needs to be paid Vmware Workstation and VMWare Server (ESX) VMWare workstation Purpose/background: run on PC (supporting different Oses is very appealing) Which means: many different devices; may not be good to require uninstall original OS to move OS upon VMM; (VMWare workstation comes out first, before VMWare server) Design: VMApp, VM Driver and VMMonitor; still have the original OS; Install VMApp upon OS; VMDriver is loaded into the kernel; VMApp uses VMDriver to start a VMM that runs directly on hardware; New OS and applications can be run inside the VMM VMWare ESX Server VMWare ESX Server runs directly on hardware, handle I/O by itself, for better performance (Xen uses Domain0 to handle I/O; VMWare workstation uses host OS) Use shadow page table (pmap for each VM to translate physical AD to machine AD) This paper talks about several memory management techniques in ESX 1. Use balloon driver to avoid VMM blindly makes bad virtual memory paging decisions. 2. Transparent page sharing (use content hashing, etc.) 3. Working-set esitmation 4. Tax-based space sharing technique .
Load more

Recommended publications
  • VM Virtualization Tasks Problems of X86 Categories of X86 Virtualization
    VM Starts from IBM VM/370 definition: "an efficient, isolated duplicate of a real machine" Old purpose: share the resource of the expensive mainframe machine New purpose: fault tolerance and security; run multiple different OSes on the same desktop; server consolidation and performance isolation, etc. type 1 vmm: vmm running on bare hardware type 2 vmm: vmm running on host os Virtualization tasks (isolate, secure, fast, transparent …) 1. cpu virtualization How to make sure privileged instructions (e.g., enable/disable interrupts; change critical registers) will not be executed directly 2. memory virtualization OS thought they can control the whole physical memory. No! 3. I/O virtualization OS thought I/O devices are all under its own control. No! Problems of X86 1. Some privileged instructions do not trap; they just fail/change-semantic silently. 2. TLB miss will be handled by hardware instead of software Categories of X86 virtualization Full-virtualization (VMWare) No change to guest OS Use binary translation to change some instructions (privilege instructions or all OS code) on the fly Para-virtualization (Xen) Change guest OS to improve performance Rewrite OS; replace privileged instructions with hypercalls Many other changes Disco On MIPS DISCO directly runs on multi-processor machine; Oses run upon DISCO Goal: use commodity OSes to leverage many-core cluster VMM runs in kernel mode VM OS runs in supervisor mode VM user runs in user mode Privileged instructions will trap to VMM and be emulated CPU virtualization No special challenge (seemed that all privilege instructions will trap) VMM maintains a process-table-entry-like data structure for each VM (virtual PC) (including registers, states, and TLB content (used for emulation …) ).
    [Show full text]
  • Security Analysis of Encrypted Virtual Machines
    Security Analysis of Encrypted Virtual Machines Felicitas Hetzelt Robert Buhren Technical University of Berlin Technical University of Berlin Berlin, Germany Berlin, Germany fi[email protected] [email protected] Abstract 1. Introduction Cloud computing has become indispensable in today’s com- Cloud computing has been one of the most prevalent trends puter landscape. The flexibility it offers for customers as in the computer industry in the last decade. It offers clear ad- well as for providers has become a crucial factor for large vantages for both customers and providers. Customers can parts of the computer industry. Virtualization is the key tech- easily deploy multiple servers and dynamically allocate re- nology that allows for sharing of hardware resources among sources according to their immediate needs. Providers can different customers. The controlling software component, ver-commit their hardware and thus increase the overall uti- called hypervisor, provides a virtualized view of the com- lization of their systems. The key technology that made this puter resources and ensures separation of different guest vir- possible is virtualization, it allows multiple operating sys- tual machines. However, this important cornerstone of cloud tems to share hardware resources. The hypervisor is respon- computing is not necessarily trustworthy or bug-free. To mit- sible for providing temporal and spatial separation of the igate this threat AMD introduced Secure Encrypted Virtu- virtual machines (VMs). However, besides these advantages alization, short SEV, which transparently encrypts a virtual virtualization also introduced new risks. machines memory. Customers who want to utilize the infrastructure of a In this paper we analyse to what extend the proposed fea- cloud provider must fully trust the cloud provider.
    [Show full text]
  • Hardware Virtualization
    Hardware Virtualization E-516 Cloud Computing 1 / 33 Virtualization Virtualization is a vital technique employed throughout the OS Given a physical resource, expose a virtual resource through layering and enforced modularity Users of the virtual resource (usually) cannot tell the difference Different forms: Multiplexing: Expose many virtual resources Aggregation: Combine many physical resources [RAID, Memory] Emulation: Provide a different virtual resource 2 / 33 Virtualization in Operating Systems Virtualizing CPU enables us to run multiple concurrent processes Mechanism: Time-division multiplexing and context switching Provides multiplexing and isolation Similarly, virtualizing memory provides each process the illusion/abstraction of a large, contiguous, and isolated “virtual” memory Virtualizing a resource enables safe multiplexing 3 / 33 Virtual Machines: Virtualizing the hardware Software abstraction Behaves like hardware Encapsulates all OS and application state Virtualization layer (aka Hypervisor) Extra level of indirection Decouples hardware and the OS Enforces isolation Multiplexes physical hardware across VMs 4 / 33 Hardware Virtualization History 1967: IBM System 360/ VM/370 fully virtualizable 1980s–1990s: “Forgotten”. x86 had no support 1999: VMWare. First x86 virtualization. 2003: Xen. Paravirtualization for Linux. Used by Amazon EC2 2006: Intel and AMD develop CPU extensions 2007: Linux Kernel Virtual Machines (KVM). Used by Google Cloud (and others). 5 / 33 Guest Operating Systems VMs run their own operating system (called “guest OS”) Full Virtualization: run unmodified guest OS. But, operating systems assume they have full control of actual hardware. With virtualization, they only have control over “virtual” hardware. Para Virtualization: Run virtualization-aware guest OS that participates and helps in the virtualization. Full machine hardware virtualization is challenging What happens when an instruction is executed? Memory accesses? Control I/O devices? Handle interrupts? File read/write? 6 / 33 Full Virtualization Requirements Isolation.
    [Show full text]
  • Protection Strategies for Direct Access to Virtualized I/O Devices
    Protection Strategies for Direct Access to Virtualized I/O Devices Paul Willmann, Scott Rixner, and Alan L. Cox Rice University {willmann, rixner, alc}@rice.edu Abstract chine virtualization [1, 2, 5, 8, 9, 10, 14, 19, 22]. How- Commodity virtual machine monitors forbid direct ac- ever, virtualization can impose performance penalties up cess to I/O devices by untrusted guest operating systems to a factor of 5 on I/O-intensive workloads [16, 19]. in order to provide protection and sharing. However, These penalties stem from the overhead of providing both I/O memory management units (IOMMUs) and re- shared and protected access to I/O devices by untrusted cently proposed software-based methods can be used to guest operating systems. Commodity virtualization ar- reduce the overhead of I/O virtualization by providing chitectures provide protection in software by forbidding untrusted guest operating systems with safe, direct ac- direct access to I/O hardware by untrusted guests— cess to I/O devices. This paper explores the performance instead, all I/O accesses, both commands and data, are and safety tradeoffs of strategies for using these mecha- routed through a single software entity that provides both nisms. protection and sharing. The protection strategies presented in this paper pro- Preferably, guest operating systems would be able to vide equivalent inter-guest protection among operating directly access I/O devices without the need for the data system instances. However, they provide varying levels to traverse an intermediate software layer within the vir- of intra-guest protection from driver software and incur tual machine monitor [17, 23].
    [Show full text]
  • Bringing Virtualization to the X86 Architecture with the Original Vmware Workstation
    12 Bringing Virtualization to the x86 Architecture with the Original VMware Workstation EDOUARD BUGNION, Stanford University SCOTT DEVINE, VMware Inc. MENDEL ROSENBLUM, Stanford University JEREMY SUGERMAN, Talaria Technologies, Inc. EDWARD Y. WANG, Cumulus Networks, Inc. This article describes the historical context, technical challenges, and main implementation techniques used by VMware Workstation to bring virtualization to the x86 architecture in 1999. Although virtual machine monitors (VMMs) had been around for decades, they were traditionally designed as part of monolithic, single-vendor architectures with explicit support for virtualization. In contrast, the x86 architecture lacked virtualization support, and the industry around it had disaggregated into an ecosystem, with different ven- dors controlling the computers, CPUs, peripherals, operating systems, and applications, none of them asking for virtualization. We chose to build our solution independently of these vendors. As a result, VMware Workstation had to deal with new challenges associated with (i) the lack of virtual- ization support in the x86 architecture, (ii) the daunting complexity of the architecture itself, (iii) the need to support a broad combination of peripherals, and (iv) the need to offer a simple user experience within existing environments. These new challenges led us to a novel combination of well-known virtualization techniques, techniques from other domains, and new techniques. VMware Workstation combined a hosted architecture with a VMM. The hosted architecture enabled a simple user experience and offered broad hardware compatibility. Rather than exposing I/O diversity to the virtual machines, VMware Workstation also relied on software emulation of I/O devices. The VMM combined a trap-and-emulate direct execution engine with a system-level dynamic binary translator to ef- ficiently virtualize the x86 architecture and support most commodity operating systems.
    [Show full text]
  • LIVE VIRTUAL MACHINE MIGRATION on AMD PROCESSORS Re-Scan the Hardware Upon Resume
    AMD WHITE PAPER Live Migration with AMD-V™ Extended Migration Technology Virtual Machine migration is a capability being increasingly utilized in today’s enterprise environments. With live migration, a Virtual Machine Monitor (VMM) moves a running Virtual Table of Contents Machine (VM) nearly instantaneously from one server to another for a seamless experience to the end user while maintaining guest uptime. However, if the processors running in the 1. Introduction . 1 computers in the pre and post migration environment are not identical, live migration can result 2. Virtual Machine in unexpected behavior of the guest software. Since the introduction of its AMD64 processor Migration . 2 technology in 2003, AMD has worked closely with virtualization software developers to defi ne the functionality necessary to ensure that live migration is possible across a broad range of 3. Determining Processor Features . 2 AMD64 processors. This whitepaper discusses the basics of live migration and describes the functionality currently available to enable virtualization software vendors to support live 4. VMM Basics. 3 migration across a broad range of AMD64 processors. 5. Problems with Live Migration . 3 1. INTRODUCTION Virtualization can increase utilization of computing System virtualization is the ability to abstract and pool resources by consolidating the workloads running on 5.1 Backward Compatibility resources on a physical platform. This abstraction many physical systems into virtual machines running Problem . 3 decouples software from hardware and enables multiple on a single physical system. Virtual machines can be 5.2 Forward Compatibility operating system images to run concurrently on a single provisioned on-demand, replicated and migrated.
    [Show full text]
  • The Evolution of an X86 Virtual Machine Monitor
    The Evolution of an x86 Virtual Machine Monitor Ole Agesen, Alex Garthwaite, Jeffrey Sheldon, Pratap Subrahmanyam {agesen, alextg, jeffshel, pratap}@vmware.com ABSTRACT While trap-and-emulate virtualization on mainframes was Twelve years have passed since VMware engineers first virtu- well understood at the time, it was – unsurprisingly – not a alized the x86 architecture. This technological breakthrough design goal for the 4004, whose first application was in fact kicked off a transformation of an entire industry, and virtu- a Busicom calculator [7]. However, as x86 CPUs expanded alization is now (once again) a thriving business with a wide their reach, the case for virtualization gradually emerged. range of solutions being deployed, developed and proposed. There was just one problem, seemingly fundamental: the But at the base of it all, the fundamental quest is still the generally accepted wisdom held that x86 virtualization was same: running virtual machines as well as we possibly can impossible, or at best merely impractical, due to inherited on top of a virtual machine monitor. architectural quirks and a requirement to retain compatibil- ity with legacy code. We review how the x86 architecture was originally virtual- ized in the days of the Pentium II (1998), and follow the This impasse was broken twelve years ago when VMware evolution of the virtual machine monitor forward through engineers first virtualized the x86 architecture entirely in the introduction of virtual SMP, 64 bit (x64), and hard- software. By basing the virtual machine monitor (VMM) ware support for virtualization to finish with a contempo- on binary translation, the architectural features that pre- rary challenge, nested virtualization.
    [Show full text]
  • Accelerating Two-Dimensional Page Walks for Virtualized Systems
    Accelerating Two-Dimensional Page Walks for Virtualized Systems Ravi Bhargava Benjamin Serebrin Francesco Spadini Srilatha Manne Computing Solutions Group Computing Solutions Group Computing Solutions Group Advanced Architecture & Technology Lab Advanced Micro Devices Advanced Micro Devices Advanced Micro Devices Advanced Micro Devices Austin, TX Sunnyvale, CA Austin, TX Bellevue, WA [email protected] [email protected] [email protected] [email protected] Abstract pervisor is the underlying software that inserts abstractions into a Nested paging is a hardware solution for alleviating the software virtualized system: an operating system (OS) becomes a guest OS, memory management overhead imposed by system virtualization. physical addresses become guest physical addresses,and,ingen- Nested paging complements existing page walk hardware to form eral, system elements that the OS presumed were real or physical a two-dimensional (2D) page walk,whichreducestheneedfor are converted into virtualized resources under control or manipula- hypervisor intervention in guest page table management. However, tion of the hypervisor [14, 16]. the extra dimension also increases the maximum number of archi- Ideally, a virtualized guest system will have comparable per- tecturally-required page table references. formance to an equivalent native, non-virtualized system. This can This paper presents an in-depth examination of the 2D page indeed be the case for compute-intensive applications. For exam- ple, the performance overhead for a virtualized system running table walk overhead and options for decreasing it. These options R include using the AMD OpteronTM processor’s page walk cache SPECint!2000 benchmarks can be less than 5% because the hy- to exploit the strong reuse of page entry references.
    [Show full text]
  • Computer Architectures an Overview
    Computer Architectures An Overview PDF generated using the open source mwlib toolkit. See http://code.pediapress.com/ for more information. PDF generated at: Sat, 25 Feb 2012 22:35:32 UTC Contents Articles Microarchitecture 1 x86 7 PowerPC 23 IBM POWER 33 MIPS architecture 39 SPARC 57 ARM architecture 65 DEC Alpha 80 AlphaStation 92 AlphaServer 95 Very long instruction word 103 Instruction-level parallelism 107 Explicitly parallel instruction computing 108 References Article Sources and Contributors 111 Image Sources, Licenses and Contributors 113 Article Licenses License 114 Microarchitecture 1 Microarchitecture In computer engineering, microarchitecture (sometimes abbreviated to µarch or uarch), also called computer organization, is the way a given instruction set architecture (ISA) is implemented on a processor. A given ISA may be implemented with different microarchitectures.[1] Implementations might vary due to different goals of a given design or due to shifts in technology.[2] Computer architecture is the combination of microarchitecture and instruction set design. Relation to instruction set architecture The ISA is roughly the same as the programming model of a processor as seen by an assembly language programmer or compiler writer. The ISA includes the execution model, processor registers, address and data formats among other things. The Intel Core microarchitecture microarchitecture includes the constituent parts of the processor and how these interconnect and interoperate to implement the ISA. The microarchitecture of a machine is usually represented as (more or less detailed) diagrams that describe the interconnections of the various microarchitectural elements of the machine, which may be everything from single gates and registers, to complete arithmetic logic units (ALU)s and even larger elements.
    [Show full text]
  • Oracle VM Server for X86 Virtualization and Management
    ORACLE DATA SHEET Oracle VM Server for x86 Virtualization and Management Oracle VM Server for x86 is a zero license cost server virtualization and management solution that makes enterprise applications easier to deploy, manage, and support. Backed worldwide by affordable enterprise-quality support for both Oracle and non-Oracle environments, Oracle VM reduces operations and support costs while increasing IT efficiency and agility. Engineered for Open Cloud Infrastructure ORACLE’S SERVER VIRTUALIZATION AND MANAGEMENT SOLUTION You are facing increased operating costs and inefficient resource utilization, and have an eye toward cloud computing. Your virtualization solution has to increase datacenter flexibility, meet KEY FEATURES AND BENEFITS your price/performance needs, and make applications easier to deploy, manage, and support. • Complete server virtualization and management solution with zero license Oracle VM delivers: cost • High performance and scalability: Low-overhead architecture with the Xen® hypervisor • Modern, low overhead architecture provides scalable performance under increasing workloads. Supports up to 384 physical based on the Xen hypervisor for leading price/performance CPUs and 6TB memory with each guest VM supporting up to 256 virtual CPUs and 2,000,000MB memory to accommodate the most demanding enterprise and cloud • Speeds application deployment with applications. Oracle VM Templates and virtual appliances • Broad guest operating system support: Oracle Linux, Oracle Solaris, Red Hat Enterprise Linux, SUSE Linux Enterprise Server, CentOS, Microsoft Windows. • Full Oracle VM Manager command-line interface (CLI) and Web Services API • Modern Dom0 kernel: Oracle Unbreakable Enterprise Kernel (UEK) Release 4 offers high (WS-API) allow greater automation and performance and streamlined partner certifications. interoperability • Dom0 UEK update on live systems: Oracle Ksplice updates the Dom0 UEK with all of the • Advanced virtualization features important security patches with no server reboot required.
    [Show full text]
  • A Comparison of Powervm and X86-Based Virtualization Performance
    A Comparison of PowerVM and x86-Based Virtualization Performance October 20, 2009 Elisabeth Stahl, Mala Anand IBM Systems and Technology Group IBM PowerVM Virtualization Virtualization technologies allow IT organizations to consolidate workloads running on multiple operating systems and software stacks and allocate platform resources dynamically to meet specific business and application requirements. Leadership virtualization has become the key technology to efficiently deploy servers in enterprise data centers to drive down costs and become the foundation for server pools and cloud computing technology. Therefore, the performance of this foundation technology is critical for the success of server pools and cloud computing. Virtualization may be employed to: • Consolidate multiple environments, including underutilized servers and systems with varied and dynamic resource requirements • Grow and shrink resources dynamically, derive energy efficiency, save space, and optimize resource utilization • Deploy new workloads through provisioning virtual machines or new systems rapidly to meet changing business demands • Develop and test applications in secure, independent domains while production can be isolated to its own domain on the same system • Transfer live workloads to support server migrations, balancing system load, or to avoid planned downtime • Control server sprawl and thereby reduce system management costs The latest advances in IBM® PowerVM™ offer broader platform support, greater scalability, higher efficiency in resource utilization, and more flexibility and robust heterogeneous server management than ever before. This paper will discuss the advantages of virtualization, highlight IBM PowerVM, and analyze the performance of virtualization technologies using industry-standard benchmarks. PowerVM With IBM Power™ Systems and IBM PowerVM virtualization technologies, an organization can consolidate applications and servers by using partitioning and virtualized system resources to provide a more flexible, dynamic IT infrastructure.
    [Show full text]
  • CS5460: Operating Systems Lecture: Virtualization
    CS5460: Operating Systems Lecture: Virtualization Anton Burtsev March, 2013 Traditional operating system Virtual machines A bit of history ● Virtual machines were popular in 60s-70s ● Share resources of mainframe computers [Goldberg 1974] ● Run multiple single-user operating systems ● Interest is lost by 80s-90s ● Development of multi-user OS ● Rapid drop in hardware cost ● Hardware support for virtualizaiton is lost What is the problem? ● Hardware is not designed to be multiplexed ● Loss of isolation Virtual machine Efficient duplicate of a real machine ● Compatibility ● Performance ● Isolation Trap and emulate What needs to be emulated? ● CPU and memory ● Register state ● Memory state ● Memory management unit ● Page tables, segments ● Platform ● Interrupt controller, timer, buses ● BIOS ● Peripheral devices ● Disk, network interface, serial line x86 is not virtualizable ● Some instructions (sensitive) read or update the state of virtual machine and don't trap (non- privileged) ● 17 sensitive, non-privileged instructions [Robin et al 2000] x86 is not virtualizable (II) ● Examples ● popf doesn't update interrupt flag (IF) – Impossible to detect when guest disables interrupts ● push %cs can read code segment selector (%cs) and learn its CPL – Guest gets confused Solution space ● Parse the instruction stream and detect all sensitive instructions dynamically ● Interpretation (BOCHS, JSLinux) ● Binary translation (VMWare, QEMU) ● Change the operating system ● Paravirtualization (Xen, L4, Denali, Hyper-V) ● Make all sensitive instructions privileged!
    [Show full text]