Virtualization Basics: Understanding Techniques and Fundamentals
Total Page:16
File Type:pdf, Size:1020Kb
Load more
Recommended publications
-
Better Performance Through a Disk/Persistent-RAM Hybrid Design
The Conquest File System: Better Performance Through a Disk/Persistent-RAM Hybrid Design AN-I ANDY WANG Florida State University GEOFF KUENNING Harvey Mudd College PETER REIHER, GERALD POPEK University of California, Los Angeles ________________________________________________________________________ Modern file systems assume the use of disk, a system-wide performance bottleneck for over a decade. Current disk caching and RAM file systems either impose high overhead to access memory content or fail to provide mechanisms to achieve data persistence across reboots. The Conquest file system is based on the observation that memory is becoming inexpensive, which enables all file system services to be delivered from memory, except providing large storage capacity. Unlike caching, Conquest uses memory with battery backup as persistent storage, and provides specialized and separate data paths to memory and disk. Therefore, the memory data path contains no disk-related complexity. The disk data path consists of only optimizations for the specialized disk usage pattern. Compared to a memory-based file system, Conquest incurs little performance overhead. Compared to several disk-based file systems, Conquest achieves 1.3x to 19x faster memory performance, and 1.4x to 2.0x faster performance when exercising both memory and disk. Conquest realizes most of the benefits of persistent RAM at a fraction of the cost of a RAM-only solution. Conquest also demonstrates that disk-related optimizations impose high overheads for accessing memory content in a memory-rich environment. Categories and Subject Descriptors: D.4.2 [Operating Systems]: Storage Management—Storage Hierarchies; D.4.3 [Operating Systems]: File System Management—Access Methods and Directory Structures; D.4.8 [Operating Systems]: Performance—Measurements General Terms: Design, Experimentation, Measurement, and Performance Additional Key Words and Phrases: Persistent RAM, File Systems, Storage Management, and Performance Measurement ________________________________________________________________________ 1. -
Evaluating and Optimizing I/O Virtualization in Kernel-Based Virtual Machine (KVM)
Evaluating and Optimizing I/O Virtualization in Kernel-based Virtual Machine (KVM) Binbin Zhang1, Xiaolin Wang1, Rongfeng Lai1, Liang Yang1, Zhenlin Wang2, Yingwei Luo1, and Xiaoming Li1 1 Dept. of Computer Science and Technology, Peking University, Beijing, China, 100871 2 Dept. of Computer Science, Michigan Technological University, Houghton, USA {wxl,lyw}@pku.edu.cn, [email protected] Abstract. I/O virtualization performance is an important problem in KVM. In this paper, we evaluate KVM I/O performance and propose several optimiza- tions for improvement. First, we reduce VM Exits by merging successive I/O instructions and decreasing the frequency of timer interrupt. Second, we simplify the Guest OS by removing redundant operations when the guest OS operates in a virtual environment. We eliminate the operations that are useless in the virtual environment and bypass the I/O scheduling in the Guest OS whose results will be rescheduled in the Host OS. We also change NIC driver’s con- figuration in Guest OS to adapt the virtual environment for better performance. Keywords: Virtualization, KVM, I/O Virtualization, Optimization. 1 Introduction Software emulation is used as the key technique in I/O device virtualization in Ker- nel-based Virtual Machine (KVM). KVM uses a kernel module to intercept I/O re- quests from a Guest OS, and passes them to QEMU, an emulator running on the user space of Host OS. QEMU translates these requests into system calls to the Host OS, which will access the physical devices via device drivers. This implementation of VMM is simple, but the performance is usually not satisfactory because multiple environments are involved in each I/O operation that results in multiple context switches and long scheduling latency. -
Effective Virtual CPU Configuration with QEMU and Libvirt
Effective Virtual CPU Configuration with QEMU and libvirt Kashyap Chamarthy <[email protected]> Open Source Summit Edinburgh, 2018 1 / 38 Timeline of recent CPU flaws, 2018 (a) Jan 03 • Spectre v1: Bounds Check Bypass Jan 03 • Spectre v2: Branch Target Injection Jan 03 • Meltdown: Rogue Data Cache Load May 21 • Spectre-NG: Speculative Store Bypass Jun 21 • TLBleed: Side-channel attack over shared TLBs 2 / 38 Timeline of recent CPU flaws, 2018 (b) Jun 29 • NetSpectre: Side-channel attack over local network Jul 10 • Spectre-NG: Bounds Check Bypass Store Aug 14 • L1TF: "L1 Terminal Fault" ... • ? 3 / 38 Related talks in the ‘References’ section Out of scope: Internals of various side-channel attacks How to exploit Meltdown & Spectre variants Details of performance implications What this talk is not about 4 / 38 Related talks in the ‘References’ section What this talk is not about Out of scope: Internals of various side-channel attacks How to exploit Meltdown & Spectre variants Details of performance implications 4 / 38 What this talk is not about Out of scope: Internals of various side-channel attacks How to exploit Meltdown & Spectre variants Details of performance implications Related talks in the ‘References’ section 4 / 38 OpenStack, et al. libguestfs Virt Driver (guestfish) libvirtd QMP QMP QEMU QEMU VM1 VM2 Custom Disk1 Disk2 Appliance ioctl() KVM-based virtualization components Linux with KVM 5 / 38 OpenStack, et al. libguestfs Virt Driver (guestfish) libvirtd QMP QMP Custom Appliance KVM-based virtualization components QEMU QEMU VM1 VM2 Disk1 Disk2 ioctl() Linux with KVM 5 / 38 OpenStack, et al. libguestfs Virt Driver (guestfish) Custom Appliance KVM-based virtualization components libvirtd QMP QMP QEMU QEMU VM1 VM2 Disk1 Disk2 ioctl() Linux with KVM 5 / 38 libguestfs (guestfish) Custom Appliance KVM-based virtualization components OpenStack, et al. -
Virtual Memory
Chapter 4 Virtual Memory Linux processes execute in a virtual environment that makes it appear as if each process had the entire address space of the CPU available to itself. This virtual address space extends from address 0 all the way to the maximum address. On a 32-bit platform, such as IA-32, the maximum address is 232 − 1or0xffffffff. On a 64-bit platform, such as IA-64, this is 264 − 1or0xffffffffffffffff. While it is obviously convenient for a process to be able to access such a huge ad- dress space, there are really three distinct, but equally important, reasons for using virtual memory. 1. Resource virtualization. On a system with virtual memory, a process does not have to concern itself with the details of how much physical memory is available or which physical memory locations are already in use by some other process. In other words, virtual memory takes a limited physical resource (physical memory) and turns it into an infinite, or at least an abundant, resource (virtual memory). 2. Information isolation. Because each process runs in its own address space, it is not possible for one process to read data that belongs to another process. This improves security because it reduces the risk of one process being able to spy on another pro- cess and, e.g., steal a password. 3. Fault isolation. Processes with their own virtual address spaces cannot overwrite each other’s memory. This greatly reduces the risk of a failure in one process trig- gering a failure in another process. That is, when a process crashes, the problem is generally limited to that process alone and does not cause the entire machine to go down. -
Oracle VM Virtualbox User Manual
Oracle VM VirtualBox R User Manual Version 5.1.20 c 2004-2017 Oracle Corporation http://www.virtualbox.org Contents 1 First steps 11 1.1 Why is virtualization useful?............................. 12 1.2 Some terminology................................... 12 1.3 Features overview................................... 13 1.4 Supported host operating systems.......................... 15 1.5 Installing VirtualBox and extension packs...................... 16 1.6 Starting VirtualBox.................................. 17 1.7 Creating your first virtual machine......................... 18 1.8 Running your virtual machine............................ 21 1.8.1 Starting a new VM for the first time.................... 21 1.8.2 Capturing and releasing keyboard and mouse.............. 22 1.8.3 Typing special characters.......................... 23 1.8.4 Changing removable media......................... 24 1.8.5 Resizing the machine’s window...................... 24 1.8.6 Saving the state of the machine...................... 25 1.9 Using VM groups................................... 26 1.10 Snapshots....................................... 26 1.10.1 Taking, restoring and deleting snapshots................. 27 1.10.2 Snapshot contents.............................. 28 1.11 Virtual machine configuration............................ 29 1.12 Removing virtual machines.............................. 30 1.13 Cloning virtual machines............................... 30 1.14 Importing and exporting virtual machines..................... 31 1.15 Global Settings................................... -
Virtualizing Your Network: Benefits & Challenges
Network Monitoring Technology Virtualizing Your Network: Benefits & Challenges What is Virtualization? factory or process plant. Virtualization can give this software Gartner Research1 defined network virtualization as the a longer operating lifecycle time, and can save both costs process of combining hardware and software network and intellectual property. resources and functionality into a single virtual network. This offers access to routing features and data streams that can provide newer, service-aware, resilient solutions; newer security services that are native within network elements; support for subscriber-aware policy control for peer-to- peer traffic management; and application-aware, real-time session control for converged voice and video applications with guaranteed bandwidth on-demand. For the most part, when we speak of virtualization, we speak of hardware virtualization. That means that we create, on a host machine, a virtual machine that looks like another computer with an operating system and software. The software on the virtual machine is separate from the host machine’s resources, and as far as it is concerned, it is running on its own computer (that we call the guest). Both in information technology (IT) and in operational technology (OT) environments the benefits of virtualization have led to its rapid adoption. This white paper is not a prescriptive guide to network virtualization, rather it is a high-level overview focused on the benefits and challenges of network virtualization. While it will Desktop virtualization separates the logical desktop from review the benefits, it will also cover the specific challenges the actual hardware. Virtual desktop infrastructure (VDI) network administrators and their respective businesses should permits the user to interact with the computer through understand to cost-effectively apply this technology to gain another host computer or device on a network connection. -
A Virtual Machine Environment for Real Time Systems Laboratories
AC 2007-904: A VIRTUAL MACHINE ENVIRONMENT FOR REAL-TIME SYSTEMS LABORATORIES Mukul Shirvaikar, University of Texas-Tyler MUKUL SHIRVAIKAR received the Ph.D. degree in Electrical and Computer Engineering from the University of Tennessee in 1993. He is currently an Associate Professor of Electrical Engineering at the University of Texas at Tyler. He has also held positions at Texas Instruments and the University of West Florida. His research interests include real-time imaging, embedded systems, pattern recognition, and dual-core processor architectures. At the University of Texas he has started a new real-time systems lab using dual-core processor technology. He is also the principal investigator for the “Back-To-Basics” project aimed at engineering student retention. Nikhil Satyala, University of Texas-Tyler NIKHIL SATYALA received the Bachelors degree in Electronics and Communication Engineering from the Jawaharlal Nehru Technological University (JNTU), India in 2004. He is currently pursuing his Masters degree at the University of Texas at Tyler, while working as a research assistant. His research interests include embedded systems, dual-core processor architectures and microprocessors. Page 12.152.1 Page © American Society for Engineering Education, 2007 A Virtual Machine Environment for Real Time Systems Laboratories Abstract The goal of this project was to build a superior environment for a real time system laboratory that would allow users to run Windows and Linux embedded application development tools concurrently on a single computer. These requirements were dictated by real-time system applications which are increasingly being implemented on asymmetric dual-core processors running different operating systems. A real time systems laboratory curriculum based on dual- core architectures has been presented in this forum in the past.2 It was designed for a senior elective course in real time systems at the University of Texas at Tyler that combines lectures along with an integrated lab. -
Chapter 3. Booting Operating Systems
Chapter 3. Booting Operating Systems Abstract: Chapter 3 provides a complete coverage on operating systems booting. It explains the booting principle and the booting sequence of various kinds of bootable devices. These include booting from floppy disk, hard disk, CDROM and USB drives. Instead of writing a customized booter to boot up only MTX, it shows how to develop booter programs to boot up real operating systems, such as Linux, from a variety of bootable devices. In particular, it shows how to boot up generic Linux bzImage kernels with initial ramdisk support. It is shown that the hard disk and CDROM booters developed in this book are comparable to GRUB and isolinux in performance. In addition, it demonstrates the booter programs by sample systems. 3.1. Booting Booting, which is short for bootstrap, refers to the process of loading an operating system image into computer memory and starting up the operating system. As such, it is the first step to run an operating system. Despite its importance and widespread interests among computer users, the subject of booting is rarely discussed in operating system books. Information on booting are usually scattered and, in most cases, incomplete. A systematic treatment of the booting process has been lacking. The purpose of this chapter is to try to fill this void. In this chapter, we shall discuss the booting principle and show how to write booter programs to boot up real operating systems. As one might expect, the booting process is highly machine dependent. To be more specific, we shall only consider the booting process of Intel x86 based PCs. -
Understanding Full Virtualization, Paravirtualization, and Hardware Assist
VMware Understanding Full Virtualization, Paravirtualization, and Hardware Assist Contents Introduction .................................................................................................................1 Overview of x86 Virtualization..................................................................................2 CPU Virtualization .......................................................................................................3 The Challenges of x86 Hardware Virtualization ...........................................................................................................3 Technique 1 - Full Virtualization using Binary Translation......................................................................................4 Technique 2 - OS Assisted Virtualization or Paravirtualization.............................................................................5 Technique 3 - Hardware Assisted Virtualization ..........................................................................................................6 Memory Virtualization................................................................................................6 Device and I/O Virtualization.....................................................................................7 Summarizing the Current State of x86 Virtualization Techniques......................8 Full Virtualization with Binary Translation is the Most Established Technology Today..........................8 Hardware Assist is the Future of Virtualization, but the Real Gains Have -
Introduction to Virtualization
z Systems Introduction to Virtualization SHARE Orlando Linux and VM Program Romney White, IBM [email protected] z Systems Architecture and Technology © 2015 IBM Corporation Agenda ° Introduction to Virtualization – Concept – Server Virtualization Approaches – Hypervisor Implementation Methods – Why Virtualization Matters ° Virtualization on z Systems – Logical Partitions – Virtual Machines 2 z Systems Virtualization Technology © 2015 IBM Corporation Virtualization Concept Virtual Resources Proxies for real resources: same interfaces/functions, different attributes May be part of a physical resource or multiple physical resources Virtualization Creates virtual resources and "maps" them to real resources Primarily accomplished with software or firmware Resources Components with architecturally-defined interfaces/functions May be centralized or distributed - usually physical Examples: memory, disk drives, networks, servers Separates presentation of resources to users from actual resources Aggregates pools of resources for allocation to users as virtual resources 3 z Systems Virtualization Technology © 2015 IBM Corporation Server Virtualization Approaches Hardware Partitioning Bare-metal Hypervisor Hosted Hypervisor Apps ... Apps Apps ... Apps Apps ... Apps OS OS OS OS OS OS Adjustable partitions Hypervisor Hypervisor Partition Controller Host OS SMP Server SMP Server SMP Server Server is subdivided into fractions Hypervisor provides fine-grained Hypervisor uses OS services to each of which can run an OS timesharing of all resources -
ELF1 7D Virtual Memory
ELF1 7D Virtual Memory Young W. Lim 2021-07-06 Tue Young W. Lim ELF1 7D Virtual Memory 2021-07-06 Tue 1 / 77 Outline 1 Based on 2 Virtual memory Virtual Memory in Operating System Physical, Logical, Virtual Addresses Kernal Addresses Kernel Logical Address Kernel Virtual Address User Virtual Address User Space Young W. Lim ELF1 7D Virtual Memory 2021-07-06 Tue 2 / 77 Based on "Study of ELF loading and relocs", 1999 http://netwinder.osuosl.org/users/p/patb/public_html/elf_ relocs.html I, the copyright holder of this work, hereby publish it under the following licenses: GNU head Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license is included in the section entitled GNU Free Documentation License. CC BY SA This file is licensed under the Creative Commons Attribution ShareAlike 3.0 Unported License. In short: you are free to share and make derivative works of the file under the conditions that you appropriately attribute it, and that you distribute it only under a license compatible with this one. Young W. Lim ELF1 7D Virtual Memory 2021-07-06 Tue 3 / 77 Compling 32-bit program on 64-bit gcc gcc -v gcc -m32 t.c sudo apt-get install gcc-multilib sudo apt-get install g++-multilib gcc-multilib g++-multilib gcc -m32 objdump -m i386 Young W. -
Enabling Intel® Virtualization Technology Features and Benefits
WHITE PAPER Intel® Virtualization Technology Enterprise Server Enabling Intel® Virtualization Technology Features and Benefits Maximizing the benefits of virtualization with Intel’s new CPUs and chipsets EXECUTIVE SUMMARY Although virtualization has been accepted in most data centers, some users have not yet taken advantage of all the virtualization features available to them. This white paper describes the features available in Intel® Virtualization Technology (Intel® VT) that work with Intel’s new CPUs and chipsets, showing how they can benefit the end user and how to enable them. Intel® Virtualization Technology Goldberg. Thus, developers found it difficult Feature Brief and Usage Model to implement a virtual machine platform on Intel VT combines with software-based the x86 architecture without significant virtualization solutions to provide maximum overhead on the host machine. system utilization by consolidating multiple environments into a single server or PC. In 2005 and 2006, Intel and AMD, working By abstracting the software away from the independently, each resolved this by creat- underlying hardware, a world of new usage ing new processor extensions to the x86 models opens up that can reduce costs, architecture. Although the actual implemen- increase management efficiency, and tation of processor extensions differs strengthen security—all while making your between AMD and Intel, both achieve the computing infrastructure more resilient in same goal of allowing a virtual machine the event of a disaster. hypervisor to run an unmodified operating system without incurring significant emula- During the last four years, Intel has intro- tion performance penalties. duced several new features to Intel VT. Most of these features are well known, but others Intel VT is Intel’s hardware virtualization for may not be.