DOCSLIB.ORG

Virtualization

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Virtualization Virtualization Nezer Jacob Zaidenberg University of Jyvaskyl¨ a,¨ Finland Abstract process virtualization as opposed to native code, cross cutting aspects implementation in virtual machines and This course is hands on course for advanced undergradu- implementations such as JVM and LLVM. ate, graduate students and experienced developers. Soft- ware development skills in C and Linux are required. We introduce aspects of virtualization. System virtu- 3.1 Java virtual machine (JVM) alization (such as VMWare), Proces virtualization (such We will discuss features and design principals of JVM as JVM) , Storage virtualization and cloud based virtual- including Just-in-time, sandboxing of applets etc. ization. 3.2 Low level virtual machine (LLVM) 1 Virtualization We will discuss features and design principals of LLVM We will discuss theoretical and practical models for vir- and the CLang compiler. tualization. The goals for running software on virtual machines. 4 Storage virtualization 2 System virtualization Introduction to storage virtualization concepts. Replica- tion, mirroring, snapshots, distributed file systems. System virtualization refers to running a complete sys- tem in a virtual environment (Products such as Virtual 5 Cloud based virtualization Box, Xen, VMWare etc.) We will review the goals of system virtualizations and why are they used. We will discuss clustering issues of virtualization. We will use Linux Virtual Server as a simple example 2.1 Lguest to using virtual IP in the cloud. We will also discuss running application in GRID sandbox. Lguest[1] is a minial system virtuaization implemented in the linux kernel for padagogic purposes. We will re- view Lguest code and understand how to build a system 6 Duration virtualization environment. The course will be given in 16 hours. 4 hours a day for 4 days. The course will be taught in English! 2.2 Virt I/O Virtual I/O are the basis for paravirtualization drivers in 6.1 First day Linux. We will review the Virt I/O infrastructure. • Introduction to virtualization. 3 Process virtualization • Introduction to Process virtualization • JVM Process virtualization refers to running a process inside a virtual machine such as CLR, JVM etc. We will discuss • LLVM & CLANG 6.2 Second day 8 Prior knowledge requirement • Introduction to system virtualization 1. Basic familiarity with OS and computer architec- • Introducing lguest ture. You need to understand how interrupts, device drivers etc. is implemented normally before you can • Code review of lguest boot process understand how to virtualize them. • Code review of lguest char dev 2. Solid C background Please examine Linux source tree for example at http: 6.3 third day //lxr.linux.no/linux+v2.6.37/ Documentation/lguest/lguest.c and • Code review of lguest launcher http://lxr.linux.no/linux+v2.6.37/ drivers/lguest/lguest device.c If you • Code review of lguest kernel module are unable to read the code please do not sign up to • Code review of virt I/O driver. the course. • Introduction to Cloud based virtualization 3. Using Linux/UNIX (shell and development tools) 4. kernel development skills recommended but not re- 6.4 forth day quired. • Linux Virtual Server (LVS) 5. Assembler for x86 is recommended but not re- • Introduction to storage virtualization quired. • Storage virtualization in Linux block layer References • Summary and projects [1] RUSSELL, R. Rusty’s remarkably unreliable guide to Lguest. http://lguest.ozlabs.org/lguest.txt. 7 Grading Following the course each student will have to submit a final project. The final project should include some of the ideas mentioned in class and will be implemented on top of lguest, llvm or other virtualization model. Project can be done in pairs and project ideas include but are not limited to: • Adding aspects to process virtual machine (such as profiling etc.) • Support for ”on the fly” insertion or removal of devices from guest (like plugging and unplugging thumb drive) • An efficient guest to guest communication system • An efficient guest to host communication system • Support for external devices in guest (like Fi- brechannel or Firewire in guest) • An interface for debugging the current kernel run- ning on lguest via kernel debugger. • Network testing tools using virtualization (simulat- ing packet losses, packets out of order etc.) • Any other project, of proper volume, that involves lguest, VirtIO or virtualization 2.
Load more

Recommended publications
  • Lguest a Journey of Learning the Linux Kernel Internals
    Lguest A journey of learning the Linux kernel internals Daniel Baluta <[email protected]> What is lguest? ● “Lguest is an adventure, with you, the reader, as a Hero” ● Minimal 32-bit x86 hypervisor ● Introduced in 2.6.23 (October, 2007) by Rusty Russel & co ● Around 6000 lines of code ● “Relatively” “easy” to understand and hack on ● Support to learn linux kernel internals ● Porting to other architectures is a challenging task Hypervisor types make Preparation ● Guest ○ Life of a guest ● Drivers ○ virtio devices: console, block, net ● Launcher ○ User space program that sets up and configures Guests ● Host ○ Normal linux kernel + lg.ko kernel module ● Switcher ○ Low level Host <-> Guest switch ● Mastery ○ What’s next? Lguest overview make Guest ● “Simple creature, identical to the Host [..] behaving in simplified ways” ○ Same kernel as Host (or at least, built from the same source) ● booting trace (when using bzImage) ○ arch/x86/boot/compressed/head_32.S: startup_32 ■ uncompresses the kernel ○ arch/x86/kernel/head_32.S : startup_32 ■ initialization , checks hardware_subarch field ○ arch/x86/lguest/head_32.S ■ LGUEST_INIT hypercall, jumps to lguest_init ○ arch/x86/lguest/boot.c ■ once we are here we know we are a guest ■ override privileged instructions ■ boot as normal kernel, calling i386_start_kernel Hypercalls struct lguest_data ● second communication method between Host and Guest ● LHCALL_GUEST_INIT ○ hypercall to tell the Host where lguest_data struct is ● both Guest and Host publish information here ● optimize hypercalls ○ irq_enabled
    [Show full text]
  • Lguest the Little Hypervisor
    Lguest the little hypervisor Matías Zabaljáuregui [email protected] based on lguest documentation written by Rusty Russell note: this is a draft and incomplete version lguest introduction launcher guest kernel host kernel switcher don't have time for: virtio virtual devices patching technique async hypercalls rewriting technique guest virtual interrupt controller guest virtual clock hw assisted vs paravirtualization hybrid virtualization benchmarks introduction A hypervisor allows multiple Operating Systems to run on a single machine. lguest paravirtualization within linux kernel proof of concept (paravirt_ops, virtio...) teaching tool guests and hosts A module (lg.ko) allows us to run other Linux kernels the same way we'd run processes. We only run specially modified Guests. Setting CONFIG_LGUEST_GUEST, compiles [arch/x86/lguest/boot.c] into the kernel so it knows how to be a Guest at boot time. This means that you can use the same kernel you boot normally (ie. as a Host) as a Guest. These Guests know that they cannot do privileged operations, such as disable interrupts, and that they have to ask the Host to do such things explicitly. Some of the replacements for such low-level native hardware operations call the Host through a "hypercall". [ drivers/lguest/hypercalls.c ] high level overview launcher /dev/lguest guest kernel host kernel switcher Launcher main() some implementations /dev/lguest write → initialize read → run guest Launcher main [ Documentation/lguest/lguest.c ] The Launcher is the Host userspace program which sets up, runs and services the Guest. Just to confuse you: to the Host kernel, the Launcher *is* the Guest and we shall see more of that later.
    [Show full text]
  • Proceedings of the Linux Symposium
    Proceedings of the Linux Symposium Volume One June 27th–30th, 2007 Ottawa, Ontario Canada Contents The Price of Safety: Evaluating IOMMU Performance 9 Ben-Yehuda, Xenidis, Mostrows, Rister, Bruemmer, Van Doorn Linux on Cell Broadband Engine status update 21 Arnd Bergmann Linux Kernel Debugging on Google-sized clusters 29 M. Bligh, M. Desnoyers, & R. Schultz Ltrace Internals 41 Rodrigo Rubira Branco Evaluating effects of cache memory compression on embedded systems 53 Anderson Briglia, Allan Bezerra, Leonid Moiseichuk, & Nitin Gupta ACPI in Linux – Myths vs. Reality 65 Len Brown Cool Hand Linux – Handheld Thermal Extensions 75 Len Brown Asynchronous System Calls 81 Zach Brown Frysk 1, Kernel 0? 87 Andrew Cagney Keeping Kernel Performance from Regressions 93 T. Chen, L. Ananiev, and A. Tikhonov Breaking the Chains—Using LinuxBIOS to Liberate Embedded x86 Processors 103 J. Crouse, M. Jones, & R. Minnich GANESHA, a multi-usage with large cache NFSv4 server 113 P. Deniel, T. Leibovici, & J.-C. Lafoucrière Why Virtualization Fragmentation Sucks 125 Justin M. Forbes A New Network File System is Born: Comparison of SMB2, CIFS, and NFS 131 Steven French Supporting the Allocation of Large Contiguous Regions of Memory 141 Mel Gorman Kernel Scalability—Expanding the Horizon Beyond Fine Grain Locks 153 Corey Gough, Suresh Siddha, & Ken Chen Kdump: Smarter, Easier, Trustier 167 Vivek Goyal Using KVM to run Xen guests without Xen 179 R.A. Harper, A.N. Aliguori & M.D. Day Djprobe—Kernel probing with the smallest overhead 189 M. Hiramatsu and S. Oshima Desktop integration of Bluetooth 201 Marcel Holtmann How virtualization makes power management different 205 Yu Ke Ptrace, Utrace, Uprobes: Lightweight, Dynamic Tracing of User Apps 215 J.
    [Show full text]
  • Bunker: a Privacy-Oriented Platform for Network Tracing Andrew G
    Bunker: A Privacy-Oriented Platform for Network Tracing Andrew G. Miklas†, Stefan Saroiu‡, Alec Wolman‡, and Angela Demke Brown† †University of Toronto and ‡Microsoft Research Abstract: ISPs are increasingly reluctant to collect closing social security numbers, names, addresses, or and store raw network traces because they can be used telephone numbers [5, 54]. to compromise their customers’ privacy. Anonymization Trace anonymization is the most common technique techniques mitigate this concern by protecting sensitive information. Trace anonymization can be performed of- for addressing these privacy concerns. A typical imple- fline (at a later time) or online (at collection time). Of- mentation uses a keyed one-way secure hash function to fline anonymization suffers from privacy problems be- obfuscate sensitive information contained in the trace. cause raw traces must be stored on disk – until the traces This could be as simple as transforming a few fields in are deleted, there is the potential for accidental leaks or the IP headers, or as complex as performing TCP connec- exposure by subpoenas. Online anonymization drasti- tion reconstruction and then obfuscating data (e.g., email cally reduces privacy risks but complicates software en- addresses) deep within the payload. There are two cur- gineering efforts because trace processing and anony- mization must be performed at line speed. This paper rent approaches to anonymizing network traces: offline presents Bunker, a network tracing system that combines and online. Offline anonymization collects and stores the software development benefits of offline anonymiz- the entire raw trace and then performs anonymization ation with the privacy benefits of online anonymization. as a post-processing step.
    [Show full text]
  • Virtual Servers and Checkpoint/Restart in Mainstream Linux
    Virtual Servers and Checkpoint/Restart in Mainstream Linux Sukadev Bhattiprolu Eric W. Biederman Serge Hallyn IBM Arastra IBM [email protected] [email protected] [email protected] Daniel Lezcano IBM [email protected] ABSTRACT one task will use one table to look for a task corresponding Virtual private servers and application checkpoint and restart to a specific pid, in other word this table is relative to the are two advanced operating system features which place dif- task. ferent but related requirements on the way kernel-provided For the past several years, we have been converting kernel- resources are accessed by userspace. In Linux, kernel re- provided resource namespaces in Linux from a single, global sources, such as process IDs and SYSV shared messages, table per resource, to Plan9-esque [31] per-process names- have traditionally been identified using global tables. Since paces for each resource. This conversion has enjoyed the for- 2005, these tables have gradually been transformed into per- tune of being deemed by many in the Linux kernel develop- process namespaces in order to support both resource avail- ment community as a general code improvement; a cleanup ability on application restart and virtual private server func- worth doing for its own sake. This perception was a tremen- tionality. Due to inherent differences in the resources them- dous help in pushing for patches to be accepted. But there selves, the semantics of namespace cloning differ for many are two additional desirable features which motivated the of the resources. This paper describes the existing and pro- authors.
    [Show full text]
  • Using KVM to Run Xen Guests Without Xen
    Using KVM to run Xen guests without Xen Ryan A. Harper Michael D. Day Anthony N. Liguori IBM IBM IBM [email protected] [email protected] [email protected] Abstract on providing future extensions to improve MMU and hardware virtualization. The inclusion of the Kernel Virtual Machine (KVM) Linux has also recently gained a set of x86 virtual- driver in Linux 2.6.20 has dramatically improved ization enhancements. For the 2.6.20 kernel release, Linux’s ability to act as a hypervisor. Previously, Linux the paravirt_ops interface and KVM driver were was only capable of running UML guests and contain- added. paravirt_ops provides a common infras- ers. KVM adds support for running unmodified x86 tructure for hooking portions of the Linux kernel that operating systems using hardware-based virtualization. would traditionally prevent virtualization. The KVM The current KVM user community is limited by the driver adds the ability for Linux to run unmodified availability of said hardware. guests, provided that the underlying processor supports The Xen hypervisor, which can also run unmodified op- either AMD or Intel’s CPU virtualization extensions. erating systems with hardware virtualization, introduced Currently, there are three consumers of the a paravirtual ABI for running modified operating sys- paravirt_ops interface: Lguest [Lguest], a tems such as Linux, Netware, FreeBSD, and OpenSo- “toy” hypervisor designed to provide an example laris. This ABI not only provides a performance advan- implementation; VMI [VMI], which provides support tage over running unmodified guests, it does not require for guests running under VMware; and XenoLinux any hardware support, increasing the number of users [Xen], which provides support for guests running under that can utilize the technology.
    [Show full text]
  • Checkpoint-Restart for a Network of Virtual Machines
    Checkpoint-Restart for a Network of Virtual Machines Rohan Garg, Komal Sodha, Zhengping Jin, Gene Cooperman∗ Northeastern University Boston, MA / USA {rohgarg,komal,jinzp,gene}@ccs.neu.edu Abstract—The ability to easily deploy parallel compu- Further, the maintainability of a proposed architecture tations on the Cloud is becoming ever more important. is important. Here, we measure maintainability by the The first uniform mechanism for checkpointing a network number of lines of new code required, beyond the base of virtual machines is described. This is important for the parallel versions of common productivity software. code of a checkpoint-restart package, or the base code Potential examples of parallelism include Simulink for of the virtual machine itself. The proposed architecture MATLAB, parallel R for the R statistical modelling relies on just 600 lines of new code: 400 lines of code language, parallel blast.py for the BLAST bioinformatics for a KVM-specific plugin used to checkpoint the virtual software, IPython.parallel for Python, and GNU parallel machine, and 200 lines of code for a TUN/TAP plugin. for parallel shells. The checkpoint mechanism is imple- mented as a plugin in the DMTCP checkpoint-restart The two DMTCP plugins above are external libraries package. It operates on KVM/QEMU, and has also been loaded into an unmodified DMTCP. Source code can be adapted to Lguest and pure user-space QEMU. The plugin found in the contrib directory of the DMTCP repository. is surprisingly compact, comprising just 400 lines of code (See Section II for further details of plugins.) to checkpoint a single virtual machine, and 200 lines of The approach described here saves the state of an code for a plugin to support saving and restoring network state.
    [Show full text]
  • Paravirtualizing Linux in a Real-Time Hypervisor
    Paravirtualizing Linux in a real-time hypervisor Vincent Legout and Matthieu Lemerre CEA, LIST, Embedded Real Time Systems Laboratory Point courrier 172, F-91191 Gif-sur-Yvette, FRANCE {vincent.legout,matthieu.lemerre}@cea.fr ABSTRACT applications written for a fully-featured operating system This paper describes a new hypervisor built to run Linux in a such as Linux. The Anaxagoros design prevent non real- virtual machine. This hypervisor is built inside Anaxagoros, time tasks to interfere with real-time tasks, thus providing a real-time microkernel designed to execute safely hard real- the security foundation to build a hypervisor to run existing time and non real-time tasks. This allows the execution of non real-time applications. This allows current applications hard real-time tasks in parallel with Linux virtual machines to run on Anaxagoros systems without any porting effort, without interfering with the execution of the real-time tasks. opening the access to a wide range of applications. We implemented this hypervisor and compared perfor- Furthermore, modern computers are powerful enough to mances with other virtualization techniques. Our hypervisor use virtualization, even embedded processors. Virtualiza- does not yet provide high performance but gives correct re- tion has become a trendy topic of computer science, with sults and we believe the design is solid enough to guarantee its advantages like scalability or security. Thus we believe solid performances with its future implementation. that building a real-time system with guaranteed real-time performances and dense non real-time tasks is an important topic for the future of real-time systems.
    [Show full text]
  • Enabling Lightweight Multi-Tenancy at the Network's Extreme Edge
    Paradrop: Enabling Lightweight Multi-tenancy at the Network’s Extreme Edge Peng Liu Dale Willis Suman Banerjee Department of Computer Sciences Department of Computer Sciences Department of Computer Sciences University of Wisconsin-Madison University of Wisconsin-Madison University of Wisconsin-Madison Email: [email protected] Email: [email protected] Email: [email protected] Abstract—We introduce, Paradrop, a specific edge computing This paper presents a specific edge computing framework, platform that provides (modest) computing and storage resources Paradrop, implemented on WiFi Access Points (APs) or other at the “extreme” edge of the network allowing third-party devel- wireless gateways (such as, set-top boxes) which allows third- opers to flexibly create new types of services. This extreme edge of the network is the WiFi Access Point (AP) or the wireless gateway party developers to bring computational functions right into through which all end-device traffic (personal devices, sensors, homes and enterprises. The WiFi AP is a unique platform for etc.) pass through. Paradrop’s focus on the WiFi APs also stems edge computing because of multiple reasons: it is ubiquitous in from the fact that the WiFi AP has unique contextual knowledge homes and enterprises, is always “on” and available, and sits of its end-devices (e.g., proximity, channel characteristics) that are directly in the data path between Internet resources and the end lost as we get deeper into the network. While different variations and implementations of edge computing platforms
    [Show full text]
  • Information Session for the Seminars “Future Internet” and “Innovative Internet Technologies and Mobile Communications”
    Chair of Network Architectures and Services Faculty of Informatics Technical University of Munich Information Session for the Seminars “Future Internet” and “Innovative Internet Technologies and Mobile Communications” Prof. Dr.-Ing. Georg Carle and I8 research staff Organization: Daniel Raumer Contact: [email protected] Content Administrative Things for all Seminars . Basic Information . Grading . Registration . About the topics www.fotoila.de Basic Information • Lecturer: Prof. Dr.-Ing. Georg Carle • Organization: [email protected] (only use this mail address!) Daniel Raumer tba. • Overview Main Language: German we will offer an English track (presuming a minimum of 4 participants) Extent: 2 SWS (4 ECTS) 4 ECTS * 30 hours = 120 working hours expected from you Course Type: For M.Sc. Students: Master’s Seminar (Master-Seminar) For B.Sc. Students: Advanced Seminar Course (Seminar) • Languages: German and English English only speakers are welcome to join (seminar will be split in two tracks if necessary) English Only Track • We offer an English only track if at least one non-German (native) speaker wants to attend the seminar • The English only track will have separate sessions Probably 1-2 sessions (depending on the number of students) • Attendance not mandatory for talks in the “standard” track Students in the “standard” track also don’t have to participate in the English track talks You are still welcome to join the other track’s talks • Usually the English track is quite small This means less attendance (if the
    [Show full text]
  • Running Xen-A Hands-On Guide to the Art of Virt
    Running Xen: A Hands-On Guide to the Art of Virtualization by Jeanna N. Matthews; Eli M. Dow; Todd Deshane; Wenjin Hu; Jeremy Bongio; Patrick F. Wilbur; Brendan Johnson Publisher: Prentice Hall Pub Date: April 10, 2008 Print ISBN-10: 0-13-234966-3 Print ISBN-13: 978-0-13-234966-6 eText ISBN-10: 0-13-207467-2 eText ISBN-13: 978-0-13-207467-4 Pages: 624 Table of Contents | Index Overview "This accessible and immediately useful book expertly provides the Xen community with everything it needs to know to download, build, deploy and manage Xen implementations." –Ian Pratt, Xen Project Leader VP Advanced Technology, Citrix Systems The Real—World, 100% Practical Guide to Xen Virtualization in Production Environments Using free, open source Xen virtualization software, you can save money, gain new flexibility, improve utilization, and simplify everything from disaster recovery to software testing. Running Xen brings together all the knowledge you need to create and manage high—performance Xen virtual machines in any environment. Drawing on the unparalleled experience of a world—class Xen team, it covers everything from installation to administration–sharing field-tested insights, best practices, and case studies you can find nowhere else. The authors begin with a primer on virtualization: its concepts, uses, and advantages. Next, they tour Xen's capabilities, explore the Xen LiveCD, introduce the Xen hypervisor, and walk you through configuring your own hard—disk—based Xen installation. After you're running, they guide you through each leading method for creating "guests" and migrating existing systems to run as Xen guests.
    [Show full text]
  • Linux Symposium 2007 As Shown by the Many Related Presentations at This Confer - Ence (E.G., KVM, Lguest, Vserver)
    Warren Henson of Google asked about other plans for For filesystems, Jon observed that disks are getting larger long-term storage and backup. Jepson said that that but not faster, so fsck time can be a problem. New filesys - sounds like a great thing for Google to do. Right now they tems such as chunkfs and tilefs are more scalable and sub - are stuck with the server (a low-power device, sealed with divide a disk so that only the active part needs to be fsck’d. a hard disk) and aren’t currently addressing that problem, The btrfs filesystem is extent-based, with subvolumes. It although Google has provided gmail accounts. In response supports snapshot checksumming, online fsck, and faster to Henson’s mention of alternative projects from other or - fsck. Jon talked about ext4 with its 48-bit block numbers ganizations, Jepson said that they would like to work with and use of extents. [Editor’s note: Also see the article in these groups and try every week. When these efforts fail, the June 2007 issue of ;login: about ext4.] the kids lose, in her opinion. Since Jepson spoke, Intel has Jon finds reiser4 interesting, but the project is unfortu - announced that they plan to work with OLPC, and Intel is nately stalled because Hans Reiser is no longer able to now listed as a supporter on the laptop.org site. work on it. It needs a new champion. Jon talked about virtualization. It is getting more attention, Linux Symposium 2007 as shown by the many related presentations at this confer - ence (e.g., KVM, lguest, Vserver).
    [Show full text]