DOCSLIB.ORG

Using Virtual Machines in Alien

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Using Virtual Machines in Alien Status for interfacing ARC and AliEn Using virtual machines in AliEn Using virtual machines in AliEn Bjarte Kileng Bergen University College June 15, 2012 Bjarte Kileng (Bergen University College) Using virtual machines in AliEn June 15, 2012 1 / 28 Status for interfacing ARC and AliEn Using virtual machines in AliEn Status for interfacing ARC and AliEn I First, a status report on the interfacing of ARC and AliEn. I Two students are following different approaches. Bjarte Kileng (Bergen University College) Using virtual machines in AliEn June 15, 2012 2 / 28 Status for interfacing ARC and AliEn Using virtual machines in AliEn AliEn-ARC interface I NDGF-T1 uses ARC middleware internally. I AliEn is run directly on some NDGF-T1 sites, but not all. I With an AliEn-ARC interface we might get access to all the NDGF-T1 sites. I With an AliEn-ARC interface, we can have a common VO-box for all the NDGF-T1 sites. I Challenge with a common VO-box for all the NDGF-T1 sites: Different architecture at different sites. I Solutions: Torrent installation: Martin Mjelde Tollefsen Virtual machines: Ann Kristin Skudal Bjarte Kileng (Bergen University College) Using virtual machines in AliEn June 15, 2012 3 / 28 Status for interfacing ARC and AliEn Using virtual machines in AliEn AliEn-ARC and torrent installation I The existing AliEn-ARC interface submit jobs into a local ARC environment. I A new solution: An ARC site is found using EGIIS (Enhanced Grid Information Indexing Service). The job is sent to the ARC site. AliEn and the necessary packages are retrieved using the torrent install method. I We need a new ARC.pm module in AliEn. Bjarte Kileng (Bergen University College) Using virtual machines in AliEn June 15, 2012 4 / 28 Status for interfacing ARC and AliEn Using virtual machines in AliEn AliEn-ARC and torrent installation What has been done? I A local AliEn setup is running: The central services. The VO-box services. Xrootd for storage. Torque as batch system. I The student has set up a computer with ARC including the indexing service and a computing element. I An ARC site for a job can be found using the indexing service. I Jobs can be submitted to ARC. Bjarte Kileng (Bergen University College) Using virtual machines in AliEn June 15, 2012 5 / 28 Status for interfacing ARC and AliEn Using virtual machines in AliEn AliEn-ARC and the use of CernVM I If all packages are preinstalled, there is no need for package installation. I CernVM uses the CernVM File System, a network file system where all the experiment software is pre-installed. I The alice.cern.ch software repository has been removed from CernVM. I The student working on this project will have a maternity leave soon. Bjarte Kileng (Bergen University College) Using virtual machines in AliEn June 15, 2012 6 / 28 Status for interfacing ARC and AliEn Using virtual machines in AliEn Why virtual machines? I Using CernVM in an AliEn-ARC interface. I Homogenizing the execution environment is interesting on its own: Can circumvent incompatibility issues between AliEn and worker nodes. Bjarte Kileng (Bergen University College) Using virtual machines in AliEn June 15, 2012 7 / 28 Status for interfacing ARC and AliEn Using virtual machines in AliEn Our setup I Torque is used as batch system. I Currently, only with XEN as hypervisor. I Some code has been taken from the ViBatch project: https://ekptrac.physik.uni-karlsruhe.de/trac/BatchVirt/wiki/ViBatch Bjarte Kileng (Bergen University College) Using virtual machines in AliEn June 15, 2012 8 / 28 Status for interfacing ARC and AliEn Using virtual machines in AliEn The setup is running I Uses libvirt: Libvirt can boot both XEN and KVM. Newer versions of libvirt can also boot VirtualBox. I Each job on a worker node will run inside its own virtual machine: A worker node can have several simultaneous running virtual machines. I The virtual machine must be set up for use by the system, but: There are no site specific configurations inside the virtual machine. I Only tested on a worker node running SLC5.8 and with a virtual machine image for SLC5.8. Bjarte Kileng (Bergen University College) Using virtual machines in AliEn June 15, 2012 9 / 28 Status for interfacing ARC and AliEn Using virtual machines in AliEn How it works I Torque can run scripts when a job is started (prologue) and when a job finishes (epilogue): Starts and stops the virtual machine. I Using the option «-S» to qsub, we can specify a shell for the job: The setup uses a script remoteshell which returns a SSH connection to the virtual machine. I The home directory of the submitter on the worker node must be accessible as a NFS share in the virtual machine. Bjarte Kileng (Bergen University College) Using virtual machines in AliEn June 15, 2012 10 / 28 Status for interfacing ARC and AliEn Using virtual machines in AliEn Prologue, epilogue and the remote shell Prologue The prologue script is run as root on the worker node before the job is executed. Epilogue The epilogue script is run as root on the worker node after the job is finished. The remote shell script The script remoteshell is run as the submitting user on the worker node. Bjarte Kileng (Bergen University College) Using virtual machines in AliEn June 15, 2012 11 / 28 Status for interfacing ARC and AliEn Using virtual machines in AliEn The prologue script I New disk images are created based on template images: Unless configured read only, a disk image can not be shared between simultaneously running virtual machines. I Disk images are mounted on the worker node and prepared for the virtual machine: The job is copied to to the image. The home directory of the submitting user is created. Creates (if necessary) and prepares a SSH key for the submitting user on the disk image. Creates an option file which is read by customized init.d scripts on the virtual machine. I The disks are unmounted end removed from the worker node. I The virtual machine is started. I The script exits when the virtual machine is up and running Bjarte Kileng (Bergen University College) Using virtual machines in AliEn June 15, 2012 12 / 28 Status for interfacing ARC and AliEn Using virtual machines in AliEn The prologue script Options stored in the option file on the disk image I Username, group, uid and gid of submitting user I NFS host, share and mount point. I Path where to put the alive directory: The alive directory is created by the virtual machine on the NFS share when the machine is up and running. The prologue script will exit when the alive directory exists. Bjarte Kileng (Bergen University College) Using virtual machines in AliEn June 15, 2012 13 / 28 Status for interfacing ARC and AliEn Using virtual machines in AliEn The remoteshell script I A SSH connection to the virtual machine is opened as the submitting user. I The environment of the job on the worker node is piped into the SSH-connection. I The virtual machine is now ready for use by Torque as the execution environment. Bjarte Kileng (Bergen University College) Using virtual machines in AliEn June 15, 2012 14 / 28 Status for interfacing ARC and AliEn Using virtual machines in AliEn The epilogue script I The virtual machine is stopped. I The disk images are deleted. I The alive directory is removed. I Lock files are removed. Bjarte Kileng (Bergen University College) Using virtual machines in AliEn June 15, 2012 15 / 28 Status for interfacing ARC and AliEn Using virtual machines in AliEn The customized init.d scripts I The submitting user is created on the virtual machine. I The NFS share is mounted. I The alive directory on the NFS share is created. Bjarte Kileng (Bergen University College) Using virtual machines in AliEn June 15, 2012 16 / 28 Status for interfacing ARC and AliEn Using virtual machines in AliEn Some network comments I MAC addresses for the virtual machines can either be configured as a list of MAC addresses or as an initial address: If an initial address is given, successive values are used for the virtual machines. A worker node will always use the same subset of MAC addresses for its virtual machines. I The network setup on the virtual machines must use DHCP: It is easy to add functionality for using static network setup. IP numbers could be given as a list or as an initial value as is done with MAC addresses. I The alive directory created by the virtual machine is identified by the MAC address of the virtual machine. I The IP number assigned to the virtual machine is stored by the virtual machine in the alive directory. Bjarte Kileng (Bergen University College) Using virtual machines in AliEn June 15, 2012 17 / 28 Status for interfacing ARC and AliEn Using virtual machines in AliEn Creating disk images from templates I Unless read only, a disk image can not be shared between simultaneous running virtual machines. I Jobs run in identical virtual machines. Bjarte Kileng (Bergen University College) Using virtual machines in AliEn June 15, 2012 18 / 28 Status for interfacing ARC and AliEn Using virtual machines in AliEn Creating disk images from templates QCOW files I Copying of complete disk images is both time- and disk consuming: Can use qcow files (qemu copy on write). I A qcow file can be based on a master file, or backing file. I The new file (or «copy») only contains the diffs from the master.
Load more

Recommended publications
  • Virtual Routing in the Cloud
    I I I I I I I I I CISCO ~ ptg17123584 Virtual Routing in the Cloud Arvind Durai Stephen Lynn ciscopress.com Amit Srivastava Exclusive Offer - 40% OFF Cisco Press Video Training live lessons·® ciscopress.com/video Use coupon code CPVIDE040 during checkout. Video Instruction from Technology Experts livelessons·® ptg17123584 Advance Your Skills Train Anywhere Learn Get started with fundamentals, Train anywhere, at your Learn from trusted author become an expert, or get certified. own pace, on any device. trainers published by Cisco Press. Try Our Popular Video Training for FREE! ciscopress.com/video Explore hundreds of FREE video lessons from our growing library of Complete Video Courses, Livelessons, networking talks, and workshops. 9781587144943_Durai_Virtual_Routing_Cloud_CVR.indd 2 4/8/16 1:25 PM Virtual Routing in the Cloud Arvind Durai, CCIE No. 7016 Stephen Lynn, CCIE No. 5507 & CCDE No. 20130056 Amit Srivastava ptg17123584 Cisco Press 800 East 96th Street Indianapolis, IN 46240 USA ii Virtual Routing in the Cloud Virtual Routing in the Cloud Arvind Durai, CCIE No. 7016 Stephen Lynn, CCIE No. 5507 & CCDE No. 20130056 Amit Srivastava Copyright© 2016 Cisco Systems, Inc. Published by: Cisco Press 800 East 96th Street Indianapolis, IN 46240 USA All rights reserved. No part of this book may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopying, recording, or by any information storage and retrieval system, without written permission from the publisher, except for the inclusion of brief quotations in a review. Printed in the United States of America First Printing April 2016 Library of Congress Control Number: 2016934921 ISBN-13: 978-1-58714-494-3 ISBN-10: 1-58714-494-8 Warning and Disclaimer ptg17123584 This book is designed to provide information about CSR 1000V router and adoption of NFV technology in the cloud environment.
    [Show full text]
  • Oracle VM Virtualbox User Manual
    Oracle VM VirtualBox R User Manual Version 5.1.0 c 2004-2016 Oracle Corporation http://www.virtualbox.org Contents 1 First steps 11 1.1 Why is virtualization useful?............................. 12 1.2 Some terminology................................... 12 1.3 Features overview................................... 13 1.4 Supported host operating systems.......................... 15 1.5 Installing VirtualBox and extension packs...................... 16 1.6 Starting VirtualBox.................................. 17 1.7 Creating your first virtual machine......................... 18 1.8 Running your virtual machine............................ 21 1.8.1 Starting a new VM for the first time.................... 21 1.8.2 Capturing and releasing keyboard and mouse.............. 22 1.8.3 Typing special characters.......................... 23 1.8.4 Changing removable media......................... 24 1.8.5 Resizing the machine’s window...................... 24 1.8.6 Saving the state of the machine...................... 25 1.9 Using VM groups................................... 26 1.10 Snapshots....................................... 26 1.10.1 Taking, restoring and deleting snapshots................. 27 1.10.2 Snapshot contents.............................. 28 1.11 Virtual machine configuration............................ 29 1.12 Removing virtual machines.............................. 30 1.13 Cloning virtual machines............................... 30 1.14 Importing and exporting virtual machines..................... 31 1.15 Global Settings...................................
    [Show full text]
  • Virtualization Best Practices
    SUSE Linux Enterprise Server 15 SP1 Virtualization Best Practices SUSE Linux Enterprise Server 15 SP1 Publication Date: September 24, 2021 Contents 1 Virtualization Scenarios 2 2 Before You Apply Modifications 2 3 Recommendations 3 4 VM Host Server Configuration and Resource Allocation 3 5 VM Guest Images 25 6 VM Guest Configuration 36 7 VM Guest-Specific Configurations and Settings 42 8 Xen: Converting a Paravirtual (PV) Guest to a Fully Virtual (FV/HVM) Guest 45 9 External References 49 1 SLES 15 SP1 1 Virtualization Scenarios Virtualization oers a lot of capabilities to your environment. It can be used in multiple scenarios. To get more details about it, refer to the Book “Virtualization Guide” and in particular, to the following sections: Book “Virtualization Guide”, Chapter 1 “Virtualization Technology”, Section 1.2 “Virtualization Capabilities” Book “Virtualization Guide”, Chapter 1 “Virtualization Technology”, Section 1.3 “Virtualization Benefits” This best practice guide will provide advice for making the right choice in your environment. It will recommend or discourage the usage of options depending on your workload. Fixing conguration issues and performing tuning tasks will increase the performance of VM Guest's near to bare metal. 2 Before You Apply Modifications 2.1 Back Up First Changing the conguration of the VM Guest or the VM Host Server can lead to data loss or an unstable state. It is really important that you do backups of les, data, images, etc. before making any changes. Without backups you cannot restore the original state after a data loss or a misconguration. Do not perform tests or experiments on production systems.
    [Show full text]
  • Ubuntu Server Guide Basic Installation Preparing to Install
    Ubuntu Server Guide Welcome to the Ubuntu Server Guide! This site includes information on using Ubuntu Server for the latest LTS release, Ubuntu 20.04 LTS (Focal Fossa). For an offline version as well as versions for previous releases see below. Improving the Documentation If you find any errors or have suggestions for improvements to pages, please use the link at thebottomof each topic titled: “Help improve this document in the forum.” This link will take you to the Server Discourse forum for the specific page you are viewing. There you can share your comments or let us know aboutbugs with any page. PDFs and Previous Releases Below are links to the previous Ubuntu Server release server guides as well as an offline copy of the current version of this site: Ubuntu 20.04 LTS (Focal Fossa): PDF Ubuntu 18.04 LTS (Bionic Beaver): Web and PDF Ubuntu 16.04 LTS (Xenial Xerus): Web and PDF Support There are a couple of different ways that the Ubuntu Server edition is supported: commercial support and community support. The main commercial support (and development funding) is available from Canonical, Ltd. They supply reasonably- priced support contracts on a per desktop or per-server basis. For more information see the Ubuntu Advantage page. Community support is also provided by dedicated individuals and companies that wish to make Ubuntu the best distribution possible. Support is provided through multiple mailing lists, IRC channels, forums, blogs, wikis, etc. The large amount of information available can be overwhelming, but a good search engine query can usually provide an answer to your questions.
    [Show full text]
  • The Open Virtual Machine Format Whitepaper for OVF Specification
    The Open Virtual Machine Format Whitepaper for OVF Specification version 0.9 OVF Whitepaper 0.9 VMware, Inc. 3401 Hillview Ave., Palo Alto CA, 94304 USA Tel 877-486-9273 Fax 650-427-5001 www.vmware.com XenSource, Inc. 2300 Geng Road, Suite 2500, Palo Alto, CA 94303 www.xensource.com Copyright © VMware, Inc. and XenSource, Inc. All rights reserved. VMware, the VMware “boxes” logo and design, Virtual SMP and VMotion are registered trademarks or trademarks of VMware, Inc. in the United States and/or other jurisdictions. Xen, XenSource, the “Circle Xen” logo and derivatives thereof are registered trademarks or trademarks of XenSource, Inc. in the United States and other countries. Microsoft, Windows and Windows NT are registered trademarks of Microsoft Corporation. Linux is a registered trademark of Linus Torvalds. All other marks and names mentioned herein may be trademarks of their respective companies. No part of this specification (whether in hardcopy or electronic form) may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means, electronic, mechanical, photocopying, recording, or otherwise, without the prior written permission of VMware, Inc. (VMware), and XenSource, Inc. (XenSource) except as otherwise permitted under copyright law or the terms of that certain Teaming and Non-Disclosure Agreement between VMware and XenSource dated March 23, 2007, as amended from time to time. Please note that the content in this specification is protected under copyright law even if it is not distributed with software
    [Show full text]
  • FVD: a High-Performance Virtual Machine Image Format for Cloud
    FVD: a High-Performance Virtual Machine Image Format for Cloud Chunqiang Tang IBM T.J. Watson Research Center [email protected] http://www.research.ibm.com/people/c/ctang/ Note: This paper describes the copy-on-write, copy- 1 Introduction on-read, and adaptive prefetching capabilities of FVD. The compact image capability of FVD is described Cloud Computing is widely considered as the next big separately in a companion paper entitled “Compact thing in IT evolution. In a Cloud like Amazon EC2 [1], Image Support in Fast Virtual Disk (FVD)”, which the storage space for virtual machines’ virtual disks can is available at https://researcher.ibm.com/ be allocated from multiple sources: the host’s direct- researcher/view project.php?id=1852 attached storage (DAS, i.e., local disk), network-attached storage (NAS), or storage area network (SAN). These op- Abstract tions offer different performance, reliability, and avail- ability at different prices. DAS is at least several times This paper analyzes the gap between existing hyper- cheaper than NAS and SAN, but DAS limits the avail- visors’ virtual disk capabilities and the requirements in ability and mobility of VMs. a Cloud, and proposes a solution called FVD (Fast Vir- tual Disk). FVD consists of an image format and a To get the best out of the different technologies, a block device driver designed for QEMU. Despite the ex- Cloud usually offers a combination of block-device stor- istence of many popular virtual machine (VM) image age services to VMs. For instance, Amazon Web Ser- formats, FVD came out of our unsatisfied needs in the vices (AWS) [2] offers to a VM both ephemeral storage IBM Cloud.
    [Show full text]
  • Gadgetpc Debian Installation Guide
    GadgetPC Single Board Computer Debian Installation Guide Document Revision: 1.03 Date: 20 October, 2009 BiPOM Electronics, Inc. 16301 Blue Ridge Road, Missouri City, Texas 77489 Telephone: 1-713-283-9970 Fax: 1-281-416-2806 E-mail: [email protected] Web: www.bipom.com All trademarked names in this manual are the property of respective owners. © 2009 BiPOM Electronics, Inc. Page 1 1. Overview Thank you for your purchase of the GadgetPC Single Board Computer. GadgetPC is a powerful computer board that is capable of running high-level operating systems such as Linux. This document is for advanced users who want to install Debian Linux system from scratch. When GadgetPC is first powered, it goes through a boot sequence and executes various components in the following order: ROM boot loader ( built-in ROM) AT91BootStrap ( DataFlash ) U-boot (DataFlash ) Linux kernel ( uImage file under USB FAT root ) Root FS ( Debian on USB Flash drive ) ROM boot loader is built into the AT91SAM9260 microcontroller and cannot be changed. As soon as the board is powered the ROM boot loader starts. It downloads and runs an application (AT91BootStrap) from external storage media (DataFlash) into internal SRAM. AT91BootStrap has been developed by BiPOM Electronics specifically for GadgetPC. AT91BootStrap is responsible for initializing hardware such as DataFlash, SDRAM, digital outputs, and USART0 serial port. AT91BootStrap downloads to SDRAM and passes control to U-Boot which is a powerful boot loader that resides also in DataFlash. U-Boot performs many low-level tasks such as detecting USB hardware, reading Linux image from external USB flash drive, uncompressing Linux image to SDRAM, and passing control to Linux image in SDRAM.
    [Show full text]
  • Oracle® Linux KVM User's Guide
    Oracle® Linux KVM User's Guide F29966-16 August 2021 Oracle Legal Notices Copyright © 2020, 2021, Oracle and/or its affiliates. This software and related documentation are provided under a license agreement containing restrictions on use and disclosure and are protected by intellectual property laws. Except as expressly permitted in your license agreement or allowed by law, you may not use, copy, reproduce, translate, broadcast, modify, license, transmit, distribute, exhibit, perform, publish, or display any part, in any form, or by any means. Reverse engineering, disassembly, or decompilation of this software, unless required by law for interoperability, is prohibited. The information contained herein is subject to change without notice and is not warranted to be error-free. If you find any errors, please report them to us in writing. If this is software or related documentation that is delivered to the U.S. Government or anyone licensing it on behalf of the U.S. Government, then the following notice is applicable: U.S. GOVERNMENT END USERS: Oracle programs (including any operating system, integrated software, any programs embedded, installed or activated on delivered hardware, and modifications of such programs) and Oracle computer documentation or other Oracle data delivered to or accessed by U.S. Government end users are "commercial computer software" or "commercial computer software documentation" pursuant to the applicable Federal Acquisition Regulation and agency-specific supplemental regulations. As such, the use, reproduction, duplication, release, display, disclosure, modification, preparation of derivative works, and/or adaptation of i) Oracle programs (including any operating system, integrated software, any programs embedded, installed or activated on delivered hardware, and modifications of such programs), ii) Oracle computer documentation and/or iii) other Oracle data, is subject to the rights and limitations specified in the license contained in the applicable contract.
    [Show full text]
  • Thin-Provisioned Disks with QEMU and KVM
    Thin-provisioned disks with QEMU and KVM Paolo Bonzini Red Hat, Inc. devconf.cz, February 2014 devconf.cz 2014 QEMU vs. KVM ● QEMU is where the cool stuff happens ● Fast paravirtualized hardware (virtio) ● Virtual machine lifecycle (including migration) ● Storage stack ● kvm.ko lets you use QEMU for virtualization ● Narrow-scoped, mature code ● Also cool :) ● This talk will be about QEMU devconf.cz 2014 Outline ● Thin provisioning concepts ● What was there ● Requirements ● Designing a thin-provisioning API ● Future work devconf.cz 2014 Thin provisioning concepts ● A disk is made of many blocks ● The user tells the disks how it's using them ● The disk can be used more efficiently ● Speed and durability gains for SSDs ● Oversubscription of networked storage ● Efficient maintenance operations ● Better name (from SCSI standard): “Logical block provisioning” devconf.cz 2014 Thin provisioning and virtualization ● The advantages extend to virtualization ● Can be applied to any storage backend ● “Software-defined storage” before it became cool ● The user is the guest administrator ● Only pay for actually used space ● Guest disks can be overprovisioned ● Host admin saves disk space devconf.cz 2014 Multiple storage layers qcow2, raw, ... file, blockqcow2, device, raw, gluster, ... iSCSI gluster, iSCSI gluster, NFS ext4/XFS Ext4, XFS SSD, NAS, dm-thinp devconf.cz 2014 What was there ● Lazy allocation of blocks ● Differential disks (copy-on-write) ● High-level watermark: management can query the highest sector in use ● QEMU-specific formats: qcow,
    [Show full text]
  • Optimizing VM Images for Openstack with KVM/QEMU
    27th Large Installation System Administration Conference November 3–8, 2013 • Washington, D.C. Optimizing VM images for OpenStack with KVM/QEMU Brian Wellman Director, Systems Operations Metacloud 27th Large Installation System Administration Conference November 3–8, 2013 • Washington, D.C. A Few Notes • All topics covered in this presentation assume the following software versions: • QEMU 1.0.0+ • OpenStack Grizzly (2013.1) • libvirt 0.9.8+ • Ubuntu 12.04 LTS • RHEL/CentOS 6.3 • There are a number of different ways of doing what is described in this presentation. This is simply one way of doing things based upon our experience running production clouds for our clients. 27th Large Installation System Administration Conference November 3–8, 2013 • Washington, D.C. Disk vs. Container Formats • Disk formats store partition and block data: • QEMU+KVM supports a cornucopia of different disk formats, too many to cover in this presentation • We will be covering RAW and QCOW2 • Container formats express metadata about a VM as well as its underlying block devices: • Typically contains files in a disk format • We will be covering AMI 27th Large Installation System Administration Conference November 3–8, 2013 • Washington, D.C. RAW Disk Format • Direct representation of a disk structure • Can be sparse • Widely supported by hypervisors • Can be treated as a block device 27th Large Installation System Administration Conference November 3–8, 2013 • Washington, D.C. QCOW2 Disk Format • QEMU Copy on Write Version 2 • Supports pre-allocation as well as on-demand allocation of blocks • Wide range of features: • Read-only backing files • Snapshots (internal and external) • Compression • Encryption 27th Large Installation System Administration Conference November 3–8, 2013 • Washington, D.C.
    [Show full text]
  • Improving Digital Forensics and Incident Analysis in Production Environments by Using Virtual Machine Introspection
    Improving Digital Forensics and Incident Analysis in Production Environments by Using Virtual Machine Introspection Benjamin Taubmann Dissertation zur Erlangung des akademischen Grades eines Doktors der Naturwissenschaften Eingereicht an der Fakultät für Informatik und Mathematik der Universität Passau Juli 2019 Gutachter: Prof. Dr. Hans P. Reiser Prof. Dr. Nuno Miguel Carvalho dos Santos Abstract Main memory forensics and its special form, virtual machine introspection (VMI), are powerful tools for digital forensics and can be used to improve the security of computer-based systems. However, their use in production systems is often not pos- sible. This work identifies the causes and offers practical solutions to apply these techniques in cloud computing and on mobile devices to improve digital forensics and incident analysis. Four key challenges must be tackled. The first challenge is that many existing solu- tions are not reproducible, for example, because the corresponding software compo- nents are not available, obsolete or incompatible. The use of these tools is also often complex and can lead to a crash of the system to be monitored in case of incorrect use. To solve this problem, this thesis describes the design and implementation of Libvmtrace, which is a framework for the introspection of Linux-based virtual ma- chines. The focus of the developed design is to implement frequently used methods in encapsulated modules so that they are easy for developers to use, optimize and test. The second challenge is that many production systems do not provide an interface for main memory forensics and virtual machine introspection. To address this problem, this thesis describes possible solutions for how such an interface can be implemented on mobile devices and in cloud environments designed to protect main memory from unprivileged access.
    [Show full text]
  • Hypervisors, Virtualization API, Cloud Computing Platforms
    Hypervisors, Virtualization API, Cloud Computing Platforms Alexander Betz 7.5.2013 What is virtualization? Separation of the OS from the hardware. OS OS Layer Hardware Hardware What is virtualization? Separation of the OS from the hardware. OS Guest Hypervisor Hardware Host What is virtualization? Separation of the OS from the hardware. OS Guest1 Guest2 Hypervisor Hypervisor Hardware Host Host Multiple different OS on the same host Transfer guest between hosts What is emulation ? • Provide an instruction set specific to a hardware type – The indirection layer converts commands from the guest into commands the host can understand – Non-host hardware can be simulated for the VM Guest Layer Host Why use it ? • Better resource usage – Average server load without virtualisation: 5% – One big instead of several small physical machines (less space, electricity, cooling required) • More flexibility for server and desktop hosting – Different OS on the same host • Only 5-10% slower Hypervisors • Also called Virtual Machine Monitor Guest Guest Guest Guest Guest Management 1 2 1 2 3 Tools Hypervisor Hypervisor Host OS Hardware Hardware Type 1 Hypervisor Type 2 Hypervisor (bare metal) (hosted) QEMU and KVM • Kernel based Virtual Machine(KVM) – Linux kernel module Guest Guest 1 2 – Acts like type 1 hypervisor – Uses Linux functions QEMU QEMU Linux Kernel KVM Hardware • Quick EMUlator(QEMU) – Provides an environment for the guest OS and passes the instructions from the guest OS to KVM Terms • Virtualization – Simulate enough hardware so a guest can run
    [Show full text]