DOCSLIB.ORG

Hands on Virtualization with Ganeti

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Hands on Virtualization with Ganeti Hands on Virtualization with Ganeti (part 1) Lance Albertson @ramereth Associate Director OSU Open Source Lab About us ● OSU Open Source Lab ● Server hosting for Open Source Projects ○ Linux Foundation, Apache Software Foundation, Drupal, Python Software Foundation, Freenode, Gentoo, Debian, CentOS, Fedora, etc etc ... ● Open Source development projects ○ Ganeti Web Manager Session Overview (part 1) ● Ganeti Introduction ● Terminology ● Major Components ● Latest Features ● Using Ganeti in Practice ● How Ganeti is deployed at OSUOSL Session Overview (part 2) ● Hands on Demo ● Installation and Initialization ● Cluster Management ● Adding instances (VMs) ● Controlling instances ● Auto Allocation ● Dealing with node failures What can Ganeti do? ● Virtual machine management software tool ● Manages clusters of physical machines ● Xen/KVM/LXC VM deployment ● Live Migration ● Resiliency to failure ● data redundancy via DRBD ● Cluster Balancing ● Ease of repairs and hardware swaps Ganeti Cluster Comparing Ganeti ● Private IaaS ● Primarily utilizes local storage ● Designed for hardware failures ● Mature project ● Low package requirements ● Simple administration ● Easily pluggable via hooks & RAPI Project Background ● Google funded project ● Used in internal corporate env ● Open Sourced in 2007 GPLv2 ● Team based in Google Switzerland ● Active mailing list & IRC channel ● Started internally before libvirt, openstack, etc Goals of Ganeti Goals: Low Entry Level ● Keeping the entry level as low as possible ● Easy to install, manage and upgrade ● No specialized hardware needed ● i.e. SANs ● Lightweight ● no "expensive" package dependencies Goals: Enterprise Scale ● Manage simultaneously from 1 to ~200 host machines ● Access to advanced features ● drbd, live migration, API, OOB control ● Batch VM deployments ● Ease of lateral expansion and rebalancing Goals: Open Source Citizen ● Design and code discussions are open ● External contributions are welcome ● Cooperate with other "big scale" Ganeti users ● Welcome third-party projects ● Ganeti Web Manager (OSL), Synnefo (GRNET) Terminology Terminology Node virtualization host Node Group homogeneous set of nodes (i.e. rack of nodes) Instance virtualization guest Cluster set of nodes, managed as a collective Job ganeti operation Architecture Components ● Linux & standard utils ○ (iproute2, bridge-utils, ssh) ● KVM, Xen or LXC ● DRBD, LVM, RDB, or SAN ● Python ○ (plus a few modules) ● socat ● Haskell (optional, for auto-allocation) Nodes Roles (management level) Runs ganeti-masterd, rapi, noded Master Node and confd Have a full copy of the config, can become master Master Candidates Run ganeti-confd and noded Cannot become master Regular Nodes Get only part of the config Offline nodes In repair or decommissioned Nodes Roles (instance hosting level) VM Capable Node Can run virtual machines Drained Nodes Are being evacuated Offline Nodes Are in repair Instances ● Virtual machine that runs on the cluster ● fault tolerant/HA entity within cluster Instance Parameters ● Hypervisor: hvparams ● General: beparams ● Networking: nicparams ● Modifiable at the instance or cluster level hvparams ● Boot order, CDROM Image ● NIC Type, Disk Type ● VNC Parameters, Serial console ● Kernel Path, initrd, args ● Other Hypervisor specific parameters beparams / nicparams ● Memory / Virtual CPUs ● Adding or removing disks ● MAC ● NIC mode (routed or bridged) ● Link Disk Template drbd LVM + DRBD between 2 nodes rbd RBD volumes residing inside a RADOS cluster * plain LVM with no redundancy diskless No disks. Useful for testing only * experimental support added in 2.6 Primary & Secondary Concepts ● Instances always runs on primary ● Uses secondary node for disk replication ● Depends on disk template (i.e. drbd, plain) Instance creation scripts also known as OS Definitions ● Requires Operating System installation script ● Provide scripts to deploy various operating systems ● Ganeti Instance Debootstrap ● upstream supported ● Ganeti Instance Image ● written by me OS Variants ● Variants of the OS Definition ● Used for defining guest operating system ● Types of deployment settings: ● Extra packages ● Filesystem ● Image directory ● Image Name Latest Features 2.4 2.5 March 2011 April 2012 ● Out of Band management ● shared storage (SAN) ● vhost net support (KVM) support ● hugepages support (KVM) ● improved node groups ● initial node groups (scalability, evacuate, commands) ● master IP turnup customization ● full SPICE support (KVM) Latest Features 2.6 Upcoming July 2012 Just ideas, not promises ● RBD support (ceph) ● Full dynamic memory support ● initial memory balloning ● Better instance networking (KVM, Xen) customization ● cpu pinning ● Rolling Reboot ● OVF export/import support ● Better automation, self- ● customized drbd parameters healing, availability ● policies for better resource ● Higher Scalability modeling ● KVM block device migration ● Optional haskell ganeti-confd ● Better OS Installation Initializing your cluster The node needs to be set up following the ganeti installation guide. gnt-cluster init [-s ip] ... \ --enabled-hypervisors=kvm cluster gnt-cluster Cluster wide operations: gnt-cluster info gnt-cluster modify [-B/H/N ...] gnt-cluster verify gnt-cluster master-failover gnt-cluster command/copyfile ... Adding nodes gnt-node add [-s ip] node2 gnt-node add [-s ip] node3 gnt-node add [-s ip] node4 Adding instances # install instance-{debootstrap, image} gnt-os list gnt-instance add -t drbd \ {-n node3:node2 | -I hail } \ -o debootstrap+default web ping web ssh web # easy with OS hooks gnt-node Per node operations: gnt-node remove node4 gnt-node modify \ [ --master-candidate yes|no ] \ [ --drained yes|no ] \ [ --offline yes|no ] node2 gnt-node evacuate/failover/migrate gnt-node powercycle -t drbd DRBD provides redundancy to instance data, and makes it possible to perform live migration without having shared storage between the nodes. "RAID1" over the network Recovering from failure # set the node offline gnt-node modify -O yes node3 Recovering from failure # failover instances to their secondaries gnt-node failover --ignore-consistency node3 # or, for each instance: gnt-instance failover \ --ignore-consistency web Recovering from failure # restore redundancy gnt-node evacuate -I hail node3 # or, for each instance: gnt-instance replace-disks \ {-n node1 | -I hail } web gnt-backup Manage instance exports/backups: gnt-backup export -n node1 web gnt-backup import -t plain \ {-n node3 | -I hail } \ --src-node node1 \ --src-dir /tmp/myexport web gnt-backup list gnt-backup remove htools: cluster resource management ● Written in Haskell ● Where do I put a new instance? ● Where do I move an existing one? ● hail: the H iallocator ● How much space do I have? ● hspace: the H space calculator ● How do I fix an N+1 error? ● hbal: the cluster balancer Controlling Ganeti ● Command line * ● Ganeti Web Manager ● Developed by OSUOSL ● RAPI (Rest-full HTTP Interface) * ● On-cluster "luxi" interface * ● luxi is currently json over unix socket ● there is code for python and haskell * programmable interfaces Job Queue ● Ganeti operations generate jobs in the master ○ with the exception of queries ● Jobs execute concurrently ● You can cancel non-started jobs, inspect the queue status, and inspect jobs gnt-job list gnt-job info gnt-job watch gnt-job cancel gnt-group Managing node groups: gnt-group add gnt-group assign-nodes gnt-group evacuate gnt-group list gnt-group modify gnt-group remove gnt-group rename gnt-instance change-group Running Ganeti in Production What should you add? ● Monitoring/Automation ● Check host disks, memory, load ● Trigger events (evacuate, send to repairs, readd node, rebalance) ● Automated host installation/setup (config management) ● Self service use ● Instance creation and resize ● Instance console access Ganeti in practice ● Medium to small virtualization environments ● High performance ● Dedicated hardware, faster disks, more spindles on local storage ● Cheap hardware to high-end hardware ● Higher reliability Ganeti as a "cloud" ● Not a traditional cloud environment ● No AWS APIs (yet at least), no object store ● Ganeti specific API ● Tools to extend it ● Ganeti Web Manager, Syssnefo, GlusterFS, Ceph ● Storage layer differences ● block devices instead of disk images (typically) How the OSL uses Ganeti ● Powers all managed virtualization ● Project hosting ● KVM based ● Hundreds of VMs ● Web hosts, code hosting, etc ● Per-project clusters: PSF, OSGeo, phpBB, Gentoo ● Powers Supercell Ganeti at OSL ● Node OS: Gentoo ● Migrating towards CentOS ● CFEngine for node configuration setup ● Utilize instance-image for guest installs ● Flexibility on guest operating systems we can deploy ● 10 clusters, 27 nodes, 230 instances ● Ganeti Web Manager Ganeti at OSL ● Production cluster ● busybox, darcs, inkscape, musicbrainz, openmrs, php.net, qemu, freenode, yum ● 5 nodes, 20 instances per machine ● 64G Ram / 3-7TB / 24 cores (2) ● 24G Ram / 670G / 4 cores (3) ● Reduced cooling footprint ● Per-project clusters enabled flexibility People running Ganeti ● Google ● Corporate Computing Infra ● osuosl.org ● Oregon State University Open Source Lab ● grnet.gr ● Greek Research & Technology Network ● nero.net ● Network for Education & Research in Oregon Questions? (Part 1 Conclusion) Lance Albertson [email protected] @ramereth http://lancealbertson.com Check it out at: http://code.google.com/p/ganeti/ Or just search for "Ganeti" Try it. Love it. Improve it. Contribute back (CLA required). © 2009-2012 Oregon State University Use under CC-by-SA / Some content borrowed/modified from Iustin Pop (with permission).
Load more

Recommended publications
  • Homework #5 Answer Key
    CS312 Homework #5 Answer Key March 9, 2016 Questions 1. Load balancers are designed to help with vertical scaling. (a) True (b) False 2. Layer 7 load balancers allow you to use both connection tracking and inspection of packets including modification of packets. (a) True (b) False 3. Linux containers do not require hypervisors (a) True (b) False 1 4. The RUN command can be used more than once in a Dockerfile. (a) True (b) False 5. The PID Namespace allows users to see processes from other containers (a) True (b) False 6. Paravirtualization uses cpu supported virtualization. (a) True (b) False 7. Which software load balancer primarily acts as an HTTP accelerator and a static cache server? (a) HAProxy (b) Apache (c) nginx (d) Varnish 8. Which of the following is not true about Linux Containers? (a) Operating system level virtualization (b) Provides little overhead (c) Fully emulates an operating system (d) Allows limits to resources with cgroups 9. Which service describes a virtual computing platform? (a) IaaS (b) PaaS (c) SaaS (d) VaaS 2 10. Describe the potential problems of using Round Robin DNS. • Depending on the implementation of the client resolver, this can be somewhat random or not. • Some resolvers always use alphabetical ordering. • DNS Caching and Time-To-Live (TTL) issues. • If the site goes down, you have to update DNS to reroute traffic which can cause problems. 11. Which scheduling algorithm is generally preferred if the web application uses sessions? Source Connection 12. Which HAProxy configuration section allows you to define a complete proxy? listen 13.
    [Show full text]
  • Using Ganeti for Running Highly Available Virtual Services
    Using Ganeti for running highly available virtual services 2016-04-21, HEPiX Spring 2016, Zeuthen Overview ● What is Ganeti ● What is it good for ● How does it work ● NDGF usage 2 What is Ganeti ● A software stack for managing virtual machines – Like VMware or OpenStack or libvirt or ... – Supporting Xen or KVM hypervisors – Handles ● Storage: volume creation and assignment ● OS installation and customization ● Networking ● Startup, shutdown, live migration, failover of instances – Written in Python and Haskell – Aimed for ease of use and fast and simple error recovery after physical failures on commodity hardware 3 What is Ganeti ● Mainly developed by Google for their own use – Handles VMs for corporate network (office servers, remote desktops etc), not production services (what non-employees see) ● Outside Google – Debian – NDGF-T1 – Lufthansa – Etc ● Maintained by Google with significant external contributions 4 What is Ganeti good at ● Running highly available services on a small set of hardware – DRBD or external reliable block devices (CEPH, Enterprise storage) – Live migrations in case of impending hardware failure ● Or reboot into new kernel security upgrade on the hardnode – Failover handled automatically in case of sudden hardware failure – No external dependencies beyond networking ● Well, if you use external storage... ● But no extra servers or services needed – Typical reasonable cluster size, 3 – 50 hardnodes ● Multiple clusters integrate well though in admin tools 5 How does Ganeti work ● gnt-cluster init ... – Creates a cluster of ganeti nodes – We'll assume DRBD for storage, as at NDGF – One member is a master node ● Others can take over with master-failover if they can get quorum A B C D E 6 How does Ganeti work ● gnt-instance add – Creates VMs, with OS install scripts (image, debootstrap, pxe) – Each VM has a secondary location (DRBD mirror, sync) A B C D E 7 How does Ganeti work ● gnt-instance migrate – No noticable service impact from live migration, <1s network pause ● Unless something is broken..
    [Show full text]
  • Managed Conversion of Guests to Ovirt
    Managed Conversion of Guests to oVirt Arik Hadas Senior Software Engineer Red Hat 21/8/15 KVM Forum, August 2015 Agenda ● Motivation ● Architecture ● Demonstration ● Implementation ● Future work KVM Forum, August 2015 Many ways to run Virtual Machines ● There are many virtualization tools ● Different hypervisors – KVM, E !"E Xi, Xen, VirtualBo$, .%% ● Different management systems – oVirt, virt'manager, v phere, Ganeti, .%% KVM Forum, August 2015 “I don't want to lose my VMs” ● Virtualization technologies are used for a long time ● +o standardization ● ,eople are tied up to the technologies they currently use ● Conversion tools are neede). KVM Forum, August 2015 virt-v2v ● ,art of virt tools – /pen source virtualization management tools ● Foreign hypervisor -0 KVM ● Standalone conversion tool KVM Forum, August 2015 Conversion to oVirt using virt-v2v ● Converts disk formats ● Enables VirtIO drivers (If needed) – Network, torage ● Fixes boot'loader ● ,roduces full oVirt-compatible OVF ● Outputs the VM in oVirt's export domain KVM Forum, August 2015 Drawbacks ● Slow ● Tedious ● Error-prone ● 5equires separate installation ● Do not support conversion of OVA 7les ● Error handlin& KVM Forum, August 2015 Our goal Improve the conversion process to oVirt – Faster – Tools are availa1le – Graphical user interface ● To con7&ure ● To monitor/cancel – Ro1ust – Support conversion of OVA files KVM Forum, August 2015 Design principles ● 8se virt-v9v capabilities – For &uest-level operations ● oVirt mana&es the conversion – -on7&ure conversion properties
    [Show full text]
  • Pipenightdreams Osgcal-Doc Mumudvb Mpg123-Alsa Tbb
    pipenightdreams osgcal-doc mumudvb mpg123-alsa tbb-examples libgammu4-dbg gcc-4.1-doc snort-rules-default davical cutmp3 libevolution5.0-cil aspell-am python-gobject-doc openoffice.org-l10n-mn libc6-xen xserver-xorg trophy-data t38modem pioneers-console libnb-platform10-java libgtkglext1-ruby libboost-wave1.39-dev drgenius bfbtester libchromexvmcpro1 isdnutils-xtools ubuntuone-client openoffice.org2-math openoffice.org-l10n-lt lsb-cxx-ia32 kdeartwork-emoticons-kde4 wmpuzzle trafshow python-plplot lx-gdb link-monitor-applet libscm-dev liblog-agent-logger-perl libccrtp-doc libclass-throwable-perl kde-i18n-csb jack-jconv hamradio-menus coinor-libvol-doc msx-emulator bitbake nabi language-pack-gnome-zh libpaperg popularity-contest xracer-tools xfont-nexus opendrim-lmp-baseserver libvorbisfile-ruby liblinebreak-doc libgfcui-2.0-0c2a-dbg libblacs-mpi-dev dict-freedict-spa-eng blender-ogrexml aspell-da x11-apps openoffice.org-l10n-lv openoffice.org-l10n-nl pnmtopng libodbcinstq1 libhsqldb-java-doc libmono-addins-gui0.2-cil sg3-utils linux-backports-modules-alsa-2.6.31-19-generic yorick-yeti-gsl python-pymssql plasma-widget-cpuload mcpp gpsim-lcd cl-csv libhtml-clean-perl asterisk-dbg apt-dater-dbg libgnome-mag1-dev language-pack-gnome-yo python-crypto svn-autoreleasedeb sugar-terminal-activity mii-diag maria-doc libplexus-component-api-java-doc libhugs-hgl-bundled libchipcard-libgwenhywfar47-plugins libghc6-random-dev freefem3d ezmlm cakephp-scripts aspell-ar ara-byte not+sparc openoffice.org-l10n-nn linux-backports-modules-karmic-generic-pae
    [Show full text]
  • Cloud Orchestration
    Cloud Orchestration What is a cloud?… ==> ▶ distributing the load automatically ▶ (and selling more than what you have – thin provisioning everywhere…) ▶ not necessarily virtualized – IaaS can also be bare-metal ▶ immediate delivery ▶ pay-as-you-use vs. unlimited ▶ QoS everywhere and accounting ▶ customers can use APIs Warning: this lecture is about setting-up your own cloud ▶ no AWS ▶ no GCP ▶ no Azure ▶ you’re the cloud administrator here, not the luser ▶ and it fits the sovereinty and privacy laws better, as long as your infrastructure is on the national territory VMM Orchestrators Different feature sets 1. just the UI 2. just an orchestrator 3. UI + orchestrator VMM farm UIs ▶ VMware vSphere vCenter ▶ Citrix XenCenter ▶ RHEV == oVirt (just like RHEL == CentOS) ▶ HyperV ▶ Proxmox (KVM & LXC) VMM orchestrators ▶ XCP-NG (XEN) ▶ Ganeti (XEN & KVM) ▶ Apache CloudStack (VMware, KVM, XenServer, XCP, Oracle VM, Hyper-V) ▶ OpenNebula (got orchestrator?) ▶ DanubeCloud (no orchestrator) ▶ OpenStack – just like k8s… why so much pain? LAB // test Apache CloudStack LAB // find the hot-migration scheduler algorithms in those code bases and discuss/compare techniques. On which hardware resources does it bases its decision? // Questions on VMM UIs and orchestrators? Anything else than Docker in mind?… ==> Container engines’ timeline ▶ chroot (1982) ▶ FreeBSD jails (2000) ▶ Solaris Zones / Containers (2004) ▶ AIX WPARs (2007) ▶ Virtuozzo / OpenVZ (resp. 2000 / 2005) ▶ Ubuntu LXC / LXD (2008) ▶ systemd-nspawn (2010) ▶ Docker (2013) ▶ runc/containerd (Jul 16, 2015) ▶ Singularity (2015) What is a chroot?… ==> the thing you sometimes need for rescuing a system There are also chroot-capable daemons (named, unbound, postfix) Anyhow, which engine are good to choose?… ==> stick with the standards ▶ (ideally jails but that’s for Free-and-DflyBSD) ▶ Docker for apps and micro-services ▶ LXC for systems LAB // are there orchestrators for BSD jails? LAB // study and retro-PoC the sysjail vuln on Net-or-OpenBSD.
    [Show full text]
  • INSTITUT F¨UR INFORMATIK Hochverf¨Ugbarkeit F¨Ur Linux
    T U M I N S T I T U T F U¨ R I N F O R M A T I K Hochverfugbark¨ eit fur¨ Linux Franz Huber, Tobias SchropfÈ TUM-I0828 August 08 T E C H N I S C H E U N I V E R S I T AÈ T M UÈ N C H E N TUM-INFO-08-I0828-0/1.-FI Alle Rechte vorbehalten Nachdruck auch auszugsweise verboten c 2008 Druck: Institut fur¨ Informatik der Technischen Universitat¨ Munchen¨ Hochverfugbarkeit¨ fur¨ Linux Franz Huber, Tobias Schr¨opf Institut fur¨ Informatik, Technische Universit¨at Munchen¨ fhuberf,[email protected] Zusammenfassung Mit dem zunehmenden Einsatz von Linux-basierten Serversystemen auch im Re- chenzentrumsbetrieb hat dieses frei verfugbare¨ Betriebssystem ein Anwendungsfeld erreicht, in dem die permanente Verfugbarkeit¨ der bereitgestellten Dienste eine we- sentliche Anforderung ist. Sogenannte Hochverfugbarkeitsl¨ ¨osungen sind in gesch¨afts- kritischen Einsatzbereichen daher auch fur¨ Linux nicht mehr wegzudenken. Hochverfugbare¨ Systeme sind dadurch gekennzeichnet, dass m¨oglichst alle Fehler- quellen, die einen Systemausfall herbeifuhren¨ k¨onnen, eliminiert werden. Das Besei- tigen dieser Single Points of Failure\ erfolgt in erster Linie durch Duplizierung der " Systeme. Diese Arbeit gibt einen Uberblick¨ uber¨ den aktuellen Stand der Hochverfugbar-¨ keitsl¨osungen fur¨ Linux und befasst sich daher haupts¨achlich mit freien, OpenSour- ce Hochverfugbarkeitsl¨ ¨osungen. Als Evaluationsplattform dient eine mit dem frei- en Hypervisor Xen virtualisierte Server-Infrastruktur. Hier werden Mechanismen untersucht, um Produktivsysteme durch passive Standby-Systeme ausfallsicher zu machen. Ein weiterer Ansatz, der ebenfalls untersucht wird, sind mehrere aktive Sys- teme, die im Normalbetrieb die Anfragen untereinander aufteilen, und bei Ausf¨allen die Anfragen der ausgefallenen Systeme bearbeiten.
    [Show full text]
  • 27Th Large Installation System Administration Conference (LISA '13)
    conference proceedings Proceedings of the 27th Large Installation System Administration Conference 27th Large Installation System Administration Conference (LISA ’13) Washington, D.C., USA November 3–8, 2013 Washington, D.C., USA November 3–8, 2013 Sponsored by In cooperation with LOPSA Thanks to Our LISA ’13 Sponsors Thanks to Our USENIX and LISA SIG Supporters Gold Sponsors USENIX Patrons Google InfoSys Microsoft Research NetApp VMware USENIX Benefactors Akamai EMC Hewlett-Packard Linux Journal Linux Pro Magazine Puppet Labs Silver Sponsors USENIX and LISA SIG Partners Cambridge Computer Google USENIX Partners Bronze Sponsors Meraki Nutanix Media Sponsors and Industry Partners ACM Queue IEEE Security & Privacy LXer ADMIN IEEE Software No Starch Press CiSE InfoSec News O’Reilly Media Computer IT/Dev Connections Open Source Data Center Conference Distributed Management Task Force IT Professional (OSDC) (DMTF) Linux Foundation Server Fault Free Software Magazine Linux Journal The Data Center Journal HPCwire Linux Pro Magazine Userfriendly.org IEEE Pervasive © 2013 by The USENIX Association All Rights Reserved This volume is published as a collective work. Rights to individual papers remain with the author or the author’s employer. Permission is granted for the noncommercial reproduction of the complete work for educational or research purposes. Permission is granted to print, primarily for one person’s exclusive use, a single copy of these Proceedings. USENIX acknowledges all trademarks herein. ISBN 978-1-931971-05-8 USENIX Association Proceedings of the 27th Large Installation System Administration Conference November 3–8, 2013 Washington, D.C. Conference Organizers Program Co-Chairs David Nalley, Apache Cloudstack Narayan Desai, Argonne National Laboratory Adele Shakal, Metacloud, Inc.
    [Show full text]
  • Dynamic Load Balancing of Virtual Machines Hosted on Xen
    Brigham Young University BYU ScholarsArchive Theses and Dissertations 2008-12-10 Dynamic Load Balancing of Virtual Machines Hosted on Xen Terry Clyde Wilcox Brigham Young University - Provo Follow this and additional works at: https://scholarsarchive.byu.edu/etd Part of the Computer Sciences Commons BYU ScholarsArchive Citation Wilcox, Terry Clyde, "Dynamic Load Balancing of Virtual Machines Hosted on Xen" (2008). Theses and Dissertations. 1654. https://scholarsarchive.byu.edu/etd/1654 This Thesis is brought to you for free and open access by BYU ScholarsArchive. It has been accepted for inclusion in Theses and Dissertations by an authorized administrator of BYU ScholarsArchive. For more information, please contact [email protected], [email protected]. DYNAMIC LOAD BALANCING OF VIRTUAL MACHINES HOSTED ON XEN by Terry C. Wilcox Jr. A thesis submitted to the faculty of Brigham Young University in partial fulfillment of the requirements for the degree of Master of Science Department of Computer Science Brigham Young University April 2009 Copyright c 2009 Terry C. Wilcox Jr. All Rights Reserved BRIGHAM YOUNG UNIVERSITY GRADUATE COMMITTEE APPROVAL of a thesis submitted by Terry C. Wilcox Jr. This thesis has been read by each member of the following graduate committee and by majority vote has been found to be satisfactory. Date Phillip J. Windley, Chair Date Kelly J. Flanagan Date Tony R. Martinez BRIGHAM YOUNG UNIVERSITY As chair of the candidate’s graduate committee, I have read the thesis of Terry C. Wilcox Jr. in its final form and have found that (1) its format, citations, and bibli- ographical style are consistent and acceptable and fulfill university and department style requirements; (2) its illustrative materials including figures, tables, and charts are in place; and (3) the final manuscript is satisfactory to the graduate committee and is ready for submission to the university library.
    [Show full text]
  • Openstack Vs. Ganeti
    OpenStack vs. Ganeti · Lance Albertson · Director, OSU Open Source Lab · http://osuosl.org · @ramereth Attribution-ShareAlike CC BY-SA ©2015 Session Overview OpenStack quick overview Ganeti overview Ganeti walk-through Comparing both About me Lance Albertson Director, OSU Open Source Lab (OSUOSL) Provide infrastructure hosting for FOSS projects Linux Foundation, Python Software Foundation, Drupal, etc Ops guy Ganeti user since 2009 OpenStack user since 2013 http://osuosl.org Virtualized Computing Resources Each organization has different needs Some are small and simple Others are more complex and larger Cost is also a major factor (both in licenses and staff maintenence) Typical Solutions VMWare VirtualBox OpenStack [insert favorite solution] OpenStack is all the hype Designed to scale and be an AWS replacement (almost) Everyone is investing in it Still maturing as a project Includes a wide-array of features, many of which most people don't need OpenStack Overview OpenStack Pros Standard Cloud API Fast VM deployment and tear down Very elastic computing needs Large community support Fast growing and new features constantly OpenStack Cons Extremely difficult to deploy and maintain Lots of moving parts Project is still maturing and unstable Fits a very specific use-case (Cloud) Requires more than one machine to effectively use So what about Ganeti? First, what is Ganeti? Cluster management tool for virtual compute resources IaaS solution that uses either KVM or Xen hypervisors Provides fast and simple recovery from hardware failures Primarily
    [Show full text]
  • Ganeti Scalable Virtualization with Ganeti by Lance Albertson About Me
    Ganeti Scalable Virtualization with Ganeti by Lance Albertson About Me OSU Open Source Lab Server hosting for Open Source projects Lead Systems Administrator / Architect Gentoo developer / contributor Jazz trumpet performer What I will cover Ganeti terminology, comparisons, & goals Cluster & virtual machine setup Dealing with outages Web Interface to Ganeti OSUOSL usage of ganeti Future roadmap State of Virtualization Citrix XenServer libvirt: oVirt, virt-manager Eucalyptus VMWare Open Stack Issues Overly complicated Multiple layers of software Lack of HA Storage integration Not always 100% open source Traditional virtualization cluster Ganeti cluster What is ganeti? Software to manage a cluster of virtual servers Project created and maintained by Google Combines virtualization & data replication Works with multiple hypervisors Automates storage management Automates OS deployment Ganeti software requirements Python various python modules DRBD LVM KVM, Xen, or LXC* Ganeti terminology Cluster - group of nodes Node - physical host Instance - virtual machine, aka guest Goals Reduce hardware cost Increase service availability Simplify VM cluster management Administration transparency Principles Not dependent on specific hardware Scales linearly Centralized administration N+1 redundancy Storage Options LVM LVM + DRBD (supports migration) File based (raw, qcow2, etc) Shared storage patch coming soon! Storage: LVM + DRBD Primary & secondary storage nodes Each instance disk synced separately Dedicated backend DRBD network Allows instance failover
    [Show full text]
  • Virtualization Explained: Definitions You Need to Know
    Virtualization Explained: Definitions You Need to Know By Stephen J. Bigelow with Edward L. Haletky Page | 1 Virtualization Explained: Definitions You Need to Know Table of contents A N AMD-V ... 3 Network virtualization … 11 C P Capacity planning ... 4 P2V … 12 Clone ... 5 Provisioning … 13 Clustered file system ... 6 S G Server consolidation … 14 Guests and hosts ... 7 Snapshot … 15 Storage virtualization … 16 I Intel VT ... 8 V Virtual hardware … 17 Virtual machine … 18 L Virtual machine monitoring … 19 Live migration ... 9 Virtual switch … 20 VMDK … 21 M Memory overcommit … 10 X Xen … 22 Page | 2 A AMD-V IT professionals should understand which AMD processors have AMD-V technology. But first, a quick history lesson: Traditional computing does not allow software to readily share hardware resources. Virtualization overcomes that limitation and allows two or more virtual machines (VMs) to share the computing resources of a single physical server, and the VM monitor is central to any virtualization deployment. It provides the abstraction that separates software from the underlying hardware and manipulates the virtual workloads so that each workload can share the common processor, memory, I/O and local storage. Early virtualization efforts relied on software emulation to replace hardware functionality. But software emulation is a slow and inefficient process. Many virtualization tasks were handled through software, so VM behavior and resource control were often poor, resulting in unacceptable VM performance on the server. By early 2005, processors still lacked the internal microcode to handle intensive virtualization tasks in hardware. Both Intel Corp. and Advanced Micro Dynamics Inc. (AMD) addressed this problem by creating a new set of processor extensions (similar to MMX, 3DNow! and others) that could offload the repetitive and inefficient work from the software.
    [Show full text]
  • Ganeti Web Manager Documentation Release 0.11.1
    Ganeti Web Manager Documentation Release 0.11.1 Oregon State University Open Source Lab Oct 22, 2019 CONTENTS 1 Getting Started 3 2 Deployment 17 3 Features 21 4 Usage 33 5 Contributing 35 6 Project Information 51 7 Deprecated 95 8 Indices and tables 105 Index 107 i ii Ganeti Web Manager Documentation, Release 0.11.1 Ganeti Web Manager is a Django based web frontend for managing Ganeti virtualization clusters. Since Ganeti only provides a command-line interface, Ganeti Web Manager’s goal is to provide a user friendly web interface to Ganeti via Ganeti’s Remote API. On top of Ganeti it provides a permission system for managing access to clusters and virtual machines, an in browser VNC console, and vm state and resource visualizations. You can see a few screenshots here. If you don’t already have a Ganeti cluster setup, these directions can help you get started. If you are looking for support, please contact us. If you are looking to deploy Ganeti Web Manager for the first time, check out our Installation guide. If you already have a Ganeti Web Manager instance running it might be time to upgrade. Finally, if you would like to report a bug or request a feature, please file an issue. Ganeti Web Manager is licensed under the GPLv2. It is currently developed and maintained by the Oregon State University Open Source Lab and a handful of volunteers. If you would like to get involved in development see our development guide. CONTENTS 1 Ganeti Web Manager Documentation, Release 0.11.1 2 CONTENTS CHAPTER ONE GETTING STARTED 1.1 Requirements 1.1.1 Operating system We officially support Ubuntu 11.10, Ubuntu 12.04 and CentOS 6.
    [Show full text]