DOCSLIB.ORG

Ganeti Extstorage Interface More Options for Your Data

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Ganeti Extstorage Interface More Options for Your Data Ganeti, the New and Arcane ganeti's best kept secrets, and exciting new developments Ganeti Eng Team - Google LinuxCon Japan 2014 - 2 Feb 2014 Introduction to Ganeti A cluster virtualization manager, in one slide What is Ganeti? · Manage clusters 1-200 of physical machines, divided in nodegroups · Deploy Xen/KVM/LXC virtual machines on them - Live migration - Resiliency to failure (DRBD, Ceph, SAN/NAS, ...) - Cluster balancing - Ease of repairs and hardware swaps · Controlled via command line, REST, web interfaces 4/53 Newest features Development status 2.10 The very stable release · Improved upgrade procedure "gnt-cluster upgrade" · CPU Load in hail/hbal (GSOC project) · Hotplug support (KVM) · RBD storage direct access (KVM) · Better Openvswitch support (GSOC project) 6/53 2.11 The latest stable release · Faster instance moves · GlusterFS support · hsqueeze (achieve maximum cluster compaction) 7/53 2.12 and future The next stable release(s) · Jobs as processes · New install model · More secure master candidates · Better container support (GSOC) · Resource reservation/Extra parallelization · Generic conversion between disk templates (GSOC) 8/53 Monitoring daemon What's going on in your cluster? Monitoring a cluster The old school way Other Systems Cluster Monitoring NICs System Instance Master Node Storage 10/53 Monitoring a cluster Using the monitoring daemon Other Systems Cluster Monitoring System Monitoring Daemons 11/53 What is the monitoring daemon? Provides information: · about the cluster state/health · live · read-only design doc: design-monitoring-agent.rst 12/53 More details · HTTP daemon · Replying to REST-like queries · Actually, GET only · Providing JSON replies · Easy to parse in any language · Already used in all the rest of Ganeti · Running on every node (Not: only master-candidates, VM-enabled) · Additionally: mon-collector: quick 'n dirty CLI tool 13/53 Data collectors · provide data to the deamon · one collector, one report · one collector, one category: - storage, hypervisor, daemon, instance · two kinds: performance reporting, status reporting · new feature: stateful data collectors 14/53 Data collectors What data can be retrieved right now? Now: · instance status (Xen only) (category: instance) · diskstats information (storage) · LVM logical volumes information (storage) · DRBD status information (storage) · Node OS CPU load average (no category, default) Soon(-ish): · instance status for KVM (instance) · Ganeti daemons status (daemon) · Hypervisor resources (hypervisor) · Node OS resources report (default) 15/53 The report format { JSON "name" : "TheCollectorIdentifier", "version" : "1.2", "format_version" : 1, "timestamp" : 1351607182000000000, "category" : null, "kind" : 0, "data" : { "plugin_specific_data" : "go_here" } } · name: the name of the plugin. Unique string. · version: the version of the plugin. A string. · format_version: the version of the data format of the plugin. Incremental integer. · timestamp: when the report was produced. Nanoseconds. Can be zero- padded. 16/53 Status reporting collectors: report They introduce a mandatory part inside the data section. "data" : { JSON ... "status" : { "code" : <value> "message: "some summary goes here" } } · <value>:by increasing criticality level · 0: working as intended · 1: temporarily wrong. Being auto-repaired · 2: unknown. Potentially dangerous state · 4: problems. External intervention required 17/53 How to use the daemon? · Accepts HTTP connections on node.example.com:1815 · Not authenticated: read only · Just firewall, or bind on local address only · GET requests to specific addresses · Each address returns different info according to the API / (return the list of supported protocol version) /1/list/collectors /1/report/all /1/report/[category]/[collector_name] 18/53 Configuration Daemon (confd) How's your cluster supposed to look like? Before confd · Configuration only available on master candidates · Few selected values replicated with ssconf · Small pieces of config in text files on all the nodes · Doesn't scale · Need for a way to access config from other nodes · Scalable · No single point of failure (so, no RAPI) 20/53 What does confd do? · Provides information from config.data · Read-only · Distributed · Multiple daemons running on master candidates · Accessible from all the nodes through confd protocol · Resilient to failures · Optional 21/53 What info does it provide? Replies to simple queries: · Ping · Master IP · Node role · Node primary IP · Master candidates primary IPs · Instance IPs · Node primary IP from Instance primary IP · Node DRBD minors · Node instances 22/53 confd protocol General description · UDP (port 1814) · keyed-Hash Message Authentication Code (HMAC) authentication · Pre-shared, cluster wide key · Generated at cluster-init · Root-only readable · Timestamp · Checked (± 2.5 mins) to prevent replay attacks · Used as HMAC salt · Queries made to any subset of master candidates · Timeout · Maximum number of expected replies 23/53 Confd protocol Request/Reply request request request request request 24/53 Confd protocol Request/Reply timeout reply (v: 57) reply (v: 57) reply (v: 57) reply (v: 56) (enough replies) 25/53 confd protocol Request plj0{ CONFD "msg": "{\"type\": 1, \"rsalt\": \"9aa6ce92-8336-11de-af38-001d093e835f\", \"protocol\": 1, \"query\": \"node1.example.com\"}\n", "salt": "1249637704", "hmac": "4a4139b2c3c5921f7e439469a0a45ad200aead0f" } · plj0: fourcc detailing the message content (PLain Json 0) · hmac: HMAC signature of salt+msg with the cluster hmac key 26/53 confd protocol Request plj0{ CONFD "msg": "{\"type\": 1, \"rsalt\": \"9aa6ce92-8336-11de-af38-001d093e835f\", \"protocol\": 1, \"query\": \"node1.example.com\"}\n", "salt": "1249637704", "hmac": "4a4139b2c3c5921f7e439469a0a45ad200aead0f" } · msg: JSON-encoded query · protocol: confd protocol version (=1) · type: What to ask for (CONFD_REQ_* constants) · query: additional parameters · rsalt: response salt == UUID identifying the request 27/53 confd protocol Reply plj0{ CONFD "msg": "{\"status\": 0, \"answer\": 0, \"serial\": 42, \"protocol\": 1}\n", "salt": "9aa6ce92-8336-11de-af38-001d093e835f", "hmac": "aaeccc0dff9328fdf7967cb600b6a80a6a9332af" } · salt: the rsalt of the query · hmac: hmac signature of salt+msg 28/53 confd protocol Reply plj0{ CONFD "msg": "{\"status\": 0, \"answer\": 0, \"serial\": 42, \"protocol\": 1}\n", "salt": "9aa6ce92-8336-11de-af38-001d093e835f", "hmac": "aaeccc0dff9328fdf7967cb600b6a80a6a9332af" } · msg: JSON-encoded answer · protocol: protocol version (=1) · status: 0=ok; 1=error · answer: query-specific reply · serial: version of config.data 29/53 Ready-made clients The protocol is simple, but clients are simpler · Ready to use confd clients · Python · lib/confd/client.py · Haskell · Since Ganeti 2.7 · src/Ganeti/ConfD/Client.hs · src/Ganeti/ConfD/ClientFunctions.hs 30/53 Expanding confd capabilities · Currently not so many queries are supported · Easy to add new ones · Just add a new query type in the constants list · ...and extend the buildResponse function (src/Ganeti/Confd/Server.hs to reply to it in the appropriate way 31/53 Ganeti and Networks How do your instances talk to the world? · Some slides contributed by Dimitris Aragiorgis <[email protected]> current nics: MAC + IP + link + mode NIC configuration · mode=bridged uses brctl addif · Hooks can deal with firewall rules, and more · External systems needed for DHCP, IPv6, etc. Management · Which VMs are on the same collision domain? · Which IP is free for a new VM to use? 33/53 gnt-network overview · manage collision domains for your instances · easy way to assign IPs to instances - If resources are shared in multiple clusters, allocation must be done externally · keep existing per-nic flexibility · hide underlying infrastructure · better networking overview 34/53 gnt-network: Who does what? masterd: config.data integrity · abstract network infrastructure: network + netparams per nodegroup · IP uniqueness inside network: IP pool management - bitarray, TemporaryReservationmanager, Locking · encapsulate network information in NIC opjects: RPC external scripts and hooks: ping vm1.ganeti.example.com · use exported environment provided by noded · brctl, iptables, ebtables, ip rule, etc. · update external dhcp/DNS server entries · let VM act unaware of the "situation" (dhclient, etc.) 35/53 gnt-network + external scripts · gnt-network alone is nothing more than a nice config.data · snf-network: node level scripts and hooks · nfdhcpd: node level DHCP server based on NFQUEUE 36/53 snf-network node level scripts and hooks · overrides Ganeti default scripts (kvm-ifup, vif-ganeti) · looks for specific tag types in NIC's network · applies corresponding rules · created nfdhcpd binding files · provides hook to update DNS entries 37/53 nfdhcpd node level DHCP server based on NFQUEUE · listens on specific NFQEUE · updates its leases db - inotify on specific directory for binding files · mangles DHCP requests and replies based on it's db · responds to RS and NS for IPv6 auto-configuration 38/53 gnt-network Examples Create and connect a new network gnt-network add --network 192.168.1.0/24 --gateway 192.168.1.1 --tags nfdhcpd net1 gnt-network connect net1 bridged prv0 Create an instance inside this network gnt-instance add --net 0:ip=pool,network=net1 ... inst1 gnt-instance info inst1 gnt-network info net1 39/53 gnt-network + snf-* Examples Use snf-network and nfdhcpd apt-get install snf-network nfdhcdpd iptables -t mangle -A PREROUTING -i prv+ -p udp -m udp --dport 67
Load more

Recommended publications
  • Homework #5 Answer Key
    CS312 Homework #5 Answer Key March 9, 2016 Questions 1. Load balancers are designed to help with vertical scaling. (a) True (b) False 2. Layer 7 load balancers allow you to use both connection tracking and inspection of packets including modification of packets. (a) True (b) False 3. Linux containers do not require hypervisors (a) True (b) False 1 4. The RUN command can be used more than once in a Dockerfile. (a) True (b) False 5. The PID Namespace allows users to see processes from other containers (a) True (b) False 6. Paravirtualization uses cpu supported virtualization. (a) True (b) False 7. Which software load balancer primarily acts as an HTTP accelerator and a static cache server? (a) HAProxy (b) Apache (c) nginx (d) Varnish 8. Which of the following is not true about Linux Containers? (a) Operating system level virtualization (b) Provides little overhead (c) Fully emulates an operating system (d) Allows limits to resources with cgroups 9. Which service describes a virtual computing platform? (a) IaaS (b) PaaS (c) SaaS (d) VaaS 2 10. Describe the potential problems of using Round Robin DNS. • Depending on the implementation of the client resolver, this can be somewhat random or not. • Some resolvers always use alphabetical ordering. • DNS Caching and Time-To-Live (TTL) issues. • If the site goes down, you have to update DNS to reroute traffic which can cause problems. 11. Which scheduling algorithm is generally preferred if the web application uses sessions? Source Connection 12. Which HAProxy configuration section allows you to define a complete proxy? listen 13.
    [Show full text]
  • Using Ganeti for Running Highly Available Virtual Services
    Using Ganeti for running highly available virtual services 2016-04-21, HEPiX Spring 2016, Zeuthen Overview ● What is Ganeti ● What is it good for ● How does it work ● NDGF usage 2 What is Ganeti ● A software stack for managing virtual machines – Like VMware or OpenStack or libvirt or ... – Supporting Xen or KVM hypervisors – Handles ● Storage: volume creation and assignment ● OS installation and customization ● Networking ● Startup, shutdown, live migration, failover of instances – Written in Python and Haskell – Aimed for ease of use and fast and simple error recovery after physical failures on commodity hardware 3 What is Ganeti ● Mainly developed by Google for their own use – Handles VMs for corporate network (office servers, remote desktops etc), not production services (what non-employees see) ● Outside Google – Debian – NDGF-T1 – Lufthansa – Etc ● Maintained by Google with significant external contributions 4 What is Ganeti good at ● Running highly available services on a small set of hardware – DRBD or external reliable block devices (CEPH, Enterprise storage) – Live migrations in case of impending hardware failure ● Or reboot into new kernel security upgrade on the hardnode – Failover handled automatically in case of sudden hardware failure – No external dependencies beyond networking ● Well, if you use external storage... ● But no extra servers or services needed – Typical reasonable cluster size, 3 – 50 hardnodes ● Multiple clusters integrate well though in admin tools 5 How does Ganeti work ● gnt-cluster init ... – Creates a cluster of ganeti nodes – We'll assume DRBD for storage, as at NDGF – One member is a master node ● Others can take over with master-failover if they can get quorum A B C D E 6 How does Ganeti work ● gnt-instance add – Creates VMs, with OS install scripts (image, debootstrap, pxe) – Each VM has a secondary location (DRBD mirror, sync) A B C D E 7 How does Ganeti work ● gnt-instance migrate – No noticable service impact from live migration, <1s network pause ● Unless something is broken..
    [Show full text]
  • Managed Conversion of Guests to Ovirt
    Managed Conversion of Guests to oVirt Arik Hadas Senior Software Engineer Red Hat 21/8/15 KVM Forum, August 2015 Agenda ● Motivation ● Architecture ● Demonstration ● Implementation ● Future work KVM Forum, August 2015 Many ways to run Virtual Machines ● There are many virtualization tools ● Different hypervisors – KVM, E !"E Xi, Xen, VirtualBo$, .%% ● Different management systems – oVirt, virt'manager, v phere, Ganeti, .%% KVM Forum, August 2015 “I don't want to lose my VMs” ● Virtualization technologies are used for a long time ● +o standardization ● ,eople are tied up to the technologies they currently use ● Conversion tools are neede). KVM Forum, August 2015 virt-v2v ● ,art of virt tools – /pen source virtualization management tools ● Foreign hypervisor -0 KVM ● Standalone conversion tool KVM Forum, August 2015 Conversion to oVirt using virt-v2v ● Converts disk formats ● Enables VirtIO drivers (If needed) – Network, torage ● Fixes boot'loader ● ,roduces full oVirt-compatible OVF ● Outputs the VM in oVirt's export domain KVM Forum, August 2015 Drawbacks ● Slow ● Tedious ● Error-prone ● 5equires separate installation ● Do not support conversion of OVA 7les ● Error handlin& KVM Forum, August 2015 Our goal Improve the conversion process to oVirt – Faster – Tools are availa1le – Graphical user interface ● To con7&ure ● To monitor/cancel – Ro1ust – Support conversion of OVA files KVM Forum, August 2015 Design principles ● 8se virt-v9v capabilities – For &uest-level operations ● oVirt mana&es the conversion – -on7&ure conversion properties
    [Show full text]
  • Pipenightdreams Osgcal-Doc Mumudvb Mpg123-Alsa Tbb
    pipenightdreams osgcal-doc mumudvb mpg123-alsa tbb-examples libgammu4-dbg gcc-4.1-doc snort-rules-default davical cutmp3 libevolution5.0-cil aspell-am python-gobject-doc openoffice.org-l10n-mn libc6-xen xserver-xorg trophy-data t38modem pioneers-console libnb-platform10-java libgtkglext1-ruby libboost-wave1.39-dev drgenius bfbtester libchromexvmcpro1 isdnutils-xtools ubuntuone-client openoffice.org2-math openoffice.org-l10n-lt lsb-cxx-ia32 kdeartwork-emoticons-kde4 wmpuzzle trafshow python-plplot lx-gdb link-monitor-applet libscm-dev liblog-agent-logger-perl libccrtp-doc libclass-throwable-perl kde-i18n-csb jack-jconv hamradio-menus coinor-libvol-doc msx-emulator bitbake nabi language-pack-gnome-zh libpaperg popularity-contest xracer-tools xfont-nexus opendrim-lmp-baseserver libvorbisfile-ruby liblinebreak-doc libgfcui-2.0-0c2a-dbg libblacs-mpi-dev dict-freedict-spa-eng blender-ogrexml aspell-da x11-apps openoffice.org-l10n-lv openoffice.org-l10n-nl pnmtopng libodbcinstq1 libhsqldb-java-doc libmono-addins-gui0.2-cil sg3-utils linux-backports-modules-alsa-2.6.31-19-generic yorick-yeti-gsl python-pymssql plasma-widget-cpuload mcpp gpsim-lcd cl-csv libhtml-clean-perl asterisk-dbg apt-dater-dbg libgnome-mag1-dev language-pack-gnome-yo python-crypto svn-autoreleasedeb sugar-terminal-activity mii-diag maria-doc libplexus-component-api-java-doc libhugs-hgl-bundled libchipcard-libgwenhywfar47-plugins libghc6-random-dev freefem3d ezmlm cakephp-scripts aspell-ar ara-byte not+sparc openoffice.org-l10n-nn linux-backports-modules-karmic-generic-pae
    [Show full text]
  • Cloud Orchestration
    Cloud Orchestration What is a cloud?… ==> ▶ distributing the load automatically ▶ (and selling more than what you have – thin provisioning everywhere…) ▶ not necessarily virtualized – IaaS can also be bare-metal ▶ immediate delivery ▶ pay-as-you-use vs. unlimited ▶ QoS everywhere and accounting ▶ customers can use APIs Warning: this lecture is about setting-up your own cloud ▶ no AWS ▶ no GCP ▶ no Azure ▶ you’re the cloud administrator here, not the luser ▶ and it fits the sovereinty and privacy laws better, as long as your infrastructure is on the national territory VMM Orchestrators Different feature sets 1. just the UI 2. just an orchestrator 3. UI + orchestrator VMM farm UIs ▶ VMware vSphere vCenter ▶ Citrix XenCenter ▶ RHEV == oVirt (just like RHEL == CentOS) ▶ HyperV ▶ Proxmox (KVM & LXC) VMM orchestrators ▶ XCP-NG (XEN) ▶ Ganeti (XEN & KVM) ▶ Apache CloudStack (VMware, KVM, XenServer, XCP, Oracle VM, Hyper-V) ▶ OpenNebula (got orchestrator?) ▶ DanubeCloud (no orchestrator) ▶ OpenStack – just like k8s… why so much pain? LAB // test Apache CloudStack LAB // find the hot-migration scheduler algorithms in those code bases and discuss/compare techniques. On which hardware resources does it bases its decision? // Questions on VMM UIs and orchestrators? Anything else than Docker in mind?… ==> Container engines’ timeline ▶ chroot (1982) ▶ FreeBSD jails (2000) ▶ Solaris Zones / Containers (2004) ▶ AIX WPARs (2007) ▶ Virtuozzo / OpenVZ (resp. 2000 / 2005) ▶ Ubuntu LXC / LXD (2008) ▶ systemd-nspawn (2010) ▶ Docker (2013) ▶ runc/containerd (Jul 16, 2015) ▶ Singularity (2015) What is a chroot?… ==> the thing you sometimes need for rescuing a system There are also chroot-capable daemons (named, unbound, postfix) Anyhow, which engine are good to choose?… ==> stick with the standards ▶ (ideally jails but that’s for Free-and-DflyBSD) ▶ Docker for apps and micro-services ▶ LXC for systems LAB // are there orchestrators for BSD jails? LAB // study and retro-PoC the sysjail vuln on Net-or-OpenBSD.
    [Show full text]
  • INSTITUT F¨UR INFORMATIK Hochverf¨Ugbarkeit F¨Ur Linux
    T U M I N S T I T U T F U¨ R I N F O R M A T I K Hochverfugbark¨ eit fur¨ Linux Franz Huber, Tobias SchropfÈ TUM-I0828 August 08 T E C H N I S C H E U N I V E R S I T AÈ T M UÈ N C H E N TUM-INFO-08-I0828-0/1.-FI Alle Rechte vorbehalten Nachdruck auch auszugsweise verboten c 2008 Druck: Institut fur¨ Informatik der Technischen Universitat¨ Munchen¨ Hochverfugbarkeit¨ fur¨ Linux Franz Huber, Tobias Schr¨opf Institut fur¨ Informatik, Technische Universit¨at Munchen¨ fhuberf,[email protected] Zusammenfassung Mit dem zunehmenden Einsatz von Linux-basierten Serversystemen auch im Re- chenzentrumsbetrieb hat dieses frei verfugbare¨ Betriebssystem ein Anwendungsfeld erreicht, in dem die permanente Verfugbarkeit¨ der bereitgestellten Dienste eine we- sentliche Anforderung ist. Sogenannte Hochverfugbarkeitsl¨ ¨osungen sind in gesch¨afts- kritischen Einsatzbereichen daher auch fur¨ Linux nicht mehr wegzudenken. Hochverfugbare¨ Systeme sind dadurch gekennzeichnet, dass m¨oglichst alle Fehler- quellen, die einen Systemausfall herbeifuhren¨ k¨onnen, eliminiert werden. Das Besei- tigen dieser Single Points of Failure\ erfolgt in erster Linie durch Duplizierung der " Systeme. Diese Arbeit gibt einen Uberblick¨ uber¨ den aktuellen Stand der Hochverfugbar-¨ keitsl¨osungen fur¨ Linux und befasst sich daher haupts¨achlich mit freien, OpenSour- ce Hochverfugbarkeitsl¨ ¨osungen. Als Evaluationsplattform dient eine mit dem frei- en Hypervisor Xen virtualisierte Server-Infrastruktur. Hier werden Mechanismen untersucht, um Produktivsysteme durch passive Standby-Systeme ausfallsicher zu machen. Ein weiterer Ansatz, der ebenfalls untersucht wird, sind mehrere aktive Sys- teme, die im Normalbetrieb die Anfragen untereinander aufteilen, und bei Ausf¨allen die Anfragen der ausgefallenen Systeme bearbeiten.
    [Show full text]
  • 27Th Large Installation System Administration Conference (LISA '13)
    conference proceedings Proceedings of the 27th Large Installation System Administration Conference 27th Large Installation System Administration Conference (LISA ’13) Washington, D.C., USA November 3–8, 2013 Washington, D.C., USA November 3–8, 2013 Sponsored by In cooperation with LOPSA Thanks to Our LISA ’13 Sponsors Thanks to Our USENIX and LISA SIG Supporters Gold Sponsors USENIX Patrons Google InfoSys Microsoft Research NetApp VMware USENIX Benefactors Akamai EMC Hewlett-Packard Linux Journal Linux Pro Magazine Puppet Labs Silver Sponsors USENIX and LISA SIG Partners Cambridge Computer Google USENIX Partners Bronze Sponsors Meraki Nutanix Media Sponsors and Industry Partners ACM Queue IEEE Security & Privacy LXer ADMIN IEEE Software No Starch Press CiSE InfoSec News O’Reilly Media Computer IT/Dev Connections Open Source Data Center Conference Distributed Management Task Force IT Professional (OSDC) (DMTF) Linux Foundation Server Fault Free Software Magazine Linux Journal The Data Center Journal HPCwire Linux Pro Magazine Userfriendly.org IEEE Pervasive © 2013 by The USENIX Association All Rights Reserved This volume is published as a collective work. Rights to individual papers remain with the author or the author’s employer. Permission is granted for the noncommercial reproduction of the complete work for educational or research purposes. Permission is granted to print, primarily for one person’s exclusive use, a single copy of these Proceedings. USENIX acknowledges all trademarks herein. ISBN 978-1-931971-05-8 USENIX Association Proceedings of the 27th Large Installation System Administration Conference November 3–8, 2013 Washington, D.C. Conference Organizers Program Co-Chairs David Nalley, Apache Cloudstack Narayan Desai, Argonne National Laboratory Adele Shakal, Metacloud, Inc.
    [Show full text]
  • Dynamic Load Balancing of Virtual Machines Hosted on Xen
    Brigham Young University BYU ScholarsArchive Theses and Dissertations 2008-12-10 Dynamic Load Balancing of Virtual Machines Hosted on Xen Terry Clyde Wilcox Brigham Young University - Provo Follow this and additional works at: https://scholarsarchive.byu.edu/etd Part of the Computer Sciences Commons BYU ScholarsArchive Citation Wilcox, Terry Clyde, "Dynamic Load Balancing of Virtual Machines Hosted on Xen" (2008). Theses and Dissertations. 1654. https://scholarsarchive.byu.edu/etd/1654 This Thesis is brought to you for free and open access by BYU ScholarsArchive. It has been accepted for inclusion in Theses and Dissertations by an authorized administrator of BYU ScholarsArchive. For more information, please contact [email protected], [email protected]. DYNAMIC LOAD BALANCING OF VIRTUAL MACHINES HOSTED ON XEN by Terry C. Wilcox Jr. A thesis submitted to the faculty of Brigham Young University in partial fulfillment of the requirements for the degree of Master of Science Department of Computer Science Brigham Young University April 2009 Copyright c 2009 Terry C. Wilcox Jr. All Rights Reserved BRIGHAM YOUNG UNIVERSITY GRADUATE COMMITTEE APPROVAL of a thesis submitted by Terry C. Wilcox Jr. This thesis has been read by each member of the following graduate committee and by majority vote has been found to be satisfactory. Date Phillip J. Windley, Chair Date Kelly J. Flanagan Date Tony R. Martinez BRIGHAM YOUNG UNIVERSITY As chair of the candidate’s graduate committee, I have read the thesis of Terry C. Wilcox Jr. in its final form and have found that (1) its format, citations, and bibli- ographical style are consistent and acceptable and fulfill university and department style requirements; (2) its illustrative materials including figures, tables, and charts are in place; and (3) the final manuscript is satisfactory to the graduate committee and is ready for submission to the university library.
    [Show full text]
  • Openstack Vs. Ganeti
    OpenStack vs. Ganeti · Lance Albertson · Director, OSU Open Source Lab · http://osuosl.org · @ramereth Attribution-ShareAlike CC BY-SA ©2015 Session Overview OpenStack quick overview Ganeti overview Ganeti walk-through Comparing both About me Lance Albertson Director, OSU Open Source Lab (OSUOSL) Provide infrastructure hosting for FOSS projects Linux Foundation, Python Software Foundation, Drupal, etc Ops guy Ganeti user since 2009 OpenStack user since 2013 http://osuosl.org Virtualized Computing Resources Each organization has different needs Some are small and simple Others are more complex and larger Cost is also a major factor (both in licenses and staff maintenence) Typical Solutions VMWare VirtualBox OpenStack [insert favorite solution] OpenStack is all the hype Designed to scale and be an AWS replacement (almost) Everyone is investing in it Still maturing as a project Includes a wide-array of features, many of which most people don't need OpenStack Overview OpenStack Pros Standard Cloud API Fast VM deployment and tear down Very elastic computing needs Large community support Fast growing and new features constantly OpenStack Cons Extremely difficult to deploy and maintain Lots of moving parts Project is still maturing and unstable Fits a very specific use-case (Cloud) Requires more than one machine to effectively use So what about Ganeti? First, what is Ganeti? Cluster management tool for virtual compute resources IaaS solution that uses either KVM or Xen hypervisors Provides fast and simple recovery from hardware failures Primarily
    [Show full text]
  • Ganeti Scalable Virtualization with Ganeti by Lance Albertson About Me
    Ganeti Scalable Virtualization with Ganeti by Lance Albertson About Me OSU Open Source Lab Server hosting for Open Source projects Lead Systems Administrator / Architect Gentoo developer / contributor Jazz trumpet performer What I will cover Ganeti terminology, comparisons, & goals Cluster & virtual machine setup Dealing with outages Web Interface to Ganeti OSUOSL usage of ganeti Future roadmap State of Virtualization Citrix XenServer libvirt: oVirt, virt-manager Eucalyptus VMWare Open Stack Issues Overly complicated Multiple layers of software Lack of HA Storage integration Not always 100% open source Traditional virtualization cluster Ganeti cluster What is ganeti? Software to manage a cluster of virtual servers Project created and maintained by Google Combines virtualization & data replication Works with multiple hypervisors Automates storage management Automates OS deployment Ganeti software requirements Python various python modules DRBD LVM KVM, Xen, or LXC* Ganeti terminology Cluster - group of nodes Node - physical host Instance - virtual machine, aka guest Goals Reduce hardware cost Increase service availability Simplify VM cluster management Administration transparency Principles Not dependent on specific hardware Scales linearly Centralized administration N+1 redundancy Storage Options LVM LVM + DRBD (supports migration) File based (raw, qcow2, etc) Shared storage patch coming soon! Storage: LVM + DRBD Primary & secondary storage nodes Each instance disk synced separately Dedicated backend DRBD network Allows instance failover
    [Show full text]
  • Virtualization Explained: Definitions You Need to Know
    Virtualization Explained: Definitions You Need to Know By Stephen J. Bigelow with Edward L. Haletky Page | 1 Virtualization Explained: Definitions You Need to Know Table of contents A N AMD-V ... 3 Network virtualization … 11 C P Capacity planning ... 4 P2V … 12 Clone ... 5 Provisioning … 13 Clustered file system ... 6 S G Server consolidation … 14 Guests and hosts ... 7 Snapshot … 15 Storage virtualization … 16 I Intel VT ... 8 V Virtual hardware … 17 Virtual machine … 18 L Virtual machine monitoring … 19 Live migration ... 9 Virtual switch … 20 VMDK … 21 M Memory overcommit … 10 X Xen … 22 Page | 2 A AMD-V IT professionals should understand which AMD processors have AMD-V technology. But first, a quick history lesson: Traditional computing does not allow software to readily share hardware resources. Virtualization overcomes that limitation and allows two or more virtual machines (VMs) to share the computing resources of a single physical server, and the VM monitor is central to any virtualization deployment. It provides the abstraction that separates software from the underlying hardware and manipulates the virtual workloads so that each workload can share the common processor, memory, I/O and local storage. Early virtualization efforts relied on software emulation to replace hardware functionality. But software emulation is a slow and inefficient process. Many virtualization tasks were handled through software, so VM behavior and resource control were often poor, resulting in unacceptable VM performance on the server. By early 2005, processors still lacked the internal microcode to handle intensive virtualization tasks in hardware. Both Intel Corp. and Advanced Micro Dynamics Inc. (AMD) addressed this problem by creating a new set of processor extensions (similar to MMX, 3DNow! and others) that could offload the repetitive and inefficient work from the software.
    [Show full text]
  • Ganeti Web Manager Documentation Release 0.11.1
    Ganeti Web Manager Documentation Release 0.11.1 Oregon State University Open Source Lab Oct 22, 2019 CONTENTS 1 Getting Started 3 2 Deployment 17 3 Features 21 4 Usage 33 5 Contributing 35 6 Project Information 51 7 Deprecated 95 8 Indices and tables 105 Index 107 i ii Ganeti Web Manager Documentation, Release 0.11.1 Ganeti Web Manager is a Django based web frontend for managing Ganeti virtualization clusters. Since Ganeti only provides a command-line interface, Ganeti Web Manager’s goal is to provide a user friendly web interface to Ganeti via Ganeti’s Remote API. On top of Ganeti it provides a permission system for managing access to clusters and virtual machines, an in browser VNC console, and vm state and resource visualizations. You can see a few screenshots here. If you don’t already have a Ganeti cluster setup, these directions can help you get started. If you are looking for support, please contact us. If you are looking to deploy Ganeti Web Manager for the first time, check out our Installation guide. If you already have a Ganeti Web Manager instance running it might be time to upgrade. Finally, if you would like to report a bug or request a feature, please file an issue. Ganeti Web Manager is licensed under the GPLv2. It is currently developed and maintained by the Oregon State University Open Source Lab and a handful of volunteers. If you would like to get involved in development see our development guide. CONTENTS 1 Ganeti Web Manager Documentation, Release 0.11.1 2 CONTENTS CHAPTER ONE GETTING STARTED 1.1 Requirements 1.1.1 Operating system We officially support Ubuntu 11.10, Ubuntu 12.04 and CentOS 6.
    [Show full text]