Xen Summit November 2007 Ganeti a Xen based high availability cluster Roman Marxer Copyright by Google Inc 1 Content · Ideas behind Ganeti · Ganeti overview · Instance failover · Administration · Usage in Google · Code · Roadmap Copyright by Google Inc 2 Combine high availability and virtualization High availability Virtualization (Xen) virt. system 1 system 1 system 2 + virt. system 2 system virt. system 1 = virt. system 2 virt. system 3 virt. system 1© system 1 system 2 system 3 Ganeti Copyright by Google Inc 3 Xen based multi-node high availability cluster instance = domU node = dom0 virtual system failover ganeti cluster instance 3 instance 1 instance 2 instance 4 node 1 node 2 node 3 node 4 master: tests and controls Copyright by Google Inc 4 Ganeti overview Overview · Xen cluster manager · High availability cluster (future) · Software used . Virtualization: Xen . Disk management: LVM / DRBD . Language and RPC: Python / Twisted · Hardware used . commodity systems Copyright by Google Inc 5 Content · Ideas behind Ganeti · Ganeti overview · Instance failover · Administration · Usage in Google · Code · Roadmap Copyright by Google Inc 6 Ganeti disk overview Xen dom0 = node Xen domU = instance instance 1 Ganeti master node 1 node 2 node 3 ¼ more Ganeti cluster Copyright by Google Inc 7 Ganeti instance failover (1/3) Xen dom0 = node Xen domU = instance instance 1 Ganeti master node 1 node 2 node 3 ¼ more Ganeti cluster Copyright by Google Inc 8 Ganeti instance failover (2/3) Xen dom0 = node Xen domU = instance primary failover instance 1 Ganeti master node 1 node 2 node 3 ¼ more Ganeti cluster Copyright by Google Inc 9 Ganeti instance failover (3/3) Xen dom0 = node Xen domU = instance instance 1 Ganeti master node 1 node 2 node 3 ¼ more Ganeti cluster secondary failover Copyright by Google Inc 10 Ganeti administration # gnt-cluster init # gnt-node add node2 # gnt-node add node3 # gnt-node list Node DTotal DFree MTotal MNode MFree Pinst Sinst node1.example.com 858240 858240 4095 511 3584 0 0 node2.example.com 858240 858240 4095 511 3584 0 0 node3.example.com 858240 858240 4095 511 3584 0 0 # gnt-instance add -t drbd -n node1:node2 -o debian-etch instance1 # gnt-instance add -t drbd -n node2:node1 -o debian-etch instance2 # gnt-instance list Instance OS Primary_node Autostart Status Memory instance1.example.com etch node1.example.com yes running 512 instance2.example.com etch node2.example.com yes running 512 # gnt-instance failover instance1 Copyright by Google Inc 11 Content · Ideas behind Ganeti · Ganeti overview · Instance failover · Administration · Usage in Google · Code · Roadmap Copyright by Google Inc 12 Ganeti usage in Google · 20-node Ganeti cluster · 64-bit node OS · 80 virtual instances · Used for internal systems · Not used for google.com · Best for non-resource intensive systems Copyright by Google Inc 13 Ganeti code · Developed at Google · August 2007: open source and released v.1.2b · License: GPL v2 · Code location: http://code.google.com/p/ganeti/ · [email protected] · December 2007: release v.1.2 Copyright by Google Inc 14 Roadmap Current: Future · Xen manager · HVM integration (Windows support) · Multi-node · Automatic failover · Mirroring over node · Transparent failover · Linux only · Manager GUI / instance allocator location A 20-node Ganeti cluster 20-node Ganeti cluster manager and allocator location B 30-node Ganeti cluster Copyright by Google Inc 15 Q & A Copyright by Google Inc 16.