CISCO Computing Research Symposium (C3RS)

November 5 & 6, 2008 San Jose, CA

Virtual Machine Management with OpenNebula in the RESERVOIR project

Ruben Santiago Montero

Distributed Systems Architecture Research Group Universidad Complutense de Madrid

1/23 Objectives

● Describe the goals of the RESERVOIR project and provide an overview of its architecture and design principles

● Discuss the challenges of managing VMs in a distributed environment and present the VM management model adopted by OpenNebula

● Study the operation of a Distributed VM Manager in clouds environments

● A simple use case: Elastic Management of Computing Clusters

2/23 RESERVOIR Virtual Machine Management with OpenNebula in the RESERVOIR project Who? • IBM (coordinator), Sun, SAP, ED, TID, UCM, UNIME, UMEA, UCL, USI, CETIC, Thales and OGF-Europe • 17-million and 3-year project partially funded by the European Commission (NESSI Strategic Project) What? • The Next Generation Infrastructure for Service Delivery, where resources and services can be transparently and dynamically managed, provisioned and relocated like utilities – virtually “without borders” How? • Integration of technologies with grid computing driven by new techniques for business service management

Virtualization - Aware Grid Grid - Aware Virtualization BSM e.g., VM as management unit + e.g., live migration across + e.g., policy- based manag. = SOI for metering and billing administrative domains of service -level agreement

3/23 RESERVOIR Virtual Machine Management with OpenNebula in the RESERVOIR project The Architecture, main Components and Interfaces

ServiceService Provider Provider Service Provider Monitor service and enforce SLA compliance by managing number and capacity of service components (VEE) Service Manager

VEE ManagerVEE VEE VEE

VEEVEE Host Host VEE VEEVEE Host Service Users

4/23 RESERVOIR Virtual Machine Management with OpenNebula in the RESERVOIR project The Architecture, main Components and Interfaces

Service Provider ServiceService Provider Provider

Organize the placement of VEEs to meet optimization Service Manager policies and constraints VMI

VEE Manager VMI VEE Manager VHI VEE Host VEE Host VEEVEE Host Host VEEVEE Host Host Support advance functionality for performance and realocation infrastructure providers optimization 5/23 Distributed Management of Virtual Machines Virtual Machine Management with OpenNebula in the RESERVOIR project Virtual Machine Model

● The service as a first-class management entity ● Service structure ● Service components run in VMs ● Inter-connection relationship ● Placement constraints ● The VM Manager is service agnostic ● However, it should provide infrastructure context

Web Server Tier Clients

Network

VM Database Tier 6/23 Distributed Management of Virtual Machines Virtual Machine Management with OpenNebula in the RESERVOIR project Distributed Virtual Machine Management System

● Provides a uniform view of the resource pool ● Resources organized in a cluster architecture ● Life-cycle management and monitoring of VM • The VM Management System integrates Image, Network and Virtualization technologies

Distributed VM Management System

The three pillars of a Distributed VM Manager Image Networking Management Virtualization 7/23 Distributed Management of Virtual Machines Virtual Machine Management with OpenNebula in the RESERVOIR project Image Management • VM Images Sources: • Master images in local repositories • Appliance supplier • Creation on the fly • Clones have to be contextualized (Context VBD)

Master Images Physical Host Virtual Machine VBD Image Repository Cloning

Virtual Context Device Installation Server 8/23 Distributed Management of Virtual Machines Virtual Machine Management with OpenNebula in the RESERVOIR project Network Management • VMs interconnected through one or more networks • Isolated, layer 2 LANs • Virtual networks are dynamically created • Medium size networks (x.x.x.x/20) with limited public IPs • TCP/IP services are not responsibility of the VM Manager

Sample Implementation

Virtual LAN Switch Virtual LAN (Red) Physical Network (Blue) Host Host Bridge Bridge tap0 tap2 tap0 tap1 tap1 VM VM VM VM VM

02:01:0A:00:01:01 02:01:0A:00:02:01 02:01:0A:00:01:02 02:01:0A:00:01:03 02:01:0A:00:02:02 10.0.1.1/24 10.0.2.1/24 10.0.1.2/24 10.0.1.3/24 10.0.2.2/24 9/23 Distributed Management of Virtual Machines Virtual Machine Management with OpenNebula in the RESERVOIR project Virtualization • Virtual Machine structure • One or more NICs attached to virtual or public networks • One or more system images (clones) • A context virtual block device • A required capacity (memory, CPU) • Use the host to create, monitor and control VMs

Distributed VM Management System

hypervisor hypervisor hypervisor hypervisor

Physical Resources 10/23 Distributed Management of Virtual Machines Virtual Machine Management with OpenNebula in the RESERVOIR project Life-cycle of a Virtual Machine (or a set of) • Resource Selection: Where do I place the VM? • Capacity planning (consolidation) • Placement requirements (e.g. affinity) • Placement Heuristics (e.g. Green IT, AR...) • Resource Preparation: What do I need for the VM? • Network preparation • Image cloning & contextualization • VM Creation: How do I start a VM? • Interface with different • VM Monitoring: How is the VM doing? • VM Migration: Is there a better resource for the VM? • Adjust placement to better fit to the infrastructure target • VM Termination: Do I need to save any VM image? 11/23 Distributed Management of Virtual Machines Virtual Machine Management with OpenNebula in the RESERVOIR project OpenNebula: Distributed VM Management System

• Open Source - Apache 2 Scheduler CLI • Flexible & Open Design OpenNebula API • Third-party components

XML-RPC • Easily adapted OpenNebula core • Easily extended Information Network • Different Hypervisors Host, Net Manager Manager & VM • Operation in Federated Pools Image VM Environments Manager Manager • Integral management of sqlite Access Drivers Virtual Services KVM EC2 libvirt Persistent Database

12/23 Distributed Management of Virtual Machines Virtual Machine Management with OpenNebula in the RESERVOIR project OpenNebula: Distributed VM Management System • Manage IP and MAC addresses Scheduler CLI libvirt• Manage VM life-cycle • Manage Image transfers OpenNebula API • Manage Physical Resources • Persistent back-end XML-RPC OpenNebula core Information Network Host, Net Manager Manager & VM Pools Image VM Manager Manager

sqlite Access Drivers XEN KVM EC2 libvirt Persistent Database

13/23 Distributed Management of Virtual Machines Virtual Machine Management with OpenNebula in the RESERVOIR project OpenNebula: Distributed VM Management System

Scheduler CLI • Imagelibvirt management systems • Physical resources OpenNebula API • Different hypervisors • Xen XML-RPC • KVM OpenNebula core • Libvirt Information Network• EC2 Host, Net Manager Manager• VMware (planned) & VM Pools Image VM Manager Manager

sqlite Access Drivers XEN KVM EC2 libvirt Persistent Database

14/23 Distributed Management of Virtual Machines Virtual Machine Management with OpenNebula in the RESERVOIR project OpenNebula: Distributed VM Management System

Scheduler CLI libvirt

OpenNebula API • VM placement policies & heuristics • Plug-inXML-RPC module •OpenNebula Available schedulers core • SimpleInformation match-makingNetwork scheduler Host, Net Manager Manager & VM Pools Image VM Manager Manager

sqlite Access Drivers XEN KVM EC2 libvirt Persistent Database

15/23 Distributed Management of Virtual Machines Virtual Machine Management with OpenNebula in the RESERVOIR project OpenNebula: Distributed VM Management System

Scheduler CLI libvirt

OpenNebula API

XML-RPC• Unix-like CLI OpenNebula core • Libvirt interface, integration with: Information Network• Virsh Host, Net Manager Manager• Virt-manager & VM • Other managers (e.g. Ovirt) Pools Image VM Manager Manager• ...

sqlite Access Drivers XEN KVM EC2 libvirt Persistent Database

16/23 Elastic Management of Computing Clusters Virtual Machine Management with OpenNebula in the RESERVOIR project

A New Infrastructure Layer...

• Separation of Resource Provisioning from Job Management • Seamless integration with the existing middleware stacks. • Completely transparent to the computing service and end users

LRMS (Job Management)

Virtual cluster nodes

Distributed VM Manager

VMM VMM VMM VMM Physical Resource

17/23 Elastic Management of Computing Clusters Virtual Machine Management with OpenNebula in the RESERVOIR project Cluster users HTTP clients

Virtual Network

Cluster Web Front-end Virtual Workernodes Server Service Layer OpenNebula

Physical Infrastructure Infrastructure Layer 18/23 Elastic Management of Computing Clusters Virtual Machine Management with OpenNebula in the RESERVOIR project A Service to Provide Hardware on Demand (IaaS)

• Cloud systems provide virtualized resources • Provide remote on-demand access to infrastructure (through VMs) • Main components of a Cloud architecture: • Front-end: Remote interface • Back-end: Local VM, image & network management

Simple Interfaces • VM Management Distr. Virtualizer • Image Management

Infrastructure Cloud Services • Commercial Cloud: Amazon EC2, GoGrid… • Scientific Cloud: (University of Chicago) 19/23 Elastic Management of Computing Clusters Virtual Machine Management with OpenNebula in the RESERVOIR project

Cluster users

Virtual Network

Service Cluster Virtual Layer Front-end Workernodes OpenNebula

Cloud Provider

Local Physical Infrastructure Infrastructure Layer 20/23 Federation of Cloud Sites Virtual Machine Management with OpenNebula in the RESERVOIR project • Consumer/Provider Relationships Clients or other Provider • Nimbus integration (WSRF + EC2) • RESERVOIR Cloud VMI • (planned)

Cloud Interface

Capacity • Federation of Cloud Providers ONE Core Manager • Image, Network and VM issues XEN / KVM Cloud Drivers Drivers OpenNebula Cloud Interface

Capacity ONE Core Manager Cloud Provider XEN / KVM Cloud Local Infrastructure Amazon EC2 Drivers Drivers OpenNebula

21/23 Virtual Machine Management with OpenNebula in RESERVOIR

THANK YOU FOR YOUR ATTENTION!!! More info, downloads, mailing lists at www.OpenNebula.org

OpenNebula is partially funded by the “RESERVOIR– Resources and Services Virtualization without Barriers” project EU grant agreement 215605

www.reservoir-fp7.eu/ Acknowledgements

• Javier Fontan • Tino Vazquez • Ignacio M. Llorente • Rafael Moreno

22/23 Elastic Management of Computing Clusters (DEMO - BACKUP)

Amazon EC2 Worker Worker Worker Node Node Connection OpenVPN Tunnels

Bridge

OpenVPN Server SGE Front-End Physical Host Bridge Local

Bridge Bridge Bridge

Worker Worker Worker Worker Worker Node Node Node Node Node

Physical Host Physical Host Physical Host 23/23