Building Service Platforms using OpenStack and CEPH
Malte Dreyer Humboldt-Universität zu Berlin
Founded in 1810
Faculties: 9
Students: 35 000 (5 000 International)
Degree Programs: 185 Professorships: 419(483) Academic Staff: 2 000 Non-Academic: 1 500
3 Campuses, ~180 Buildings
Computer and Media Service Broad Range of Services
ADS Hardware Service Terminalserver Service AFS Identity Management Repositories Accounts Image Editing Training Anti Virus Software Mail Service Trouble Ticket Systems Audio / Video Equipment Mailing Lists User Certificates Backup-Service Map Service User Support Calendar Service Media Repository VPN Campus Management SW Moodle Video Conferences Chat Service Printing Service Video Service Compute Clusters Public Computers WLAN Computer Networks Publication Repository Web Content Management Computer Training Rooms Resource Management SW Web Files Database Service Scanning Service Web Hosting Managed Desktops Shibboleth Website Management File Exchange Software Service Wiki and Blog Farms File Service Statistics SW Service Windows Domains Graphics Service Storage Area Network ePublishing Survey Service
Projects in Close Cooperation with Scholars
Customizations and feature developments Media Repository Farm
Joint projects and software developments Laudatio Corpus Repository
"Products of research are not just publications.”
NSF senior policy specialist Beth Strausser.
Biographical Sketch(es), has been revised to rename the “Publications” section to “Products” and amend terminology and instructions accordingly.
13 January 2013: "National Science Foundation’s Merit Review Criteria: Review and Revisions” Chapter II.C.2.f(i)(c),
1. Recommended open access to scholarly papers of publicly funded research
2. Recommended open access to all digital outputs of publicly funded research
3. Mandated open access to scholarly papers of publicly funded research
4. Mandated open access to all digital outputs of publicly funded research
5. Enforced, mandated open access to scholarly papers of publicly funded research
6. Enforced, mandated open access to all digital outputs of publicly funded research
From a Data Center Point of View
80 individual projects 80 setups 80 databases 80 configs
Custom software Fedora Repository CakePHP Elastic Search PID-Server (Handle) jQuery / Bootstrap Apache Xerces +RedMine for Tickets Cloud as e-Infrastructure Supporting Research
SaaS
Applications Applications and Data PaaS
Platform Platform OS & Basic Services OS & Basic Services IaaS
Infrastructure Infrastructure Infrastructure Servers–Storage- Servers–Storage- Servers–Storage- Network Network Network XaaS/EaaS: Software-led Infrastructure (Software Defined Data Center)
Backup Database Identity Monitoring Network Security Storage …………………. as a Service
Media Repository Seafile Repository xyz …………………. as a Service Flexible e-Infrastructure…
keep services alive reduce technology beyond project lifetime stack for project staff
concentrate on what easy testing you are really environments interested in
We Took a Look At… (Being aware that these are different in type)
• amazon web services • Apache Cloudstack • Docker / LXC • Eucalyptus • Microsoft Azure • OpenNebula • OpenStack • VMware
And still use Proxmox for VM management for many services … and Chose OpenStack still under heavy development
Macro Scripting „Manage VMs like a Linux Process“
Pet or Cattle
Images: Sheep: Public Domain; Toy Dog: By Toyloverz (Own work) [CC BY-SA 3.0 (http://creativecommons.org/licenses/by-sa/3.0)], via Wikimedia Commons OpenStack Component Architecture
Horizon Heat Ceilometer
Dashboard Orchestration Metering
Trove Sahara
Database Mgmt Hadoop
Glance Keystone Cinder
Image Mgmt Identity Mgmt Block Storage
Nova Neutron Swift
Compute Network Object Storage • Preise [TEUR]
2016 2017 2018
WDM (Licht) 200 200
Linecards 10 200 Gbps x20 MLX (Router) 175 175 Chassis 8x Linecards
(26) 10Gbps 286 286
x8 Summe 661 661 200
CEPH Server SAN
MB/s Aggregated 160 MB/s1200 140 1000 Read from disk 120 800 100 Read from cache 80 600 Aggregated read 60 400 from disk 40 Aggregated read 200 20 from cache 0 0 1 2 3 4 5 6 7 8 Number of instances
Figure to the left: By Inktank, CC-BY-SA http://ceph.com/docs/master/architecture/ Seafile: Deployment in OpenStack Using Ansible and Heat
ANSIBLE Playbook HEAT my_port01: # update packet cache type: OS::Neutron::Port - hosts: all properties: tasks: admin_state_up: true - apt: name: Port01 network_id: { get_resource: my_first_network } - name: apply common configuration to all nodes my_server07: hosts: all type: OS::Nova::Server roles: properties: - common name: seaf07 key_name: { get_param: keypair_name } - name: deploy and configure memcached server flavor: { get_param: machine_flavor } hosts: memcached image: { get_param: image_id } roles: networks: [{ "port": { get_resource: my_port07 }, …}] - memcached user_data: | #!/bin/bash -v - name: deploy MySQL and configure the databases …. hosts: db cinder_volume07: roles: type: OS::Cinder::Volume - db properties: size: { get_param: volume_size } - name: deploy seafile and apply configuration hosts: nodes volume_attachment07: roles: type: OS::Cinder::VolumeAttachment - nodes properties: volume_id: { get_resource: cinder_volume07 } - name: deploy loadbalancer instance_uuid: { get_resource: my_server07 } hosts: loadbalancer mountpoint: /dev/vdb roles: - loadbalancer tags: - loadbalancer Organizational Responsibilities
Horizon Heat Ceilometer
Dashboard Orchestration Metering
Trove Sahara
Database Mgmt Hadoop
Glance Keystone Cinder
Image Mgmt Identity Mgmt Block Storage
Nova Neutron Swift
Compute Network Object Storage Organizational Responsibilities /1
Horizon Heat Ceilometer
Dashboard Orchestration Metering
Trove Sahara
Database Mgmt Hadoop
Glance Keystone Cinder
Image Mgmt Identity Mgmt Block Storage
Nova Neutron Swift
Compute Network Object Storage Organizational Responsibilities /2
Horizon Heat Ceilometer
Dashboard Orchestration Metering
Trove Sahara
Database Mgmt Hadoop
Glance Keystone Cinder
Image Mgmt Identity Mgmt Block Storage
Nova Neutron Swift
Compute Network Object Storage Organizational Responsibilities /3
Horizon Heat Ceilometer
Dashboard Orchestration Metering
Trove Sahara
Database Mgmt Hadoop
Glance Keystone Cinder
Image Mgmt Identity Mgmt Block Storage
Nova Neutron Swift
Compute Network Object Storage Organizational Responsibilities /4
Horizon Heat Ceilometer
Dashboard Orchestration Metering
Trove Sahara
Database Mgmt Hadoop
Glance Keystone Cinder
Image Mgmt Identity Mgmt Block Storage
Nova Neutron Swift
Compute Network Object Storage Organizational Responsibilities /5
Horizon Heat Ceilometer
Dashboard Orchestration Metering
Trove Sahara
Database Mgmt Hadoop
Glance Keystone Cinder
Image Mgmt Identity Mgmt Block Storage
Nova Neutron Swift
Compute Network Object Storage Organizational Responsibilities /6
Horizon Heat Ceilometer
Dashboard Orchestration Metering
Trove Sahara
Database Mgmt Hadoop
Glance Keystone Cinder
Image Mgmt Identity Mgmt Block Storage
Nova Neutron Swift
Compute Network Object Storage Organizational Responsibilities: New Distributed Competency
Horizon Heat Ceilometer
Dashboard Orchestration Metering
Trove Sahara
Database Mgmt Hadoop
Glance Keystone Cinder
Image Mgmt Identity Mgmt Block Storage
Nova Neutron Swift
Compute Network Object Storage Organizational Challenges and Conclusions
Build new competencies • OpenStack is complex and additional technologies are needed • An advanced training program
Changing system administration styles • Develop vs. configure manually step by step
Re-Use what others built already (and share again) • Read scripts from other people
Be patient when OpenStack is still immature, or Neutron fails again • So far it develops/improves very fast • Dogfooding builds trust
..…Thanks…..
Questions?