Compass – a Streamlined Openstack Deployment System
Total Page:16
File Type:pdf, Size:1020Kb
2013年11月7日星期四 Compass – A Streamlined OpenStack Deployment System Shuo Yang Principal Architect of Cloud Computing, US R&D Center Outline of This Talk 1 Scope of Problem for Compass 2 Compass Explained 3 DRY, Truly Open Deployment 1 Compass at a Glimpse Think Big, Start Small A General System to Deploy Distributed Systems, Extensibility as a Primary Design Goal Not Limited to OpenStack, but Streamlined Our OpenStack Deployment Like a Charm To Be Open Sourced – Apache 2.0 Soon 100% Python, 5000 Line of Python Code Successfully Deployed Several Dogfood Clusters Compass Wiki Page: https://wiki.openstack.org/wiki/Compass 2 Data Center as a Computer Open Cloud OS (OpenStack) Open Deployment (Compass) OpenStackLinux Quantumeth0, lo Nova/proc Cinder/dev Live Auto CD / Deploy GRUB NIC CPU Disk SwitchSwitchSwitch CPUCPUServer DiskStorage NIC CPU Disk (5020) (2285/1285) (N8000/N900) OpenStack Control Channel LILO/GRUB/LiveCD for OpenStack HW/SW Configuration Deployment Channel 3 Why We Are Doing This? Full HW Portfolio in Data Center No.1 as Storage Revenue Growth No. 2 as `x86 Server Revenue Growth Needless to Say, Networking Gears.. OpenStack Makes the Above a Full Global Excellent Telecom Cloud Solution Cloud Solution Provider of year 2012 4 OpenStack Deployment System Overview Crowbar TripleO (“under the cloud” mode) Pioneer effort, a Ruby web app, Chef based Attractive concept to OpenStack folks: configuration management deploy OpenStack from OpenStack Fuel DevStack A great web apps, Puppet based A great tool for simple OpenStack setup: configuration management developers use it 5 Streamlining the Deployment of a System Blueprint AI Rack and Stack Robot Zero Touch Installation Compass 6 Real Demo Video Clips Recording Our First Deployment 7 Outline of This Talk 1 Scope of Problem for Compass 2 Compass Explained 3 DRY, Truly Open Deployment 8 A Life of Deployment – How to Streamline All Controls Plus a lot of scripts: Boilerplate Target system description Chef / Puppet / … Host OS Host OS Host OS Cobbler / Razor / … IPMI Server 1 Server 2 Server n SNMP Switch 1 Switch 2 Switch k 9 Compass Philosophies – Programmability and Extensibility Programmability – To Be a Programmable Framework A Restful API service instead of a web app Extensibility – NOT To Be A Conglomerate Functional modules with plug-in architecture (e.g., 200 Loc to support HP Switch) Carefully designed boundary to mature tools like Cobbler, Chef and etc. Work WITH the above tools, not to replace any of them. 10 Compass Internals Rest rd Huawei UI Client 3 Party ISV UI Compass -- Open Source Restful API Server Message bus (RabbitMQ) DB Hardware Discover Package Deployment OS Provisioning Huawei Plugin microkernel Chef plugin Ansible plugin Cobbler plugin Razor plugin 11 Compass – A Programmable Framework Mapping Fundamental Concepts during Software Deployment into Restful Resources Machine – physical compute resource Switch – management network for machines Cluster – pool of machines managed together Host – machine in a cluster Adapter – plug-in module to discover/deploy/configure hosts 12 Resource Example: Machines GET /machines Lists all discovered machines GET /machines/id Lists a machine’s hardware details and status POST /machines/id/action Modify the hardware configuration for a machine Query parameters for filtering switch=ID marker=markerID address=string limit=int cluster=ID changes-size=dateTime status=string details=ID,switch,address,cluster,devices,status,modified Actions modify(hwConfig) 13 13 A Life of OpenStack Deployment -- as Restful Calls (1) Find all available machines: /machines?switch=0e8756dc (2) Select machines for my system to be deployed on to: {1, 2, 3, 4} (3) Select an adapter, which describes the result system: /?adapter=OpenStack (4) Find components for result systems {“controller”, “compute”} (5) Tell Compass a plan {1 -> “controller”; [2,3,4] -> “compute”} (or specify ‘?auto=true’) (6) Poll the deployment progress Time 14 NOT to Be Conglomerate – Plug-in Architecture Plug-in architecture for configuration management system Currently we use Chef as deployment management Engine Puppet or Ansible devopers, we want to work with you Plug-in architecture for Hardware vendors IPMI / SNMP Networking gear based bare metal discovery (topolology awareness) Currently we are working on RAID Controller for Huawei OCP community, what’s your thought 15 NOT to Be Conglomerate – Roles in the System Policy Databag (e.g., exclusive; default) OpenStack Cookbook Lib writer Chef-Server Restful Reqs Compass snippets kickstart files App writer Cobbler-server 16 Outline of This Talk 1 Scope of Problem for Compass 2 Compass Explained 3 DRY, Truly Open Deployment 17 Extensible in Use Cases OS ESXi ? Windows Server? CentOS Ubuntu Huawei OpenStack Hadoop Ceph? Distributed System HP OCP? Hardware 18 Extensible in Tool Chain OS Provisioning Tools Razor Cobbler Chef Puppet Ansible Salt Configuration Management Tools 19 To Summarize Open Sourced under Apache 2.0 General Purpose Deployment System (Ultimate Goal) Deploys OpenStack Cloud Infrastructure Smoothly (Dogfoods Done) Restful API: Allow Third-party UI Integration Open to Collaboration Contact us: [email protected] Santa Clara R&D Center, US 20 Copyright©2013 Huawei Technologies Co., Ltd. All Rights Reserved. The information in this document may contain predictive statements including, without limitation, statements regarding the future financial and operating results, future product portfolio, new technology, etc. There are a number of factors that could cause actual results and developments to differ materially from those expressed or implied in the predictive statements. Therefore, such information is provided for reference purpose only and constitutes neither an offer nor an acceptance. Huawei may change the information at any time without notice. Compass – Web Enabled Deployment Simple Web zzz… From baremetal actions (40mins) to OpenStack DC Internet Intranet Compass Administrators Administrators Server Farm 22 Compass Internal: Current Implementation Flask for Web Server Layer RabbitMQ as Message Broker Celery for Task Management Chef as Package Management Executor Relational DB as Transaction Records Cobbler as Host System Management 23 Compass API Design Convention Follow OpenStack API conventions Authentication Changes-since query parameter JSON or XML request/response data formats Version in URI or header Paginated collections Common representation for faults Allow multi-tenancy as an option ( tenantID not required) Use POST to create a new resource and return its ID Use PUT to replace, PATCH/POST for a partial update 24 .