OpenStack Architecture for the Enterprise

Keith Tobin Greg Jacobs Cloud Architect Network Architect

Speaker Profiles

Keith Tobin Greg Jacobs

• Cloud Architect • Dell Network Architect • Located in the Dell Cloud Center of • 20 years working as a network Excellence in Dublin, Ireland architecture and solutions specialist • Over two decades of , • Network evangelist and packet head platform and cloud development

• Organizer for OpenStack Ireland

• Loves all things OpenStack and Cloud

2 OpenStack Architecture for the Enterprise Services

Design Goals • Design an OpenStack architecture to run both enterprise and cloud applications • Meet enterprise performance expectations for VM’s, networking and storage • Zero downtime on the control layer • No data loss in the event of a single node failure • Must be durable, reliable and highly available • Must automatically recover after a fault situation has been fixed • Minimize management tasks

3 OpenStack Architecture for the Enterprise Services

Solution Stack

Dell Private Managed Cloud Tested Qualified Solution Solution

Enstratius Dell Multi-Cloud Manager OpenStack OpenStack Titanium CEPH on R720 Storage Deployment Crowbar

SUSE Linux SUSE Linux

Networking Physical Layer SonicWall R620 R720 C6100 Servers

Security SecureWorks

Cloud Services Managed

4 OpenStack Architecture for the Enterprise Services Deployment With Crowbar

5 OpenStack Architecture for the Enterprise Services Deploying OpenStack with Crowbar Is Easy

• Crowbar boots and installs from ISO on a management node (server) • Provides user with web UI for configuration and operations management • Discovers all potential targets (servers) automatically • Performs hardware configuration – BIOS configuration – Performs updates to BIOS and firmware • Installs OS on targets • Deploys OpenStack from a set of configuration modules called barclamps

6 OpenStack Architecture for the Enterprise Services Crowbar Deployment

OpenStack Database Controller Barclamp Node 01 Identity Controller Barclamp DHCP Node 02 Image Controller Barclamp Node 03 Block storage Crowbar PXE Barclamp Compute Networking Node 01 Compute Barclamp Server Nova Node 02 Barclamp Compute Horizon Node 03 Barclamp

7 OpenStack Architecture for the Enterprise Services Crowbar 2 Status

8 OpenStack Architecture for the Enterprise Services Load Balancer

9 OpenStack Architecture for the Enterprise Services Active/Passive (Normal)

Incoming API Requests

Controller Controller VIP 01 02 KeepaliveD KeepaliveD Load Load Balancer 01 Balancer 02 (Inactive)(Active) (Passive)(Active)

Keystone Keystone Keystone etc… etc… etc…

10 OpenStack Architecture for the Enterprise Services Active/Passive (Fault)

Incoming API Requests

Controller Controller VIP 01 02 KeepaliveD KeepaliveD Load Load Balancer 01 Balancer 02 (Inactive)(Active) (Passive)(Active)

Keystone Keystone Keystone etc… etc… etc…

11 OpenStack Architecture for the Enterprise Services Single HAProxy Bottleneck

Controller iVIP eVIP Controller 01 02 HAProxy HAProxy (Active LB) Inactive LB MySQL MySQL

RabbitMQ RabbitMQ

Horizon Horizon

Glance-API Glance-API Glance- Glance- registry registry Nova- Nova- scheduler scheduler Nova-API Nova-API

Keystone-API Keystone-API

Cinder-API Cinder-API

Neutron-API Neutron-API

Neutron L2 Neutron L2

Neutron L3 Neutron L3 Neutron Neutron DHCP DHCP

12 OpenStack Architecture for the Enterprise Services Dynamic Load Balancing Incoming API Traffic

Network Network Switch Switch

OSPF ECMP

Controller Controller Controller 01 Quagga 02 Quagga 03 Quagga

HAProxy HAProxy HAProxy

Keystone Keystone Keystone etc… etc… etc…

13 OpenStack Architecture for the Enterprise Services Neutron

14 OpenStack Architecture for the Enterprise Services Neutron (Single Networking Node)

Controller 01 Controller 02 Compute 01 Compute 03

Keystone Keystone Nova Nova Glance Glance Compute 02 Compute 04 MySQL MySQL etc… etc… Nova Nova

Network 01

Neutron API L3 Agent OVS Agent DHCP Agent

15 OpenStack Architecture for the Enterprise Services Neutron (Separate Networking Nodes)

Controller 01 Controller 02 Controller 03 Compute 01 Compute 02

HAProxy HAProxy HAProxy L3 Agent L3 Agent

Quagga Quagga Quagga DHCP Agent DHCP Agent

OVS Agent OVS Agent Neutron API Neutron API Neutron API Neutron Neutron Neutron Scheduler Scheduler Scheduler Other Other EctOther…. OpenStack OpenStack OpenStack Compute 03 Compute 04 Services Services Services L3 Agent L3 Agent

DHCP Agent DHCP Agent OVS Agent OVS Agent

Internet

16 OpenStack Architecture for the Enterprise Services RabbitMQ

17 OpenStack Architecture for the Enterprise Services RabbitMQ Cluster (With Mirrored Queues)

Physical View Logical View

Controller 01 Controller 02 Controller 03 End Point

HAProxy HAProxy HAProxy

Quagga Quagga Quagga

RabbitMQ RabbitMQ RabbitMQ Dynamic Load Balancer Other Other Other OpenStack OpenStack OpenStackEct…. Services Services Services

RabbitMQ 01 RabbitMQ 02 RabbitMQ 03

18 OpenStack Architecture for the Enterprise Services MySQL Database

19 OpenStack Architecture for the Enterprise Services MySQL Single Server (Normal Operation)

Application

DB Server

Data

20 Openstack Architecture for the Enterprise Services MySQL Single Server (Failed)

Application

DB Server 1 (Active)

Data

21 Openstack Architecture for the Enterprise Services MySQL Single Server (Failed)

• Disadvantage Application • Single point of failure

DB Server 1 (Active)

Data

22 Openstack Architecture for the Enterprise Services MySQL Active/Passive Failover

23 Openstack Architecture for the Enterprise Services MySQLMySQL Active/PassiveActive/Passive FailoverFailover (Failed(Normal Node) Operation) Disadvantages • Inactive Server Application • Limits scale • Reduces the potential workload processing ability of the solution by half DB Server 1 DBDB Server Server 2 2 • Async replication has (Active)(Failed) (Passive)(Active) potential for data

inconsistencies across Async Replication server nodes in fault DataData Data situations

24 Openstack Architecture for the Enterprise Services MySQL With DRDB

25 Openstack Architecture for the Enterprise Services MySQL With DRDB (Normal Operation)

Disadvantages Application • Inactive server • Limits scale • Reduces the potential workload processing ability of the solution DB Server 1 DB Server 2 by half (Failed) Server Not Available ! Event

Data Data

Event Confirmed 26 Openstack Architecture for the Enterprise Services MySQL Multi- Master

27 Openstack Architecture for the Enterprise Services MySQL Multi-Master Cluster Application Application 01 02

Load balancer

DB DB Server 1 Server 2

Data Data

DB Server 3

Data

28 Openstack Architecture for the Enterprise Services MySQL Multi-Master Cluster (Failed Node) Application Application 01 02

Load balancer

DB DB Server 1 Server 2

Data Data

DB Server 3

Data

29 Openstack Architecture for the Enterprise Services MySQL Async Replication

• No guarantee that data will be consistent across all nodes at any point in time Server 1 Server N

Data Data

30 Openstack Architecture for the Enterprise Services MySQL Single Thread replication

Processor

C1 C2 C3 C4

C5 C6 C7 C8

31 Openstack Architecture for the Enterprise Services MySQL Multi-Master Cluster Application Application 01 02 Disadvantages

• At any point in time there Load balancer is no guarantee that data is in a consistent state within the cluster • One thread allocated for DB DB replication tasks Server 1 Server 2 Data Data

DB Server 3 Data

32 Openstack Architecture for the Enterprise Services Percona MySQL Cluster

33 Openstack Architecture for the Enterprise Services Percona MySQL Cluster

Capability Synchronous Replication Data Consistency Parallel Applying on All Nodes Automatic Node Provisioning Multi-Master

34 Openstack Architecture for the Enterprise Services Synchronous Replication

Application 01

Capability Controller Controller Synchronous Replication DB 1 DB 2 Data Consistency Event Parallel Applying on All Nodes Data Data Automatic Node Provisioning Event Multi-Master Confirmed

35 Openstack Architecture for the Enterprise Services Data Consistency

Controller Controller Server DB 1 DB 2 DB 3

Capability

Synchronous Replication Data Data Data Data Consistency

Parallel Applying on All Nodes Automatic Node At any point in time data is always Provisioning consistent across all nodes in the cluster Multi-Master

36 Openstack Architecture for the Enterprise Services Parallel Applying on All Nodes

Processor

C1 C2 C3 C4

C5 C6 C7 C8

Capability

Synchronous Replication

Data Consistency

Parallel Applying on All Nodes Automatic Node Provisioning Multi-Master

37 Openstack Architecture for the Enterprise Services Automatic Node Provisioning

New Node Controller DB 1 Controller DB 4 Join Cluster Data

Data Join Cluster Capability Cluster Synchronous Replication Controller Controller DB 2 DB 3 Data Consistency

Parallel Applying on All Data Data Nodes Automatic Node Provisioning Multi-Master

38 Openstack Architecture for the Enterprise Services Multi-Master Cluster Application Application 01 02

Dynamic Load balancer

Capability

Controller Controller Synchronous Replication DB 1 DB 2 Data Consistency Data Data Parallel Applying on All Nodes Controller DB 3 Automatic Node Provisioning

Data Multi-Master

39 Openstack Architecture for the Enterprise Services Block Storage

40 OpenStack Architecture for the Enterprise Services Block Storage

CEPH on R720

41 OpenStack Architecture for the Enterprise Services Logical Architecture

42 OpenStack Architecture for the Enterprise Services Thank you

Chat with Keith and Greg @ the Dell booth

Keith Tobin Greg Jacobs Cloud Architect Network Architect

43 OpenStack Architecture for the Enterprise Services