Mysql Cluster for Real Time, HA Services

MySQL Cluster for Real Time, HA Services

Mario Beck ([email protected]) MySQL Sales Consultant MySQL Cluster

• CARRIER GRADE Shared Nothing in Memory Parallel Database DATABASE • ACID Compliant Relational Database

• HIGHLY Five Nines Availability (99.999%) AVAILABLE • Sub Second Failover, Self-Healing

• LINEARLY Incrementally grow out with Application Demands SCALABLE • Linearly Scalable with Distribution Awareness

• REAL TIME High Load, Real Time Performance PERFORMANCE • Predictable Low-Latency, Bounded Access Time

• OPEN Open Source, Multiple Data Access DEVELOPMENT • High Performance APIs (C++,Java), SQL, LDAP MySQL Cluster – Users & Applications HA, Transactional Services: Web & Telecoms • Telecoms • Subscriber Databases (HLR/HSS) • Service Delivery Platforms • VoIP, IPTV & VoD • Mobile Content Delivery • On-Line app stores and portals • IP Management • Payment Gateways • Web • User profile management • Session stores • eCommerce • On-Line Gaming

• Application Servers http://www.mysql.com/customers/cluster/ Comparison of Commercial vs. GPL Licensing

GPL Commercial • Free to use, open source • Commercial (paid) license • Great to quickly start • Carrier Grade Edition (CGE) prototyping of new • Removes obligations of GPL applications and services to share application code • Responsibility to share/GPL • Access to support contracts code you extend • Access to MySQL Cluster Manager

• Standard SQL Interface SQL Node • Scale-out for Performance (MySQL) • Enables Replication

• Data Storage (Memory/Disk) • Automatic & User-Defined Partitioning Data Node • Local & Global Checkpoints (NDB Storage Engine) • Scale-out or scale-up for Capacity & Redundancy • Scale dynamically with on-line add node

• High Performance NDB API • C, C++ & Java, LDAP, HTTP API (Application) • Developer’s Guide

Management • Administration and Configuration • Arbitration Node • Use Two for Redundancy

Clients

MySQL Cluster Application Nodes

MySQL MySQL Cluster Cluster Mgmt Mgmt

MySQL Cluster Data Nodes

Synchronous Replication within Node Group Transaction Handling

1. 5. 2. F1 F2

4. F2 F1

Transaction Coordinator

Out of the Box Scalability: Data Partitioning

• Data partitioned across Data Nodes • Rows are divided into partitions, based on a hash of all or part of the primary key • Each Data Node holds primary fragment for 1 partition • Also stores secondary fragment of another partition • Records larger than 8KB stored as BLOBs MySQL Cluster vs MySQL MEMORY: 30x Higher Throughput / 1/3rd the Latency on a single node

• Table level locking inhibits MEMORY scalability beyond a single client connection • Check-pointing & logging enabled, MySQL Cluster still delivers durability • 4 socket server, 64GB RAM, running Linux MySQL Cluster 7.1 Micro-benchmarks

• Scale-out on commodity hardware • NDB API Performance 6.82 M Queries per second! – 16 servers, dual-6-core CPUs @2.93 GHz – 2 Data Nodes per server • Interim results from 2 days testing – watch this space: mikaelronstrom.blogspot.com Node Failure Detection & Self-Healing Recovery Online Add Node (1) – add node group

authid (PK)‏ fname lname country

1 Albert Camus France

2 Ernest Hemingway USA

Application 3 Johann Goethe Germany

4 Junichiro Tanizaki Japan

Node Group New Node Group

authid (PK)‏ fname lname country

1 Albert Camus France

2 Ernest Hemingway USA

3 Johann Goethe Germany

4 Junichiro Tanizaki Japan

http://www.mysql.com/why-mysql/white-papers/mysql_wp_cluster7_architecture.php

authid (PK)‏ fname lname country

1 Albert Camus France

2 Ernest Hemingway USA

Application 3 Johann Goethe Germany

4 Junichiro Tanizaki Japan

Node Group New Node Group

authid (PK)‏ fname lname country authid (PK)‏ fname lname country 1 Albert Camus France authid (PK)‏ fname lname country 1 Albert Camus France 2 Ernest Hemingway USA 2 Ernest Hemingway USA 2 Ernest Hemingway USA 3 Johann Goethe Germany 4 Junichiro Tanizaki Japan 3 Johan Goethe Germany 4 Junichiro Tanizaki Japan 4 Junichiro Tanizaki Japan No extra space needed on existing nodes!

http://www.mysql.com/why-mysql/white-papers/mysql_wp_cluster7_architecture.php

authid (PK)‏ fname lname country

1 Albert Camus France

2 Ernest Hemingway USA

Application 3 Johan Goethe Germany

4 Junichiro Tanizaki Japan

Node Group New Node Group

authid (PK)‏ fname lname country

1 Albert Camus France authid (PK)‏ fname lname country

2 Ernest Hemingway USA 2 Ernest Hemingway USA

3 Johann Goethe Germany 4 Junichiro Tanizaki Japan

4 Junichiro Tanizaki Japan

http://www.mysql.com/why-mysql/white-papers/mysql_wp_cluster7_architecture.php

authid (PK)‏ fname lname country

1 Albert Camus France Dynamic scaling of a 2 Ernest Hemingway USA running Cluster – no Application 3 Johan Goethe Germany interruption to service 4 Junichiro Tanizaki Japan

Node Group 1 Node Group 2

authid (PK)‏ fname lname country authid (PK)‏ fname lname country

1 Albert Camus France 2 Ernest Hemingway USA

3 Johann Goethe Germany 4 Junichiro Tanizaki Japan

http://www.mysql.com/why-mysql/white-papers/mysql_wp_cluster7_architecture.php

binlog async replication relaylog

Master Slave

log_bin=1 SQL thread

binlog async replication relaylog *always Master row-based Slave binlog injector

Data Nodes Data Nodes

MySQL Cluster MySQL Cluster

failover replication channel

binlog binlog async replication relaylog *always row-based Master Master Slave

binlog binlog injector injector

Data Nodes Data Nodes

MySQL Cluster MySQL Cluster

• Synchronous replication within a Cluster node group for HA • Bi-Direction asynchronous replication to remote Cluster Cluster 1 Cluster 2 for geographic redundancy • Asynchronous replication to non-Cluster databases for specialised activities such as report generation • Mix and match replication types

MyISAM MyISAM InnoDB

Synchronous replication Asynchronous replication

• New Domain Object Model Persistence API (ClusterJ) : • Java API • High performance, low latency • Feature rich • JPA interface built upon this new Java

Network layer: • Java Persistence API compliant • Implemented as an OpenJPA plugin • Uses ClusterJ where possible, reverts to JDBC for some operations • Higher performance than JDBC • More natural for most Java designers • Easier Cluster adoption for web Data Nodes applications Performance MySQL Cluster Manager 1.1 Features

Automated Management Monitoring HA Operations

• Cluster-Wide • Status Monitoring & • Disk Persistence Management Recovery • Configuration • Process Management Consistency • On-Line Upgrade • HA Agent Operation • On-Line Reconfig • On-Line Add Node Creating & Starting a Cluster

mysql client 1. Define the site:

Mysql> create site --hosts=192.168.0.10,192.168.0.11, -> 192.168.0.12,192.168.0.13 mysite; 2. Expand the MySQL Cluster tar-ball(s)

7. mysqld 8. mysqld from mysql.com to known directory

1. ndb_mgmd 2. ndb_mgmd 3. Define the package(s):

Mysql> add package --basedir=/usr/local/mysql_6_3_26 6.3; agent agent Mysql> add package --basedir=/usr/local/mysql_7_0_7 7.0; Note that the basedir should match the 192.168.0.10 192.168.0.11 directory used in Step 2.

3. ndbd 4. ndbd 4. Create the Cluster Mysql> create cluster --package=6.3 5. ndbd 6. ndbd -> [email protected],[email protected], -> [email protected],[email protected], [email protected], -> [email protected],[email protected],[email protected] agent agent -> mycluster; This is where you define what 192.168.0.12 192.168.0.13 nodes/processes make up the Cluster and where they should run 5. Start the Cluster:

Mysql> start cluster mycluster; Online Upgrade with MySQL Cluster Manager

mysql client

• Upgrade from MySQL Cluster 6.3.26 to 7. mysqld 8. mysqld 7.0.7:

1. ndb_mgmd 2. ndb_mgmd mysql> upgrade cluster --package=7.0 mycluster; agent agent • Automatically upgrades each node and

192.168.0.10 192.168.0.11 restarts the process – in the correct order to avoid any loss of service

3. ndbd 4. ndbd • Without MySQL Cluster Manager, the

5. ndbd 6. ndbd administrator must stop each process in turn, start the process with the new version agent agent and wait for the node to restart before moving onto the next one 192.168.0.12 192.168.0.13