Mysql Cluster
Total Page:16
File Type:pdf, Size:1020Kb
MySQL Cluster <Insert Picture Here> Craig Sylvester MySQL Sales Consultant Monday, August 15, 2011 1 Safe Harbor Statement The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decision. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle. © 2011 Oracle Corporation – Proprietary and Confidential Monday, August 15, 2011 2 Agenda • MySQL Cluster • MySQL Cluster Manager <Insert Picture • MySQL ClusterJ/ClusterJPA Here> © 2011 Oracle Corporation – Proprietary and Confidential Monday, August 15, 2011 3 MySQL Cluster Carrier Grade Database © 2011 Oracle Corporation – Proprietary and Confidential Monday, August 15, 2011 4 MySQL Cluster Architecture Clients MySQL Cluster Application Nodes MySQL MySQL Cluster Cluster Mgmt Mgmt MySQL Cluster Data Nodes © 2011 Oracle Corporation – Proprietary and Confidential Monday, August 15, 2011 5 MySQL Cluster - Components • SQL Access SQL Node • STD API’s (JDBC, ODBC, PERL, (MySQL) Python, LDAP, C, .NET, etc) NDB API • High Performance C/C++ API (Application) • Developer’s Guide Data Node • Data Storage (Memory/Disk) (NDB Storage Engine) • Automatic & User-Defined Partitioning Management • Administration & Configuration Node • Arbitration © 2011 Oracle Corporation – Proprietary and Confidential Monday, August 15, 2011 6 MySQL Cluster – Black Box Concept Clients (JDBC, ODBC, C, PHP, etc) High Perf API MySQL Server(s) BLACK BOX Mgm Node Data Nodes NDB Storage Engine © 2011 Oracle Corporation – Proprietary and Confidential Monday, August 15, 2011 7 MySQL Cluster – Distributed DB Two Data Nodes (Physical) authid (PK) fname lname country * 100% of database on each node 1 Albert Camus France Two Replicas (Logical) 2 Ernest Hemingway USA * One copy of each partition 3 Johan Goethe Germany One Node Groups 4 Junichiro Tanizaki Japan * One set of identical Data Nodes Node Group 1 Data Nodes NDB Storage Engine © 2011 Oracle Corporation – Proprietary and Confidential Monday, August 15, 2011 8 Auto data distribution and mirroring Four Data Nodes (Physical) authid (PK) fname lname country * Half of database on each node 1 Albert Camus France Two Replicas (Logical) 2 Ernest Hemingway USA * Two copies of each partition 3 Johan Goethe Germany Two Node Groups 4 Junichiro Tanizaki Japan * Two sets of identical Data Nodes Node Group 1 Node Group 2 authid (PK) fname lname country authid (PK) fname lname country 1 Albert Camus France 2 Ernest Hemingway USA 3 Johan Goethe Germany 4 Junichiro Tanizaki Japan © 2011 Oracle Corporation – Proprietary and Confidential Monday, August 15, 2011 9 Synchronous Data Replication Update: Two-phase UPDATE author commit SET country = ‘France’ WHERE authid = 1; protocol Node Group 1 Node Group 2 authid (PK) fname lname country authid (PK) fname lname country 1 Albert Camus France 2 Ernest Hemingway USA 3 Johan Goethe Germany 4 Junichiro Tanizaki Japan © 2011 Oracle Corporation – Proprietary and Confidential Monday, August 15, 2011 10 High Availability Query: Result: SELECT fname, lname Albert Camus FROM author Ernest Hemingway WHERE authid BETWEEN 1 AND 3; Johan Goethe Node Group 1 Node Group 2 X X authid (PK) fname lname country authid (PK) fname lname country 1 Albert Camus France 2 Ernest Hemingway USA 3 Johan Goethe Germany 4 Junichiro Tanizaki Japan © 2011 Oracle Corporation – Proprietary and Confidential Monday, August 15, 2011 11 Automatic Failure Detection - Heartbeats © 2011 Oracle Corporation – Proprietary and Confidential Monday, August 15, 2011 12 Automatic resynchronization of failed node © 2011 Oracle Corporation – Proprietary and Confidential Monday, August 15, 2011 13 Online Add Node Application Node Group 1 Dynamic scaling of a NG 1 NG 2 running Cluster – no interruption to service authid (PK) fname lname country authid (PK) fname lname country 1 Albert Camus France 1 Albert Camus France 3 Johan Goethe Germany 2 Ernest Hemingway USA authid (PK) fname lname country 3 Johan Goethe Germany 2 Ernest Hemingway USA 4 Junichiro Tanizaki Japan 4 Junichiro Tanizaki Japan http://www.mysql.com/why-mysql/white-papers/mysql_wp_cluster7_architecture.php © 2011 Oracle Corporation – Proprietary and Confidential Monday, August 15, 2011 14 Geo Redundancy • Asynchronous replication between Clusters • Supports Master-Master Replication with automatic conflict resolution Asynchronous MySQL Replication Cluster 1 Cluster 2 © 2011 Oracle Corporation – Proprietary and Confidential Monday, August 15, 2011 15 Replication Flexibility • Synchronous replication within a Cluster node group for HA • Bi-Direction asynchronous Cluster 1 Cluster 2 replication to remote Cluster for geographic redundancy • Asynchronous replication to non-Cluster databases for specialized activities such as report generation InnoDB MyISAM InnoDB • Mix and match replication types Synchronous replication Asynchronous replication © 2011 Oracle Corporation – Proprietary and Confidential Monday, August 15, 2011 16 Backup & Restore • Built-in Online Backup capability – Active transactions continue unaffected!!! – Non-locking/blocking • Backs up each data node – Stored on local disk on each data node • Restore – Restore one data node at-a-time • Point-In-Time Recovery – Need to enable Binary-Logging on MySQL Server • Logging used for Async Replication – Procedure • Restore cluster backup • Replay changes up-to a point using binary logs © 2011 Oracle Corporation – Proprietary and Confidential Monday, August 15, 2011 17 Cluster Use Cases • Subscriber Data Management Systems – Five 9’s or greater availability – Scalability: Cluster supports up to 255 total nodes (48 data nodes) – Multiple data access API’s (LDAP, SQL, C/C++, JDBC/ODBC, PERL, Python, PHP, Ruby) • Web Session Management – Manage persistent sessions requiring fast lookup and HA • AAA Session Management • In-Memory Cache / Persistent Cache – High speed access with guaranteed latencies – Disk-based storage • Applications requiring Automatic Sharding (or Partitioning) of data • Applications requiring High Insert/Update rates – In-memory nature enables high update rates – HLR/HSS © 2011 Oracle Corporation – Proprietary and Confidential Monday, August 15, 2011 18 ndbinfo: Online Usage Metrics mysql> use ndbinfo mysql> show tables; +-------------------+ • ndbinfo database presents real- | Tables_in_ndbinfo | +-------------------+ time metric data in the form of tables | blocks | • Examples include: | config_params | – Resource usage (memory, buffers) | counters | – Event counters (such as number of READ | logbuffers | operations since last restart) | logspaces | – Data node status and connection status | memoryusage | | nodes | | resources | | transporters | +-------------------+ http://www.mysql.com/why-mysql/white-papers/mysql_wp_cluster7_architecture.php © 2011 Oracle Corporation – Proprietary and Confidential Monday, August 15, 2011 19 ndbinfo examples 1. Check memory usage/availability mysql> select * from ndbinfo.memoryusage; +---------+--------------+--------+------------+-----------+-------------+ | node_id | memory_type | used | used_pages | total | total_pages | +---------+--------------+--------+------------+-----------+-------------+ | 3 | Data memory | 917504 | 28 | 104857600 | 3200 | | 3 | Index memory | 221184 | 27 | 11010048 | 1344 | | 4 | Data memory | 917504 | 28 | 104857600 | 3200 | | 4 | Index memory | 221184 | 27 | 11010048 | 1344 | +---------+--------------+--------+------------+-----------+-------------+ 2. Check how many table scans performed on each data node since the last restart mysql> select node_id as 'data node', val as 'Table Scans' from ndbinfo.counters where counter_name='TABLE_SCANS'; +-----------+-------------+ | data node | Table Scans | +-----------+-------------+ | 3 | 3 | | 4 | 4 | +-----------+-------------+ http://www.mysql.com/why-mysql/white-papers/mysql_wp_cluster7_architecture.php © 2011 Oracle Corporation – Proprietary and Confidential Monday, August 15, 2011 20 Enterprise Monitor Integration © 2011 Oracle Corporation – Proprietary and Confidential Monday, August 15, 2011 21 MySQL Cluster Manager Reducing database TCO and creating a more agile, highly available IT environment Automated Management Monitoring & High Availability Recovery Operation http://www.mysql.com/products/database/cluster/mcm/ © 2011 Oracle Corporation – Proprietary and Confidential Monday, August 15, 2011 22 MySQL Cluster Manager 1.1 Features http://www.mysql.com/why-mysql/white-papers/mysql_wp_cluster_manager.php © 2011 Oracle Corporation – Proprietary and Confidential Monday, August 15, 2011 23 Cluster Manager example configuration mysql client • One agent per physical host – no central manager 7. mysqld 8. mysqld each agent is responsible for its 1. ndb_mgmd 2. ndb_mgmd – local nodes agent agent – responsible for managing all 192.168.0.10 192.168.0.11 nodes in the cluster • Management responsibilities 3. ndbd 4. ndbd – Starting, stopping & restarting nodes 5. ndbd 6. ndbd – Configuration changes agent agent – Upgrades Host & Node status reporting 192.168.0.12 192.168.0.13 – – Recovering failed nodes n. mysqld MySQL Server (ID=n) n. ndb_mgmd Management Node (ID=n) n. ndbd Data Node (ID=n) agent MySQL Cluster Manager agent © 2011 Oracle Corporation – Proprietary and Confidential Monday, August 15, 2011 24 Creating & Starting a Cluster