Mysql Cluster Wann Brauche Ich Das?

MySQL Cluster Wann brauche ich das?

Mario Beck Principal Sales Consultant [email protected] The presentation is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.

2.1BN USERS 8X DATA GROWTH IN 5 YRS 750M USERS 70+ NEW DOMAINS EVERY 60 SECONDS 20M APPS PER DAY 40% DATA GROWTH PER YEAR 600 NEW VIDEOS EVERY 60 SECONDS

$1TR BY 2014 100K TWEETS PER MINUTE $700BN IN 2011

5.3BN MOBILE SUBS IN 2010 (78% PENETRATION) 13K iPHONE APPS 370K CALL MINUTES EVERY 60 SECONDS DOWNLOADED PER MINUTE

Driving new Database Requirements

EXTREME WRITE SCALABILITY REAL TIME USER EXPERIENCE

ROCK SOLID RELIABILITY ELIMNATE BARRIERS TO ENTRY

No Trade-Offs

Transactional Integrity

EXTREME WRITE SCALABILITYComplex REALQueries TIME USER EXPERIENCE Standards & Skillsets

ROCK SOLID RELIABILITY ELIMNATE BARRIERS TO ENTRY

No Trade-Offs: Cellular Network

HLR / HSS Location Updates

AuC, Call Routing, Billing

Pre & Post Paid

• Massive volumes of write traffic • <3ms database response • Downtime & lost transactions = lost $

Billing, AuC, VLR MySQL Cluster in Action: http://bit.ly/oRI5tF

No Trade-Offs: eCommerce

• Integrated Service Provider platform • eCommerce • Payment processing • Fulfillment • Supports 1k+ manufacturers & 18k retail partners • Requirements • Scaling, On-Demand • HA: failures & on-line upgrades • High batch & real time loads • Low TCO: capex and opex http://mysql.com/customers/view/?id=1080

No Trade-Offs: Flight Control

• US Navy aircraft carriers • Consolidated flight operations management system • Maintenance records • Fuel loads • Weather conditions • Flight deck plans • Requirements • No Single Points of Failure • Complete redundancy • Small footprint, harsh environment • 4 x MySQL Cluster nodes, Linux and Windows

MySQL User Conference Session: http://bit.ly/ogeid3 ACID Compliant Relational Database • SQL & NoSQL interfaces

Write-Scalable & Real-Time • Distributed, multi-master, auto-sharding, optimized in-memory structures & indices

99.999% Availability • Shared-nothing, integrated clustering & sub-second recovery, local & geographic replication, on-line operations

Low Barriers to Entry • Open-source, elastic, multiple APIs, management tools, commodity hardware

MySQL Cluster Architecture

JPA REST Application Nodes

Node Group 1 Node Group 2

Cluster F1 F2 Cluster

Mgmt F3 F4 Mgmt

Node 1 Node

Node 3 Node

F3 F4

F1 F2 Node 4 Node Node 2 Node Data Nodes

MySQL Cluster - Extreme Resilience

JPA REST Application Nodes

Node Group 1 Node Group 2

Cluster F1 F2 Cluster

Mgmt F3 F4 Mgmt

Node 1 Node

Node 3 Node

F3 F4

F1 F2 Node 4 Node Node 2 Node Data Nodes

Handling Scheduled Maintenance On-Line Operations

• Scale the cluster (add nodes on-line) • Repartition tables • Upgrade / patch servers & OS • Upgrade / patch MySQL Cluster • Back-Up • Evolve the schema on-line, in real-time

Auto-Sharding with High Availability

• Transparent to the application • No need for application-layer sharding logic • Partitioning based on hashing all or part of the primary key • Each node stores primary fragment for 1 partition and back-up fragment for another • Transparency maintained during failover, upgrades and scale-out • No need to limit application to single-shard transactions

Best Practice : Primary Keys

• ALWAYS DEFINE A PRIMARY KEY ON THE TABLE! • A hidden PRIMARY KEY is added if no PK is specified. BUT.. • .. NOT recommended • The hidden primary key is for example not replicated (between Clusters)!! • There are problems in this area, so avoid the problems! • So always, at least have id BIGINT AUTO_INCREMENT PRIMARY KEY • Even if you don't “need” it for you applications

Scale-Out on Commodity Hardware

4.3M QPS

• 8 Intel servers • Dual-6-core CPUs @2.93 GHz • 24GB RAM • 2 Data Nodes per server • flexAsync benchmark • 16 parallel threads, each issuing 256 simultaneous transactions • Read / Write 100KB attribute • Access via NDB API

Enabling Next Generation Web Services

• 70x Higher Complex Query Performance: Adaptive Query Localization • Native memcached API • MySQL 5.5 Server Integration

Enhancing Cross Data Center Scalability • Multi-Site Clustering • Simplified Active / Active Replication

Simplifying Provisioning & Administration • Consolidated Privileges

Adaptive Query Localization Scaling Distributed Joins 70x More Performance

• Perform Complex Queries mysqld across Shards • JOINs pushed down to data nodes Data Nodes A • Executed in parallel Q • Returns single result set to MySQL L • Opens Up New Use-Cases • Real-time analytics • Recommendations engines

mysqld • Analyze click-streams

Data Nodes DON’T COMPROMISE FUNCTIONALITY TO SCALE-OUT !!

The existence, content and timing of future releases described here is included for information only and may be changed at Oracles discretion. October 3rd, 2011 MySQL Cluster 7.2 AQL Test Query Web-Based Content Management System

MySQL Server

Data Data Node1 Node2

mysql> ANALYZE TABLE ;

7.1 = 87.23s

The existence, content and timing of future releases described here is included for information only and may be changed at Oracles discretion. October 3rd, 2011 Memcached Key-Value API

• Persistent, Scalable, HA Back-End to memcached • No application changes: re- uses standard memcached clients & libraries • Consolidate Caching & Database Tiers • Eliminate cache invalidation • Simpler re-use of data across services • Improved service levels New • Flexible Deployment NoSQL • Schema or Schema-less Access storage

The existence, content and timing of future releases described here is included for information only and may be changed at Oracles discretion. October 3rd, 2011 API Flexibility: SQL & NoSQL Combined

Clients

JDBC / ODBC PHP / PERL Native memcached HTTP/REST Python / Ruby

NDB API

Data Nodes

Mix • SQL: Complex, relational queries & • HTTP / memcached: Key-Value web services Match • Java: Enterprise Apps • NDB API: Real-time services

The existence, content and timing of future releases described here is included for information only and may be changed at Oracles discretion. October 3rd, 2011 MySQL 5.5 Server Integration • Exploit Latest Server / Storage Engine Developments within a Single Application • InnoDB: Foreign Keys, XA Transactions, Large Rows • MySQL Cluster: HA, High Write Rates, Real- Time • Reduces Complexity, Simplifies DevOps • MySQL 5.5 • 3x higher performance • Improved partitioning, diagnostics, availability, etc.

• Replicating complete clusters across data centers • DR & data locality • No passive resources Geographic Replication • Simplified Active / Active Replication • Eliminates requirement for application & schema changes • Transaction-level rollback

The existence, content and timing of future releases described here is included for information only and may be changed at Oracles discretion. October 3rd, 2011 Enhanced Cross Data Center Scalability Multi-Site Clustering • Split data nodes & node groups across data centers Data Node 1 Data Node 2 • Synchronous replication Synchronous and auto-failover between Replication sites Data Node 3 Data Node 4 • Improved heartbeating to handle network partitions • Extends Disaster Recovery Options • Active/Active with no need for conflict handling

The existence, content and timing of future releases described here is included for information only and may be changed at Oracles discretion. October 3rd, 2011 Simplified Provisioning & Maintenance User Privilege Consolidation

The existence, content and timing of future releases described here is included for information only and may be changed at Oracles discretion. October 3rd, 2011 MySQL Cluster Manager

Reducing TCO and creating a more agile, highly available database environment

Automated Management

High Monitoring & Availability Recovery Operation

Example: Initiating upgrade from MySQL Cluster 6.3 to 7.1

Before MySQL Cluster Manager With MySQL Cluster Manager • 1 x preliminary check of cluster state upgrade cluster --package=7.1 mycluster; • 8 x ssh commands per server • 8 x per-process stop commands • 4 x scp of configuration files (2 x mgmd & 2 x Total: 1 Command - mysqld) Unattended Operation • 8 x per-process start commands • 8 x checks for started and re-joined processes • Results • 8 x process completion verifications • 1 x verify completion of the whole cluster. • Reduces the overhead and complexity of managing database clusters • Excludes manual editing of each configuration file. • Reduces the risk of downtime resulting from administrator error Total: 46 commands - • Automates best practices in database 2.5 hours of attended operation cluster management

Download & Configure Start processes Extract • Cluster-wide • Management Nodes • edelivery.oracle.com “config.ini” • Data Nodes • www.mysql.com • Per-mysqld “my.cnf” • MySQL Servers • dev.mysql.com

• Up & running in 10-15 minutes using Quick Start guides from http://dev.mysql.com/downloads/cluster/ • Versions for Linux, Windows & Solaris

1. Download MCM from edelivery.oracle.com: • Package including Cluster 2. Unzip 3. Run agent, define, create & start Cluster! $> bin\mcmd –bootstrap MySQL Cluster Manager 1.1.2 started Connect to MySQL Cluster Manager by running "D:\Andrew\Documents\MySQL\mcm\bin\mcm" -a NOVA:1862 Configuring default cluster 'mycluster'... Starting default cluster 'mycluster'... Cluster 'mycluster' started successfully ndb_mgmd NOVA:1186 ndbd NOVA ndbd NOVA mysqld NOVA:3306 mysqld NOVA:3307 ndbapi * Connect to the database by running "D:\Andrew\Documents\MySQL\mcm\cluster\bin\mysql" -h NOVA -P 3306 -u root 4. Connect to Cluster & start using database To bootstrap with Cluster 7.2 replace contents of mcm/cluster directory

MySQL Servers MySQL Servers MySQL Master

Node Group 1 Node Group 2 Node Group 1 Node Group 2

F1 F2 F1 F2 Slave N

F3 F4 F3 F4

Node 3 Node

F1 F2 F1 F2 F3 F4

F4 4 Node

F3 4 Node Node 4 Node Node 4 Node Slave 6 Slave 7 Slave 8 Slave 9 Slave 10

MySQL Cluster Data Nodes MySQL Cluster Data Nodes

Slave 1 Slave 2 Slave 3 Slave 4 Slave 5 • 4 x Data Nodes: 6k Analytics page hits per second MySQL Master XOR • Each page hit generating 8 – 12 database operations

Distributed Slave 1 Slave 2 Slave 3 Storage

Whitepaper: http://www.mysql.com/why-mysql/white-papers/mysql_wp_high-availability_webrefarchs.php Getting Started

Learn More Scaling Web Databases Guide

www.mysql.com/cluster/

Evaluate MySQL Cluster 7.2 Bootstrap a Cluster!

Download, No Download Today Obligation http://dev.mysql.com/do https://edelivery.oracl wnloads/cluster/ e.com/

Scale Web Services with Carrier-Grade Availability

Don’t Trade Functionality for Scale

Evaluate the Development Milestone Today

Scale the Cluster Capacity & Performance

Application Nodes

Data Node 1 Data Node2

Cluster F1 F2 Mgmt F2 F1

Data Nodes

Scale the Cluster Capacity & Performance

Application Nodes

Node Group 1 Node Group 2

Cluster F1 F2 Cluster

Mgmt F3 F4 Mgmt Node 1 Node

3 Node

F3 F4

F1 F2 Node 4 Node Node 2 Node Data Nodes

Node Failure Detection & Self-Healing Recovery

On-Line Scaling & Maintenance 1. New node group added 2. Data is re-partitioned 3. Redundant data is deleted 4. Distribution is switched to share load with new node group

• Can also update schema on- line • Upgrade hardware & software with no downtime • Perform back-ups on-line

Only MySQL Can….. blend the agility & innovation of the web….

….with the trust & capability of the network.