Mysql Cluster Wann Brauche Ich Das?
Total Page:16
File Type:pdf, Size:1020Kb
MySQL Cluster Wann brauche ich das? Mario Beck Principal Sales Consultant [email protected] The presentation is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle. 2.1BN USERS 8X DATA GROWTH IN 5 YRS 750M USERS 70+ NEW DOMAINS EVERY 60 SECONDS 20M APPS PER DAY 40% DATA GROWTH PER YEAR 600 NEW VIDEOS EVERY 60 SECONDS $1TR BY 2014 100K TWEETS PER MINUTE $700BN IN 2011 5.3BN MOBILE SUBS IN 2010 (78% PENETRATION) 13K iPHONE APPS 370K CALL MINUTES EVERY 60 SECONDS DOWNLOADED PER MINUTE Driving new Database Requirements EXTREME WRITE SCALABILITY REAL TIME USER EXPERIENCE ROCK SOLID RELIABILITY ELIMNATE BARRIERS TO ENTRY No Trade-Offs Transactional Integrity EXTREME WRITE SCALABILITYComplex REALQueries TIME USER EXPERIENCE Standards & Skillsets ROCK SOLID RELIABILITY ELIMNATE BARRIERS TO ENTRY No Trade-Offs: Cellular Network HLR / HSS Location Updates AuC, Call Routing, Billing Pre & Post Paid • Massive volumes of write traffic • <3ms database response • Downtime & lost transactions = lost $ Billing, AuC, VLR MySQL Cluster in Action: http://bit.ly/oRI5tF No Trade-Offs: eCommerce • Integrated Service Provider platform • eCommerce • Payment processing • Fulfillment • Supports 1k+ manufacturers & 18k retail partners • Requirements • Scaling, On-Demand • HA: failures & on-line upgrades • High batch & real time loads • Low TCO: capex and opex http://mysql.com/customers/view/?id=1080 No Trade-Offs: Flight Control • US Navy aircraft carriers • Consolidated flight operations management system • Maintenance records • Fuel loads • Weather conditions • Flight deck plans • Requirements • No Single Points of Failure • Complete redundancy • Small footprint, harsh environment • 4 x MySQL Cluster nodes, Linux and Windows MySQL User Conference Session: http://bit.ly/ogeid3 ACID Compliant Relational Database • SQL & NoSQL interfaces Write-Scalable & Real-Time • Distributed, multi-master, auto-sharding, optimized in-memory structures & indices 99.999% Availability • Shared-nothing, integrated clustering & sub-second recovery, local & geographic replication, on-line operations Low Barriers to Entry • Open-source, elastic, multiple APIs, management tools, commodity hardware MySQL Cluster Architecture JPA REST Application Nodes Node Group 1 Node Group 2 Cluster F1 F2 Cluster Mgmt F3 F4 Mgmt Node 1 Node Node 3 Node F3 F4 F1 F2 Node 4 Node Node 2 Node Data Nodes MySQL Cluster - Extreme Resilience JPA REST Application Nodes Node Group 1 Node Group 2 Cluster F1 F2 Cluster Mgmt F3 F4 Mgmt Node 1 Node Node 3 Node F3 F4 F1 F2 Node 4 Node Node 2 Node Data Nodes Handling Scheduled Maintenance On-Line Operations • Scale the cluster (add nodes on-line) • Repartition tables • Upgrade / patch servers & OS • Upgrade / patch MySQL Cluster • Back-Up • Evolve the schema on-line, in real-time Auto-Sharding with High Availability • Transparent to the application • No need for application-layer sharding logic • Partitioning based on hashing all or part of the primary key • Each node stores primary fragment for 1 partition and back-up fragment for another • Transparency maintained during failover, upgrades and scale-out • No need to limit application to single-shard transactions Best Practice : Primary Keys • ALWAYS DEFINE A PRIMARY KEY ON THE TABLE! • A hidden PRIMARY KEY is added if no PK is specified. BUT.. • .. NOT recommended • The hidden primary key is for example not replicated (between Clusters)!! • There are problems in this area, so avoid the problems! • So always, at least have id BIGINT AUTO_INCREMENT PRIMARY KEY • Even if you don't “need” it for you applications Scale-Out on Commodity Hardware 4.3M QPS • 8 Intel servers • Dual-6-core CPUs @2.93 GHz • 24GB RAM • 2 Data Nodes per server • flexAsync benchmark • 16 parallel threads, each issuing 256 simultaneous transactions • Read / Write 100KB attribute • Access via NDB API Copyright 2011 Oracle Corporation 18 MySQL Cluster 7.2.1 DMR Enabling Next Generation Web Services • 70x Higher Complex Query Performance: Adaptive Query Localization • Native memcached API • MySQL 5.5 Server Integration Enhancing Cross Data Center Scalability • Multi-Site Clustering • Simplified Active / Active Replication Simplifying Provisioning & Administration • Consolidated Privileges Adaptive Query Localization Scaling Distributed Joins 70x More Performance • Perform Complex Queries mysqld across Shards • JOINs pushed down to data nodes Data Nodes A • Executed in parallel Q • Returns single result set to MySQL L • Opens Up New Use-Cases • Real-time analytics • Recommendations engines mysqld • Analyze click-streams Data Nodes DON’T COMPROMISE FUNCTIONALITY TO SCALE-OUT !! The existence, content and timing of future releases described here is included for information only and may be changed at Oracles discretion. October 3rd, 2011 MySQL Cluster 7.2 AQL Test Query Web-Based Content Management System MySQL Server Data Data Node1 Node2 Copyright 2011 Oracle Corporation 21 MySQL Cluster 7.2 70x Speedup! (DM) 7.2 = 1.26s 7.2 analyzed • 70x Speedup for real = 65.3s customer JOIN across 11 tables • Must Analyze tables for best results mysql> ANALYZE TABLE <tab-name>; 7.1 = 87.23s The existence, content and timing of future releases described here is included for information only and may be changed at Oracles discretion. October 3rd, 2011 Memcached Key-Value API • Persistent, Scalable, HA Back-End to memcached • No application changes: re- uses standard memcached clients & libraries • Consolidate Caching & Database Tiers • Eliminate cache invalidation • Simpler re-use of data across services • Improved service levels New • Flexible Deployment NoSQL • Schema or Schema-less Access storage The existence, content and timing of future releases described here is included for information only and may be changed at Oracles discretion. October 3rd, 2011 API Flexibility: SQL & NoSQL Combined Clients JDBC / ODBC PHP / PERL Native memcached HTTP/REST Python / Ruby NDB API Data Nodes Mix • SQL: Complex, relational queries & • HTTP / memcached: Key-Value web services Match • Java: Enterprise Apps • NDB API: Real-time services The existence, content and timing of future releases described here is included for information only and may be changed at Oracles discretion. October 3rd, 2011 MySQL 5.5 Server Integration • Exploit Latest Server / Storage Engine Developments within a Single Application • InnoDB: Foreign Keys, XA Transactions, Large Rows • MySQL Cluster: HA, High Write Rates, Real- Time • Reduces Complexity, Simplifies DevOps • MySQL 5.5 • 3x higher performance • Improved partitioning, diagnostics, availability, etc. The existence, content and timing of future releases described here is included for information only and may be changed at Oracles discretion. October 3rd, 2011 Enhanced Cross Data Center Scalability Geographic Replication • Replicating complete clusters across data centers • DR & data locality • No passive resources Geographic Replication • Simplified Active / Active Replication • Eliminates requirement for application & schema changes • Transaction-level rollback The existence, content and timing of future releases described here is included for information only and may be changed at Oracles discretion. October 3rd, 2011 Enhanced Cross Data Center Scalability Multi-Site Clustering • Split data nodes & node groups across data centers Data Node 1 Data Node 2 • Synchronous replication Synchronous and auto-failover between Replication sites Data Node 3 Data Node 4 • Improved heartbeating to handle network partitions • Extends Disaster Recovery Options • Active/Active with no need for conflict handling The existence, content and timing of future releases described here is included for information only and may be changed at Oracles discretion. October 3rd, 2011 Simplified Provisioning & Maintenance User Privilege Consolidation The existence, content and timing of future releases described here is included for information only and may be changed at Oracles discretion. October 3rd, 2011 MySQL Cluster Manager Reducing TCO and creating a more agile, highly available database environment Automated Management High Monitoring & Availability Recovery Operation Copyright 2011 Oracle Corporation 33 How Does MySQL Cluster Manager Help? Example: Initiating upgrade from MySQL Cluster 6.3 to 7.1 Before MySQL Cluster Manager With MySQL Cluster Manager • 1 x preliminary check of cluster state upgrade cluster --package=7.1 mycluster; • 8 x ssh commands per server • 8 x per-process stop commands • 4 x scp of configuration files (2 x mgmd & 2 x Total: 1 Command - mysqld) Unattended Operation • 8 x per-process start commands • 8 x checks for started and re-joined processes • Results • 8 x process completion verifications • 1 x verify completion of the whole cluster. • Reduces the overhead and complexity of managing database clusters • Excludes manual editing of each configuration file. • Reduces the risk of downtime resulting from administrator error Total: 46 commands - • Automates best practices in database 2.5 hours of attended operation cluster management Copyright 2011 Oracle Corporation 34 Creating & running your first Cluster - the “manual” way (without MCM) Download & Configure Start