Mysql High Available by Design Carsten Thalheimer
Total Page:16
File Type:pdf, Size:1020Kb
MySQL High available by design Carsten Thalheimer Sales Consultant MySQL GBU EMEA ([email protected]) Safe Harbor Statement The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle. Oracle ©|Nov 2016 Slide 2 Copyright 2015, Oracle 2 and/or its affiliates. All rights reserved Agenda • MySQL today • MySQL 5.7 ( Fall 2015) Recap • MySQL 5.7 and MySQL Cluster 7.5 ( Fall 2016) X-Protocol // Group Replication & InnoDB Cluster • MySQL 8.0 - What’s next? MySQL 8.0 Oracle ©|Nov 2016 Slide 3 2.4BN USERS 8x DATA GROWTH IN 5 YRS >1B USERS 70+ NEW DOMAINS EVERY 60 SECONDS 40% DATA GROWTH PER YEAR 600 NEW VIDEOS EVERY 60 SECONDS $1TR BY 2014 5700 TWEETS PER SECOND $700BN IN 2011 6.7BN MOBILE SUBS IN 2012 85% HANDSETS SHIPPED WITH A BROWSER Oracle ©|Nov 2016 Slide 4 Logarithmic plotting ! Source: http://db-engines.com/en/ranking Oracle ©|Nov 2016 Slide 5 MySQL – The view from the moon • MySQL < 5.0/5.1 [pre Oracle] - MyISAM default, InnoDB optional Before 2010 - Very often defaults in Linux distributions • MySQL 5.5 more ‘SQLish’ - First release under Oracle Dec 2010 - InnoDB default (MVCC, ACID), MyISAM optional - MS Windows optimized • MySQL 5.6 Replication revamp - InnoDB+, GTID, NoSQL, Performance SchemaIn Feb 2013 - Used by some of the biggest Websites - Introduction of MySQL Fabric • MySQL 5.7 New ways - InnoDB++, GIS Support, QRP, Instrinc Tables Oct 2015 - MultiThreaded Sls, MultiSourceRpl, Group Rpl - MySQL Fabric, SemiSync+ • MySQL 8.0 ??? - Check Labs! Two Years Cycle? Oracle ©|Nov 2016 Slide 6 MySQL 5.7 Oracle ©|Nov 2016 Slide 7 MySQL 5.7 is GA (August 2015) Performance & Scalability Manageability 3 X Faster than MySQL 5.6 Native JSON Support Enhanced InnoDB: faster online & bulk Improved Security: safer initialization, load operations setup & management Replication Improvements (incl. multi- Performance Schema Improvements source, multi-threaded slaves...) New Optimizer Cost Model: greater user MySQL SYS Schema control & better query performance MySQL 5.7 http://www.thecompletelistoffeatures.com Oracle ©|Nov 2016 Slide 8 Core New JSON features in MySQL 5.7 • Native JSON data type Native internal binary format for efficient processing & storage. Up to 10x faster than storing as text • Built-in JSON functions Allowing you to efficiently store, search, update, and manipulate Documents • Indexing of Documents using Generated Columns Automatically uses the best “functional” index available for even faster results • JSON Comparator & New inline syntax for easy SQL integration Allows for easy integration of Document data within your SQL queries Oracle ©|Nov 2016 Slide 9 JSON Functions • 5.7 supports functions to CREATE, SEARCH, MODIFY and RETURN JSON values: JSON_ARRAY_APPEND() JSON_INSERT() JSON_REPLACE() JSON_ARRAY_INSERT() JSON_KEYS() JSON_SEARCH() JSON_ARRAY() JSON_LENGTH() JSON_SET() JSON_CONTAINS_PATH() JSON_MERGE() JSON_TYPE() JSON_CONTAINS() JSON_OBJECT() JSON_UNQUOTE() JSON_DEPTH() JSON_QUOTE() JSON_VALID() JSON_EXTRACT() JSON_REMOVE() https://dev.mysql.com/doc/refman/5.7/en/json-functions.html Oracle ©|Nov 2016 Slide 10 Agenda • MySQL today • MySQL 5.7 ( Fall 2015) Recap • MySQL 5.7 and MySQL Cluster 7.5 ( Fall 2016) X-Protocol // Group Replication & InnoDB Cluster • MySQL 8.0 - What’s next? MySQL 8.0 Oracle ©|Nov 2016 Slide 11 MySQL 5.7 [2016] - DocumentStore Oracle ©|Nov 2016 Slide 12 Architecture SQL API CRUD API MySQL Std Protocol X Protocol Shell Std Protocol X Protocol X and Std 3306 33060 Protocols MySQL Plugins Core X Protocol Plugin Memcached Plugin Oracle ©|Nov 2016 Slide 13 MySQL as a Document Store – a FULL Stack Store, Retrieve, Search and Managing JSON documents • Native JSON Datatype, Indexes on JSON Documents • Native Collections with Key Value Semantics • Interactive Shell “MySQL Shell” – Javascript, Python, SQL modes • Connectors include NoSQL CRUD APIs – Java, NodeJS, NET, C++/C, PHP, Python – Method Chaining and Pipelining – Supports Combined Document and Relational Oracle ©|Nov 2016 Slide 14 MySQL Group Replication (RC) Oracle ©|Nov 2016 Slide 15 Group Replication Plugin, what Is It? and it is the foundation for the MySQL InnoDB Cluster! “Multi-master update anywhere replication plugin for MySQL with built-in conflict detection and resolution, automatic distributed recovery, and group membership.” App Servers with • Group Replication library MySQL Router – Implementation of Replicated Database State Machine theory • MySQL GCS is based on Paxos (variant of Mencius) – Provides virtually synchronous replication for MySQL/InnoDB 5.7+ – Supported on all MySQL platforms MySQL Group Replication • Linux, Windows, Solaris, OSX, FreeBSD Oracle ©|Nov 2016 Slide 16 16 Oracle/MySQL Group Replication – Distributed as part of the official MySQL binaries • from www.mysql.com, edelivery.oracle.com or support.oracle.com – Starting MySQL 5.7.17 and MySQL 8.0.1 (next MySQL) it is available in all MySQL versions by default. – Is/will be integrated with other products of the MySQL Productsuite • MySQL Workbench, MySQL Backup and MySQL Monitor – Easy to monitor (Performance Schema tables for group and node status/stat). – Predictable roadmap and backuped by Oracle MySQL Premier Support Oracle ©|Nov 2016 Slide 17 1 7 The Theory Behind It • Implementation is based on “Replicated Database State Machines” – Group Communication primitives resemble general properties of Databases – Distributed systems meet Databases: Pedone, Guerraoui, and Schiper paper • Deferred update replication: before committing locally we certify it on all nodes – In order to implement it one needs Atomic Broadcast – the change occurs everywhere or nowhere – This is necessary to ensure data consistency across all nodes • Membership Service – Group Membership: it allows one to know that at a given moment in time all the members that are participating in the protocol are associated with the same logical identifier (view id) – View Synchrony: ensure that messages from past views are all delivered before a new view is installed Oracle ©|Nov 2016 Slide 18 1 8 MySQL Group Replication “Multi-master update anywhere replication plugin for MySQL with built-in conflict detection and resolution, automatic distributed recovery, and group membership.” App Servers with MySQL Router • Active/Active update anywhere MySQL Shell • Distributed fault tolerance Setup, Manage, Orchestrate • Automatic server fail-over • Automatic reconfiguration MySQL Group Replication • Automatic conflict detection & resolution Oracle ©|Nov 2016 Slide 19 1 9 MySQL Group Replication • What does the MySQL Group Replication plugin do for the user? – Removes the need for handling server fail-over. – Provides fault tolerance. – Enables update everywhere setups. – Automates group reconfiguration (handling of crashes, failures, re-connects). – Provides a highly available replicated database. – Automatic distributed coordination (protects against split-brain and message loss). – Less admin overhead, means more fun time! Oracle ©|Nov 2016 Slide 20 2 0 MySQL Group Replication: Architecture Node Types R: Traffic routers/proxies: mysqlrouter, haproxy, sqlproxy, ... M: mysqld nodes participating in Group Replication Oracle ©|Nov 2016 Slide 21 2 1 Multi-Master Update Anywhere! • Any two transactions on different servers can write to the same row • Conflicts will automatically be detected and handled – First committer wins rule UPDATE t1 SET a=3 WHERE a=1 UPDATE t1 SET a=4 WHERE a=2 OK OK M M M M M UPDATE t1 SET a=3 WHERE a=1 UPDATE t1 SET a=2 WHERE a=1 OK M M M M M Oracle ©|Nov 2016 Slide 22 2 2 MySQL Router ... • Motivation – Primary Master Mode by default (Single Master) App – (Mulit-Master possible) – Do not need to know which server is the master – Transparent fail-over Router M M M • Generic and versatile framework: Oracle ©|Nov 2016 Slide 23 MySQL Router. MySQL InnoDB Cluster (DMR) Oracle ©|Nov 2016 Slide 24 2 4 MySQL InnoDB Cluster: Vision “A single product — MySQL — with high availability and scaling features baked in; providing an integrated end-to-end solution that is easy to use.” Oracle ©|Nov 2016 Slide 25 2 5 MySQL InnoDB Cluster: Goals • One Product: MySQL • Flexible and Modern – All components created together – SQL and NoSQL together – Tested together – Protocol Buffers – Packaged together – Developer friendly • Easy to Use • Support Read/Write Scale Out – One client: MySQL Shell – Sharded clusters – Easy packaging – Federated system of N replica sets – Integrated orchestration – Each replica set manages a shard – Homogenous servers Oracle ©|Nov 2016 Slide 26 2 6 MySQL InnoDB Cluster: Architecture – Step 2 RC Application Application MySQL Connector MySQL Connector MySQL Router MySQL Router M HA MySQL Shell M M Group Replication Oracle ©|Nov 2016 Slide 27 Labs MySQL InnoDB Cluster: Architecture – Step 3 Application Application MySQL Connector MySQL Connector MySQL Router MySQL Router M HA MySQL Shell M M Group Replication S1 S2 S3 S4 S… Read-Only Slaves Oracle ©|Nov 2016 Slide 28 Labs MySQL InnoDB Cluster: Architecture – Step 4 Application Application Application Application MySQL Connector MySQL Connector MySQL Connector MySQL Connector MySQL Router MySQL Router MySQL Router MySQL Router M HA M HA