MySQL for Distributed Transaction and The fusion between SQL & NOSQL using JSON Ajo Robert Software Engineer, MySQL, Oracle Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | Confidential – Oracle Internal/Restricted/Highly Restricted Safe Harbor Statement The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle. Confidential – Oracle Internal/Restricted/Highly Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | 2 Restricted Program Agenda 1 Introduction to Distributed Transaction (XA) 2 Role of MySQL in XA Topology & Use Cases 3 A short recap. 4 Introduction to MySQL JSON & Document Store 5 NoSQL Interface and SQL Data Access 6 Summary. Confidential – Oracle Internal/Restricted/Highly Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | 3 Restricted MySQL Distributed Transaction Confidential – Oracle Internal/Restricted/Highly Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | 4 Restricted Concepts What is a Transaction? Benefits? ACID (Atomicity, Consistency, Isolation, Durability) Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | Concepts What is a Transaction? Benefits? ACID (Atomicity, Consistency, Isolation, Durability) What is a distributed transaction(DT)? Benefits? ACID across heterogeneous and/or geographically diverse nodes. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | Concepts What is a Transaction? Benefits? ACID (Atomicity, Consistency, Isolation, Durability) What is a distributed transaction(DT)? Benefits? ACID across heterogeneous and/or geographically diverse nodes. XA? Is an X/Open group standard for DT. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | Concepts What is a Transaction? Benefits? ACID (Atomicity, Consistency, Isolation, Durability) What is a distributed transaction(DT)? Benefits? ACID across heterogeneous and/or geographically diverse nodes. XA? Is an X/Open group standard for DT. How popular is it? It is used for Databases, Message Queues, File Systems, etc. Eg: Few XA Supported DBMS are MySQL, Oracle, IBM DB/2, Sybase, Informix. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | XA Basics... The X/Open Distributed Transaction Processing standard, designed by Open Group. The XA interfaces enable the resource managers to join transactions, to perform 2PC Phase 1: PREPARE on All Nodes Phase 2: If PREPARE SUCCEED on All Nodes COMMIT on All Nodes Else ROLLBACK on All Nodes End If Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | XA Basics... The X/Open Distributed Transaction Processing standard, designed by Open Group. The XA interfaces enable the resource managers to join transactions, to perform 2PC Phase 1: PREPARE on All Nodes APP Phase 2: If PREPARE SUCCEED on All Nodes COMMIT on All Nodes Else XA ROLLBACK on All Nodes TM RM End If Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | XA Use Case Application + TM XA Transaction 1 Part 1 Part 2 Part 3 RM 1 RM 2 RM 3 Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | XA Application Level Sharding Application + TM XA Transaction 1 Part 1 Part 2 Part 3 MySQL 1 MySQL 2 MySQL 3 Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | Databases : MySQL File system : XADisk* XA Heterogeneous Transaction Message Queue : Apache Activemq* *Random choice and not a recommendation. Application + TM XA Transaction 1 Part 1 Part 2 Part 3 MySQL 1 XADisk Activemq Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | XA Example App + TM MySQL IN MySQL US XA Start ‘x1’ DELETE FROM t1 WHERE id = 10; Transaction XA Start ‘x1’ Body INSERT INTO t1(id, ... ) VALUES (10, .... ); APP + TM XA End ‘x1’ XA End ‘x1’ XA XA XA Prepare ‘x1’ Success MySQL IN MySQL US 2 Phase XA Prepare ‘x1’ Success Commit XA Commit ‘x1’ XA Commit ‘x1’ Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | A Short Recap ● XA is an industry adopted distributed transaction standard. ● MySQL supports XA from 5.0 and matured in 5.7. ● XA transactions can be performed across functionally different nodes. ● XA can be used for ACID compliant transaction across different MySQL instances in same the premise or at a remote place. Confidential – Oracle Internal/Restricted/Highly Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | 15 Restricted Enhanced XA Support for Replication in MySQL-5.7 By Nisha Gopalakrishnan 25-March-2018 15:30 Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | MySQL JSON & Document Store Confidential – Oracle Internal/Restricted/Highly Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | 17 Restricted Everyone is looking for? SQL Databases NoSQL Databases - Schema Support - Handle data as key/value pair. - Foreign Keys - Supports Structured/ - SQL Language Support Semi-structured and - JOIN Capability Unstructured data. - ACID Compliance - Flexible data type - Tools, expertise and support. - Store/Retrive JSON Objects Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | DBMS or NoSQL ? Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | DBMS or NoSQL ? Why not both ? Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | MySQL Document Store: From 5.7.12 Powered by MySQL JSON datatype Supports CRUD Operations Enabled by all famous client interfaces & MySQL Shell Supports ALL native JSON types * Numbers, strings, bool, Objects, arrays And some extended types * Date, time, datetime, timestamp, etc. Confidential – Oracle Internal/Restricted/Highly Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | 21 Restricted The Fusion: Creation... mysql-js> \use test Schema `test` accessible through db. mysql-js> var SuperHeroes= db.createCollection("SuperHeroes") mysql-js> db.getCollections() [ <Collection:SuperHeroes> ] Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | The Fusion: Creation... mysql-js> \sql Switching to SQL mode... Commands end with ; mysql-sql> SHOW TABLES; +----------------+ | Tables_in_test | +----------------+ | SuperHeroes | +----------------+ CREATE TABLE `SuperHeroes` ( `doc` json DEFAULT NULL, `_id` varchar(32) GENERATED ALWAYS AS (json_unquote(json_extract(`doc`,'$._id'))) STORED NOT NULL, PRIMARY KEY (`_id`) ) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 | Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | The Fusion: Addition... mysql-sql> \js Switching to JavaScript mode... mysql-js> SuperHeroes.add({name: "Thor", publisher: "Marvel"}) Query OK, 1 item affected (0.05 sec) mysql-js> \sql Switching to SQL mode... Commands end with ; mysql-sql> mysql-sql> INSERT INTO SuperHeroes(doc) VALUES( '{"_id":"abc1234567890", "name": "Batman", "publisher": "DC"}'); Query OK, 1 row affected (0.05 sec) Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | The Fusion: Selection... mysql-sql> CREATE VIEW SuperHeroList AS SELECT doc->"$.name" as Name, doc->>"$.publisher" as Publisher FROM SuperHeroes; Query OK, 0 rows affected (0.01 sec) mysql-sql> SELECT * FROM SuperHeroList; +--------+-----------+ | Name | Publisher | +--------+-----------+ | Thor | Marvel | | Batman | DC | +--------+-----------+ 2 rows in set (0.05 sec) Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | The Fusion: Selection... mysql-js> SuperHeroes.find() [ { "_id": "ac5573b2ba1ae811eb7ceacb7b28cf39", "name": "Thor", "publisher": "Marvel" }, mysql-js> SuperHeroes.find("publisher like 'DC%'") { [ "_id": "abc1234567890", { "name": "Batman", "_id": "abc1234567890", "publisher": "DC" "name": "Batman", } "publisher": "DC" ] } 2 documents in set (0.04 sec) ] 1 document in set (0.04 sec) Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | Fusion: Integrity Constraint in NoSQL mysql-sql> CREATE TABLE publishers(name VARCHAR(100), PRIMARY KEY(name)) CHARSET=utf8mb4; Query OK, 0 rows affected (0.06 sec) mysql-sql> INSERT INTO publishers VALUES("DC"),("Marvel"); Query OK, 2 rows affected (0.05 sec) mysql-sql> ALTER TABLE SuperHeroes ADD COLUMN Publisher VARCHAR(100) AS (doc->>"$.publisher") STORED; Query OK, 3 rows affected (0.11 sec) mysql-sql> ALTER TABLE SuperHeroes ADD FOREIGN KEY(Publisher) REFERENCES publishers(name); Query OK, 3 rows affected (0.12 sec) Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | Fusion: Integrity Constraint in NoSQL mysql-js> SuperHeroes.add({name: "Sakthiman", publisher: "Raj Comics"}); Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | Fusion: Integrity Constraint in NoSQL mysql-js> SuperHeroes.add({name: "Sakthiman", publisher: "Raj Comics"}); ERROR: 1452: Cannot add or update a child row: a foreign key constraint fails (`test`.`SuperHeroes`, CONSTRAINT `SuperHeroes_ibfk_1` FOREIGN KEY (`Publisher`) REFERENCES `publishers` (`name`)) Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | Summary ● SQL and NoSQL can be used simultaneously to work with data. ● MySQL uses JSON for the schemaless datastore. ● GC and View’s can be created over NoSQL data for convenient relational data access. ● Foreign Key’s can be created over NoSQL collections to impose integrity constraints. ● MySQL Document Store is a flexible schema, ACID compliant and FK capable NoSQL Solution. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | Thank You... Confidential – Oracle Internal/Restricted/Highly Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | 31 Restricted .