Mysql Server 8.0: the Oracle Mysql Team Writes the Code

MySQL 8.0 Driving Innovation with MySQL and Docker Kathy Forte MySQL Curriculum Developer Microservices World 2019

(With Insights from MySQL Development and Product Management Teams)

CopyrightCopyright © © 2019 2019, Oracle, Oracle and/or and/or its its affiliates. affiliates. All All rights rights reserved. reserved. | Safe Harbor Statement The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.

Feature Request Feature Request from DBAs from Developers

Drawing from https://www.dreamstime.com/royalty-free- stock-image-source-code-puzzle-shows-software-program- programming-showing-image34210956

Developer First Data Driven Hybrid data model and data Optimizing services with real access APIs for flexibility for time data analysis developers

Scalable & Secure Mobile Friendly Improved query/index x maintenance tools, security Ready for location based 24 7 tools, and performance services. Handling Emoji improvements and Unicode characters at Scale

Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | 6 Quote from External, Independent Source “Let me therefore boldly claim: MySQL 8.0 is one small step for a database, one giant leap for SQL” - Markus Winand, Modern SQL

https://modern-sql.com/blog/2019-04/mysql-8.0

https://twitter.com/MarkusWinand

1 Top Five 8.0 Features for Developers

Drawing from https://www.dreamstime.com/royalty- free-stock-image-source-code-puzzle- 2 shows-software-program-programming- Top Five 8.0 Features for DBAs showing-image34210956

3 Containers and MySQL

Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | MySQL – Next Version – 8.0: Top Features for Developers 1) MySQL Document Store 2) Window Functions or Analytical Functions 3) Common Table Expressions

/ (CTEs) 4) InnoDB NO WAIT & SKIP LOCKED 5) utf8mb4 Character Set

MySQL 8.0 – Hybrid Model

Developer First Hybrid data model and data access APIs for flexibility for developers

MySQL Document Store – NoSQL within MySQL product

https://www.youtube.com/watch?v=E8zwnXjIjPM

Business Benefit: Both worlds in one database product: Saves time - don’t have to learn yet another database product

Reduces complexity of dealing with multiple vendors

https://www.upwork.com/hiring/data/sql-vs-nosql-databases-whats-the-difference/

• All the existing features of MySQL – Replication – ACID transaction http://datastoragecharlotte.com/services/document-storage/ – Documents have data integrity – All of our tools – Collection joins; analytics on collections

• With the addition of schemaless – Documents using JSON – Easy to program CRUD APIs

NodeJS C#

• How to combine SQL and NoSQL (video): https://www.youtube.com/watch?v=yQvdJ2mVHjg&index=2&list=PLWx5 a9Tn2EvGe-LGUpXYkv8-5LyKGh_DF

• Blogs - MySQL Document Store: https://mysqlserverteam.com/category/docstore/

Data Driven Optimizing services with real time data analysis and locking control

More Top Features for SQL Developers /

MySQL 8.0 – Data-Driven Models

Business Benefit: Real-time data analysis and more: Less complex programming with Window Functions, Common Table Expressions (CTEs), ‘NO WAIT’ & ‘SKIP LOCKED’ Saves time - don’t have to switch to analytics database

2. MySQL Window Functions

Improving Support for Analytics

Feature Request from Developers

https://www.google.com/search?biw=1646&bih=907&tbm=isch&sa=1&q=window++cat&oq=window+ +cat&gs_l=psy-ab.3..0l6j0i67k1j0l3.20057.20669.0.21178.4.4.0.0.0.0.217.601.0j3j1.4.0....0...1.1.64.psy- ab..0.2.211...0i7i30k1j0i8i7i30k1.0.lMDZgoQOoG0#imgrc=w_TkHuUSM07wtM:

Improving Support for Analytics

https://www.google.com/search?biw=1646&bih=907&tbm=isch&sa=1 &q=window++cat&oq=window++cat&gs_l=psy- ab.3..0l6j0i67k1j0l3.20057.20669.0.21178.4.4.0.0.0.0.217.601.0j3j1.4.0.. ..0...1.1.64.psy- ab..0.2.211...0i7i30k1j0i8i7i30k1.0.lMDZgoQOoG0#imgrc=w_TkHuUSM 07wtM:

Window functions gather results into a set of rows, returning a single aggregated value for each of the rows in the result set

Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | 21 Window Functions Justin Turner Dodgers 348 Charlie Blackman Rockies 333 Consider: SELECT * Daniel Murphy Nationals 332 FROM player; Bryce Harper Nationals 327 Paul Goldschmidt Diamondbacks 322

Buster Posey Giants 321 The top ten batters in the DJ LeMahieu Rockies 316 National League - 2017 Nolan Arenado Rockies 315

Joey Votto Reds 315 Ryan Zimmerman Nationals 310

Here’s an aggregate function: give me the top batting average from each team of top ten batters for each NL team: SELECT team, MAX(baverage) FROM player GROUP BY team;

Diamondbacks 322 Nationals 332 Dodgers 348 Giants 321 Reds 315 Rockies 333

Window Functions Team Name baverage MAX (baverage) SELECT team, name, baverage, Diamondbacks Paul Goldschmidt 322 322 MAX(baverage) OVER (PARTITION BY team) Dodgers Justin Turner 348 348 FROM player; Giants Buster Posey 321 321 Nationals Ryan Zimmerman 310 332 Nationals Daniel Murphy 332 332

Nationals Bryce Harper 327 332

Reds Joey Votto 315 315 Rockies Charlie Blackmon 333 333 Rockies DJ LeMahieu 316 24 333 Rockies Nolan Arenado 315 333

Dodgers Justin Turner 348 1 Giants Buster Posey 321 1 SELECT team, name, baverage, RANK() OVER (PARTITION BY team Nationals Daniel Murphy 332 1 ORDER BY baverage DESC) FROM player Nationals Bryce Harper 327 2 ORDER BY team, baverage DESC; Nationals Ryan Zimmerman 310 3

Reds Joey Votto 315 1

Rockies Charlie Blackmon 33325 1

Rockies DJ LeMahieu 316 2 Rockies Nolan Arenado 315 3

25 Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | More on Window Functions http://mysqlserverteam.com/mysql-8-0-2-introducing-window-functions/ https://dev.mysql.com/doc/refman/8.0/en/window-functions.html

3. MySQL Common Table Expressions (CTEs)

Feature Request from Developers

https://forums.anandtech.com/threads/could-you-live-a-recursive-life.2393849/

Common Table Expressions (CTEs)

• “With queries” – Statement scoped views – Valid only in the query they are being used – Benefit: Simplifies writing complex SQL: https://forums.anandtech.com/threads/could- you-live-a-recursive-life.2393849/ WITH t1 AS (SELECT * FROM tblA WHERE a=‘b’) SELECT * FROM t1;

) a SELECT * FROM qn; 1 https://forums.anandtech.com/threads/could-you-live-a- 2 recursive-life.2393849/ 3 4 5 6 7 8 9 10

Consider: A table with one parent, one child, family tree

parent child Charles Jeffrey Jeffrey Kathy Kathy Samuel

Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | 31 Recursive CTE Benefit: Iteration How many levels in this family descendants tree? parent descendant level WITH RECURSIVE descendants AS Charles Jeffrey 1 ( SELECT parent, child AS descendant, 1 as level Charles Kathy 2 FROM family UNION ALL Charles Samuel 3 SELECT d.parent, f.child, d.level + 1 ….. FROM descendants as d JOIN family f ON d.descendant = f.parent ) SELECT * FROM descendants ORDER BY parent, level, descendant ;

Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | 32 More on CTEs First blog in a series: http://mysqlserverteam.com/mysql-8-0-labs-recursive-common-table- expressions-in-mysql-ctes/

Performance and CTEs: https://mysqlserverteam.com/mysql-8-0-improved-performance-with-cte/

SKIP LOCKS and NOWAIT Constructs

Feature Request from Developers

• https://workwiththebest.intraway.com/blog-post/the-use-of-oracle-feature-select-for-update-to-prevent-race-conditions-on-our-exposed-services/

https://workwiththebest.intraway.com/blog-post/the-use-of- oracle-feature-select-for-update-to-prevent-race-conditions-on- our-exposed-services/

Locking

Feature Request from Developers http://mysqlserverteam.com/mysql-8-0-1-using- skip-locked-and-nowait-to-handle-hot-rows/

https://dev.mysql.com/doc/refman/8.0/en/data -locks-table.html

• MySQL 8.0 defaults to utf8mb4 • Latest Unicode 9.0 Support – Emoji characters • Support for 21 languages!

https://www.google.com/search?tbm=isch&q=emojis+happy&chips=q:emoji+happy,g_3:happiness&sa= X&ved=0ahUKEwjo9c6q9ZnbAhWIAnwKHeqpBwwQ4lYIKigA&biw=1860&bih=846&dpr=0.8#imgrc=OFM SFSANDEYHDM:

1 Top Five 8.0 Features for Developers

Drawing from https://www.dreamstime.com/royalty- free-stock-image-source-code-puzzle- 2 shows-software-program-programming- Top Five 8.0 Features for DBAs showing-image34210956

3 Containers and MySQL

6) Invisible Indexes 7) Check Constraints 8) Roles 9) Transactional Data Dictionary 10) Performance

MySQL 8.0 – Top Features for DBAs

Scalable & Secure 24x7 Improved query/index at Scale maintenance tools, security tools, and performance improvements

Business Benefits of New Features: More control of index placement with ‘invisible’ indexes

Improves data integrity with check constraints Reduces complexity of setting up security with roles Better performance and reliability with data dictionary

6. MySQL Invisible Indexes – Toggling Indexes

‘Off’ and ‘On’

Feature Request from DBAs

https://www.google.com/search?q=invisible+man+old+movie&source=lnms&tbm=isch&sa=X&ved=0ahUKEwjwhJaj0vnaAhUCwFQKHZTiD7sQ_AUICygC&biw=1422&bih=1002#imgrc=l5aW4QqAIIEqdM:

• Indexes are “hidden” from the MySQL Optimizer – Index pages are updated but not used in queries https://www.google.com/search?q=invi sible+man+old+movie&source=lnms&t by optimizer bm=isch&sa=X&ved=0ahUKEwjwhJaj0v naAhUCwFQKHZTiD7sQ_AUICygC&biw= 1422&bih=1002#imgrc=l5aW4QqAIIEqd M:

http://viralpatel.net/blogs/invisible-indees-in-oracle-11g/

• Two use cases: –Soft Delete (Recycle Bin) https://www.google.com/search?q=invisible+man+old+movie&source=lnms&tbm=isch&sa=X&ved=0ah UKEwjwhJaj0vnaAhUCwFQKHZTiD7sQ_AUICygC&biw=1422&bih=1002#imgrc=l5aW4QqAIIEqdM: –Staged Rollout

• “I don’t think this index is used any more": https://www.google.com/searc ALTER TABLE Country ALTER INDEX c INVISIBLE; h?q=invisible+man+old+movie &source=lnms&tbm=isch&sa=X &ved=0ahUKEwjwhJaj0vnaAhU CwFQKHZTiD7sQ_AUICygC&biw • “Queries not slower. Okay, it is now safe to drop the index”: =1422&bih=1002#imgrc=l5aW4 QqAIIEqdM: ALTER TABLE Country DROP INDEX c;

• Adding any new index can change existing execution plans • All change introduces risk of regression

https://www.google.com/search?q=invisible+ma n+old+movie&source=lnms&tbm=isch&sa=X&ve ALTER TABLE Country ADD INDEX c (Continent) INVISIBLE;d=0ahUKEwjwhJaj0vnaAhUCwFQKHZTiD7sQ_AUI CygC&biw=1422&bih=1002#imgrc=l5aW4QqAIIE • “I want to test just in my session” qdM SET optimizer_switch=‘use_invisible_indexes=on’; • “Okay, it works after my experiments, turn it on”

ALTER TABLE Country ALTER INDEX c VISIBLE;

• Remove checks from your programs and place in the database where they belong • But once defined, many of those rules can be enforced by the DBMS, thereby improving data quality … and your business processes.

• INSERT and UPDATE processing) will cause the expression to be evaluated - If the modification conforms to the Boolean expression, the modification is permitted to continue - not, the statement will fail with a constraint violation.

Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | 49 7. MySQL Check Constraints CREATE TABLE products ( product_no INTEGER, name TEXT, price NUMERIC CHECK (price > 0), cost_to_produce NUMERIC CHECK (cost_to_produce > 0), CHECK (price > cost_to_produce) ); INSERT INTO products VALUES (1, 'gum', 1, 1); ERROR 3819 (HY000): Check constraint 'products_chk_3‘ is violated.

Here’s my latest blog on check constraints: ‘MYSQL CHECK CONSTRAINTS TO THE RESCUE’ (https://mysqlsolutionsblog.wordpress.com/2019/09/24/mysql-check-constraints-to-the-rescue/)

8. MySQL Roles

Improving MySQL Access Controls Feature Request from DBAs

https://cougardaily.org/7241/student-life/advanced-acting-and-advanced-drawing-draw-together/

Improving MySQL Access Controls

https://cougardaily.org/7241/student-life/advanced-acting-and- advanced-drawing-draw-together/

Improving MySQL Access Controls • Easier to manage user and applications rights • As standards compliant as practically possible https://cougardaily.org/7241/student-life/advanced-acting-and- • Multiple default roles advanced-drawing-draw-together/

https://dev.mysql.com/doc/refman/8.0/en/roles.html

• Increased reliability and performance

• Using InnoDB tables internally for data dictionary

– No FRM files – No DB.OPT files Data Dictionary – No TRG files Increased reliability and – No TRN files consistency with transactional meta data – No PAR files repository

Additional Benefits

• Better cross-platform experience – No dependencies on filesystem semantics Data Dictionary Increased reliability and consistency with • Atomic DDL transactional meta data repository – Robust replication (Group Replication)

• Flexible metadata API – Easier path to adding new features

• Applies to statements performing multiple logical operations: –DROP TABLE t1, t2;

• Uses a custom Meta Data Lock (MDL) for Data Definition Language (DDL)

• Store information in system catalogues and only changes your table when you add new rows: –ALTER TABLE t1 ADD COLUMN c3 INT DEFAULT 1, ALGORITHM = INSTANT;

• Cannot ‘ALTER DROP COLUMN’ (coming in later releases)

http://mysqlserverteam.com/mysql-8-0-data-dictionary-background-and- motivation/

MySQL 8.0 is 2x faster than MySQL 5.7 and sets new benchmarks records

1,200,000

1,000,000

MySQL 8.0 800,000

600,000 MySQL 5.7

400,000 Queries per Second perQueries MySQL 5.6 200,000

0 1 2 4 8 16 32 64 128 256 512 OS : Oracle Linux 7.4 Users CPU : 48cores-HT Intel Skylake 2.7Ghz (2CPU sockets, Intel(R) Xeon(R) Platinum 8168 CPU) RAM: 256GB Storage : x2 Intel Optane flash devices (Intel (R) Optane (TM) SSD P4800X Series)

300,000

250,000 MySQL 8.0

200,000 MySQL 5.7 150,000

100,000

Queries per Second per Queries MySQL 5.6

50,000

0 1 2 4 8 16 32 64 128 256 512 1,024 OS : Oracle Linux 7.4 CPU : 48cores-HT Intel Skylake 2.7Ghz Users (2CPU sockets, Intel(R) Xeon(R) Platinum 8168 CPU) RAM: 256GB Storage : x2 Intel Optane flash devices (Intel (R) Optane (TM) SSD P4800X Series)

Old Design New Design

• For more information on the MySQL 8.0 benchmarks: http://www.mysql.com/why-mysql/benchmarks/ • The SysBench benchmark tool is freely available for specific benchmarks and can be downloaded here: http://dev.mysql.com/downloads/benchmarks.html • And this blog: http://dimitrik.free.fr/blog/

SELECT TABLE_SCHEMA, TABLE_NAME, TABLE_TYPE, Over 30x faster! ENGINE, ROW_FORMAT FROM information_schema.tables WHERE TABLE_SCHEMA LIKE 'db%';

1000 active sessions Query Time MySQL 8.0 0. 10. 20. 30. 40. MySQL 5.7

Time in Seconds (Lower is better)

1 Top Five 8.0 Features for Developers

Drawing from https://www.dreamstime.com/royalty- free-stock-image-source-code-puzzle- 2 shows-software-program-programming- Top Five 8.0 Features for DBAs showing-image34210956

3 Containers and MySQL

Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Why Docker “You create an illusion for your application that it is getting its very own OS instance, although there may be other containers running on same system. Docker is what enables us to run, create and manage containers on a single operating system.”

https://containerjournal.com /2019/01/14/kubernetes-vs- docker-a-primer/

• Advantages of MySQL with Docker - Fastest way to test 8.0! - Portable - Easy to use - Lightweight (less than 1GB)

Uber is using MySQL with Docker: https://eng.uber.com/dockerizing-mysql/

• First, install Docker: https://docs.docker.com/ or go to https://labs.play-with-docker.com/ for online Alpine sandbox

• Then, pull 8.0 and play with it:

docker run --name mysql8 -e MYSQL_ROOT_PASSWORD=Newpasswd# -d mysql/mysql-server

docker exec -it mysql8 mysql -uroot –p Enter password:

My blog on setting up 8.0 for testing: https://mysqlsolutionsblog.wordpress .com/2018/09/25/driving-innovation- with-mysql-8-0-and-docker/

• First, install Docker: https://docs.docker.com/ or go to https://labs.play-with-docker.com/ for online Alpine Linux sandbox

• Then, pull 8.0 and connect to MySQL Shell: $ docker run --name mysql8 -e MYSQL_ALLOW_EMPTY_PASSWORD=yes -d mysql/mysql- server

/* have to point to the directory of mysqlx.sock */ $ docker exec -it mysql8 mysqlsh --mysqlx -uroot -S/var/run/mysqld/mysqlx.sock

http://datastoragecharlotte.com/se rvices/document-storage

Copyright © 2018, Oracle and/or its affiliates. All rights reserved. Confidential – Oracle Internal/Restricted/Highly Restricted 74 MySQL Document Store – MySQL Shell Move Back and Forth from CRUD to SQL

http://datastoragecharlotte.com/services/d ocument-storage

Copyright © 2018, Oracle and/or its affiliates. All rights reserved. Confidential – Oracle Internal/Restricted/Highly Restricted 75 More Information on Docker and MySQL Shell My blog on setting up MySQL 8.0 with MySQL Shell for testing Document Store:

https://mysqlsolutionsblog.wordpress .com/2018/09/27/running-mysql-shell- 8-0-with-docker/

Docker Engine $ docker login container-registry.oracle.com Username: [email protected] Password: $ docker pull container-registry.oracle.com/mysql/enterprise-server:8.0.13 8.0.13: Pulling from mysql/enterprise-server e64f6e679e1a: Pull complete

Agent Object Kubernetes storage API Operator CRD

MySQLCluster MySQLBackup MySQLScheduledBackup MySQLRestore

Release Notes for Latest Release 8.0.17: https://dev.mysql.com/doc/relnotes/mysql/8.0/en/new s-8-0-17.html

Blog on 8.0.17: https://mysqlserverteam.com/the-mysql-8-0-17-maintenance- release-is-generally-available/

• Strategies for upgrade, changes:

https://dev.mysql.com/doc/refman/8.0/en/upgrading.html/

• Tool to check for incompatibilities (from 5.7 to 8.0):

http://lefred.be/content/how-to-safely-upgrade-to-mysql-8-0/

• CTEs • Replication Applier Thread Progress Reports • Window Functions • • SKIP LOCKED and NOWAIT (hot row handling) GTID_PURGED always settable • Document Store • Undo tablespaces now dynamic • Unicode (utf8mb4 default) encoding • Smaller Package Downloads • Data Dictionary • JSON table function • Roles • Invisible Indexes • JSON path expressions (ranges) • Histograms • JSON performance improvements • Descending Indexes • Expanded Query Hints • Redo Undo Log Encryption • Binary and Relay Log Encryption • Improved usability of cost constant configuration • Password Rotation Policy • Improved Parallel Replication • GIS Improvements (3-D) • SSD by default • Scalability Improvements • Resource Groups • Atomic DDL • Cost Model Improvements • Check Constraint

• Dedicated server: self-tune Cloud Friendly • SET PERSIST 81 Q & A Your Thoughts