MySQL 8.0: Performance & Scalibility Bill Papp MySQL Solutions Architect 1 CopyrightCopyright © © 2016, 2017, Oracle Oracle and/or and/or its its affiliates. affiliates. All All rights rights reserved. reserved. | Safe Harbor Statement The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | Improved UTF-8 Support Copyright © 2018, Oracle and/or its affiliates. All rights reserved. Improved UTF-8 Support in MySQL 8.0 • Support for the latest Unicode 9.0 • utf8mb4 made default character set! –utf8mb4_0900_ai_ci default collation • Accent and case sensitive collations –Including 20+ language specific collations –Now also Japanese and Russian • Significantly improved performance Copyright © 2018, Oracle and/or its affiliates. All rights reserved. 5 What Is in MySQL 5.7 and Earlier Versions? • Default charset is “latin1” and default collation is “latin1_swedish_ci” • utf8 = utf8mb3: support BMP Plane 0 only • utf8mb4 character set: –Only accent and case insensitive collations –Default collation is utf8mb4_general_ci, compares all characters beyond BMP, e.g. emojis, to be equal –20+ language specific collations –Recommend to use: utf8mb4_unicode_520_ci Copyright © 2018, Oracle and/or its affiliates. All rights reserved. 6 New Default Character Set • No changes to existing tables • Only has effect on new tables/schemas where character set is not explicitly defined. • Separating character set/collation change from server upgrade – Upgrade first, change charset/collation afterwards • Recommend users to not mixing collations – Error “Illegal mix of collations” – Slower query because index can no longer be used Copyright © 2018, Oracle and/or its affiliates. All rights reserved. 7 Transactional Data Dictionary Copyright © 2018, Oracle and/or its affiliates. All rights reserved. Data Dictionary Definition • Metadata is information about data in the RDBMS – Column definitions, Index definitions, Foreign key definitions ... • Data Dictionary collects metadata for all things in RDBMS – User table structure, Stored program definitions ... Metadata ID NAME WEIGHT HEIGHT GENDER 3 Bob 80 185 M Data 5 Liz 55 165 F Copyright © 2018, Oracle and/or its affiliates. All rights reserved. MySQL Data Dictionary before MySQL 8.0 Data Dictionary Files File system FRM TRG OPT SQL System Tables (mysql.) MyISAM user events proc InnoDB System Tables InnoDB Copyright © 2018, Oracle and/or its affiliates. All rights reserved. 10 MySQL Data Dictionary before MySQL 8.0 Problems • Performance of INFORMATION_SCHEMA • Inconsistencies between persistent storage in files and tables • Inconsistencies between InnoDB DD and Server DD • Showstopper for crash-safe / atomic DDL • Difficult for replication • Hard to extend Copyright © 2018, Oracle and/or its affiliates. All rights reserved. 11 What do you get with the Transactional Data Dictionary? • All DD tables and system tables in a DD tablespace (mysql) • Atomic DDL (this is actually a big thing!!) – Especially important for replicated topologies – Crash situations will not end up in out-of-synch information sql-layer/SE-layer • Reliability and Redundancy – Serialized Dictionary Information • Better scaling INFORMATION_SCHEMA queries – Reimplemented with views over DD tables – Scaling both on database size and query load • Improved Upgrade Copyright © 2018, Oracle and/or its affiliates. All rights reserved. 12 MySQL 8.0: Transactional Data Dictionary Atomic and crash-safe DDL Data Dictionary DD Table DDDD Table Table SQL InnoDB Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | 13 Transactional Data Dictionary in MySQL 8.0 Main features • All dictionary metadata stored in tables • Single repository of metadata for SEs and the MySQL server • Reliable, crash-safe InnoDB tables • INFORMATION_SCHEMA tables implemented as VIEWs over DD tables – Queries can be optimized by the optimizer – Improved performance – Simpler, uniform implementation, easier to maintain Copyright © 2018, Oracle and/or its affiliates. All rights reserved. Transactional Data Dictionary in MySQL 8.0 Main features, cont’ed • Extensible –The data dictionary schema is based on the SQL standard definitions –The data dictionary is designed to be easily extended for new requirements –Designed to provide automated upgrade of dictionary data –Designed to allow plugins to add and extend system tables, I_S tables, Performance Schema tables Copyright © 2018, Oracle and/or its affiliates. All rights reserved. Transactional Data Dictionary in MySQL 8.0 Overview INFORMATION SCHEMA Data Dictionary Optimizer InnoDB Views DD Table User Table Archive User Table CSV User Table Copyright © 2018, Oracle and/or its affiliates. All rights reserved. 16 Transactional Data Dictionary API in MySQL 8.0 Design goals • The only way to deal with Data Dictionary – For the server core – For SEs – For other plugins • Less Error-prone • Internal API (non-stable) and external API (stable) • Provide a way to handle SE private data Copyright © 2018, Oracle and/or its affiliates. All rights reserved. 17 Transactional Data Dictionary in MySQL 8.0 Architecture Query Executor Parser Optimizer Plugin Data Dictionary External API Data Dictionary Internal API Data Storage Dictionary Tablespace User Table Engine Tablespace InnoDB Internal SE Plugin Copyright © 2018, Oracle and/or its affiliates. All rights reserved. 18 Reliability and Redundancy Copyright © 2018, Oracle and/or its affiliates. All rights reserved. Transactional Data Dictionary in MySQL 8.0 Reliability and Redundancy SDI: Serialized Dictionary Information - Used for data migration and is a redundant write-only copy in normal operation The InnoDB Data Dictionary tablespace ”mysql” is the metadata storage InnoDB User Tablespaces InnoDB Data Dictionary tablespace mysql.tables User Table ID Name 1 User Table 1 SDI 2 User Table 2 Copyright © 2018, Oracle and/or its affiliates. All rights reserved. 20 Example - DROP SCHEMA at high level MySQL 5.7 MySQL 8.0 • Delete tables • Delete tables – Metadata, TRN/TRG/FRM files – Metadata, rows in InnoDB – Data, InnoDB tables – Data, InnoDB tables • Delete stored programs Delete stored programs – Metadata, rows in MyISAM (non- • transactional) – Metadata, rows in InnoDB • Delete schema • Delete schema – Metadata, DB.OPT file – Metadata, rows in InnoDB Mix of filesystem, non- Updates to transactional storage, transactional/transactional one commit storage and multiple commits Copyright © 2018, Oracle and/or its affiliates. All rights reserved. 21 Reliability – Disaster Recovery Use case – IMPORT using Serialized Dictionary Information • All data stored in InnoDB (transactional SE) – Metadata stored in an SDI embedded in .ibd – Data stored in .ibd • InnoDB logging DDL operations as «Dictionary Storage Engine» – After restart and recovery, the DDL operation will be complete or rolled back Copyright © 2018, Oracle and/or its affiliates. All rights reserved. 22 Reliability – Automated Dictionary Upgrade Use case – Upgrade • The Data Dictionary definition will have a version attached • The MySQL Server executable will support upgrading dictionary tables Copyright © 2018, Oracle and/or its affiliates. All rights reserved. 23 Reliability – Protecting Dictionary Tables • Data Dictionary tables are protected against user DML – This means there will be no possibility for corruption from user DML on these tables – All information in the dictionary tables will be available through INFORMATION_SCHEMA Copyright © 2018, Oracle and/or its affiliates. All rights reserved. 24 Data Dictionary CREATE TABLE customers( CREATE PROCEDURE p1(v INT) id INT AUTO_INCREMENT SQL SECURITY INVOKER ... BEGIN PRIMARY KEY (id), ... INDEX ... END FOREIGN KEY ... ) Data Dictionary Table Definitions SP Definitions ACL View Definitions Plugins Copyright © 2018, Oracle and/or its affiliates. All rights reserved. Hot Row Contention Improvements Copyright © 2018, Oracle and/or its affiliates. All rights reserved. SELECT ... FOR UPDATE SKIP LOCKED • Common problem: – Hot row contention, multiple worker threads accessing the same rows • Solution 1: – Only read rows that are not locked – InnoDB skips a locked row, and the next one goes to the result set • Example: – Booking system: Skip orders that are pending START TRANSACTION; SELECT * FROM seats WHERE seat_no BETWEEN 2 AND 3 AND booked = 'NO' FOR UPDATE SKIP LOCKED; Copyright © 2018, Oracle and/or its affiliates. All rights reserved. 27 SELECT… FOR UPDATE NOWAIT • Common problem: – Hot row contention, multiple worker threads accessing the same rows • Solution 2: – If any of the rows are already locked, the statement should fail immediately – Without NOWAIT, have to wait for innodb_lock_wait_timeout (default: 50 sec) while trying to acquire lock • Usage: START TRANSACTION; SELECT * FROM seats WHERE seat_no BETWEEN 2 AND 3 AND booked = 'NO' FOR UPDATE NOWAIT; ERROR 3572 (HY000): Statement aborted because lock(s) could not be acquired … Copyright © 2018, Oracle and/or its affiliates. All rights reserved. 28 Index Extensions Copyright © 2018, Oracle and/or its affiliates. All rights reserved. Invisible Index • Index is maintained