The new MariaDB offering MariaDB 10, MaxScale, and more
Rasmus Johansson, VP Engineering, SkySQL
24/06/2014 © SkySQL Corpora on Ab. Company Confiden al. 1 MariaDB intro
24/06/2014 © SkySQL Corpora on Ab. Company Confiden al. 2 What is MariaDB?
• A free fork of MySQL with extra features • features, requested by the users • Backward compa ble • file formats, replica on, configura on files • aim at 100% drop-in replacement • Community developed • 50% of maria-captains are from the community • no hidden agenda
© SkySQL Corpora on Ab. Company Confiden al. Who’s using it?
© SkySQL Corpora on Ab. Company Confiden al. Where to get it
• Distribu ons: • RedHat Enterprise Linux (default in RHEL 7), Debian, Ubuntu (since 14.04), Fedora, Mageia (default), openSUSE (default), Gentoo, Slackware (default), Arch (default), ALTLinux, TurboLinux, Chakra Project (default), Kdu, … • FreeBSD, OpenBSD • Mac OS X with MacPorts or Homebrew • From MariaDB.org • sources, binaries in .tar.gz or .zip (Windows) • Windows MSI installer • MariaDB apt and yum repositories © SkySQL Corpora on Ab. Company Confiden al. Before MariaDB 10.0
• MariaDB 5.1 – 17 releases since Oct 2009 • Aria, XtraDB, PBXT, FederatedX • table elimina on, pool of threads, • MariaDB 5.2 – 15 releases since Apr 2010 • OQGraph, SphinxSE • virtual columns, pluggable auth, segmented key cache, extended sta s cs, • MariaDB 5.3 – 13 releases since July 2011 • major op mizer improvements • replica on: group commit, checksums • HandlerSocket, dynamic columns, Windows performance improvements, microsecond support, GIS precise opera ons, progress repor ng, …
© SkySQL Corpora on Ab. Company Confiden al. Before MariaDB 10.0
• MariaDB 5.5 – 20 releases since Feb 2012 • new thread pool • non-blocking client API • extended keys op miza on • dynamic replicate_do_*, replicate_ignore_*, replicate_wild_* variables
© SkySQL Corpora on Ab. Company Confiden al. Other MariaDB projects
• MaxScale • MariaDB Galera Cluster • Connectors/Drivers • MariaDB C Client Library (LGPL) • MariaDB JDBC driver (LGPL) • MariaDB ODBC driver (LGPL) • MariaDB Audit Plugin
© SkySQL Corpora on Ab. Company Confiden al. Community ac vity
• Past month (March 2014): • Added/Changed Ar cles in the KB: 123 • On Freenode IRC #maria, 550 people wrote 10173 lines
• Source code (Launchpad): : GitHub • 25 ac ve branches MariaDB is now on • 395 commits https://github.com/MariaDB/server/
24/06/2014 9 © SkySQL Corpora on Ab. Company Confiden al. Community Contribu ons
MariaDB 10.0 major contribu ons: • Spider storage engine metadata_lock_info Informa on schema • Per thread memory coun ng and usage • Code by Kentoku Shiba, Spiral Arms • Base code and idea by Lixun Peng, Taobao • Roles • Mul -source replica on • Code by Vicen u Ciorbaru, Google Summer of Code 2013 • Base code by Lixun Peng, Taobao • PCRE Regular Expressions • GET_LOCK • Code by Sudheera Palihakkara, Google • Code by Konstan n "Kostja" Osipov, Summer of Code 2013 mail.ru • Global Transac on IDs • CONNECT storage engine • Some patches by Pavel Ivanov, Google • Code by Olivier Bertrand
https://mariadb.com/kb/en/log-of-mariadb-contributions/
24/06/2014 © SkySQL Corpora on Ab. Company Confiden al. 10 Community Ongoing Dev Projects
• WebScaleSQL patches • Sta s cally op mize mysql-test runs by h ps://mariadb.atlassian.net/browse/MDEV-6039 running less tests, Pablo Estrada, Google Summer of Code • LevelDB/RocksDB storage engine (persistent key-value h ps://mariadb.atlassian.net/browse/ store for fast storage) MDEV-5776 h ps://mariadb.com/kb/en/leveldb-storage-engine/ • CREATE OR REPLACE, CREATE IF NOT • Fusion-IO page compression EXISTS, DROP IF EXISTS, Sriram Pa l, h ps://blog.mariadb.org/significant-performance- Google Summer of Code h ps://mariadb.atlassian.net/browse/ boost-with-new-mariadb-page-compression-on- MDEV-5359 fusionio/ • Self-Tuning Op mizer, Anshu Avinash, • Kerberos authen ca on plugin Google Summer of Code h ps://mariadb.atlassian.net/browse/MDEV-4691 h ps://mariadb.atlassian.net/browse/ • Mroonga storage engine (fast fulltext search) MDEV-350 h ps://mariadb.atlassian.net/browse/MDEV-5222 • Support of UseServerPrepStmts to MariaDB Java Connector, Puneet Dewan, Google Summer of Code
24/06/2014 © SkySQL Corpora on Ab. Company Confiden al. 11 MariaDB 10.0
© SkySQL Corpora on Ab. Company Confiden al. MariaDB 10
● MariaDB 10.0 GA launched March 31st, 2014. ● Application-compatible with MySQL. ● Includes significant unique new features in scalability, replication, performance, NoSQL, operations, security. ● Available under the GPL v2.
© SkySQL Ab. Company Confiden al. MariaDB 10 Scalability: MySQL MariaDB ✘ ✔ Single in 10.0. Parallel Slave Replica on threaded per database. ● Sponsored by Google. ● Allows slaves to process update events in parallel. ● Uses MariaDB 10’s improved Global Transaction ID (GTID). ● MariaDB unlike MySQL can process multiple updates for a single database in parallel. Much better throughput. ● Preliminary benchmarks: almost 10x faster at 12 threads. Improves scalability, reduces slave lag, making slaves more consistent.
© SkySQL Ab. Company Confiden al. Parallel Slave Benchmark
sysbench OLTP single database slave tps relative to master
© SkySQL Ab. Company Confiden al. MariaDB 10 Scalability: MySQL MariaDB ✘ ✔ Multi-Source Replication in 10.0.
Content Online E- Management Commerce System Application
Master Master ● Collects data for S S S S S S S S analytics using built-in replication. Click-stream data
Master ● Aids in administration S S S S example: consolidated backups of multiple databases. ● Uses MariaDB 10’s improved
Global Transaction ID (GTID). Slave ETL Data Warehouse
Easier analytics, more insight, simpler administration, fewer headaches.
© SkySQL Ab. Company Confiden al. Technology Preview
MariaDB 10 Scalability: MySQL MariaDB ✘ ✔ Sharding with Spider optional 3rd in 10.0. party install
● Contributed by a 3rd party developer. Application ● Storage engine partitions large tables
across multiple DBs. Spider
● No changes to the client application. Customers ● Developers aren’t aware, can keep A-H I-P applications simple. Q-Z ● Allows for more
parallelism, scale-out. Shard 1 Shard 2 Shard 3
● Transactional storage Customers Customers Customers engine. A-H I-P Q-Z
Web-scale, without the development hassle.
© SkySQL Ab. Company Confiden al. MariaDB 10 Performance: MySQL MariaDB ✘ ✔ TokuDB Storage Engine optional 3rd in 10.0. party install
● 3rd party partner - TokuTek. ● Drop-in replacement for InnoDB/XtraDB. ● Advanced indexing and compression algorithms. ● Transactional. ● Up to 20x performance gain for inserts/updates. ● Up to 90% less disk storage. ● Online schema changes. ● Reduces or eliminates slave lag. ● Best when DB doesn’t fit in memory, and SSD longevity is important. © SkySQL Ab. Company Confiden al. MariaDB 10 Performance: Op mizer Improvements ● Of 29 distinct enhancements noted, 28 are in MariaDB 10. Just 1 only in MySQL 5.6. ● Enhancements include: ○ Disk access optimizations. ○ JOIN optimizations. ○ Subquery optimizations. ○ Optimized derived tables and views. ○ Execution control. ○ Optimizer control. ○ EXPLAIN improvements.
Less I/O, CPU, memory requirements. Faster execution. © SkySQL Ab. Company Confiden al. MariaDB 10 Performance: More Enhancements and Features
● Fusion-io Atomic Writes - improves performance on popular SSD hardware.✔ exclusive to MariaDB 10 ● XtraDB storage engine - enhanced from InnoDB. ● Improved InnoDB storage engine - based on MySQL 5.6. ● Performance schema - real-time performance management. ● Improved thread pool - better concurrent user performance.✔ exclusive to MariaDB 10 ● More optimizer improvements - higher efficiency in I/O accesses. ✔ exclusive to MariaDB 10
Faster performance, lower cost.
© SkySQL Ab. Company Confiden al. Fusion-IO page compression
• Atomic writes gives a performance increase of about 30%. By enabling fast checksum for XtraDB it’s 50% • By using page compression the compression ratio is leading to better performance and there are less writes to disk. • Multi-threaded flush provides better throughput and decreases operation latencies delivering a performance boost
https://blog.mariadb.org/significant-performance-boost-with-new-mariadb-page-compression-on-fusionio
© SkySQL Ab. Company Confiden al. Technology Preview
MariaDB 10 Interoperability: MySQL MariaDB CONNECT Storage Engine ✘ ✔ in 10.0.
● From 3rd party developer. Application ● Maps diverse data to tables. MariaDB Parser/Optimizer/Connection Pool ● JOIN mapped data Spi der to DB tables. CONNECT Other Engine Engines ● Flat files including CSV. ● Tables in external DBs. ● Generated tables
(PIVOT etc.) .log
CSV Database ● Plug-in API for your own mappings. XML Tables
Powerful tool for data integration, federation. © SkySQL Ab. Company Confiden al. Technology Preview
MariaDB 10 Interoperability: MySQL MariaDB Cassandra Storage Engine ✘ ✔ in 10.0.
● Window into a Cassandra ring: Application read/write like a table in MariaDB. ● Use standard SQL queries. ● JOIN Cassandra data MariaDB Parser/Optimizer/ConnectionSpi Pool to MariaDB tables. der Cassandra Other Engine Engines ● Use a MariaDB cluster for high-availability access. ● Bring data from
Cassandra into OLTP Database applications. Tables Interoperate with Cassandra. Use Cassandra data in OLTP applications.
© SkySQL Ab. Company Confiden al. MariaDB 10 NoSQL Features: MySQL MariaDB Dynamic Columns ✘ ✔ in 10.0.
Cust ID Account Balance Dyn_Col_BLOBs
2035 $154.04 NAME: John Smith|LOC: 45.35243, -74.98348|IMAGE: x27A8B8C ...
2036 $929.10 NAME: Jane Doe|LOC: 45.35243, -74.98348|AGE: 32| GENDER: F...
2037 $377.53 NAME: Carol Jones|AGE: 43|GENDER: F||IMAGE: xA9674DE678 ...
● Store unstructured data in MariaDB tables with a simple API. ● Use MariaDB’s indexing and transactions to manipulate “document” style data fast and consistently. ● Nest sets of dynamic columns inside of other dynamic columns - hierarchical structuring. ● Include multiple rows with dynamic columns in transactions. Process unstructured data in the same way as NoSQL,but with the power of MariaDB.
© SkySQL Ab. Company Confiden al. MariaDB 10 NoSQL Features: MySQL MariaDB HandlerSocket Plugin ✘ ✔ in 10.0.
Transactional NoSQL ● Simple, low-overhead Application Application
API direct to the SQL Statements Simple CRUD API storage engine. Connection Pool ● Bypasses SQL Spi SQL Interface statement processing. der Parser
● Less CPU, memory, Optimizer HandlerSocket Plugin I/O, network demands. InnoDB/XtraDB and Spider Storage Engines ● Batches requests for even less I/O operations. ● Building block for application Database level NoSQL processing. Tables
Mix RDBMS applications with high performance NoSQL applications. Leverage MariaDB’s storage engine architecture for both. © SkySQL Ab. Company Confiden al. MariaDB 10 Operations: MySQL MariaDB ✘ ✔ SHOW EXPLAIN Command in 10.0.
Oh! Sequential scan of a 18M row Why is this so table… no wonder! slow? Lets find I’ll let the team To: DevOps Team out… SHOW know... EXPLAIN FOR 28; From: DBA Debby
Subj. Slow Queries
Hey guys, figured out those blocking queries. The new reservation app is looking for the shortest route but that column has no index - we need one! I’ll add it now. Debby Thread Time
27 .003761 ID select_type table possible_ rows keys And it works in the slow query log too! 28 34.2529 1 SIMPLE tbl NULL 1855174 Also for UPDATE and 29 .000713 DELETE. © SkySQL Ab. Company Confiden al. MariaDB 10 Operations: More Enhancements and Features
● Improved Table Discovery - less work for DBAs; integrates storage engines. ✔ exclusive to MariaDB 10 ● SHOW PLUGINS SONAME - dynamic report on installed components. ✔ exclusive to MariaDB 10 ● SHUTDOWN Command - apps can shutdown the database programmatically. ✔ exclusive to MariaDB 10 ● Kill Query by Query ID - fix performance blocks . ● Online ALTER TABLE - schema changes without downtime. ● Per-thread Memory Statistics - easier tuning and problem diagnosis. ✔ exclusive to MariaDB 10 ● Improved Error Messages - faster problem solving.✔ exclusive to MariaDB 10 Manage larger configurations with lower cost, less downtime.
© SkySQL Ab. Company Confiden al. MariaDB 10: Security: MySQL MariaDB
✘ ✔ Role-Based Access Control in 10.0.
Role: DBA
Permissions: ● Update Schema ● View Statistics DBA ● Create Database
MariaDB 10
Developer
Database Sysadmin Tables
© SkySQL Ab. Company Confiden al. MariaDB 10 Security: More Enhancements and Features
● Audit Plugin - Identify and correct potential security breaches. Comply with auditability requirements. ✔ included with MariaDB 10, supported by SkySQL
● PAM Authentication Module - delegate authentication to the popular Unix single-signon authentication API.✔ exclusive to MariaDB 10
Integrate the database with your security policies. Simplify administration and compliance.
© SkySQL Ab. Company Confiden al. MariaDB Audit Plugin 2.0 Confidential Roadmap
● Audit Plugin 1.1 includes table event logging such as triggers and stored procedure calls, unique to its integration with MariaDB. ● Building on these capabilities, V2.0 of the Audit Plugin is planned to include: ○ Optional field substitution of placeholders in query logs to improve privacy and security. ○ Integration with MariaDB 10 Role-based Access Control - filtering audit logs by role. ○ Recording of privilege changes as audit-able events. ○ Log rotation.
© SkySQL Ab. Company Confiden al. Confidential MariaDB 10 Roadmap
10.0.X Releases - updates to 10.0 over 2014. • Merge addi onal MySQL 5.6 changes. • Improvements to Fusion-io SSD atomic write support. • More thread pool improvements including Percona enhancements. • SHOW EXPLAIN support for PARTITIONS and EXTENDED a ributes. • CONNECT engine improvements. • Perform extensive replica on tes ng between MySQL 5.6 and MariaDB 10.0 to establish a more complete supportability matrix. • Open Query Graph (OQGRAPH) engine improvements. • Experimental Mroonga storage engine - full-text for Asian languages.
© SkySQL Ab. Company Confiden al. Confidential MariaDB 10 Roadmap 10.1.X Preliminary Major Feature List (all dot.dot versions): • Integra on of Galera mul -master clusters into MariaDB • Switch on/off Galera support in conf file • Merged features from MySQL, Percona, WebscaleSQL: • All func ons visible from SQL in 5.6. • Character set and colla on changes. • default_tmp_storage_engine. • Op mizer features including semi-joins+outer-joins, op mizer trace, EXPLAIN JSON, EXPLAIN FOR CONNECTION. • Percona features: per-query variables, userstat improvements, SHOW ENGINE INNODB STATUS improvements, NUMA improvements. • Selected 5.7 features: UNION ALL without temporary tables, mul ple triggers on table, new InnoDB, new Performance Schema instruments, MDL improvements, mul ple triggers per table, new InnoDB/XtraDB. • Merge WebscaleSQL improvements and integrate stress-test. • Kerberos authen ca on support • GIS improvements - full support for OGC compliance
• Windowing func ons © SkySQL Ab. Company Confiden al. Confidential MariaDB 10 Roadmap
10.1.X Preliminary Major Feature List, con nued: • Performance boosts: thread management, 16K InnoDB buffers, remote and portable tablespaces, op mizer enhancements, temporary table improvements, InnoDB persistent auto-increment, query meouts. • Performance tuning Parallel Replica on and GTID, row-level PR enhancements, slave execu on of triggers, more benchmarks. • Se ng to force InnoDB tables to have a primary key. • Role-based access control: features and performance improvements. • Progress report on mysqldump • Memory engine varchar and BLOB types • Community: move version control to GitHub (DONE).
© SkySQL Ab. Company Confiden al. Confidential MariaDB 10 Roadmap
10.1 and following - addi onal • More storage engines and feature ideas under interoperability with other inves ga on: DBs - poten ally: • Transac onal DDL • RocksDB/LevelDB • Indexes on expressions • InfiniDB • Field/Column type plug-in • MongoDB • JSON/BSON • Hadoop/HDFS • YAML • HBase/Honeycomb • IPv6/IPv4 • Message queues • UUID • Own datatype for Dynamic • Global DDL lock Columns
© SkySQL Corpora on Ab. Company Confiden al. MariaDB Enterprise
24/06/2014 © SkySQL Corpora on Ab. Company Confiden al. 35 MariaDB Enterprise: Integra ng the Complete Solu on DevOps Admin NoSQL Sharding Clustering Innovation MariaDB Enterprise Subscriptions Support Consulting Training RDBA Portal NRE
Partner Products Customer Care 36 © SkySQL Corpora on Ab. Company Confiden al. MariaDB Enterprise: What’s In A Subscrip on?
Benefit What you get beyond the core MariaDB server binaries Documentation Customer portal includes subscription-only material beyond KnowledgeBase. Connectors Tested by SkySQL, certified with ISVs. Product Roadmaps Roadmaps shared with subscribers, who have stronger influence on direction. Upgrades, Patches Subscribers receive proactive upgrade push, with release notes and advice. Helpdesk Support Subscribers receive consultative support and hot-fixes, beyond break/fix. Bundled Partner Selected partner tools offered as part of subscription. Tools
© SkySQL Corpora on Ab. Company Confiden al.