The New Mariadb Offering

The new MariaDB oﬀering MariaDB 10, MaxScale, and more

Rasmus Johansson, VP Engineering, SkySQL

24/06/2014 © SkySQL Corporaon Ab. Company Conﬁdenal. 1 MariaDB intro

24/06/2014 © SkySQL Corporaon Ab. Company Conﬁdenal. 2 What is MariaDB?

• A free fork of MySQL with extra features • features, requested by the users • Backward compable • file formats, replicaon, configuraon files • aim at 100% drop-in replacement • Community developed • 50% of maria-captains are from the community • no hidden agenda

• Distribuons: • RedHat Enterprise Linux (default in RHEL 7), Debian, Ubuntu (since 14.04), Fedora, Mageia (default), openSUSE (default), Gentoo, Slackware (default), Arch (default), ALTLinux, TurboLinux, Chakra Project (default), Kdu, … • FreeBSD, OpenBSD • Mac OS X with MacPorts or Homebrew • From MariaDB.org • sources, binaries in .tar.gz or .zip (Windows) • Windows MSI installer • MariaDB apt and yum repositories © SkySQL Corporaon Ab. Company Conﬁdenal. Before MariaDB 10.0

• MariaDB 5.1 – 17 releases since Oct 2009 • Aria, XtraDB, PBXT, FederatedX • table eliminaon, pool of threads, • MariaDB 5.2 – 15 releases since Apr 2010 • OQGraph, SphinxSE • virtual columns, pluggable auth, segmented key cache, extended stascs, • MariaDB 5.3 – 13 releases since July 2011 • major opmizer improvements • replicaon: group commit, checksums • HandlerSocket, dynamic columns, Windows performance improvements, microsecond support, GIS precise operaons, progress reporng, …

• MariaDB 5.5 – 20 releases since Feb 2012 • new thread pool • non-blocking client API • extended keys opmizaon • dynamic replicate_do_*, replicate_ignore_*, replicate_wild_* variables

• MaxScale • MariaDB Galera Cluster • Connectors/Drivers • MariaDB C Client Library (LGPL) • MariaDB JDBC driver (LGPL) • MariaDB ODBC driver (LGPL) • MariaDB Audit Plugin

• Past month (March 2014): • Added/Changed Arcles in the KB: 123 • On Freenode IRC #maria, 550 people wrote 10173 lines

• Source code (Launchpad): : GitHub • 25 acve branches MariaDB is now on • 395 commits https://github.com/MariaDB/server/

24/06/2014 9 © SkySQL Corporaon Ab. Company Conﬁdenal. Community Contribuons

MariaDB 10.0 major contribuons: • Spider storage engine metadata_lock_info Informaon schema • Per thread memory counng and usage • Code by Kentoku Shiba, Spiral Arms • Base code and idea by Lixun Peng, Taobao • Roles • Mul-source replicaon • Code by Vicenu Ciorbaru, Google Summer of Code 2013 • Base code by Lixun Peng, Taobao • PCRE Regular Expressions • GET_LOCK • Code by Sudheera Palihakkara, Google • Code by Konstann "Kostja" Osipov, Summer of Code 2013 mail.ru • Global Transacon IDs • CONNECT storage engine • Some patches by Pavel Ivanov, Google • Code by Olivier Bertrand

https://mariadb.com/kb/en/log-of-mariadb-contributions/

24/06/2014 © SkySQL Corporaon Ab. Company Conﬁdenal. 10 Community Ongoing Dev Projects

• WebScaleSQL patches • Stascally opmize mysql-test runs by hps://mariadb.atlassian.net/browse/MDEV-6039 running less tests, Pablo Estrada, Google Summer of Code • LevelDB/RocksDB storage engine (persistent key-value hps://mariadb.atlassian.net/browse/ store for fast storage) MDEV-5776 hps://mariadb.com/kb/en/leveldb-storage-engine/ • CREATE OR REPLACE, CREATE IF NOT • Fusion-IO page compression EXISTS, DROP IF EXISTS, Sriram Pal, hps://blog.mariadb.org/signiﬁcant-performance- Google Summer of Code hps://mariadb.atlassian.net/browse/ boost-with-new-mariadb-page-compression-on- MDEV-5359 fusionio/ • Self-Tuning Opmizer, Anshu Avinash, • Kerberos authencaon plugin Google Summer of Code hps://mariadb.atlassian.net/browse/MDEV-4691 hps://mariadb.atlassian.net/browse/ • Mroonga storage engine (fast fulltext search) MDEV-350 hps://mariadb.atlassian.net/browse/MDEV-5222 • Support of UseServerPrepStmts to MariaDB Java Connector, Puneet Dewan, Google Summer of Code

24/06/2014 © SkySQL Corporaon Ab. Company Conﬁdenal. 11 MariaDB 10.0

● MariaDB 10.0 GA launched March 31st, 2014. ● Application-compatible with MySQL. ● Includes signiﬁcant unique new features in scalability, replication, performance, NoSQL, operations, security. ● Available under the GPL v2.

© SkySQL Ab. Company Conﬁdenal. MariaDB 10 Scalability: MySQL MariaDB ✘ ✔ Single in 10.0. Parallel Slave Replicaon threaded per database. ● Sponsored by Google. ● Allows slaves to process update events in parallel. ● Uses MariaDB 10’s improved Global Transaction ID (GTID). ● MariaDB unlike MySQL can process multiple updates for a single database in parallel. Much better throughput. ● Preliminary benchmarks: almost 10x faster at 12 threads. Improves scalability, reduces slave lag, making slaves more consistent.

sysbench OLTP single database slave tps relative to master

Content Online E- Management Commerce System Application

Master Master ● Collects data for S S S S S S S S analytics using built-in replication. Click-stream data

Master ● Aids in administration S S S S example: consolidated backups of multiple databases. ● Uses MariaDB 10’s improved

Global Transaction ID (GTID). Slave ETL Data Warehouse

Easier analytics, more insight, simpler administration, fewer headaches.

MariaDB 10 Scalability: MySQL MariaDB ✘ ✔ Sharding with Spider optional 3rd in 10.0. party install

● Contributed by a 3rd party developer. Application ● Storage engine partitions large tables

across multiple DBs. Spider

● No changes to the client application. Customers ● Developers aren’t aware, can keep A-H I-P applications simple. Q-Z ● Allows for more

parallelism, scale-out. Shard 1 Shard 2 Shard 3

● Transactional storage Customers Customers Customers engine. A-H I-P Q-Z

Web-scale, without the development hassle.

● 3rd party partner - TokuTek. ● Drop-in replacement for InnoDB/XtraDB. ● Advanced indexing and compression algorithms. ● Transactional. ● Up to 20x performance gain for inserts/updates. ● Up to 90% less disk storage. ● Online schema changes. ● Reduces or eliminates slave lag. ● Best when DB doesn’t ﬁt in memory, and SSD longevity is important. © SkySQL Ab. Company Conﬁdenal. MariaDB 10 Performance: Opmizer Improvements ● Of 29 distinct enhancements noted, 28 are in MariaDB 10. Just 1 only in MySQL 5.6. ● Enhancements include: ○ Disk access optimizations. ○ JOIN optimizations. ○ Subquery optimizations. ○ Optimized derived tables and views. ○ Execution control. ○ Optimizer control. ○ EXPLAIN improvements.

Less I/O, CPU, memory requirements. Faster execution. © SkySQL Ab. Company Conﬁdenal. MariaDB 10 Performance: More Enhancements and Features

● Fusion-io Atomic Writes - improves performance on popular SSD hardware.✔ exclusive to MariaDB 10 ● XtraDB storage engine - enhanced from InnoDB. ● Improved InnoDB storage engine - based on MySQL 5.6. ● Performance schema - real-time performance management. ● Improved thread pool - better concurrent user performance.✔ exclusive to MariaDB 10 ● More optimizer improvements - higher eﬃciency in I/O accesses. ✔ exclusive to MariaDB 10

Faster performance, lower cost.

• Atomic writes gives a performance increase of about 30%. By enabling fast checksum for XtraDB it’s 50% • By using page compression the compression ratio is leading to better performance and there are less writes to disk. • Multi-threaded flush provides better throughput and decreases operation latencies delivering a performance boost

https://blog.mariadb.org/significant-performance-boost-with-new-mariadb-page-compression-on-fusionio

MariaDB 10 Interoperability: MySQL MariaDB CONNECT Storage Engine ✘ ✔ in 10.0.

● From 3rd party developer. Application ● Maps diverse data to tables. MariaDB Parser/Optimizer/Connection Pool ● JOIN mapped data Spi der to DB tables. CONNECT Other Engine Engines ● Flat ﬁles including CSV. ● Tables in external DBs. ● Generated tables

(PIVOT etc.) .log

CSV Database ● Plug-in API for your own mappings. XML Tables

Powerful tool for data integration, federation. © SkySQL Ab. Company Conﬁdenal. Technology Preview

MariaDB 10 Interoperability: MySQL MariaDB Cassandra Storage Engine ✘ ✔ in 10.0.

● Window into a Cassandra ring: Application read/write like a table in MariaDB. ● Use standard SQL queries. ● JOIN Cassandra data MariaDB Parser/Optimizer/ConnectionSpi Pool to MariaDB tables. der Cassandra Other Engine Engines ● Use a MariaDB cluster for high-availability access. ● Bring data from

Cassandra into OLTP Database applications. Tables Interoperate with Cassandra. Use Cassandra data in OLTP applications.

Cust ID Account Balance Dyn_Col_BLOBs

2035 $154.04 NAME: John Smith|LOC: 45.35243, -74.98348|IMAGE: x27A8B8C ...

2036 $929.10 NAME: Jane Doe|LOC: 45.35243, -74.98348|AGE: 32| GENDER: F...

2037 $377.53 NAME: Carol Jones|AGE: 43|GENDER: F||IMAGE: xA9674DE678 ...

● Store unstructured data in MariaDB tables with a simple API. ● Use MariaDB’s indexing and transactions to manipulate “document” style data fast and consistently. ● Nest sets of dynamic columns inside of other dynamic columns - hierarchical structuring. ● Include multiple rows with dynamic columns in transactions. Process unstructured data in the same way as NoSQL,but with the power of MariaDB.

Transactional NoSQL ● Simple, low-overhead Application Application

API direct to the SQL Statements Simple CRUD API storage engine. Connection Pool ● Bypasses SQL Spi SQL Interface statement processing. der Parser

● Less CPU, memory, Optimizer HandlerSocket Plugin I/O, network demands. InnoDB/XtraDB and Spider Storage Engines ● Batches requests for even less I/O operations. ● Building block for application Database level NoSQL processing. Tables

Mix RDBMS applications with high performance NoSQL applications. Leverage MariaDB’s storage engine architecture for both. © SkySQL Ab. Company Conﬁdenal. MariaDB 10 Operations: MySQL MariaDB ✘ ✔ SHOW EXPLAIN Command in 10.0.

Oh! Sequential scan of a 18M row Why is this so table… no wonder! slow? Lets ﬁnd I’ll let the team To: DevOps Team out… SHOW know... EXPLAIN FOR 28; From: DBA Debby

Subj. Slow Queries

Hey guys, ﬁgured out those blocking queries. The new reservation app is looking for the shortest route but that column has no index - we need one! I’ll add it now. Debby Thread Time

27 .003761 ID select_type table possible_ rows keys And it works in the slow query log too! 28 34.2529 1 SIMPLE tbl NULL 1855174 Also for UPDATE and 29 .000713 DELETE. © SkySQL Ab. Company Conﬁdenal. MariaDB 10 Operations: More Enhancements and Features

● Improved Table Discovery - less work for DBAs; integrates storage engines. ✔ exclusive to MariaDB 10 ● SHOW PLUGINS SONAME - dynamic report on installed components. ✔ exclusive to MariaDB 10 ● SHUTDOWN Command - apps can shutdown the database programmatically. ✔ exclusive to MariaDB 10 ● Kill Query by Query ID - ﬁx performance blocks . ● Online ALTER TABLE - schema changes without downtime. ● Per-thread Memory Statistics - easier tuning and problem diagnosis. ✔ exclusive to MariaDB 10 ● Improved Error Messages - faster problem solving.✔ exclusive to MariaDB 10 Manage larger conﬁgurations with lower cost, less downtime.

✘ ✔ Role-Based Access Control in 10.0.

Role: DBA

Permissions: ● Update Schema ● View Statistics DBA ● Create Database

MariaDB 10

Developer

Database Sysadmin Tables

● Audit Plugin - Identify and correct potential security breaches. Comply with auditability requirements. ✔ included with MariaDB 10, supported by SkySQL

● PAM Authentication Module - delegate authentication to the popular Unix single-signon authentication API.✔ exclusive to MariaDB 10

Integrate the database with your security policies. Simplify administration and compliance.

● Audit Plugin 1.1 includes table event logging such as triggers and stored procedure calls, unique to its integration with MariaDB. ● Building on these capabilities, V2.0 of the Audit Plugin is planned to include: ○ Optional ﬁeld substitution of placeholders in query logs to improve privacy and security. ○ Integration with MariaDB 10 Role-based Access Control - ﬁltering audit logs by role. ○ Recording of privilege changes as audit-able events. ○ Log rotation.

10.0.X Releases - updates to 10.0 over 2014. • Merge addional MySQL 5.6 changes. • Improvements to Fusion-io SSD atomic write support. • More thread pool improvements including Percona enhancements. • SHOW EXPLAIN support for PARTITIONS and EXTENDED aributes. • CONNECT engine improvements. • Perform extensive replicaon tesng between MySQL 5.6 and MariaDB 10.0 to establish a more complete supportability matrix. • Open Query Graph (OQGRAPH) engine improvements. • Experimental Mroonga storage engine - full-text for Asian languages.

© SkySQL Ab. Company Confidenal. Confidential MariaDB 10 Roadmap 10.1.X Preliminary Major Feature List (all dot.dot versions): • Integraon of Galera mul-master clusters into MariaDB • Switch on/off Galera support in conf file • Merged features from MySQL, Percona, WebscaleSQL: • All funcons visible from SQL in 5.6. • Character set and collaon changes. • default_tmp_storage_engine. • Opmizer features including semi-joins+outer-joins, opmizer trace, EXPLAIN JSON, EXPLAIN FOR CONNECTION. • Percona features: per-query variables, userstat improvements, SHOW ENGINE INNODB STATUS improvements, NUMA improvements. • Selected 5.7 features: UNION ALL without temporary tables, mulple triggers on table, new InnoDB, new Performance Schema instruments, MDL improvements, mulple triggers per table, new InnoDB/XtraDB. • Merge WebscaleSQL improvements and integrate stress-test. • Kerberos authencaon support • GIS improvements - full support for OGC compliance

10.1.X Preliminary Major Feature List, connued: • Performance boosts: thread management, 16K InnoDB buﬀers, remote and portable tablespaces, opmizer enhancements, temporary table improvements, InnoDB persistent auto-increment, query meouts. • Performance tuning Parallel Replicaon and GTID, row-level PR enhancements, slave execuon of triggers, more benchmarks. • Seng to force InnoDB tables to have a primary key. • Role-based access control: features and performance improvements. • Progress report on mysqldump • Memory engine varchar and BLOB types • Community: move version control to GitHub (DONE).

10.1 and following - addional • More storage engines and feature ideas under interoperability with other invesgaon: DBs - potenally: • Transaconal DDL • RocksDB/LevelDB • Indexes on expressions • InﬁniDB • Field/Column type plug-in • MongoDB • JSON/BSON • Hadoop/HDFS • YAML • HBase/Honeycomb • IPv6/IPv4 • Message queues • UUID • Own datatype for Dynamic • Global DDL lock Columns

24/06/2014 © SkySQL Corporaon Ab. Company Conﬁdenal. 35 MariaDB Enterprise: Integrang the Complete Soluon DevOps Admin NoSQL Sharding Clustering Innovation MariaDB Enterprise Subscriptions Support Consulting Training RDBA Portal NRE

Benefit What you get beyond the core MariaDB server binaries Documentation Customer portal includes subscription-only material beyond KnowledgeBase. Connectors Tested by SkySQL, certified with ISVs. Product Roadmaps Roadmaps shared with subscribers, who have stronger influence on direction. Upgrades, Patches Subscribers receive proactive upgrade push, with release notes and advice. Helpdesk Support Subscribers receive consultative support and hot-fixes, beyond break/fix. Bundled Partner Selected partner tools offered as part of subscription. Tools