Operating and Troubleshooting with MySQL Percona Training http://www.percona.com/training © 2011 – 2013 PERCONA Table of Contents 1. Introduction 8. Hardware and OS 2. Basics and Tools 9. InnoDB Basics 3. Backup and Recovery 10. InnoDB Internals 4. Replication 11. Concurrency & Locking 5. Server Troubleshooting 12. InnoDB Diagnostics 6. Monitoring 13. Wrap-Up 7. Security © 2011 – 2013 PERCONA 2 Welcome Everybody! • Thank you for choosing a #Percona Training course. • We also offer additional services: – Consulting – Support – Remote DBA – Development • We support all MySQL distributions, including Oracle MySQL, Maria DB and Percona Server © 2011 – 2013 PERCONA 3 House Keeping • Bathrooms • When do we break for lunch? • When does the day end? • Can I get a copy of the slides? – Yes: http://learning.percona.com © 2011 – 2013 PERCONA 4 What You Need to Already Know • Basic MySQL (DDL, DML, etc.) • How to tune queries (using EXPLAIN) • Basic operating systems (Linux) and system administration concepts and commands – processes, threads, ssh, top, rsync, nc, ... • Basic hardware understanding – RAID, SSD vs. HDD, cache, filesystems, ... © 2011 – 2013 PERCONA 5 Content in Detail • Course introduction (this!) • Basics and Tools – Storage engines and its main features – MySQL history, versions and current support – Different MySQL flavors and forks – MySQL provided tools – Essential third party tools (incl. Percona Toolkit) © 2011 – 2013 PERCONA 6 Content in Detail (cont.) • Backup and Recovery – Logical backups (mysqldump, mydumper) – Physical backups (mylvmbackup, XtraBackup) – Backups strategies and validation • Replication – Replication internals – Setup and administration – Troubleshooting Replication, helper tools and alternatives © 2011 – 2013 PERCONA 7 Content in Detail (cont.) • Server Troubleshooting – Understanding hardware components that affect MySQL performance – Learning 3rd party tools to profile server performance – Evaluating current status of the server to find common bottlenecks and increasing performance by changing the configuration (SHOW GLOBAL STATUS) • Monitoring – Best practices for monitoring – Alerting Tools (Nagios) – Trending tools (Cacti) © 2011 – 2013 PERCONA 8 Content in Detail (cont.) • Security – Configuration Best practices for Security – Authentication and authorization in MySQL – Potential security threats for MySQL • Hardware and operating systems – Discussion on best practices for hardware purchase and operating system tuning © 2011 – 2013 PERCONA 9 Content in Detail (cont.) • InnoDB internals: – On Disk Information: tablespaces, innodb-file-per-table, row formats, log files, undo space – In Memory information: Buffer pool and other structures; – Logical InnoDB organization for data and indexes: Clustered index – Advanced structures: adaptive hash, insert buffer and double write – Essential configuration parameters © 2011 – 2013 PERCONA 10 Content in Detail (cont.) • Concurrency and Locking: – MVCC implementation and isolation levels – Old data purging – Locking and its performance implications – Maintenance tasks and background threads • InnoDB Diagnostics – Solving issues and improving performance of InnoDB by using SHOW ENGINE INNODB STATUS, SHOW GLOBAL STATUS, INFORMATION_SCHEMA tables and other tools, including Percona XtraDB extensions © 2011 – 2013 PERCONA 11 Useful Resources • Online – MySQL Reference Manual – MySQL Internals Manual: InnoDB Storage Engine – MySQL Performance Blog • Books – High Performance MySQL, 3rd edition © 2011 – 2013 PERCONA 13 MySQL Basics and Tools Percona Training http://www.percona.com/training © 2011 – 2013 PERCONA 1 Table of Contents 1. Storage Engines 4. Essential Third Party DBA Tools 2. MySQL Versions and Support 5. Percona Toolkit 3. MySQL Supplied Tools © 2011 – 2013 PERCONA 2 InnoDB Basics and Tools STORAGE ENGINES © 2011 – 2013 PERCONA 3 Storage Engines • MySQL Separates SQL from Storage. – Replication, Partitioning, Stored Procedures all happen above the storage engine layer. – Storage happens in the Storage Engines. • The most popular storage engine is InnoDB. – The default is InnoDB as of MySQL 5.5. – For 99% of people MyISAM is probably the wrong choice. © 2011 – 2013 PERCONA 4 Storage Engines: InnoDB • Most popular, default since MySQL 5.5. • Row-level locking. • ACID transactions. • Automatic crash recovery. • Caching data and indexes. • Referential integrity (foreign keys). • Better performance and scalability when tuned well. • Fulltext indexing in MySQL 5.6. © 2011 – 2013 PERCONA 5 Storage Engines: MyISAM • Default storage engine until MySQL 5.1. • Table-level locking. • Relies on filesystem caching—risk of corruption. • Fulltext indexing. • GIS indexing. © 2011 – 2013 PERCONA 6 Storage Engines: Others • MEMORY – Stores data in volatile memory. • BLACKHOLE – Stores no data, like /dev/null. Very fast! Small footprint! – Useful as a dummy target, while DML is written to the binlog. • CSV – Stores data in text files using a comma-separated value format. • ARCHIVE – Store large amounts of unindexed data with transparent compression. – Supports only INSERT and SELECT. © 2011 – 2013 PERCONA 7 Storage Engines: Not Recommended • MERGE – Interface to a collection of identical MyISAM tables as one table. – Use Partitioning instead. • FEDERATED – Lets you access data from remote MySQL instances without using replication or cluster technology. – Roughly analogous to Oracle Database Links. – Not recommended; stability and performance issues. © 2011 – 2013 PERCONA 8 Changing a Table’s Storage Engine • Simple to convert: mysql> ALTER TABLE name ENGINE=InnoDB; • It performs a table restructure (just like many ALTER statements do), and the table is locked for the duration. • Test carefully—you could truncate data or lose table details if data types or index types are not supported in the new storage engine. © 2011 – 2013 PERCONA 9 InnoDB Basics and Tools MYSQL VERSIONS AND SUPPORT © 2011 – 2013 PERCONA 10 Show of Hands • What version of MySQL is everyone using? – MySQL 5.0 or earlier? – MySQL 5.1? – MySQL 5.5? – MySQL 5.6? – ... a third party release? © 2011 – 2013 PERCONA 11 MySQL Lifetime Support Policies Extended Support Sustaining Support Release GA Date Premier Support End End End MySQL Database 5.0 Oct 2005 Dec 2011 Not available Indefinite MySQL Database 5.1 Dec 2008 Dec 2013 Not available Indefinite MySQL Database 5.5 Dec 2010 Dec 2015 Dec 2018 Indefinite MySQL Database 5.6 Feb 2013 Feb 2018 Feb 2021 Indefinite MySQL Cluster 6 Aug 2007 Mar 2013 Not available Indefinite MySQL Cluster 7 Apr 2009 Apr 2014 Not available Indefinite http://www.oracle.com/us/support/library/lifetime-support-technology-069183.pdf © 2011 – 2013 PERCONA 12 Upgrading • Migrating from 5.1.xx to 5.1.yy “should be safe”. – Internally MySQL has a policy of “no new features in a point release”*. – The hardest regressions to catch are performance and optimizer related. They affect everyone differently. * This policy is occasionally broken. © 2011 – 2013 PERCONA 13 Upgrading (cont.) • Migrating between major releases (e.g. 5.5.x to 5.6.x) is not guaranteed to be safe. – Features can be added or removed. – The on-disk format can change in an incompatible way. • Incompatibilities are documented: – http://dev.mysql.com/doc/refman/5.0/en/upgrading-from-previous-series.html – http://dev.mysql.com/doc/refman/5.1/en/upgrading-from-previous-series.html – http://dev.mysql.com/doc/refman/5.5/en/upgrading-from-previous-series.html – http://dev.mysql.com/doc/refman/5.6/en/upgrading-from-previous-series.html © 2011 – 2013 PERCONA 14 Version Number Soup • It is possible to get confused looking at numbers: – MySQL Cluster (NDB) has its own versioning scheme; it is currently up to 7.2. – 5.2 was rebranded as 6.0; but 6.0 was canceled. – 5.4 was created to replace 6.0; some code in 6.0 was backported. – 5.4 was rebranded as 5.5. – InnoDB (plugin) now has its own versioning: • InnoDB 1.0.x for MySQL 5.1 • InnoDB 1.1.x for MySQL 5.5 © 2011 – 2013 PERCONA 15 MySQL in Linux Repositories Version RHEL CentOS Ubuntu Debian Open Rpmfusi Webtati Mariadb Percona SuSE on.org c.com .com .com MySQL 5.0 5.x 5.x 8.04 -- 11.0 -- -- -- Yes1 11.1 MySQL 5.1 6.x 6.x 10.04 6.0 11.2 -- -- -- Yes1 11.10 (squeeze) 11.3 11.4 MySQL 5.5 -- -- 12.04 (wheezy) 12.2 Yes Yes Yes2 Yes1 12.10 13.04 MySQL 5.6 -- -- -- -- -- -- -- -- Yes1 1 Percona.com offers Percona Server packages. 2 Mariadb.com offers MariaDB packages. © 2011 – 2013 PERCONA 16 Forks and Patches Sun Microsystems Oracle Oracle MySQL 5.1 MySQL 5.5 MySQL 5.6 MyISAM InnoDB InnoDB InnoDB InnoDB (builtin) Plugin 1.0 Plugin 1.1 Plugin 1.2 Aria XtraDB XtraDB XtraDB Percona Server 5.1 Percona Server 5.5 Percona Server 5.6 MariaDB 5.1 MariaDB 5.5 (alpha) Drizzle MariaDB 10 © 2011 – 2013 PERCONA 17 Percona Server • At the MySQL level, very little is changed. – There is some additional instrumentation in the slow query log and some additional user statistics. • As newer versions of MySQL Server are released, Percona will rebase its patches against them. – It’s not a true fork. – On disk format remains the same*. – There is no intention to rebase against every new MySQL release, and no promises as to reaction time when security vulnerabilities are found. * Unless some specific features are enabled. © 2011 – 2013 PERCONA 18 Percona Server (cont.) • Main change is in storage engines. – Percona Server is built with XtraDB, a modified version of the InnoDB storage engine. • XtraDB has a number of performance and usability enhancements over InnoDB. – But it itself is not a true fork