MySQL für Einsteiger
Philipp Michaly
@michaly_philipp doag2018 Wir helfen Mehrwerte aus Daten zu generieren
2 11/16/2018 Trivadis DOAG18: MySQL für Einsteiger Mit über 650 IT- und Fachexperten bei Ihnen vor Ort.
16 Trivadis Niederlassungen mit über 650 Mitarbeitenden. Erfahrung aus mehr als 1'900 Projekten pro Jahr bei über 800 Kunden. Über 250 Service Level Agreements. Mehr als 4'000 Trainingsteilnehmer. Forschungs- und Entwicklungsbudget: CHF 5.0 Mio. / EUR 4.0 Mio. Finanziell unabhängig und nachhaltig profitabel.
3 11/16/2018 Trivadis DOAG18: MySQL für Einsteiger mysql@DOAG18:~/ echo $speaker Philipp Michaly Senior Consultant, Trivadis GmbH, München . Seit 2004 im IT Ops & Engineering . Seit 2010 bei Trivadis . Seit 2016 Fokus auf MySQL
IT Erfahrung Skills . Oracle DB Administration . Backup & Recovery . MySQL Administration . Performance Analysen . OS Administration (Linux, Unix) . MySQL Architektur . Infrastruktur . Referent für MySQL Kurs
2 11/16/2018 Trivadis DOAG18: MySQL für Einsteiger Agenda
1. Considerations about ODBMS
2. Introducing MySQL
3. Oracle DBA tricks for MySQL
5 11/16/2018 Trivadis DOAG18: MySQL für Einsteiger Considerations about ODBMS
6 11/16/2018 Trivadis DOAG18: MySQL für Einsteiger Considerations about ODBMS – Gartner Study
“Open-source relational DBMSs have matured significantly and can be used to replace commercial RDBMSs at a considerable TCO saving. Information leaders, DBAs and application development management can now consider them as a standard choice for deploying applications.” The State of Open-Source RDBMSs, 2015 - Gartner
7 11/16/2018 Trivadis DOAG18: MySQL für Einsteiger Considerations about ODBMS – Gartner Study
Gartner Strategic assumptions for 2018:
• More than 70% of new in-house applications will be developed on an OSDBMS • 50% of existing commercial RDBMS instances will have been converted or will be in process
8 11/16/2018 Trivadis DOAG18: MySQL für Einsteiger Considerations about ODBMS – Gartner Study
Significant maturity Progress for OSDBMS in the past years:
2009 2015
9 11/16/2018 Trivadis DOAG18: MySQL für Einsteiger Considerations about ODBMS – Why MySQL?
Source: https://db-engines.com/de/ranking 10 11/16/2018 Trivadis DOAG18: MySQL für Einsteiger Considerations about ODBMS – Why MySQL?
11 11/16/2018 Trivadis DOAG18: MySQL für Einsteiger Considerations about ODBMS – Why MySQL?
Source: https://db-engines.com/de/ranking_trend 12 11/16/2018 Trivadis DOAG18: MySQL für Einsteiger Considerations about ODBMS – Why MySQL?
13 11/16/2018 Trivadis DOAG18: MySQL für Einsteiger Introducing MySQL
14 11/16/2018 Trivadis DOAG18: MySQL für Einsteiger Introducing MySQL – History of MySQL
1994 1997 • MySQL project started by Michael • Official MySQL 3.1 release Widenius (Monty), David Axmark and • Releases as binary distribution for Allan Larsson Linux and Solaris • For the first 6 years commercially • Based on „matured“ internal kernel represented by TCX, predecessor of from 23th Mai 1995 MySQL AB • Focus on performance and large tables • Stability was not the primary target
15 11/16/2018 Trivadis DOAG18: MySQL für Einsteiger Introducing MySQL – History of MySQL
2000 2003 2004 • MySQL goes Open Source • MySQL 4.0 released • MySQL 4.1 released and releases software under a dual license in the terms of the GPL • Foundation and transition to MySQL AB • Revenues dropped 80% as a result, and it took a year to make up for it. • Release of MySQL 3.23 • Introducing InnoDB and Replication
16 11/16/2018 Trivadis DOAG18: MySQL für Einsteiger Introducing MySQL – History of MySQL
2005 2006 • MySQL 5.0 released • Oracle tries to buy MySQL and fails • Introducing: • Buys Sleepycat instead • Views, Triggers, Stored Procedures, • MySQL is estimated to have a 33% User defined Functions market share measured in install base • Now supporting all SQL3-Standard and 0.2% market share measured in Objects revenue (the database market was a • Oracle buys Innobase, the 4-person $15 billion market in 2006). Finnish company behind MySQL's InnoDB storage backend.
17 11/16/2018 Trivadis DOAG18: MySQL für Einsteiger Introducing MySQL – History of MySQL
2008 2009 2010 • MySQL 5.1 released, • Oracle acquires Sun • MySQL 5.5 released Features Partitioning Microsystems for • InnoDB now default • Sun Microsystems approximately 7.4 storage engine acquires MySQL AB for Billion • Performance Schema approximately $1 introduced billion. • Michael Widenius (Monty) and David Axmark, two of MySQL AB's co-founders, leave Sun shortly after the acquisition.
18 11/16/2018 Trivadis DOAG18: MySQL für Einsteiger Introducing MySQL – The Present
2012 2015 • MySQL 5.6 released • MySQL 5.7 released • Replication improvements • Security enhancements „secure (Multithreaded slaves, Crash- by default“ Safe slaves) • Significant optimizer improvements
19 11/16/2018 Trivadis DOAG18: MySQL für Einsteiger Introducing MySQL – The Present
2016 (September): MySQL 8.0.0 Milestone Release available • GA Release released in 2018-04-19
Release 8.0 features • Roles • Transactional Data Dictionary • Invisible indexes • Persistent global variables • Optimizer improvements (Histograms, advanced cost model, improved range scan performance…) • New plugin infrastructure (Plugins as a service) • IO performance improvements • Much more
20 11/16/2018 Trivadis DOAG18: MySQL für Einsteiger Introducing MySQL - MySQL Editions
MySQL Community Edition Free and opensource edition (GPL) No commercial support
Linux Distribution Editions The most distributions ship modified non-standard version Modifications are not very transparent The only reliable source for original and unmodified MySQL binaries is Oracle
MySQL Commercial Editions Standard Edition: Opensource version + Oracle Premier Support Enterprise Edition: Additional enterprise features + Oracle Premier Support
21 11/16/2018 Trivadis DOAG18: MySQL für Einsteiger Introducing MySQL– Enterprise Edition
MySQL Enterprise Edition Features
Oracle Premier Monitoring Backup Encryption Audit Support
MySQL Enterprise MySQL Enterprise MySQL Enterprise MySQL Enterprise 24x7 support TDE (Transparent Monitor Backup Audit Data Encryption)
Access to My Cloud Control Oracle Support Plugin
22 11/16/2018 Trivadis DOAG18: MySQL für Einsteiger Introducing MySQL – Forks and Branches
Maria DB • Popularity leadership in opensource community • Big Players migrating from MySQL to MariaDB • Oracle SQL Mode!
Percona Sever • Focus on Performance improvements + performance stability • Own administration toolkit • XtraDB storage Engine
23 11/16/2018 Trivadis DOAG18: MySQL für Einsteiger Introducing MySQL – Forks and Branches
Amazon Aurora – MySQL Edition • Forked from MySQL 5.6 • Closed source due to hole in GPL • Completely new IO redesign • Promising performance
24 11/16/2018 Trivadis DOAG18: MySQL für Einsteiger Introducing MySQL – MySQL Architecture MySQL Layer 1 MySQL Layer 2
Application NoSQL SQL- MySQL- Connection/Thread Manager Connector Statement Connector Plugin 1 SQL Processing
Query Parser Plugin 2 Cache Query Cache Buffer Plugin 3 Security Execution Optimizer . Manager Engine . Storage-Engine API . More Transaction Management plugins
Recovery Management
Referential Integrity
MySQL Layer 3 Segment Management Redo,Undo, Datafiles Binary Logs Memory Memory Management
Buffer cache
Manager
rd Party Vendors Party rd
InnoDB 3 MyISAM .. Memory
26 11/16/2018 Trivadis DOAG18: MySQL für Einsteiger Introducing MySQL- HA Architectures
Active/Passive • Oracle: Clusterware (Failover) Cluster: • MySQL: MySQL Replication, Clusterware
Master/Slave • Oracle: Dataguard • MySQL: Synchronous, Semi-Synchronous Replication (DG Replication: Fast-Sync), Synchronous Replication (MySQL Cluster only)
Active/Active • Oracle: Real Application Cluster • MySQL: MySQL Cluster: Multi Master Replication and Cluster: Active/Active Cluster
31 11/16/2018 Trivadis DOAG18: MySQL für Einsteiger HA Architectures - Single Instance Architecture
Connectors Native C API, JDBC, ODBC... R/W
VM 1 VM 2
Prod_DB 1
Test_DB1
Dev_DB1
32 11/16/2018 Trivadis DOAG18: MySQL für Einsteiger HA Architectures - Performance + Data Availability
Connectors Native C API, JDBC, ODBC... R/W RO
VM 1 Asynch. Replication VM 2 Master_Prod_DB 1 Slave_Prod_DB1
Test_DB1
Dev_DB1
33 11/16/2018 Trivadis DOAG18: MySQL für Einsteiger HA Architectures – Asynchronous Replication
Source: https://dev.mysql.com/doc/refman/8.0/en/group-replication-primary-secondary-replication.html
34 11/16/2018 Trivadis DOAG18: MySQL für Einsteiger HA Architectures - Maximum Safety
Connectors Native C API, JDBC, ODBC... R/W RO
VM 1 Semi Synch. VM 2 Master_Prod_DB 1 Replication Slave_Prod_DB1
Test_DB1
Dev_DB1
35 11/16/2018 Trivadis DOAG18: MySQL für Einsteiger HA Architectures – Semisynchronous Replication
Source: https://dev.mysql.com/doc/refman/8.0/en/group-replication-primary-secondary-replication.html
36 11/16/2018 Trivadis DOAG18: MySQL für Einsteiger HA Architectures - InnoDB Cluster
Replicate data within the Cluster
Automatic failover
Transparent auto failover
Read scalability
Load balancing
Source: https://dev.mysql.com/doc/refman/8.0/en/mys ql-innodb-cluster-introduction.html 37 11/16/2018 Trivadis DOAG18: MySQL für Einsteiger HA Architectures – Group Replication
Source: https://dev.mysql.com/doc/refman/8.0/en/group-replication-summary.html
38 11/16/2018 Trivadis DOAG18: MySQL für Einsteiger Introducing MySQL - Licensing
Server Setup: X86, 2 Sockets, each CPU 6 cores
Number of Yearly per License per Yearly Total Three Year License Metric License Fee sockets/cores Metric Server Support Cost Oracle Database 2 per socket $17,500 $3,850 $35,000 $7,700 $58,100 Standard Edition 2 Oracle Database 12 0.5 per core $47,500 $10,450 $285,000 $62,700 $473,100 Enterprise Edition MySQL Enterprise n/a Server n/a $5,000 n/a $5,000 $15,000 Edition
Source: Oracle´(November 2018)
39 11/16/2018 Trivadis DOAG18: MySQL für Einsteiger Introducing MySQL– Installing MySQL
3 possible ways to install MySQL:
1. Package Manager installation (apt, yum, yast, dpkg, rpm…) yum install mysql-commercial-server.x86_64
Pros: • Simplicity of maintenance • Comfortable Patch/Upgrade/Removal maintenance Cons: • Hard coded default locations • Sensitive repository maintenance
40 11/16/2018 Trivadis DOAG18: MySQL für Einsteiger Introducing MySQL– Installing MySQL
2. Pre-built binaries (binary tarballs) tar –xzfbv Archive.tar.gz
Pros: • High flexibility • Easier upgrade/downgrade management • Custom locations Cons: • Manage dependency issues manually • A bit more manual work
41 11/16/2018 Trivadis DOAG18: MySQL für Einsteiger Introducing MySQL– Installing MySQL
3. Custom built binaries (Compile your own) cmake --build .
Pros: • Maximum freedom to modify mysql
Cons: • Requires engineering effort • Release/Patch Maintenance is enormous
42 11/16/2018 Trivadis DOAG18: MySQL für Einsteiger Introducing MySQL – Schema Concept 1/3
MySQL: Schema/Database
• Schema is a Synonym for Database
• Database/Schema is a container for objects
• Database/Schema is user independent
43 11/16/2018 Trivadis DOAG18: MySQL für Einsteiger Introducing MySQL – Schema Concept 2/3
mysql> show databases; +------+ | Database | +------+ | information_schema | | mysql | | performance_schema | | s1 | | sakila | | sys | | test | | world_x | +------+
44 11/16/2018 Trivadis DOAG18: MySQL für Einsteiger Introducing MySQL – Schema Concept 3/3 mysql> use test;
Database changed mysql> show tables; +------+ | Tables_in_test | +------+ | badges | | comments | | post_history | | post_links | | posts | | tags | | users | | votes | +------+
45 11/16/2018 Trivadis DOAG18: MySQL für Einsteiger Introducing MySQL - Start and Stop MySQL
Control MySQL Instance with Systemd:
• Status systemctl status db1
• Start/Stop systemctl start/stop db1
• Restart systemctl restart db1
46 11/16/2018 Trivadis DOAG18: MySQL für Einsteiger Introducing MySQL: Backup & Recovery
Logical Backup:
• mysqldump + mysqlump (datapump)
Physical Backup: MySQL Enterprise Backup (MEB)
• Incremental Backup • Mulitple IO Threads • Compression • TDE support • Much more additional Features • See https://www.mysql.de/products/enterprise/backup.html
Point in time recovery: • Manually with mysqlbinlog
47 11/16/2018 Trivadis DOAG18: MySQL für Einsteiger Oracle DBA tricks for MySQL
48 11/16/2018 Trivadis DOAG18: MySQL für Einsteiger Oracle DBA tricks for MySQL
Trick 1: Adapt the Oracle SID to MySQL
• MySQL datadir variable offers local uniqueness mysql> select @@datadir; +------+ | @@datadir | +------+ |/u01/mysqldata/BRONKO/ | +------+
49 11/16/2018 Trivadis DOAG18: MySQL für Einsteiger Oracle DBA tricks for MySQL
• Simple string modifications generate a SID like identifier mysql> set @sid=replace(@@datadir,'/u01/mysqldata/',''); Query OK, 0 rows affected (0.00 sec) mysql> set @sid=replace(@sid,'/',''); Query OK, 0 rows affected (0.00 sec) mysql> select @sid; +------+ | @sid | +------+ |BRONKO | +------+
50 11/16/2018 Trivadis DOAG18: MySQL für Einsteiger Oracle DBA tricks for MySQL
Trick 2: Adapt the Oracle Optimal Flexible Architecture to MySQL
• Much more flexibility for: • Mutli Instance Setups • In-Place Upgrades • Filesystem resizes
MySQL_BASE: /u00/app/mysql/… MySQL_HOME /u00/app/mysql/product/8.0.* MySQL Datadir: /u01/mysqldata/
51 11/16/2018 Trivadis DOAG18: MySQL für Einsteiger Oracle DBA tricks for MySQL
Trick 3: Secure passwordless login
• Use mysql socket_auth Authentication Plugin • Local socket based login • Compareable to oracle bequeath connection • Oracle syntax: sqlplus / as sysdba • MySQL syntax: mysql --defaults- file=/u00/app/mysql/admin/$SID/conf/$SID.my.cnf --user $USER
52 11/16/2018 Trivadis DOAG18: MySQL für Einsteiger Oracle DBA tricks for MySQL
Trick 3: Safe passwordless login
• Example: mysql@vm01:~/ [BRONKO] mysql --defaults- file=/u00/app/mysql/admin/BRONKO/conf/BRONKO.my.cnf --user $USER
Your MySQL connection id is 23 Server version: 5.7.23-enterprise-commercial-advanced-log MySQL Enterprise Server - Advanced Edition (Commercial) Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. mysql>
53 11/16/2018 Trivadis DOAG18: MySQL für Einsteiger Trivadis @ DOAG 2018 #opencompany
Stand: 3ter Stock, direkt an der Rolltreppe
Wir teilen unser Know how! Einfach vorbei kommen, Live-Präsentationen und Dokumentenarchiv T-Shirts, Gewinnspiel und mehr
Wir freuen uns wenn Sie vorbei schauen
54 11/16/2018 Trivadis DOAG18: MySQL für Einsteiger