Mysql Cluster • Mysql, Heartbeat Plus DRBD Architectures • Setup and Configuration • Managing Failover • High Availability Services from Mysql •Q & A
Total Page:16
File Type:pdf, Size:1020Kb
High Availability Architectures with MySQL and DRBD Jimmy Guerrero Senior Product Manager [email protected] Copyright 2007 MySQL AB The World’s Most Popular Open Source Database 1 Agenda • Introduction to MySQL • Comparing MySQL HA Architectures – MySQL Replication – MySQL Replication plus Linux Heartbeat – MySQL Cluster • MySQL, Heartbeat plus DRBD Architectures • Setup and Configuration • Managing Failover • High Availability Services from MySQL •Q & A Copyright 2007 MySQL AB The World’s Most Popular Open Source Database 2 Overview of MySQL AB • Founded in 1995 • Operations in 24 countries • World’s Most Popular Open Source Database – Over 10 million installations – 50,000 downloads each day • Reduces TCO of database management – Reliable, high performance, easy to use • Bundled by over 100 software & hardware companies • Partners HP, Dell, Novell, Red Hat, Unisys, Accenture, LogicaCMG, T-Systems and IPL Copyright 2007 MySQL AB The World’s Most Popular Open Source Database 3 Established & Emerging Companies High Volume Websites – Web 2.0 – Dynamic content – eCommerce – Look to Book – Session Management – Gaming & Entertainment craigslist –Scale Out Enterprise – Data Warehousing – High-Volume OLTP –Scale Out Embedded – Software & Hardware Copyright 2007 MySQL AB The World’s Most Popular Open Source Database 4 MySQL, Heartbeat and DRBD Cluster • High availability solution built on open source software – MySQL: Database – Linux Heartbeat: Virtual IP management – DRBD: Synchronous block level replication and fast data resynchronization after node failure • Low cost – Many popular Linux distributions supported – Can be built on COTS hardware components – No migration hassles • Supports MySQL “scale out” for reads • Supports transactional storage engines – InnoDB –Falcon Copyright 2007 MySQL AB The World’s Most Popular Open Source Database 5 What is DRBD? • Block device which leverages synchronous replication – Open source Linux kernel module – Bundled by several popular Linux distributions – Project maintained by LinBit • Core functionality – The entire block device is mirrored via a dedicated network – Similar to RAID-1 over Ethernet – DRBD operates at disk I/O layer – Transparent to MySQL and the underlying file system – MySQL fails over to passive DRBD server via "virtual IP management" • Typically accomplished with the Linux Heartbeat package. – MySQL transactional storage engine independent • InnoDB, Falcon http://www.drbd.org/ http://www.linux-ha.org/ Copyright 2007 MySQL AB The World’s Most Popular Open Source Database 6 MySQL HA Solutions Clustering & Geographical Redundancy MySQL Clustering Cluster Technologies MySQL + DRBD Replication MySQL + SAN Technologies Well-Managed MySQL Replication Unmanaged Small ISPs & Telco Data Banking Military Business Mainstream Centers Medical Business Defense 9 9 . 9 9 9 % 35 days 4 days 8 hours 50 mins 5 mins Copyright 2007 MySQL AB The World’s Most Popular Open Source Database 7 What is Replication? Asynchronous Replication update customer Synchronous Replication update customer Copyright 2007 MySQL AB The World’s Most Popular Open Source Database 8 MySQL Replication • Asynchronous Requirements Support – Statement Availability – Row-based (5.1) Automated IP Fail Over No – Mixed (5.1) Automated DB Fail Over No • Read vs. Write Typical Fail Over Time Varies – Response times – Load balancing Resynchronization of Data No –Scale Out Geographic Redundancy Yes • Administration Scalability – Backups Built-in Load Balancing Scale-Out – Upgrades Read Intensive Yes – Testing Write Intensive No • Geographic Redundancy Dozens for # of Nodes Reads Copyright 2007 MySQL AB The World’s Most Popular Open Source Database 9 MySQL Replication Topologies Master > Slave Master > Slaves Masters > Slave (Multi-Source) Master > Slave > Slaves Circular (Multi-Master) Master < > Master (Multi-Master) Copyright 2007 MySQL AB The World’s Most Popular Open Source Database 10 MySQL Replication - Scale Out • Write to one master • Read from many slaves • Perfect for read intensive apps Load Balancer Writes & Reads Reads Reads MySQL Replication Copyright 2007 MySQL AB The World’s Most Popular Open Source Database 11 MySQL Replication & Linux Heartbeat IP Management Active Passive MySQL MySQL Server Server Asynchronous MySQL Replication Heartbeat implements serial, UDP, and PPP/UDP heartbeats together with IP address takeover, this solves MySQL Replication’s “fail over” problem, but, data resynchronization is not automatic and can be complex. Copyright 2007 MySQL AB The World’s Most Popular Open Source Database 12 MySQL Cluster • In-Memory Database Requirements Support – Disk Data (5.1) Availability • Shared-nothing storage Automated IP Fail Over No • Synchronous replication Automated DB Fail Over Yes • Fast-automatic data fail over Typical Fail Over Time < 3 Secs • High performance Resynchronization of Data Yes – NDB API Yes Geographic Redundancy • No special hardware (w/ Replication) • Best for primary key lookups Scalability • Transactional Load Balancing Scale-Out • Online backup Read Intensive Yes (Look ups) • No automated IP failover Write Intensive Yes 5.0: 64 # of Nodes 5.1: 255 Copyright 2007 MySQL AB The World’s Most Popular Open Source Database 13 MySQL Cluster: Architecture Clients NDB API Application Nodes (NDB API and/or MySQL Server) Management Nodes (Typically 2) Data Nodes NDB Storage Engine Copyright 2007 MySQL AB The World’s Most Popular Open Source Database 14 MySQL Cluster: Data Distribution PK col1 col2 col3 Partitions - Automatic or User-Defined 1 data data data - Horizontal 2 data data data - MD5 hash of Primary Key 3 data data data 4 data data data Two Data Nodes * Complete copy of database on each node Two Replicas * Two copies of each partition Node Group One Node Group P1 - Primary P1 - Replica * One set of identical Data Nodes P2 - Replica P2 - Primary Copyright 2007 MySQL AB The World’s Most Popular Open Source Database 15 MySQL, Heartbeat & DRBD Cluster • Distributed storage Requirements Support • Similar to network RAID 1 Availability • Synchronous replication Automated IP Fail Over Yes • No special networking Automated DB Fail Over Yes components Typical Fail Over Time < 30 secs • Excellent performance Resynchronization of Data Yes – Blocks vs. rows of data MySQL Geographic Redundancy • Manages inconsistencies of Replication data during a failure Scalability • Hides the complexity of many Built-in Load Balancing No recovery actions Read Intensive Yes • Heartbeat manages fail over and virtual IPs Write Intensive Possible # of Nodes 1 Active Copyright 2007 MySQL AB The World’s Most Popular Open Source Database 16 MySQL, Heartbeat & DRBD Cluster IP Management Active Passive DRBD DRBD Server Server Synchronous Block Replication Copyright 2007 MySQL AB The World’s Most Popular Open Source Database 17 Scale Out MySQL Replication w/ DRBD Cluster IP Management Active Passive Server Server MySQL Replication Slaves – Read Scalability - Asynchronous Copyright 2007 MySQL AB The World’s Most Popular Open Source Database 18 Application Partitioning MySQL Replication w/ DRBD Cluster • Data is separated into distinct, autonomous data silos or “shards” within DRBD Clusters • Applications access “shards” directly or through load balancing • Used in conjunction with MySQL Replication, can increase read & write scalability • Potential to also increase availability depending on partitioning scheme • Partitioning on different levels – Application – MySQL data partitioning in 5.1 – Reads vs Writes Copyright 2007 MySQL AB The World’s Most Popular Open Source Database 19 Application Partitioning MySQL Replication w/ DRBD Cluster Load Balancer/SQL Relay Shard 1 Shard 2 IP Management IP Management Copyright 2007 MySQL AB The World’s Most Popular Open Source Database 20 Block Devices • A device node is a type of special file – Used on many Unix-like operating systems • Facilitates transparent communication – Between user space applications and hardware • Correspond to resources allocated by the operating system kernel • Accessed using standard system calls and treated like regular files • Used to correspond to devices – Data is transmitted in the form of blocks. • Make use of buffered input and output routines Copyright 2007 MySQL AB The World’s Most Popular Open Source Database 21 DRBD Basics • Each device has one of two states – Primary – Secondary • Application runs and accesses the device on the primary node • Writes are sent to the local lower level block device and to the node with the device in secondary state • Secondary device writes the data to its lower level block device • Reads are always carried out locally – Key advantage over SAN/NAS • If the primary node fails – Heartbeat switches the secondary device into a primary state and starts the application (MySQL) Copyright 2007 MySQL AB The World’s Most Popular Open Source Database 22 DRBD Basics • When the failed node comes up – Becomes a secondary node – Must synchronize its data to the primary – Synching happens in the background without service interruption – Only the parts of the device which have changed since the failure will be resynchronized. • Total resynchronization in just several minutes – Regardless of device size – Even after the hard crash of an active node – Resynch rates may be configured per device • Device sizes currently limited to 4 TB Copyright 2007 MySQL AB The World’s Most Popular Open Source Database 23 Setup & Configuration System Requirement Support Hosts Minimum 2 (More for scale-out or partitioning) COTS or SMP - 32