Gluster File System 3.2.5 Administration Guide Red Hat Engineering Content Services Administration Guide Gluster File System 3.2.5 Administration Guide Edition 1 Author Red Hat Engineering Content [email protected] Services Copyright © 2011 Red Hat Inc. The text of and illustrations in this document are licensed by Red Hat under a Creative Commons Attribution–Share Alike 3.0 Unported license ("CC-BY-SA"). An explanation of CC-BY-SA is available at http://creativecommons.org/licenses/by-sa/3.0/. In accordance with CC-BY-SA, if you distribute this document or an adaptation of it, you must provide the URL for the original version. Red Hat, as the licensor of this document, waives the right to enforce, and agrees not to assert, Section 4d of CC-BY-SA to the fullest extent permitted by applicable law. Red Hat, Red Hat Enterprise Linux, the Shadowman logo, JBoss, MetaMatrix, Fedora, the Infinity Logo, and RHCE are trademarks of Red Hat, Inc., registered in the United States and other countries. Linux® is the registered trademark of Linus Torvalds in the United States and other countries. Java® is a registered trademark of Oracle and/or its affiliates. XFS® is a trademark of Silicon Graphics International Corp. or its subsidiaries in the United States and/or other countries. MySQL® is a registered trademark of MySQL AB in the United States, the European Union and other countries. All other trademarks are the property of their respective owners. 1801 Varsity Drive Raleigh, NC27606-2072USA Phone: +1 919 754 3700 Phone: 888 733 4281 Fax: +1 919 754 3701 This guide describes Gluster File System (GlusterFS), how to configure, operate, and manage GlusterFS. Preface vii 1. Audience ....................................................................................................................... vii 2. License ......................................................................................................................... vii 3. Document Conventions .................................................................................................. vii 3.1. Typographic Conventions .................................................................................... vii 3.2. Pull-quote Conventions ........................................................................................ ix 3.3. Notes and Warnings ............................................................................................ ix 4. Feedback ........................................................................................................................ x 1. Introducing Gluster File System 1 2. Starting and Stopping the glusterd Daemon 3 2.1. Starting and Stopping glusterd Manually ........................................................................ 3 2.2. Starting glusterd Automatically ...................................................................................... 3 2.2.1. Red Hat-based Systems .................................................................................... 3 2.2.2. Debian-based Systems ...................................................................................... 3 2.2.3. Systems Other than Red Hat and Debain ........................................................... 4 3. Gluster Console Manager – Command Line Utility for Configuration and Management 5 4. Trusted Storage Pools – Preparing GlusterFS for Management 7 4.1. Adding Servers to Trusted Storage Pool ........................................................................ 7 4.2. Removing Server from the Trusted Storage Pool ............................................................ 8 5. Setting Up GlusterFS Server Volumes 9 5.1. Creating Distributed Volumes ........................................................................................ 9 5.2. Creating Replicated Volumes ..................................................................................... 11 5.3. Creating Striped Volumes ........................................................................................... 12 5.4. Creating Distributed Striped Volumes ......................................................................... 12 5.5. Creating Distributed Replicated Volumes .................................................................... 13 5.6. Starting Volumes ....................................................................................................... 14 5.7. Displaying Volume Information ................................................................................... 14 6. Accessing Data - Setting Up GlusterFS Client 17 6.1. Gluster Native Client .................................................................................................. 17 6.1.1. Installing the Gluster Native Client .................................................................... 17 6.1.2. Mounting Volumes ........................................................................................... 21 6.2. NFS ........................................................................................................................... 23 6.2.1. Using NFS to Mount Volumes .......................................................................... 23 6.3. CIFS .......................................................................................................................... 26 6.3.1. Using CIFS to Mount Volumes ......................................................................... 26 7. Managing GlusterFS Volumes 29 7.1. Tuning Volume Options ............................................................................................... 29 7.2. Expanding Volumes .................................................................................................... 37 7.3. Shrinking Volumes ...................................................................................................... 38 7.4. Migrating Volumes ...................................................................................................... 39 7.5. Rebalancing Volumes ................................................................................................. 41 7.5.1. Rebalancing Volume to Fix Layout Changes ...................................................... 42 7.5.2. Rebalancing Volume to Migrate Existing Data .................................................... 42 7.5.3. Rebalancing Volume to Fix Layout and Migrate Existing Data ............................. 43 7.6. Stopping Volumes ...................................................................................................... 43 7.7. Deleting Volumes ....................................................................................................... 43 7.8. Triggering Self-Heal on Replicate ................................................................................ 44 iii Administration Guide 8. Managing GlusterFS Geo-replication 45 8.1. Replicated Volumes vs Geo-replication ........................................................................ 45 8.2. Preparing to Deploy GlusterFS Geo-replication ............................................................ 45 8.2.1. Exploring Geo-replication Deployment Scenarios ............................................... 46 8.2.2. GlusterFS Geo-replication Deployment Overview ............................................... 47 8.2.3. Checking Geo-replication Minimum Requirements ............................................. 48 8.2.4. Setting Up the Environment for Geo-replication ................................................. 48 8.3. Starting GlusterFS Geo-replication .............................................................................. 49 8.3.1. Starting Geo-replication .................................................................................... 49 8.3.2. Verifying Successful Deployment ...................................................................... 50 8.3.3. Displaying Geo-replication Status Information .................................................... 50 8.3.4. Configuring Geo-replication .............................................................................. 51 8.3.5. Stopping Geo-replication .................................................................................. 51 8.4. Restoring Data from the Slave .................................................................................... 52 8.5. Best Practices ............................................................................................................ 54 9. Monitoring your GlusterFS Workload 57 9.1. Running GlusterFS Volume Profile Command .............................................................. 57 9.1.1. Start Profiling .................................................................................................. 57 9.1.2. Displaying the I/0 Information ........................................................................... 57 9.1.3. Stop Profiling ................................................................................................... 58 9.2. Running GlusterFS Volume TOP Command ................................................................ 59 9.2.1. Viewing Open fd Count and Maximum fd Count ................................................ 59 9.2.2. Viewing Highest File Read Calls ...................................................................... 60 9.2.3. Viewing Highest File Write Calls ...................................................................... 61 9.2.4. Viewing Highest Open Calls on Directories ....................................................... 61 9.2.5. Viewing Highest Read Calls on Directory ........................................................