CXFSTM 5 Administration Guide for SGI® InfiniteStorage 007–4016–029 COPYRIGHT © 1999–2009 SGI. All rights reserved; provided portions may be copyright in third parties, as indicated elsewhere herein. No permission is granted to copy, distribute, or create derivative works from the contents of this electronic documentation in any manner, in whole or in part, without the prior written permission of SGI. The following copyright notice applies to the LZF algorithm: Copyright (c) 2000-2005 Marc Alexander Lehmann <[email protected]> Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: 1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. 2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. 3. The name of the author may not be used to endorse or promote products derived from this software without specific prior written permission. THIS SOFTWARE IS PROVIDED BY THE AUTHOR ‘‘AS IS’’ AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY,WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. LIMITED RIGHTS LEGEND The software described in this document is "commercial computer software" provided with restricted rights (except as to included open/free source) as specified in the FAR 52.227-19 and/or the DFAR 227.7202, or successive sections. Use beyond license provisions is a violation of worldwide intellectual property laws, treaties and conventions. This document is provided with limited rights as defined in 52.227-14. TRADEMARKS AND ATTRIBUTIONS SGI, Altix, the SGI cube, the SGI logo, IRIX, O2, Octane, Onyx, Onyx2, Origin, Silicon Graphics Prism, Silicon Graphics Tezro, and XFS are registered trademarks and CXFS, FDDIXPress, NUMAlink, Octane2, Performance Co-Pilot, Silicon Graphics Fuel, and SGI ProPack are trademarks of SGI in the United States and/or other countries worldwide. AIX and IBM are registered trademarks of IBM Corporation. Brocade is a trademark of Brocade Communication Systems, Inc. Digi is a trademark of Digi International, Inc. Java and Sun are registered trademarks and Solaris is a trademark of Sun Microsystems, Inc. Linux is a registered trademark of Linus Torvalds in several countries. Legato NetWorker is a registered trademark of Legato Systems, Inc. Mac OS is a registered trademark of Apple Computer, Inc. Netscape is a trademark of Netscape Communications Corporation. QLogic is a registered trademark of QLogic Corporation. Red Hat is a registered trademark and RPM is a trademark of Red Hat, Inc. UNIX is a registered trademark of The Open Group in the United States and other countries. VERITAS is a trademark of VERITAS Software Corporation. Microsoft, Windows, Windows NT, and Windows Vista are registered trademarks or trademarks of Microsoft Corporation. X/Open is a registered trademark of X/Open Company Ltd. All other trademarks mentioned herein are the property of their respective owners. New Features in This Guide Note: Be sure to read the release notes for the SGI InfiniteStorage Software Platform, your specific CXFS platforms, and the late-breaking caveats pages on Supportfolio to learn about any changes to the installation and configuration procedures. This version includes corrections and the following: • "Unmount Filesystems Before Adding or Deleting Server-Capable Administration Nodes" on page 51 • "Change the Brocade Password when Prompted" on page 70 • "Use a Time Synchronization Application" on page 81 • "Turn Off Local XVM on Linux Nodes if Unused" on page 81 • "Configuring SuSEfirewall2" on page 115 • "Dynamic Parameters for Debugging Purposes Only" on page 468 007–4016–029 iii Record of Revision Version Description 001 September 1999 Supports the CXFS 1.1 product in the IRIX 6.5.6f release. 002 October 1999 Supports the CXFS 1.1 product in the IRIX 6.5.6f release. 003 December 1999 Supports the CXFS product in the IRIX 6.5.7f release. 004 March 2000 Supports the CXFS product in the IRIX 6.5.8f release. 005 June 2000 Supports the CXFS product in the IRIX 6.5.9f release. 006 September 2000 Supports the CXFS product in the IRIX 6.5.10f release. 007 January 2001 Supports the CXFS product in the IRIX 6.5.11f release. 008 March 2001 Supports the CXFS product in the IRIX 6.5.12f release. 009 June 2001 Supports the CXFS product in the IRIX 6.5.13f release. 011 September 2001 Supports the CXFS product in the IRIX 6.5.14f release. (Note, there was no 010 version due to an internal numbering mechanism.) 012 December 2001 Supports the CXFS Version 2 product in IRIX 6.5.15f. 013 March 2002 Supports the CXFS Version 2 product in IRIX 6.5.16f. 007–4016–029 v Record of Revision 014 June 2002 Supports the CXFS Version 2 product in IRIX 6.5.17f. 015 September 2002 Supports the CXFS Version 2 product in IRIX 6.5.18f. 016 December 2002 Supports the CXFS Version 2 product in IRIX 6.5.19f. 017 March 2003 Supports the CXFS Version 2 product in IRIX 6.5.20f. 018 September 2003 Supports the CXFS 3.0 product in IRIX 6.5.22 and CXFS 3.0 for SGI Altix 3000 running SGI ProPack 2.3 for Linux. 019 December 2003 Supports the CXFS 3.1 product in IRIX 6.5.23 and CXFS 3.1 for SGI Altix 3000 running SGI ProPack 2.4 for Linux. 020 March 2004 Supports the CXFS 3.2 product in IRIX 6.5.24 and CXFS 3.2 for SGI Altix 3000 running SGI ProPack 3 for Linux. 021 November 2004 Supports the CXFS 3.2 product in IRIX 6.5.24 and CXFS 3.2 for SGI Altix 3000 running SGI ProPack 3 for Linux. 022 April 2005 Supports the CXFS 3.3 product 023 July 2005 Supports the CXFS 3.4 product 024 May 2006 Supports the CXFS 4.0 product 025 January 2007 Supports the CXFS 4.1 product vi 007–4016–029 CXFSTM 5 Administration Guide for SGI® InfiniteStorage 026 September 2007 Supports the CXFS 4.2 product 027 March 2008 Supports the CXFS 5.0 product 028 September 2008 Supports the CXFS 5.2 product 029 March 2009 Supports the CXFS 5.4 product in ISSP 1.6 007–4016–029 vii Contents About This Guide .....................xxxvii Related Publications . ...................xxxvii Obtaining Publications . ...................xxxix Conventions . ........................ xl Reader Comments ....................... xli 1. Introduction to CXFS . ................. 1 What is CXFS? ........................ 1 Comparison of XFS and CXFS ................... 3 Differences in Filesystem Mounting and Management . ......... 3 Supported XFS Features . ................... 4 When to Use CXFS . ................... 5 Performance Considerations ................... 6 Comparison of Network and CXFS Filesystems .............. 7 Network Filesystems . ................... 7 CXFS Filesystems . ................... 7 CXFS Features ....................... 8 CXFS Restrictions . ................... 9 Cluster Environment Concepts ................... 9 Metadata . ........................ 10 Node . ........................ 10 RAID . ........................ 10 LUN.......................... 10 Cluster . ........................ 11 Pool . ........................ 11 007–4016–029 ix Contents Node Types, Node Functions, and the Cluster Database . ......... 12 Membership ........................ 22 Private Network ....................... 22 Data Integrity Protection . ................... 24 CXFS Tiebreaker ....................... 24 Relocation . ........................ 25 Recovery . ........................ 25 CXFS Services ....................... 28 CXFS Interoperability With Other Products and Features . ......... 28 Data Migration Facility (DMF) .................. 29 Highly Available Services . ................... 29 GPT Labels Overview . ................... 30 Guaranteed-Rate I/O (GRIO) Version 2 Overview . ......... 30 Storage Management . ................... 31 Storage Backups Differences .................. 31 NFS Differences . ................... 31 Quotas Differences . ................... 32 Samba Differences . ................... 32 Volume Management with XVM .................. 33 XVM Failover Version 2 Overview . .............. 34 Hardware and Software Requirements for Server-Capable Administration Nodes . 34 CXFS Software Products Installed on Server-Capable Administration Nodes . 36 CXFS Tools . ........................ 37 Configuration Commands . ................... 37 CXFS GUI Overview . ................... 38 cxfs_admin Command-Line Configuration Tool Overview ........ 40 Cluster Administration Daemons .................. 41 CXFS Control Daemons . ................... 42 x 007–4016–029 CXFSTM 5 Administration Guide for SGI® InfiniteStorage Other Commonly Used Administrative Commands . ......... 42 2. Best Practices . ..................... 45 Configuration Best Practices . ................... 45 Fix Network Issues First . ................... 46 Save the Current Configuration Before Making Changes . ......... 46 Save the CXFS Configuration .................. 47 Save the XVM Configuration