IBM Spectrum Scale Version 4 Release 2.0 Advanced Administration Guide IBM SC23-7032-06 IBM Spectrum Scale Version 4 Release 2.0 Advanced Administration Guide IBM SC23-7032-06 Note Before using this information and the product it supports, read the information in “Notices” on page 425. This edition applies to version 4 release 2 of the following products, and to all subsequent releases and modifications until otherwise indicated in new editions: v IBM Spectrum Scale ordered through Passport Advantage® (product number 5725-Q01) v IBM Spectrum Scale ordered through AAS/eConfig (product number 5641-GPF) v IBM Spectrum Scale for Linux on z Systems (product number 5725-S28) Significant changes or additions to the text and illustrations are indicated by a vertical line (|) to the left of the change. IBM welcomes your comments; see the topic “How to send your comments” on page xiv. When you send information to IBM, you grant IBM a nonexclusive right to use or distribute the information in any way it believes appropriate without incurring any obligation to you. © Copyright IBM Corporation 2014, 2016. US Government Users Restricted Rights – Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM Corp. Contents Figures .............. vii Deleting a snapshot ........... 71 Managing snapshots with IBM Spectrum Scale GUI 72 Tables ............... ix Chapter 4. Creating and managing file About this information ........ xi clones ............... 75 Prerequisite and related information ...... xiii Creating file clones ............ 75 Conventions used in this information ..... xiii Listing file clones ............ 76 How to send your comments ........ xiv Deleting file clones ............ 77 Splitting file clones from clone parents ..... 77 Summary of changes ........ xv File clones and disk space management ..... 77 File clones and snapshots.......... 77 File clones and policy files ......... 78 Chapter 1. Accessing a remote GPFS file system ............... 1 Chapter 5. Scale Out Backup and Remote user access to a GPFS file system..... 3 Restore (SOBAR) .......... 79 Mounting a remote GPFS file system ...... 4 Backup procedure with SOBAR........ 79 Managing remote access to a GPFS file system ... 6 Restore procedure with SOBAR........ 80 Using remote access with public and private IP addresses ............... 7 Using multiple security levels for remote access .. 9 Chapter 6. Establishing disaster Changing security keys with remote access .... 10 recovery for a cluster ........ 83 NIST compliance ............ 11 Synchronous mirroring with GPFS replication ... 84 Important information about remote access .... 11 Configuring a cluster with GPFS replication .. 85 Recovery from a disaster with GPFS replication 87 Chapter 2. Information Lifecycle Synchronous mirroring with IBM TotalStorage ESS Management for IBM Spectrum Scale . 13 PPRC ................ 92 An active-active cluster ......... 93 Storage pools .............. 13 An active-passive cluster ......... 97 Internal storage pools .......... 14 Data integrity and the use of PPRC consistency External storage pools .......... 19 groups .............. 102 Policies for automating file management..... 20 Asynchronous mirroring with ESS FlashCopy .. 103 Overview of policies .......... 20 Setting up FlashCopy using file-system-level Policy rules ............. 21 suspension ............. 105 The mmapplypolicy command and policy rules 38 Policy rules: Examples and tips....... 42 Managing policies ........... 46 Chapter 7. Implementing a clustered Working with external storage pools ..... 48 NFS environment on Linux ..... 107 Backup and restore with storage pools .... 54 NFS monitoring ............ 107 Filesets ................ 55 NFS failover.............. 107 Fileset namespace ........... 56 NFS locking and load balancing ....... 107 Filesets and quotas ........... 57 CNFS network setup ........... 108 Filesets and storage pools ........ 57 CNFS setup .............. 108 Filesets and global snapshots ....... 57 CNFS administration ........... 109 Fileset-level snapshots .......... 58 Filesets and backup........... 58 Chapter 8. Implementing Cluster Managing filesets ........... 59 Export Services .......... 111 Immutability and appendOnly features ..... 61 CES features .............. 111 CES cluster setup ........... 111 Chapter 3. Creating and maintaining CES network configuration ........ 112 snapshots of file systems ...... 67 CES address failover and distribution policies 113 Creating a snapshot ........... 67 CES protocol management ........ 114 Listing snapshots ............ 68 CES management and administration .... 114 Restoring a file system from a snapshot ..... 69 CES NFS support ............ 115 Reading a snapshot with the policy engine .... 70 CES SMB support ............ 117 Linking to a snapshot ........... 70 CES OBJ support ............ 118 © Copyright IBM Corp. 2014, 2016 iii Migration of CNFS clusters to CES clusters ... 119 Example mmpmon scenarios and how to analyze and interpret their results ......... 186 Chapter 9. Performance monitoring 123 fs_io_s and io_s output - how to aggregate and Network performance monitoring ...... 123 analyze the results........... 186 Performance monitoring tool overview ..... 125 Request histogram (rhist) output - how to Configuring the performance monitoring tool 126 aggregate and analyze the results...... 189 Restarting the performance monitoring tool .. 130 Using request source and prefix directive once 189 Defining the metrics .......... 131 Other information about mmpmon output.... 195 Performance monitoring......... 131 Counter sizes and counter wrapping .... 195 List of queries ............ 132 Return codes from mmpmon ....... 196 List of metrics ............ 136 Performance monitoring through IBM Spectrum Chapter 12. GPFS SNMP support ... 197 Scale GUI .............. 150 Installing Net-SNMP ........... 197 Configuring Net-SNMP .......... 198 Chapter 10. Monitoring system health Configuring management applications ..... 198 through the IBM Spectrum Scale GUI . 153 Installing MIB files on the collector node and management node............ 199 Collector node administration ........ 199 Chapter 11. Monitoring GPFS I/O Starting and stopping the SNMP subagent.... 200 performance with the mmpmon The management and monitoring subagent ... 200 command ............. 157 SNMP object IDs ........... 201 Overview of mmpmon .......... 157 MIB objects ............. 201 Specifying input to the mmpmon command ... 157 Cluster status information ........ 201 Running mmpmon on multiple nodes .... 158 Cluster configuration information ..... 201 Running mmpmon concurrently from multiple Node status information......... 202 users on the same node ......... 158 Node configuration information ...... 202 Display I/O statistics per mounted file system .. 159 File system status information ....... 203 Example of mmpmon fs_io_s request .... 159 File system performance information .... 204 Display I/O statistics for the entire node .... 160 Storage pool information ........ 204 Example of mmpmon io_s request ..... 161 Disk status information ......... 205 Understanding the node list facility ...... 162 Disk configuration information ...... 205 Add node names to a list of nodes for mmpmon Disk performance information ....... 206 processing ............. 162 Net-SNMP traps ........... 207 Delete a node list ........... 164 Create a new node list ......... 164 Chapter 13. Identity management on Show the contents of the current node list ... 165 Windows ............. 209 Delete node names from a list of nodes for Auto-generated ID mappings ........ 209 mmpmon processing .......... 166 Installing Windows IMU ......... 209 Node list examples and error handling .... 166 Configuring ID mappings in IMU ...... 210 Reset statistics to zero .......... 169 Example of mmpmon reset request ..... 170 Chapter 14. Active file management 213 Understanding the request histogram facility ... 170 Specifying the size ranges for I/O histograms 171 Active file management architecture ...... 213 Specifying the latency ranges for I/O .... 171 Caching modes ............. 214 Changing the request histogram facility request File system caching and synchronization .... 216 size and latency ranges ......... 172 Asynchronous operations ........ 216 Disabling the request histogram facility ... 174 Synchronous operations ......... 216 Enabling the request histogram facility .... 175 Update synchronization ......... 217 Displaying the request histogram facility pattern 176 Components of a cluster that is running AFM .. 217 Resetting the request histogram facility data to Global namespace ............ 218 zero ............... 179 Cache eviction ............. 219 Displaying the request histogram facility Disconnected operation .......... 219 statistics values............ 180 Expiration .............. 220 Understanding the Remote Procedure Call (RPC) Cache states .............. 220 facility................ 182 Failure and recovery ........... 222 Displaying the aggregation of execution time for Handling an IW cache fileset disaster ..... 223 Remote Procedure Calls (RPCs) ...... 182 Prefetching .............. 224 Displaying the Remote Procedure Call (RPC) Peer snapshots ............. 225 execution time according to the size of messages 184 Viewing snapshots at home ........ 225 Displaying mmpmon version ........ 185 Failover of cache filesets.......... 225 Example of mmpmon ver request ..... 186 Partial file caching............ 226 iv IBM Spectrum Scale 4.2: Advanced Administration