Front cover Configuring and Managing AIX Clusters Using xCAT 2 An xCAT guide for CSM system administrators with advanced features How to transition your AIX cluster to xCAT xCAT 2 deployment guide for a new AIX cluster Octavian Lascu Tim Donovan Saul Hiller Simeon McAleer László Niesz Sean Saunders Kai Wu Peter Zutenis ibm.com/redbooks International Technical Support Organization Configuring and Managing AIX Clusters Using xCAT 2 October 2009 SG24-7766-00 Note: Before using this information and the product it supports, read the information in “Notices” on page ix. First Edition (October 2009) This edition applies to Version 2, Release 2, Modification 1 of Extreme Cluster Administration Tool (xCAT 2) for AIX. © Copyright International Business Machines Corporation 2009. All rights reserved. Note to U.S. Government Users Restricted Rights -- Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM Corp. Contents Notices . ix Trademarks . x Preface . xi The team that wrote this book . xi Become a published author . xiii Comments welcome. xiv Part 1. Introduction . 1 Chapter 1. Introduction to clustering . 3 1.1 Clustering concepts. 4 1.1.1 Cluster types . 4 1.1.2 Cluster components . 5 1.1.3 Cluster nodes . 7 1.1.4 Cluster networks . 9 1.1.5 Hardware, power control, and console access . 11 1.2 Suggested cluster diagram . 13 1.3 Managing high performance computing (HPC) clusters . 15 1.3.1 What is CSM . 15 1.3.2 xCAT 2: An 0pen source method for cluster management . 16 1.3.3 xCAT 1.1.4 and CSM to xCAT 2 - History and evolution . 17 1.3.4 IBM (Systems) Director vs. xCAT 2 . 18 Chapter 2. xCAT 2 architecture . 21 2.1 Overview of xCAT 2 features . 22 2.1.1 Operating systems and distributions supported . 23 2.1.2 Supported hardware . 24 2.1.3 Hardware control. 24 2.1.4 Hierarchical clusters . 25 2.2 Overview of the xCAT 2 architecture. 26 2.2.1 xCAT 2 database . 28 2.2.2 xCAT 2 directory structure . 30 2.2.3 xCAT network fundamentals . 31 2.2.4 Monitoring infrastructure . 33 2.2.5 Parallel remote command execution and file copy, File Sync. 37 2.2.6 Additional features . 38 2.3 Support offering terms. 39 © Copyright IBM Corp. 2009. All rights reserved. iii Chapter 3. CSM and xCAT functional comparison. 41 3.1 Architectural comparison. 42 3.1.1 Hierarchical versus flat . 42 3.1.2 Modular structure . 42 3.1.3 Database. 43 3.1.4 Notification infrastructure . 44 3.1.5 Deploying both diskless and diskful nodes . 44 3.1.6 Automatic setup of services . 44 3.1.7 Role-based system administration . 44 3.1.8 Dynamic nodegroups . 44 3.1.9 Secondary adapters . 45 3.1.10 RMC monitoring . 45 3.2 Features . 54 3.3 Hardware and software considerations . 55 3.4 Hardware considerations. 55 3.5 SW considerations . 55 3.6 Database . 56 3.7 Administrative task comparison. 59 3.7.1 Table comparing functionality . 59 3.7.2 Comparison of common admin commands. 62 Part 2. Implementation . 69 Chapter 4. CSM to xCAT 2 transition . 71 4.1 Introduction to implementation scenarios . 72 4.1.1 Hardware. 72 4.1.2 Software . 72 4.1.3 Test environment diagram . 73 4.1.4 Scenarios description . 73 4.2 Preparing your CSM cluster for transition . 76 4.2.1 Things to check before the transition . 76 4.2.2 Remote shell/copy back-end considerations. 78 4.2.3 Backing up CSM data . 78 4.2.4 Saving secondary adapter information . 82 4.2.5 Save RMC monitoring data . 83 4.3 Transitioning nodes to a different MN . 84 4.3.1 Preparing the xCAT 2 management node (MN) for transition. 84 4.3.2 Gather CSM data using the csm2xcat script on CSM MS . 89 4.3.3 Update /etc/hosts . 90 4.3.4 Set up user and group on the new xCAT MN . 90 4.3.5 Configure remote shell . 91 4.3.6 Check the ulimit. 91 4.3.7 Enabling NTP . 92 iv Configuring and Managing AIX Clusters Using xCAT 2 4.3.8 Enabling inetd entries . 92 4.3.9 Define HMC and VIO servers in xCAT . 93 4.3.10 Discovering HW in xCAT (rscan) . 94 4.3.11 Defining the nodes (populating the tables) . 94 4.3.12 Configuring and testing conserver . 99 4.3.13 Testing remote power control (rpower) . 100 4.3.14 Configuring NIM master and creating NIM resources . 100 4.3.15 Add installp bundles to xCAT osimage . 102 4.3.16 Define xCAT networks . 102 4.3.17 Create NIM secondary adapter definitions . 102 4.3.18 Creating NIM client and group definitions . 103 4.3.19 Removing nodes from the NIM configuration on CSM MS . 105 4.3.20 Register the nodes with the new xCAT NIM master . 105 4.3.21 Install xCAT dependency packages on the client nodes. 106 4.3.22 Removing the nodes from CSM . 107 4.3.23 Setting up syslog. 109 4.3.24 Prepare SSH keys for distribution to the nodes . 109 4.3.25 Run the xcataixscript utility on the nodes . 109 4.3.26 Configure and run postscripts with updatenode . ..