SUSE Linux Enterprise High Availability Extension 12 SP4 Administration Guide Administration Guide SUSE Linux Enterprise High Availability Extension 12 SP4 by Tanja Roth and Thomas Schraitle This guide is intended for administrators who need to set up, congure, and maintain clusters with SUSE® Linux Enterprise High Availability Extension. For quick and ecient conguration and administration, the High Availability Extension includes both a graphical user interface (GUI) and a command line interface (CLI). For performing key tasks, both approaches (GUI and CLI) are covered in detail in this guide. Thus, administrators can choose the appropriate tool that matches their needs. Publication Date: September 10, 2021 SUSE LLC 1800 South Novell Place Provo, UT 84606 USA https://documentation.suse.com Copyright © 2006–2021 SUSE LLC and contributors. All rights reserved. Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or (at your option) version 1.3; with the Invariant Section being this copyright notice and license. A copy of the license version 1.2 is included in the section entitled “GNU Free Documentation License”. For SUSE trademarks, see http://www.suse.com/company/legal/ . All other third-party trademarks are the property of their respective owners. Trademark symbols (®, ™ etc.) denote trademarks of SUSE and its aliates. Asterisks (*) denote third-party trademarks. All information found in this book has been compiled with utmost attention to detail. However, this does not guarantee complete accuracy. Neither SUSE LLC, its aliates, the authors nor the translators shall be held liable for possible errors or the consequences thereof. Contents About This Guide xv 1 Available Documentation xvi 2 Giving Feedback xvii 3 Documentation Conventions xviii 4 Product Life Cycle and Support xix Support Statement for SUSE Linux Enterprise High Availability Extension xx • Technology Previews xxi I INSTALLATION, SETUP AND UPGRADE 1 1 Product Overview 2 1.1 Availability as Extension 2 1.2 Key Features 3 Wide Range of Clustering Scenarios 3 • Flexibility 3 • Storage and Data Replication 4 • Support for Virtualized Environments 4 • Support of Local, Metro, and Geo Clusters 4 • Resource Agents 5 • User-friendly Administration Tools 5 1.3 Benefits 6 1.4 Cluster Configurations: Storage 9 1.5 Architecture 12 Architecture Layers 12 • Process Flow 15 2 System Requirements and Recommendations 16 2.1 Hardware Requirements 16 2.2 Software Requirements 17 2.3 Storage Requirements 17 iii Administration Guide 2.4 Other Requirements and Recommendations 18 3 Installing the High Availability Extension 20 3.1 Manual Installation 20 3.2 Mass Installation and Deployment with AutoYaST 20 4 Using the YaST Cluster Module 23 4.1 Definition of Terms 23 4.2 YaST Cluster Module 25 4.3 Defining the Communication Channels 27 4.4 Defining Authentication Settings 32 4.5 Transferring the Configuration to All Nodes 33 Configuring Csync2 with YaST 34 • Synchronizing Changes with Csync2 35 4.6 Synchronizing Connection Status Between Cluster Nodes 37 4.7 Configuring Services 38 4.8 Bringing the Cluster Online 40 5 Upgrading Your Cluster and Updating Software Packages 42 5.1 Terminology 42 5.2 Upgrading your Cluster to the Latest Product Version 43 Supported Upgrade Paths for SLE HA and SLE HA Geo 44 • Required Preparations Before Upgrading 46 • Cluster Offline Upgrade 47 • Cluster Rolling Upgrade 49 5.3 Updating Software Packages on Cluster Nodes 52 5.4 For More Information 53 iv Administration Guide II CONFIGURATION AND ADMINISTRATION 54 6 Configuration and Administration Basics 55 6.1 Use Case Scenarios 55 6.2 Quorum Determination 56 Global Cluster Options 57 • Global Option no-quorum- policy 57 • Global Option stonith-enabled 58 • Corosync Configuration for Two-Node Clusters 58 • Corosync Configuration for N- Node Clusters 59 6.3 Cluster Resources 60 Resource Management 60 • Supported Resource Agent Classes 61 • Types of Resources 63 • Resource Templates 63 • Advanced Resource Types 64 • Resource Options (Meta Attributes) 67 • Instance Attributes (Parameters) 70 • Resource Operations 72 • Timeout Values 74 6.4 Resource Monitoring 75 6.5 Resource Constraints 77 Types of Constraints 77 • Scores and Infinity 80 • Resource Templates and Constraints 81 • Failover Nodes 82 • Failback Nodes 83 • Placing Resources Based on Their Load Impact 84 • Grouping Resources by Using Tags 87 6.6 Managing Services on Remote Hosts 87 Monitoring Services on Remote Hosts with Monitoring Plug- ins 88 • Managing Services on Remote Nodes with pacemaker_remote 89 6.7 Monitoring System Health 90 6.8 For More Information 92 7 Configuring and Managing Cluster Resources with Hawk2 94 7.1 Hawk2 Requirements 94 7.2 Logging In 95 v Administration Guide 7.3 Hawk2 Overview: Main Elements 96 Left Navigation Bar 97 • Top-Level Row 98 7.4 Configuring Global Cluster Options 98 7.5 Configuring Cluster Resources 100 Showing the Current Cluster Configuration (CIB) 101 • Adding Resources with the Wizard 102 • Adding Simple Resources 103 • Adding Resource Templates 105 • Modifying Resources 105 • Adding STONITH Resources 107 • Adding Cluster Resource Groups 108 • Adding Clone Resources 110 • Adding Multi-state Resources 111 • Grouping Resources by Using Tags 112 • Configuring Resource Monitoring 113 7.6 Configuring Constraints 116 Adding Location Constraints 116 • Adding Colocation Constraints 117 • Adding Order Constraints 119 • Using Resource Sets for Constraints 121 • For More Information 122 • Specifying Resource Failover Nodes 123 • Specifying Resource Failback Nodes (Resource Stickiness) 124 • Configuring Placement of Resources Based on Load Impact 125 7.7 Managing Cluster Resources 127 Editing Resources and Groups 128 • Starting Resources 128 • Cleaning Up Resources 129 • Removing Cluster Resources 130 • Migrating Cluster Resources 130 7.8 Monitoring Clusters 132 Monitoring a Single Cluster 132 • Monitoring Multiple Clusters 133 7.9 Using the Batch Mode 136 7.10 Viewing the Cluster History 140 Viewing Recent Events of Nodes or Resources 140 • Using the History Explorer for Cluster Reports 141 • Viewing Transition Details in the History Explorer 143 7.11 Verifying Cluster Health 145 vi Administration Guide 8 Configuring and Managing Cluster Resources (Command Line) 146 8.1 crmsh—Overview 146 Getting Help 147 • Executing crmsh's Subcommands 148 • Displaying Information about OCF Resource Agents 150 • Using crmsh's Shell Scripts 151 • Using crmsh's Cluster Scripts 152 • Using Configuration Templates 155 • Testing with Shadow Configuration 157 • Debugging Your Configuration Changes 158 • Cluster Diagram 158 8.2 Managing Corosync Configuration 158 8.3 Configuring Global Cluster Options 160 8.4 Configuring Cluster Resources 160 Loading Cluster Resources from a File 161 • Creating Cluster Resources 161 • Creating Resource Templates 162 • Creating a STONITH Resource 163 • Configuring Resource Constraints 164 • Specifying Resource Failover Nodes 167 • Specifying Resource Failback Nodes (Resource Stickiness) 168 • Configuring Placement of Resources Based on Load Impact 168 • Configuring Resource Monitoring 171 • Configuring a Cluster Resource Group 171 • Configuring a Clone Resource 172 8.5 Managing Cluster Resources 173 Showing Cluster Resources 173 • Starting a New Cluster Resource 175 • Cleaning Up Resources 175 • Removing a Cluster Resource 176 • Migrating a Cluster Resource 176 • Grouping/Tagging Resources 177 • Getting Health Status 177 8.6 Setting Passwords Independent of cib.xml 178 8.7 Retrieving History Information 178 8.8 For More Information 180 9 Adding or Modifying Resource Agents 181 9.1 STONITH Agents 181 9.2 Writing OCF Resource Agents 181 9.3 OCF Return Codes and Failure Recovery 183 vii Administration Guide 10 Fencing and STONITH 185 10.1 Classes of Fencing 185 10.2 Node Level Fencing 186 STONITH Devices 186 • STONITH Implementation 187 10.3 STONITH Resources and Configuration 188 Example STONITH Resource Configurations 188 10.4 Monitoring Fencing Devices 191 10.5 Special Fencing Devices 192 10.6 Basic Recommendations 194 10.7 For More Information 195 11 Storage Protection and SBD 196 11.1 Conceptual Overview 196 11.2 Overview of Manually Setting Up SBD 198 11.3 Requirements 198 11.4 Number of SBD Devices 199 11.5 Calculation of Timeouts 200 11.6 Setting Up the Watchdog 201 Using a Hardware Watchdog 201 • Using the Software Watchdog (softdog) 203 11.7 Setting Up SBD with Devices 204 11.8 Setting Up Diskless SBD 209 11.9 Testing SBD and Fencing 211 11.10 Additional Mechanisms for Storage Protection 212 Configuring an sg_persist Resource 212 • Ensuring Exclusive Storage Activation with sfex 214 11.11 For More Information 216 viii Administration Guide 12 Access Control Lists 217 12.1 Requirements and Prerequisites 217 12.2 Enabling Use of ACLs in Your Cluster 218 12.3 The Basics of ACLs 219 Setting ACL Rules via XPath Expressions 219 • Setting ACL Rules via Abbreviations 221 12.4 Configuring ACLs with Hawk2 222 12.5 Configuring ACLs with crmsh 224 13 Network Device Bonding 225 13.1 Configuring Bonding Devices with YaST 225 13.2 Hotplugging of Bonding Slaves 228 13.3 For More Information 230 14 Load Balancing 231 14.1 Conceptual Overview 231 14.2 Configuring Load Balancing with Linux Virtual Server 233 Director 233 • User Space Controller and Daemons 233 • Packet Forwarding 234 • Scheduling Algorithms 234 • Setting Up IP Load Balancing with YaST 235 • Further Setup 241 14.3 Configuring Load Balancing with HAProxy 241