robert vaupel _ of mainframe environments high availability and scalability of mainframe environments r. vaupel Robert Vaupel High Availability and Scalability of Mainframe Environments using System z and z/OS as example High Availability and Scalability of Mainframe Environments using System z and z/OS as example by Robert Vaupel Impressum Karlsruher Institut für Technologie (KIT) KIT Scientific Publishing Straße am Forum 2 D-76131 Karlsruhe www.ksp.kit.edu KIT – Universität des Landes Baden-Württemberg und nationales Forschungszentrum in der Helmholtz-Gemeinschaft Diese Veröffentlichung ist im Internet unter folgender Creative Commons-Lizenz publiziert: http://creativecommons.org/licenses/by-nc-nd/3.0/de/ KIT Scientific Publishing 2013 Print on Demand ISBN 978-3-7315-0022-3 Contents 1. Introduction 1 1.1. Motivation . 1 1.2. High Availability . 3 1.3. Scalability . 5 2. z/Architecture 9 2.1. A Little History . 9 2.2. System z CMOS Heritage . 12 2.3. System zEC12 Central Electronic Complex . 13 2.4. System zEC12 Components . 14 2.5. System z Multi Chip Module . 16 2.5.1. Memory . 17 2.5.2. Book . 18 2.5.3. Processor Characterization . 18 2.6. System z High Availability Design . 19 2.6.1. Transparent CPU Sparing . 20 2.6.2. CPU Error Detection for newer System z machines . 21 2.6.3. Redundant Array of Independent Memory . 22 2.7. System z Software and Firmware Layers . 25 2.8. Instruction Execution . 27 2.8.1. CISC versus RISC Architecture . 27 2.8.2. Register Sets . 30 2.8.3. Program Status Word . 31 2.8.4. System z Instructions . 33 2.8.5. Decimal Arithmetic . 35 2.8.6. Floating Point Arithmetic . 37 2.8.7. View of Instruction Execution . 40 2.8.8. Super-Scalar Instruction Execution . 42 2.8.9. System z10 Microprocessor Pipeline . 43 2.8.10. Instruction Re-Ordering . 46 2.8.11. Register Renaming . 47 2.8.12. Instruction Cracking . 49 ii Contents 2.8.13. Out-Of-Order Execution . 49 2.8.14. System z196 Microprocessor Core . 51 2.8.15. System z196 Instruction Handling . 54 2.8.16. System z196 Instruction Optimization . 54 2.8.17. System z196 Microprocessor Summary . 57 2.9. Interrupts . 57 2.9.1. Enabling and Disabling . 59 2.10. Timing Facilities . 60 2.10.1. Time-of-Day Clock . 60 2.10.2. Clock Comparator . 62 2.10.3. CPU Timer . 62 2.11. Storage Addressing . 63 2.11.1. Address Types . 63 2.11.2. Dynamic Address Translation . 64 2.11.3. Large Page Support . 66 2.11.4. Storage Protection . 66 2.11.5. Prefixing . 67 2.12. Multiprocessing . 68 2.12.1. Atomic Instructions . 69 2.13. Input and Output . 70 2.13.1. z196 I/O Infrastructure . 71 2.13.2. I/O System Overview . 72 2.13.3. I/O Drawer . 74 2.13.4. I/O Operation . 74 2.13.5. Logical Channel Subsystem . 76 2.13.6. I/O Configuration . 78 2.14. Logical Partitioning . 78 2.14.1. z/VM . 80 2.14.2. Start Interpretive Execution . 81 2.14.3. Logical Processor Management . 82 2.14.4. Storage of a Logical Partition . 84 2.14.5. Storage of a Virtual Machine under z/VM . 85 2.15. Summary . 86 3. z/OS 87 3.1. z/OS Structure . 87 3.2. Address Spaces . 89 3.2.1. Address Space Types and Storage . 91 3.2.2. Control Block Areas . 93 3.2.3. Storage Protection . 94 Contents iii 3.2.4. Address Space Creation . 94 3.3. Program Execution . 96 3.3.1. Reentrant Programming . 97 3.3.2. Program Recovery . 99 3.3.3. Recovery Processing . 101 3.4. Program Data Exchange . 102 3.4.1. Data Exchange via Common Storage . 103 3.4.2. Data Exchange via Cross Memory . 104 3.4.3. Access Registers . 108 3.5. Storage Management . 108 3.5.1. z/OS Storage Managers . 109 3.5.2. Managing Storage . 110 3.6. z/OS Data Sets . 114 3.6.1. Sequential Data Sets . 116 3.6.2. Partitioned Data Sets . 117 3.6.3. Allocating a Data Set . 118 3.6.4. Virtual Storage Access Method . 118 3.6.5. Data Set Organization . 119 3.7. Starting z/OS . 119 3.8. Job Entry System . 123 3.9. Time Sharing Option . 124 3.10. Unix System Services . 126 3.11. Summary . 127 4. Dispatching 129 4.1. Dispatching Requirements . 129 4.2. z/OS Dispatching . 132 4.2.1. CPU and Storage Management . 133 4.2.2. Dispatcher Queue . 134 4.2.3. Dispatching Work . 135 4.2.4. Preemption and Time Slicing . 137 4.2.5. Interrupt Processing . 138 4.2.6. I/O Enablement . 139 4.2.7. CPU Report Example . 141 4.3. PR/SM Dispatching . 143 4.3.1. Dispatching Logical Processors . 144 4.3.2. LPAR Report Example . 147 4.3.3. CEC Utilization Example . 149 4.4. Offload Processors . 151 4.4.1. Using zIIPs and zAAPs . 152 iv Contents 4.4.2. Processor Pools . 154 4.4.3. LPAR Report Example including Offload Processors . 155 4.5. Example on how z/OS and PR/SM Dispatcher work together . 156 4.5.1. z/OS Serialization . 156 4.5.2. z/OS Spin Lock Processing . 158 4.6. Limitations of Dispatching . 160 4.6.1. Large System Effects . 161 4.7. Hiperdispatch . 163 4.7.1. Vertical CPU Management . 164 4.7.2. Processor Shares . 165 4.7.3. Example for Unparking Low Processors . 167 4.7.4. Hiperdispatch in z/OS . 170 4.7.5. Affinity Nodes . 171 4.7.6. Assessment for Parking and Unparking Low Processors 173 4.7.7. Balancing Work . 174 4.7.8. Helper Processing . 175 4.7.9. System Work . 177 4.8. Hiperdispatch Analysis . 178 4.8.1. Processor Utilization . 178 4.8.2. Cycles per Instruction . 179 4.8.3. Hiperdispatch on z196 and zEC12 . 181 4.9. Summary . 182 5. Workload Management 185 5.1. Workload Management Concepts . 185 5.2. Why Workload Management . 186 5.3. Workload Management on z/OS . 186 5.4. z/OS Workload Management Basics . 187 5.4.1. Work Classification . 187 5.4.2. Service Class Goals . 190 5.4.3. Response Time Goals . 191 5.4.4. Execution Velocity Goals . 193 5.4.5. Managing Work in z/OS . 195 5.4.6. WLM Transaction Management . 196 5.4.7. Enclaves . 197 5.4.8. Managing CICS and IMS Transactions . 199 5.4.9. Service Definition . 201 5.4.10. Service Unit . 202 5.4.11. Goal Achievement . 203 Contents v 5.5. z/OS Workload Manager Algorithms . 204 5.5.1. Data Sampling . 205 5.5.2. Histories . 207 5.5.3. Data Plots . 209 5.5.4. Policy Adjustment . 210 5.5.5. Policy Adjustment Example: Fix MPL Delay . 213 5.5.6. Projecting a Change for Accessing a Resource . 215 5.5.7. Receiver Value Assessment . 216 5.5.8. Net Value Assessment . 217 5.5.9. Policy Adjustment Example: Fix CPU Delay . 218 5.5.10. Policy Adjustment Example: Time-line for Changes . 221 5.5.11. Test Scenario versus Real Environment . 224 5.5.12. Resource Adjustment . 226.