Guide to OpenVMS Performance Management Order Number: AA–PV5XA–TE May 1993 This manual is a conceptual and tutorial guide for experienced users responsible for optimizing performance on OpenVMS systems. Revision/Update Information: This manual supersedes the Guide to VMS Performance Management, Version 5.2. Software Version: OpenVMS VAX Version 6.0 Digital Equipment Corporation Maynard, Massachusetts May 1993 Digital Equipment Corporation makes no representations that the use of its products in the manner described in this publication will not infringe on existing or future patent rights, nor do the descriptions contained in this publication imply the granting of licenses to make, use, or sell equipment or software in accordance with the description. Possession, use, or copying of the software described in this publication is authorized only pursuant to a valid written license from Digital or an authorized sublicensor. © Digital Equipment Corporation 1993. All rights reserved. The postpaid Reader’s Comments forms at the end of this document request your critical evaluation to assist in preparing future documentation. The following are trademarks of Digital Equipment Corporation: ACMS, ALL–IN–1, Bookreader, CI, DBMS, DECnet, DECwindows, Digital, MicroVAX, OpenVMS, PDP–11, VAX, VAXcluster, VAX DOCUMENT, VMS, and the DIGITAL logo. All other trademarks and registered trademarks are the property of their respective holders. ZK4521 This document was prepared using VAX DOCUMENT Version 2.1. Contents Preface ............................................................ ix 1 Introduction to Performance Management 1.1 Knowing Your Workload ....................................... 1–2 1.1.1 Workload Management .................................... 1–3 1.1.2 Workload Distribution ..................................... 1–4 1.1.3 Code Sharing ............................................ 1–4 1.2 Evaluating User Complaints . ................................ 1–5 1.2.1 Recognizing Hardware and MWAIT Problems ................... 1–6 1.2.2 Recognizing Unrealistic Expectations . ....................... 1–7 1.3 Preparing for a Performance Investigation . ....................... 1–7 1.3.1 Understanding What Tuning Is .............................. 1–7 1.3.2 Predicting When Tuning Is Required . ....................... 1–8 1.4 Evaluating Tuning Success .................................... 1–9 1.5 Deciding When To Stop Tuning . ................................ 1–9 1.6 Performance Options for the System Manager ...................... 1–10 2 Review of OpenVMS Resource Management 2.1 OpenVMS Memory Management Analogy . ....................... 2–1 2.2 Advanced Memory Management Mechanisms and Policies ............ 2–6 2.2.1 Automatic Working Set Adjustment ........................... 2–6 2.2.1.1 Working Set Sizes ..................................... 2–12 2.2.1.2 Initial Working Set Limits and Characteristics ............... 2–13 2.2.1.3 Adjustments to AWSA Parameters . ....................... 2–14 2.2.1.4 Performance Management Strategies and Automatic Working Set Adjustment . ........................................ 2–15 2.2.2 Swapper Trimming ........................................ 2–16 2.2.3 Virtual Balance Slots ...................................... 2–18 2.2.4 Proactive Reclamation of Memory from Idle Processes ............ 2–19 2.2.4.1 How Is This Policy Enabled? ............................. 2–19 2.2.4.2 Reclaiming Memory from Long-Waiting Processes ............. 2–20 2.2.4.3 Reclaiming Memory from Periodically Waking Processes . ....... 2–21 2.2.5 Memory Sharing . ........................................ 2–21 2.2.6 OpenVMS Scheduling ..................................... 2–25 iii 3 Managing System Resources 3.1 Evaluation Ground Rules . .................................... 3–2 3.2 Collecting and Interpreting Image-Level Accounting Data . ........... 3–3 3.3 Maintaining and Interpreting MONITOR Summaries ................ 3–6 3.4 Understanding System Responsiveness ........................... 3–8 3.4.1 Evaluating Responsiveness of System Resources ................. 3–8 3.4.2 Improving Responsiveness of System Resources ................. 3–9 3.5 Understanding the CPU Resource . ............................ 3–9 3.5.1 Evaluating CPU Responsiveness . ............................ 3–9 3.5.1.1 Compute Queue . .................................... 3–9 3.5.1.2 Estimating Available CPU Capacity ........................ 3–11 3.5.1.2.1 Voluntary Wait States . ............................ 3–11 3.5.1.2.2 Involuntary Wait States . ............................ 3–12 3.5.2 Improving CPU Responsiveness . ............................ 3–13 3.5.2.1 Equitable CPU Sharing ................................. 3–13 3.5.2.2 Reduction of CPU Consumption by the System ............... 3–14 3.5.2.2.1 Interpreting MONITOR MODES Data ................... 3–15 3.5.2.3 CPU Offloading . .................................... 3–18 3.5.2.4 CPU Offloading Between Processors on the Network ........... 3–19 3.5.2.5 CPU Load Balancing in a VAXcluster . ................... 3–19 3.6 Understanding the Memory Resource ............................ 3–20 3.6.1 Evaluating Memory Responsiveness .......................... 3–23 3.6.1.1 Page Faulting ......................................... 3–23 3.6.1.1.1 Secondary Page Cache . ............................ 3–24 3.6.1.2 Swapping and Swapper Trimming ......................... 3–25 3.6.2 Improving Memory Responsiveness ........................... 3–26 3.6.2.1 Equitable Memory Sharing . ............................ 3–26 3.6.2.2 Reduction of Memory Consumption by the System . ........... 3–27 3.6.2.3 Memory Offloading . .................................... 3–28 3.6.2.4 Memory Load Balancing ................................ 3–29 3.7 Understanding the Disk I/O Resource ............................ 3–29 3.7.1 Components of a Disk Transfer . ............................ 3–30 3.7.2 Disk Capacity and Demand ................................. 3–30 3.7.2.1 Seek Capacity ........................................ 3–30 3.7.2.2 Data Transfer Capacity ................................. 3–30 3.7.2.3 Demand . ............................................ 3–31 3.7.3 Evaluating Disk I/O Responsiveness .......................... 3–31 3.7.4 Improving Disk I/O Responsiveness ........................... 3–33 3.7.4.1 Solid-State Disks . .................................... 3–33 3.7.4.2 Equitable Disk I/O Sharing . ............................ 3–34 3.7.4.3 Reduction of Disk I/O Consumption by the System . ........... 3–34 3.7.4.3.1 Paging I/O Activity ................................. 3–34 3.7.4.3.2 Swapping I/O Activity . ............................ 3–35 3.7.4.3.3 File System (XQP) I/O Activity ........................ 3–35 3.7.4.4 Disk I/O Offloading .................................... 3–37 3.7.4.4.1 Using the Virtual I/O Cache ........................... 3–37 3.7.4.4.2 Using a RAM Disk .................................. 3–39 3.7.4.4.3 Other Disk I/O Offloading Techniques ................... 3–40 3.7.4.5 Disk I/O Load Balancing ................................ 3–41 3.8 Summary of Important MONITOR Data Items . ................... 3–42 iv 4 Diagnosing Resource Limitations 4.1 Diagnostic Strategy . ........................................ 4–1 4.2 Isolating Memory Limitations . ................................ 4–4 4.2.1 Analyzing the Excessive Page Faulting Symptom . ............... 4–4 4.2.1.1 Image Activations Are Excessive . ....................... 4–5 4.2.1.2 Characterizing Hard Versus Soft Faults .................... 4–6 4.2.1.3 Total Working Set Size Is Too Small—Overview of the Problem . 4–7 4.2.1.4 WSDEFAULT, WSQUOTA, and WSEXTENT Values Are Inappropriate . ........................................ 4–10 4.2.1.5 Borrowing Is Ineffective . ................................ 4–11 4.2.1.6 AWSA Might Be Disabled ............................... 4–11 4.2.1.7 AWSA Is Ineffective—Overview ........................... 4–11 4.2.1.8 Swapper Trimming Is Too Vigorous . ....................... 4–13 4.2.2 Analyzing the Swapping Symptom ............................ 4–14 4.2.3 Detecting Harmful Swapping ................................ 4–15 4.2.4 Investigating Why Processes Consume Unreasonable Amounts of Memory ................................................ 4–16 4.2.4.1 Large, Compute-Bound Process Gains Inordinate Control of Memory ............................................. 4–19 4.2.4.2 Large Waiting Process Is Never Outswapped . ............... 4–20 4.2.4.3 Too Many Processes Compete for Available Memory ........... 4–20 4.2.4.4 Borrowing Is Too Generous .............................. 4–21 4.2.4.5 Swapper Trimming Is Ineffective . ....................... 4–21 4.2.4.6 Many Working Sets Are Too Large . ....................... 4–21 4.2.4.7 Disk Thrashing Occurs . ................................ 4–22 4.2.4.8 System Swaps Rather Than Pages . ....................... 4–23 4.2.4.9 Demand Exceeds Available Memory . ....................... 4–23 4.2.5 Analyzing the Limited Free Memory Symptom . ............... 4–23 4.3 Isolating I/O Limitations ...................................... 4–24 4.3.1 Disk or Tape Operation Problems (Direct I/O) ................... 4–24 4.3.2 Determining I/O Rates ..................................... 4–25 4.3.2.1 Device I/O Rate Is Below Capacity . ....................... 4–25 4.3.2.2 Direct I/O Rate Is Abnormally High . ....................... 4–27 4.3.2.3 Disk Activity Is Due to Paging or Swapping . ............... 4–29 4.3.3 Reduce I/O Demand or Add Capacity . ......................