Red Hat Enterprise Linux 7 Performance Tuning Guide Monitoring and optimizing subsystem throughput in RHEL 7 Last Updated: 2021-08-31 Red Hat Enterprise Linux 7 Performance Tuning Guide Monitoring and optimizing subsystem throughput in RHEL 7 Milan Navrátil Red Hat Customer Content Services Laura Bailey Red Hat Customer Content Services Charlie Boyle Red Hat Customer Content Services Edited by Marek Suchánek Red Hat Customer Content Services [email protected] Legal Notice Copyright © 2018 Red Hat, Inc. This document is licensed by Red Hat under the Creative Commons Attribution-ShareAlike 3.0 Unported License. If you distribute this document, or a modified version of it, you must provide attribution to Red Hat, Inc. and provide a link to the original. If the document is modified, all Red Hat trademarks must be removed. Red Hat, as the licensor of this document, waives the right to enforce, and agrees not to assert, Section 4d of CC-BY-SA to the fullest extent permitted by applicable law. Red Hat, Red Hat Enterprise Linux, the Shadowman logo, the Red Hat logo, JBoss, OpenShift, Fedora, the Infinity logo, and RHCE are trademarks of Red Hat, Inc., registered in the United States and other countries. Linux ® is the registered trademark of Linus Torvalds in the United States and other countries. Java ® is a registered trademark of Oracle and/or its affiliates. XFS ® is a trademark of Silicon Graphics International Corp. or its subsidiaries in the United States and/or other countries. MySQL ® is a registered trademark of MySQL AB in the United States, the European Union and other countries. Node.js ® is an official trademark of Joyent. Red Hat is not formally related to or endorsed by the official Joyent Node.js open source or commercial project. The OpenStack ® Word Mark and OpenStack logo are either registered trademarks/service marks or trademarks/service marks of the OpenStack Foundation, in the United States and other countries and are used with the OpenStack Foundation's permission. We are not affiliated with, endorsed or sponsored by the OpenStack Foundation, or the OpenStack community. All other trademarks are the property of their respective owners. Abstract The Red Hat Enterprise Linux 7 Performance Tuning Guide explains how to optimize Red Hat Enterprise Linux 7 performance. It also documents performance-related upgrades in Red Hat Enterprise Linux 7. The Performance Tuning Guide presents only field-tested and proven procedures. Nonetheless, all prospective configurations should be set up and tested in a testing environment before being applied to a production system. Backing up all data and configuration settings prior to tuning is also recommended. To expand your expertise, you might also be interested in the Red Hat Enterprise Linux Performance Tuning (RH442) training course. Table of Contents Table of Contents .C . H. .A . P. .T .E . R. 1.. .I .N . T. .R .O . .D . U. .C . T. .I O. N. .4 . Who Should Read This Book 4 .C . H. .A . P. .T .E . R. 2. P. .E . R. .F .O . .R . M. A. .N . C. .E . .M . O. N. .I T. .O . .R .I .N . G. T. .O . O. L. S. 5. 2.1. /PROC 5 2.2. GNOME SYSTEM MONITOR 5 2.3. BUILT-IN COMMAND-LINE TOOLS 6 2.4. PERF 7 2.5. TURBOSTAT 7 2.6. IOSTAT 7 2.7. IRQBALANCE 7 2.8. SS 8 2.9. NUMASTAT 8 2.10. NUMAD 8 2.11. SYSTEMTAP 9 2.12. OPROFILE 9 2.13. VALGRIND 9 2.14. PQOS 10 .C . H. .A . P. .T .E . R. 3. T. .U . N. E. D. 1. 2. 3.1. TUNED OVERVIEW 12 3.2. PERFORMANCE TUNING WITH TUNED AND TUNED-ADM 23 .C . H. .A . P. .T .E . R. 4. .T .U . .N . A. .2 . 6. 4.1. REVIEWING THE SYSTEM WITH TUNA 26 4.2. TUNING CPUS WITH TUNA 27 4.3. TUNING IRQS WITH TUNA 27 4.4. TUNING TASKS WITH TUNA 28 4.5. EXAMPLES OF USING TUNA 29 .C . H. .A . P. .T .E . R. 5. P. .E . R. .F . O. .R . M. A. .N . C. .E . .C . O. .- .P . I.L . O. .T . .( .P . C. .P . ). 3. .1 . 5.1. PCP OVERVIEW AND RESOURCES 31 5.2. XFS FILE SYSTEM PERFORMANCE ANALYSIS WITH PERFORMANCE CO-PILOT 31 5.3. PERFORMING MINIMAL PCP SETUP TO GATHER FILE SYSTEM DATA 38 .C . H. .A . P. .T .E . R. 6. .C . P. .U . .4 .0 . 6.1. CONSIDERATIONS 40 6.2. MONITORING AND DIAGNOSING PERFORMANCE PROBLEMS 45 6.3. CONFIGURATION SUGGESTIONS 46 .C . H. .A . P. .T .E . R. 7. M. E. .M . .O . R. .Y . .5 . 4. 7.1. CONSIDERATIONS 54 7.2. MONITORING AND DIAGNOSING PERFORMANCE PROBLEMS 54 7.3. CONFIGURING HUGETLB HUGE PAGES 58 7.4. CONFIGURING TRANSPARENT HUGE PAGES 61 7.5. CONFIGURING SYSTEM MEMORY CAPACITY 62 .C . H. .A . P. .T .E . R. 8. .S .T . O. .R . A. .G . .E . A. .N . .D . .F .I .L .E . .S . Y. .S .T . E. .M . .S . .6 .6 . 8.1. CONSIDERATIONS 66 8.2. MONITORING AND DIAGNOSING PERFORMANCE PROBLEMS 71 8.3. SOLID-STATE DISKS 74 8.4. CONFIGURATION TOOLS 75 1 Performance Tuning Guide .C . H. .A . P. .T .E . R. 9. .N . E. .T .W . .O . .R . K. I. N. .G . .8 . 7. 9.1. CONSIDERATIONS 87 9.2. MONITORING AND DIAGNOSING PERFORMANCE PROBLEMS 88 9.3. CONFIGURATION TOOLS 89 .A .P . P. .E . N. .D . I. X. A . .T .O . .O . L. R. .E .F . E. .R . E. N. C. .E . .9 .6 . A.1. IRQBALANCE 96 A.2. ETHTOOL 97 A.3. SS 97 A.4. TUNED 97 A.5. TUNED-ADM 97 A.6. PERF 99 A.7. PERFORMANCE CO-PILOT (PCP) 100 A.8. VMSTAT 104 A.9. X86_ENERGY_PERF_POLICY 105 A.10. TURBOSTAT 106 A.11. NUMASTAT 107 A.12. NUMACTL 108 A.13. NUMAD 109 A.14. OPROFILE 111 A.15. TASKSET 111 A.16. SYSTEMTAP 112 .A .P . P. .E . N. .D . I. X. B . R. .E . V. .I S. .I O. N. H. .I S. .T . O. .R . Y. ..