Red Hat Enterprise Linux for Real Time 7 Tuning Guide
Total Page:16
File Type:pdf, Size:1020Kb
Red Hat Enterprise Linux for Real Time 7 Tuning Guide Advanced tuning procedures for Red Hat Enterprise Linux for Real Time Radek Bíba David Ryan Cheryn Tan Lana Brindley Alison Young Red Hat Enterprise Linux for Real Time 7 Tuning Guide Advanced tuning procedures for Red Hat Enterprise Linux for Real Time Radek Bíba Red Hat Customer Content Services [email protected] David Ryan Red Hat Customer Content Services [email protected] Cheryn Tan Red Hat Customer Content Services Lana Brindley Red Hat Customer Content Services Alison Young Red Hat Customer Content Services Legal Notice Copyright © 2015 Red Hat, Inc. This document is licensed by Red Hat under the Creative Commons Attribution-ShareAlike 3.0 Unported License. If you distribute this document, or a modified version of it, you must provide attribution to Red Hat, Inc. and provide a link to the original. If the document is modified, all Red Hat trademarks must be removed. Red Hat, as the licensor of this document, waives the right to enforce, and agrees not to assert, Section 4d of CC-BY-SA to the fullest extent permitted by applicable law. Red Hat, Red Hat Enterprise Linux, the Shadowman logo, JBoss, MetaMatrix, Fedora, the Infinity Logo, and RHCE are trademarks of Red Hat, Inc., registered in the United States and other countries. Linux ® is the registered trademark of Linus Torvalds in the United States and other countries. Java ® is a registered trademark of Oracle and/or its affiliates. XFS ® is a trademark of Silicon Graphics International Corp. or its subsidiaries in the United States and/or other countries. MySQL ® is a registered trademark of MySQL AB in the United States, the European Union and other countries. Node.js ® is an official trademark of Joyent. Red Hat Software Collections is not formally related to or endorsed by the official Joyent Node.js open source or commercial project. The OpenStack ® Word Mark and OpenStack Logo are either registered trademarks/service marks or trademarks/service marks of the OpenStack Foundation, in the United States and other countries and are used with the OpenStack Foundation's permission. We are not affiliated with, endorsed or sponsored by the OpenStack Foundation, or the OpenStack community. All other trademarks are the property of their respective owners. Abstract This book contains advanced tuning procedures for Red Hat Enterprise Linux for Real Time. For installation instructions, see the Red Hat Enterprise Linux for Real Time Installation Guide. T able of Cont ent s Table of Contents .P .r e. f. a. c. e. 2. .C .h .a .p . t.e .r . 1. .. B. .e .f o. r. e. .Y .o .u . .S .t .a .r t. .T .u . n. i.n .g . .Y .o .u . r. R. .e .d . H. .a .t .E . n. t. e. r.p .r .i s. e. .L .i n. u. x. .f .o .r .R . e. a. l. T. .i m. .e . S. y. s. t. e. m. 3. .C .h .a .p . t.e .r . 2. .. G. .e .n .e .r .a .l .S .y .s .t .e .m . .T .u . n. i.n .g . 5. 2.1. Using the Tuna Interface 5 2.2. Setting Persistent Tuning Parameters 5 2.3. Setting BIOS Parameters 6 2.4. Interrup t and Pro cess Bind ing 7 2.5. File System Determinism Tip s 10 2.6 . Using Hard ware Clo cks fo r System Timestamp ing 11 2.7. Avo id Running Extra Ap p licatio ns 13 2.8 . Swap p ing and Out o f Memo ry Tip s 14 2.9 . Netwo rk Determinism Tip s 16 2.10 . syslo g Tuning Tip s 17 2.11. The PC Card Daemo n 18 2.12. Red uce TCP Perfo rmance Sp ikes 18 .C .h .a .p . t.e .r . 3. .. R. e. a. l.t .i m. .e .- .S .p .e .c .i f. i.c . T. u. .n .i n. g. 2. 0. 3.1. Setting Sched uler Prio rities 20 3.2. Using kd ump and kexec with the Red Hat Enterp rise Linux fo r Real Time Kernel 22 3.3. TSC Timer Synchro nizatio n o n Op tero n CPUs 25 3.4. Infinib and 26 3.5. Ro CEE and Hig h Perfo rmance Netwo rking 26 3.6 . No n-Unifo rm Memo ry Access 27 3.7. Red ucing the TCP Delayed Ack Timeo ut 27 3.8 . Using d eb ug fs 27 3.9 . Using the ftrace Utility fo r Tracing Latencies 27 3.10 . Latency Tracing Using trace-cmd 31 3.11. Using sched _nr_mig rate to Limit SCHED_OTHER Task Mig ratio n. 33 3.12. Real Time Thro ttling 33 .C .h .a .p . t.e .r . 4. .. A. p. .p .l i.c .a .t .i o. n. .T . u. n. .i n. g. .a .n . d. .D . e. p. l.o .y .m . e. n. .t . 3. 5. 4.1. Sig nal Pro cessing in Realtime Ap p licatio ns 35 4.2. Using sched _yield and Other Synchro nizatio n Mechanisms 35 4.3. Mutex Op tio ns 36 4.4. TCP_NODELAY and Small Buffer Writes 38 4.5. Setting Realtime Sched uler Prio rities 39 4.6 . Lo ad ing Dynamic Lib raries 39 4.7. Using _COARSE POSIX Clo cks fo r Ap p licatio n Timestamp ing 40 4.8 . Ab o ut Perf 42 .C .h .a .p . t.e .r . 5. .. M. .o .r e. .I n. .f o. r. m. .a .t i.o . n. 4. 6. 5.1. Rep o rting Bug s 46 .E .v .e .n .t . T. .r a. c. i.n .g . 4. 7. .D .e .t .a .i l.e .d . D. .e .s .c .r i.p . t.i o. .n . o. f. .F .t r. a. c. e. 4. 8. .R .e .v .i s. i.o . n. .H . i.s .t o. .r y. 1. 0. 5. 1 Red Hat Ent erprise Linux for Real T ime 7 T uning Guide Preface This book detailed tuning information about Red Hat Enterprise Linux for Real Time. Many industries and organizations need extremely high performance computing and may require low and predictable latency, especially in the financial and telecommunications industries. Latency, or response time, is defined as the time between an event and system response and is generally measured in microseconds (μs). For most applications running under a Linux environment, basic performance tuning can improve latency sufficiently. For those industries where latency not only needs to be low, but also accountable and predictable, Red Hat has now developed a 'drop-in' kernel replacement that provides this. Red Hat Enterprise Linux for Real Time provides seamless integration with Red Hat Enterprise Linux 7 and offers clients the opportunity to measure, configure, and record latency times within their organization. You will need to have the Red Hat Enterprise Linux for Real Time kernel installed before you begin the tuning procedures in this book. If you have not yet installed the Red Hat Enterprise Linux for Real Time kernel, or need help with installation issues, read the Red Hat Enterprise Linux for Real Time Installation Guide. 2 Chapt er 1 . Before You St art T uning Your Red Hat Ent erprise Linux for Real T ime Syst em Chapter 1. Before You Start Tuning Your Red Hat Enterprise Linux for Real Time System Red Hat Enterprise Linux for Real Time is designed to be used on well-tuned systems for applications with extremely high determinism requirements. Kernel system tuning offers the vast majority of the improvement in determinism. For example, in many workloads thorough system tuning improves consistency of results by around 90%. This is why we typically recommend that customers first perform the Chapter 2, General System Tuning of standard Red Hat Enterprise Linux before using Red Hat Enterprise Linux for Real Time. Things to Remember While You Are Tuning Your Red Hat Enterprise Linux for Real Time Kernel 1. Be Patient Realtime tuning is an iterative process; you will almost never be able to tweak a few variables and know that the change is the best that can be achieved. Be prepared to spend.