Linux® User’s Guide 0898004-520 May 2007 Copyright 2007 by Concurrent Computer Corporation. All rights reserved. This publication or any part thereof is intended for use with Concurrent products by Concurrent personnel, customers, and end–users. It may not be reproduced in any form without the written permission of the publisher. The information contained in this document is believed to be correct at the time of publication. It is subject to change without notice. Concurrent makes no warranties, expressed or implied, concerning the information contained in this document. To report an error or comment on a specific portion of the manual, photocopy the page in question and mark the correction or comment on the copy. Mail the copy (and any additional comments) to Concurrent Computer Corporation, 2881 Gateway Drive, Pompano Beach, Florida, 33069. Mark the envelope “Attention: Publications Department.” This publication may not be reproduced for any other reason in any form without written permission of the publisher. Concurrent Computer Corporation and its logo are registered trademarks of Concurrent Computer Corporation. All other Concurrent product names are trademarks of Concurrent while all other product names are trademarks or registered trademarks of their respective owners. Linux® is used pursuant to a sublicense from the Linux Mark Institute. Printed in U. S. A. Revision History: Date Level Effective With August 2002 000 RedHawk Linux Release 1.1 September 2002 100 RedHawk Linux Release 1.1 December 2002 200 RedHawk Linux Release 1.2 April 2003 300 RedHawk Linux Release 1.3, 1.4 December 2003 400 RedHawk Linux Release 2.0 March 2004 410 RedHawk Linux Release 2.1 July 2004 420 RedHawk Linux Release 2.2 May 2005 430 RedHawk Linux Release 2.3 March 2006 500 RedHawk Linux Release 4.1 May 2006 510 RedHawk Linux Release 4.1 May 2007 520 RedHawk Linux Release 4.2 Preface Scope of Manual This manual consists of three parts. The information in Part 1 is directed towards real-time users. Part 2 is directed towards system administrators. Part 3 consists of backmatter: appendixes, glossary and index. An overview of the contents of the manual follows. Structure of Manual This guide consists of the following sections: Part 1 - Real-Time User • Chapter 1, Introduction, provides an introduction to the RedHawk Linux operating system and an overview of the real-time features included. • Chapter 2, Real-Time Performance, discusses issues involved with achieving real-time performance including interrupt response, process dispatch latency and deterministic program execution. The shielded CPU model is described. • Chapter 3, Real-Time Interprocess Communication, discusses procedures for using the POSIX® and System V message-passing and shared memory facilities. • Chapter 4, Process Scheduling, provides an overview of process scheduling and describes POSIX scheduling policies and priorities. • Chapter 5, Interprocess Synchronization, describes the interfaces provided by RedHawk Linux for cooperating processes to synchronize access to shared resources. Included are: POSIX counting semaphores, System V semaphores, rescheduling control tools and condition synchronization tools. • Chapter 6, Programmable Clocks and Timers, provides an overview of some of the RCIM and POSIX timing facilities available under RedHawk Linux. • Chapter 7, System Clocks and Timers, describes system timekeeping and the per-CPU local timer. • Chapter 8, File Systems and Disk I/O, explains the xfs journaling file system and procedures for performing direct disk I/O on the RedHawk Linux operating system. • Chapter 9, Memory Mapping, describes the methods provided by RedHawk Linux for a process to access the contents of another process’ address space. • Chapter 10, Non-Uniform Memory Access (NUMA), describes the NUMA support available on certain systems. iii RedHawk Linux User’s Guide Part 2 - Administrator • Chapter 11, Configuring and Building the Kernel, provides information on how to configure and build a RedHawk Linux kernel. • Chapter 12, Kernel Debugging, provides guidelines for saving, restoring and analyzing the kernel memory image using kdump and crash and basic use of the kdb and kgdb kernel debuggers. • Chapter 13, Pluggable Authentication Modules (PAM), describes the PAM authentication capabilities of RedHawk Linux. • Chapter 14, Device Drivers, describes RedHawk functionality and real-time issues involved with writing device drivers. • Chapter 15, PCI-to-VME Support, describes RedHawk’s support for a PCI- to-VME bridge. Part 3 - Common Material • Appendix A, Example Message Queue Programs, contains example programs illustrating the POSIX and System V message queue facilities. • Appendix B, Kernel Tunables for Real-time Features, contains a listing of the kernel tunables that control unique features in RedHawk Linux and their default values in pre-built kernels. • Appendix C, Capabilities, lists the capabilities included in RedHawk Linux and the permissions provided by each. • Appendix D, Kernel Trace Events, lists pre-defined kernel trace points and methods for defining and logging custom events within kernel modules. • Appendix E, Migrating 32-bit Code to 64-bit Code, provides information needed to migrate 32-bit code to 64-bit processing on an x86_64 processor. • Appendix F, Kernel-level Daemons on Shielded CPUs, describes how kernel-level daemons execute on shielded CPUs and provides methods for improving performance. • Appendix G, Cross Processor Interrupts on Shielded CPUs, describes how cross-processor interrupts execute on shielded CPUs and provides methods for improving performance. • Appendix H, Serial Console Setup, provides instructions for configuring a serial console. • Appendix I, Boot Command Line Parameters, discusses the boot parameters unique to RedHawk. • The Glossary provides definitions for terms used throughout this Guide. • The Index contains an alphabetical reference to key terms and concepts and the pages where they occur in the text. iv Preface Syntax Notation The following notation is used throughout this manual: italic Books, reference cards, and items that the user must specify appear in italic type. Special terms may also appear in italic. list bold User input appears in list bold type and must be entered exactly as shown. Names of directories, files, commands, options and man page references also appear in list bold type. list Operating system and program output such as prompts, messages and listings of files and programs appears in list type. [] Brackets enclose command options and arguments that are optional. You do not type the brackets if you choose to specify these options or arguments. hypertext links When viewing this document online, clicking on chapter, section, fig- ure, table and page number references will display the corresponding text. Clicking on Internet URLs provided in blue type will launch your web browser and display the web site. Clicking on publication names and numbers in red type will display the corresponding manual PDF, if accessible. Related Publications The following table lists RedHawk Linux documentation. Click on the red entry to display the document PDF (optional product documentation is available for viewing only if the optional product has been installed). These documents are also available by clicking on the “Documents” icon on the desktop and from Concurrent’s web site at www.ccur.com. RedHawk Linux Operating System Documentation Pub. Number RedHawk Linux Online Documentation Roadmap 0898002 RedHawk Linux OS Release Notes 0898003 RedHawk Linux User’s Guide 0898004 Real-Time Clock & Interrupt Module (RCIM) PCI Form Factor User’s Guide 0898007 iHawk Optimization Guide 0898011 RedHawk Linux FAQ N/A Partner Documentation SBS Technologies Model 1003 Linux Support Software 85221990 SBS Technologies Model 618-3 Adapter Hardware 85851150 Guide to SNARE for Linux N/A Optional RedHawk Product Documentation RedHawk Linux Frequency-Based Scheduler (FBS) User’s Guide 0898005 RedHawk Linux Cluster Manager User’s Guide 0898016 RedHawk High Availability NFS (HA-NFS) Installation Guide 0898018 RedHawk OpenFabrics Enterprise Distribution (OFED) Installation Guide 0898019 RedHawk Global File System (GFS) Installation Guide 0898020 v RedHawk Linux User’s Guide vi Chapter 0Contents Preface . iii Chapter 1 Introduction Overview. 1-1 RedHawk Linux Kernels . 1-3 System Updates. 1-4 Real-Time Features. 1-4 Processor Shielding . 1-4 Processor Affinity . 1-4 User-level Preemption Control . 1-5 Fast Block/Wake Services . 1-5 RCIM Driver . 1-5 Frequency-Based Scheduler . 1-5 /proc Modifications . 1-6 Kernel Trace Facility . 1-6 ptrace Extensions. 1-6 Kernel Preemption. 1-6 Real-Time Scheduler . 1-7 Low Latency Enhancements . 1-7 Priority Inheritance . 1-7 High Resolution Process Accounting . 1-7 Capabilities Support . 1-7 Kernel Debuggers . 1-8 Kernel Core Dumps/Crash Analysis . 1-8 User-level Spin Locks . 1-8 usermap and /proc mmap. 1-8 Hyper-threading. 1-9 XFS Journaling File System . 1-9 POSIX Real-Time Extensions . 1-9 User Priority Scheduling . 1-9 Memory Resident Processes. 1-10 Memory Mapping and Data Sharing . 1-10 Process Synchronization. 1-10 Asynchronous Input/Output . 1-10 Synchronized Input/Output . 1-11 Real-Time Signal Behavior . 1-11 Clocks and Timers . 1-11 Message Queues . 1-11 Chapter 2 Real-Time Performance Overview of the Shielded CPU Model . 2-1 Overview of Determinism . 2-2 Process Dispatch Latency . 2-2 Effect of Disabling Interrupts . 2-4 Effect of Interrupts. 2-5 Effect of Disabling Preemption . 2-8 vii RedHawk Linux User’s Guide Effect of Open Source Device Drivers . 2-9 How Shielding Improves Real-Time Performance. 2-9 Shielding From Background Processes . 2-9 Shielding From Interrupts . 2-10 Shielding From Local Interrupt . 2-11 Interfaces to CPU Shielding . 2-12 Shield Command . 2-12 Shield Command Examples . 2-13 Exit.