SOLARIS INTERNALS Core Kernel Components i SOLARIS INTERNALS Core Kernel Components Jim Mauro and Richard McDougall Sun Microsystems Press A Prentice Hall Title © 2000 Sun Microsystems, Inc. — Printed in the United States of America. 901 San Antonio Road, Palo Alto, California 94303 U.S.A. All rights reserved. This product and related documentation are protected by copyright and distributed under licenses restricting its use, copying, distribution and decompilation. No part of this product or related documentation may be reproduced in any form by any means without prior written authoriza- tion of Sun and its licensors, if any. RESTRICTED RIGHTS LEGEND: Use, duplication, or disclosure by the United States Government is subject to the restrictions as set forth in DFARS 252.227-7013 (c)(1)(ii) and FAR 52.227-19. The product described in this manual may be protected by one or more U.S. patents, foreign patents, or pending applications. TRADEMARKS—Sun, Sun Microsystems, the Sun logo, HotJava, Solaris, SunExpress, SunScreen, SunDocs, SPARC, SunOS, and SunSoft are trademarks or registered trademarks of Sun Microsystems, Inc. All other products or services mentioned in this book are the trademarks or service marks of their respective companies or organizations. 109 87654321 ISBN 0-13-022496-0 Sun Microsystems Press A Prentice Hall Title For Traci. .. for your love and encouragement .......................................... Richard For Donna, Frankie and Dominick. All my love, always... .......................................... Jim ACKNOWLEDGEMENTS It ‘s hard to thank all people that helped us with this book. As a minimum, we owe: • Thanks to Brian Wong, Adrian Cockcroft, Paul Strong, Lisa Musgrave and Fraser Gardiner for all your help and advise for the structure and content of this book. • Thanks to Tony Shoumack, Phil Harman, Jim Moore, Robert Miller, Martin Braid, Robert Lane, Bert Beals, Magnus Bergman, Calum Mackay, Allan Packer, Magnus Bergman, Chris Larson, Bill Walker, Keith Bierman, Dan Mick and Raghunath Shenbagam for helping to review the material. • A very special thanks to David Collier-Brown, Norm Shulman, Dominic Kay, Jarod Jenson, Bob Sneed, and Evert Hoogendoorn for painstaking page by page reviews of the whole book. • Our thanks to the engineers in the Solaris business unit - Jim Litchfield, Michael Shapiro, Jeff Bonwick, Wolfgang Thaler, Bryan Cantrill, Roger Faulker, Andy Tucker, Casper Dik, Tim Marsland, Andy Rudoff, Greg Onufer, Rob Gingell, Devang Shah, Deepankar Das, Dan Price and Kit Chow for their advise and guidance. We’re quite sure there are others, and we apolo- gize up front to those whose names we have missed. • Thank you to the systems engineers and technical support staff at Sun for the corrections and suggestions along the way. • Thanks to Lou Marchant - for the endless search for engine pictures, and Dwayne Schumate at Lotus Cars USA for coordinating permission to use the images of the Lotus V8 engine. • Thanks to the folks at Prentice Hall - Greg Doench for his patience (we did slip this thing a few times) and support. vii viii Acknowledgements • Thanks to our enduring copy editor, Mary Lou Nohr for her top notch edito- rial work and style suggestions. Without your help, this book wouldn’t be what it is today. From Jim: I wish to personally acknowledge Jeff Bonwick and Andy Tucker of Solaris ker- nel engineering. They demonstrated great patience in clarifying things that were complex to me but second nature to them. They answered innumerous emails, which contributed significantly to the accuracy of the text, as well as insuring all the key points were made. They also provided some wonderful explanations in var- ious areas of the source code, which definitely helped. Roger Faulkner and Jim Litchfield, also of Solaris kernel engineering, deserve and additional note of thanks for their efforts and time. Thanks to Nobel Shelby and Casey Palowitch for reviewing sections of the manuscript and providing insightful feedback and suggestions. I owe a debt of gratitude to Hal Stern that goes way beyond his support for this work. His mentoring, guidance and friendship over the years have had a profound impact on my development at Sun. Last, but certainly not least, comes the family acknowledgment. This may appear cliche’, as every technical book I’ve ever seen recognizes the writers family in the acknowledgements section. Well, there’s a very good reason for that. There are only 24 hours in a day and 7 days in a week. That doesn’t change just because you decide to write a book, nor do the other things that demand your time, like your job, your house, your lawn, etc., all of a sudden become less demanding. So the ones that end up getting the short end of the stick is invariably your family. Thus, my deepest gratitude goes to my wife Donna, and my sons, Frankie and Dominick. Without their love, sacrifice and support, I would not have been able to complete this work. Thanks guys, I’m back now (of course, there is that pesky lit- tle matter of the updated version for Solaris 8...). Jim Mauro [email protected] Green Brook, New Jersey June, 2000 Acknowledgements ix From Richard: I would like to thank Adrian Cockcroft and Brian Wong for first giving me the opportunity to join their engineering group in 1995, working from my remote out- post in Australia. Their leadership and guidance has meant a lot to me during my career at Sun. Thank you to our friends, visitors and family who seemingly understood for 2 years when I abstained from many invites to dinners, day trips and fun events cit- ing “when the books done...”. Yes - it is done now! And yes, a special thank you to my wife Traci, who provided a seemingly end- less amount of encouragement and personal sacrifice along the way. This project would have been forever unfinished without her unquestionable co-operation and support. Richard McDougall [email protected] Cupertino, California June, 2000 x Acknowledgements PREFACE The internals of the UNIX kernel is fairly well documented, most notably by Good- heart and Cox [10], Bach [1], McKusick et al. [19], and Vahalia [39]. These texts have become a common source of reference information for those who want to bet- ter understand the internals of UNIX. However little has been written about the specifics of the Solaris kernel. The paucity of Solaris specific information led us to create our own reference material. As we published information through white papers, magazine columns, and tutorials, the number of folks expressing interest motivated us to produce a complete work that discussed Solaris exclusively. About This Book This book is about the internals of Sun’s Solaris Operating Environment. The rapid growth of Solaris has created a large number of users, software developers, systems administrators, performance analysts, and other members of the techni- cal community, all of whom require in-depth knowledge about the environment in which they work. Since the focus of this book is the internals of the Solaris kernel, the book pro- vides a great deal of information on the architecture of the kernel and the major data structures and algorithms implemented in the operating system. However, rather than approach the subject matter from a purely academic point of view, we wrote the book with an eye on the practical application of the information con- xi xii Preface tained herein. Thus, we have emphasized the methods and tools that can be used on a Solaris system to extract information that otherwise is not easily accessible with the standard bundled commands and utilities. We want to illustrate how you can apply this knowledge in a meaningful way, as your job or interest dictates. To maximize the usefulness of the text, we included specific information on Solaris versions 2.5.1, 2.6, and Solaris 7. We cover the major Solaris subsystems, including memory management, process management, threads, files, and file sys- tems. We do not cover details of low-level I/O, device drivers, STREAMS, and net- working. For reference material on these topics, see “Writing Device Drivers” [28], the “STREAMS Programming Guide” [29], and “UNIX Network Programming” [32]. The material included in this book is not necessarily presented at an introduc- tory level, although whenever possible we begin discussing a topic with some con- ceptual background information. We assume that you have some familiarity with operating systems concepts and have used a Unix-based operating system. Some knowledge of the C programming language is useful but not required. Because of the variety of hardware platforms on which Solaris runs, it is not practical to discuss the low-level details of all the different processors and architec- tures, so our hardware focus, when detail is required, is admittedly UltraS- PARC-centric. This approach makes the most sense since it represents the current technology and addresses the largest installed base. In general, the concepts put forth when detail is required apply to other processors and platforms supported. The differences are in the specific implementation details, such as per-processor hardware registers. Throughout the book we refer to specific kernel functions by name as we describe the flow of various code segments. These routines are internal to the oper- ating system and should not be construed as, or confused with, the public inter- faces that ship as part of the Solaris product line—the systems calls and library interfaces. The functions referenced throughout the text, unless explicitly noted, are private to the kernel and not callable or in any way usable by application pro- grams. Intended Audience We hope that this book will serve as a useful reference for a variety of technical staff members working with the Solaris Operating Environment.