This page intentionally left blank Operating Systems in Depth This page intentionally left blank OPERATING SYSTEMS IN DEPTH Thomas W. Doeppner Brown University JOHN WILEY & SONS, INC. vice-president & executive publisher Donald Fowley executive editor Beth Lang Golub executive marketing manager Christopher Ruel production editor Barbara Russiello editorial program assistant Mike Berlin senior marketing assistant Diana Smith executive media editor Thomas Kulesa cover design Wendy Lai cover photo Thomas W. Doeppner Cover photo is of Banggai Cardinalfi sh (Pterapogon kauderni), taken in the Lembeh Strait, North Sulawesi, Indonesia. This book was set in 10/12 Times Roman. The book was composed by MPS Limited, A Macmillan Company and printed and bound by Hamilton Printing Company. This book is printed on acid free paper. ϱ Founded in 1807, John Wiley & Sons, Inc. has been a valued source of knowledge and understanding for more than 200 years, helping people around the world meet their needs and fulfi ll their aspirations. Our company is built on a foundation of principles that include responsibility to the communities we serve and where we live and work. In 2008, we launched a Corporate Citizenship Initiative, a global effort to address the environmental, social, economic, and ethical challenges we face in our business. Among the issues we are addressing are carbon impact, paper specifi cations and procurement, ethical conduct within our business and among our vendors, and community and charitable support. For more information, please visit our website: www.wiley.com/go/citizenship. Copyright © 2011 by John Wiley & Sons, Inc. All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, electronic, mechanical, photocopying, recording, scanning or otherwise, except as permitted under Sections 107 or 108 of the 1976 United States Copyright Act, without either the prior written permission of the Publisher, or authorization through payment of the appropriate per-copy fee to the Copyright Clearance Center, Inc., 222 Rosewood Drive, Danvers, MA 01923 (Web site: www.copyright.com). Requests to the Publisher for permission should be addressed to the Permissions Department, John Wiley & Sons, Inc., 111 River Street, Hoboken, NJ 07030-5774, (201) 748-6011, fax (201) 748-6008, or online at: www. wiley.com/go/permissions. Evaluation copies are provided to qualifi ed academics and professionals for review purposes only, for use in their courses during the next academic year. These copies are licensed and may not be sold or transferred to a third party. Upon completion of the review period, please return the evaluation copy to Wiley. Return instructions and a free of charge return shipping label are available at: www.wiley.com/ go/returnlabel. Outside of the United States, please contact your local representative. Library of Congress Cataloging in Publication Data: Doeppner, Thomas W. Operating systems in depth / Thomas W. Doeppner. p. cm. Includes index. ISBN 978-0-471-68723-8 (hardback) 1. Operating systems (Computers) I. Title. QA76.76.O63D64 2010 005.4’3—dc22 2010034669 Printed in the United States of America 10 9 8 7 6 5 4 3 2 1 To the memory of my father, Thomas W. Doeppner Sr. This page intentionally left blank Preface he goal of this book is to bring together and explain current practice in operating systems. TThis includes much of what is traditionally covered in operating-system textbooks: concur- rency, scheduling, linking and loading, storage management (both real and virtual), fi le systems, and security. However, I also cover issues that come up every day in operating-systems design and implementation but are not often taught in undergraduate courses. For example, I cover: • Deferred work, which includes deferred and asynchronous procedure calls in Windows, tasklets in Linux, and interrupt threads in Solaris. • The intricacies of thread switching, on both uniprocessor and multiprocessor systems. • Modern fi le systems, such as ZFS and WAFL. • Distributed fi le systems, including CIFS and NFS version 4. AUDIENCE This book is based on material I’ve developed over the past 30+ years for my operating-systems course at Brown University and is suitable for a one-semester course for advanced undergradu- ates and beginning graduate students. Students taking the course at Brown have generally had an introductory course on computer architecture and an advanced programming course. The course investigates in depth what’s done in current operating systems, and its signifi cant programming projects make students come to grips with major operating-system components and attain an intimate understanding of how they work. But certainly not all students in an OS course want to go on to concentrate in the area, let alone work in it. A course based on this text must be accessible to these students as well. This issue is handled at Brown by letting students choose one of two projects (discussed later in this preface). The fi rst, relatively straightforward, project involves writing a user-level threads library, a fi le-system cache manager, and a simple fi le system. The second, for the truly interested gung-ho students, is to implement a good portion of a simple but fully functional operating system. (Those portions that are not pedagogically useful for students to write are provided to them.) Students completing this latter project get a half course of additional credit (Brown’s full courses and half courses are equivalent to four-credit and two-credit courses in schools using the credit system). TOPIC COVERAGE Part of the challenge in writing a textbook is not only choosing the topics to cover but also determining the order of presentation. There is the usual conundrum: to appreciate the individual viivii viii PREFACE topics one must fi rst know how they fi t together, but to appreciate how they fi t together one must fi rst know the individual topics. Compounding this problem is my belief that students must complete a comprehensive programming project in order to understand and appreciate how operating systems work. Thus the material must be presented so as to make such a project possible. Chapter 1, Introduction, introduces the reader to what an operating system is. It discusses at an abstract level both how they’re structured and what they do and provides a brief history. Then, to make things concrete and to introduce the notions of processes and fi les, I briefl y describe early Unix (Sixth Edition, from 1975). This 35-year-old OS is attractive both because of its simplicity and elegance and because, at the level presented here, it’s not all that different from either modern Unix systems or modern Windows. Exploring the concept of a process has traditionally been a fundamental part of an operating systems course. However, the meaning of this term has changed over the years. As used in most of today’s operating systems, a process is what was originally called a computation. Most systems now use the term thread to refer to what was originally meant by process. Our introduction to the process concept in Chapter 1 uses the contemporary notion of a process; we spend most of Chapter 2, Multithreaded Programming, discussing threads. It’s really important that students doing an operating-systems project understand concurrency—and by “understand” I mean not simply comprehending the concepts but also becoming profi cient in multithreaded programming. Chapter 2 is a tutorial on how to write multithreaded programs, both using POSIX threads and Windows threads. Though the material covers such programming from an application-program perspective, both the concepts and the practice apply to programming within an operating system. In this chapter, I go over the notion of deadlocks. Earlier operating-systems textbooks have spent a great deal of time on deadlocks, describing algorithms for deadlock detection and avoidance. In practice, such algorithms have little or no role in operating systems, so I simply discuss deadlock avoidance. Chapter 3, Basic Concepts, covers a collection of topics students may have met in previous courses; they aren’t all strictly operating-systems material, but they are essential for understanding operating systems. Included here is a discussion of procedure calls and various forms of context switching at an assembler-language level. We talk a lot about context switching and its cost when discussing operating systems, so students really must understand what’s involved. This discussion is given both in terms of x86 assembler language and SPARC assembler language. The latter is done to balance the x86-orientation of this part of the chapter but may safely be skipped. Also included in the chapter is a rather simple discussion of I/O architectures, important both for programming projects and for understanding I/O handling in operating systems. I cover dynamic storage-allocation algorithms, following in part Knuth’s classic treatment, with additional material on slab allocation. Though this topic could well be considered appropriate for an algorithms course, it doesn’t seem to be taught there and it’s defi nitely important in operating systems. I discuss linkers and loaders, concepts that also seem to be much neglected elsewhere in the curriculum. Finally, I go over how a system is booted. Up to Chapter 4, Operating-System Design, I really haven’t explained what’s in an operating system, except briefl y in the introduction. In Chapter 4, I go over standard operating-system components, discuss fl ow of control in terms of both threads and interrupt handling, and introduce structuring techniques for operating systems such as monolithic kernels, microkernels, and virtual machines. I talk about some of the layering that’s used, such as the virtual fi le system (VFS) layer in Unix systems and its equivalent in Windows.