FLEXATo ol for Building Ecient and Flexible Systems John B Carter Bryan Ford Mike Hibler RavindraKuramkote Jerey Law Jay Lepreau Douglas B Orr Leigh Stol ler and Mark Swanson University of Utah Department of Computer Science Abstract Mo dern op erating systems must supp ort a wide variety of services for a diverse set of users Designers of these systems face a tradeo b etween functionality and p erformance Systems likeMachprovide a set of general abstractions and attempt to handle every situation which can lead to p o or p erformance for common cases Other systems such as Unix provide a small set of abstractions that can b e made very ecient at the exp ense of functionalityWe are implementing a exible system building to ol FLEX that allows us to supp ort a p owerful op erating systems interface eciently by constructing sp ecialized mo dule implementations at runtime FLEX improves the p erformance of existing systems by optimizing interpro cess communications paths and relo cating servers and clients to reduce communications overhead These facilities improve the p erformance of Unix system calls on Mach from Furthermore FLEX can dynamically extend the kernel in a controlled fashion which gives user programs access to privileged data and devices not envisioned by the original op erating system implementor Overview Op erating systems consist of a collection of interrelated entities that provide a variety of services Beyond this general description the sp ecics of individual op erating systems varies widely dep ending on the decisions made by each systems designers We b elieve that there is no optimal organization b ecause the optimal design is dep endent on a particular systems resources and the needs of its clients No single op erating system structure can p erform optimally in a wide varietyofenvironments The most imp ortant asp ect of an op erating system from the users p oint of view is its application programming interface denition As long as the op erating system eciently supp orts its interface and protection requirements it is irrelevant whether this is provided via functions found at known lo cations in the user address space eg MSDOS via traps to the kernel eg Unix via remote pro cedure calls RPCs to server pro cesses eg much of Mach or any combination of these options To exploit the exibility that this p ersp ectiveprovides we are developing the FLEX system building to ol FLEX is a coarsegrained system building service that allows systems to b e dynamically constructed using the implementation most appropriate to a given situation In this context a system is an entity that interacts with the op erating system plus the op erating system itself FLEX provides functionalitytoan op erating system similar to but more p owerful than that provided to a programming system by a linker FLEX works by reading modules manipulating them as sp ecied by mo dule manipulation les and writing the resulting executable either to a users address space or to a le for later use The activeentityin FLEX is the OMOS server which runs as a privileged pro cess It p erforms several functions including generating relo cating and binding ob ject les caching relo cated ob ject les for later use managing address spaces and interacting with the kernel to add or delete kernel routines on demand FLEX can improvean op erating system in two dierentways i by improving the p erformance of existing system implementations and ii by dynamically extending op erating system functionality One of the goals of FLEX is to allow the implementations of existing systems to b e improved without requiring their comp onent mo dules or interfaces to change FLEX uses interface denition sp ecications to dene how individual system comp onents interact with one another This sp ecication allows FLEX to sp ecialize instances of a system resource or communications channel to a particular situation For example when a client program rst connects to a particular service FLEX can determine where the server is phys ically lo cated and provide the most ecientinterpro cess communications mechanism for this clientserver interaction If the server is on a remote machine then RPC is appropriate while if the client and server have b een placed in the same address space a capabilityof FLEX discussed b elow then some form of pro cedure call is preferable Similarlyif FLEX is able to determine that most of the interactions with a server are coming from a particular client it can intervene to make those interactions less exp ensive eg by mapping the clientinto the server address space in a controlled manner FLEXs other asp ect is its ability to extend or mo dify the op erating system dynamically in a mostly transparentway FLEX can mo dify the op erating system on the y by binding routines to the kernel on a p er client basis This abilityallows users or op erating systems designers to add or mo dify kernel functionality The means by which this can b e done safely is discussed in the following section This feature of FLEX can b e exploited in a number of ways For example this might b e used to give an unprivileged user program direct control over its physical memory so that it can use a paging algorithm tuned to its particular access b ehavior or realtime needs A long term goal of this asp ect of FLEX is to allow op erating systems researchers and vendors to distribute functional mo dications or extensions to existing op erating systems to sites without access to the op erating system source co de Design of FLEX FLEXs p ower comes from OMOSs ability to manipulate mo dules in complex ways its full access to interface denitions its ability to derive sp ecialized implementations of mo dules and their interactions and its gener alized constraint system for managing address spaces A mo dule is an entitycontaining a number of symbols or linkage p oints and optionally a fragment of executable co de in the machine language of the target sys tem Most mo dules pro cessed by FLEX are normal relo catable ob ject les pro duced by a compiler but there are also several kinds of interface denition mo dules FLEX uses a Lisplike mo dule manipulation language develop ed as part of the Jigsaw framework to sp ecify how particular mo dules are to b e combined This language provides a complete implementation of mo dule name manipulation muchmorepowerful than exist ing linkers Augmented with detailed knowledge of communication mechanisms and corresp onding argument semantics this allows FLEX to transparently sp ecialize intermo dule communication Within a mo dule a symb ol can either b e a declaration or a reference Declarations are most often asso ciated with co de fragments that implement a particular system service while references are most often asso ciated with co de fragments that invoke a particular system service FLEX is able to improveupon existing system implementations by transparently substituting a particular declaration of a symb ol with a functionally equivalent implementation that provides the same interface and semantics The details of the lowlevel binding mechanism are b eyond the scop e of this pap er but are describ ed elsewhere For example if a client pro cess contains a reference to a system call that normally is implemented as an RPC to a userlevel server pro cess whichishowmany Unix system calls are emulated in Mach it can replace the RPC interface with an LRPClikeinterface if it recognizes that the server is always lo cal LRPC exploits memory sharing and thread migration to reduce the overhead of an RPC but it is only p ossible between two pro cesses on the same machine Implementing the full LRPC mechanism requires mo dications to the Mach thread mo del to supp ort thread migration Similarly if the service is currently b eing provided by a server that has b een loaded into the kernel address space FLEX can replace the RPC with a trap that directly invokes the desired service These examples demonstrate the p otential that FLEX has for improving the qualityofexisting systems In addition to b eing able to sp ecialize interfaces FLEX is able to dynamically extend the op erating system kernel or relo cate services in a transparentway The simplest example of how this can b e accomplished arises when FLEX detects that a desired service is not currently b eing provided as can o ccur if the server crashes or cho oses to quit due to inactivity In this case FLEX can restart the appropriate server An incremental improvementuponthisisthat FLEX can start a new instance of the server that is sp ecialized to the clients needs and make the appropriate binding FLEX go es b eyond these simple mechanisms by allowing mo dules to b e relo cated b etween address spaces to provide more ecientintermo dule interactions For example if FLEX observes or is informed that a particular userlevel service is used frequently and the server is trusted it can bind the service into the kernel and replace calls to that server with traps This use of FLEX blurs the distinction b etween macrokernels and microkernels whichwe b elieve should b e the case The microkernel organization improves co de mo dularityandprovides an easy to understand design philosophy but when a rigid enforcement of the microkernel philosophy impacts p erformance without signicant gains it should b e relaxed transparently to regain the p erformance b enets of a macrokernel system Enabling optimization of more interactions FLEX can transparently load clients and trusted servers in a single address space while protecting the servers from client accesses