The Design qnd Implementation of the Clouds Distributed Operating System P. Dasgupta, R. C. Chen, S. Menon, M. P. Pearson, R. Ananthanarayanan, U. Ramachandran, M. Ahamad, R. J. LeBlanc, W. F. Appelbe, J. M. Bernabéu-Aubán, P. W. Hutto, M. Y. A. Khalidi, and C. J. Wilkenloh Georgia Institute of Technology ABSTRACT: Clouds is a native operating system for a distributed environment. The Clouds operating system is built on top of a kernel called Ra. Ra ís a second generation kernel derived from our experi- ence with the first version of the Clouds operating system. Rø is a minimal, flexible kernel that pro- vides a framework for implementing a variety of distributed operating systems. This paper presents the Clouds paradigm and a brief overview of its first implementation. We then present the details of the Rø kernel, the rationale for its design, and the system services that consti- tute the Clouds operating system. This work was funded by NSF grant CCR-8619886. @ Computing Systems, Vol. 3 . No. I . Winter 1990 11 I. Introduction Clouds is a distributed operating system built on top of a minimal kernel called Rø. The paradigm supported by Clouds provides an abstraction of storage called objects and an abstraction of execu- tion called threads. 1.1 Basic Philosophies A primary research topic of the Clouds project is the development of a set of techniques that can be used to construct a simple, usable distributed operating system. A distributed operating sys- tem should integrate a set of loosely-coupled machines into a cen- talized,work environment. The following are the requirements of such a distributed system: . The operating system must integrate a number of comput- ers, both compute servers and data servers, into one operat- ing environment. The system structuring paradigm is important in deciding the appeal of the system. This should be clean, elegant, sim- ple to use and feasible. A simple, efficient, yet effective implementation. To attain this end, we proposed the following basic philosophies: . We use the much advocated minimalíst philosophy towards operating system design. The operating system is divided into several clean, well defined modules and layers. The kernel is one such layer ofthe operating system and sup- ports only features that cannot be supported elsewhere. 1,2 Dasgupta et al. Modularity in these layers limits the amount of interference (side-efects) and gives rise to easier upgrading and debugging. Three basic mechanisms common to all general-purpose sys- tems are computation, storage and I/O. We use simple primitives to support these: namely, light-weight processes and persistent object memory. Light-weight processes are involved in computation. Persistent object memory serves for all storage needs. The need for user-level disk I/O in our model is eliminated. Terminal I/O is provided as a special case ofobject access. 1.2 Clouds Design Objectives Clouds is designed to run on a set of general purpose computers (uniprocessors or multiprocessors) that are connected via a local- area network. The major design objectives lor Clouds are: . Integration of resources through cooperation and location transparency, leading to simple and uniform interfaces for distributed processing. Support for various forms of atomicity and data con- sistency, including transaction processing, and the ability to tolerate failures. Portability, extensibility and efficient implementation. Clouds coalesces a distributed network of computers into an integrated computing environment with the look and feel of a cen- tralized timesharing system. In addition to the integration, the paradigm used for defrning and implementing the system structure of the Clouds system is a persistent object/thread model. This model provides threads to support computation and objects to support an abstraction of storage. The model has been augmented to to provide support for reliable programs [Chen & Dasgupta 1989; Chen 19901. The consistency techniques have been designed, but not implemented, and are outside the scope of this paper. The rest of this paper is organized as follows. An overview of the Clouds project is provided in Section 2, followed by an over- view of the Clouds paradigm in Section 3. The paradigm is the The Design ønd Implementation of the Clouds Distributed Operating System r3 common link between the first implementation of Cbuds (Clouds u.1) and the current version (Clouds v.2). We present the imple- mentation of Clouds v.1 in Section 4,the structure and implemen- tation of Clouds v.2 in more detail in Section 5, and the reasons behind the Clouds redesign in Section 6. Section 7 presents some of details of the Ra implementation and Section 8 describes the current state of the implementation of Cbuds on Ra. 2. Project Overview The first version of the Clouds kernel was implemented in 1986. This version is referred to as Clouds v.l and was used as an exper- imental testbed by the implementors. This implementation was successful in demonstrating the feasibility of a native operating system supporting the object model. Our experience with Clouds v./ provided insights into developing better implementations of the object/thread paradigm. The lessons learned from the first implementation have been used to redesign the kernel and build a new version of the operat- ing system called Clouds v.2. Clouds v.2 uses an object/thread paradigm which is derived from the object/process/action system [Allchin 1983; Spafford 1986; Wilkes 1987] used by Clouds v.I. However, most of the design and implementation of the system are substantially different. Clouds v.1 was targeted to be a testbed for distributed operating system research. Clouds v.2 is tatgeted to be a distributed computing platform for research in a wide variety of areas in computer science. The present implementation of Cbuds supports the persistent object/thread paradigm, a networking protocol, distributed virtual memory support, user-level I/O with virtual terminals on UNIX, a Cr+ based programming language, and some system management software. Work in progress includes user interfaces, consistency support, fault tolerance, language environments, object programming con- ventions and better programming language support for object typ- ing, instantiation, and inheritance. 14 Dasgupta et al. 3. The Clouds Paradigm All data, programs, devices, and resources in Clouds are encapsu- lated in objects. Objects represent the passive entities in the sys- tem. Activity is provided by threads, which execute within objects. 3.1 Objects A Clouds object, at the conceptual level, is a virtual address space. Unlike virtual address spaces in conventional operating systems, a Clouds object is persistent and is not tied to any thread. A Clouds object exists forever and survives system crashes and shutdowns (as does a frle) unless explicitly deleted. As will be seen in the fol- lowing description of objects, Clouds objects are somewhat "heavyweight" and are better suited for storage and execution of large-grained data and programs because invocation and storage of objects bear some non-trivial overhead. An object consists of a named address space and the contents of the address space. Since it does not contain a process, it is completely passive. Hence, unlike objects in some object based systems, a Clouds object is not associated with any server process. (The frrst system to use passive objects was Hydra [Wulf et al. 1974; Wulf et al. 19811.) The contents of each virtual address space are protected from outside access so that memory (data) in an object is accessible only by the code in that object and the operating system. Each object is an encapsulated address space with entry points at which threads may commence execution. The code that is accessible through an entry point is known as an object operation. Data cannot be transmitted in or out of the object freely, but can be passed as parameters (see the discussion on threads in Sec- tion 3.2). Each Clouds object has a global system-level name called a sysname, which is a bit-string that is unique over the entire distri- buted system. Sysnames do not include the current location of the object (objects may migrate). Therefore, the sysname-based nam- ing scheme in Clouds creates a uniform, flat system name space The Design and Implementation of the Clouds Distributed Operating System 15 for objects, and allows the object mobility needed for load balanc- ing and reconfiguration. User-level names are translated to sysnames using a nameserver. Clouds objects are programmed by the user. System utilities can be implemented using Clouds objects. A complete Clouds object can contain user-deflned code and data, and system-defrned code and data that handles synchronization and recovery. In addition, it contains a volatile heap for temporary memory alloca- tion, and a permanent heap for allocating memory that becomes a part of the data structures in the object. Locks and sysnames of other objects are also a part of the object data space. 3.2 Threads The only form of user activity in the Clouds system is the user thread. A thread can be viewed as a thread of control that runs code in objects, traversing objects and machines as it executes. A thread executes in an object by entering it through one of several entry points; after the execution is complete the thread leaves the object. The code in the object can contain a call to an operation in another object with arguments. When the thread executes this call, it temporarily leaves the calling object, enters the called object and commences execution there. The thread returns to the calling object after the execution in the called object terminates with returned results. These argurnents/results are strictly data; they may not be addresses. (Note that sysnames are data.) This restriction is necessary as addresses which are meaningful in the context of one object are meaningless in the context of another object.