ABSTRACT SOFTWARE-ORIENTED MEMORY-MANAGEMENT DESIGN by Bruce Ledley Jacob Chair: Trevor N. Mudge Changing trends in technologies, notably cheaper and faster memory hierarchies, have made it worthwhile to revisit many hardware-oriented design decisions made in previous decades. Hardware-oriented designs, in which one uses special-purpose hardware to perform some dedi- cated function, are a response to a high cost of executing instructions out of memory; when caches are expensive, slow, and/or in scarce supply, it is a perfectly reasonable reaction to build hardware state machines that do not compete with user applications for cache space and do not rely on the performance of the caches. In contrast, when the caches are large enough to withstand competition between the application and operating system, the cost of executing operating system functions out of the memory subsystem decreases significantly, and software-oriented designs become viable. Software-oriented designs, in which one dispenses with special-purpose hardware and instead per- forms the same function entirely in software, offer dramatically increased flexibility over hardware state machines at a modest cost in performance. This dissertation explores a software-oriented design for a virtual memory management system. It shows not only that a software design is more flexible than hardware designs, but that a software scheme can perform as well as most hardware schemes. Eliminating dedicated special- purpose hardware from processor design saves chip area and reduces power consumption, thus lowering the overall system cost. Moreover, a flexible design aids in the portability of system soft- ware. A software-oriented design methodology should therefore benefit architects of many differ- ent microprocessor designs, from general-purpose processors in PC-class and workstation-class computers, to embedded processors where cost tends to have a higher priority than performance. The particular implementation described in the following chapters, which is centered around a vir- tual cache hierarchy managed by the operating system, is shown to be useful for real-time systems, shared-memory multiprocessors, and architecture emulation. SOFTWARE-ORIENTED MEMORY-MANAGEMENT DESIGN by Bruce Ledley Jacob A dissertation submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy (Computer Science and Engineering) in the University of Michigan 1997 Doctoral Committee: Professor Trevor Mudge, Chair Adjunct Assistant Professor Charles Antonelli Professor Richard Brown Assistant Professor Peter Chen Associate Professor Farnam Jahanian This book is merely a personal narrative, and not a pretentious history or a philosophical dissertation. It is a record of several years of variegated vagabondizing, and its object is rather to help the resting reader while away an idle hour than afflict him with metaphysics, or goad him with science. Still, there is quite a good deal of information in the book. I regret this very much; but really it could not be helped: information appears to stew out of me naturally, like the precious ottar of roses out of the otter. Sometimes it has seemed that I would give worlds if I could retain my facts; but it cannot be. The more I caulk up the sources, and the tighter I get, the more I leak wisdom. Therefore, I can only claim indulgence at the hands of the reader, not justification. Excerpted from Prefatory, Roughing It — Mark Twain Bruce Ledley Jacob © 1997 All Rights Reserved For my Family (my family, my family-in-law, and most especially my truly adorable wife) ii ACKNOWLEDGMENTS Thanks to the members of my thesis committee, who have given me good advice and per- spective over the last few years in the classroom, at informal hallway meetings, and at various local restaurants. Thanks especially to my thesis chair, whose dogged question-asking indirectly unearthed much found herein. iii PREFACE In the fall of 1995 we began a project to build a 1GHz PowerPC processor in gallium ars- enide (GaAs); my duties included the design of the memory management system, both software and hardware. The problem to solve initially was threefold: 1. Reduce transistor count 2. Reduce complexity 3. Increase performance These could be restated as: 1. Make it small 2. Make it simple 3. Make it fast These goals may appear over-simplistic; I believe that they are not. What is not immedi- ately obvious is that they are in order of priority, and that when viewed as requirements and not simply good engineering advice, they define a clear path to achieving our performance goals. To explain their rationale: if we could not make our design small, we would almost certainly not be able to build it at all—GaAs does not allow one the luxury of a large design. If it could not be made simple, we would probably not be able to debug it. If it could not be made fast, we would lose face but the design would still work. Therefore, we considered it worthwhile to sacrifice a small amount of performance if doing so would make the design smaller or simpler. One maxim that can be drawn from these requirements is: whenever it is possible to imple- ment a given function in software it is worthwhile to do so, unless the performance cost of doing so is prohibitive. The thesis work presented in this dissertation is the result of investigating the valid- ity and implications of that maxim in the domain of memory management. The memory-management design that adheres to this maxim is a software-oriented one. A software-oriented design is one in which the designer eliminates a piece of special-purpose hard- ware that performs some dedicated function and instead performs the same function entirely in software. For instance, the translation lookaside buffer (TLB) is a specialized hardware structure iv that performs a dedicated function; it provides protection and address translation for physically indexed or physically tagged caches. It is not needed if one uses virtually addressed caches (except for the protection function, which can be ignored or supported by keeping protection information in each cache line). We eliminated the traditional TLB structure and replaced it with virtually indexed, virtually tagged caches that are managed by the operating system—in such a scheme, address translation is performed not in hardware but entirely in software by system-level software routines. A hardware-oriented PowerPC memory-management architecture has essentially three TLBs: the segment registers, the traditional page-oriented TLB, and the superpage-oriented BAT registers (Block-Address Translation). The segment registers are required to implement the Pow- erPC’s segmented address space. With a software-oriented design, we were able to eliminate the remaining two TLB structures at virtually no performance cost. Once our investigations showed the scheme to be viable (its performance is roughly that of a system with a TLB), it occurred to us that we had designed a system that could potentially com- pete in performance with any memory-management scheme, but which offered dramatically increased flexibility over traditional designs. Thus, this memory-management organization might be useful even if one is not building a processor in a resource-poor technology such as GaAs. This dissertation explores the possibilities for such a design, comparing the performance and physical requirements (e.g. chip area) of a hardware-oriented scheme against the requirements of a soft- ware-oriented one. We discuss the flexibility of a software-oriented design and, briefly, its benefits for multiprocessor systems, real-time systems, architecture emulation, and reconfigurable comput- ing. v TABLE OF CONTENTS DEDICATION ................................................................................................................................ ii ACKNOWLEDGMENTS ............................................................................................................. iii PREFACE ...................................................................................................................................... iv TABLE OF CONTENTS .............................................................................................................. vi LIST OF FIGURES ....................................................................................................................... xi LIST OF TABLES ........................................................................................................................ xv CHAPTER 1 INTRODUCTION: VIRTUAL MEMORY, THREE DECADES LATER ............................................................................................. 1 1.1 ...... Motivation ........................................................................................................... 1 1.2 ...... Background and Previous Work in Memory Management Design ..................... 5 1.3 ...... Dissertation Overview ......................................................................................... 7 1.4 ...... Scope of the Thesis .............................................................................................. 9 CHAPTER 2 A VIRTUAL MEMORY PRIMER ................................................................................. 10 2.1 ...... Introduction ....................................................................................................... 10 2.2 ...... What Is Virtual Memory? .................................................................................. 11 2.2.1 ... Three Models