Portland State University PDXScholar Dissertations and Theses Dissertations and Theses 1-1-2011 A Functional Approach to Memory-Safe Operating Systems Rebekah Leslie Portland State University Follow this and additional works at: https://pdxscholar.library.pdx.edu/open_access_etds Let us know how access to this document benefits ou.y Recommended Citation Leslie, Rebekah, "A Functional Approach to Memory-Safe Operating Systems" (2011). Dissertations and Theses. Paper 499. https://doi.org/10.15760/etd.499 This Dissertation is brought to you for free and open access. It has been accepted for inclusion in Dissertations and Theses by an authorized administrator of PDXScholar. Please contact us if we can make this document more accessible: [email protected]. A Functional Approach to Memory-Safe Operating Systems by Rebekah Leslie A dissertation submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy in Computer Science Dissertation Committee: Mark P. Jones, Chair James Hook Andrew Tolmach Jonathan Walpole Gernot Heiser Jeanette R. Palmiter Portland State University c 2011 i ABSTRACT Purely functional languages|with static type systems and dynamic memory man- agement using garbage collection|are a known tool for helping programmers to reduce the number of memory errors in programs. By using such languages, we can establish correctness properties relating to memory-safety through our choice of implementation language alone. Unfortunately, the language characteristics that make purely functional languages safe also make them more difficult to apply in a low-level domain like operating systems construction. The low-level features that support the kinds of hardware manipulations required by operating systems are not typically available in memory-safe languages with garbage collection. Those that are provided may have the ability to violate memory- and type-safety, destroying the guarantees that motivate using such languages in the first place. This work demonstrates that it is possible to bridge the gap between the require- ments of operating system implementations and the features of purely functional languages without sacrificing type- and memory-safety. In particular, we show that this can be achieved by isolating the potentially unsafe memory operations required by operating systems in an abstraction layer that is well integrated with a purely functional language. The salient features of this abstraction layer are that the operations it exposes are memory-safe and yet sufficiently expressive to support the implementation of realistic operating systems. The abstraction layer enables systems programmers to perform all of the low-level tasks necessary in an OS implementation, such as manipulating an MMU and executing user-level ii programs, without compromising the static memory-safety guarantees of program- ming in a purely functional language. A specific contribution of this work is an analysis of memory-safety for the abstraction layer by formalizing a meaning for memory-safety in the presence of virtual-memory using a novel application of non- interference security policies. In addition, we evaluate the expressiveness of the abstraction layer by implementing the L4 microkernel API, which has a flexible set of virtual memory management operations. iii DEDICATION For Joe Hurd, who inspires me every day and without whose loving support this work would not have been possible. iv ACKNOWLEDGMENTS This research could not have been accomplished without the help of countless others. First and foremost, I would like to thank my advisor, Mark Jones, for his unwavering support throughout my graduate school career. In my early days as a student his courses inspired and challenged me, and our initial projects together set the direction for my research career. In the years that followed, Mark has made every effort to be involved with my work and to help me succeed. This work has benefited immensely from his contributions, in both research content and in presentation. The members of the Hasp and Programatica research projects have always provided an intellectually stimulating and friendly climate in which to conduct research. Andrew Tolmach made significant contributions to the design of the ab- straction layer presented here, including the memory-safety analysis, and directly influenced its implementation through his work on the House operating system and the original H interface. James Hook has also been extremely supportive, encouraging me to see the bigger picture and the broader connections of my work. My fellow students have provided me with many valuable discussions and pointers over the years. I am particularly grateful to Iavor Diatchki and Emir Pa˘sali´c,who befriended me in those early days when graduate school was a frightening and overwhelming place. I am also very grateful to Tim Chevalier and John Ochsner for proofreading this document. I owe a great debt to my colleagues upon whose work my abstraction layer v is based, as well as the maintainers of the bare metal Haskell compiler. Thomas Hallgren made significant contributions to the House operating system and the first version of the H interface, and has always been available to answer questions and to help with debugging. Kenneth Graunke created the version of the Haskell run- time system that my implementation relies on, leveraging the work of Adam Wick. In addition to his technical contributions, Adam has been an excellent resource throughout the process of writing my dissertation, providing a rare example of person in the local functional programming community doing primarily low-level systems work. Don Stewart taught me about compiler pragmas in GHC and helped me to optimize my L4 implementation. I would also like to thank Rex Page at the University of Oklahoma. He gave me my first start in research and functional programming as an undergraduate and without his guidance I would not be where I am today. Finally, I would like to thank Joe Hurd. There are not words to express how valuable his love and support have been in enabling me to complete this process. Moreover, the quality of this work has benefited immeasurably from our countless discussions and from his proofreading efforts. vi CONTENTS Abstract ::::::::::::::::::::::::::::::::::::: i Dedication :::::::::::::::::::::::::::::::::::: iii Acknowledgments ::::::::::::::::::::::::::::::: iv List of Tables :::::::::::::::::::::::::::::::::: xi List of Figures ::::::::::::::::::::::::::::::::: xiv 1 Introduction ::::::::::::::::::::::::::::::::: 1 1.1 Correctness in Operating Systems . .3 1.2 Purely Functional Languages for Memory-Safety . .5 1.3 Example: Low-level Programming in Haskell . .7 1.4 Assurance Through A Small Trusted Computing Base . .7 1.5 This Work . .9 2 Background: Core Concepts of Intel IA32 Processors ::::::: 11 2.1 Execution Environment . 12 2.1.1 Registers . 12 2.1.2 Modes of Execution . 14 2.1.3 Faults and Interrupts . 15 2.1.4 Input/Output . 17 2.2 Virtual-Memory Management . 18 2.2.1 Address Translation Structures . 18 2.2.2 Protection . 21 2.2.3 Translation Table Formats . 23 2.2.4 Example: Adding a Virtual-To-Physical Mapping . 25 vii 3 Background: Fundamentals of Haskell Programming ::::::: 27 3.1 Datatypes . 29 3.2 Parameterized Datatypes . 33 3.3 Type Classes and Qualified Types . 36 3.4 Monads . 39 3.5 Modules . 46 4 A Memory-Safe Abstraction Layer for Operating System Con- struction in Haskell :::::::::::::::::::::::::::: 49 4.1 Characterizing Memory . 53 4.2 Example: Adding a Memory Mapping in H . 56 4.3 System Configuration . 60 4.3.1 Physical Memory Configuration . 60 4.3.2 User-Code Modules . 66 4.4 Virtual-Memory Management . 67 4.4.1 Address Spaces . 68 4.4.2 User Memory . 71 4.4.3 Kernel Memory . 74 4.5 User Process Execution . 77 4.6 Input/Output . 80 4.6.1 Ports . 80 4.6.2 Debugging . 81 5 Formalizing Operating System Memory-Safety as a Noninterfer- ence Property :::::::::::::::::::::::::::::::: 82 5.1 Background: Rushby's Noninterference Framework . 86 5.1.1 System Model . 86 5.1.2 Characterizing Domain Interactions . 88 5.1.3 Establishing Security . 91 5.2 Notation . 92 5.3 Machine Model . 94 5.3.1 Virtual and Physical Memory . 95 5.3.2 Memory Regions . 100 5.3.3 Machine State . 102 5.4 Protection Domains . 106 5.4.1 Domain Actions . 108 viii 5.5 Policy . 117 5.5.1 Modeling Approach . 118 5.5.2 Interference Between Domains . 122 5.6 Well-Formedness . 126 5.6.1 CR3 and Reference Page-Directory Are Page-Directories . 127 5.6.2 Reference Page-Directory Maps Kernel-Space Addresses . 127 5.6.3 Reference Page-Directory Maps Every Environment Page . 128 5.6.4 All Page-Directories Contain Reference Mappings . 129 5.6.5 Environment Pages Are Only Mapped to Addresses That Are Mapped in the Reference Page-Directory . 130 5.6.6 Mapped Pages Are Available . 131 5.6.7 Page-Table Pointers Are Page-Tables . 131 5.6.8 Regions Are Consistent and Disjoint From Environment . 132 5.6.9 Putting It All Together . 133 5.7 Connecting the Model and Implementation . 134 5.7.1 Specification of WriteE . 136 5.7.2 Specification of DeriveRegionH . 137 5.7.3 Specification of AllocatePageDirectoryH . 138 5.7.4 Specification of FreePageDirectoryH . 139 5.7.5 Specification of AddMappingH . 140 5.7.6 Specification of RemoveMappingH . 144 5.7.7 Specification of AddKernelMappingH . 146 5.7.8 Specification of ExecuteH . 147 5.7.9 Specification of WriteK . 148 5.7.10 Specification of WriteU . 149 5.8 Verifying Memory-Safety . 150 5.8.1 Completing the Rushby Instantiation . 150 5.8.2 Properties of the H Specification and System Model . 152 5.8.3 Proving the Unwinding Conditions . 156 5.8.4 Model Validation . 165 5.9 Summary . 167 6 Implementing the Abstraction Layer ::::::::::::::::: 169 6.1 Safely Encapsulating the Abstraction Layer Operations . 169 6.2 Type Classes for Fine-Grained Effect Tracking . 172 6.3 Booting . 176 ix 6.4 Precise Kernel Control Over Page-Map Memory .