Device driver reuse via virtual machines A dissertation submitted to the School of Computer Science and Engineering of The University of New South Wales in partial fulfillment of the requirements for the degree of Doctor of Philosophy. Joshua Thomas LeVasseur May 2009 PLEASE TYPE THE UNIVERSITY OF NEW SOUTH WALES Thesis/Dissertation Sheet Surname or Family name: LeVasseur First name: Joshua Other name/s: Abbreviation for degree as given in the University calendar: PhD School: Computer Science and Engineering Faculty: Engineering Title: Device driver reuse via virtual machines Abstract 350 words maximum: (PLEASE TYPE) Device drivers constitute a significant portion of an operating system's source code. The effort to develop a new driver set is a sobering hurdle to the pursuit of novel operating system ventures. A practical solution is to reuse drivers, but this can contradict design goals in a new operating system. We offer a new approach to de\ ice-driver reuse, v\ ith a focus on promoting novel operating-SNStem construction, which insulates the new operating system from the invariants of the reused dri\ ers, while also addressing development effort. Our solution runs the drivers along with their original operating systems inside virtual machines, with some minor reuse infrastructure added to the driver's operating s} stem to interface with the rest of the s) stem. This approach turns the drivers into de-privileged applications of the new operating system, which separates their architectures and reduces cross-influences, and improves system dependability. Virtual machines help reuse drivers, but the> also penalize performance. The known solution for improving virtual machine performance, para-virtualization, modifies the operating system to run on a h} per\ isor, w hich has an enormous cost; substantial development effort, and abandonment of many of v irtualization's benefits such as modularit}. These costs contradict our goals for driver reuse: to reduce development effort, and to easih reuse from a variety of operating s\ stems. Thus we introduce a new approach to constructing virtual machines: pre-virtualization. Our solution combines the performance of para-\ irtualization w ith the modularity of traditional \ irtual machines. We still modify the operating system, but according to a set of principles called soft layering that preserves modularity, and via automation which reduces implementation costs. With pre-virtualization we can easily reuse device drivers. We describe our driver-reuse approach applied to a real system: we run virtual machines on the L4Ka::Pistachio microkernel, with reused Linux drivers. We include an evaluation and demonstrate that we achieve throughput comparable to the native Linux drivers, but with moderately higher CPU and memor} utilization. Additionall>, we describe hov\ to appl) pre- virtualization to multiple hypervisor environments. We include an evaluation of pre-virtualization, and demonstrate that it achieves comparable performance to para-virtualization for both the L4Ka::Pistachio and Xen h\ pervisors, with modularity. Declaration relating to disposition of project thesis/dissertation I hereby grant to the University of New South Wales or its agents the right to archive and to make available my thesis or dissertation in whole or in part in the University libraries in all forms of media, now or here after known, subject to the provisions of the Copyright Act 1968. I retain all property rights, such as patent rights. I also retain the right to use in future works (such as articles or books) all or part of this thesis or dissertation. I also authorise University Microfilms to use the 350 word abstract of my thesis in Dissertation Abstracts International (this is applicable to doctoral theses only). 20 Jully 2009 Signature Witness Date The University recognises that there may be exceptional circumstances requiring restrictions on copying or conditions on use. Requests for restriction for a period of up to 2 years must be made in writing. Requests for a longer period of restriction may be considered in exceptional circumstances and require the approval of the Dean of Graduate Research. FOR OFFICE USE ONLY Date of completion of requirements for Award: C) - c5 - THIS SHEET IS TO BE GLUED TO THE INSIDE FRONT COVER OF THE THESIS Abstract Device drivers constitute a significant portion of an operating system's source code. The effort to develop a new driver set is a sobering hurdle to the pursuit of novel operating system ventures. A practical solution is to reuse drivers, but this can contradict design goals in a new operating system. We offer a new approach to device-driver reuse, with a focus on promoting novel operating-system construc- tion, which insulates the new operating system from the invariants of the reused drivers, while also addressing development effort. Our solution runs the drivers along with their original operating systems inside virtual machines, with some mi- nor reuse infrastructure added to the driver's operating system to interface with the rest of the system. This approach turns the drivers into de-privileged applica- tions of the new operating system, which separates their architectures and reduces cross-influences, and improves system dependability. Virtual machines help reuse drivers, but they also penalize performance. The known solution for improving virtual machine performance, para-virtualization, modifies the operating system to run on a hypervisor, which has an enormous cost: substantial development effort, and abandonment of many of virtualization's bene- fits such as modularity. These costs contradict our goals for driver reuse: to reduce development effort, and to easily reuse from a variety of operating systems. Thus we introduce a new approach to constructing virtual machines: pre-virtualization. Our solution combines the performance of para-virtualization with the modularity of traditional virtual machines. We still modify the operating system, but accord- ing to a set of principles called soft layering that preserves modularity, and via automation which reduces implementation costs. With pre-virtualization we can easily reuse device drivers. We describe our driver-reuse approach applied to a real system: we run vir- tual machines on the L4Ka::Pistachio microkernel, with reused Linux drivers. We include an evaluation and demonstrate that we achieve throughput comparable to the native Linux drivers, but with moderately higher CPU and memory utiliza- tion. Additionally, we describe how to apply pre-virtualization to multiple hyper- visor environments. We include an evaluation of pre-virtualization, and demon- strate that it achieves comparable performance to para-virtualization for both the L4Ka:: Pistachio and Xen hypervisors, with modularity. Originality statement 'I hereby declare that this submission is my own work and to the best of my knowl- edge it contains no materials previously published or written by another person, or substantial proportions of material which have been accepted for the award of any other degree or diploma at UNSW or any other educational institution, except where due acknowledgement is made in the thesis. Any contribution made to the research by others, with whom I have worked at UNSW or elsewhere, is explic- itly acknowledged in the thesis. I also declare that the intellectual content of this thesis is the product of my own work, except to the extent that assistance from others in the project's design and conception or in style, presentation and linguistic expression is acknowledged.' Joshua LeVasseur May 2009 vn vin Copyright and authenticity statements 'I hereby grant the University of New South Wales or its agents the right to archive and to make available my thesis or dissertation in whole or part in the University libraries in all forms of media, now or here after known, subject to the provisions of the Copyright Act 1968. I retain all proprietary rights, such as patent rights. I also retain the right to use in future works (such as articles or books) all or part of this thesis or dissertation. I also authorise University Microfilms to use the 350 word abstract of my thesis in Dissertation Abstract International (this is applicable to doctoral theses only). I have either used no substantial portions of copyright ma- terial in my thesis or I have obtained permission to use copyright material; where permission has not been granted I have applied/will apply for a partial restriction of the digital copy of my thesis or dissertation.' Joshua LeVasseur May 2009 'I certify that the Library deposit digital copy is a direct equivalent of the final officially approved version of my thesis. No emendation of content has occurred and if there are any minor variations in formatting, they are the result of the con- version to digital format.' Joshua LeVasseur May 2009 Acknowledgments My experience working on this project was rewarding and enriching, and a turn- ing point in my life, as all periods of self growth must be. I thank Prof. Gemot Heiser, my supervisor, for relentlessly supporting me throughout the project, and for believing in my ability to succeed. He worked with me on the research and helped guide me through paper writing, and spent considerable time reviewing my thesis and providing feedback. He worked to bring me from the American edu- cational system into the German, which have different requirements for entering a PhD program. He also continued to supervise me from remote, after he transferred to UNSW, while I remained in Germany. 1 thank Prof. Gerhard Goos, of the University of Karlsruhe, for his wonderful advice, guidance, and challenges. He graciously accepted me as a student after Prof. Heiser moved to UNSW. His demanding requests guided me into the person I wanted to become. I enjoyed participating in his PhD seminars, which were important training for paper writing, public speaking, and finally the thesis writing. I thank Prof. Frank Bellosa for accepting me after he become the head of the System Architecture Group in Karlsruhe.