Fully Countering Trusting Trust through Diverse Double-Compiling A dissertation submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy at George Mason University By David A. Wheeler Master of Science George Mason University, 1994 Bachelor of Science George Mason University, 1988 Co-Directors: Dr. Daniel A. Menascé and Dr. Ravi Sandhu, Professors The Volgenau School of Information Technology & Engineering Fall Semester 2009 George Mason University Fairfax, VA Copyright © 2009 David A. Wheeler You may use and redistribute this work under the Creative Commons Attribution-Share Alike (CC-BY-SA) 3.0 United States License. You are free to Share (to copy, distribute, display, and perform the work) and to Remix (to make derivative works), under the following conditions: (1) Attribution. You must attribute the work in the manner specified by the author or licensor (but not in any way that suggests that they endorse you or your use of the work). (2) Share Alike. If you alter, transform, or build upon this work, you may distribute the resulting work only under the same, similar or a compatible license. Alternatively, permission is also granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation. As a third alternative, permission is also granted to copy, distribute and/or modify this document under the terms of the GNU General Public License (GPL) version 2 or any later version published by the Free Software Foundation. All trademarks, service marks, logos, and company names mentioned in this work are the property of their respective owners. ii Dedication This is dedicated to my wife and children, who sacrificed many days so I could perform this work, to my extended family, and to the memory of my former mentors Dennis W. Fife and Donald Macleay, who always believed in me. Soli Deo gloria—Glory to God alone. iii Acknowledgments I would like to thank my PhD committee members and former members Dr. Daniel A. Menascé, Dr. Ravi Sandhu, Dr. Paul Ammann, Dr. Jeff Offutt, Dr. Yutao Zhong, and Dr. David Rine, for their helpful comments. The Institute for Defense Analyses (IDA) provided a great deal of help. Dr. Roger Mason and the Honorable Priscilla Guthrie, former directors of IDA’s Information Technology and Systems Division (ITSD), partly supported this work through IDA’s Central Research Program. Dr. Margaret E. Myers, current IDA ITSD director, approved its final release. I am very grateful to my IDA co-workers (alphabetically by last name) Dr. Brian Cohen, Aaron Hatcher, Dr. Dale Lichtblau, Dr. Reg Meeson, Dr. Clyde Moseberry, Dr. Clyde Roby, Dr. Ed Schneider, Dr. Marty Stytz, and Dr. Andy Trice, who had many helpful comments on this dissertation and/or the previous ACSAC paper. Reg Meeson in particular spent many hours carefully reviewing the proofs and related materials, and Clyde Roby carefully reviewed the whole dissertation; I thank them both. Aaron Hatcher was immensely helpful in working to scale the Diverse Double- Compiling (DDC) technique up to a real-world application using GCC. In particular, Aaron helped implement many applications of DDC that we thought should have worked with GCC, but didn’t, and then helped to determine why they didn’t work. These “Edison successes” (which successfully found out what did not work) were important in helping to lead to a working application of DDC to GCC. Many others also helped create this work. The work of Dr. Paul A. Karger, Dr. Roger R. Schell, and Ken Thompson made the world aware of a problem that needed solving; without knowing there was a problem, there would have been no work to solve it. Henry Spencer posted the first version of this idea that eventually led to this dissertation (though this dissertation expands on it far beyond the few sentences that he wrote). Henry Spencer, Eric S. Raymond, and the anonymous ACSAC reviewers provided helpful comments on the ACSAC paper. I received many helpful comments and other information after publication of the ACSAC paper, including comments from (alphabetically by last name) Bill Alexander, Dr. Steven M. Bellovin, Terry Bollinger, Ulf Dittmer, Jakub Jelinek, Dr. Paul A. Karger, Ben Laurie, Mike Lisanke, Thomas Lord, Bruce Schneier, Brian Snow, Ken Thompson, Dr. Larry Wagoner, and James Walden. Tawnia Wheeler proofread both the ACSAC paper and this document; thank you! My thanks to the many developers of the OpenDocument specification and the OpenOffice.org implementation, who made developing this document a Joy. iv Table of Contents Page List of Tables...............................................................................................................................viii List of Figures................................................................................................................................ix List of Abbreviations and Symbols.................................................................................................x Abstract.......................................................................................................................................xiv 1 Introduction.................................................................................................................................1 2 Background and related work......................................................................................................4 2.1 Initial revelation: Karger, Schell, and Thompson.................................................................4 2.2 Other work on corrupted compilers.....................................................................................6 2.3 Compiler bootstrap test........................................................................................................9 2.4 Analyzing software............................................................................................................10 2.4.1 Static analysis............................................................................................................11 2.4.2 Dynamic analysis.......................................................................................................14 2.5 Diversity for security.........................................................................................................16 2.6 Subversion of software is a real problem...........................................................................17 2.7 Previous DDC paper..........................................................................................................21 3 Description of threat..................................................................................................................23 3.1 Attacker motivation............................................................................................................23 3.2 Triggers, payloads, and non-discovery...............................................................................27 4 Informal description of Diverse Double-Compiling (DDC).......................................................30 4.1 Terminology and notation..................................................................................................30 4.2 Informal description of DDC.............................................................................................32 4.3 Informal assumptions.........................................................................................................35 4.4 DDC does not require that different compilers produce identical executables...................37 4.5 Special case: Self-parenting compiler................................................................................38 4.6 Why not always use the trusted compiler?.........................................................................40 4.7 Why is DDC different from N-version programming?.......................................................41 4.8 DDC works with randomly-corrupting compilers..............................................................43 5 Formal proof..............................................................................................................................44 5.1 Graphical model for formal proof .....................................................................................45 5.1.1 Types..........................................................................................................................46 5.1.2 DDC components.......................................................................................................47 5.1.3 Claimed origin...........................................................................................................48 5.2 Formal notation: First-Order Logic (FOL).........................................................................49 5.3 Proof step rationales (derivation rules or rules of inference)..............................................51 5.4 Tools and rationale for confidence in the proofs................................................................54 5.4.1 Early DDC proof efforts............................................................................................54 5.4.2 Prover9, mace4, and ivy.............................................................................................54 v 5.4.3 Tool limitations..........................................................................................................56 5.4.4 Proofs’ conclusions follow from their assumptions....................................................57