An Agile and Rapidly Reconfigurable Test Bed for Hardware-Based Security Features by Daniel Smith Beard Master of Science Computer Information Systems Florida Institute of Technology 2009 Bachelor of Science Engineering, Electrical Option University of South Florida 1980 A dissertation submitted to the College of Engineering and Computer Science at Florida Institute of Technology in partial fulfillment of the requirements for the degree of Doctorate of Philosophy in Computer Science Melbourne, Florida December, 2019 © Copyright 2019 Daniel Smith Beard All Rights Reserved The author grants permission to make single copies. We the undersigned committee hereby approve the attached dissertation An Agile and Rapidly Reconfigurable Test Bed for Hardware-Based Security Features by Daniel Smith Beard Marco Carvalho, Ph.D. Professor and Dean College of Engineering and Science Committee Chair Stephen K. Cusick, J.D. Associate Professor College of Aeronautics Outside Committee Member William H. Allen, Ph.D. Associate Professor Computer Engineering and Sciences Committee Member Heather Crawford, Ph.D. Assistant Professor Computer Engineering and Sciences Committee Member Philip J. Bernhard, Ph.D. Associate Professor and Department Head Computer Engineering and Sciences ABSTRACT Title: An Agile and Rapidly Reconfigurable Test Bed for Hardware-Based Security Features Author: Daniel Smith Beard Major Advisor: Marco Carvalho, Ph.D. Current general-purpose computing hardware and the software that runs on it have evolved over more than a half century from large mainframe systems in corporate, military, and research use to interconnected commodity devices more common than wrist watches. Computational power, storage capacity, and communication capa- bilities have increased in wonderful and staggering ways; however, when we read about the latest vulnerability or data breach it seems that cybersecurity is stuck somewhere between 1983 when Matthew Broderick first heard a synthesized voice ask \Shall we play a game?", [93] and 1988 when the Morris worm hit the Internet [116]. Multics [82] and Scomp [54] had a shot at establishing secure computing but functionality, cost, and ease of use have largely trumped security so far. For the present, as Jaeger said, \. security features fail to protect the system in a myriad of ways." [77] This study and research effort briefly surveys the roots of secure computing and present vulnerabilities that contribute to insecurity, and presents technological changes that could help stem this tide. We have gleaned a collec- tion of demonstrated security features that could be hardware-based and therefore hardware-enforced, but would require no adaptation of existing legacy applica- tions beyond recompiling already-existing high level source code. In this effort iii we demonstrate a prototype CPU with hardware-based security features that is amenable to FPGA or ASIC implementation and provide a hardware testbed based on DARPA's Cyber Grand Challenge cybersecurity \experimentation ecosystem" [39]. This will answer the question of whether hardware-based security features can produce a significant security improvement in unadapted legacy C/C++ code, and provide a testbed for further evaluation and testing of hardware-based features. iv Table of Contents Abstract iii List of Figures xii List of Tables xv Acknowledgments xvi Dedication xviii 1 Introduction 1 2 Foundations 4 2.1 Foci in Security . 5 2.2 Security Defined . 5 2.3 Problem Statement . 7 2.3.1 Narrowing the Focus { Security in Hardware . 7 2.4 Legacy Secure (Trusted) Systems . 8 2.4.1 Multics . 8 2.4.2 Honeywell Scomp . 10 2.4.3 Drawbacks . 10 2.5 A Modern Trusted Computer System Effort { CHERI . 11 v 2.5.1 Object-Capability Security Overview . 12 2.5.2 Object-Capability Hardware Enhancements . 16 2.5.3 Memory Protection in the Object-Capability Model . 18 2.5.4 CHERI Object-Capability Example . 19 2.5.5 Hardware-Software Integration . 20 2.5.6 Relevance to the Secure Processor . 22 2.6 Common Vulnerability Patterns for Modern Computers . 23 2.6.1 Definition of Terms . 24 2.6.2 From Attack to Intrusion . 27 2.7 Stack Based Buffer Overflows . 31 2.7.1 Stack Basics . 32 2.7.1.1 Stack Operations { Physical View . 32 2.7.1.2 Stack Operations { Computer Memory Represen- tation . 33 2.7.1.3 Stack Width and Growth Direction . 34 2.7.1.4 Stack Operation in Procedure Calls { CALL, RET vs. PUSH, POP . 36 2.7.1.5 Stack Use for Parameters and Variables . 37 2.7.2 Stack Overflow Details . 39 2.7.3 Co-mingled Control and Data on a Common Stack . 41 2.7.4 `Reverse' Stack Growth . 43 2.7.5 Stack Based Buffer Overflow Protection Techniques . 45 2.7.5.1 Stack Execution Prevention . 46 2.7.5.2 Stack Canaries . 49 2.7.5.3 Return Address Protection or Repair . 52 vi 2.7.5.4 Reverse Stack . 55 2.8 Non-Stack Buffer Overflows . 56 2.9 Return- and Jump-Oriented Programming . 59 2.9.1 Gadgets . 60 2.9.2 Return-Oriented Programming Details . 64 2.9.3 Jump-Oriented Programming Details . 65 2.9.4 Control Flow Protection . 67 2.10 Code Injection . 70 2.11 Memory Protection . 71 2.12 Address Space Layout Randomization . 74 2.13 Harvard Architecture . 75 2.14 Instruction Set Architecture . 77 2.15 Instruction Set Randomization . 78 2.16 Hardware-enhanced Authentication . 81 2.16.1 Random Number Sources . 81 2.16.1.1 Physical Uncloneable Functions . 84 2.17 Current State of the Art Summary . 84 3 Secure Host CPU 86 3.1 Introduction . 86 3.2 Secure Host CPU Design Features . 87 3.2.1 High Level Architecture . 87 3.2.2 Memory Architecture . 87 3.2.3 Register Architecture . 88 3.2.4 Stack Architecture . 89 3.2.4.1 Reverse Stack Growth . 90 vii 3.2.4.2 Dual Stack . 91 3.2.5 Instruction Set Architecture . 91 3.2.5.1 LAND Group . 93 3.2.6 Instruction Set Randomization . 94 3.3 Field Programmable Gate Arrays . 97 3.3.1 Example Logic Functions in FPGAs . 98 3.3.2 FPGA Manufacture and Function Implementation . 99 3.3.3 Hardware Description Language . 101 3.3.4 Possible Alternatives to FPGAs . 102 3.4 Exception Handling . 104 3.5 Application Summary . 105 4 Secure Host CPU Implementation 106 4.1 Early FPGA Prototype . 107 4.1.1 C99 Emulator . 108 4.2 Review and Introduction . 109 4.3 CPU High Level Architecture . 109 4.3.1 Secure Host CPU Emulator . 110 4.3.2 Data Types in the C99 Emulator . 112 4.4 Memory Architecture . 113 4.5 Register Architecture . 114 4.5.1 Register Implementation . 116 4.5.1.1 Register Identifiers . 116 4.5.2 Flags Register (eflags) . 119 4.6 Stack Architecture . 119 4.6.1 Reverse Stacks . 120 viii 4.6.2 Dual Stacks . 121 4.7 Instruction Pointer Management . 123 4.8 Instruction Set Architecture . 124 4.8.1 Instruction Word Overview . 124 4.8.2 Instruction Word Architecture . 125 4.8.3 Opclass . 127 4.8.4 Transfer Width . 128 4.8.5 Arguments, Operands, and Operand Types . 128 4.8.6 Operands 1 and 2 Differences . 129 4.8.7 Instruction Transfer and Argument Sizes . 131 4.8.8 Relocation Flag . 132 4.8.9 Instruction Word Binary Implementation . 133 4.9 Other Security Features of the ISA . 133 4.9.1 Jump/Land Flow Control Instructions . 133 4.9.2 Other Flow Control Instructions . 137 4.9.3 Instruction Set Density . 138 4.9.4 Instruction Set Randomization . 139 4.9.4.1 ISR Keys . 140 5 CPU Testbed and Evaluation 141 5.1 Linux Host . 141 5.2 Secure Host Tool Chain . 147 5.2.1 Secure Host CPU Compiler . 148 5.2.1.1 IR to Assembly Register Allocation . 148 5.2.2 Assembler . 149 5.2.2.1 Assembler Dictionaries . 151 ix 5.2.2.2 Assembler Field Codes . 152 5.2.3 Relocating Loader . 153 5.2.4 Console Monitor/Debugger . 153 5.3 OS Support for the Secure Host CPU . 157 5.4 Performance Tuning of the Emulator . 157 5.5 Proof of Concept Demonstration . 158 5.6 Demonstration Results . 159 5.6.1 Invalid Instructions . 159 5.6.2 ROP and JOP Gadget Reduction . 160 5.6.3 Control Flow Protection . 161 5.7 Proof of Concept Demonstration Summary . 162 6 DARPA CGC and DECREE 164 6.1 DARPA Cyber Grand Challenge . 165 6.2 DARPA DECREE . 165 6.2.1 DECREE OS Syscalls . 166 6.2.2 DECREE Syscall Interface . 168 7 Future Work and Concluding Remarks 170 7.1 Architecture Retrospectives . 170 7.1.1 x86 Patterning . 170 7.1.2 Concurrent Registers . 171 7.1.3 Additional Registers . 171 7.1.4 Instruction Set Architecture Changes . 172 7.2 Testbed Enhancements . ..