Open Butler-Thesis.Pdf
Total Page:16
File Type:pdf, Size:1020Kb
The Pennsylvania State University The Graduate School LEVERAGING EMERGING STORAGE FUNCTIONALITY FOR NEW SECURITY SERVICES A Dissertation in Computer Science and Engineering by Kevin Raymond Boyce Butler c 2010 Kevin Raymond Boyce Butler Submitted in Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy August 2010 The dissertation of Kevin Raymond Boyce Butler was reviewed and approved∗ by the following: Patrick D. McDaniel Associate Professor of Computer Science and Engineering Dissertation Advisor, Chair of Committee Trent R. Jaeger Associate Professor of Computer Science and Engineering Bhuvan Urgaonkar Assistant Professor of Computer Science and Engineering Stephen G. Simpson Professor of Mathematics Matthew A. Blaze Associate Professor of Computer and Information Science University of Pennsylvania Raj Acharya Professor of Computer Science and Engineering Head of the Department of Computer Science and Engineering ∗Signatures are on file in the Graduate School. Abstract The complexity of modern operating systems makes securing them a challenging problem. However, changes in the computing model, such as the rise of cloud computing and smarter peripherals, have presented opportunities to reconsider system architectures, as we move from traditional “stove-pipe” computing to distributed systems. In particular, we can build trustworthy components that act to provide security in complex systems. The focus of this dissertation is on how new disk architectures may be exploited to aid the protection of systems by acting as policy decision and enforcement points. We prototype disks that enforce data immutability at the block level on critical system data, preventing malicious code from inserting itself into system configuration and boot files. We then examine how storage may be used to ensure the integrity state of hosts prior to allowing access to data, and how such a design improves the security of portable storage devices. Through the continual measuring of system state, we show through formal reasoning that such a device enforces guarantees that data is read and written while the host is in a good state. Finally, we discuss future directions and how secure disk architectures can be used as the basis for large-scale and distributed system security. iii Table of Contents List of Figures ix List of Tables xiii Acknowledgments xiv Chapter 1 Introduction 1 1.1 Using Storage to Reduce the TCB . 4 1.2 Thesis Statement . 6 1.3 Contributions of Dissertation . 7 1.4 Trust Model and Limitations . 8 1.5 Publications . 10 Chapter 2 Foundational Work 11 2.1 Database Machines and Augmented Disks . 11 2.2 Security in Operating Systems . 13 2.2.1 Process Isolation . 15 2.2.2 System Threats from Rootkits . 16 2.2.3 Autonomous Security . 18 2.3 Secure Autonomous Storage . 19 2.3.1 Securing Data with the Filesystem . 19 2.3.2 Augmented Secure Disks . 21 2.4 Trust Establishment . 23 2.4.1 Trusting the Boot Process . 23 2.4.2 Reducing the TCB . 25 2.4.3 Devices for Trust Establishment . 26 2.4.4 Trusted Storage . 27 iv 2.5 Usability Concerns . 27 2.5.1 Threats from Improper Token Usage . 28 2.5.2 Insertion vs. Proximity-Based Tokens . 29 2.5.3 Tokens vs. Secure Cryptographic Channel . 30 2.5.4 Studies of Token-Based Interfaces . 31 Chapter 3 Designing Disks to Autonomously Enforce Security 33 3.1 ASD Design . 34 3.1.1 Requirements . 34 3.1.2 ASD Design . 35 3.2 Applications . 38 3.2.1 Support for Authenticated Encryption . 38 3.2.2 Capability-Based Access Control . 41 3.2.2.1 Capability Representation . 42 3.2.2.2 OS Constructions . 44 3.2.3 Preserving Information Flow . 45 3.3 Emulating an ASD . 47 3.3.1 Modeling ASDs . 48 3.3.2 Flash Memory Emulator . 49 3.3.3 Experimental Setup . 50 3.4 Evaluation . 51 3.4.1 Small Random Access Workload . 53 3.4.2 Large Contiguous Access Workload . 54 3.4.3 Large Disk Analysis . 54 Chapter 4 Autonomous Disk Enforcement for Rootkit Protection 56 4.1 The Problem of Persistent Rootkits . 56 4.2 Rootkit-Resistant Disks . 58 4.2.1 Goals for an RRD . 59 4.2.2 RRD Design . 60 4.2.3 Tokens and Disk Policy . 61 4.2.4 RRD Operation . 64 4.3 Prototype Implementation . 65 4.3.1 RRD . 66 4.3.1.1 Label Management . 67 4.3.2 Host Machine . 67 4.3.3 Installer . 68 4.4 Evaluation . 69 4.4.1 Experimental Setup . 70 4.4.2 Performance . 71 4.4.2.1 Postmark . 71 v 4.4.2.2 System Install . 72 4.4.3 Scalability . 73 4.4.3.1 Measuring Label Creep . 74 4.4.3.2 Label Space Constraints . 75 4.4.4 Security . 75 4.5 Discussion . 76 4.5.1 System Tokens and atime . 76 4.5.2 File system Modification . 77 4.5.3 Maintenance Concerns . 78 4.5.3.1 Token Duplication . 78 4.5.3.2 Backup . 78 4.5.3.3 Revocation . 79 4.5.3.4 Large-Scale Installations and Upgrades . 79 4.5.4 Considerations for Windows Systems . 79 4.6 Summary . 80 Chapter 5 Enforcing Inter-OS Isolation with SwitchBlade 82 5.1 Introduction . 82 5.2 System Goals . 84 5.2.1 Design Criteria . 84 5.2.2 Securing On-Disk Storage . 85 5.2.3 Threat Model . 85 5.3 Architecture . 86 5.3.1 SwitchBlade Policy . 87 5.3.2 SwitchBlade Management . 89 5.3.3 Boot Process . 90 5.3.4 System Management with SwitchBlade . 91 5.4 Prototype Implementation . 92 5.4.1 SwitchBlade implementation . 92 5.4.1.1 Audit log . 93 5.4.1.2 Policy management . 94 5.4.1.3 Boot sector integrity requirements . 94 5.4.2 Client System . 94 5.4.2.1 Multiboot Operation . 95 5.4.2.2 Virtualized Operation . 96 5.4.3 Experience . 97 5.4.3.1 Segment Creation . 97 5.4.3.2 Regular Operation . 98 5.4.4 Abnormal Token Removal . 99 5.5 Evaluation . 99 5.5.1 Security Analysis . 99 vi 5.5.2 Performance Evaluation . 101 5.6 Discussion . 103 5.6.1 Token Management . 103 5.6.2 Disk Administration . 103 5.6.3 SwitchBlade Applications . 104 5.7 Towards More Complex System Security with Autonomous Storage . 105 Chapter 6 Storage-Enabled Secure Boot with Firma 107 6.1 The Need for Commodity Secure Boot . 107 6.2 Design of Firma . 109 6.3 Management and Operation . 111 6.3.1 Installation . 111 6.3.2 Deployment to the Host . 113 6.3.3 Regular Operation . 114 6.3.4 Error Handling . ..