Design and Implementation of a Hypervisor-Based Platform for Dynamic Information Flow Tracking in a Distributed Environment Andrey Ermolinskiy Scott Shenker Electrical Engineering and Computer Sciences University of California at Berkeley Technical Report No. UCB/EECS-2011-50 http://www.eecs.berkeley.edu/Pubs/TechRpts/2011/EECS-2011-50.html May 12, 2011 Copyright © 2011, by the author(s). All rights reserved. Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission. Design and Implementation of a Hypervisor-Based Platform for Dynamic Information Flow Tracking in a Distributed Environment by Andrey Ermolinskiy A dissertation submitted in partial satisfaction of the requirements for the degree of Doctor of Philosophy in Computer Science in the GRADUATE DIVISION of the UNIVERSITY OF CALIFORNIA, BERKELEY Committee in charge: Professor Scott Shenker, Chair Professor Ion Stoica Professor Deirdre Mulligan Spring 2011 Design and Implementation of a Hypervisor-Based Platform for Dynamic Information Flow Tracking in a Distributed Environment Copyright c 2011 by Andrey Ermolinskiy Abstract Design and Implementation of a Hypervisor-Based Platform for Dynamic Information Flow Tracking in a Distributed Environment by Andrey Ermolinskiy Doctor of Philosophy in Computer Science University of California, Berkeley Professor Scott Shenker, Chair One of the central security concerns in managing an organization is protecting the flow of sensitive information, by which we mean either maintaining an audit trail or ensuring that sensitive documents are disseminated only to the authorized parties. A promising approach to securing sensitive data involves designing mechanisms that interpose at the software-hardware boundary and track the flow of information with high precision — at the level of bytes and machine instructions. Fine-grained information flow tracking (IFT) is conceptually simple: memory and registers containing sensitive data are tagged with taint labels and these labels are propagated in accordance with the computation. However, previous efforts have demonstrated that full-system IFT faces two major practi- cal limitations — enormous performance overhead and taint explosion. These challenges render existing IFT implementations impractical for deployment outside of a laboratory setting. This dissertation describes our progress in addressing these challenges. We present the design and implementation of PIFT (for Practical Information Flow Tracking) — a hypervisor-based IFT platform that achieves substantial performance improvements over previous systems and largely eliminates the problem of kernel taint explosion. PIFT takes advantage of spare CPU cores to track the flow of information asynchronously and in par- allel with the primary instruction stream. To the best of our knowledge, PIFT is the most efficient full-system IFT platform avail- able at the time of writing and is the only implementation that supports real-time tracking of information flow in graphical desktop environments. 1 Dedicated to my parents Ludmila and Alexandr, to my brother Pavel, and to my loving wife Olga. i Contents Contents ii List of Figures v List of Tables vii Acknowledgements viii 1 Introduction 1 1.1 DesignGoalsforaComprehensiveIFTPlatform . ..... 3 1.2 TheDesignPhilosophyofPIFT . 4 1.3 EvaluatingPIFT................................ 6 1.4 SummaryofContributions . 7 1.5 ThesisRoadmap................................ 8 2 Background and Related Work 9 2.1 InformationFlowControl. 10 2.1.1 StaticLanguage-LevelAnalysis . 10 2.1.2 DynamicEnforcementofInformationFlowRules . ... 11 2.2 DynamicTaintAnalysis. 14 2.2.1 ApplicationsofTaintTrackingTechniques . .... 14 2.2.2 Improving the Performance of Dynamic Taint Analysis . ...... 17 2.2.3 Hardware Extensionsfor Dynamic Taint Analysis . ..... 18 3 The PIFT Architecture and Information Flow Tracking Model 22 3.1 DataLabelsandthePolicyModel . 22 ii 3.2 TheModelofInformationFlowTracking . ... 23 3.3 SystemArchitecture. .. .. .. .. .. .. .. .. 26 3.3.1 PIFT-ext3:ALabel-AwareFilesystem. .. 29 3.3.2 Comparing PIFT to Existing Hypervisor-Based IFT Systems. .. 31 4 Prototype Implementation 33 4.1 TheHypervisor-LevelComponentofPIFT. .... 34 4.1.1 OverviewofXen ........................... 34 4.1.2 Transforming Xen into a Comprehensive IFT Platform . ..... 36 4.2 InformationFlowTrackingwithQEMU . .. 42 4.2.1 OverviewofQEMU ......................... 43 4.2.2 Extending QEMU with Taint Tracking: Approach Overview.... 47 4.2.3 ThePIFTInstructionSet . 52 4.2.4 TaintTrackingCodeGeneration . 59 4.2.5 TaintProcessorInternals . 60 4.2.6 AsynchronousParallelTaintTracking . ... 64 4.2.7 IntegrationwiththeOverallPIFT Architecture . ...... 66 4.3 ATaint-AwareStorageLayer. 68 4.3.1 ext3:DesignOverview. 69 4.3.2 OurExtensionstoext3 . 71 4.3.3 OurExtensionstotheNFSLayer . 74 4.3.4 Xen-RPC: An Efficient RPC Transport for Inter-VM Communication 75 4.3.5 EvaluationofPIFT-ext3 . 80 4.4 PolicyEnforcement.............................. 88 4.4.1 EnforcementforVirtualBlockDevices . .. 89 4.4.2 Enforcement for Virtual Network Interfaces . ..... 91 4.5 ExtendingPIFTtoaDistributedEnvironment . ...... 91 4.5.1 BandwidthOverheadEvaluation . 92 5 Full-System Performance Evaluation 95 5.1 Computationally-IntensiveWorkloads . ...... 96 5.1.1 CopyingandCompressing . 96 iii 5.1.2 TextSearch.............................. 98 5.2 BenefitsandLimitationsofAsynchrony . .... 99 5.3 Interactivity and User Productivity in Graphical Environments . .102 5.4 EvaluationSummary .............................107 6 Correctness of Taint Label Propagation 109 6.1 ExperimentalResults . .. .. .. .. .. .. .. .. 110 6.2 FundamentalLimitations . 113 6.3 EliminatingTaintExplosion . 116 6.3.1 Case Study: Eliminating Taint Explosion in the Linux Kernel . 118 7 Summary and Conclusions 123 7.1 DirectionsforFutureWork . 126 7.1.1 BridgingtheSemanticGap. .126 7.1.2 FurtherPerformanceImprovements . 127 7.1.3 OtherUsesofDynamicTaintAnalysis . 129 7.2 FinalRemarks.................................130 Bibliography 132 iv List of Figures 3.1 An example of instruction-level IFT: the compute_sum function.. 24 3.2 An example of instruction-level IFT: the table_lookup function. 25 3.3 An example of instruction-level IFT: the is_nonzero function. 26 3.4 Thehigh-levelarchitectureofPIFT. ..... 27 4.1 The format of a leaf page table entry on a 32-bit PAE-enabled x86machine. 38 4.2 Theformat ofthe shared_info_xen_qemu structure. 39 4.3 The contents of the hypervisor-level stack upon entry to restore_all_guest............................. 39 4.4 The implementation of the pift_emulate_guest function. 40 4.5 AnexampleofdynamiccodetranslationinQEMU. ..... 45 4.6 An example of code translation with static instruction handlerroutines.. 46 4.7 OurextensionstoQEMU’sdynamiccodetranslator. ....... 48 4.8 An illustrative example of dynamic code translation in PIFT. ........ 49 4.9 The general format of a PIFT taint tracking instruction. ........... 52 4.10 The PageTaintSummary lookupprocedure. 61 4.11 The implementation of the pre-generated handler for Set(Dst=eax, Src=MEM_BYTE) in the source code form (left) and in the final form com- piledforthex86platform(right).. .. 63 4.12 The implementation of the pre-generated handler for Merge(Dst=eax, Src=edx) in the source code form (left) and in the final form compiled for thex86platform(right). 63 4.13 Theon-disklayoutofanext2/ext3filesystem. ........ 69 4.14 The on-disk layout of taint label metadata in PIFT-ext3............ 73 4.15 New VFS callbacks in the inode_operations structure. 75 4.16 The implementation of the get_filerange_taint callback in PIFT-ext3. 76 v 4.17 TheformatofanXDRbufferstructure. .... 77 4.18 Performance of the BTD cache module at varying levels of concurrency. 81 4.19 Performance of the on-disk filesystem under data- and metadata-intensive workloads.................................... 83 4.20 Overall performance of the PIFT storage subsystem under data- and metadata-intensivefilesystemworkloads. .... 86 4.21 Function prototypes of policy enforcement handlers for paravirtualized diskandnetworkdevices.. 90 4.22 The effective network bandwidth, as measured by iperf, for varying amounts of tainted data (P ) and at varying levels of label fragmentation (F ). ...................................... 94 5.1 Performance of PIFT-A(512) and Neon on file copying and compression tasks with a varying amount of tainted data (F )................ 97 5.2 Performance on worst-case CPU-bound computational tasks in all config- urationsofinterest. .. .. .. .. .. .. .. ..101 5.3 Application launch time for Abiword and OOCalc in all configurations of interest. ....................................104 5.4 User-perceptible overhead in the text entry experiment. ...........105 5.5 Time taken to complete the spreadsheet editing task in all configurations of interest. ....................................106 6.1 A time series showing the level of computational activity within the pro- tected VM in Experiment 1. The highlighted overlay illustrates the number of basic blocks that touch at least one tainted operand. ........111 6.2 A schematic depiction of the data flow graph that emerged from our em- pirical study of kernel taint explosion. The shaded nodes depict