On Using the System Management Mode for Security Purposes William Augusto Rodrigues de Souza Thesis submitted to the University of London for the degree of Doctor of Philosophy 2016 On Using the System Management Mode for Security Purposes Department of Mathematics Royal Holloway, University of London Seeing much, suffering much and studying much are the three pillars of learning. (Benjamin Disraeli) Declaration of Authorship I, William Augusto Rodrigues de Souza, hereby declare that this thesis and the work presented in it is entirely my own. Where I have consulted the work of others, this is always clearly stated. Signed: (William Augusto Rodrigues de Souza) Date: Summary Computer systems are by design insecure and therefore are many security issues around them. So, security practitioners are always trying to enhance security and performing verification tasks to minimise the risk of potential threats become suc- cessful attacks. These tasks are usually performed by security tools. Thus concepts as: isolation, privilege and view are important in the context of computer systems. Security tools must have good isolation, privilege and view of the system. Then, security tools must operate isolated, have high privilege and must have a global view of the system, but also good ability to view and act timely in its own environment to enhance the chances of success when performing their tasks and for not being hit by the problems they are trying to solve. In this context, this research investigates the System Management Mode (SMM) in the context of Intel processors, current security tools capitalising on SMM and attacks and misuses of SMM to establish a set of requirements and then design a generic architecture for SMM-based security tools. That generic architecture is tested by building a proof of concept to measure the integrity of a file of the Xen hypervisor. This measurement is limited to the minimum necessary to prove the concept of the architecture. The problem context addressed is a cloud computing environment, compris- ing of one or more machines (chipsets). Each chipset hosts in its main memory (DRAM) a virtualised environment comprising of one manager virtual machine, one or more guest virtual machines and a hypervisor. We address our research in- vestigation in two levels: the vertical and the horizontal security level. The vertical security level puts the problem in context, relating it to security issues on: cloud, chipset, memory, virtualisation layer and cache memory. The horizontal security level considers the research problem in its environment, relating it to security is- sues on components of the bootup process and the processor, such as: Intel VMX, TXT and SGX, BIOS and so on. First, we investigate the SMM, its resources and components. Then, we analyse SMM-based security tools and the opportunities to improve them. We also analyse SMM attacks and how to thwart them. From the acquired knowledge, we establish a set of requirements to use SMM for security purposes. Having the requirements, we design a generic architecture for SMM-based security tools. To test the architec- ture, we build a proof of concept comprising of a module to probe chipsets and a SMM-based hypervisor integrity measurement tool. The implementation of that architecture was done in a proof of concept de- signed to have two modules: a manager and an agent. The manager module is used for learning about and researching on the target machine, as for probing, set- ting and clearing registers related to SMM. The manager can be used in the target i machine or in a machine with the same chipset of the target machine. So, it can be deployed in main memory. The agent basically comprises of two parts: a basic code embodying management functions and a payload, where the security functions are implemented. So the use of a payload is what makes the architecture generic since any security task might be implemented and added in the agent by changing the payload. We conclude that any security tool can capitalising on SMM resources provided that it meets the set of requirements established in this research: small, fast, per- sistent, cooperative, isolated, resistant, complete and SMI-independent (meaning that it can be started by any System management interruption, which occur in the chipset); and stick to the proposed generic architecture. ii Acknowledgments My heartfelt thanks to my supervisor Dr. Allan Tomlinson for his guidance, encourage- ment, patience and dedication. I would like to also thank my examiners Professor William Buchanan and Professor Lorenzo Caballaro for taking the time to examine my thesis. I gratefully acknowledge all professors, specially Professors Chris Mitchell and Carlos Cid, colleagues and staff at ISG and at the Department of Mathematics for making my life easier and for all support, guidance and fruitful discussions. I would like to also thank all support and encouragement I received from The Centre for Naval System Analysis of Brazilian Navy. I am also so grateful for all support and understanding received from my family. iii Contents 1 Introduction1 1.1 Motivation.................................. 1 1.2 Limits and Scope .............................. 3 1.3 Significance ................................. 8 1.4 Research Questions............................. 9 1.5 Contribution................................. 11 1.6 List of Publications............................. 12 1.7 Overview of the Research ......................... 13 2 Background 15 2.1 Introduction................................. 15 2.2 Definitions.................................. 17 2.3 Context and Technologies ......................... 19 2.4 Environment and Technologies...................... 35 2.5 Data Integrity with Hash Functions.................... 39 2.6 Related work: System Executive Software Integrity Issues . 40 2.7 Discussion.................................. 43 2.8 Summary................................... 44 3 The System Management Mode (SMM) 45 3.1 Introduction................................. 45 3.2 Components................................. 46 3.3 SMM operation and relations ....................... 54 3.4 Security implementations using SMM.................. 58 3.5 Launching attacks using SMM resources................. 62 3.6 Discussion.................................. 66 3.7 Summary................................... 66 4 Requirements 69 4.1 Introduction................................. 69 4.2 Threat model................................. 69 4.3 Assumptions................................. 72 4.4 Requirements for using SMM for security purposes.......... 72 4.5 Discussion.................................. 74 4.6 Summary................................... 76 5 A Generic Architecture for SMM-Based Security Tools 79 5.1 Introduction................................. 79 v CONTENTS 5.2 Requirements Specification ........................ 79 5.3 General Architecture............................ 82 5.4 Architecture Design............................. 82 5.5 Discussion.................................. 97 5.6 Summary................................... 99 6 Implementation and Evaluation - Manager Module and SBST 105 6.1 Introduction.................................105 6.2 Functions in the Manager Module ....................108 6.3 Manager Module Computational Experiments . 117 6.4 SBST Implementation and Evaluation ..................130 6.5 SBST Limits and Constraints........................132 6.6 Manager Limits and Constraints .....................133 6.7 Discussion..................................133 6.8 Summary...................................133 7 Conclusion 135 7.1 Directions for Future work.........................137 7.2 Investigate the interaction of an SBST with technologies in the chipset137 7.3 Investigate the Impact of SMI Latency..................137 7.4 Optimize the Proof of Concept Execution Time . 138 7.5 Embed the Tool in a BIOS to Test It in a More Realistic Scenario . 138 A Specific SMM Registers 139 A.1 Chipset 1 Specific Registers ........................139 A.2 Chipset 2 System Management RAM Control register . 148 Bibliography 155 vi List of Figures 1.1 Machine and Chipset 1............................. 4 1.2 Machine and Chipset 2............................. 5 1.3 First set of machines used in the experiments................ 5 1.4 Second set of Machines used in the experiments .............. 6 1.5 General architecture .............................. 8 2.1 Security Context................................. 16 2.2 The Cloud Reference Model (figure from [131])............... 21 2.3 Virtualisation Layer............................... 24 2.4 Consolidation.................................. 24 2.5 Containment................................... 25 2.6 Full virtualisation (figure from[140])..................... 28 2.7 Paravirtualisation (figure from [140])..................... 29 2.8 Hardware-assisted virtualisation (figure from [140])............ 30 2.9 Xen Architecture ................................ 31 2.10 Intel Hub Architecture, based on [57]..................... 32 2.11 The Intel Platform Controller Hub ...................... 33 2.12 Ring security scheme, based on [57] ..................... 35 2.13 The PI and UEFI layers, based on [153].................... 37 2.14 TPM Overview (figure from [54]) [57] .................... 39 3.1 SMM components................................ 47 3.2 SMRAM space for 32-bit machines...................... 48 3.3 SMRAM space for 64-bit machines...................... 49 3.4 SMRAM control register...........................