Bootstrapping and Maintaining Trust in the Cloud ∗ Nabil Schear Patrick T. Cable II Thomas M. Moyer MIT Lincoln Laboratory Threat Stack, Inc. MIT Lincoln Laboratory [email protected] [email protected] [email protected] Bryan Richard Robert Rudd MIT Lincoln Laboratory MIT Lincoln Laboratory [email protected] [email protected] ABSTRACT 1. INTRODUCTION Today's infrastructure as a service (IaaS) cloud environ- The proliferation and popularity of infrastructure-as-a- ments rely upon full trust in the provider to secure appli- service (IaaS) cloud computing services such as Amazon cations and data. Cloud providers do not offer the ability Web Services and Google Compute Engine means more cloud to create hardware-rooted cryptographic identities for IaaS tenants are hosting sensitive, private, and business critical cloud resources or sufficient information to verify the in- data and applications in the cloud. Unfortunately, IaaS tegrity of systems. Trusted computing protocols and hard- cloud service providers do not currently furnish the build- ware like the TPM have long promised a solution to this ing blocks necessary to establish a trusted environment for problem. However, these technologies have not seen broad hosting these sensitive resources. Tenants have limited abil- adoption because of their complexity of implementation, low ity to verify the underlying platform when they deploy to performance, and lack of compatibility with virtualized en- the cloud and to ensure that the platform remains in a good vironments. In this paper we introduce keylime, a scal- state for the duration of their computation. Additionally, able trusted cloud key management system. keylime pro- current practices restrict tenants' ability to establish unique, vides an end-to-end solution for both bootstrapping hard- unforgeable identities for individual nodes that are tied to a ware rooted cryptographic identities for IaaS nodes and for hardware root of trust. Often, identity is based solely on a system integrity monitoring of those nodes via periodic at- software-based cryptographic solution or unverifiable trust testation. We support these functions in both bare-metal in the provider. For example, tenants often pass unprotected and virtualized IaaS environments using a virtual TPM. secrets to their IaaS nodes via the cloud provider. keylime provides a clean interface that allows higher level Commodity trusted hardware, like the Trusted Platform security services like disk encryption or configuration man- Module (TPM) [40], has long been proposed as the solution agement to leverage trusted computing without being trusted for bootstrapping trust, enabling the detection of changes to computing aware. We show that our bootstrapping proto- system state that might indicate compromise, and establish- col can derive a key in less than two seconds, we can detect ing cryptographic identities. Unfortunately, TPMs have not system integrity violations in as little as 110ms, and that been widely deployed in IaaS cloud environments due to a keylime can scale to thousands of IaaS cloud nodes. variety of challenges. First, the TPM and related standards for its use are complex and difficult to implement. Second, ∗ This material is based upon work supported by the As- since the TPM is a cryptographic co-processor and not an sistant Secretary of Defense for Research and Engineering accelerator, it can introduce substantial performance bottle- under Air Force Contract No. FA8721-05-C-0002 and/or necks (e.g., 500+ms to generate a single digital signature). FA8702-15-D-0001. Any opinions, findings, conclusions or recommendations expressed in this material are those of the Lastly, the TPM is a physical device by design and most author(s) and do not necessarily reflect the views of the As- IaaS services rely upon virtualization, which purposefully sistant Secretary of Defense for Research and Engineering. divorces cloud nodes from the hardware on which they run. c 2016 Massachusetts Institute of Technology. Delivered At best, the limitation to physical platforms means that to the U.S. Government with Unlimited Rights, as defined only the cloud provider would have access to the trusted in DFARS Part 252.227-7013 or 7014 (Feb 2014). Notwith- hardware, not the tenants [17, 20, 31]. The Xen hypervisor standing any copyright notice, U.S. Government rights in this work are defined by DFARS 252.227-7013 or DFARS includes a virtualized TPM implementation that links its se- 252.227-7014 as detailed above. Use of this work other than curity to a physical TPM [2, 10], but protocols to make use as specifically authorized by the U.S. Government may vio- of the vTPM in an IaaS environment do not exist. late any copyrights that exist in this work. To address these challenges we identify the following de- sirable features of an IaaS trusted computing system: ACM acknowledges that this contribution was authored or co-authored by an em- ployee, or contractor of the national government. As such, the Government retains a nonexclusive, royalty-free right to publish or reproduce this article, or to allow oth- • Secure Bootstrapping { the system should enable ers to do so, for Government purposes only. Permission to make digital or hard copies the tenant to securely install an initial root secret into for personal or classroom use is granted. Copies must bear this notice and the full ci- each cloud node. This is typically the node's long term tation on the first page. Copyrights for components of this work owned by others than ACM must be honored. To copy otherwise, distribute, republish, or post, requires prior cryptographic identity and the tenant chains other se- specific permission and/or a fee. Request permissions from [email protected]. crets to it to enable secure services. ACSAC ’16, December 05-09, 2016, Los Angeles, CA, USA c 2016 ACM. ISBN 978-1-4503-4771-6/16/12. $15.00 • System Integrity Monitoring { the system should DOI: http://dx.doi.org/10.1145/2991079.2991104 allow the tenant to monitor cloud nodes as they oper- ate and react to integrity deviations within one second. Software-based Cryptographic Services ID key Software • Secure Layering (Virtualization Support) { the revoked? ID Keys system should support tenant controlled bootstrapping Keylime and integrity monitoring in a VM using a TPM in the Trusted Computing Services provider's infrastructure. This must be done in collab- Valid Signed TPM? EKs oration with the provider in least privilege manner. TPM / Platform Manufacturer Enrollment • Compatibility { the system should allow the ten- ant to leverage hardware-rooted cryptographic keys in Figure 1: Interface between trusted hardware and software to secure services they already use (e.g., disk existing software-based security services via the encryption or configuration management). keylime trusted computing service layer. • Scalability { the system should scale to support boot- strapping and monitoring of thousands of IaaS resources provisioning a key using keylime takes less than two sec- as they are elastically instantiated and terminated. onds. Finally, we find that our system can detect integrity Prior cloud trusted computing solutions address a subset measurement violations in as little as 110ms. of these features, but none achieve all. Excalibur [31] sup- ports bootstrapping at scale, but does not allow for system 2. BACKGROUND integrity monitoring or offer full support for tenant trusted Trusted Computing The TPM provides the means for computing inside a VM (i.e., layering). Manferdelli et al. creating trusted systems that are amenable to system in- created a system that supports secure layering and boot- tegrity monitoring. The TPM, as specified by the Trusted strapping, but does not support system integrity monitor- Computing Group (TCG)4, is a cryptographic co-processor ing, is incompatible with existing cryptographic services, that provides key generation, protected storage, and crypto- and has not demonstrated cloud scale operation [25]. Fi- graphic operations. The protected storage includes a set of nally, the Cloud Verifier [34] enables system integrity mea- Platform Configuration Registers (PCRs) where the TPM surement and cloud scalability but does not fully address stores hashes. The TPM uses these registers to store mea- secure layering or enable secure bootstrapping. surements of integrity-relevant components in the system. In this paper, we introduce keylime; an end-to-end IaaS To store a new measurement in a PCR, the extend opera- trusted cloud key management service that supports all the tion concatenates the existing PCR value with the new mea- above desired features. The key insight of our work is to surement, securely hashes5 that value, and stores the result- utilize trusted computing to bootstrap identity in the cloud ing hash in the register. This hash chain allows a verifier to and provide integrity measurement to support revocation, confirm that a set of measurements reported by the system but then allow high-level services that leverage these iden- has not been altered. This report of measurements is called tities to operate independently. Thus, we provide a clean an attestation, and relies on the quote operation, which ac- and easy to use interface that can integrate with existing cepts a random nonce and a set of PCRs. These PCRs can security technologies (see Figure 1). include measurements of the BIOS, firmware, boot loader, We introduce a novel bootstrap key derivation protocol hypervisor, OS, and applications, depending on the configu- that combines both tenant intent and integrity measurement ration of the system. The TPM reads the PCR values, and to install secrets into cloud nodes. We then leverage the then signs the nonce and PCRs with a key that is only ac- Cloud Verifier [34] pattern of Schiffman et al. to enable pe- cessible by the TPM. The key the TPM uses to sign quotes riodic attestation that automatically links to identity revo- is called an attestation identity key (AIK). We denote a cation.