RC23964 (W0605-129) May 24, 2006 Computer Science IBM Research Report Shame on Trust in Distributed Systems Trent Jaeger, Patrick McDaniel, Luke St. Clair Pennsylvania State University Reiner Sailer, Ramón Cáceres IBM Research Division Thomas J. Watson Research Center P.O. Box 704 Yorktown Heights, NY 10598 Research Division Almaden - Austin - Beijing - Haifa - India - T. J. Watson - Tokyo - Zurich LIMITED DISTRIBUTION NOTICE: This report has been submitted for publication outside of IBM and will probably be copyrighted if accepted for publication. I thas been issued as a Research Report for early dissemination of its contents. In view of the transfer of copyright to the outside publisher, its distribution outside of IBM prior to publication should be limited to peer communications and specific requests. After outside publication, requests should be filled only by reprints or legally obtained copies of the article (e.g ,. payment of royalties). Copies may be requested from IBM T. J. Watson Research Center , P. O. Box 218, Yorktown Heights, NY 10598 USA (email: [email protected]). Some reports are available on the internet at http://domino.watson.ibm.com/library/CyberDig.nsf/home . Shame on Trust in Distributed Systems Trent Jaeger, Patrick McDaniel, Luke St. Clair Ramon´ Caceres,´ Reiner Sailer Pennsylvania State University IBM T. J. Watson Research Center 1 Introduction of the distributed system that is also resilient to dynamic Approaches for building secure, distributed systems have changes in the application. fundamental limitations that prevent the construction of dy- The Shamon approach addresses the fundamental chal- namic, Internet-scale systems. In this paper, we propose a lenges described above. First, trust is built from the bottom- concept of a shared reference monitor or Shamon that we up via secure hardware credentials that enable attestations believe will provide a basis for overcoming these limita- of virtual machine-based enforcement for each machine. tions. First, distributed systems lack a principled basis for Second, the MAC policy enforced by the Shamon is used trust in the trusted computing bases of member machines. to prove enforcement of system security goals. We de- In most distributed systems, a trusted computing base is as- fine a logical representation for verifying these criteria that sumed. However, the fear of compromise due to miscon- enables scalable management of large Shamon even under figuration or vulnerable software limits the cases where this changes in application configuration. Each of the five tasks assumption can be applied in practice. Where such trust that convert a reference monitor into a Shamon presents sub- is not assumed, current solutions are not scalable to large stantial research challenges, but we aim to demonstrate that systems [7, 20]. Second, current systems do not ensure each has tractable solution potential and that the resultant the enforcement of the flexible, distributed system secu- Shamon system will provide a foundation for large-scale rity goals. Mandatory access control (MAC) policies aim distributed authorization. To motivate its design, we intro- to describe enforceable security goals, but flexible MAC duce our prototype application of the Shamon in the follow- solutions, such as SELinux, do not even provide a scal- ing section. able solution for a single machine (due to the complexity of UNIX systems), much less a distributed system. A signifi- 2 Application cant change in approach is necessary to develop a principled The Playpen is a Xen-based, virtual machine (VM) environ- trusted computing base that enforces system security goals ment for the students taking security courses at Pennsylva- and scales to large distributed systems. nia State University. Each student is given their own virtual Our proposal is to develop scalable mechanisms for com- machines in the Playpen. Over the course of the semester, posing a verifiable reference monitoring infrastructure that students are required to configure and build security appara- spans Internet-wide distributed systems. We refer to a tus to defend their machines against attacks from the faculty set of reference monitors that provides coherent security and TAs. The isolation, persistence, and mobility of the VM guarantees across multiple physical machines as a Sha- environment provides ideal conditions for pedagogy: users mon 1. While this may sound like a mere extension of can experiment with security apparatus under the controlled the well-known reference monitor concept, we propose sev- environment. eral key differences: (1) the credentials of secure hardware The current Playpen is the prototype for a larger project (e.g., Trusted Computing Group’s Trusted Platform Mod- supporting wide-area mobile and secure computing envi- ule), rather than users, are used to authenticate individual ronments. The long term goal is to extend the Playpen to en- reference monitoring systems in the Shamon ; (2) trust in compass all aspects of university life. In this, a user would the Shamon is based on attestation of reference monitoring be given one or more virtual machines that would migrate properties: tamperproofing, mediation, and simplicity of to the location where they are working. The central chal- design; (3) virtual machine monitoring is used to establish lenge of this work is to support the users’ ability to move coarse-grained domains, which results in significant sim- freely within the university environment. The system must plification of MAC policies; (4) policy analyses verify that securely support arbitrary migration to previously unknown these MAC policies satisfy the Shamon application’s secu- hardware at a previously unknown location and share data rity goals when enforced by the Shamon; and (5) based on with previously unknown collaborators. Note that while the this restricted definition of trust, a focused logic is defined environment aims at a single university system, we are not that enables scalable evaluation of this trust by components centrally-administered: there is different administration at each campus, and some departments also administer their 1The name is short for Shared Monitor and related to the word shaman meaning “... a medium ... who practices ... control over natural events” own machines. words removed for effect, not necessarily accuracy). Consider a typical day of Alice the graduate student in 1 this new university. She wakes up at noon and goes to class. vide a consistent view of security; and it must scale – there Alice joins a live coalition of class participants by logging are over 41,000 students at Penn State spread out over 24 into a host in her classroom. She exits the coalition at the campuses. end of class, and at lunch she surfs the Internet and ex- changes personal communication within her protected en- 3 Coalitions and Shamon vironment at the local student union. After lunch, she heads to the laboratory and performs research and shares data with Physical Machine Physical Machine VM the other graduate students. At the end of the day, she meets VM Coalition with her advisor and shares summary data and exchanges VM VM VM VM results. She heads home and plays a massively multiplayer VM VM VM game with thousands of other gamers until dawn over the Internet. Untrusted Such is the nature of university life. The ”roles” of Network Alice’s computing environment and the environments in VM VM which she interacts evolve constantly; from class partic- VM ipant, personal communication, researcher, advisee, and VM VM gamer. Moreover, the set of hosts to which she has an as- VM VM sociation is also changing. What is interesting here is not Physical Machine Physical Machine that this somehow changes the way Alice lives, but that her computing environment follows her throughout her life. Figure 1: Example of a distributed coalition. Virtual ma- The security challenges of this environment are non- chine (VM) instances sharing common Mandatory Access Con- trivial. The physical machines within the open university trol (MAC) labels on multiple physical hypervisor systems are all environment are largely unknown and often compromised.2 members of the same coalition. The applications are as diverse as the environments in which Figure 1 illustrates the conceptual idea for future dis- Alice lives, from classroom to research to gaming. Further- tributed applications. A distributed application is a coali- more, the collaborations in which Alice participates change tion of VMs that executes across multiple physical plat- hourly, and often form and disband organically. It is clear forms. Each member of the coalition may reside on a dif- that: (1) supporting this environment requires significant se- ferent physical machine, which may itself execute multiple curity, and (2) current commodity environments (e.g., dis- coalitions. The physical machines themselves each have a tributed file systems and VPNs) do not support it. Note that reference monitor capable of enforcing MAC policies over large corporate environments are similar–users will move all of their VMs. freely through a largely insecure complex and use data and We define the Shamon as follows. A Shamon is a set of applications as needed. reference monitors serving a coalition by enforcing its se- Research that enables articulation of finer-grained poli- curity goals. A reference monitor may belong to multiple cies across distributed systems, for distributed file access Shamon, so its enforcement must ensure the satisfaction of (e.g., [15, 3]) and trust management (e.g., [4, 14]), often the security goals for each. The challenge is to establish assume trust in the trusted computing base as well. An ex- trust in the Shamon reference monitors’ enforcement of a ception is the Taos operating system approach [2] which coalition security goal. This trust must be upheld in a scal- has a form of secure booting for establishing trust in the in- able fashion as VMs join the coalition or migrate between frastructure. However, building trust in a single machine is machines. In so doing, the Shamon provides authorization insufficient.