Inlined Reference Monitors: Certification, Concurrency and Tree Based Monitoring
Total Page：16
File Type：pdf, Size：1020Kb
Inlined Reference Monitors: Certification, Concurrency and Tree Based Monitoring ANDREAS LUNDBLAD Doctoral Thesis Stockholm, Sweden 2013 TRITA-CSC-A 2013:01 ISSN-1653-5723 KTH CSC TCS ISRN-KTH/CSC/A–13/01-SE SE-100 44 Stockholm ISBN 978-91-7501-654-2 SWEDEN Akademisk avhandling som med tillstånd av Kungl Tekniska högskolan framlägges till offentlig granskning för avläggande av teknologie doktorsexamen i datalogi 15 mars 2013, 10:00 i F3, Kungl Tekniska Högskolan, Stockholm. © Andreas Lundblad, 2013 Tryck: E-print iii Abstract Reference monitor inlining is a technique for enforcing security policies by injecting security checks into the untrusted software in a style similar to aspect-oriented programming. The intention is that the injected code enforces compliance with the policy (security), without adding behavior (conservativity) or affecting existing policy compliant behavior (transparency). This thesis consists of four papers which covers a range of topics including formalization of monitor inlining correctness properties, certification of inlined monitors, limitations in multithreaded settings and extensions using data-flow monitoring. The first paper addresses the problem of having a potentially complex pro- gram rewriter as part of the trusted computing base. By means of proof- carrying code we show how the inliner can be replaced by a relatively simple proof-checker. This technique also enables the use of monitor inlining for qual- ity assurance at development time, while minimizing the need for post-shipping code rewrites. The second paper focuses on the issues associated with monitor inlining in a concurrent setting. Specifically, it discusses the problem of maintaining transparency when introducing locks for synchronizing monitor state reads and updates. Due to Java’s relaxed memory model, it turns out to be impossible for a monitor to be entirely transparent without sacrificing the security property. To accommodate for this, the paper proposes a set of new correctness properties shown to be realistic and realizable. The third paper also focuses on problems due to concurrency and identi- fies a class of race-free policies that precisely characterizes the set of inlineable policies. This is done by showing that inlining of a policy outside this class is either not secure or not transparent, and by exhibiting a concrete algorithm for inlining of policies inside the class which is secure, conservative, and transpar- ent. The paper also discusses how certification in the style of proof-carrying code could be supported in multithreaded Java programs. The fourth paper formalizes a new type of data centric runtime monitoring which combines monitor inlining with taint tracking. As opposed to ordinary techniques which focus on monitoring linear flows of events, the approach pre- sented here relies on tree shaped traces. The paper describes how the approach can be efficiently implemented and presents a denotational semantics for a sim- ple “while” language illustrating how the theoretical foundations is to be used in a practical setting. Each paper is concluded by a practical evaluation of the theoretical results, based on a prototype implementation and case studies on real-world applica- tions and policies. iv Sammanfattning Referensmonitorinvävning, eller monitorinvävning, är en teknik som an- vänds för att se till att en given säkerhetspolicy efterföljs under exekvering av potentiellt skadlig kod. Tekniken går ut på att bädda in en uppsättning sä- kerhetskontroller (en säkerhetsmonitor) i koden på ett sätt som kan jämföras med aspektorienterad programmering. Syftet med den invävda monitorn är att garantera att policyn efterföljs (säkerhet) utan att påverka ursprungsprogram- mets beteende, såvida det följer policyn (transparans och konservativitet). Denna avhandling innefattar fyra artiklar som tillsammans täcker in en rad ämnen rörande monitorinvävning. Bland annat diskuteras formalisering av korrekthetsegenskaper hos invävda monitorer, certifiering av invävda monito- rer, begränsningar i multitrådade program och utökningar för hantering av dataflödesmonitorering. Den första artikeln behandlar problemen associerade med att ha en po- tentiellt komplex programmodifierare som del i den säkerhetskritiska kompo- nenten av ett datorsystem. Genom så kallad bevisbärande kod visar vi hur en monitorinvävare kan ersättas av en relativt enkel beviskontrollerare. Denna teknik möjliggör även användandet av monitorinvävning som hjälpmedel för programutvecklare och eliminerar behovet av programmodifikationer efter att programmet distribuerats. Den andra artikeln fokuserar på problemen kring invävning av monitorer i multitrådade program. Artikeln diskuterar problemen kring att upprätthålla transparans trots införandet av lås för synkronisering av läsningar av och skriv- ningar till säkerhetstillståndet. På grund av Javas minnesmodell visar det sig dock omöjligt att bädda in en säkerhetsmonitor på ett säkert och transparent sätt. För att ackommodera för detta föreslås en ny uppsättning korrekthetse- genskaper som visas vara realistiska och realiserbara. Den tredje artikeln fokuserar även den på problemen kring flertrådad ex- ekvering och karaktäriserar en egenskap för en policy som är tillräcklig och nödvändig för att både säkerhet och transparens ska uppnås. Detta görs ge- nom att visa att en policy utan egenskapen inte kan upprätthållas på ett säkert och transparent sätt, och genom att beskriva en implementation av en moni- torinvävare som är säker och transparent för en policy som har egenskapen. Artikeln diskuterar också hur certifiering av säkerhetsmonitorer i flertrådade program kan realiseras genom bevisbärande kod. Den fjärde artikeln beskriver en ny typ av datacentrisk säkerhetsmonitore- ring som kombinerar monitorinvävning med dataflödesanalys. Till skillnad mot existerande tekniker som fokuserar på linjära sekvenser av säkerhetskritiska händelser förlitar sig tekniken som presenteras här på trädformade händelse- sekvenser. Artikeln beskriver hur tekniken kan implementeras på ett effektivt sätt med hjälp av abstraktion. Varje artikel avslutas med en praktisk evaluering av de teoretiska resultaten baserat på en prototypimplementation och fallstudier av verkliga program och säkerhetsegenskaper. v Acknowledgements First and foremost, I would like to take the opportunity to thank all those who have supported me and encouraged me during the past six years. I owe my deepest gratitude to my adviser professor Mads Dam for all his help and guidance during my time as a graduate student. I want to thank him for his confidence in me and my work and for always finding the time and patience to stimulate me and keep me on track. Mads, with his deep knowledge of the subject and academical experience have been truly invaluable to me and I am honored and proud to have had Mads as my adviser. I also want to thank Dilian Gurov for introducing me to the subject of formal methods. If it was not for his excellent teaching and inspiring course on semantics for programming languages, I would most likely never have pursued a doctoral degree in computer science to begin with. I thank all my co-authors; Mads Dam, Frank Piessens, Bart Jacobs and Gurvan Le Guernic for all great discussions and fruitful collaboration. My office mates, Fei Niu and Muddassar Azam Sindhu and my former ones Mika Cohen and Irem Aktug–I have really enjoyed your company. Thank you. Musard Balliu, Torbjörn Granlund, Karl Palmskog, Gunnar Kreitz, Oliver Schwartz, Siavash Soleimanifard, Pedro de Carvalho Gomes, Benjamin Greschbach, Emma Enström, thanks for providing such friendly and enjoyable atmosphere here at the department. David Wennström, you are a better friend than I could ever wish for. I have shared more laughs and technically interesting discussions with you than with any one else during my graduate studies. Thank you. A special thanks goes to my father-in-law Professor Björn Bergenståhl for all the discussions and advice on research in general. I would like to thank mom, dad and my sister for all your warmth, encourage- ment and support the past years. Tora–without your patience, your support and your love I would never have come this far. I simply can not thank you enough. I love you. Finally, two very special persons have come into my life during the work on this thesis; Siri and Isabelle, my daughters. I dedicate this thesis to you. Contents Contents vi 1 Introduction 1 1.1 Background . 2 1.2 Reference Monitors . 6 1.2.1 Reference Monitor Inlining . 7 1.2.2 Linear vs Tree based Monitoring . 12 1.3 Research Issues and Motivation . 13 1.4 This Thesis . 16 1.4.1 Included Papers . 17 1.4.2 Author Contributions . 20 2 A Proof-Carrying Code Framework for Inlined Reference Mon- itors in Sequential Java Bytecode 21 2.1 Introduction . 22 2.1.1 Related Work . 24 2.2 A Single Threaded Program Model . 26 2.2.1 Programs . 27 2.2.2 Configurations . 27 2.2.3 Types . 27 2.2.4 API Method Calls . 27 2.2.5 Transition Semantics . 28 2.3 Assertions . 30 2.4 Extended Method Definitions . 31 2.5 Security Specifications . 35 2.5.1 Security Automata . 35 2.6 Example Inlining Algorithm . 36 2.7 The Ghost Monitor . 38 2.8 Contract Adherence Proofs . 40 2.8.1 Example Proof Generation . 41 2.8.2 Proof Recognition . 42 2.9 Implementation and Evaluation . 45 2.9.1 MobileJam . 45 vi CONTENTS vii 2.9.2 Snake . 45 2.9.3 Statistics . 46 2.10 Conclusions . 46 3 Provably Correct Inline Monitoring for Multithreaded Java-like Programs 49 3.1 Introduction . 49 3.2 Security by Contract . 51 3.2.1 Security for mobile applications and services . 52 3.2.2 Application contracts and policies . 53 3.2.3 Example: Mobile 2-player Chess . 54 3.3 A Multithreaded Program Model . 56 3.3.1 Executions and Traces . 56 3.3.2 Field Accesses and Legal Executions . 56 3.3.3 Transition Semantics . 57 3.4 Security Automata . 58 3.5 Inlining . 58 3.5.1 Inlining Correctness Properties . 59 3.5.2 Example Inliner . 63 3.6 Case Studies and Benchmarks . 70 3.6.1 Inlining Overhead . 71 3.7 Conclusions . 71 3.8 Acknowledgements . 72 4 Security Monitor Inlining and Certification for Multithreaded Java 73 4.1 Introduction .