TRICKING HARDWARE INTO EFFICIENTLY SECURING SOFTWARE Koen Koning VRIJE UNIVERSITEIT
Total Page：16
File Type：pdf, Size：1020Kb
TRICKING HARDWARE INTO EFFICIENTLY SECURING SOFTWARE Koen Koning VRIJE UNIVERSITEIT TRICKING HARDWARE INTO EFFICIENTLY SECURING SOFTWARE PH.D. THESIS Koen Koning The research reported in this dissertation was conducted at the Faculty of Science, at the Department of Computer Science, of the Vrije Universiteit Amsterdam. This work is part of the research programme Vici with project number 639.023.309 titled “Dowsing”, which is funded by the Dutch Research Council (NWO). Copyright © 2020 by Koen Koning Cover: illustration by Mei-Li Nieuwland, typography by Gabor Roozen VRIJE UNIVERSITEIT TRICKING HARDWARE INTO EFFICIENTLY SECURING SOFTWARE ACADEMISCH PROEFSCHRIFT ter verkrijging van de graad Doctor aan de Vrije Universiteit Amsterdam, op gezag van de rector magnificus prof.dr. V. Subramaniam, in het openbaar te verdedigen ten overstaan van de promotiecommissie van de Faculteit der Bètawetenschappen op dinsdag 26 januari 2021 om 15.45 uur in de aula van de universiteit, De Boelelaan 1105 door Koen Koning geboren te Alkmaar promotor: prof.dr.ir. H.J. Bos copromotor: dr. C. Giuffrida /* * "Sam sat on the ground and put his head in his hands. ’I wish I had never * come here, and I don’t want to see no more magic,’ he said, and fell silent." */ 400.perlbench/src/mg.c, SPEC CPU2006 Acknowledgements This thesis would not have been possible without the support of family, friends and colleagues. I am grateful to you all for giving me both a great environment to work and learn in, and for helping me maintain my sanity during it all. Herbert, thank you for being a great supervisor, and for giving me the opportu- nity to do a PhD in the first place. You taught me a lot about research, and always allowed me to pursue work in areas I was interested in. Moreover, you were always accommodating and supportive when I needed it.1 I could not have wished for a better (co)promoter than Cristiano. Your energy and infinite optimism are awesome, and always gave me motivation to persevere. I greatly appreciate how you somehow always seemed to make time for me, be it to help out with a project or to despair together about the state of academia. I would also like to express my gratitude towards my committee, Pramod Bha- totia, Fabio Massacci, Nele Mentens, Mathias Payer, and Frank Piessens, for taking the time to review this thesis. During my PhD I had the great pleasure of working together with Taddeüs, both directly on Delta Pointers, and helping me out on other projects (including this thesis!). You taught me most of what I know about compilers, and I am grateful I had someone to share the pain of debugging SPEC with. But moreover, I am happy to have you as a friend. Alyssa, you are one of the smartest, most helpful and kindest people I know, and you have been supporting me even before joining VUSec. Be it debugging a kernel, fixing a printer, designing a custom GameBoy CPU/PCB, or acquiring cute animal pictures, you were always there to help. I would also like to thank my former office mate, Kaveh, not only for putting up with this young and naive PhD student back then, but for teaching me so much about academia and research. And of course for the healthy(?) supply of alcoholic beverages. There are many more (former) VUSec colleagues I would like to thank, for our work together and being there to chat with over lunch or beers. Thank you Andrea, 1Just too bad about that whole Emacs thing. vii ACKNOWLEDGEMENTS Andrei B, Andrei T, Angelos, Ben, Brian, Chen, Dennis, Elia, Elias, Emanuele, Enes, Enrico, Erik B, Erik vdK, Hany, Istvan, Jakob, Koustubha, Lucian, Manolis, Manuel, Marco, Marius, Michael, Natalie, Pietro, Radhesh, Sanjay, Sebastian, Stephan, Vic- tor D, and Victor vdV. And Caroline and Mojca for shielding me from the dangers of VU bureaucracy, During my PhD I also had the opportunity to do two amazing internships. Thank you Manuel and others at Microsoft Research Cambridge for teaching me about practical security research, and the wonder of UK pubs. And thank you Sanjay, Philip and the rest of the SAL team at Intel Labs for making me feel part of the team and teaching me so much. Also thank you Dmitrii, Gurunath, Marcela, Palak, and others for the hiking trips and game nights in Oregon. I count myself lucky to have so many wonderful friends who support me and distract me from work. Thank you Jos for getting me into this computer mess all those years ago, and the beautiful trips to Sweden. Thanks Koen for always looking out for me, and the many evenings of gaming (and sharing this beautiful name of ours). Thank you Arjen, Bas, Cédric, Floris (for so much, including unborking of my Dutch in this thesis), Rex (you were an awesome neighbor), and Victor for the countless evenings and beers we shared the past ten years, may there be many more. Vera, your emotional support has been invaluable, and I can always count on you to cheer me up. Thank you Raphael for giving me my first taste of doing research, and for imparting wisdom on me ever since. Thank you Mei-Li for being able to bring my research to life through illustrations, including the beautiful cover design. Last but not least, for my parents and sister: mijn hele leven lang hebben jullie mij geholpen, in mij geloofd, en voor mij gevochten. Ik kan altijd op jullie rekenen, en zonder jullie was ik hier zeker nooit gekomen. Thank you! Koen Koning Amsterdam, The Netherlands, December 2020 Contents Acknowledgements vii Contents ix List of Figures xi List of Tables xiii Publications xv 1 Introduction 1 1.1 Memory errors and attacks . 3 1.1.1 Partial defenses . 5 1.1.2 Protecting binaries . 6 1.2 Contributions and roadmap . 7 2 No Need to Hide: Protecting Safe Regions on Commodity Hardware 9 2.1 Introduction . 10 2.2 Memory isolation in review . 12 2.2.1 Deterministic vs probabilistic isolation . 12 2.2.2 Defenses that rely on isolation . 13 2.2.3 Threat model . 15 2.3 Deterministic memory isolation . 16 2.3.1 Domain-based isolation . 18 2.3.2 Address-based isolation . 21 2.4 MemSentry applications . 22 2.5 Implementation . 23 2.5.1 VMFUNC . 24 2.5.2 MPK . 24 2.5.3 Encryption . 25 2.5.4 MPX and SFI . 25 2.5.5 LLVM & points-to analysis . 26 2.6 Evaluation . 27 ix x CONTENTS 2.6.1 Microbenchmarks . 28 2.6.2 Real-world performance . 29 2.6.3 Discussion . 32 2.7 Related work . 34 2.8 Conclusion . 35 3 Delta Pointers: Buffer Overflow Checks Without the Checks 37 3.1 Introduction . 38 3.2 Background . 39 3.3 Threat model . 41 3.4 Delta Pointers . 41 3.5 Pointer tagging . 45 3.5.1 C pointer operations . 46 3.5.2 Compiler support . 47 3.5.3 Coverage considerations . 49 3.5.4 Performance considerations . 50 3.6 Implementation . 51 3.6.1 Address space reduction . 51 3.6.2 Instrumentation . 52 3.6.3 Coverage . 52 3.6.4 Optimization . 54 3.7 Evaluation . 55 3.7.1 Runtime performance . 55 3.7.2 Security . 58 3.8 Discussion . 60 3.9 Relatedwork ............................... 62 3.10 Conclusion . 64 Appendices . 67 4 Secure and Efficient Multi-variant Execution Using Hardware-assisted Process Virtualization 67 4.1 Introduction . 68 4.2 Background . 70 4.2.1 Monitor . 70 4.2.2 Variant generation . 72 4.3 Threat model . 73 4.4 Overview . 73 4.5 MvArmor: fast and secure MVX . 75 4.5.1 Variant generator . 75 4.5.2 Security manager . 77 4.5.3 Syscall frontend . 77 CONTENTS xi 4.5.4 Variant manager . 78 4.5.5 Syscall backend . 79 4.5.6 Namespace manager . 80 4.5.7 Detector . 81 4.5.8 Implementation . 81 4.6 Limitations . 82 4.7 Evaluation . 82 4.7.1 Server performance . 83 4.7.2 SPEC performance . 86 4.7.3 Microbenchmark performance . 87 4.7.4 Security . 88 4.8 Related work . 90 4.9 Conclusion . 91 5 kMVX: Detecting Kernel Information Leaks with Multi-variant Execution 93 5.1 Introduction . 94 5.2 Background . 95 5.3 Threat model . 97 5.4 kMVX: Kernel multi-variant execution . 98 5.4.1 Syscall synchronization . 99 5.4.2 I/O sync . 100 5.4.3 Variant generation . 100 5.5 Implementation . 102 5.5.1 Variant generation . 103 5.5.2 Syscall sync . 104 5.5.3 I/O sync . 105 5.6 Evaluation . 108 5.6.1 Performance evaluation . 108 5.6.2 Security analysis . 114 5.7 Related work . 116 5.8 Conclusion . 117 6 Conclusion 119 6.1 Future directions . 120 References 123 Summary 139 Samenvatting.