DEFENDING OPERATING SYSTEMS FROM MALICIOUS PERIPHERALS By JING TIAN A DISSERTATION PRESENTED TO THE GRADUATE SCHOOL OF THE UNIVERSITY OF FLORIDA IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF DOCTOR OF PHILOSOPHY UNIVERSITY OF FLORIDA 2019 c 2019 Jing Tian For my mom, who gave up her chance to go to a college for her family but firmly believes \Knowledge is power". For my dad, who knows nothing about computer but bought me one in 1998. ACKNOWLEDGMENTS I am extremely grateful to my family, who always love and support me unconditionally; my advisor Dr. Kevin Butler, who inspires me to a faculty career and keeps inspiring me; and Xie and Fubao, who accompanied me in the past five years. I am also indebted to Dr. Patrick Traynor, who provide guidance and support all along the way. I am also grateful to Dr. Adam Bates, Dr. Bradley Reaves, Dr. Benjamin Mood, and Dr. Nolen Scaife { it is my honor to work with you guys. I would like to thank Dr. Adam Bates, Dr. Patrick McDaniel, Dr. Michael Bailey, Dr. Prabhat Mishra, Dr. Raju Rangaswami, Dr. Tom Shrimpton, and Dr. Vincent Bindschaedler, who gave strong support during my job hunting. Special thanks to Dr. Emily Rine Butler, who taught me how to write an academic paper. I had great pleasure of working with all the talented and motivated students of the FICS research. Many thanks to Grant Hernandez and Joseph Choi for their consistent support whenever my paper is on fire. I would like to thank all my co-authors on the projects described in this work { these projects would not happen without your contribution. I wish to thank our graduate coordinator Adrienne Cook, who always does her best to make sure everything is on track. I am grateful to Gary and Margurite, who keep encouraging and praying for me whenever I am blue. I owe Dr. Dejing Dou a debt of gratitude for his support during my first year in Oregon. Lastly, I would also like to thank my committee members, Dr. Swarup Bhunia, Dr. Prabhat Mishra, and Dr. Patrick Traynor, who have been accommodating in spite of tight schedules and helpful in provided feedback that helped shape this work. 4 TABLE OF CONTENTS page ACKNOWLEDGMENTS.................................4 LIST OF TABLES.....................................8 LIST OF FIGURES....................................9 ABSTRACT........................................ 12 CHAPTER 1 INTRODUCTION.................................. 13 2 BACKGROUND................................... 16 2.1 USB Security.................................. 16 2.1.1 USB Protocol.............................. 16 2.1.2 USB Attacks and Defenses....................... 18 2.2 Bluetooth and NFC Security.......................... 22 2.3 BPF/eBPF................................... 23 3 GOODUSB...................................... 25 3.1 Design...................................... 26 3.1.1 Threat Model and Assumptions.................... 27 3.1.2 Mediating USB Interfaces and Drivers................. 28 3.1.3 Identifying USB Devices........................ 29 3.1.4 Profiling Malicious USB Devices.................... 31 3.2 Implementation................................. 32 3.2.1 User Space Daemon........................... 33 3.2.2 USB Honeypot............................. 37 3.2.3 Device Class Identifier......................... 38 3.2.4 Limited HID Driver........................... 39 3.3 Evaluation.................................... 40 3.3.1 Attack Analysis............................. 40 3.3.1.1 HID-based attacks...................... 41 3.3.1.2 Other USB interfaces and composite devices........ 42 3.3.1.3 Smartphone-based USB attacks............... 42 3.3.2 Performance Analysis.......................... 43 3.4 Discussion.................................... 45 4 USBFILTER..................................... 49 4.1 Design...................................... 51 4.1.1 Threat and Trust Models........................ 51 4.1.2 Design Goals............................... 52 5 4.1.3 Design and Implementation....................... 53 4.1.3.1 Packet filtering rules..................... 54 4.1.3.2 Traceback........................... 55 4.1.3.3 Userspace control....................... 56 4.1.4 Deployment............................... 57 4.1.4.1 Platform integrity....................... 57 4.1.4.2 Runtime integrity....................... 57 4.2 Security Analylsis................................ 58 4.3 Evaluation.................................... 62 4.3.1 Case Studies............................... 62 4.3.2 Benchmarks............................... 66 4.3.2.1 Microbenchmark....................... 67 4.3.2.2 Macrobenchmark....................... 70 4.3.3 Real-world Workloads.......................... 71 4.3.4 Summary................................. 72 4.4 Discussion.................................... 72 4.4.1 Process Table.............................. 72 4.4.2 System Caching............................. 73 4.4.3 Packet Analysis From USB Devices.................. 74 4.4.4 Malicious USB Drivers and USB Covert Channels.......... 74 4.4.5 Usability Issues............................. 75 5 LINUX (E)BPF MODULES............................. 76 5.1 Design...................................... 77 5.1.1 Security Model............................. 77 5.1.2 Goals: Beyond A Reference Monitor.................. 77 5.1.3 LBM Kernel Infrastructure....................... 78 5.1.4 LBM User Space............................ 83 5.2 Implementation................................. 84 5.2.1 LBM Kernel Space........................... 84 5.2.2 LBM User Space............................ 90 5.3 Evaluation.................................... 91 5.3.1 Case Studies............................... 92 5.3.2 Benchmark Setup............................ 98 5.3.3 Micro-Benchmark............................ 99 5.3.4 Macro-Benchmark............................ 99 5.3.5 Scalability................................ 101 5.4 Discussion.................................... 105 5.4.1 LBM vs. USBFILTER vs. USBFirewall................ 105 5.4.2 L2CAP Signaling in Bluetooth..................... 105 5.4.3 BPF Memory Write........................... 106 5.4.4 BPF Helper Kernel Modules...................... 106 5.4.5 LLVM Support............................. 107 5.5 Limitations................................... 107 6 5.5.1 Stateless vs. Stateful Policy...................... 107 5.5.2 DMA-Oriented Protocols........................ 108 5.5.3 Operating Systems Dependency.................... 108 5.5.4 Lbmtool Limitations.......................... 109 6 USB TYPE-C AUTHENTICATION........................ 110 6.1 Authentication Protocol............................ 110 6.1.1 USB Certificate Authorities...................... 112 6.1.2 Authentication Protocol........................ 112 6.1.3 Secure Key Storage and Processing.................. 113 6.1.4 Security Policy.............................. 113 6.2 Formal Verification............................... 114 6.3 Other Issues................................... 117 7 REFLECTIONS ON PERIPHERAL SECURITY................. 120 7.1 Future Work................................... 121 7.2 Conclusion.................................... 123 APPENDIX A A LUM EXAMPLE TO BLOCK SCSI WRITES................. 124 B LBMTOOL FRONTEND GRAMMAR....................... 125 C LBMTOOL COMPILATION EXAMPLE..................... 126 D LMBENCH RESULTS FOR LBM......................... 128 REFERENCES....................................... 130 BIOGRAPHICAL SKETCH................................ 141 7 LIST OF TABLES Table page 2-1 Notable real-world attacks on the USB/Peripheral ecosystem, grouped by the layer at which they operate and the offensive primitive of which they are an instance. 20 3-1 Microbenchmarking GoodUSB operation (in microseconds) averaged over 20 runs. 44 4-1 Prolog reasoning time (µs) averaged by 100 runs.................. 67 4-2 Rule adding operation time (ms) averaged by 100 runs............... 67 4-3 USB enumeration time (ms) averaged by 20 runs.................. 68 4-4 Packet filtering time (µs) averaged by 1500 packets................. 68 4-5 Latency (ms) of the fileserver workload with different mean file sizes....... 69 5-1 LBM compared to USBFILTER and USBFirewall. LBM unifies USBFILTER and USBFirewall, providing a superset their properties via extensible protocol support......................................... 81 5-2 LBM vs. USBFILTER vs. USBFirewall, specifically with respect to filter design of each......................................... 82 5-3 LBM statistics per subsystem, including # of fields exposed to the user space, # of BPF helpers implemented, and # of lines of code changes.......... 87 5-4 The number of lines added to support NFC..................... 98 5-5 Details about the five LBM rules used during the benchmarks........... 98 5-6 LBM overhead in µs based on processing 10K packets on the RX path. For each subsystem, the 1st row is for normal LBM and the 2nd row is for LBM-JIT. In most cases, the overhead of is within 1 µs when JIT is enabled.......... 99 D-1 lmbench results for a Vanilla kernel, LBM, and LBM-JIT............. 129 8 LIST OF FIGURES Figure page 2-1 Peripheral vulnerabilities can be classified by the abstracted communications layer at which they operated. A successful attack involves violating a design assumption or implementation error at a given layer................ 17 2-2 A USB device containing two configurations. Configuration 1 contains two interfaces, and configuration 2 contains one interface. Each interface supports two unidirectional communication channels (In/Out) with the host machine. Each channel