Air Force Institute of Technology AFIT Scholar Theses and Dissertations Student Graduate Works 3-26-2020 Implications and Limitations of Securing an InfiniBand Network Lucas E. Mireles Follow this and additional works at: https://scholar.afit.edu/etd Part of the Computer Engineering Commons Recommended Citation Mireles, Lucas E., "Implications and Limitations of Securing an InfiniBand Network" (2020). Theses and Dissertations. 3183. https://scholar.afit.edu/etd/3183 This Thesis is brought to you for free and open access by the Student Graduate Works at AFIT Scholar. It has been accepted for inclusion in Theses and Dissertations by an authorized administrator of AFIT Scholar. For more information, please contact [email protected]. IMPLICATIONS AND LIMITATIONS OF SECURING AN INFINIBAND NETWORK THESIS Lucas E. Mireles, Second Lieutenant, USAF AFIT-ENG-MS-20-M-44 DEPARTMENT OF THE AIR FORCE AIR UNIVERSITY AIR FORCE INSTITUTE OF TECHNOLOGY Wright-Patterson Air Force Base, Ohio DISTRIBUTION STATEMENT A APPROVED FOR PUBLIC RELEASE; DISTRIBUTION UNLIMITED. The views expressed in this document are those of the author and do not reflect the official policy or position of the United States Air Force, the United States Department of Defense or the United States Government. This material is declared a work of the U.S. Government and is not subject to copyright protection in the United States. AFIT-ENG-MS-20-M-44 IMPLICATIONS AND LIMITATIONS OF SECURING AN INFINIBAND NETWORK THESIS Presented to the Faculty Department of Electrical and Computer Engineering Graduate School of Engineering and Management Air Force Institute of Technology Air University Air Education and Training Command in Partial Fulfillment of the Requirements for the Degree of Master of Science in Computer Engineering Lucas E. Mireles, B.S. Second Lieutenant, USAF March 2020 DISTRIBUTION STATEMENT A APPROVED FOR PUBLIC RELEASE; DISTRIBUTION UNLIMITED. AFIT-ENG-MS-20-M-44 IMPLICATIONS AND LIMITATIONS OF SECURING AN INFINIBAND NETWORK THESIS Lucas E. Mireles, B.S. Second Lieutenant, USAF Committee Membership: Scott R. Graham, Ph.D. Chair Patrick J. Sweeney, Ph.D., Lt. Col Member Stephen Dunlap, M.S. Member Matthew J. Dallmeyer, M.S. Member AFIT-ENG-MS-20-M-44 Abstract The InfiniBand Architecture is one of the leading network interconnects used in high performance computing, delivering very high bandwidth and low latency. As the popularity of InfiniBand increases, the possibility for new InfiniBand applications arise outside the domain of high performance computing, thereby creating the op- portunity for new security risks. In this work, new security questions are considered and addressed. The study demonstrates that many common traffic analyzing tools cannot monitor or capture InfiniBand traffic transmitted between two hosts. Due to the kernel bypass nature of InfiniBand, many host-based network security systems cannot be executed on InfiniBand applications. Those that can impose a significant performance loss for the network. The research concludes that not all network secu- rity practices used for Ethernet translate to InfiniBand as previously suggested and that an answer to meeting specific security requirements for an InfiniBand network might reside in hardware offload. iv AFIT-ENG-MS-20-M-44 This work is dedicated to my wife and family for their unfailing love and support. v Acknowledgements Foremost, I would like to express my sincere gratitude to my advisor Dr. Scott Graham for guiding me throughout my graduate education with his expertise and immense knowledge. His enthusiasm and personal generosity made this research process invaluable and I could not have completed it without him. I would also like to thank my professors and committee members Stephen Dun- lap, Matthew Dallmeyer, and Lt Col Patrick Sweeney for mentoring me throughout this process and providing crucial feedback that not only improved my research, but improved me as a learner. Finally, I must thank my wife for her continuous support, encouragement, and patience at all times. Throughout this entire process, you never once doubted me or let me doubt myself. This accomplishment would not have been possible without you. Lucas E. Mireles vi Table of Contents Page Abstract . iv Dedication . .v Acknowledgements . vi List of Figures . ix List of Tables . .x List of Acronyms . xi I. Introduction . .1 1.1 Background and Motivation . .1 1.2 Problem Statement . .2 1.3 Research Objectives . .3 1.4 Organization . .4 II. Background and Related Work . .6 2.1 Overview . .6 2.2 NIST Cybersecurity Framework . .6 2.3 The InfiniBand Architecture . .8 2.3.1 Infiniband Components. .9 2.3.2 Software Architecture . 12 2.3.3 InfiniBand Architecture (IBA) Stack Layers . 15 2.3.4 Communication Model . 19 2.3.5 Current Security Features . 19 2.4 Example Application: Vehicle Networks and ADAS . 21 2.5 Relevant Technologies . 23 2.5.1 Field Programmable Gate Array . 24 2.5.2 Peripheral Component Interconnect Express . 25 2.5.3 Linux Device Drivers. 26 2.6 Related Work in IBA Security. 28 2.6.1 Insights into IBA vulnerabilities . 28 2.6.2 IBA GUID Spoofing . 29 2.6.3 Security Analysis of InfiniBand Protocol Implementation . 29 2.6.4 A Framework for Cyber Vulnerability Assessments of Infiniband Networks . 30 2.6.5 An FPGA implementation for a high-speed optical link with a PCIe interface . 31 vii Page 2.7 Summary . 31 III. Infiniband Case Studies . 33 3.1 Objective . 33 3.1.1 Testbed Setup . 33 3.1.2 Case Study 1: Traffic Monitoring . 36 3.1.3 Case Study 2: Implementation of a Network Security System on InfiniBand Verbs. 38 3.1.4 Case Study 3: Performance of Software-Based Security . 40 3.2 Results . 41 3.2.1 Case Study 1: Results . 41 3.2.2 Case Study 2: Results . 43 3.2.3 Case Study 3: Results . 44 3.3 Conclusion . 45 IV. Hardware Security Solutions . 47 4.1 Objective . 47 4.2 Possible Technology . 47 4.3 Requirements . 48 4.4 Exploration Approach . 50 4.5 Hardware Accelerated Security Protocol . 51 4.5.1 Procedure . 54 4.5.2 Findings and Implications . 57 4.6 Programmable SmartNIC via FPGA . 58 4.6.1 Procedure . 60 4.6.2 Findings and Implications . 66 4.7 Programmable SmartNIC via System on Chip . 67 4.7.1 Future Implications . 68 4.8 Summary . 69 V. Conclusion . 71 5.1 Overview . 71 5.2 Summary . 71 5.3 Research Contributions . 75 5.4 Future Work . 75 5.5 Conclusion . 76 Bibliography . ..