DOT/FAA/TC-15/27 Reverse Engineering for Federal Aviation Administration William J. Hughes Technical Center Software and Digital Systems Aviation Research Division Atlantic City International Airport New Jersey 08405 February 2016 Final Report This document is available to the U.S. public through the National Technical Information Service (NTIS), Springfield, Virginia 22161. This document is also available from the Federal Aviation Administration William J. Hughes Technical Center at actlibrary.tc.faa.gov. U.S. Department of Transportation Federal Aviation Administration NOTICE This document is disseminated under the sponsorship of the U.S. Department of Transportation in the interest of information exchange. The U.S. Government assumes no liability for the contents or use thereof. The U.S. Government does not endorse products or manufacturers. Trade or manufacturers’ names appear herein solely because they are considered essential to the objective of this report. The findings and conclusions in this report are those of the author(s) and do not necessarily represent the views of the funding agency. This document does not constitute FAA policy. Consult the FAA sponsoring organization listed on the Technical Documentation page as to its use. This report is available at the Federal Aviation Administration William J. Hughes Technical Center’s Full-Text Technical Reports page: actlibrary.tc.faa.gov in Adobe Acrobat portable document format (PDF). Technical Report Documentation Page 1. Report No. 2. Government Accession No. 3. Recipient's Catalog No. DOT/FAA/TC-15/27 4. Title and Subtitle 5. Report Date REVERSE ENGINEERING FOR SOFTWARE AND DIGITAL SYSTEMS February 2016 6. Performing Organization Code 7. Author(s) 8. Performing Organization Report No. George Romanski1, Mike DeWalt2 for phase 1 – research, and Dewi Daniels3for phase 2 – Validation. 9. Performing Organization Name and Address 10. Work Unit No. (TRAIS) 1Verocel, Inc. 234 Littleton Road, Suite 2B, Westford MA 01886 2Certification Services, Inc., P.O. Box 1569, Eastsound, WA 98245-1569 11. Contract or Grant No. 3 Verocel Limited, 129 Devizes Road, Hilperton, Trowbridge, UK, BA14 7AZ DTFACT-09-C-00023 12. Sponsoring Agency Name and Address 13. Type of Report and Period Covered Final Report U.S. Department of Transportation Federal Aviation Administration 950 L’Enfant Plaza SW, 5th Floor Washington, DC 20024 14. Sponsoring Agency Code AIR-134 15. Supplementary Notes The Federal Aviation Administration William J. Hughes Technical Center Aviation Research Division COR was Charles Kilgore. 16. Abstract Reverse engineering (RE) is a class of development processes that starts with detailed representations of system software or hardware description for a device and applies various techniques to produce more generalized, less-detailed representations. The goal is to have more abstract representations that can be used to understand and consider the structure and intent of the more detailed representations. RE has been used in many industries, including aircraft applications for mechanical, hardware and software components. The scope of this report covers RE software and electronic hardware device applications for airborne systems and equipment. The Federal Aviation Administration sponsored this research project to provide a clear understanding of what should be considered RE for airborne software and airborne electronic hardware (AEH) devices and under what conditions it could be deployed or restricted in the aircraft certification environment. This report provides an overview of the aviation industry’s views of RE, the potential issues of employing RE for safety-critical airborne systems, recommendations for when RE application is acceptable, and some associated criteria for successful implementation. The RE development process is the opposite of the traditional waterfall model that has been a well-known commodity to the certification authorities; therefore, RE, which is much less known, poses a concern to the certification authorities. Without a common set of recognized and accepted terminology, definitions, and constraints on the processes and other issues, the certification authorities are forced to provide case-by-case evaluations of the different RE proposals. This report provides the research results for RE. The intended audience includes practitioners who develop compliance evidence, as well as evaluators for airborne software and AEH to be approved under RTCA documents DO-178B and DO-254, respectively. The report proposes a framework for RE of software and electronic hardware for airborne systems and equipment. This report also validates that framework by presenting two case studies. The software case study chosen is a subset of an Ada runtime library. This library was chosen because it is distributed under the GNU General Public License and because it has previously been approved to DO-178B Level A as part of a specific aircraft project. The AEH case study was based on the certification of a system with two programmable logic devices. The intent of these case studies was to validate the framework against two examples that were developed using RE and approved by the FAA. 17. Key Words 18. Distribution Statement This document is available to the U.S. public through the Reverse engineering, Software, DO-178, CAST-18, DO-254, National Technical Information Service (NTIS), Springfield, Complex electronic devices Virginia 22161. This document is also available from the Federal Aviation Administration William J. Hughes Technical Center at actlibrary.tc.faa.gov. 19. Security Classif. (of this report) 20. Security Classif. (of this page) 21. No. of Pages 22. Price Unclassified Unclassified 120 Form DOT F 1700.7 (8-72) Reproduction of completed page authorized ACKNOWLEDGEMENTS We would like to thank Kelly Hayhurst from the NASA Langley Research Center for her valuable contribution to the survey. We also thank the Federal Aviation Administration Review Team, consisting of Barbara Lingberg, Charles Kilgore, Richard Spencer, Srini Mandalapu, and Will Struck, for their technical support and guidance throughout the project. iii TABLE OF CONTENTS EXECUTIVE SUMMARY viii 1. INTRODUCTION 1 1.1 Background 1 1.2 Purpose and Scope 2 1.2.1 Why is This Report Needed? 2 1.2.2 Intent of the Report 3 1.3 Section Overviews 4 2. SURVEY AND INDUSTRY EXPERIENCE 5 2.1 Industry experience 5 2.2 Industry perspective from survey 5 2.2.1 Software Survey Results 5 2.2.2 CEH Survey Results 6 2.3 Terminology 7 2.3.1 Reverse Engineering 7 2.3.2 Configuration Management of Reverse Engineered Artifacts 8 2.3.3 Certifier 8 2.3.4 Forward Engineering 8 3. FRAMEWORK FOR PERFORMING RE 9 3.1 Software Development Processes and Sequence Dependencies 9 3.2 Compatibility of Regulatory Guidance With RE Processes 9 3.3 Roles 12 3.4 Processes 13 3.4.1 An Example of Source Code to LLR Development 13 3.4.2 Generic RE Processes 15 3.4.3 Inputs 15 3.4.4 Outputs 16 3.4.5 Entry Criteria 17 3.4.6 Exit Criteria 17 3.4.7 Process Description 18 3.5 The RE Aspects of Software Verification 18 3.6 Generic RE SME Verification Process 20 iv 3.6.1 Inputs 20 3.6.2 Outputs 21 3.6.3 Entry Criteria 21 3.6.4 Exit Criteria 22 3.6.5 Process Description 22 3.7 Acceptance Criteria 22 4. VALIDATION OF FRAMEWORK 23 5. RECOMMENDATIONS 23 6. CONCLUSIONS 25 7. REFERENCES 25 APPENDICES A—CASE STUDY: ANALYSIS OF PROJECTS COMPLETED AT VEROCEL B—REVERSE ENGINEERING SURVEY C—REVERSE ENGINEERING SURVEY RESULTS D—ANALYSIS OF ISSUES AND THEIR POTENTIAL MITIGATIONS E—VALIDATION OF FRAMEWORK FOR SOFTWARE F—VALIDATION OF FRAMEWORK FOR AIRBORNE ELECTRONIC HARDWARE v LIST OF FIGURES Figure Page 1 Example processes for development and review of LLRs using RE 14 2 Generic processes for development and review of different abstraction layers 15 vi LIST OF ACRONYMS A/D Analog/digital AEH Airborne electronic hardware (includes complex electronic hardware and simple electronic hardware) ARTE Ada Runtime Environment CAST Certification Authorities Software Team CEH Complex electronic hardware COTS Commercial off-the-shelf DER Designated engineering representative DOD Department of Defense DVD-ROM Digital Versatile Disc – Read-Only Memory ELOC Effective lines of code FAA Federal Aviation Administration GNAT GNU New York University Ada 9X Translator GNU A recursive acronym for “GNU’s Not Unix!”—Unix-like computer operation system developed by the GNU Project GPL General Public License HAS Hardware accomplishment summary HIS High integrity software HLR High-level requirement ICU Interface Control Unit KLOC Kilo (thousand) lines of code LAL Less abstract layer LLR Low-level requirement MAL More abstract layer NaN Not a number PHAC Plan for Hardware Aspects of Certification PLD Programmable logic device PR Problem report PSAC Plan for Software Aspects of Certification RE Reverse engineering RESP Reverse Engineering Software Plan SAS Software Accomplishment Summary SDP Software Development Plan SME Subject matter expert SOI Stage of involvement SUNECO Training aid identifying the combination of SUfficient, NEcessary, and COrrect TSO Technical Standard Order UAS Unmanned aircraft system VHDL Very High Speed Integrated Circuit (VHSIC) Hardware Description Language vii EXECUTIVE SUMMARY Reverse Engineering (RE) is a class of development processes that start with detailed representations of software for a system, or a hardware description for a device, and apply various techniques to produce more generalized, less detailed representations. The goal is to have more abstract representations that can be used to understand and consider the structure and intent of the more detailed representations. RE has been used in many industries, including aircraft applications for mechanical, hardware, and software components. The scope of this report is the electronic hardware devices and software applications of RE for airborne systems and equipment. The Federal Aviation Administration (FAA) sponsored this research task to provide a clear understanding of what should be considered acceptable RE for airborne software and airborne electronic hardware (AEH) devices, and under what conditions it could be deployed or restricted in the aircraft certification environment.