Software Engineering Technology The Qualification of Software Development Tools From the DO-178B Certification Perspective Dr. Andrew J. Kornecki Dr. Janusz Zalewski Embry Riddle Aeronautical University Florida Gulf Coast University Software development tools are in wide use among safety-critical system developers. Examples of such use include aviation, auto- motive, space, nuclear, railroad, medical, and military applications. However, verification of tool output to ensure safety, man- dated in highly regulated industries, requires enormous effort. If a tool is qualified, this effort can be reduced or even eliminat- ed. The Radio Technical Commission for Aeronautics Document Order-178B and related documents provide guidelines by which to qualify these tools. However, current regulations, business models, and industry practice make this goal difficult to accomplish. This article discusses the qualification of development tools and the potential impact of this process on the aviation industry. oftware development tools are com- (SC-167) in 1989 with the objective of context of a specific airborne system.” It is puter programs that help developers updating the DO-178A by focusing on the certification authority that decides on createS other programs. Such tools have five areas: documentation integration and the outcome of the qualification process. been in use since the early days of com- production, system issues, software devel- Moreover, qualification, if claimed, is a puting to improve the efficiency of the opment, software verification, and soft- requirement in getting a system certified. development process by automating mun- ware configuration management and soft- dane translation operations and bringing ware quality assurance. The resulting doc- Types of Software the level of abstraction closer to the appli- ument, DO-178B, provides guidelines for Development Tools cation engineer. Nowadays, development applicants developing software-intensive DO-178B differentiates between verifica- tools are used in a variety of safety-critical airborne systems [1, 2]. It discusses objec- tion tools that cannot introduce errors but may applications, including the aviation, auto- tives that need to be met to show that the fail to detect them and development tools motive, space, nuclear, railroad, medical, software development process provides whose output is part of airborne software and and military industries, and contribute to specified levels of safety assurance. It also thus can introduce errors. There is a signifi- the risks associated with using respective describes the processes and means of cant amount of effort involved to qualify products. Despite these risks to society, compliance. a verification tool, and much more to development tools are rarely qualified in a Systems are categorized by DO-178B qualify a development tool. However, sense comparable to product certification as meeting safety assurance levels A numerous development tools have been in regulated industries. The objective of through E based on their criticality in sup- used successfully in many certified pro- this article is to look at the current state of porting safe aircraft flight. The level A sys- jects without being qualified. To define a the tool qualification process, identify the tem is the most critical: The failure of subject matter more narrowly, we need to issues, and propose recommendations for such a system could result in a catastroph- take a closer look at the entire domain of potential improvement, focusing on the ic failure condition for the aircraft. The software development tools. aviation industry. level E system is the least critical: Such a The landscape of modern software System Certification Versus system has no effect on the operational development tools is very broad, as illus- capability of the aircraft or pilot workload. trated in Figure 1 (see page 20). Following Software Tool Qualification Although the RTCA DO-178B is the lead- the traditional model of the development Certification of airborne equipment is typ- ing source of guidelines for software process from requirements to implemen- ically achieved through the Federal developers engaged in such system con- tation, we can identify the following: Aviation Administration (FAA) authoriza- struction, two other documents have criti- • The requirements category that includes tion of a type certificate (the entire air- cal bearing on the subject. RTCA DO- tools used early in the life cycle to craft), supplemental type certificate (new 248B [3] clarifies some of the misinterpre- identify and specify the software equipment in a specific aircraft), or a tech- tation of the DO-178B. The FAA Order requirements. nical standard order (minimum perfor- 8110.49 compiles a variety of guidelines • The design category that includes tools mance standard for materials, parts, and related to the use of software in airborne allowing developers to create architec- appliances used on civil aircraft). A special systems. Chapter 9 is specifically dedicated tural and detailed design of the soft- committee (SC-145) of the Radio Tech- to tool qualification [4]. ware in a notation of their choice sup- nical Commission for Aeronautics (RTCA) A key component of the updated ver- ported by the tool; often in this cate- convened in 1980 to establish guidelines sion of DO-178B is the concept of tool gory, tools translate the model to for developing airborne systems.The qualification elaborated in Section 12. source code. report “Software Considerations in Air- Qualification is a supplementary process • The implementation category that borne Systems and Equipment Certifica- that the applicant may elect to follow in includes all support required to trans- tion” was published in January 1982 as the the course of certifying an airborne sys- late the computer code and transfer it RTCA Document Order (DO)-178 (and tem. According to the definition given in to the target computer. revised as DO-178A in 1985). DO-178B, tool qualification is defined as, As illustrated in Figure 1, three other Due to rapid advances in technology, “The process necessary to obtain certifica- categories of tools can be identified: those the RTCA established a new committee tion credit for a software tool within the related to analysis, testing, and target. April 2006 www.stsc.hill.af.mil 19 Software Engineering Technology e.g.: a Tool Configuration Management Index, Structural Rhapsody Typically With Design RoseRT Code Generator Tool Development Data, Tool Verification Functionality Tool STOOD Records, Tool Quality Assurance Records, Requirements Artisan Tool Integrated Tool Configuration Management Records, or/and e.g.: Development e.g.: Functional SCADE Environment etc. These requirements are also described Reqtify Design Matlab (IDE) in [4]. Tool qualification data are approved BEACON DOORS Tool e.g.: SpecTRM Sildex Tornado only in the context of the overall software DOME Multi development for the specific system Analysis Implementation where the intention to use the tool is stat- Tool Tool ed in the PSAC. The tool itself does not receive a separate qualification stamp of e.g.: Testing RapidRMA Tool approval. Therefore, using the tool on TimeWiz another system/project requires a separate e.g.: Target qualification, although some qualification CodeTest (with RTOS) TestRT credits may be reused. VectorCast e.g.: Surveys requesting which tools are used Insure++ VxWorks QNX by industry were conducted at two national OSE conferences: the 2002 FAA National Tool Categories Integrity LynxOS Software Conference and the 2004 Embry Riddle Aeronautical University/FAA Soft- Figure 1: Software Tool Categories ware Tool Forum. In addition, two follow- Figure 1:Software Tool Categories up e-mail solicitations were sent to more However, for this article, we will focus pri- producer) is to transform an input artifact than 500 professionals working on air- marily on the tools used in the design into output, thus creating another software borne systems. These surveys and solicita- phase, the central component of the soft- artifact. The current process mandates ver- tions resulted in a relatively small sample of ware development life cycle. They reflect ification after each transformation. If this Can responses that did not provide a base for two divertseool vie inwsertpoints error on real-time, safety- transformation has an impact on the final statistically significant results. The com- NO critical systemsinto a irdevelopment,borne which result airborne product, the producer needs to be ments included industry discouragement NO QUALIFICATION from differentso fdevelopers’tware? backgrounds: qualified, but only if the transformation regarding the rigor of development tool • Control engineers consider a system to output wouldNEC notESS beA verifiedRY and the trans- YES qualification, and a justified perception of be a dynamic model consisting of well- formation leads to elimination, reduction, the extensive cost of qualification. defined blocks of specific functionality or automation of any of the DO-178B Potential solutions to assist in commer- (logic, arithmetic,W dynamic).ill The func- processes. The conditions under which a the tool’s output cial off-the-shelf (COTS) development tional paradigm of the model is the development tool requires qualification are NOT be verified tool qualification included extensive ven- basis for system simulation and analysis presented TinOO FigureL M U2 S[4].T (as specified in NO dor collaboration and using alternate means