Software BETTER PLAY IT SAFE Using static analysis tools for safety certification By Robert Dewar, PhD, and Ben Brosgol, PhD Building reliable software is difficult but achievable. Choosing an appropriate language is important but is only the first step. The careful selection of a coordinated set of tools is just as important or perhaps even more so. For safety-critical systems, using qualified verification tools that tell as much as possible about the software as early as possible helps increase confidence in the system’s correctness while reducing the costs for the system’s certification. Complying with safety-critical standards Tool qualification and software many such optimizations but still does not such as DO-178B involves demonstrat- certification realize the full potential of the Itanium ing that a system meets its requirements Before considering specific tools that chip. Development continues at a rapid and does not introduce safety hazards. An assist in the production of safety-critical pace, but so far no compiler achieves the underlying issue is that the certification certified code, let’s first have a look at the chip’s full performance potential. evidence is based on the system’s static issue of achieving confidence in the tools source text but needs to relate to the sys- themselves. Obviously it is desirable to Not only are the compilers themselves tem’s dynamic, runtime, behavior. How use tools that are known to be 100 percent out of reach of formal safety certification does one guarantee prevention of runtime reliable and free of any errors. because of their inherent complexity, but insecurities such as misusing an integer as to make things worse, there are no formal an address, exhausting memory resources, A first view would be that the tools specifications of these languages suitable selecting a nonexistent element in a data should be certified with the same rig- for certification. International standard structure, buffer overflow, referencing a orous approach that is used for safety- documents define the languages in suf- variable before it is initialized, or access- critical applications. After all, if life ficient detail for programmers and com- ing shared data across concurrent activities safety depends on the program’s correct- piler writers, but these definitions are not without protection? ness, shouldn’t any tools that are used in at the formal complete level needed for its production also be totally reliable? certification using DO-178B or similar The programming language chosen and approaches. the set of features used are obviously Unfortunately, this is not practical. Why important in the development of safety- not? The difficulty is that large systems So that sounds bad. What do software critical systems. But even with a language are written using large and complex lan- developers do if the tools they use cannot designed for reliable programming, such guages. Experience has shown that in be shown to be completely reliable? The as Ada, it will be possible to write pro- order to develop maintainable applica- answer is that in the certification process, grams that compile but that encounter tions, languages with rich features are they either certify at the object code level, such issues at runtime. needed. It is certainly possible to design bypassing the issue of whether the com- very simple languages, but in practice piler is fully reliable, or they certify at the A solution is to use static analysis tools such languages are not suitable for build- source level and include a detailed analy- that can detect and thus prevent poten- ing large, modern, complex applications. sis showing the correspondence of the tial runtime insecurities. If such a tool is source code with the object code. Vendors being used to replace a process that would Even C now has an international standard such as Verocel provide tools that assist at otherwise be done manually during the document that is very large, and compil- the object code level. In either case, the system certification, then, in DO-178B ers for modern languages are themselves focus is on the object code, so if the com- parlance, the tool needs to be “qualified very large and complex programs. For piler fails to generate correct code, this as a verification tool.” If the tool is such example, the 1999 version exceeds 500 will be discovered during the certification that its failure could introduce errors into pages – comparable in size to the Ada process. the system, then it would need to be quali- standard. A compiler for a language fied as a development tool, a much more such as Ada can comprise a million lines The situation is similar for any tool that stringent requirement. of code or more. Furthermore, modern is involved in actual code generation. microprocessor design requires compil- However, there is a brighter side to this Among the static analysis tools applica- ers to perform extensive optimizations. picture. Software developers also use ble in a DO-178B certification context, a Indeed the requirement for sophisticated tools that are not directly involved in code stack usage analyzer is particularly impor- optimization is becoming more stringent generation, but instead provide informa- tant for data space predictability. This tool as time goes on; for example, reasonable tion about the program. Such analysis calculates the maximum amount of stack performance on the Itanium, ia64, micro- tools have a completely different sta- space that a program would ever need. processor architecture is only possible for tus from the compiler generating object Such information is especially relevant in compilers carrying out advanced opti- code. An error in an analysis tool does not memory-constrained environments such mization algorithms. The Free Software directly cause any error in the resulting as VMEbus systems. Foundation’s GCC compiler implements program. At worst, the tool may complain Reprinted from VMEbus Systems / April 2006 Copyright 2006 Software BETTER PLAY IT SAFE about a nonexistent problem, which can be ignored, or it may miss something it Does the tool replace processes otherwise required by DO-178B? should have caught, which will be found later during object code certification. Yes No DO-178B recognizes the difference between a tool such as a compiler, which No need to qualify generates code that is part of the opera- tional system, and an analysis tool that does not generate code. The differences between the qualification requirements Does the tool produce output that is part of the airborne software Can the tools fail to detect errors? for these tools are illustrated in Figure 1. (i.e., can the tool introduce errors?) The qualification procedure for analysis tools or verification tools still requires Yes Yes careful generation of objectives, and thor- ough testing, but does not operate at the same stringent level as certification of the avionics application code itself. Need to qualify as development tool Need to qualify as verification tool Qualified tools can be an important com- Figure 1 ponent in the production of safety-critical code. By using analysis tools, the software Ada, however, the two variables have dis- language is only as good as the software developer can find errors earlier, which tinct types. Since Ada’s predefined “>” development environment that supports it. reduces development expense. By using operation requires operands of the same A compiler that correctly implements the qualified analysis tools, the developer can type, the error would be detected at com- language is important but is not sufficient. get credit for some of the automated veri- pile time in Ada, and the compiler would Let’s now look at some other important fication work that otherwise would need reject the program. It is, of course, much attributes for a compiler in the context to be done manually, which reduces the cheaper when the compiler detects such of safety-critical development, and then certification expense. errors than if time is spent debugging to consider the role of static analysis tools track them down. in general and a stack usage analyzer in Programming language choice particular. Although they cannot be certified with Although the compiler cannot be abso- regard to object code accuracy, compil- lutely trusted to generate correct code, in The compiler as a static analysis ers actually do far more than just generate practice it is safe to rely on the compiler tool code. Modern languages have been care- to reject incorrect programs, and to make Although compilers are usually thought fully designed so that the compiler can use of these error messages to find prob- of as tools for generating code, they can detect many problems before a program is lems early. If an incorrect error message often perform much more extensive tasks even run. Most notably, the strong typing is generated, then that’s annoying but of program analysis. In addition to the system of a language like Ada can find does not compromise safety. If a message detection of incorrect programs, a good many errors early on. As an example, sup- is missed, that’s also annoying because compiler can provide extensive warnings pose a program needs to deal with physi- the error won’t be found until later in the about suspicious code that is not actually cal units; in Ada, this might be expressed certification process, but still there is no wrong but which represents likely errors as follows, using distinct types: compromise to safety. or at least code that should be reviewed. type Temperature is range 20 .. In the case of Ada, there is a formal As an example, consider an Ada program 30; -- Celsius validation process for Ada compilers that contains: type Pressure is range 0 .. 1000; that was initially supervised by the U.S.