Static analysis for Ada, C/C++ and Python: different languages, different needs. Maurizio Martignano Spazio IT – Soluzioni Informatiche s.a.s. San Giorgio Bigarello, Italy [email protected] Abstract—Spazio IT has been working on the Independent • Bugs Finding. Software Verification and Validation of several codebases, some written in Ada, others in C/C++ and more recently also in Metrics Gathering, Structure Analysis Python; in all cases Spazio IT has used static analysis techniques In some projects the codebase under analysis has to and tools facilitating code inspection. Static analysis has always proven to be beneficial, but depending on the programming satisfy a set of quality requirements, it has to “meet some language, its advantages have emerged in different areas and specific set of quality objectives”. This is usually verified namely: metrics and structural analysis for Ada, bugs finding at according to a particular software quality model, e.g. ISO/IEC execution/semantic level for C/C++ and errors finding at 25010 [1]. These models identify some quality compilation/syntactical level for Python. The paper presents first “characteristics” (e.g. “maintainability”, that derive from some some general considerations on static analysis, then it sub-characteristics (e.g. in the case of “maintainability” its concentrates on static analysis specifically applied to Ada, C/C++ sub-characteristic are “modularity”, “reusability”, and Python codebases. Finally, the paper describes Spazio IT “analysability”, “modifiability”, “testability”). Sub- future activities in the area of static analysis. characteristics in turn derive/are computed from “metrics” like Keywords— static analysis, code inspection, Ada, C/C++, “lines of code”, “ratio comment to code”, “cyclomatic Python, open source complexity”, “construct nesting”, and so on. Metrics are gathered by static analyzers; some static analyzers are also able to compute sub-characteristics and, in I. INTRODUCTION turn, characteristics according to selected models. Spazio IT has been working on several Independent In Spazio IT experience, several times, these Software Verification and Validation projects, especially in “apparently theoretical” quality measurements, have allowed application domains like avionics and automotive, where the the identification of critical areas in the codebase actually codebases under analysis have been written mostly in C/C++, requiring fixes to improve their readability and sometimes in Ada and more recently also in Python. Though maintainability. in all these projects static analysis has always proven to be beneficial, its advantages have emerged in different areas and Checking Guidelines and Standards namely: metrics and structural analysis for Ada, bugs finding The adoption of guidelines and coding standards is at execution/semantic level for C/C++ and errors finding at supported by the principle that, given a programming compilation/syntactical level for Python. language, it is possible to identify a subset of that language Section two of this paper will present static analysis aiming at improving the portability, security and safety aspects in general, why it is used in ISVV projects, in which areas it of programs written in that language. MISRA C 2012, MISRA provides the more benefits and its relationships with testing. C++ 2008 [2] and AUTOSAR C++ 14 [3] are example of such Section three will concentrate on static analysis and Ada. “subsets” for C/C++. SPARK 2014 [4] is another example of Section four on static analysis and C/C++. Section five will these language “subsets”, but this time for Ada. While, at the concentrate on static analysis and Python. Section six will moment, the C/C++ “subsets” can only be checked via static present Spazio IT next activities in the area of static analysis. analyzers, SPARK 2014 stands on top of the GNAT compilation system. In Spazio IT experience, only relating to C/C++ II. WHY STATIC ANALYSIS? “subsets”, adopting these guidelines and standards has often Static analysis techniques and tools are used in generated a considerable amount of noise, false positives, Independent Software Verification and Validation mostly for especially when applied to legacy codebases. This is why in three reasons: these cases, a fine-tuning activity, identifying with checks, • Metrics Gathering, Structure Analysis; rules to allow and which ones to disable before the actual • Checking the adherence of the codebase under analysis execution had been necessary. analysis to Guidelines and Standards; Bugs Finding DO-178B/C [8] and EN 50128 / IEC 62279 [9]. Likewise, for Static analysis has proven to be very helpful also in the very same two reasons, also MathWorks PolySpace for bugs finding. This capacity has been relying on techniques Ada [10] and SciTools Understand [11] have bene used. like abstract interpretation, data flow analysis during symbolic In some limited cases AdaCore CodePeer [12] and Polyspace (i.e. virtual) execution, bounded model checking and the like. have also been used as supporting tool when looking for bugs With time most of these techniques, that originally during coding inspections activities. were only available in some advanced commercial tools or in In 2015 Spazio IT developed for Airbus an Ada some kind or research/experimental tools, have been SonarQube Plugin able to compute the so called implemented also in “popular” and well supported tools like “Maintainability Index” based on a quality model developed Clang Static Analyzer, Clang Tidy and FB-Infer. These “new by Airbus and similar to ISO/IEC 25010; in this model generation” tools, being based on actual compiler “maintainability” was a function of “analyzability”, technologies, are able to process real codebases in a “changeability”, “stability”, and “testability”. reasonable time; on the contrary some of the original tools could only work on very limited portions of code. IV. STATIC ANALYSIS IN C/C++ Table 2 shows the C/C++ static analyzers used at Syntactical Checks Spazio IT over the years. A fourth reason, only appeared recently with “interpreted” languages like Python and justifying the use of static analysis, is that on “interpreted” codebases the static analyzer, the “linter” can be used as a sort of compiler to verify the syntactical correctness of the entire codebase, very difficult to prove otherwise. More on this in section five, on Python. Static Analysis vs Testing Static analysis, especially bugs finding, has Table 2 – C/C++ Static Analyzers used at Spazio IT sometimes been considered as a replacement for testing. (M = mostly used for, A = also used for) Spazio IT believes it is better to consider static analysis as a Gimpel Software PC-Lint(-Plus) [13] has been used complementary activity to testing. The quality of testing mostly for metrics gathering and checking the adherence of depends on the quality and completeness of the test cases. the codebases to MISRA and AUTOSAR coding standards. In Static analysis efficiency in bugs finding depends on the used some cases, PC-Lint has also been used for bugs finding (e.g. techniques and tools and not on the quality of the test cases or identification of recursive loops, logical errors in data flow their completeness. In huge codebases static analysis processing and the like). techniques could help identifying which areas require more The open-source tools Cppcheck [14], Clang-SA testing than others, so they could help in maximizing the (Static analyzer) [15], Clang-Tidy [16] and FB Infer [17] have results that can be obtained with testing – also when resources been used for bugs finding. Cppcheck has also been used for are limited. metrics gathering and checking the adherence of the codebases to MISRA. III. STATIC ANALYSIS IN ADA It has to be noticed that, recognizing the power and Table 1 shows the Ada static analyzers used at Spazio continuous improvement of the Clang tools, also Cppcheck is IT over the years. more and more relying not only on its internal analysis capabilities but on these Clang tools. Table 1 – Ada Static Analyzers used at Spazio IT (M = mostly used for, A = also used for) AdaLog AdaControl [5] has been used for metrics gathering and checking the adherence of the codebases to project-specific coding standards.AdaCore GNATmetric [6] Figure 1 – Dangling references spotted by the Ada and GNATcheck [7] have also been used for metrics gathering compiler and by the C compiler and checking the adherence of the codebases to standards like Figure 1 shows that C/C++ compilers, if properly used, can be as strict as Ada compilers (e.g. Ada Gem #33 [18]). In the Clang compilation system, the compiler and the static analyzers (Clang-SA and Clang-Tidy) are based on the very same libraries, so, in a way, the “Compiler” is the “Static Analyzer”. Static analyzers as CBMC [19] and Frama-C [20] though very interesting from a theoretical and research point of view, at the moment, that is April 2021, are not a viable option for the analysis of large codebases. It is also interesting Table 3 - Python Static Analyzers used at Spazio IT to notice that, like Cppcheck, also Frama-C is more and more (M = mostly used for, A = also used for) trying to integrate with the Clang compilation system and The open-source tool Flake8 [22] has been used for static analyzers. metrics gathering and checking the adherence of the codebases to some project-specific coding standards. Like in the case of C/C++, also in Python, the open- source tools Bandit [23], Pylint [24] and SonarQube Python Plugin [25] have been used for bugs finding, Figure 2 – a bug discovered by Clang Static Analyzer and showed in its GUI Figure 2 shows a bug caught by Clang Static Analyzer and Figure 3 shows the very same bug, this time displayed from by SonarQube (a code quality platform used Figure 4 – SonarQube platform showing a bug caught by by Spazio IT for all its ISVV activities [21]). the SonarQube Python Plugin Somehow static analysis is applied to Python codebases in the same way that is applied to C/C++ codebases: mostly for bugs finding.