Faculty of Informatics, Masaryk University wyA 1 Compiling Applications for Analysis with DIVINE Master’s thesis Zuzana Baranová Brno, 2019 Declaration Hereby I declare that this thesis is my original work, which I have created on my own. All sources and literature used in writing the thesis, as well as any quoted material, are properly cited, including full reference to its source. Advisor: doc. RNDr. Petr Švenda, Ph.D. Consultant: RNDr. Petr Ročkai, Ph.D. Abstract Verification and other formal analysis techniques are often demanding tasks, both in skill and time. This is why non-critical software is seldom subjected to the same rigorous analysis as safety-critical software. However, all software would benefit from an extra level of assurance of its reliability and there has been prolonged effort on the side of analysis tools developers to make their use easier. Ideally, the aim is to integrate analysis techniques into the normal software development process. Among other tools, DIVINE is one such verifier whose long-term key goal is to bring verification closer to the developers of everyday software. A big step forward was direct verification of C and C++ programs. The programs are compiled into a more analysis-friendly form to be verified, notably LLVM bitcode (LLVM IR). Another big step in lowering barriers for adopting formal verification is re-using automated build tools and existing build instructions of projects, which would prevent the need for manual compilation of software. The purpose of this thesis is to replace the existing compilation toolchain of DIVINE with a tool which could be transparently used in automated build systems and which would produce bitcode of the whole program. We have successfully implemented such a tool, and evaluated its practicality on a number of real projects which use automated build systems. The resulting bitcode can be loaded into DIVINE and verified. Keywords C, C++, DIVINE, LLVM IR, ELF, compilation, formal verification, POSIX, build automation, implementation iii Acknowledgements Firstly, I would like to thank the ParaDiSe group, for welcoming me, and for the endless supply of obscure words and programming language concepts. I would also like to thank my Finnish family, for making sure I survived the long winter. Most of all, I have to thank Mornfall, for the unreasonable amount of time he dedicated to fixing my own problems, for explaining the same things to me over and over again, and for not giving up on me. Notes This thesis is loosely based on our papers Reproducible Execution of POSIX Programs with DiOS [35] and Compiling C and C++ Programs for Dynamic White-Box Analysis [34]. v | Contents 1 Introduction1 1.1 Motivation ..................................... 3 1.2 Goals ........................................ 3 1.3 LLVM IR ...................................... 4 1.4 DIVINE....................................... 6 2 Compilation Process9 2.1 Binaries....................................... 9 2.2 Header files and Libraries ............................. 10 2.3 Compiler Architecture............................... 14 2.4 Anatomy of an ELF file .............................. 22 3 Related Work 25 3.1 Program Analysis.................................. 25 3.2 Link-Time Optimizations ............................. 26 3.3 wllvm ........................................ 26 4 Compilation Automation 29 4.1 Build Systems.................................... 29 4.2 Make......................................... 30 4.3 CMake........................................ 31 4.4 configure script.................................. 33 5 DiOS 39 5.1 Motivation ..................................... 39 5.2 Building verified executables with DiOS..................... 41 5.3 The DiVM Virtual Machine............................ 42 5.4 DiOS Libraries ................................... 44 5.5 Kernel........................................ 46 5.6 Changes in DiOS.................................. 48 6 dioscc 51 6.1 divcc and dioscc ................................. 51 6.2 Intended Use .................................... 52 6.3 Limitations ..................................... 53 vii 6.4 Design........................................ 54 6.5 Implementation................................... 55 6.6 Linking ....................................... 55 6.7 API & ABI Compatibility............................. 56 6.8 Internal Structure ................................. 60 7 Evaluation 67 7.1 System Information ................................ 67 7.2 Feasibility with Real Projects........................... 67 7.3 DiOS......................................... 71 8 Conclusions 73 8.1 Future Work .................................... 73 A Archive and Manual 75 Bibliography 77 1| Introduction The rise of information systems has brought the need to prove that the programs these systems use function correctly. This is true not only for safety-critical systems, since an undetected error in, say, a ticket reservation system can also prove very costly. In fact, anyone who writes programs wishes them to behave as they intended. To ensure the correctness of programs, various analysis tools can be used, with wide-ranging goals: from simple testing of correct behaviour, through more rigorous methods, including memory error detection (such as the tool valgrind [31]), to more formal methods – like static analysis or model checking, used for instance for parallel-program-specific analysis (checking for deadlocks, starvation, etc.). The analyses can be performed at different stages of development of the program, or even on the final executable. As a result, the tools necessarily work with different representations of the program, e.g. source code, or machine code – processor instructions.1 Finally, some tools require that the program is first transformed (manually or automatically) into a representation that the analysis tool can work with. There are two basic approaches: the program is either described in a modelling language that can better articulate its properties (these often have to be written by hand, based on domain-specific knowledge); or the program is first translated using another tool into a language that is more analysis-friendly. Examples of the former are: • the DVE modelling language, used to model asynchronous systems, • the SPIN model checker and its input language (PROMELA) for system descriptions. Depending on the type of analysis, some of the intermediate languages that are used in the second approach, i.e. those that are obtained automatically from the source language using a specialized compiler, include (the following examples assume C and C++ programs as inputs): • GOTO programs (a type of control-flow graph representation), which are generated by goto-cc2, and which in turn serve as input for the CBMC analyser, • the LLVM intermediate representation, which is used in a number of tools, including the symbolic executor KLEE, a static analyser called PhASAR, or the DIVINE model checker [3]. For C and C++ programs, LLVM IR can be obtained using clang, although other languages (such as Rust) can also be translated into this intermediate form. 1Some static analysis tools work directly with the source code; other tools (like valgrind) work with the resulting executable program, analysing it at runtime and making use of debug metadata. The respective stages that a program goes through and the forms a program takes are described in Chapter2. 2More information on goto-cc can be found in Related Work (Section 3.1). 1 With programs written in a compiled, high-level programming language, it is desirable for the analysis to be performed as late as possible in the compilation process, to ensure that the verification result applies to the executable program. Ideally, verification should be done on the fully-compiled, executable binary, which consists of the actual instructions that will be used once the program is run. However, the binary is not portable, i.e. it can only be executed on the system it was compiled for. As such, the choice of the level of abstraction poses a trade-off between veracity and applicability of analysis (analysis of low-level code is more accurate but only applies to the particular binary, while higher-level analysis is more widely applicable, but less precise). With this in mind, and because the analysis should cover a broad range of systems, many tools instead opt for an intermediate representation of the code as verification input. In this thesis, we will particularly work with LLVM IR3. Another reason for the abstraction of code into LLVM IR is that at the level of processor instructions, after the compiler has possibly performed many optimisations already, it is extremely difficult to reason about the original source code. It is also not trivial to map the instructions back to the source code, so that a potential problem can be traced. In practice, we have observed that the use of LLVM IR in verification is a good trade-off, common in verification tools. Program analyses can be broadly classified into two categories: static and dynamic. The term dynamic analysis refers to a type of analysis where the program is inspected at runtime, i.e. during execution or interpretation. This is in contrast with static analysis, during which the program’s source code or the intermediate representation is analysed directly. [2] Therefore, static analysers do not need to have the entire program available to analyse it – they can analyse it partially and still get valuable results. On the other hand, dynamic analysis requires that the whole