Open-source static code analyzer review

April 2018 Table of contents Introduction ...... 2 Part 1 : The research of static code analyser ...... 2 Part 2 : Installing the analyzers ...... 3 Part 3 : Reviewing the analyzers ...... 3 Part 4 : Testing the analyzers on AGL source code...... 4 Conclusion ...... 5

Introduction

We are a group of 3 students from ENSIBS, a French engineer school. We have had a project in collaboration with IOT BZH whose goal was to improve the continuous integration chain of AGL with the integration of some static analyzer in the process. This document will explain the whole process of research, installation, and test of the different solutions.

Part 1 : The research of static code analyzer

The first part of our research was to find some static code analyzer. The requirements were :

- The analyzer should work on linux - The analyzer should support /C++ languages - The analyzer has to be open source - The analyzer has to have been updated recently. - The analyzer has to be well documented.

This step was quite difficult because lots of analyzers are proprietary software and the remaining ones are often out of date.

With these requirements, we have found these analyzers :

- Sonarqube : https://www.sonarqube.org/ - CppCheck : http://cppcheck.sourceforge.net/ - Frama-c : https://frama-c.com/ - infer : http://fbinfer.com/docs/getting-started.html Part 2 : Installing the analyzers

All of the analyzers have been installed on a Debian 9 virtual machine. Installing cppcheck was very easy because it is in the Debian repository. Installing sonarqube was a bit more difficult because we have a service to install and some configuration to do.

Frama and infer are using ocaml so they have to be install with opam. It was very difficult because some dependencies needed to be resolved.

At the end, we have ignored infer because we haven’t reached to install some dependencies.

Part 3 : Reviewing the analyzers

The next step was to review the analyzers to determine if they were powerful or not. We have designed a set of test, with some piece of code that should trigger vulnerability warnings.

Here is an array with the discovered vulnerabilities.

Vulnerabilities CppCheck Sonarqube Frama-c Boundary error

Dead code

Invalid memory access

Memory leaks

Mismatched allocation deallocation

Missing allocation

Uninitialized memory access

Uninitialized variables

Unused variables

We can see that cppcheck can detect most of the common vulnerabilities. Frama-c is also quite powerful but is not compatible with c++. Finally, sonerqube completely unable to detect the common c/c++ vulnerabilities. This analyzer is more powerful with java and has to be completed with additional plugins to be powerful on c/c++ languages.

Part 4 : Testing the analyzers on AGL source code.

We have then been able to test the analyzers on AGL source code. We have chosen to work on agl 5.0 electrical eel because master is likely to change a lot. We have chosen to test the analyzers on two projects first :

- The low level can service, which has been developed by a beginner developer - The binder, which has been developed by an advanced one.

CAN service :

On that project, just cppcheck was able to work because that project is written in cpp

Binder :

Cppcheck :

Frama-C :

As you can see below, Frama-c don’t work on AGL source code because it isn’t able to resolve the dependencies. We didn’t reach to solve that issue, so we don’t advise to use frama-c on AGL project.

Conclusion

Open source static analyzers are very limited : Either they don’t work well with C/C++, or they are difficult to use with large projects. Cppcheck is the best one between all the one we have tested. It should be quite easy to include Cppcheck on to add several test at each AGL build.