Detecting Memory Errors in a Constrained Embedded Linux System Fredrik Nyberg, Emil Bengtsson
Total Page:16
File Type:pdf, Size:1020Kb
MASTER’S THESIS 2020 Detecting Memory Errors in a Constrained Embedded Linux System Fredrik Nyberg, Emil Bengtsson ISSN 1650-2884 LU-CS-EX: 2020-28 DEPARTMENT OF COMPUTER SCIENCE LTH j LUND UNIVERSITY EXAMENSARBETE Datavetenskap LU-CS-EX: 2020-28 Detecting Memory Errors in a Constrained Embedded Linux System Fredrik Nyberg, Emil Bengtsson Detecting Memory Errors in a Constrained Embedded Linux System (Perspective from the Automotive Industry) Fredrik Nyberg Emil Bengtsson [email protected] [email protected] June 22, 2020 Master’s thesis work carried out at Volvo Cars. Supervisors: Flavius Gruian, [email protected] Paul Asterland, [email protected] Examiner: Jonas Skeppstedt, [email protected] Abstract Memory errors are the cause of many bugs in software written in C and C++. Embedded systems often run software written in these languages, and it is com- mon for these systems to have limited resources. Embedded systems are also susceptible to errors, since they are often supposed to run for long times with- out crashing. Finding errors is therefore important, which makes analysing the software rewarding, but the low resource availability can make dynamic analysis tools challenging to use. We have evaluated different dynamic memory analy- sis tools by assessing their error finding capabilities, and their impact on the analysed programs’ performance. We have carried out our work for Volvo Cars, and performed our evaluations on one of their embedded systems. As part of the evaluation, we also used the most promising tools to analyse some of Volvo’s pro- grams in a real setting. We managed to find two memory leaks in one of Volvo’s programs by analysing them with the tools. The leaks were confirmed to be real by a developer, and Volvo could fix them. Based on our findings, we recommend a selection of tools that we consider useful for analysing software on embedded Linux. We conclude that using 2-3 less advanced tools in separate analysis runs works better than using a more advanced tool on Volvo’s embedded system. This approach introduces less overhead while still being able to find as many errors as the more advanced tools. Keywords: Memory Errors, Limited Resources, Embedded Systems, Automotive, Mem- ory Analysis 2 Acknowledgements We would like to thank our LTH supervisor Flavius Gruian for his guidance during the entire thesis process, especially for his detailed and constructive feedback on our writing. We would also like to thank Paul Asterland at Volvo Cars for guiding our work in a direction which made the thesis relevant for Volvo Cars and for making sure we felt welcome and had all the support and equipment we needed. Additionally, we want to thank all developers at Volvo who helped us with any technical questions we had, and with giving suggestions for the thesis: Johan Bohlin, Ola Lilja, Piotr Tomaszewski, and Mikael Persson. Lastly, we would like to thank everyone else who helped proof-reading our thesis; your feedback has greatly improved our writing. 3 4 Contents 1 Introduction 9 1.1 Problem Definition . 10 1.2 Research Questions . 11 1.3 Limitations . 11 1.4 Delimitations . 11 1.5 Division of Work . 12 2 Background 13 2.1 Embedded systems . 13 2.1.1 Embedded Software . 14 2.2 Hardware Dependent Software . 15 2.3 Memory Leaks and Memory Faults . 16 2.3.1 Memory Leaks . 17 Memory Leak Detection and Prevention . 17 2.3.2 Memory Faults . 18 2.4 Common Weakness Enumeration . 18 2.4.1 List of Errors and Their CWE Classification . 19 2.5 Memory Analysis . 22 2.5.1 Static Analysis . 22 2.5.2 Dynamic Analysis . 24 2.6 The Automotive Industry and Software . 25 2.7 Automotive Standards and Guidelines . 27 2.8 Volvo and the TCAM .............................. 28 2.9 Related Work . 29 3 Method 33 3.1 Literature Review . 33 3.2 Initial Tool Selection . 34 3.3 Benchmarks . 35 3.3.1 Memory Errors . 35 5 CONTENTS 3.3.2 Overhead . 36 Description of Benchmarking Programs . 37 3.4 Evaluation . 39 3.4.1 Memory Faults . 39 3.4.2 Overhead . 40 Execution Time . 40 Memory Consumption . 40 3.5 Revised Tool Selection . 40 3.6 Testing on Volvo’s Programs . 42 3.7 Final Tool Selection . 43 4 Tool Survey 45 4.1 General Usage of Tools . 45 4.2 Implementation Details of Tools . 46 4.2.1 Valgrind . 46 4.2.2 Dr. Memory . 47 4.2.3 AddressSanitizer . 47 4.2.4 Debug_new . 48 4.2.5 Heaptrack . 48 4.2.6 Heapusage . 48 4.2.7 Gperftools . 48 5 Results 51 5.1 Initial Tool Selection . 51 5.2 Evaluation . 51 5.2.1 Memory Fault and Leak Detection . 52 5.2.2 Tool Feedback . 53 5.2.3 Execution Time . 54 5.2.4 Memory Consumption . 56 5.3 Analysing Volvo’s Programs . 58 5.3.1 AddressSanitizer . 58 5.3.2 Debug_new . 59 5.3.3 Heapusage . 60 5.3.4 Heaptrack, Dr. Memory, and Gperftools . 60 6 Discussion 61 6.1 Memory Leaks and Memory Faults . 61 6.2 Overhead . 63 6.3 Development and Maintenance of Tools . 64 6.4 Testing on Volvo’s Programs . 64 6.5 Tool Usage in the Workflow . 65 6.6 Possible Pitfalls . 65 7 Conclusions 67 7.1 Answers to Research Questions . 67 7.2 Future Work . 69 6 CONTENTS References 73 Appendix A Description of Terms 85 Appendix B Usage of Tools 87 B.1 Volvo Script . 87 B.2 Debug_new . 87 B.3 Memwatch . 88 B.4 Heaptrack . 89 B.5 Heapusage . 90 B.6 Valgrind (Memcheck) . 91 B.7 Leaktracer . 91 B.8 Gperftools Heapcheck . 92 B.9 AddressSanitizer . 92 B.10 Duma . 92 B.11 Malloc_count . 93 B.12 Libleak . 93 B.13 Mtrace . 93 B.14 Rmdebug . 94 B.15 Dr. Memory . 94 Appendix C Tables 95 Appendix D Graphs 119 7 CONTENTS 8 Chapter 1 Introduction “Since human beings themselves are not fully debugged yet, there will be bugs in your code no matter what you do.” — Chris Mason [67] Embedded systems are present in many objects we use every day. For convenience and safety, it is important to minimise the risk of these systems malfunctioning. One way an embedded system can malfunction is if there is an error in the software running on it. Some of these errors can be found with analysis tools, but since embedded systems often have limited memory and processing power, not all tools are able to run on them. We have evaluated different tools, to see if they can be used to find memory errors in an embedded environment. We have carried out our work at Volvo Cars and performed our evaluations on an embedded Linux system used for telecommunications in future cars. The final product of our evaluations is a suggestion of tools which we consider useful for finding memory errors in embedded software. In this chapter, we define the problem we want to solve and why it is important, present our research questions, list the limitations and delimitations we worked with, and describe how we divided the work. 9 1. Introduction 1.1 Problem Definition Finding errors related to how memory is used in software is important, since memory errors can cause malfunctions and crashes. There are many tools that can be used for finding these errors: some of them only find a specific subset of errors, and some of them find manydiffer- ent kinds. A few of these tools, especially the ones that find many types of errors, introduce large overheads on the programs they analyse, both in memory consumption and in execu- tion time. One such tool, which is arguably one of the more common memory analysis tools, is the Valgrind skin Memcheck. Since it is able to find multiple types of errors, Memcheck can be a useful asset during software development. However, the usefulness comes at a price; as described in Section 4.2.1, Memcheck imposes considerable overheads on the programs it analyses. Finding memory errors in embedded software is especially important, since embedded systems are often intended to run for extended periods of time without crashing. Many of these systems have limited amounts of memory and are not as powerful as desktop computers, which can make demanding analysis tools, such as Memcheck, difficult to use on them. This is not a problem if the software can be compiled for a more powerful computer. However, sometimes the software is dependent on the embedded system. For example, it might need some specific hardware component to be present. Because of this, it would be desirable to find other tools that are not just effective at finding errors, but are also efficient inhowmuch resource overhead they introduce.