JUL 2 0 2004 LIBRARIES BARKER Securing Software: an Evaluation of Static Source Code Analyzers by Misha Zitser
Total Page:16
File Type:pdf, Size:1020Kb
Securing Software: An Evaluation of Static Source Code Analyzers by Misha Zitser Submitted to the Department of Electrical Engineering and Computer Science in Partial Fulfillment of the Requirements for the Degrees of Bachelor of Science in Electrical [Computer] Science and Engineering and Master of Engineering in Electrical Engineering and Computer Science at the Massachusetts Institute of Technology August 29, 2003 Copyright 2003 Misha Zitser. All rights reserved. The author hereby grants to M.I.T. permission to reproduce and distribute publicly paper and electronic copies of this thesis and to grant others the right to do so. Author Department of Electrical Engineering and Computer Science A August 29, 2003 Certified by Richard Lippmann 'Thgfs gupervisor Accepted by Arthur C. Smith Chairman, Department Committee on Graduate Theses MASSACHJSET-Ts INS E OF TECHNOLOGy JUL 2 0 2004 LIBRARIES BARKER Securing Software: An Evaluation of Static Source Code Analyzers by Misha Zitser Submitted to the Department of Electrical Engineering and Computer Science August 29, 2003 In Partial Fulfillment of the Requirements for the Degree of Bachelor of Science in Computer Science and Engineering and Master of Engineering in Electrical Engineering and Computer Science ABSTRACT This thesis evaluated five static analysis tools - Polyspace C Verifier, ARCHER, BOON, Splint, and UNO - using 14 code examples that illustrated actual buffer overflow vulnerabilities found in various versions of Sendmail, BIND, and WU-FTPD. Each code example included a "BAD" case with one or more buffer overflow vulnerabilities and a "PATCHED" case without buffer overflows. The buffer overflows varied and included stack, heap, bss and data buffers; access above and below buffer bounds; access using pointers, indices, and functions; and scope differences between buffer creation and use. Detection rates for the "BAD" examples were low except for Splint and Polyspace C Verifier, which had average detection rates of 57% and 87% respectively. However, average false alarm rates, as measured using the "PATCHED" programs, were high for these two systems. The frequency of false alarms per lines of code was high for both of these tools; Splint gave on average one false alarm per 50 lines of code, and PolySpace gave on average one false alarm per 10 lines of code. This result shows that current approaches can detect buffer overflows, but that false alarm rates need to be lowered substantially. Thesis Supervisor: Richard Lippmann Title: Senior Scientist, MIT Lincoln Laboratory 2 Acknowledgements I would like thank my advisor, Rich Lippmann, for his guidance and suggestions during the whole process. I am also very grateful for the time he spent in proofreading the thesis and in offering constructive criticism. I would also like to thank all the people at Lincoln Lab whom I've had many interesting discussions with on the subject of static code analysis, including Robert Cunnigham, Tim Leek, Roger Khazan, Kendra Kratkiewicz, and Jesse Rabek. I specifically would like to thank Rob and Tim - Rob for suggesting that I look at the off-by-one vulnerability in wu-ftpd, and Tim for all his help in setting up the tools, for his help in writing the small test cases and his help in analyzing the results. In addition, I would like to thank David Evans for his help with Splint, David Wagner for answering questions about BOON, Michael Howard for pointing out the availability of PREfast, Yichen Xie and Dawson Engler for their help with ARCHER, and Chris Hote and Vince Hopson for all their help on answering questions about PolySpace. I would also like to thank Doug Stetson for being a great office-mate, and fellow students, Dave Messing and Nick Malyska, for being cheerful neighbors. Last but not least, I would like to thank my parents and my sister, Katrina, for providing me with plenty of moral support. I sincerely apologize to anyone whom I might have forgotten to thank. 3 Table of Contents CH A PTER 1 IN TR O D U C TION ................................................................................................................ 6 1.0 W HY IS SOFTW ARE SECURITY IM PORTANT?....................................................................................6 1.1 TYPES OF SOFTW ARE BUGS..................................................................................................................6 1.2 W HY FOCUS ON BUFFER OVERFLOW S?........................................................................................... 7 1.3 A BASIC OVERVIEW OF BUFFER OVERFLOW S ...................................................................................... 8 1.4 W HY ARE BUFFER OVERFLOW S SO COM M ON?.................................. ............................................... 9 1.5 W HAT'S BEING DONE TO STOP BUFFER OVERFLOW S?.................................................................. 11 1.6 G OALS OF THIS THESIS.......................................................................................................................12 CHAPTER 2 TYPES OF BUFFER OVERFLOW ATTACKS ............................................................ 14 2.0 H OW DO BUFFER OVERFLOW S W ORK?...................................... ...................................................... 14 2.1 O VERW RITING CODE POINTERS........................................................................................................18 2.2 LOGIC-BASED A TTACKS..................................................................................................................... 19 2.3 A TTACKER INJECTS CODE.................................................................................................................19 2.4 ATTACKER U SES EXISTING CODE ................................................................................................. 20 2.5 H EAP-BASED BUFFER O VERFLOW S...................................................................................................21 2.6 IS THE BUFFER O VERFLOW EXPLOITABLE?.....................................................................................21 CHAPTER 3 APPROACHES TO DETECTING/PREVENTING BUFFER OVERFLOWS...........23 3.0 TYPES OF SOFTW ARE SECURITY TOOLS ...................................................................................... 23 3.0.0 Dynam ic Testing Tools .............................................................................................................. 23 3.0.1 Other Dynam ic A nalysis Approaches .................................................................................... 24 3.0.2 Compiler-basedDynam ic Prevention Tools.......................................................................... 26 3.0.3 Language-basedApproach ................................................................................................... 29 3.0.4 Static/Dynamic Hybrids ....................................................................................................... 31 3.0.5 OperatingSystem Solutions.......................................................................................................33 3.0.6 Static Source Code Analyzers ............................................................................................... 34 3.1 W HY FOCUS ON STATIC ANALYSIS?............................................. ............................................... 37 CHAPTER 4 DESCRIPTION OF THE STATIC ANALYSIS TOOLS .......................................... 40 4.0 SPECIFIC TOOLS................................................................................................................................40 4.1 LEXICAL ANALYSIS .......................................................................................................................... 40 4.1.0 Flawfinder 1.22 ........................................................................................................................ 41 4.1.1 ITS4..........................................................................................................................................42 4.1.2 RA T S 2.1 .................................................................................................................................. 43 4.2 SEM ANTIC A NALYSIS TOOLS ........................................................................................................... 44 4.2.0 What is abstractinterpretation? .......................................................................................... 44 4.2.1 Splint (Secure Programm ing Lint)...................................................................................... 46 4.2.2 BO ON ....................................................................................................................................... 48 4.2.3 AR CHER (A rray CHeckER) ............................................................................................... 50 4.2.4 PREfix - not included in evaluation .................................................................................... 52 4.2.5 PREfast - not included in evaluation ................................................................................. 54 4.2.6 PolySpace C Verifier ................................................................................................................ 54 4.2.7 Uno ........................................................................................................................................... 55 4.2.8 M C (Meta Compiler) - Not included in evaluation ............................................................. 56 4.3