Fuzzing analysis: Evaluation of properties for developing a feedback driven fuzzer tool Master's thesis Kris Gundersen 22/4 - 2014 22.04.2014 Table of Contents 1 Table of Contents 1 Table of Contents ................................................................................................................ 2 2 Abstract ............................................................................................................................... 5 3 Foreword ............................................................................................................................. 7 3.1 Acknowledgments ....................................................................................................... 7 4 Introduction ........................................................................................................................ 8 4.1 Software security and quality assurance ..................................................................... 8 4.2 Fuzzing and thesis theme introduction ....................................................................... 8 4.3 Overview, Plan and research question(s) .................................................................. 10 4.3.1 Nature of the thesis ............................................................................................ 10 4.3.2 Thesis structure .................................................................................................. 11 4.3.3 Research question(s) .......................................................................................... 11 5 Theory ............................................................................................................................... 12 5.1 Software security and quality assurance ................................................................... 12 5.2 Fuzzing ....................................................................................................................... 13 5.2.1 Concepts and purpose ........................................................................................ 13 5.2.2 Techniques ......................................................................................................... 14 5.2.3 History ................................................................................................................ 16 5.2.4 Available tools/frameworks ............................................................................... 18 5.2.5 Evolutionary fuzzers ........................................................................................... 20 5.2.6 Limitations .......................................................................................................... 21 5.3 Instrumentation ......................................................................................................... 23 5.3.1 Concepts ............................................................................................................. 23 5.3.2 Tools ................................................................................................................... 24 5.4 Minimal set ................................................................................................................ 25 6 Presentation ...................................................................................................................... 26 6.1 Target programs ........................................................................................................ 26 6.1.1 SumatraPDF ........................................................................................................ 26 6.1.2 xPDF .................................................................................................................... 27 6.2 Metrics ....................................................................................................................... 27 6.2.1 Code Coverage ................................................................................................... 28 6.2.2 Time .................................................................................................................... 29 Kris Gundersen Page 2 of 107 22.04.2014 Table of Contents 6.2.3 Unique Crashes................................................................................................... 29 6.3 Tools to be used......................................................................................................... 29 6.3.1 Fuzzing ................................................................................................................ 29 6.3.2 Instrumentation ................................................................................................. 30 6.3.3 BinNavi and IDA .................................................................................................. 31 6.3.4 !exploitable ......................................................................................................... 31 6.3.5 Immunity Debugger ........................................................................................... 32 6.3.6 010 Editor ........................................................................................................... 32 6.4 Working environment ............................................................................................... 32 6.5 Tool interaction- and program flow overview .......................................................... 33 6.5.1 Peach operation flow ......................................................................................... 33 6.5.2 Getting trace and code coverage information operation flow .......................... 34 6.6 Assumptions and decisions ....................................................................................... 36 6.6.1 Thesis division .................................................................................................... 36 6.6.2 ASLR .................................................................................................................... 36 6.6.3 !exploitable and it’s categorizing of faults ......................................................... 36 6.6.4 Iteration count.................................................................................................... 36 6.6.5 Unique crashes ................................................................................................... 36 7 The work process, results and analysis ............................................................................. 39 7.1 Code Coverage ........................................................................................................... 42 7.1.1 Work process ...................................................................................................... 42 7.1.1.1 Fuzzing: producing crashes ......................................................................... 43 7.1.1.2 Extracting the trace ..................................................................................... 44 7.1.1.3 Modifying the trace..................................................................................... 44 7.1.1.4 Finding the number of hops between the location and the trace ............. 45 7.1.1.5 Locating the branch-off location ................................................................. 47 7.1.2 Results ................................................................................................................ 47 7.1.3 Analysis ............................................................................................................... 53 7.2 Pre-fuzz ...................................................................................................................... 55 7.2.1 Work Process ...................................................................................................... 55 7.2.1.1 Investigating the impact of various features .............................................. 56 7.2.1.2 Comparing samples to find key areas to focus on ...................................... 58 Kris Gundersen Page 3 of 107 22.04.2014 Table of Contents 7.2.1.3 Generating samples .................................................................................... 60 7.2.2 Results ................................................................................................................ 61 7.2.3 Analysis ............................................................................................................... 66 7.3 Fuzzing ....................................................................................................................... 69 7.3.1 Work Process ...................................................................................................... 69 7.3.1.1 Getting baselines for the required time ..................................................... 69 7.3.1.2 Identifying the three most interesting samples ......................................... 71 7.3.1.3 Developing a smart-fuzz tool ...................................................................... 72 7.3.2 Results ................................................................................................................ 75 7.3.3 Analysis ............................................................................................................... 82 8 Conclusion and further work ............................................................................................ 87 8.1 Code Coverage ........................................................................................................... 87 8.2 Pre-fuzz .....................................................................................................................