Advanced Features for an Integrated Verification Environment Ruben Kälin [email protected] Master Thesis November 2016 https://bitbucket.org/viperproject/viper-ide/ Supervisor Arshavir Ter-Gabrielyan Abstract Currently, in software verification, the IDE support is insufficient, especially for symbolic exe- cution. Many verification technologies are being developed, but only few IDEs target software verification. For example, users of the Viper framework had only little assistance in writing, and no assistance in debugging their software so far. The common way of invoking the verifier is through the command line, as a sufficient IDE support has been missing. An IDE capable of handling more than one language is desirable, as there are many small verification languages. Without proper IDE support writing, verifying, and debugging software is cumbersome. Many of the tasks normally handled by an IDE need to be performed manually. A capable IDE so- lution could reduce the overhead of verifying software and thereby improve the programmer’s productivity. In this Master’s thesis, we built a cross-platform IDE for creating Viper programs. The IDE assists the user in the process of writing, verifying, and debugging Viper source code. This support ranges from automatically performing tasks at the right time to visualizing the symbolic states and counterexamples. We provide intuitive debugging support for traversing all states of symbolic execution and allow the user to visually compare states. The Viper IDE is based on Microsoft Visual Studio Code. We use the Viper toolchain as a verification engine. We provide a solution for symbolic execution, whereas the existing solutions focus on a different verification approach. We manage to visualize the internal state of the verification in a usable way. The implemented solution is highly configurable and allows for an easy integration of additional tools, for example, for specification inference. These novel features allow the user to explore failed verification attempts and thereby understand the cause of the problem. Moreover, the implemented solution generally improves the usability of the Viper framework. i ii Acknowledgment First and foremost, I would like to thank my thesis advisor Arshavir Ter-Gabrielyan of the Chair of programming methodology at ETH Zürich. He guided the project and provided invaluable feedback. His door was always open whenever I had a question, and he took the time for testing the system and proofreading the thesis. My thanks go to Prof. Dr. Peter Müller, head of the Chair of programming methodology at ETH Zürich for providing the big picture and valuable feedback. During the course of this project many people were involved in answering technical and con- ceptual questions and testing the system. I am grateful for Malte Schwerhoff’s help with all verification backend-related issues. He also assisted the project planning with insightful ideas. I would like to thank Dimitar Asenov for his support in taking the right decisions for the user interface of the project. My thanks go to Marco Eilers, Dr. Alex Summers, and all other testers for detecting and reporting issues. Finally, I want to express my gratitude to my family and friends, especially Amanda Jenny for their support and continuous encouragement throughout the thesis. Thank you for taking your time to proofread the thesis. Without you this thesis would not have been possible. iii iv Contents 1. Introduction1 1.1. Problem Statement................................1 1.2. Goals.......................................2 1.3. Previous Solution.................................3 1.4. Viper.......................................3 1.5. Abbreviations...................................5 1.6. Structure of this Document............................5 2. State-of-the-Art7 2.1. Verification Technologies.............................7 2.1.1. Satisfiability Modulo Theories Solver..................8 2.1.2. Verification Condition Generation....................8 2.1.3. Symbolic Execution...........................8 2.2. Integrated Development Environments for Verification.............8 2.2.1. Design Concepts for Debugging Symbolic Execution..........9 2.2.2. Dafny Integrated Development Environment..............9 2.3. Previous Solution.................................9 2.4. Text Editor Comparison..............................9 2.4.1. Sublime Text............................... 10 2.4.2. Atom................................... 11 2.4.3. Visual Studio Code............................ 11 3. Requirement Analysis and Overview of Planned Features 15 3.1. Pre-Verification Phase............................... 15 3.2. Pre-Verification Information Reporting...................... 16 3.3. Interaction with the SE Verification Backend................... 17 v Contents 3.4. Error Reporting.................................. 18 3.5. Supported Operating Systems........................... 19 4. Graphical User Interface 21 4.1. Basic GUI Features................................ 21 4.1.1. Syntax Highlighting........................... 22 4.1.2. Code Completion............................. 22 4.1.3. Automatic Verification Invocation.................... 23 4.2. Features for Debugging Symbolic Execution Faliures.............. 24 4.2.1. Marking Symbolic Execution States................... 25 4.2.2. Simplified Debugging Mode....................... 27 4.2.3. Advanced Debugging Mode....................... 29 4.2.4. State Visualization............................ 31 4.2.5. Comparing States............................. 36 4.3. Status Bar..................................... 37 4.3.1. Error Reporting.............................. 37 4.3.2. Progress Reporting............................ 39 4.4. Commands and Shortcuts............................. 41 5. Implementation 43 5.1. Infrastructure................................... 43 5.1.1. Language Client............................. 44 5.1.2. Language Server............................. 45 5.1.3. Debugger................................. 46 5.1.4. External Tools.............................. 46 5.1.5. Third-Party Tools............................. 47 5.2. The Viper Protocol................................ 47 5.2.1. Client-Server Communication...................... 47 5.2.2. Communication of the Debugger with Client and Server........ 50 5.2.3. Communication between Backends and the IDE............. 51 5.2.4. Control Flow of the Viper IDE...................... 54 5.3. Configuration................................... 56 5.3.1. Configuration Validation......................... 56 5.3.2. Path Resolution.............................. 57 5.3.3. Verification Backend Settings...................... 59 5.3.4. Extensibility............................... 59 5.4. Logging...................................... 61 5.5. Adaptation of Tools................................ 62 5.5.1. Output for Progress Reporting...................... 62 5.5.2. Outputting SE States........................... 63 5.5.3. Outputting the Counterexample..................... 64 5.5.4. Backwards Compatibility......................... 64 5.5.5. Validation of Changes.......................... 65 5.6. Corner Cases................................... 65 5.6.1. Opening Individual Files Without an Opened Workspace........ 65 5.6.2. State Markers............................... 66 vi Contents 5.6.3. Installation................................ 66 5.6.4. Platform-Independent Path Configuration................ 67 5.6.5. Output Buffer Size............................ 68 5.7. Syntax Highlighting................................ 68 5.8. Automatic Code Formatting........................... 68 5.9. Time Management of the Viper IDE....................... 69 5.10. The Shutdown Sequence of Viper IDE...................... 69 6. Evaluation 71 6.1. Achieved Goals.................................. 71 6.1.1. Core Features............................... 71 6.1.2. Extension Features............................ 72 6.1.3. Unplanned Features........................... 73 6.1.4. Extensibility............................... 74 6.2. Comparison of Visual Features with Design Concepts.............. 74 6.2.1. Design Principles............................. 75 6.2.2. Comparison of the Debug Controls................... 76 6.3. Comparison to Dafny IDE............................ 76 6.3.1. State Markers............................... 76 6.3.2. Error List................................. 80 6.3.3. State Visualization............................ 80 6.3.4. Comparison of Progress Reporting.................... 80 6.4. Performance Benchmark of the Adopted Backend................ 81 6.5. Test Coverage................................... 82 6.6. Limitations.................................... 82 7. Conclusion and Future Work 85 7.1. Future Work.................................... 86 List of Figures 89 List of Tables 91 Bibliography 94 A. Infrastructure 95 B. Debug Launch Configuration 97 C. List.vpr 99 D. Temporary Files 101 E. Default Configuration 103 F. Output Messages 107 vii Contents viii 1 Introduction Tools for verifying program correctness are becoming more and more powerful and widely used. However, it is still difficult for a non-expert in program verification to debug verification errors. Current verification environments provide helpful support for verifying properties of software, such as syntax highlighting, auto-completion, and visualization of syntax errors, but lack assistance for fixing errors in proofs. 1.1. Problem Statement Modern verification tools are complicated. In case of a verification failure, the warnings and error