Software Performance Engineering Using Virtual Time Program Execution
Total Page：16
File Type：pdf, Size：1020Kb
View metadata, citation and similar papers at core.ac.uk brought to you by CORE provided by Spiral - Imperial College Digital Repository IMPERIAL COLLEGE LONDON DEPARTMENT OF COMPUTING Software Performance Engineering using Virtual Time Program Execution Nikolaos Baltas Submitted in part fullment of the requirements for the degree of Doctor of Philosophy in Computing of Imperial College London and the Diploma of Imperial College London To my mother Dimitra for her endless support The copyright of this thesis rests with the author and is made available under a Creative Commons Attribution Non-Commercial No Derivatives licence. Re- searchers are free to copy, distribute or transmit the thesis on the condition that they attribute it, that they do not use it for commercial purposes and that they do not alter, transform or build upon it. For any reuse or redistribution, researchers must make clear to others the licence terms of this work. 4 Abstract In this thesis we introduce a novel approach to software performance engineering that is based on the execution of code in virtual time. Virtual time execution models the timing-behaviour of unmodified applications by scaling observed method times or replacing them with results acquired from performance model simulation. This facilitates the investigation of \what-if" performance predictions of applications comprising an arbitrary combination of real code and performance models. The ability to analyse code and models in a single framework enables performance testing throughout the software lifecycle, without the need to to extract perfor- mance models from code. This is accomplished by forcing thread scheduling decisions to take into account the hypothetical time-scaling or model-based performance specifications of each method. The virtual time execution of I/O operations or multicore targets is also investigated. We explore these ideas using a Virtual EXecution (VEX) framework, which provides perfor- mance predictions for multi-threaded applications. The language-independent VEX core is driven by an instrumentation layer that notifies it of thread state changes and method profiling events; it is then up to VEX to control the progress of application threads in virtual time on top of the operating system scheduler. We also describe a Java Instrumentation Environment (JINE), demonstrating the challenges involved in virtual time execution at the JVM level. We evaluate the VEX/JINE tools by executing client-side Java benchmarks in virtual time and identifying the causes of deviations from observed real times. Our results show that VEX and JINE transparently provide predictions for the response time of unmodified applications with typically good accuracy (within 5-10%) and low simulation overheads (25-50% additional time). We conclude this thesis with a case study that shows how models and code can be integrated, thus illustrating our vision on how virtual time execution can support performance testing throughout the software lifecycle. Acknowledgements I would like to express my sincerest gratitude to my supervisor, Tony Field, whose knowledge, perceptiveness and positive attitude has supported, guided and inspired me throughout the course of this thesis. He has been a remarkable tutor and working with him has been an absolute privilege. I would also like to thank Paul Kelly for presenting to me the opportunity to work on this project and Alexander Wolf for bringing me in contact with the wider performance modelling community in a Dagstuhl seminar. I am grateful to my examiners Stephen Jarvis and Will Knottenbelt for their insightful comments and suggestions that have significantly improved the quality of this thesis. I would like to particularly thank Andrew Cheadle for all the technical support and research guidance that he offered in the early stages of the project, as well as for helping us envision how our approach could be used in performance engineering. I would like to thank all the students who have been involved with the virtual time execution project: Richard Bounds, Damola Adeagbo, Andreas Matsikaris, James Greenhaulgh and Cyrus Lyons. They comprise the community that has used the virtual time execution framework and their feedback has contributed to the further improvement of the tool. I am especially grateful to my office colleagues and friends Wolfram Wiesemann, Phoebe Vayanos, Angelos Georgiou, Raquel Fonseca and Adil Hussain for all the fun, interesting and stimulating discussions we have held over the years and for the great experience of sharing my time at the college with them. Finally, I would like to thank my family and friends for being there for me in the better and tougher times of the research effort. I would also like to express my deepest gratitude to Delia for standing by me throughout this journey; for her constant encouragement and unconditional support. 6 Contents 1 Introduction 25 1.1 The idea . 26 1.2 Contributions . 27 1.3 Publications and Statement of Originality . 29 2 Background 30 2.1 Software Performance Engineering (SPE) . 30 2.1.1 Model-based SPE . 32 2.1.2 Measurements . 36 2.2 Simulation . 41 2.2.1 Execution-driven simulation . 42 2.2.2 Network emulation . 50 2.2.3 Real time simulation . 52 2.2.4 Controlling program execution in Virtual Time . 53 2.3 Java Virtual Machine overview . 55 2.3.1 Java . 55 2.3.2 JNI . 55 7 8 CONTENTS 2.3.3 JVMTI . 55 2.3.4 Java instrumentation . 56 2.4 Pthreads . 56 2.5 I/O . 57 2.5.1 I/O in Linux . 57 2.5.2 I/O in execution-driven simulation . 58 2.5.3 Prediction methods . 60 3 Virtual Time Execution 62 3.1 Scheduler . 66 3.1.1 VEX states . 67 3.1.2 Controlling threads in virtual time . 69 3.1.3 Implementation issues . 71 3.2 Profiler . 75 3.2.1 Measuring virtual time . 76 3.2.2 Logging method times . 78 3.2.3 Compensating for observer effects . 79 3.2.4 Simulation output . 82 3.3 Evaluation . 87 3.3.1 Validation . 87 3.3.2 Overhead . 90 3.3.3 Limitations . 93 CONTENTS 9 4 I/O in virtual time 95 4.1 I/O in VEX . 97 4.1.1 Requirements . 97 4.1.2 Methodology . 98 4.1.3 Misprediction handling . 103 4.1.4 I/O scaling . 105 4.1.5 I/O exclusion . 107 4.2 Empirical I/O prediction parameters . 108 4.2.1 Measurement aggregation . 109 4.2.2 Prediction methods . 110 4.3 Evaluation . 113 4.3.1 Benchmarks . 114 4.3.2 Optimal configuration investigation . 117 4.3.3 Investigating I/O effects . 123 4.3.4 Database prediction . 125 5 Java Instrumentation Environment 131 5.1 Design and implementation . 132 5.1.1 System initialisation and termination . 132 5.1.2 Thread monitoring . 134 5.1.3 Method handling . 142 5.1.4 Instrumentation techniques . 144 5.1.5 Usage . 146 10 CONTENTS 5.1.6 Summary . 146 5.2 Challenges in Java virtual time execution . 148 5.2.1 Implementation note: Deadlocks in JVM . 148 5.2.2 Native waiting threads . 150 5.2.3 Safepoint synchronisation . 153 5.2.4 Leaps forward in virtual time . 154 5.2.5 Heuristics for countering premature VLFs . 157 5.2.6 Java system threads . 162 5.2.7 JIT compilation . 163 5.2.8 Garbage collection . 164 6 Results 166 6.1 M/M/1 simulation . 166 6.2 JGF single-threaded . 168 6.3 JGF multi-threaded . 171 6.3.1 Basic results . 171 6.3.2 Thread and time scaling . 174 6.4 SPECjvm2008 . 176 6.4.1 Single-threaded . 176 6.4.2 Multithreaded . 178 7 Multicore VEX 191 7.1 Methodology . 194 7.1.1 Multiple core simulation . 194 CONTENTS 11 7.1.2 Synchronisation . 195 7.2 Extensions for VEX thread handling . 203 7.2.1 I/O . 203 7.2.2 Native waiting . 204 7.3 Evaluation . 205 7.3.1 JGF investigation of scaling . 205 7.3.2 SPEC investigation ..