A Game-Theoretic Approach to Software Process Improvement Carlos Gavidia-Calderon A dissertation submitted in fulfillment of the requirements for the degree of Doctor of Philosophy of University College London Department of Computer Science University College London 15 April 2020 Declarations I, Carlos Gavidia-Calderon, confirm that the work presented in this thesis is my own. Where information has been derived from other sources, I confirm that this has been indicated in the thesis. Parts of this document are based on the following publications: 1. Chapters 2 and 3 are based on: Carlos Gavidia-Calderon, Emmanuel Letier and Earl T. Barr. A Survey of Game Theory Applied to Software Engineering. In progress. 2. Chapter 4 is based on: Carlos Gavidia-Calderon, Federica Sarro, Mark Harman and Earl T. Barr. Game-Theoretic Analysis of Development Practices: Challenges and Opportunities. Journal of Systems and Soft- ware. 2019. 3. Chapter 5 and 6 are based on: Carlos Gavidia-Calderon, Federica Sarro, Mark Harman and Earl T. Barr. The Assessor’s Dilemma: Improving Bug Repair via Empirical Game Theory. IEEE Transactions on Software Engineering. 2019. This research project is a collaborative enterprise. While I led this work, I did it in close collaboration with my supervisors. I use “I” throughout this thesis in this sense. Date Signature 2 Abstract There is a plethora of software development practices. Practice adoption by a development team is a challenge by itself. This makes software process improvement very hard for organisations. I believe a key factor in successful practice adoption is proper incentives. Wrong incentives can lead a process improvement effort to failure. I propose to address this problem using game-theory. Game theory studies cooperation and conflict. I believe its use can speed the development of effective software processes. I surveyed game-theory applications to software engineering problems, showing the potential of this technique. By using game-theoretic models of software development practices, we can verify if the behaviour at equilibrium converges towards team cooperation. Modern software development is performed by large teams, working multi- ple iterations over long periods of time. Classic game representations do not scale well to model such scenarios, so abstraction is needed. In this thesis, I propose GTPI (game-theoretic process improvement), a software process improvement approach based on empirical game-theoretic analysis (EGTA) abstractions. EGTA enables the production of software process models of manageable size. I use GTPI to address technical debt, modelling developers that prefer quick and cheap solutions instead of high-quality time-consuming fixes. I have also approached bug prioritisation with GTPI, proposing the assessor-throttling prioritisation process, and developing a tool to support its adoption. 3 Impact Statement Software is part of many aspects of modern life: from trivial things like ordering takeaway to vital tasks like aeroplane navigation. Software quality does not only depend on the technical aspects of engineering, but also on the process that drives how people build it. In the words of Linus Torvalds, ”All the really stressful times for me have been about process: they haven’t been about code”. In this thesis, I propose game theory as a formal approach for the analysis and design of software processes. Game theory is widely adopted in fields like economics, biology and cybersecurity. I bring game theory to software process design, using it to identify and remove incentive problems. I used game theory to resolve real problems affecting software teams in industry. Both technical debt and priority inflation are still relevant problems in software development. I recommended actionable process interventions for their removal. I used game-theoretic models to confirm the effectiveness of established processes — like code reviews — and to challenge the adequacy of others — like bug triage teams. Finally, I have established the foundations of a new exciting research field. I believe that technical debt and priority inflation are two of the many in- centive problems that affect software processes. In this thesis, I developed a framework to address these problems with game-theoretic models. I expect that both practitioners and researchers adopt this novel approach of software process improvement. 4 Acknowledgements First, I would like to thank my supervisory team at UCL: Dr Earl Barr, Dr Federica Sarro and Prof. Mark Harman. And Paul Baker, my mentor during my time at Visa Europe. The path from practitioner to academic is not an easy one, and it was your guidance that made it possible. I would also like to thank my fellow PhD students: DongGyun Han, Matheus Paixao, Chaiyong Ragkhitwetsagul and Bobby Bruce. The PhD was definitely more enjoyable by sharing it with you. And also my friends at Ifor Evans Hall: Hillary Ingram, Silvia Schmidt, Lauren Coyne, Vibhav Mishra, Razvan Marinescu, Robbie Wilson and Mariflor Vega. Ifor Evans was my very first home-away-from-home, thanks to your friendship. I also like to thank Dr Angel Gavidia and Lilia Calderon, my parents, and Dr Jorge Gavidia and Mario Gavidia, my brothers. Leaving Peru to pursue my academic dream was only possible with your help. It is an honour to become the third “Doctor Gavidia”. Finally, I would like to especially thank Giovani Guizzo. In the good times, you were always up for a pint; and in the bad ones, I could always count on your support. And to Ireen Islam. I came to this country for a Doctorate, and I ended up finding you. For me, that is even more valuable. 5 Contents 1 Introduction 14 1.1 Research Problem.......................... 15 1.2 Objectives.............................. 17 1.3 Contributions............................ 17 1.4 Thesis Outline............................ 18 2 Background 20 2.1 Concepts and Terminology.................... 20 2.2 Normal-form Games........................ 22 2.3 Extensive-form Games....................... 23 2.4 Multistage Games.......................... 26 2.5 Bayesian Games........................... 27 2.6 Mechanism Design......................... 28 2.7 Cooperative Game-Theory..................... 30 3 Literature Review 32 3.1 Methodology............................ 33 3.2 Game Models in Software Engineering.............. 35 3.2.1 Software Requirements.................. 36 3.2.2 Software Design...................... 37 3.2.3 Software Construction................... 37 3.2.4 Software Testing...................... 38 3.2.5 Software Maintenance................... 39 3.2.6 Software Engineering Management........... 41 3.2.7 Software Engineering Professional Practice....... 43 3.3 Challenges and Opportunities................... 44 3.3.1 The Need for Game Abstractions............. 44 3.3.2 Beyond the Rationality Assumptions.......... 48 6 Contents 4 GTPI: A Game-Theoretic Approach to Process Improvement 50 4.1 Motivating Example........................ 52 4.2 An Introduction to GTPI...................... 54 4.2.1 Software Process Modelling................ 54 4.2.2 Software Process Improvement.............. 59 4.3 Practical considerations...................... 61 4.3.1 Data Gathering....................... 62 4.3.2 Technical Validation.................... 63 4.3.3 Securing Acceptance.................... 64 4.4 Related Work............................ 65 5 TaskAssessor: A Game-Theoretic Model of Priority Inflation 67 5.1 The Assessor’s Dilemma...................... 69 5.2 Identifying the Process Anomaly................. 71 5.2.1 Shared Prioritisation Tooling Adoption......... 72 5.2.2 The Cost of Priority Inflation............... 73 5.3 Empirical Game Design...................... 77 5.3.1 Bug Repair and Issue Resolution Corpus........ 78 5.3.2 Game Models with TaskAssessor .............. 80 5.3.3 TaskAssessor under Twins and EGTA........... 82 5.3.4 Validating TaskAssessor ................... 85 5.3.5 Threats to Validity..................... 88 5.3.6 Using TaskAssessor ..................... 89 6 Assessor-Throttling: A Novel Task Prioritisation Process 91 6.1 Empirical Game Improvement.................. 93 6.1.1 Distributed Bug Prioritisation............... 94 6.1.2 Do Gatekeepers Prevent Priority Inflation?....... 95 6.1.3 The Assessor-Throttling Process............. 98 6.2 Process Deployment........................ 106 7 Conclusion and Future Work 111 7.1 Summary of Contributions..................... 111 7.2 Future Work............................. 112 7.3 Final Remarks............................ 113 Bibliography 114 7 List of Figures 2.1 Normal-form representation of a two-player game (DEV1 and DEV2), where each player has two actions (cooperate and oppose). At equilibrium, both players adopt the opposing action with a probability of 100%.................. 21 3.1 Scope of this review with respect to the SWEBOK guide [1]. For this literature review, we only consider papers focussing on practice core knowledge areas.................... 34 3.2 EGTA-abstracted game for a 2-player-3-round rock-paper- scissors game: each round victory is rewarded with 1 point and draws give no points to either player............. 46 4.1 Extensive form game for a multi-project freelancer’s dilemma: this figure contains the part of the tree corresponding to the first two projects. Nodes in the tree correspond to players (Freelancer A and Freelance B) and edges to actions (C for cooperation, and NC for no cooperation) The size of this rep- resentation grows with the number of projects, freelancers and actions available