Model of Agile Software Measurement: A Case Study Master of Science Thesis in the Programme Software engineering and Technology JOHAN GUSTAFSSON Chalmers University of Technology University of Gothenburg Department of Computer Science and Engineering Göteborg, Sweden, June 2011 The Author grants to Chalmers University of Technology and University of Gothenburg the non-exclusive right to publish the Work electronically and in a non-commercial purpose make it accessible on the Internet. The Author warrants that he/she is the author to the Work, and warrants that the Work does not contain text, pictures or other material that violates copyright law. The Author shall, when transferring the rights of the Work to a third party (for example a publisher or a company), acknowledge the third party about this agreement. If the Author has signed a copyright agreement with a third party regarding the Work, the Author warrants hereby that he/she has obtained any necessary permission from this third party to let Chalmers University of Technology and University of Gothenburg store the Work electronically and make it accessible on the Internet. Model of Agile Software Measurement: A Case Study JOHAN GUSTAFSSON © JOHAN GUSTAFSSON, June 2011. Supervisor: ROBERT FELDT Chalmers University of Technology University of Gothenburg Department of Computer Science and Engineering SE-412 96 Göteborg Sweden Telephone + 46 (0)31-772 1000 Department of Computer Science and Engineering Göteborg,Sweden 1 Model of Agile Software Measurement: A Case Study Johan Gustafsson Computer Science and Engineering. Chalmers University of Technology Gothenburg, Sweden [email protected] Abstract— This master thesis examines what type of What companies need is a quick, flexible and cost efficient measurement should be conducted in agile software development. The paradigm shift that agile methodology offers is in conflict way to measure. with traditional measurement approaches and there is a need to The Adaptable Measurement Framework clarify if and how measurement can benefit the agile practice. In (AMF) particularly focuses on these aspects and is tested in this paper we develop a model that addresses both performance and process optimization measurement in agile processes. The this research project [6]. The framework brings the principles model is evaluated at bwin Games AB, a web game software of agile into the measurement practice, providing a way to company based in Sweden. As a part of the evaluation, an agile develop flexible metrics to a low cost and just-in-time. inspired measurement framework for implementation of cost- Combining the agile processes with the lean methodology is efficient and flexible metrics is successfully tested. The study shows that measurement can be beneficial for the agile practice, becoming increasingly popular in attempts to enhance the but that careful consideration to dysfunctional behavior is agile methods [7]. In the past it has primarily been done by necessary. removing waste. More recently the focus has changed to optimizing the flow of the value chain. Particularly, the trend Index Terms— Measurement, KPI, Metrics, Agile software in agile development applied in a large scale, is to adopt lean development, Lean software development, Process improvement software development’s end-to-end perspective [8]. In order to account for this trend, it is important to consider the lean I. INTRODUCTION principles in the recommendations about agile software measurement. UANTIFIABLE measurement has been described as the Qmajor weakness of agile software development, while others To study the introduction of a measurement system in agile claim that measurement is an integral part of the agile practice software development, a case study is conducted at bwin [1]. In today’s software development, traditional life cycle Games. bwin Games is a poker network operator in a market processes such as the V model, are still predominant. leading position. The company also has its own development However, the adaption rate of the agile methodology is where their next generation of poker products is currently increasing rapidly [2]. In any case, many organizations are developed using agile methods. bwin Games is a part of the looking for ways to let Key Performance Indicators (KPIs) bwin.party group and has recently received its own profit-and- measure the performance of the agile software development loss responsibility. [3]. To facilitate this transformation, they want to introduce Traditional measurement does not account for the agile agile software measurement. Some KPIs are already culture, and it often leads to undesirable effects [4]. Resulting implemented. However, bwin Games are not sure if the KPIs in a need for a measurement approach that affirms and perform its function and measure the right things, nor are there reinforces the principles that agile is based upon. any measurements in the improvement efforts. Besides performance measurement, there is a demand for The purpose of this thesis is to reveal the best practices of measurement in the continuous improvement process [3, 5]. agile software measurement and apply parts of it at bwin These types of metrics have a different character than the Games. The goal is to understand how performance KPIs. They diagnose specific problems and are intended for measurement should be conducted in agile software occasional use. development and how measurement can facilitate the process improvement effort. 2 The intended audience of this paper can be divided into In order facilitate correct understanding of different types three categories. The first category is managers and decision of measurement the measurement practice should: makers in agile software development. The second category is Communicate the definition of the different types of those who have an interest in process improvement. The last metrics. category is those involved in the coordination of the set of Communicate the difference between performance and measurements performed in an organization, known as process optimization measurement. measurement systems. Understand how the KPIs and diagnostics complement The main research question is: each other. Table 1 Best practice checklist for facilitating the correct understanding of What type of performance and process optimization different types of measurement. Based on the sources in this section. measurement should be performed in agile software development? B. Industrial Context bwin Games is in a transition phase and the whole bwin The report is structured as follows: Section II is a summary group has merged with another large online gaming company. of relevant theories. In Section III the research methodology is The merge has introduced changes and uncertainty in the described. Section IV presents the results and Section V organization. In addition, the technology department discusses the result’s validity. Finally, Section VI presents the performed a major reorganization with new management. conclusions. These factors made the thesis’ decision process slow and ambiguous. II. BACKGROUND The thesis’ theoretical validation and strategic This section contains a brief description of relevant theories recommendations are mainly done in collaboration with the and practices associated with agile measurement. To get a technology department’s process manager. The empirical common understanding of key concepts, some definitions are work is conducted at the unit that develops the poker client. first given. The unit follows the Scrum methodology and has three scrum A. Definitions teams composed of programmers, quality analysts and two Hartmann and Dymond [3] propose a division between scrum masters. long-term organizational performance measurement and C. Agile Software Development measurement that diagnose locally to achieve process Agile software development has emerged from the recent improvement. years’ changes in the nature of software development [12]. The current software market is very volatile and requires Definition: Metric flexible development methods. Traditional pre-defined and “A measure or a combination of measures for quantitatively plan-driven methods such as the waterfall model [13], often assessing, controlling or improving a process, a product, a fail to meet today’s dynamic market needs. Agile software team”[9] development recognizes that it is difficult for the customer to know all the requirements upfront and instead takes an Definition: Key Performance Indicator empirical approach. “Quantifiable metrics which reflect the performance of an organization in achieving its goals and objectives”[10] The methods included in the agile software development such as Scrum [14] and Extreme Programming [15], have a Definition: Diagnostic common basis in the Agile Manifesto and its principles[16]. “A metric used to diagnose and improve the processes that produce business value. What it measures might not have a The Agile Manifesto states [16]: direct value to the customer.” [11] ―We are uncovering better ways of developing software by doing it and helping others do it. Through this work we have The general term metric covers all types of measurement come to value: statistics, including both KPIs and diagnostics. KPIs are used in the long-term performance measurement and measure the Individuals and interactions over processes and tools value produced. Working software over comprehensive documentation Customer collaboration over contract negotiation Diagnostics are used in the