Measuring Process Flow Using Metrics in Agile Software Development a Systematic Literature Review and a Case Study

Total Page:16

File Type:pdf, Size:1020Kb

Measuring Process Flow Using Metrics in Agile Software Development a Systematic Literature Review and a Case Study Master of Science in Software Engineering June 2019 Measuring Process Flow using Metrics in Agile Software Development A Systematic Literature Review and a Case Study Krishna Sai Bitla Sairam Sagar Veesamsetty Faculty of Computing, Blekinge Institute of Technology, 371 79 Karlskrona, Sweden This thesis is submitted to the Faculty of Computing at Blekinge Institute of Technology in partial fulfilment of the requirements for the degree of Master of Science in Software Engineering. The thesis is equivalent to 20 weeks of full-time studies. The authors declare that they are the sole authors of this thesis and that they have not used any sources other than those listed in the bibliography and identified as references. They further declare that they have not submitted this thesis at any other institution to obtain a degree. Contact Information: Author(s): Krishna Sai Bitla E-mail: [email protected] Sairam Sagar Veesamsetty E-mail: [email protected] University advisor: DR. Krzysztof Wnuk Department of Software Engineering Industry Advisor: Dani Benjamin Plant Manager VCEZ at Volvo Cars Faculty of Computing Internet : www.bth.se Blekinge Institute of Technology Phone : +46 455 38 50 00 SE-371 79 Karlskrona, Sweden Fax : +46 455 38 50 57 ii ACKNOWLEDGEMENTS Our Journey in the thesis has been tremendously intense learning, value added and a remarkable experience. First and foremost, we would first like to thank from our bottom of our hearts to our university supervisor, Dr. Krzysztof Wnuk who has always been a third wheel in the thesis study. Whenever we have faced trouble during thesis time he has made sure that his doors were always open for supervisions, comments on the study, suggestions for unique considerations during the thesis period. His perseverance, motivation on the study and endless enthusiasm has given the researchers inspirations unparalleled. Without his supervision, we could not have achieved this. We simply could not have asked for a better academic supervisor for our thesis study and we are very much honoured to have someone like him. Secondly, we look like to thank Mr. Dani Benjamin for giving us the opportunity to conduct the thesis study at the organization. We would like to thank all the people who have involved in the organization and have supported us in all ways and to make us understand in learning new things at the company. Our external supervisor has treated us extraordinarily and has helped us in many ways and has stayed back even after his office works to help us out personally during our study at the organization. We would like to make a special acknowledgement to all the interview participants who have taken part in the study, without them we would not have completed our thesis and thus making the completing the thesis was inevitable. “I would like to make a personal note and take this opportunity to express my deepest heartful gratitude towards my parents Shri. Veesamsetty Karuna Sagar and Smt. Veesamsetty Udaya Rani and my brother Mr. Veesamsetty Amruth Sagar who have always been with me throughout the thick and thin skin of my life. They have always supported me in all the decisions I have made in life and I would thank them for always constructing a good environment and have made sure that I am always abreast to my goals in life. I would like to make a special mention to my best friends Surya, Shiva, Satya and Shashank. I would like to also take a note on thanking my friends, cousins and family for their long-lasting love and uplifting my confidence levels.” - SAIRAM SAGAR VEESAMSETTY “Finally, I would like to show my heartfelt gratitude on this occasion to my parents B. Srinivas and B. Kavitha my brother B. Bhanu Teja, Gopi Krishna Devulapally and my friends Vishal, Manideep, Saikumar, Imran and Yakub Pasha for their everlasting support, love and wishes. I would forward my thanks to all people, whom I missed to mention inadvertently, and from whom I received direct or indirect help” - Krishna Sai Bitla iii ABSTRACT Context. Software Project management focuses on planning and executing the activities for developing software. Agile Software Project Management helps to plan shorter iterations and frequent changes to customer requirement. Developing the process flow metrics helps to monitor the process and to tune the process for the given context. Objectives. The main objectives in the thesis are to identify process flow metrics and frameworks that are suitable for measuring the process flow in Agile projects especially projects with significant dependence on hardware components. Apart from identified metrics from the literature, we identify the impact, challenges, and advantages of using agile models with the help of productivity and process flow metrics and implement them on a test phase project and compare the productivity of agile model with waterfall model. Methods. The thesis presents a two-step study. The first step was to perform a Systematic Literature Review (SLR) and collect the metrics from the literature study that can be used for the comparison of the productivity of both the processes. The Second step was to conduct a case study at Volvo Cars to get us a better understanding, the impact of the agile and how the process flow metrics can be used in real time for measuring and comparison. Results. In the first step of SLR, 363 metrics that can be used by software teams have been identified of which 10 were suitable for the comparison of our current case study required by the second step of our thesis. In the second step, after using the metrics in the first iteration after the transition, an increase in productivity of 6.25% is achieved by the team following the agile process over the team following the traditional process. Several advantages and challenges faced during the transition have been identified which might have affected the achieved productivity. Conclusions. We conclude from the results achieved that metrics can be used as a tool to enhance the benefits of the Agile process. Process Flow metrics can be of good use to compare the difference of productivity between different processes and make improvements to the current processes. Use of process flow metrics increases the insight of all the team members who are on the progress of the project and guides them to enhance team performance and stay on track with the project schedule. Keywords: Process Flow, Agile software development, traditional software development, software metrics. iv CONTENTS ACKNOWLEDGEMENTS ................................................................................................................................ III ABSTRACT ......................................................................................................................................................... IV CONTENTS ........................................................................................................................................................... V LIST OF FIGURES .......................................................................................................................................... VII LIST OF TABLES ........................................................................................................................................... VIII 1 INTRODUCTION ........................................................................................................................................ 1 1.1 AIMS AND OBJECTIVES OF THE RESEARCH ............................................................................................. 3 1.2 THESIS OUTLINE..................................................................................................................................... 3 1.3 RESEARCH GAP OF THE THESIS .............................................................................................................. 4 2 BACKGROUND ........................................................................................................................................... 5 2.1 “TRADITIONAL” SOFTWARE DEVELOPMENT .......................................................................................... 5 2.2 AGILE SOFTWARE DEVELOPMENT .......................................................................................................... 7 2.3 SOFTWARE METRICS .............................................................................................................................. 8 2.4 PROCESS FLOW ...................................................................................................................................... 9 2.5 CONTEXT DESCRIPTION ........................................................................................................................ 10 3 RELATED WORK .................................................................................................................................... 11 4 METHOD .................................................................................................................................................... 13 4.1 RESEARCH QUESTIONS ......................................................................................................................... 13 4.2 SYSTEMATIC LITERATURE REVIEW ...................................................................................................... 14 4.2.1 Study Identification ......................................................................................................................... 15 4.2.1.1 Identify Start Set Papers ........................................................................................................................
Recommended publications
  • Measuring the Software Size of Sliced V-Model Projects
    2014 Joint Conference of the International Workshop on Software Measurement and the International Conference on Software Process and Product Measurement Measuring the Software Size of Sliced V-model Projects Andreas Deuter PHOENIX CONTACT Electronics GmbH Gregor Engels Dringenauer Str. 30 University of Paderborn 31812 Bad Pyrmont, Germany Zukunftsmeile 1 Email: [email protected] 33102 Paderborn, Germany Email: [email protected] Abstract—Companies expect higher productivity of their soft- But, the “manufacturing” of software within the standard ware teams when introducing new software development methods. production process is just a very short process when bringing Productivity is commonly understood as the ratio of output the binaries of the software to the device. This process is hardly created and resources consumed. Whereas the measurement of to optimize and does not reflect the “production of software” the resources consumed is rather straightforward, there are at all. The creation of the software is namely done in the several definitions for counting the output of a software de- developing teams by applying typical software engineering velopment. Source code-based metrics create a set of valuable figures direct from the heart of the software - the code. However, methods. However, to keep up with the high demands on depending on the chosen process model software developers and implementing new functionality (e.g. for PLC) the software testers produce also a fair amount of documentation. Up to development process within these companies must improve. now this output remains uncounted leading to an incomplete Therefore, they start to analyze their software processes in view on the development output. This article addresses this open detail and to identify productivity drivers.
    [Show full text]
  • Software Error Analysis Technology
    NIST Special Publication 500-209 Computer Systems Software Error Analysis Technology U.S. DEPARTMENT OF COMMERCE Wendy W. Peng Technology Administration Dolores R. Wallace National Institute of Standards and Technology NAT L INST. OF ST4ND & TECH R I.C. NISI PUBLICATIONS A111D3 TTSTll ^QC ' 100 .U57 //500-209 1993 7he National Institute of Standards and Technology was established in 1988 by Congress to "assist industry in the development of technology . needed to improve product quality, to modernize processes, to reliability . manufacturing ensure product and to facilitate rapid commercialization . , of products based on new scientific discoveries." NIST, originally founded as the National Bureau of Standards in 1901, works to strengthen U.S. industry's competitiveness; advance science and engineering; and improve public health, safety, and the environment. One of the agency's basic functions is to develop, maintain, and retain custody of the national standards of measurement, and provide the means and methods for comparing standards used in science, engineering, manufacturing, commerce, industry, and education with the standards adopted or recognized by the Federal Government. As an agency of the U.S. Commerce Department's Technology Administration, NIST conducts basic and applied research in the physical sciences and engineering and performs related services. The Institute does generic and precompetitive work on new and advanced technologies. NIST's research facilities are located at Gaithersburg, MD 20899, and at Boulder, CO 80303.
    [Show full text]
  • Software Metrics: Successes, Failures and New Directions
    Software Metrics: Successes, Failures and New Directions Norman E Fenton Centre for Software Reliability Agena Ltd City University 11 Main Street Northampton Square Caldecote London EC1V OHB Cambridge CB3 7NU United Kingdom United Kingdom Tel: 44 171 477 8425 Tel 44 1223 263753 / 44 181 530 5981 Fax: 44 171 477 8585 Fax: 01223 263899 [email protected] [email protected] www.csr.city.ac.uk/people/norman.fenton/ www.agena.co.uk Abstract The history of software metrics is almost as old as the history of software engineering. Yet, the extensive research and literature on the subject has had little impact on industrial practice. This is worrying given that the major rationale for using metrics is to improve the software engineering decision making process from a managerial and technical perspective. Industrial metrics activity is invariably based around metrics that have been around for nearly 30 years (notably Lines of Code or similar size counts, and defects counts). While such metrics can be considered as massively successful given their popularity, their limitations are well known, and mis-applications are still common. The major problem is in using such metrics in isolation. We argue that it is possible to provide genuinely improved management decision support systems based on such simplistic metrics, but only by adopting a less isolationist approach. Specifically, we feel it is important to explicitly model: a) cause and effect relationships and b) uncertainty and combination of evidence. Our approach uses Bayesian Belief nets which are increasingly seen as the best means of handling decision-making under uncertainty.
    [Show full text]
  • Software Measurement: a Necessary Scientific Basis Norman Fenton
    IEEE TRANSACTIONS ON SOFTWARE ENGINEERING. VOL. 20, NO. 3, MARCH I994 I99 Software Measurement: A Necessary Scientific Basis Norman Fenton Abstruct- Software measurement, like measurement in any 11. MEASUREMENTFUNDAMENTALS other discipline, must adhere to the science of measurement if it is to gain widespread acceptance and validity. The observation In this section, we provide a summary of the key concepts of some very simple, but fundamental, principles of measurement from the science of measurement which are relevant to soft- can have an extremely beneficial effect on the subject. Measure- ware metrics. First, we define the fundamental notions (which ment theory is used to highlight both weaknesses and strengths of are generally not well understood) and then we summarise the software metrics work, including work on metrics validation. We representational theory of measurement. Finally, we explain identify a problem with the well-known Weyuker properties, but also show that a criticism of these properties by Cherniavsky and how this leads inevitably to a goal-oriented approach. Smith is invalid. We show that the search for general software complexity measures is doomed to failure. However, the theory A. What is Measurement? does help us to define and validate measures of specific complexity attributes. Above all, we are able to view software measurement Measurement is defined as the process by which numbers or in a very wide perspective, rationalising and relating its many symbols are assigned to attributes of entities in the real world diverse activities. in such a way as to describe them according to clearly defined Index Terms-Software measurement, empirical studies, met- rules [ 131, [36].
    [Show full text]
  • Ssfm BPG 1: Validation of Software in Measurement Systems
    NPL Report DEM-ES 014 Software Support for Metrology Best Practice Guide No. 1 Validation of Software in Measurement Systems Brian Wichmann, Graeme Parkin and Robin Barker NOT RESTRICTED January 2007 National Physical Laboratory Hampton Road Teddington Middlesex United Kingdom TW11 0LW Switchboard 020 8977 3222 NPL Helpline 020 8943 6880 Fax 020 8943 6458 www.npl.co.uk Software Support for Metrology Best Practice Guide No. 1 Validation of Software in Measurement Systems Brian Wichmann with Graeme Parkin and Robin Barker Mathematics and Scientific Computing Group January 2007 ABSTRACT The increasing use of software within measurement systems implies that the validation of such software must be considered. This Guide addresses the validation both from the perspective of the user and the supplier. The complete Guide consists of a Management Overview and a Technical Application together with consideration of its use within safety systems. Version 2.2 c Crown copyright 2007 Reproduced with the permission of the Controller of HMSO and Queen’s Printer for Scotland ISSN 1744–0475 National Physical Laboratory, Hampton Road, Teddington, Middlesex, United Kingdom TW11 0LW Extracts from this guide may be reproduced provided the source is acknowledged and the extract is not taken out of context We gratefully acknowledge the financial support of the UK Department of Trade and Industry (National Measurement System Directorate) Approved on behalf of the Managing Director, NPL by Jonathan Williams, Knowledge Leader for the Electrical and Software team Preface The use of software in measurement systems has dramatically increased in the last few years, making many devices easier to use, more reliable and more accurate.
    [Show full text]
  • Objectives and Context of Software Measurement, Analysis, and Control Michael A
    Objectives and Context of Software Measurement, Analysis, and Control Michael A. Cusumano Massachusetts Institute of Technology Sloan School WP#3471-92/BPS October 8, 1992 Objectives and Context of Software Measurement, Analysis and Control Michael A. Cusumano Sloan School of Management Massachusetts Institute of Technology Cambridge, Massachusetts USA 02142 Abstract This paper focuses on the what and why of measurement in software development, and the impact of the context of the development organization or of specific projects on approaches to measurement, analysis, and control. It also presents observations on contextual issues based on studies of the major Japanese software producers. 1 Introduction This paper focuses on the what and why of measurement in software development, and the impact of the context of the development organization or of specific projects on approaches to measurement, analysis, and control. It begins by discussing three basic questions: (1) Why do we measure? (2) What is it we want to know? And (3) how does the context affect the ability to measure? The second section presents observations on contextual issues based on my ongoing study of the Japanese [3]. This paper also relies heavily on extensive interviews conducted at Hitachi, Fujitsu, NEC, and Toshiba during May, August, and September 1992. 2 Basic Questions Why we measure aspects of the software development process and conduct analytical exercises or experiments, as well as what we try to do with this information, depends very much on who is doing or requesting the measuring and analysis. There already exists a broad literature on technical aspects software measurement and the relationship to process management, which will not be reviewed here [1, 2, 5, 6].
    [Show full text]
  • 12 Steps to Useful Software Metrics
    12 Steps to Useful Software Metrics Linda Westfall The Westfall Team [email protected] PMB 101, 3000 Custer Road, Suite 270 Plano, TX 75075 972-867-1172 (voice) 972-943-1484 (fax) Abstract: 12 Steps to Useful Software Metrics introduces the reader to a practical process for establishing and tailoring a software metrics program that focuses on goals and information needs. The process provides a practical, systematic, start-to-finish method of selecting, designing and implementing software metrics. It outlines a cookbook method that the reader can use to simplify the journey from software metrics in concept to delivered information. Bio: Linda Westfall is the President of The Westfall Team, which provides Software Metrics and Software Quality Engineering training and consulting services. Prior to starting her own business, Linda was the Senior Manager of the Quality Metrics and Analysis at DSC Communications where her team designed and implemented a corporate wide metric program. Linda has more than twenty-five years of experience in real-time software engineering, quality and metrics. She has worked as a Software Engineer, Systems Analyst, Software Process Engineer and Manager of Production Software. Very active professionally, Linda Westfall is a past chair of the American Society for Quality (ASQ) Software Division. She has also served as the Software Division’s Program Chair and Certification Chair and on the ASQ National Certification Board. Linda is a past-chair of the Association for Software Engineering Excellence (ASEE) and has chaired several conference program committees. Linda Westfall has an MBA from the University of Texas at Dallas and BS in Mathematics from Carnegie- Mellon University.
    [Show full text]
  • Model of Agile Software Measurement: a Case Study Master of Science Thesis in the Programme Software Engineering and Technology
    Model of Agile Software Measurement: A Case Study Master of Science Thesis in the Programme Software engineering and Technology JOHAN GUSTAFSSON Chalmers University of Technology University of Gothenburg Department of Computer Science and Engineering Göteborg, Sweden, June 2011 The Author grants to Chalmers University of Technology and University of Gothenburg the non-exclusive right to publish the Work electronically and in a non-commercial purpose make it accessible on the Internet. The Author warrants that he/she is the author to the Work, and warrants that the Work does not contain text, pictures or other material that violates copyright law. The Author shall, when transferring the rights of the Work to a third party (for example a publisher or a company), acknowledge the third party about this agreement. If the Author has signed a copyright agreement with a third party regarding the Work, the Author warrants hereby that he/she has obtained any necessary permission from this third party to let Chalmers University of Technology and University of Gothenburg store the Work electronically and make it accessible on the Internet. Model of Agile Software Measurement: A Case Study JOHAN GUSTAFSSON © JOHAN GUSTAFSSON, June 2011. Supervisor: ROBERT FELDT Chalmers University of Technology University of Gothenburg Department of Computer Science and Engineering SE-412 96 Göteborg Sweden Telephone + 46 (0)31-772 1000 Department of Computer Science and Engineering Göteborg,Sweden 1 Model of Agile Software Measurement: A Case Study Johan Gustafsson Computer Science and Engineering. Chalmers University of Technology Gothenburg, Sweden [email protected] Abstract— This master thesis examines what type of What companies need is a quick, flexible and cost efficient measurement should be conducted in agile software development.
    [Show full text]
  • An Approach for Deriving, Analyzing, and Organizing Requirements Metrics and Related Information in Systems Projects
    An Approach for Deriving, Analyzing, and Organizing Requirements Metrics and Related Information in Systems Projects Ibtehal Noorwali1, Nazim H. Madhavji1 and Remo Ferrari2 1Computer Science Department, The University of Western Ontario, London, Ontario, Canada 2Siemens Mobility, New York, NY, U.S.A. Keywords: Requirements Metrics, Requirements Management, Measurement Approach. Abstract: Large systems projects present unique challenges to the requirements measurement process: large sets of re- quirements across many sub-projects, requirements existing in different categories (e.g., hardware, interface, software, etc.), varying requirements meta-data items (e.g., ID, requirement type, priority, etc.), to name few. Consequently, requirements metrics are often incomplete, metrics and measurement reports are often unor- ganized, and meta-data items that are essential for applying the metrics are often incomplete or missing. To our knowledge, there are no published approaches for measuring requirements in large systems projects. In this paper, we propose a 7-step approach that combines the use of the goal-question-metric paradigm (GQM) and the identification and analysis of four main RE measurement elements: attributes, levels, metrics, and meta-data items—that aids in the derivation, analysis, and organization of requirements metrics. We illustrate the use of our approach by applying it to real-life data from the rail automation systems domain. We show how the approach led to a more comprehensive set of requirements metrics, improved organization
    [Show full text]
  • Metrics, Models, and Measures for Managing Programs & Projects
    Lean & Agile Performance Measurement Metrics, Models, and Measures for Managing Programs & Projects Dr. David F. Rico, PMP, CSEP, FCP, FCT, ACP, CSM, SAFE, DEVOPS Twitter: @dr_david_f_rico Website: http://www.davidfrico.com LinkedIn: http://www.linkedin.com/in/davidfrico Agile Capabilities: http://davidfrico.com/rico-capability-agile.pdf Agile Cost of Quality: http://www.davidfrico.com/agile-vs-trad-coq.pdf DevOps Return on Investment (ROI): http://davidfrico.com/rico-devops-roi.pdf Dave’s NEW Business Agility Video: https://www.youtube.com/watch?v=-wTXqN-OBzA Dave’s NEWER Development Operations Security Video: https://vimeo.com/214895416 DoD Fighter Jets vs. Amazon Web Services: http://davidfrico.com/dod-agile-principles.pdf Author Background Gov’t contractor with 35+ years of IT experience B.S. Comp. Sci., M.S. Soft. Eng., & D.M. Info. Sys. Large gov’t projects in U.S., Far/Mid-East, & Europe Career systems & software engineering methodologist Lean-Agile, Six Sigma, CMMI, ISO 9001, DoD 5000 NASA, USAF, Navy, Army, DISA, & DARPA projects Published seven books & numerous journal articles Intn’l keynote speaker, 196+ talks to 14,400 people Specializes in metrics, models, & cost engineering Cloud Computing, SOA, Web Services, FOSS, etc. Professor at 7 Washington, DC-area universities 2 On Metrics—Peter Drucker 3 Large TRADITIONAL Projects Size vs. Quality Size vs. Change 16.00 40% 12.80 32% 9.60 24% 6.40 16% CHANGE DEFECTS 3.20 8% 0.00 0% 0 2 6 25 100 400 0 2 6 25 100 400 SIZE SIZE Size vs. Productivity Size vs. Success 5.00 60% 4.00 48% 3.00 36% 2.00 24% 1.00 SUCCESS 12% PRODUCTIVITY 0.00 0% 0 2 6 25 100 400 0 2 6 25 100 400 SIZE SIZE Jones, C.
    [Show full text]
  • Contextual Injection of Quality Measures Into Software Engineering Processes
    International Journal on Advances in Software, vol 4 no 1 & 2, year 2011, http://www.iariajournals.org/software/ 76 Contextual Injection of Quality Measures into Software Engineering Processes Gregor Grambow and Roy Oberhauser Manfred Reichert Computer Science Dept. Institute for Databases and Information Systems Aalen University, Germany Ulm University, Germany {gregor.grambow, roy.oberhauser}@htw-aalen.de [email protected] Abstract—Despite improvements in software engineering process (e.g., junior vs. senior engineers, information needs), processes and tools, concrete preventative and analytical activity coordination with the overall team process, the software quality assurance activities are still typically contextual and project influences on the processes (e.g., manually triggered and determined, resulting in missed or schedule, resource availability), and software quality untimely quality opportunities and increased project overhead. assurance (SQA) dependencies (e.g., quality plan, reactive Quality goals, when defined, lack holistic environmental quality measures, metric dependencies). support for automated performance measurement and Historically software development projects have also governance that is tightly integrated in the low-level faced difficulties in meeting budget, schedule, functionality, operational software engineering processes. This results in and quality targets [6][7][8]. A more recent study in 2002 by higher quality risks and cost risks. Based on adaptive process the National Institute of Standards and Technology (NIST) management, an approach is presented that injects situationally-determined quality measure proposals into the found that most delivered software products are still stricken concrete workflows of software engineers, using contextual by bugs and defects [9]. While some of these difficulties semantic knowledge and multi-agent quality goal tracking and might be ascribed to a misaligned planning environment in decision-making.
    [Show full text]
  • Measuring Productivity in Agile Software Development Process: a Scoping Study
    Measuring Productivity in Agile Software Development Process: A Scoping Study Syed Muhammad Ali Shah Efi Papatheocharous Jaana Nyfjord SICS Swedish ICT AB SICS Swedish ICT AB SICS Swedish ICT AB [email protected] [email protected] [email protected] ABSTRACT In 2001, a philosophy for developing software called ‘agile software An agile software development process is often claimed to increase development’ was introduced. Agile software development is a productivity. However, productivity measurement in agile software group of software development methods, based on a collection of development is little researched. Measures are not explicitly defined iterative and incremental concepts, principles and practices [5]. The nor commonly agreed upon. In this paper, we highlight the agile agile software development manifesto [6] defines the essential productivity measures reported in literature by means of a research principles of agile methods by valuing: method called scoping study. We were able to identify 12 papers Individuals and interactions over processes and tools, reporting the productivity measures in agile software development Working software over comprehensive documentation, processes. We found that finding, understanding and putting into use Customer collaboration over contract negotiation, agile productivity definitions is not an easy task. From the Responding to change over following a plan. perspective of common roles in agile software development process and existing knowledge workers’ productivity dimensions, we also Agile comes up with a different philosophy of developing software emphasize that none of the productivity measures satisfy these fully. requiring a set of different measurement metrics and methods, e.g., We recommend that future effort should be focused on defining story points and planning poker, for measuring the complexity of agile productivity in measurable, practicable and meaningful form.
    [Show full text]