Acceptance Testing in Agile Software Development Perspectives from Research and Practice

Total Page:16

File Type:pdf, Size:1020Kb

Acceptance Testing in Agile Software Development Perspectives from Research and Practice Master Thesis Software Engineering Thesis no: MSE-2020-NN 01 2021 Acceptance Testing in Agile Software Development Perspectives from Research and Practice Nayla Nasir Dept. Software Engineering Blekinge Institute of Technology SE–371 79 Karlskrona, Sweden This thesis is submitted to the Department of Software Engineering at Blekinge Institute of Technology in partial fulfillment of the requirements for the degree of Master of Science in Software Engineering. The thesis is equivalent to 20 weeks of full-time studies. Contact Information: Author(s): Nayla Nasir E-mail: [email protected] University advisor: Davide Fucci Dept. Software Engineering Dept. Software Engineering Internet : www.bth.se/dipt Blekinge Institute of Technology Phone : +46 455 38 50 00 SE–371 79 Karlskrona, Sweden Fax : +46 455 38 50 57 Abstract Context: Acceptance testing is an important activity that verifies the confor- mance of a system to its acceptance criteria. It aims to provide a detailed com- munication of domain knowledge and is used to evaluate whether the customer requirements are met. Existing literature lacks the empirical evidence for accep- tance testing. Especially in context of industry practice, it is not in the authors’ consideration, except for a few studies, where the authors have investigated the state of practice in a specific domain. Objective: This study aims to recognize the state of research and practice of acceptance testing in Agile Software Development and investigate the similari- ties and differences in both perspectives. The study contributes to identify the industry-academia gap in the context of acceptance testing. Research Method: To identify the acceptance testing practices and challenges from research, I have conducted a literature review. For the industry perspective on acceptance testing practices and challenges, I have conducted an interview- based survey of the practitioners working in Agile Software Development envi- ronment. I followed the snowball search strategy to search the primary studies, whereas to select the respondents, I used the convenience and snowball sampling method. For data analysis, I followed the approach of thematic synthesis. Results: The results of this thesis are the outcome of a literature review of 20 selected studies and an interview-based survey with 12 practitioners representing 10 companies. I identified acceptance testing practices and challenges from re- search and industry. In the research, the most recommended form of acceptance testing is acceptance test-driven development (ATDD), and the majority of the studies are referring to the use of FIT for the acceptance testing. Customer in- volvement in different phases of acceptance testing is recommended in research. From the interviews, I come across that acceptance testing is manual at large in the industry, and the most challenging aspect is the customer’s involvement. Conclusions: From the findings of this thesis, it is concluded that there is a gap between the research and industry perspective of acceptance testing prac- tices. Currently, acceptance testing in industry is mostly manual, the research is not focusing on this aspect of acceptance testing. Despite the differences, there are some commonalities as well. Specially, most challenges of acceptance testing are similar in both perspectives. Researchers have to consider the commonalities, and they have to look at how they can minimize the acceptance testing challenges from the perspective of industry. Keywords: Acceptance testing, ATDD, Agile Software Development, practition- ers’ perspective, industry-academia gap. Acknowledgments First of all, I would like to thank my supervisor, Davide Fucci, for guiding me in this study. Whenever I asked for any guidance, he ensured his availability. Secondly, I would like to thank all the experts who took active and volun- teer participation in the interviews. Without their volunteer participation, I would not be able to finish this study. Lastly, I would like to extend my heartiest thanks to my family and friends for their unconditional support during this work. I owe a lot of gratitude to my husband, Nasir Mehmood Minhas, who has always been a source of guidance and motivation for me. His confidence in me, makes me set higher goals for myself. Nayla Nasir ii List of Figures 3.1 Example color coding to extract AT practices and challenges . 12 4.1 Collaboration challenges identified by the practitioners. ...... 32 4.2 AT practices – similarities and differences in two perspectives. 35 4.3 AT practices – extent of commonality in two perspectives. .... 36 4.4 AT challenges—similarities and differences in two perspectives. 37 4.5 AT challenges – extent of commonality in two perspectives. .... 38 iii List of Tables 3.1 Snowball iterations. .......................... 11 3.2 Data extraction form. ......................... 12 3.3 Interview Questions. ......................... 14 3.4 Interview participants. ........................ 16 3.5 Participants’ Organizational Context. ................ 16 3.6 Analysis procedure adopted for step2 and step 3. ......... 17 4.1 Acceptance testing practices in the literature. ........... 21 4.2 Acceptance testing challenges in the literature. ........... 22 4.3 State of acceptance testing practice/organizational policy. 24 4.4 Acceptance test automation. ..................... 25 4.5 Practitioners’ perspective of acceptance testing. .......... 27 4.6 Participants’ opinion on ATDD. ................... 28 4.7 Practitioners’ perspective of AT practices. ............. 29 4.8 Practitioners perspective of AT Challenges. ............. 31 iv Contents Abstract i 1 Introduction 1 1.1 Background .............................. 1 1.2 What is acceptance testing? ..................... 2 1.3Whatisthegap?........................... 2 1.4Focusofthiswork........................... 3 1.4.1 Objectives . ......................... 3 1.4.2 Contributions ......................... 4 1.5 Thesis arrangement .......................... 4 2 Related Work 5 3 Method 8 3.1 Research Questions .......................... 8 3.2 Selection of research method ..................... 9 3.3 Detailed study design ......................... 10 3.3.1 Literature review ....................... 10 3.3.2 Survey ............................. 12 3.3.3 Data analysis ......................... 15 3.4 Threats to validity .......................... 18 4 Results 20 4.1 Acceptance testing in research perspective – RQ-1 ......... 20 4.1.1 Acceptance testing practices in literature .......... 20 4.1.2 Acceptance testing challenges in literature ......... 22 4.2 Acceptance testing in industry perspective– RQ-2 ......... 23 4.2.1 Organizational policy of acceptance testing ......... 23 4.2.2 Acceptance test automation ................. 25 4.2.3 AT definition ......................... 26 4.2.4 Practitioners’ Perception on ATDD ............. 27 4.2.5 Acceptance testing practices in industry .......... 29 4.2.6 Acceptance testing challenges in industry .......... 31 4.3 Similarities and differences in the two perspectives – RQ-3 .... 34 v 4.3.1 Similarities and differences in AT practices ......... 34 4.3.2 Similarities and differences in AT challenges ........ 36 5 Analysis and Discussion 40 5.1 Who is involved in acceptance testing ................ 40 5.2 When acceptance tests are written ................. 41 5.3 How acceptance tests are written .................. 42 6 Conclusions and Future Work 44 6.1 Conclusions .............................. 44 6.2FutureWork.............................. 45 vi Chapter 1 Introduction 1.1 Background Testing is an essential activity of the software development life cycle. It is a complicated and costly activity and can consume up to 50% of the total cost [1]. The goal of testing is to ensure the quality of the system under test, and it is performed at various levels, including unit, integration, and system level. It verifies the correctness of the system’s functional and non-functional aspects, and this fact classifies the testing into two categories: functional testing and non- functional testing. Some of the common testing types are unit testing, integration testing, system testing, and acceptance testing [2]. The success of any software product lies with the fact that the customers/users accept it. Acceptance testing has a central role in this regard. It advocates the involvement of the domain experts (customers) in the verification/validation process. It gives them a chance to check if everything matches their expectations and if the requirements have been communicated and implemented correctly [3]. Despite its significance, empirical evidence on acceptance testing practices in Agile is very low [4], many authors [4, 5, 6, 7] suggested more research in this area. This fact motivated the author to carry out an empirical investigation on ac- ceptance testing in Agile Software Development. Agile Software Development is a controversial topic since the outset. Its proponents and opponents are in both research and practice. Despite many criticisms, there is no doubt that Agile methods are widely used in the software industry. Customer involvement, early and frequent feedback, and quick releases are some of the benefits of Agile de- velopment methodologies. Unlike classical software development methods, Agile methods advocate for early and frequent acceptance testing [8]. The subsequent sections define acceptance testing, describe the research gap, and put a light on the focus of current research. 1 Chapter 1. Introduction 2 1.2 What is acceptance testing?
Recommended publications
  • Effectiveness of Software Testing Techniques in Enterprise: a Case Study
    MYKOLAS ROMERIS UNIVERSITY BUSINESS AND MEDIA SCHOOL BRIGITA JAZUKEVIČIŪTĖ (Business Informatics) EFFECTIVENESS OF SOFTWARE TESTING TECHNIQUES IN ENTERPRISE: A CASE STUDY Master Thesis Supervisor – Assoc. Prof. Andrej Vlasenko Vilnius, 2016 CONTENTS INTRODUCTION .................................................................................................................................. 7 1. THE RELATIONSHIP BETWEEN SOFTWARE TESTING AND SOFTWARE QUALITY ASSURANCE ........................................................................................................................................ 11 1.1. Introduction to Software Quality Assurance ......................................................................... 11 1.2. The overview of Software testing fundamentals: Concepts, History, Main principles ......... 20 2. AN OVERVIEW OF SOFTWARE TESTING TECHNIQUES AND THEIR USE IN ENTERPRISES ...................................................................................................................................... 26 2.1. Testing techniques as code analysis ....................................................................................... 26 2.1.1. Static testing ...................................................................................................................... 26 2.1.2. Dynamic testing ................................................................................................................. 28 2.2. Test design based Techniques ...............................................................................................
    [Show full text]
  • Studying the Feasibility and Importance of Software Testing: an Analysis
    Dr. S.S.Riaz Ahamed / Internatinal Journal of Engineering Science and Technology Vol.1(3), 2009, 119-128 STUDYING THE FEASIBILITY AND IMPORTANCE OF SOFTWARE TESTING: AN ANALYSIS Dr.S.S.Riaz Ahamed Principal, Sathak Institute of Technology, Ramanathapuram,India. Email:[email protected], [email protected] ABSTRACT Software testing is a critical element of software quality assurance and represents the ultimate review of specification, design and coding. Software testing is the process of testing the functionality and correctness of software by running it. Software testing is usually performed for one of two reasons: defect detection, and reliability estimation. The problem of applying software testing to defect detection is that software can only suggest the presence of flaws, not their absence (unless the testing is exhaustive). The problem of applying software testing to reliability estimation is that the input distribution used for selecting test cases may be flawed. The key to software testing is trying to find the modes of failure - something that requires exhaustively testing the code on all possible inputs. Software Testing, depending on the testing method employed, can be implemented at any time in the development process. Keywords: verification and validation (V & V) 1 INTRODUCTION Testing is a set of activities that could be planned ahead and conducted systematically. The main objective of testing is to find an error by executing a program. The objective of testing is to check whether the designed software meets the customer specification. The Testing should fulfill the following criteria: ¾ Test should begin at the module level and work “outward” toward the integration of the entire computer based system.
    [Show full text]
  • Acceptance Testing Tools - a Different Perspective on Managing Requirements
    Acceptance Testing Tools - A different perspective on managing requirements Wolfgang Emmerich Professor of Distributed Computing University College London http://sse.cs.ucl.ac.uk Learning Objectives • Introduce the V-Model of quality assurance • Stress the importance of testing in terms of software engineering economics • Understand that acceptance tests are requirements specifications • Introduce acceptance and integration testing tools for Test Driven Development • Appreciate that automated acceptance tests are executable requirements specifications 2 V-Model in Distributed System Development Requirements Acceptance Test Software Integration Architecture Test Detailed System Design Test See: B. Boehm Guidelines for Verifying and Validating Software Unit Requirements and Design Code Specifications. Euro IFIP, P. A. Test Samet (editor), North-Holland Publishing Company, IFIP, 1979. 3 1 Traditional Software Development Requirements Acceptance Test Software Integration Architecture Test Detailed System Design Test Unit Code Test 4 Test Driven Development of Distributed Systems Use Cases/ User Stories Acceptance QoS Requirements Test Software Integration & Architecture System Test Detailed Unit Design Test These tests Code should be automated 5 Advantages of Test Driven Development • Early definition of acceptance tests reveals incomplete requirements • Early formalization of requirements into automated acceptance tests unearths ambiguities • Flaws in distributed software architectures (there often are many!) are discovered early • Unit tests become precise specifications • Early resolution improves productivity (see next slide) 6 2 Software Engineering Economics See: B. Boehm: Software Engineering Economics. Prentice Hall. 1981 7 An Example Consider an on-line car dealership User Story: • I first select a locale to determine the language shown at the user interface. I then select the SUV I want to buy.
    [Show full text]
  • Model-Based Fuzzing Using Symbolic Transition Systems
    Model-Based Fuzzing Using Symbolic Transition Systems Wouter Bohlken [email protected] January 31, 2021, 48 pages Academic supervisor: Dr. Ana-Maria Oprescu, [email protected] Daily supervisor: Dr. Machiel van der Bijl, [email protected] Host organisation: Axini, https://www.axini.com Universiteit van Amsterdam Faculteit der Natuurwetenschappen, Wiskunde en Informatica Master Software Engineering http://www.software-engineering-amsterdam.nl Abstract As software is getting more complex, the need for thorough testing increases at the same rate. Model- Based Testing (MBT) is a technique for thorough functional testing. Model-Based Testing uses a formal definition of a program and automatically extracts test cases. While MBT is useful for functional testing, non-functional security testing is not covered in this approach. Software vulnerabilities, when exploited, can lead to serious damage in industry. Finding flaws in software is a complicated, laborious, and ex- pensive task, therefore, automated security testing is of high relevance. Fuzzing is one of the most popular and effective techniques for automatically detecting vulnerabilities in software. Many differ- ent fuzzing approaches have been developed in recent years. Research has shown that there is no single fuzzer that works best on all types of software, and different fuzzers should be used for different purposes. In this research, we conducted a systematic review of state-of-the-art fuzzers and composed a list of candidate fuzzers that can be combined with MBT. We present two approaches on how to combine these two techniques: offline and online. An offline approach fully utilizes an existing fuzzer and automatically extracts relevant information from a model, which is then used for fuzzing.
    [Show full text]
  • An Analysis of Current Guidance in the Certification of Airborne Software
    An Analysis of Current Guidance in the Certification of Airborne Software by Ryan Erwin Berk B.S., Mathematics I Massachusetts Institute of Technology, 2002 B.S., Management Science I Massachusetts Institute of Technology, 2002 Submitted to the System Design and Management Program In Partial Fulfillment of the Requirements for the Degree of Master of Science in Engineering and Management at the ARCHIVES MASSACHUSETrS INS E. MASSACHUSETTS INSTITUTE OF TECHNOLOGY OF TECHNOLOGY June 2009 SEP 2 3 2009 © 2009 Ryan Erwin Berk LIBRARIES All Rights Reserved The author hereby grants to MIT permission to reproduce and to distribute publicly paper and electronic copies of this thesis document in whole or in part in any medium now known or hereafter created. Signature of Author Ryan Erwin Berk / System Design & Management Program May 8, 2009 Certified by _ Nancy Leveson Thesis Supervisor Professor of Aeronautics and Astronautics pm-m A 11A Professor of Engineering Systems Accepted by _ Patrick Hale Director System Design and Management Program This page is intentionally left blank. An Analysis of Current Guidance in the Certification of Airborne Software by Ryan Erwin Berk Submitted to the System Design and Management Program on May 8, 2009 in Partial Fulfillment of the Requirements for the Degree of Master of Science in Engineering and Management ABSTRACT The use of software in commercial aviation has expanded over the last two decades, moving from commercial passenger transport down into single-engine piston aircraft. The most comprehensive and recent official guidance on software certification guidelines was approved in 1992 as DO-178B, before the widespread use of object-oriented design and complex aircraft systems integration in general aviation (GA).
    [Show full text]
  • Manual on Quality Assurance for Computer Software Related to the Safety of Nuclear Power Plants
    SIMPLIFIED SOFTWARE LIFE-CYCLE DIAGRAM FEASIBILITY STUDY PROJECT TIME I SOFTWARE P FUNCTIONAL I SPECIFICATION! SOFTWARE SYSTEM DESIGN DETAILED MODULES CECIFICATION MODULES DESIGN SOFTWARE INTEGRATION AND TESTING SYSTEM TESTING ••COMMISSIONING I AND HANDOVER | DECOMMISSION DESIGN DESIGN SPECIFICATION VERIFICATION OPERATION AND MAINTENANCE SOFTWARE LIFE-CYCLE PHASES TECHNICAL REPORTS SERIES No. 282 Manual on Quality Assurance for Computer Software Related to the Safety of Nuclear Power Plants f INTERNATIONAL ATOMIC ENERGY AGENCY, VIENNA, 1988 MANUAL ON QUALITY ASSURANCE FOR COMPUTER SOFTWARE RELATED TO THE SAFETY OF NUCLEAR POWER PLANTS The following States are Members of the International Atomic Energy Agency: AFGHANISTAN GUATEMALA PARAGUAY ALBANIA HAITI PERU ALGERIA HOLY SEE PHILIPPINES ARGENTINA HUNGARY POLAND AUSTRALIA ICELAND PORTUGAL AUSTRIA INDIA QATAR BANGLADESH INDONESIA ROMANIA BELGIUM IRAN, ISLAMIC REPUBLIC OF SAUDI ARABIA BOLIVIA IRAQ SENEGAL BRAZIL IRELAND SIERRA LEONE BULGARIA ISRAEL SINGAPORE BURMA ITALY SOUTH AFRICA BYELORUSSIAN SOVIET JAMAICA SPAIN SOCIALIST REPUBLIC JAPAN SRI LANKA CAMEROON JORDAN SUDAN CANADA KENYA SWEDEN CHILE KOREA, REPUBLIC OF SWITZERLAND CHINA KUWAIT SYRIAN ARAB REPUBLIC COLOMBIA LEBANON THAILAND COSTA RICA LIBERIA TUNISIA COTE D'lVOIRE LIBYAN ARAB JAMAHIRIYA TURKEY CUBA LIECHTENSTEIN UGANDA CYPRUS LUXEMBOURG UKRAINIAN SOVIET SOCIALIST CZECHOSLOVAKIA MADAGASCAR REPUBLIC DEMOCRATIC KAMPUCHEA MALAYSIA UNION OF SOVIET SOCIALIST DEMOCRATIC PEOPLE'S MALI REPUBLICS REPUBLIC OF KOREA MAURITIUS UNITED ARAB
    [Show full text]
  • Acceptance Testing How Cslim and Fitnesse Can Help You Test Your Embedded System
    Acceptance Testing How CSlim and FitNesse Can Help You Test Your Embedded System Doug Bradbury Software Craftsman, 8th Light Tutorial Environment git clone git://github.com/dougbradbury/c_learning.git cd c_learning ./bootstrap.sh or with a live CD: cp -R cslim_agile_package c_clearning cd c_learning git pull ./bootstrap.sh Overview Talk w/ exercises: Acceptance Tests Tutorial: Writing Acceptance tests Tutorial: Fitnesse Tutorial: CSlim Talk: Embedded Systems Integration Bonus Topics Introductions Who are you? Where do you work? What experience do you have with ... embedded systems? acceptance testing? FitNesse and Slim? Objectives As a result of this course you will be able to: Understand the purposes of acceptance testing; Use acceptance tests to define and negotiate scope on embedded systems projects; Integrate a CSlim Server into your embedded systems; Objectives (cont) As a result of this course you will be able to: Add CSlim fixtures to your embedded system; Write Fitnesse tests to drive the execution of CSlim fixtures; Write and maintain suites of tests in a responsible manner. Points on a star How many points does this star have? Star Point Specification Points on a star are counted by the number of exterior points. Points on a star How many points does this star have? By Example 3 5 9 Points on a star Now, how many points does this star have? Robo-draw Pick a partner ... Acceptance Testing Collaboratively producing examples of what a piece of software is supposed to do Unit Tests help you build the code right. Acceptance Tests
    [Show full text]
  • Test-Driven Development in Enterprise Integration Projects
    Test-Driven Development in Enterprise Integration Projects November 2002 Gregor Hohpe Wendy Istvanick Copyright ThoughtWorks, Inc. 2002 Table of Contents Summary............................................................................................................. 1 Testing Complex Business Applications......................................................... 2 Testing – The Stepchild of the Software Development Lifecycle?............................................... 2 Test-Driven Development............................................................................................................. 2 Effective Testing........................................................................................................................... 3 Testing Frameworks..................................................................................................................... 3 Layered Testing Approach ........................................................................................................... 4 Testing Integration Solutions............................................................................ 5 Anatomy of an Enterprise Integration Solution............................................................................. 5 EAI Testing Challenges................................................................................................................ 6 Functional Testing for Integration Solutions................................................................................. 7 EAI Testing Framework ..................................................................................
    [Show full text]
  • Software Testing: Essential Phase of SDLC and a Comparative Study Of
    International Journal of System and Software Engineering Volume 5 Issue 2, December 2017 ISSN.: 2321-6107 Software Testing: Essential Phase of SDLC and a Comparative Study of Software Testing Techniques Sushma Malik Assistant Professor, Institute of Innovation in Technology and Management, Janak Puri, New Delhi, India. Email: [email protected] Abstract: Software Development Life-Cycle (SDLC) follows In the software development process, the problem (Software) the different activities that are used in the development of a can be dividing in the following activities [3]: software product. SDLC is also called the software process ∑ Understanding the problem and it is the lifeline of any Software Development Model. ∑ Decide a plan for the solution Software Processes decide the survival of a particular software development model in the market as well as in ∑ Coding for the designed solution software organization and Software testing is a process of ∑ Testing the definite program finding software bugs while executing a program so that we get the zero defect software. The main objective of software These activities may be very complex for large systems. So, testing is to evaluating the competence and usability of a each of the activity has to be broken into smaller sub-activities software. Software testing is an important part of the SDLC or steps. These steps are then handled effectively to produce a because through software testing getting the quality of the software project or system. The basic steps involved in software software. Lots of advancements have been done through project development are: various verification techniques, but still we need software to 1) Requirement Analysis and Specification: The goal of be fully tested before handed to the customer.
    [Show full text]
  • Continuous Quality and Testing to Accelerate Application Development
    Continuous Quality and Testing to Accelerate Application Development How to assess your current testing maturity level and practice continuous testing for DevOps Continuous Quality and Testing to Accelerate Application Development // 1 Table of Contents 03 Introduction 04 Why Is Continuous Quality and Testing Maturity Important to DevOps? 05 Continuous Testing Engineers Quality into DevOps 07 Best Practices for Well- Engineered Continuous Testing 08 Continuous Testing Maturity Levels Level 1: Chaos Level 2: Continuous Integration Level 3: Continuous Flow Level 4: Continuous Feedback Level 5: Continuous Improvement 12 Continuous Testing Maturity Assessment 13 How to Get Started with DevOps Testing? 14 Continuous Testing in the Cloud Choosing the right tools for Continuous Testing On-demand Development and Testing Environments with Infrastructure as Code The Right Tests at the Right Time 20 Get Started 20 Conclusion 21 About AWS Marketplace and DevOps Institute 21 Contributors Introduction A successful DevOps implementation reduces the bottlenecks related to testing. These bottlenecks include finding and setting up test environments, test configurations, and test results implementation. These issues are not industry specific. They can be experienced in manufacturing, service businesses, and governments alike. They can be reduced by having a thorough understanding and a disciplined, mature implementation of Continuous Testing and related recommended engineering practices. The best place to start addressing these challenges is having a good understanding of what Continuous Testing is. Marc Hornbeek, the author of Engineering DevOps, describes it as: “A quality assessment strategy in which most tests are automated and integrated as a core and essential part of DevOps. Continuous Testing is much more than simply ‘automating tests.’” In this whitepaper, we’ll address the best practices you can adopt for implementing Continuous Quality and Testing on the AWS Cloud environment in the context of the DevOps model.
    [Show full text]
  • Devops Point of View an Enterprise Architecture Perspective
    DevOps Point of View An Enterprise Architecture perspective Amsterdam, 2020 Management summary “It is not the strongest of the species that survive, nor the most intelligent, but the one most responsive to change.”1 Setting the scene Goal of this Point of View In the current world of IT and the development of This point of view aims to create awareness around the IT-related products or services, companies from transformation towards the DevOps way of working, to enterprise level to smaller sizes are starting to help gain understanding what DevOps is, why you need it use the DevOps processes and methods as a part and what is needed to implement DevOps. of their day-to-day organization process. The goal is to reduce the time involved in all the An Enterprise Architecture perspective software development phases, to achieve greater Even though it is DevOps from an Enterprise Architecture application stability and faster development service line perspective, this material has been gathered cycles. from our experiences with customers, combined with However not only on the technical side of the knowledge from subject matter experts and theory from organization is DevOps changing the playing within and outside Deloitte. field, also an organizational change that involves merging development and operations teams is Targeted audience required with an hint of cultural changes. And last but not least the skillset of all people It is specifically for the people within Deloitte that want to involved is changing. use this as an accelerator for conversations and proposals & to get in contact with the people who have performed these type of projects.
    [Show full text]
  • Testing Guidelines
    Guidance Testing Guidelines Purpose This document provides guidelines for testing changes to the BSC Software, Systems and Processes. It also provides a high level view of test procedures that are followed by various parties involved in testing. The management of testing and test deliverables is detailed in the Test Management procedure (Reference 1). The scope of testing for each software release is detailed in the relevant Release Test Strategy document, which is developed by ELEXON and the relevant Service Providers. The scope is agreed by all test participants and captured in the Test Strategy. This document is specifically written as a guideline for testing the BSC Systems. However, the principles described may be used when planning for testing of other software systems. Contents 1. Testing Process Overview ...................................................................................................... 2 2. Application Manager and Developer Testing Overview ........................................................ 4 3. Business Process Operator Testing Overview ....................................................................... 9 4. Terms Used ............................................................................................................................ 13 5. References ............................................................................................................................. 13 6. Appendix A – Test Results ...................................................................................................
    [Show full text]