DOCSLIB.ORG

Detecting and Correcting Duplication in Behaviour Driven Development Specifications

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Detecting and Correcting Duplication in Behaviour Driven Development Specifications DETECTING AND CORRECTING DUPLICATION IN BEHAVIOUR DRIVEN DEVELOPMENT SPECIFICATIONS A THESIS SUBMITTED TO THE UNIVERSITY OF MANCHESTER FOR THE DEGREE OF DOCTOR OF PHILOSOPHY IN THE FACULTY OF SCIENCE AND ENGINEERING 2020 By Leonard Peter Binamungu School of Engineering, Department of Computer Science Contents Abstract 10 Declaration 12 Copyright 13 Acknowledgements 14 Dedication 16 1 Introduction 17 1.1 Behaviour Driven Development . 20 1.2 The Problem . 24 1.3 Hypothesis and Research Questions . 26 1.4 Thesis Contributions . 27 1.5 Publication Activity . 29 1.6 Thesis Organisation . 29 2 Literature Review 31 2.1 Terminology . 32 2.1.1 Terminology on Duplication . 32 2.1.2 Other Terminology . 35 2.2 Duplication in Software Systems . 35 2.2.1 Evidence of Duplication in Real World Systems . 35 2.2.2 Causes of Duplication in Software Systems . 36 2.2.3 Upsides and Downsides of Duplication in Software Systems . 37 2.3 Duplication Detection . 43 2.3.1 Static analysis approaches . 44 2 2.3.2 Dynamic analysis approaches . 46 2.3.3 Combination of static and dynamic analysis approaches . 47 2.4 Duplication Removal . 49 2.4.1 Refactoring of duplicates in production code . 50 2.4.2 Removing duplication in test code . 53 2.4.3 Test Suite Reduction . 54 2.4.4 Removing duplication in natural language tests . 55 2.4.5 Assessing the quality of tests and requirements . 56 2.5 Analysis and Improvement of BDD Specifications . 58 2.6 Research Gap . 63 2.7 Summary . 65 3 Challenges and Opportunities for Maintaining BDD Suites 68 3.1 Introduction . 68 3.2 Study Design . 69 3.2.1 Survey Design . 69 3.2.2 Recruitment of Respondents . 70 3.2.3 Survey Respondents and Data Analysis Approach . 72 3.2.4 Ethical Considerations . 75 3.3 Results . 76 3.3.1 RQ1. Extent of Active Use of BDD . 76 3.3.2 RQ2. Perceived Benefits and Challenges Involved in Using BDD 79 3.3.3 RQ3. Challenges of Maintaining BDD Specifications . 84 3.3.4 RQ4. Duplication in BDD Suites . 85 3.4 Discussion . 92 3.5 Research Opportunities . 96 3.6 Threats to Validity . 97 3.7 Summary . 99 4 Benchmark Development 101 4.1 Introduction . 101 4.2 Semantically Equivalent Scenarios . 101 4.3 Benchmark Development . 102 4.3.1 Context . 102 4.3.2 The Host Systems . 103 4.3.3 Duplicates between Original Scenarios . 105 3 4.3.4 Duplicate Injection . 106 4.4 Summary . 109 5 Detecting Duplicate Scenarios in BDD Suites 110 5.1 Introduction . 110 5.2 The Duplicate Detection Problem in BDD . 113 5.3 Existing Tools . 115 5.3.1 Tools Selection . 115 5.3.2 Tools Configuration and Experiment Setup . 116 5.3.3 Results and Discussion . 117 5.4 Detecting Duplicate Scenarios . 118 5.4.1 Hypothesis . 118 5.4.2 Overall Duplicate Detection Framework . 119 5.4.3 Definitions of Key Terms . 122 5.4.4 Using Default Traces to Detect Duplicate Scenarios . 123 5.4.5 Using Essential Traces to Detect Duplicate Scenarios . 125 5.4.6 Algorithm and Implementation . 129 5.5 Evaluation . 133 5.5.1 Experiment Design . 133 5.5.2 Results . 134 5.5.3 Discussion . 138 5.6 Summary . 140 6 Quality-Oriented Removal of Duplicate Scenarios in BDD Suites 142 6.1 Introduction . 142 6.2 BDD Suite Quality Principles . 145 6.2.1 Aspects of Quality in BDD Specifications . 146 6.2.2 Principle of Conservation of Steps . 148 6.2.3 Principle of Conservation of Domain Vocabulary . 149 6.2.4 Principle of Elimination of Technical Vocabulary . 149 6.2.5 Principle of Conservation of Proper Abstraction . 150 6.3 Community Support for the Quality Principles . 150 6.3.1 Survey Design . 151 6.3.2 Ethical Considerations . 152 6.3.3 Respondents and Data Analysis . 152 6.3.4 Results . 154 4 6.3.5 Conclusions from the Survey . 158 6.4 Operationalization of BDD Quality Principles . 161 6.4.1 Assessing Conservation of Steps . 161 6.4.2 Assessing Conservation of Domain Vocabulary . 162 6.4.3 Assessing Elimination of Technical Vocabulary . 163 6.4.4 Assessing Conservation of Proper Abstraction . 165 6.4.5 Implementation . 167 6.5 Evaluation . 169 6.5.1 Context and Research Questions . 169 6.5.2 Lab Experiment: Design, Results and Discussion . 172 6.5.3 Industry Experiment: Design, Results and Discussion . 179 6.6 Threats to Validity . 183 6.7 Summary . 185 7 Conclusions and Future Work 188 7.1 Summary of Research Contributions . 188 7.2 Answers to the Research Questions . 190 7.3 Future Work . 192 7.3.1 Future work on duplicate detection in BDD . 192 7.3.2 Future work on quality of BDD specifications . 193 7.3.3 Future work for BDD as a whole . 194 Bibliography 195 A A Survey on BDD Use by Software Engineering Teams and the Challenges of Duplication in BDD Specifications 215 A.1 Survey Questions . 215 A.2 Codes and Themes from the Challenges Reported by BDD Practitioners 217 B Sample Duplicate Report 228 B.1 Example duplication report . 228 C Sample Remove Suggestions 229 C.1 Sample remove suggestions . 229 Word Count: 56184 5 List of Tables 2.1 Example code fragments for different types of duplication in program code (adapted from Roy and Cordy [1]) . 33 2.2 Evidence, advantages, and disadvantages of duplication in software systems . 43 2.3 Summary of duplicate detection approaches . 50 2.4 Approaches for removing duplication, as well as assessing and improv- ing the quality of test suites and requirements . 58 3.1 Survey on BDD maintenance challenges: Distribution of respondents by continent . 73 3.2 Survey on BDD maintenance challenges: Distribution of respondents’ organisations . 73 3.3 Survey on BDD maintenance challenges: Distribution of job roles of respondents . 74 3.4 Benefits and challenges of using BDD . 81 3.5 Activeness of BDD use in industry . 93 3.6 Challenges and research opportunities for maintenance and evolution of BDD specifications . 98 4.1 Characteristics of selected evaluation software . 105 4.2 Number of duplicate scenario pairs between original scenarios . 106 4.3 Number of volunteers per system . 108 4.4 Distribution of duplicate scenario pairs across the three evaluation sys- tems . 108 5.1 Results of experimentation with existing clone detection tools . 118 5.2 Detailed results for the 8 different hypotheses on the 3 systems . 136 5.3 Comparison between our tool and the existing duplicate detection tools 138 6 6.1 BDD quality aspects from scientific and grey literature . 147 6.2 Survey on BDD quality principles: roles of respondents . 153 6.3 Survey on BDD quality principles: sizes of respondent organisations . 153 6.4 Other recommended practices for all project stakeholders on how to keep BDD specifications readable, easy to extend, and maintainable . 159 6.5 Other recommended practices for testers and developers on how to keep BDD specifications readable, easy to extend, and maintainable . 160 6.6 Performance of the four principles on the scenarios injected by each of the 13 volunteers . 177 6.7 Suggestions of injected scenarios for removal: combined algorithm . 178 6.8 Expert opinions: scenarios suggested for removal and the reasons . 181 A.1 Codes for the challenges facing respondent BDD practitioners . 218 A.2 BDD challenges: themes with related codes . 224 7 List of Figures 1.1 Typical TDD cycle (Source: [2]) . 19 1.2 Typical ATDD cycle (Source: [3]) . 20 1.3 Organisation of a typical BDD project . 25 2.1 Possible distribution of duplicates and non-duplicates in a software artefact (Adapted from Roy and Cordy [4]) . 34 2.2 Classification of the literature we surveyed . 66 3.1 Mapping between the research questions in section 3.1 and the ques- tions which are not about respondents’ demographics on the survey in Appendix A.1 . 71 3.2 Extent of BDD use . ..
Load more

Recommended publications
  • Methods & Tools
    METHODS & TOOLS Practical knowledge for the software developer, tester and project manager ISSN 1661-402X Spring 2011 (Volume 19 - number 1) www.methodsandtools.com The Salesmen/Developers Ratio at Software Vendors Many of us might think that software is a technological industry. Maybe. But maybe not. If you consider Oracle or Microsoft, I suppose that few of us would consider them as technology leaders, but we will all recognize their financial strength and marketing power. Long lasting organizations in the software industry might have more financial strength than technological capabilities. The recent conflict between Oracle and the creator of Hudson, an open source continuous integration server, is just another episode in the opposition between developers- and salesmen-driven software companies. On one side you have Oracle, for which Hudson is just small project inherited from the Sun buyout and that owns the "Hudson" brand. On the other side, you find Kohsuke Kawaguchi, who created Hudson and wants keep some control on its development. Hudson has been "forked", the open source code being used to start another project. You now have a Jenkins CI project that includes most of the active Hudson contributors and a Hudson CI project backed by Oracle and Sonatype, the commercial company behind the Maven project. Everything is however not black and white. Kohsuke Kawaguchi has joined Cloudbees, a company that has some management and financing coming from ex-JBoss managers, people that know how to make money with open source software. These people are not working hard for the only sake of technology evolution. You can judge the main orientation of a company looking at its salesmen/developers ratios.
    [Show full text]
  • Summer 2012 (Volume 20 - Number 2)
    METHODS & TOOLS Practical knowledge for the software developer, tester and project manager ISSN 1661-402X Summer 2012 (Volume 20 - number 2) www.methodsandtools.com Do Software Developers Want to Be Managers? According to surveys, Agile is now implemented by a majority of organizations and Scrum is the project management framework the most widely used. A characteristic of Scrum teams is that they are (or should be) self-organized. Team members are now responsible to divide between themselves the workload and they will need to deal with conflicts between team members. The former project manager authority and responsibilities are spread amongst the Scrum team. But do Agile developers really want this? Self-organization relies on the ability of team members to communicate and negotiate. The transition from a hierarchical team organization to a flat relationship where everybody participates is not easy and faces many challenges. Software developers have in majority an introverted personality type and this is not the best setting to interact with colleagues or even just to express needs or feelings. Teams might end us as just being organized by the most vocal members. Another implied hypothesis of self-organization is that all team members should have an equal weight in decisions, even if technical superiority or leadership might be recognized. This might not exist when you have teams that mix internal employees and contractors that might be denied authority. Self- organization also works in a culture of individual responsibility, affirmation and (positive) contradiction. This is not always the case in all organizations or countries. The team has now to deal with non-performing members, people that are either technically limited or that have a difficult personality.
    [Show full text]
  • {Download PDF} Use Cases Requirements in Context 2Nd Edition
    USE CASES REQUIREMENTS IN CONTEXT 2ND EDITION PDF, EPUB, EBOOK Daryl Kulak | --- | --- | --- | 9780321154989 | --- | --- R.S. Pressman & Associates, Inc. Managing Software Requirements, Second Edition , specifically addresses the ongoing challenge of managing change and describes a process for assuring that project scope is successfully defined and agreed upon by all stakeholders. Domain-Driven Design DDD software modeling delivers powerful results in practice, not just in theory, which is …. It is a very good book made …. Being a successful software architect is more than just possessing technical knowledge. Upon completion of an object-oriented design, you are faced with a troubling question: "Is it good, …. Skip to main content. Start your free trial. Book description "Many projects fail because developers fail to build the right thing. Topics covered include: The five steps in problem analysis Business modeling and system engineering Techniques for eliciting requirements from customers and stakeholders Establishing and managing project scope Applying and refining use cases Product management Transitioning from requirements to design and implementation Transitioning from use cases to test cases Agile requirements methods XB Show and hide more. Use case diagrams are typically developed in the early stage of development and people often apply use case modeling for the following purposes:. A standard form of use case diagram is defined in the Unified Modeling Language as shown in the Use Case Diagram example below:. Use cases share different kinds of relationships. Defining the relationship between two use cases is the decision of the software analysts of the use case diagram. A relationship between two use cases is basically modeling the dependency between the two use cases.
    [Show full text]
  • Software Requirements As Executable Code Karen Eileen Wasielewski Morand Regis University
    Regis University ePublications at Regis University All Regis University Theses Fall 2012 Software Requirements As Executable Code Karen Eileen Wasielewski Morand Regis University Follow this and additional works at: https://epublications.regis.edu/theses Part of the Computer Sciences Commons Recommended Citation Wasielewski Morand, Karen Eileen, "Software Requirements As Executable Code" (2012). All Regis University Theses. 232. https://epublications.regis.edu/theses/232 This Thesis - Open Access is brought to you for free and open access by ePublications at Regis University. It has been accepted for inclusion in All Regis University Theses by an authorized administrator of ePublications at Regis University. For more information, please contact [email protected]. Regis University College for Professional Studies Graduate Programs Final Project/Thesis Disclaimer Use of the materials available in the Regis University Thesis Collection (“Collection”) is limited and restricted to those users who agree to comply with the following terms of use. Regis University reserves the right to deny access to the Collection to any person who violates these terms of use or who seeks to or does alter, avoid or supersede the functional conditions, restrictions and limitations of the Collection. The site may be used only for lawful purposes. The user is solely responsible for knowing and adhering to any and all applicable laws, rules, and regulations relating or pertaining to use of the Collection. All content in this Collection is owned by and subject to the exclusive control of Regis University and the authors of the materials. It is available only for research purposes and may not be used in violation of copyright laws or for unlawful purposes.
    [Show full text]
  • Our Thoughts on the Technology and Trends That Are Shaping the Future
    TECHNOLOGY RADAR VOL.18 Our thoughts on the technology and trends that are shaping the future thoughtworks.com/radar #TWTechRadar CONTRIBUTORS The Technology Radar is prepared by the ThoughtWorks Technology Advisory Board, comprised of: Rebecca Parsons (CTO) | Martin Fowler (Chief Scientist) | Bharani Subramaniam | Camilla Crispim | Erik Doernenburg Evan Bottcher | Fausto de la Torre | Hao Xu | Ian Cartwright | James Lewis Jonny LeRoy | Ketan Padegaonkar | Lakshminarasimhan Sudarshan | Marco Valtas | Mike Mason Neal Ford | Rachel Laycock | Scott Shaw | Shangqi Liu | Zhamak Dehghani This edition of the ThoughtWorks Technology Radar based on a meeting of the Technology Advisory Board, which met in Sydney in March 2018 WHAT’S NEW? Highlighted themes in this edition: BROWSER BULKS UP, SERVER TRUST TEAMS BUT VERIFY SLIMS DOWN Security remains of paramount concern for virtually all The browser continues to expand its capabilities as a software development. But we see a shift in the traditional deployment target for application logic. As platforms take “lock everything down globally” approach to a more care of more cross-cutting concerns and nonfunctional nuanced, localized approach. Many systems now manage requirements, we see a trend toward reduced complexity trust within smaller domains and use modern mechanisms in back-end logic. The introduction of WebAssembly to create transitive trust between disparate systems. The opens up new language options to create logic for web philosophy has shifted from “never trust anything” outside applications and pushes processing closer to the metal of the domain and “never verify anything” inside the domain (and the GPU). Web Bluetooth enables browsers to handle to “trust teams but verify” everywhere — that is to say, functionality formerly reserved for native applications, assume well-intentioned interactions with other parts of and we increasingly see open standards such as CSS Grid the system but verify trust at the local level.
    [Show full text]
  • Concordion Environment Setup
    CCOONNCCOORRDDIIOONN -- EENNVVIIRROONNMMEENNTT http://www.tutorialspoint.com/concordion/concordion_environment.htm Copyright © tutorialspoint.com Here we will see how to prepare a development environment to make use of Concordion. Before using Concordion, you need to set up JDK, Tomcat, and Eclipse on your system. Let's go step by step. Step 1 - Setup Java Development Kit JDK You can download the latest version of JDK from Oracle's Java site: Java SE Downloads. You will find instructions for installing JDK in downloaded files. Just follow the given instructions to install and configure the setup. Finally set the PATH and JAVA_HOME environment variables to refer to the directory that contains java and javac, typically java_install_dir/bin and java_install_dir respectively. If you are running Windows and you have installed the JDK in C:\jdk1.7.0_75, you would have to put the following line in your C:\autoexec.bat file. set PATH=C:\jdk1.7.0_75\bin;%PATH% set JAVA_HOME=C:\jdk1.7.0_75 Alternatively, on Windows NT/2000/XP, you could also right-click on My Computer, select Properties, then Advanced, and Environment Variables. Then, you would update the PATH value and press the OK button. On Unix Solaris, Linux, etc. , if the SDK is installed in /usr/local/jdk1.7.0_75 and you use the C shell, you would put the following into your .cshrc file. setenv PATH /usr/local/jdk1.7.0_75/bin:$PATH setenv JAVA_HOME /usr/local/jdk1.7.0_75 Alternatively, if you are using an Integrated Development Environment IDE like Borland JBuilder, Eclipse, IntelliJ IDEA, or Sun ONE Studio, then compile and run a simple program to confirm that the IDE knows where you installed Java, otherwise do proper setup as given in the document of the IDE.
    [Show full text]
  • Arxiv:2102.11678V2 [Cs.SE] 30 Jul 2021 Mostly Used
    Computing and Informatics, Vol. 32, 2013, 1001{1030, V 2021-Aug-2 AUTOMATING TEST CASE IDENTIFICATION IN JAVA OPEN SOURCE PROJECTS ON GITHUB Matej Madeja, Jaroslav Poruban¨ , Michaela Bacˇ´ıkova´ Department of Computers and Informatics Faculty of Electrical Engineering and Informatics Technical University of Koˇsice, Letn´a9, 042 00 Koˇsice, Slovakia e-mail: [email protected] Mat´uˇs Sul´ır, J´an Juhar´ , Sergej Chodarev, Filip Gurba´ˇl Department of Computers and Informatics Faculty of Electrical Engineering and Informatics Technical University of Koˇsice, Letn´a9, 042 00 Koˇsice, Slovakia e-mail: [email protected] Abstract. Software testing is one of the very important Quality Assurance (QA) components. A lot of researchers deal with the testing process in terms of tester motivation and how tests should or should not be written. However, it is not known from the recommendations how the tests are written in real projects. In this paper, the following was investigated: (i) the denotation of the word \test" in different natural languages; (ii) whether the number of occurrences of the word \test" correlates with the number of test cases; and (iii) what testing frameworks are arXiv:2102.11678v2 [cs.SE] 30 Jul 2021 mostly used. The analysis was performed on 38 GitHub open source repositories thoroughly selected from the set of 4.3M GitHub projects. We analyzed 20,340 test cases in 803 classes manually and 170k classes using an automated approach. The results show that: (i) there exists a weak correlation (r = 0:655) between the number of occurrences of the word \test" and the number of test cases in a class; (ii) the proposed algorithm using static file analysis correctly detected 97% of test cases; (iii) 15% of the analyzed classes used main() function whose represent regular Java programs that test the production code without using any third-party framework.
    [Show full text]
  • Concordion Quick Guide
    CCOONNCCOORRDDIIOONN -- QQUUIICCKK GGUUIIDDEE http://www.tutorialspoint.com/concordion/concordion_quick_guide.htm Copyright © tutorialspoint.com CCOONNCCOORRDDIIOONN -- OOVVEERRVVIIEEWW What is Concordion? Concordion is a powerful tool to write and manage automated acceptance tests in Java based projects. It directly integrates with JUnit framework, making it ready to be used with all popular JAVA based IDEs like Netbeans, Eclipse, IntelliJ IDEA. Active software specification Active software specification is a way to specify the behaviour of a feature. It also provides a way to implement and verify the software specification by having a connection with the system under development. An active specification in Concordion is of two parts: 1. A cleanly written requirement document which describes the desired functionality written using XHTML. The XHTML based specifications contain descriptions of the functionality provided with acceptance test examples. Example's data is marked using simple HTML tags. 2. Acceptance tests are written in Java language called fixture code. Using a Concordion extension of a standard JUnit test case, test are implemented. It is the responsiblity of Fixture Code to find the example's data marked by tag and use them to verify the software under development. Output of Cncordion When Concordion active specification tests are run, outpu XHTML files shows the original specification and test results. Successful test are highlighted using "green" color and failed test are highlighted using "red". As any change in system will result in failing the test, which helps to identify that specifiction are to be updated, Concordion terms these specifications as active specifications. Therefore specifications are always up-to-date. Key Features Following are the key features of Concordion: Specifications as documentation - Concordion specification being highly readable can be used as active system documentation.
    [Show full text]
  • Java Convert Markdown to Pdf
    Java convert markdown to pdf Continue Curated a list of delightful things Markdown. Markdown is a marking language created by John Gruber in collaboration with Aaron Schwartz. It was designed to be easily converted to HTML and has been expanded to different types of implementations. Please read the recommendations on contributions before you contribute. means really awesome/helpful. means it's a web application/web service. means that works on Android. means it works on iOS. means it works on macOS. means that works on Linux. means that works on Windows. Content CMS / CMS Blogs or blogs that support Markdown by default. Ghost - Publishing platform for professional bloggers. Grav - Modern flat CMS file. Hexo - Fast, simple and powerful blog framework Hugo - Fast and modern static engine website. Jekyll - Turn your simple text into static websites and blogs. Pelican is a static site generator that does not require a database or logic on the server side. Svbtle - Blog platform is designed to help you think. Vuepress is a minimalist vue generator of a static site. Libraries libraries to visualize Markdown documents. C cmark - Analysis and visualization of commonMark Library and Program in C. Hoedown - Standard, Fast, Safe Markdown Processing Library in C (Sundown Fork). peg-markdown - Implementation of Markdown in C using PEG grammar. Sundown - Standards compatible, fast, secure Markdown Processing Libraries in C. Java commonmark-java - Java implementation CommonMark, Markdown format specification. flexmark-java - Java Markdown review implementation with the original AST level. pegdown is a Pure-Java Markdown processor based on the PEG parser, which supports a number of extensions.
    [Show full text]
  • Tdd, Bdd, and Atdd) to the Software Life Cycle
    Dissertation Master in Computer Engineering – Mobile Computing ANALYSIS OF THE IMPACT OF TEST BASED DEVELOPMENT TECHNIQUES (TDD, BDD, AND ATDD) TO THE SOFTWARE LIFE CYCLE Luis Alberto Cisneros Gómez Leiria, September 2018 This page was intentionally left blank Dissertation Master in Computer Engineering – Mobile Computing ANALYSIS OF THE IMPACT OF TEST BASED DEVELOPMENT TECHNIQUES (TDD, BDD, AND ATDD) TO THE SOFTWARE LIFE CYCLE Luis Alberto Cisneros Gómez Dissertation developed under the supervision of Catarina Isabel Ferreira Viveiros Tavares dos Reis, Professor at the School of Technology and Management of the Polytechnic Institute of Leiria and co-supervision of Marisa da Silva Maximiano, Professor at the School of Technology and Management of the Polytechnic Institute of Leiria. Leiria, September 2018 This page was intentionally left blank ii Acknowledgements First at all, I want to thank to God for giving me intelligence, fortitude and illuminating this academic progress. To my mother, Kevin, my grandmother and my sisters; who have given me their unconditional affection and, always have been supporting me to continue forward in the more difficult moments of my life. At the Polytechnic of Leiria for offering me a space where I could complete my master´s degree studies. To my thesis supervisors: Catarina Reis PhD. and, Marisa Maximiano PhD., who have collaborated in the development of my work. To my thesis coordinator, Antonio Quiña MsC., who allowed me to execute the experimentation in the Universidad Técnica del Norte. To Filipe Mota Pinto PhD., for his orientation to decide my specialization studies. To Oscar Dieste PhD., for his cooperation in the planning of the experiment.
    [Show full text]
  • Download File
    Unit Test Virtualization with VMVM Jonathan Bell Gail Kaiser Columbia University Columbia University 500 West 120th St, MC 0401 500 West 120th St, MC 0401 New York, NY USA New York, NY USA [email protected] [email protected] ABSTRACT To cope with long running test suites, testers might turn Testing large software packages can become very time in- to Test Suite Minimization or Test Suite Prioritization [43]. tensive. To address this problem, researchers have inves- Test Suite Minimization techniques such as [14, 15, 22, 23, tigated techniques such as Test Suite Minimization. Test 27, 28, 38, 41] reduce the total number of tests to execute Suite Minimization reduces the number of tests in a suite by identifying duplicate tests. However, identifying which by removing tests that appear redundant, but risks a reduc- tests are duplicates is hard, and Test Suite Minimization tion in fault-finding ability since it can be difficult to identify approaches typically rely on coverage measures to identify which tests are truly redundant. We take a completely dif- overlap, which may not be completely accurate. Addition- ferent approach to solving the same problem of long running ally, Test Suite Minimization is an NP-complete problem test suites by instead reducing the time needed to execute [23], and therefore existing algorithms rely on heuristics. each test, an approach that we call Unit Test Virtualiza- Test Suite Prioritization techniques such as [18,19,36,37,40] tion. We describe the empirical analysis that we performed re-order test cases, for example so that given the set of to ground our approach and provide an implementation of changes to the application since the last test execution, the Unit Test Virtualization targeting Java applications.
    [Show full text]
  • Viktor Österholm
    Overview of Behaviour-Driven Development tools for web applications Viktor Österholm MSc in Computer Science Åbo Akademi University Faculty of Science and Engineering Department of Information Technologies May 2021 Supervisor: Dragos Truscan 1 Abstract Testing is an important part of software engineering. To avoid failure, software needs to be tested to ensure that it fulfils the requirements and that it is as defect-free as possible. Additionally, software needs to help the customer reach their goals, such as gaining a competitive advantage, launching a new product or feature, etc. The goal of Behaviour-Driven Development (BDD) is to help software development teams deliver highly valuable, high-quality software faster. BDD focuses on close customer interaction and writing tests based on scenarios before the code is written. The goal of the thesis was to help software development teams choose the correct BDD tool for developing and testing web applications, as choosing the wrong testing tool can be time-consuming and costly. The thesis compared three testing tools that can be used for BDD: Robot Framework, Behave (the Python port of Cucumber), and Concordion. The tools were compared based on 10 questions within five categories that were found to be important to testers when choosing a testing tool. To help evaluate the tools, a test case was implemented on a web application. The assessment found that both Robot Framework and Cucumber are good choices for BDD for web applications, as they are popular, well-supported, and relatively easy to use. Concordion was less popular and harder to use due to a lack of learning resources.
    [Show full text]