D5.06 Test Strategy and Environment

Total Page:16

File Type:pdf, Size:1020Kb

D5.06 Test Strategy and Environment Ref. Ares(2018)3343807 - 25/06/2018 Title: Test Strategy and Environment Abstract Multi-cloud Execution-ware This document presents a strategy for testing, acceptance criteria, for Large-scale Optimized test related products and responsibilities related to quality Data -Intensive Computing assurance tasks in Melodic. The document should be use as a H2020-ICT-2016-2017 guideline for performing all activities within the testing process. Leadership in Enabling and Also, it should be used in conjunction with the 'D5.10 Quality Industrial Technologies; Information and Assurance Guide' deliverable, which contains detailed Communication Technologies descriptions on how to accomplish test related tasks and should be Grant Agreement No.: use as a manual for those tasks. The deliverable's intended 731664 audiences is as follows: It could be used by all participants in the Duration: project. Especially it should be use by development teams (the 1 December 2016 - unit/integration tests and the bugs definition), test teams (the 30 November 2019 whole document), architects (bugs and test cases processes), use www.melodic.cloud case application users (test reporting and bugs handling) and Deliverable reference: managers (management of the releases and the testing process). D5.6 In detail, this document defines the test strategy, and describes Date: various levels of testing (e.g. unit testing, integration testing, 31 May 2018 functional testing, and non-functional testing). The next main Responsible partner: topic covered is the environments configurations, including their 7bulls purposes, followed by test process descriptions. For the test Editor(s): process descriptions, the document explains the process of testing, Katarzyna Materka how to use it in the project, the purpose of the process, and where Author(s): to find more information. Furthermore, this document describes Katarzyna Materka, Michał Semczuk test related products delivered within the Melodic project, including a Test Plan, Test Cases with scenarios, and a Test Report. Approved by: Ernst Gunnar Gran Finally, the document covers communication and responsibilities ISBN Number: related to quality assurance in the project. Altogether, this N/A deliverable should represent a complete and comprehensive guide Document URL: for the most important quality assurance tasks with regards to http://www.melodic.cloud/ testing in the project. deliverables/D5.6 Test Strategy and Environment.pdf This project has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 731664 www.melodic.cloud Deliverable reference: Editor(s): D5.6 Katarzyna Materka Document Period Covered M6-12 Deliverable No. D5.06 Deliverable Title Test Strategy and Environment Editor(s) Katarzyna Materka Author(s) Katarzyna Materka, Michał Semczuk Reviewer(s) Kyriakos Kritikos, Ioannis Patiniotakis Work Package No. 5 Work Package Title Integration and security Lead Beneficiary 7bulls Distribution PU Version Final Draft/Final Final Total No. of Pages 35 This project has received funding from the European Union’s Horizon 2020 www.melodic.cloud 2 research and innovation programme under grant agreement No 731664 Deliverable reference: Editor(s): D5.6 Katarzyna Materka Table of Contents 1 Introduction ................................................................................................................................................. 6 1.1 The purpose and Objective ................................................................................................................ 6 1.1.1 Purpose ................................................................................................................................................. 6 1.1.2 Objective ............................................................................................................................................... 7 1.2 Assumptions ........................................................................................................................................... 7 1.3 Intended Audience ............................................................................................................................... 8 1.4 Document Structure ............................................................................................................................. 8 2 Test Strategy ............................................................................................................................................. 10 2.1 Definition of Project Testing .......................................................................................................... 10 2.2 Planning of Test Activities .............................................................................................................. 12 2.2.1 Development .................................................................................................................................... 12 2.2.2 Acceptance Test .............................................................................................................................. 12 2.2.3 Scope of Testing .............................................................................................................................. 12 3 Levels of Testing ...................................................................................................................................... 13 3.1 Unit Testing ........................................................................................................................................... 13 3.1.1 Definition ........................................................................................................................................... 13 3.1.2 Objectives .......................................................................................................................................... 14 3.1.3 Tools ..................................................................................................................................................... 14 3.2 Smoke Testing ..................................................................................................................................... 15 3.2.1 Definition ........................................................................................................................................... 15 3.2.2 Objectives .......................................................................................................................................... 15 3.2.3 Tools ..................................................................................................................................................... 15 3.3 Functional Testing ............................................................................................................................. 16 3.3.1 Definition ........................................................................................................................................... 16 3.3.2 Objectives .......................................................................................................................................... 17 3.3.3 Tools ..................................................................................................................................................... 18 3.4 Regression Testing ............................................................................................................................. 18 This project has received funding from the European Union’s Horizon 2020 www.melodic.cloud 3 research and innovation programme under grant agreement No 731664 Deliverable reference: Editor(s): D5.6 Katarzyna Materka 3.4.1 Definition ........................................................................................................................................... 18 3.4.2 Objectives .......................................................................................................................................... 19 3.4.3 Tools ..................................................................................................................................................... 19 3.5 Non-Functional Testing ................................................................................................................... 19 3.5.1 Definition ........................................................................................................................................... 19 3.5.2 Objectives .......................................................................................................................................... 20 4 Test Environments ................................................................................................................................. 21 4.1 Development environment (Unit Testing Environment) ................................................... 21 4.2 Integration environment (Smoke Testing Environment) .................................................. 21 4.3 Acceptance environment (Functional Test Environment) ................................................ 22 4.4 Non-Functional Test Environment – optional ....................................................................... 22 5 Test Process ............................................................................................................................................... 23 5.1 Test cases and test scenario relations ....................................................................................... 24 5.2
Recommended publications
  • Smoke Testing
    International Journal of Scientific and Research Publications, Volume 4, Issue 2, February 2014 1 ISSN 2250-3153 Smoke Testing Vinod Kumar Chauhan Quality Assurance (QA), Impetus InfoTech Pvt. Ltd. (India) Abstract- Smoke testing is an end-to-end testing which determine the stability of new build by checking the crucial functionality of the application under test and used as criteria of accepting the new build for detailed testing. Index Terms- Software testing, acceptance testing, agile, build, regression testing, sanity testing I. INTRODUCTION The purpose of this research paper is to give details of smoke testing from the industry practices. II. SMOKE TESTING The purpose of smoke testing is to determine whether the new software build is stable or not so that the build could be used for detailed testing by the QA team and further work by the development team. If the build is stable i.e. the smoke test passes then build could be used by the QA and development team. On the stable build QA team performs functional testing for the newly added features/functionality and then performs regression testing depending upon the situation. But if the build is not stable i.e. the smoke fails then the build is rejected to the development team to fix the build issues and create a new build. Smoke Testing is generally done by the QA team but in certain situations, can be done by the development team. In that case the development team checks the stability of the build and deploy to QA only if the build is stable. In other case, whenever there is new build deployment to the QA environment then the team first performs the smoke test on the build and depending upon the results of smoke the decision to accept or reject the build is taken.
    [Show full text]
  • Types of Software Testing
    Types of Software Testing We would be glad to have feedback from you. Drop us a line, whether it is a comment, a question, a work proposition or just a hello. You can use either the form below or the contact details on the rightt. Contact details [email protected] +91 811 386 5000 1 Software testing is the way of assessing a software product to distinguish contrasts between given information and expected result. Additionally, to evaluate the characteristic of a product. The testing process evaluates the quality of the software. You know what testing does. No need to explain further. But, are you aware of types of testing. It’s indeed a sea. But before we get to the types, let’s have a look at the standards that needs to be maintained. Standards of Testing The entire test should meet the user prerequisites. Exhaustive testing isn’t conceivable. As we require the ideal quantity of testing in view of the risk evaluation of the application. The entire test to be directed ought to be arranged before executing it. It follows 80/20 rule which expresses that 80% of defects originates from 20% of program parts. Start testing with little parts and extend it to broad components. Software testers know about the different sorts of Software Testing. In this article, we have incorporated majorly all types of software testing which testers, developers, and QA reams more often use in their everyday testing life. Let’s understand them!!! Black box Testing The black box testing is a category of strategy that disregards the interior component of the framework and spotlights on the output created against any input and performance of the system.
    [Show full text]
  • Smoke Testing What Is Smoke Testing?
    Smoke Testing What is Smoke Testing? Smoke testing is the initial testing process exercised to check whether the software under test is ready/stable for further testing. The term ‘Smoke Testing’ is came from the hardware testing, in the hardware testing initial pass is done to check if it did not catch the fire or smoked in the initial switch on.Prior to start Smoke testing few test cases need to created once to use for smoke testing. These test cases are executed prior to start actual testing to checkcritical functionalities of the program is working fine. This set of test cases written such a way that all functionality is verified but not in deep. The objective is not to perform exhaustive testing, the tester need check the navigation’s & adding simple things, tester needs to ask simple questions “Can tester able to access software application?”, “Does user navigates from one window to other?”, “Check that the GUI is responsive” etc. Here are graphical representation of Smoke testing & Sanity testing in software testing: Smoke Sanity Testing Diagram The test cases can be executed manually or automated; this depends upon the project requirements. In this types of testing mainly focus on the important functionality of application, tester do not care about detailed testing of each software component, this can be cover in the further testing of application. The Smoke testing is typically executed by testers after every build is received for checking the build is in testable condition. This type of testing is applicable in the Integration Testing, System Testing and Acceptance Testing levels.
    [Show full text]
  • Leading Practice: Test Strategy and Approach in Agile Projects
    CA SERVICES | LEADING PRACTICE Leading Practice: Test Strategy and Approach in Agile Projects Abstract This document provides best practices on how to strategize testing CA Project and Portfolio Management (CA PPM) in an agile project. The document does not include specific test cases; the list of test cases and steps for each test case are provided in a separate document. This document should be used by the agile project team that is planning the testing activities, and by end users who perform user acceptance testing (UAT). Concepts Concept Description Test Approach Defines testing strategy, roles and responsibilities of various team members, and test types. Testing Environments Outlines which testing is carried out in which environment. Testing Automation and Tools Addresses test management and automation tools required for test execution. Risk Analysis Defines the approach for risk identification and plans to mitigate risks as well as a contingency plan. Test Planning and Execution Defines the approach to plan the test cases, test scripts, and execution. Review and Approval Lists individuals who should review, approve and sign off on test results. Test Approach The test approach defines testing strategy, roles and responsibilities of various team members, and the test types. The first step is to define the testing strategy. It should describe how and when the testing will be conducted, who will do the testing, the type of testing being conducted, features being tested, environment(s) where the testing takes place, what testing tools are used, and how are defects tracked and managed. The testing strategy should be prepared by the agile core team.
    [Show full text]
  • Opportunities and Open Problems for Static and Dynamic Program Analysis Mark Harman∗, Peter O’Hearn∗ ∗Facebook London and University College London, UK
    1 From Start-ups to Scale-ups: Opportunities and Open Problems for Static and Dynamic Program Analysis Mark Harman∗, Peter O’Hearn∗ ∗Facebook London and University College London, UK Abstract—This paper1 describes some of the challenges and research questions that target the most productive intersection opportunities when deploying static and dynamic analysis at we have yet witnessed: that between exciting, intellectually scale, drawing on the authors’ experience with the Infer and challenging science, and real-world deployment impact. Sapienz Technologies at Facebook, each of which started life as a research-led start-up that was subsequently deployed at scale, Many industrialists have perhaps tended to regard it unlikely impacting billions of people worldwide. that much academic work will prove relevant to their most The paper identifies open problems that have yet to receive pressing industrial concerns. On the other hand, it is not significant attention from the scientific community, yet which uncommon for academic and scientific researchers to believe have potential for profound real world impact, formulating these that most of the problems faced by industrialists are either as research questions that, we believe, are ripe for exploration and that would make excellent topics for research projects. boring, tedious or scientifically uninteresting. This sociological phenomenon has led to a great deal of miscommunication between the academic and industrial sectors. I. INTRODUCTION We hope that we can make a small contribution by focusing on the intersection of challenging and interesting scientific How do we transition research on static and dynamic problems with pressing industrial deployment needs. Our aim analysis techniques from the testing and verification research is to move the debate beyond relatively unhelpful observations communities to industrial practice? Many have asked this we have typically encountered in, for example, conference question, and others related to it.
    [Show full text]
  • Exploring Languages with Interpreters and Functional Programming Chapter 11
    Exploring Languages with Interpreters and Functional Programming Chapter 11 H. Conrad Cunningham 24 January 2019 Contents 11 Software Testing Concepts 2 11.1 Chapter Introduction . .2 11.2 Software Requirements Specification . .2 11.3 What is Software Testing? . .3 11.4 Goals of Testing . .3 11.5 Dimensions of Testing . .3 11.5.1 Testing levels . .4 11.5.2 Testing methods . .6 11.5.2.1 Black-box testing . .6 11.5.2.2 White-box testing . .8 11.5.2.3 Gray-box testing . .9 11.5.2.4 Ad hoc testing . .9 11.5.3 Testing types . .9 11.5.4 Combining levels, methods, and types . 10 11.6 Aside: Test-Driven Development . 10 11.7 Principles for Test Automation . 12 11.8 What Next? . 15 11.9 Exercises . 15 11.10Acknowledgements . 15 11.11References . 16 11.12Terms and Concepts . 17 Copyright (C) 2018, H. Conrad Cunningham Professor of Computer and Information Science University of Mississippi 211 Weir Hall P.O. Box 1848 University, MS 38677 1 (662) 915-5358 Browser Advisory: The HTML version of this textbook requires a browser that supports the display of MathML. A good choice as of October 2018 is a recent version of Firefox from Mozilla. 2 11 Software Testing Concepts 11.1 Chapter Introduction The goal of this chapter is to survey the important concepts, terminology, and techniques of software testing in general. The next chapter illustrates these techniques by manually constructing test scripts for Haskell functions and modules. 11.2 Software Requirements Specification The purpose of a software development project is to meet particular needs and expectations of the project’s stakeholders.
    [Show full text]
  • Infotek Solutions Inc
    Infotek Solutions Inc. Test Strategy Test Strategy is summarize or outline which describes the approach of software development cycle. Or The test levels to be performed in testing and the description of testing activities within those test levels are known as test strategy. It is created to inform project managers, testers and developers about some key points of testing process. Strategy also includes testing objective, methods to testing new functions, time and resource what will be used in project and testing environment. Test strategies describe how to mitigate the product risk at test level, which of types of test are to be performed in testing and which entry and exit criteria apply. Different Testing strategies may be adopted depending on the type of system to be test and the development process used. In this document we are going to discuss testing strategies on two different types of software testing. 1. Conventional Software 2. Object Oriented Software(OO software) Conventional Software Testing: It is traditional approach. It takes place mostly when water fall life cycle is used for development. Conventional software Testing focuses more on decomposition. Conventional software testing always takes place during the test phase of life cycle, which usually follows the development phase and proceeds the implementation phase. During the conventional testing phase, Three types of testing will be conducted 1. System Testing 2. Integration Testing 3. Unit Testing Object Oriented Software Testing: Using Object Oriented design or design along with Agile development methodology leads to Object Oriented Software Testing. Object Oriented testing is done having emphasis on Composition.
    [Show full text]
  • Agile Test Automation Strategy, V2.Pptx
    MJ Half-day Tutorials 10/3/16 13:00 Test Automation Strategies for the Agile World Presented by: Bob Galen Velocity Partners Brought to you by: 350 Corporate Way, Suite 400, Orange Park, FL 32073 888---268---8770 ·· 904---278---0524 - [email protected] - http://www.starwest.techwell.com/ Bob Galen Velocity Partners An agile methodologist, practitioner, and coach, Bob Galen ([email protected]) helps guide companies in their adoption of Scrum and other agile methodologies and practices. Bob is a principal agile evangelist at Velocity Partners; president of RGCG; and frequent speaker on software development, project management, software testing, and team leadership. He is a Certified Scrum Coach, Certified Scrum Product Owner, and an active member of the Agile and Scrum Alliances. Bob published Scrum Product Ownership–Balancing Value from the Inside Out. Test Automation Strategies for the Agile World Bob Galen President & Principal Consultant RGCG, LLC [email protected] Introduction Bob Galen n Independent Agile Coach (CEC) at RGCG, LLC n Director, Agile Practices at n Somewhere ‘north’ of 30 years overall experience J n Wide variety of technical stacks and business domains n Developer first, then Project Management / Leadership, then Testing n Senior/Executive software development leadership for 20+ years n Practicing formal agility since 2000 n XP, Lean, Scrum, and Kanban experience n From Cary, North Carolina Bias Disclaimer: Agile is THE BEST Methodology for Software Development… However, NOT a Silver Bullet! Copyright © 2016 RGCG, LLC 2 1 Outline n Traditional Automation – Business Case & ROI n 3-Pillars n Agile Test Automation Pyramid n Agile Automation – Business Case & ROI n Implementation Strategy n Communication n Wrap-up Copyright © 2016 RGCG, LLC 3 Let’s start with… Traditional Automation Strategy n What are your current strategies towards: q Test Automation q Frameworks q Tooling q Maintenance q ROI q Team structure n Get together in “pairs” and chat about this for 20 minutes.
    [Show full text]
  • Beginners Guide to Software Testing
    Beginners Guide To Software Testing Beginners Guide To Software Testing - Padmini C Page 1 Beginners Guide To Software Testing Table of Contents: 1. Overview ........................................................................................................ 5 The Big Picture ............................................................................................... 5 What is software? Why should it be tested? ................................................. 6 What is Quality? How important is it? ........................................................... 6 What exactly does a software tester do? ...................................................... 7 What makes a good tester? ........................................................................... 8 Guidelines for new testers ............................................................................. 9 2. Introduction .................................................................................................. 11 Software Life Cycle ....................................................................................... 11 Various Life Cycle Models ............................................................................ 12 Software Testing Life Cycle .......................................................................... 13 What is a bug? Why do bugs occur? ............................................................ 15 Bug Life Cycle ............................................................................................... 16 Cost of fixing bugs .......................................................................................
    [Show full text]
  • Getting to Continuous Testing
    T2 Continuous Testing Thursday, October 3rd, 2019 9:45 AM Getting to Continuous Testing Presented by: Max Saperstone Coveros Brought to you by: 888---268---8770 ·· 904---278---0524 - [email protected] - http://www.starwest.techwell.com/ Max Saperstone Max Saperstone has been working as a software and test engineer for over a decade, with a focus on test automation within the CI/CD process. He specializes in open source tools, including the Selenium tool suite, JMeter, AutoIT, Cucumber, and Chef. Max has led several test automation efforts, including developing an automated suite focusing on web-based software to operate over several applications for Kronos Federal. He also headed a project with Delta Dental, developing an automated testing structure to run Cucumber tests over multiple test interfaces and environments, while also developing a system to keep test data "ageless." He currently heads up the development of Selenified, an open source testing framework to allow for testing of multiple interfaces, custom reporting, and minimal test upkeep. Getting to Continuous Testing A Greenfield Test Automation Story [email protected] maxsaperstone @Automate_Tests Max Saperstone @Automate_Tests 1 Max Saperstone @Automate_Tests Max Saperstone • Director of Testing and Automation at Coveros • Over a decade of test automation experience • Often Test Architect on consulting projects • Helped transform multiple organizations to test effectively within sprints • Mainly focuses on open source tools • Lover of the outdoors and cheap beer Max
    [Show full text]
  • Test Plan and Test Strategy Document
    Test Plan And Test Strategy Document Aamir never admixes any saccharoses untuned ava, is Adlai clever and heteronomous enough? Minute DomenicoDionysus fables concenter his dissemblances quite unexceptionably unbent unadvisedly. but osculating Bold her Tedd cline stillmysteriously. hurtled: ungenuine and sizzling What is a test strategy? The Test Strategy may also describe the test levels to be carried out and it should give guidance on the entry and exit criteria for each level, but also on the relationships among levels. Verify if the login button is displaying or not. This picture will show whenever you leave a comment. Your PDF request was successfully submitted. While testing software projects, certain test cases will be treated as the most important ones, and if they fail the product cannot be released. Standards must be tailored to meet your needs. We were unable to process your PDF request. This data must be available to the test leader and the project manager, along with all the team members, in a central location. It should also list steps taken to mitigate any risks that arise from skipping regression testing, like releasing a maintenance patch after two weeks. The fundamental difference between test strategy and test plan is their scope. In my career, I have mostly operated without written test plans. The product and principles of defects in the activities and pass and test plan is to prepare a detailed and. Go into as much detail as possible. Sorry, your blog cannot share posts by email. Proper and effective documentation work plays a vital role in carrying out any sort of process, either development or testing in an efficient manner for its success.
    [Show full text]
  • An Automated Testing Strategy for Efficient Use in the Consultancy
    Master Thesis Software Engineering Thesis no: MSE-2007:18 June 2007 An automated testing strategy targeted for efficient use in the consulting domain Teddie Stenvi School of Engineering Blekinge Institute of Technology Box 520 SE – 372 25 Ronneby Sweden This thesis is submitted to the School of Engineering at Blekinge Institute of Technology in Master Thesis partial fulfillment of the requirements for the degree of Master of Science in Software Software Engineering Engineering. The thesis is equivalent to 20 weeks of full time studies. Thesis no: MSE-2007-xx Month Year Contact Information: Author(s): Teddie Stenvi Address: Jaktstigen 18 22652 Lund E-mail: [email protected] External advisor(s): Per Sigurdson Testway AB Address: Hans Michelsengatan 9 211 20 Malmö University advisor(s): Dr. Robert Feldt Department of Systems and Software Engineering School of Engineering Internet : www.bth.se/tek School of Engineering Blekinge Institute of Technology Phone : +46 457 38 50 00 Blekinge Institute of Technology Box 520 Fax : + 46 457 271 25 Box 520 SE – 372 25 Ronneby SE – 372 25 Ronneby Sweden Sweden ii ABSTRACT Test automation can decrease release cycle time for software systems compared to manual test execution. Manual test execution is also considered inefficient and error-prone. However, few companies have gotten far within the field of test automation. This thesis investigates how testing and test automation is conducted in a test consulting setting. It has been recognized that low test process maturity is common in customer projects and this has led to equally low system testability and stability. The study started with a literature survey which summarized the current state within the field of automated testing.
    [Show full text]