DOCSLIB.ORG

The Communication Problem Solved with Cucumber

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
The Communication Problem Solved with Cucumber The Communication Problem Solved with Cucumber Mary Thorn About me(Mary Thorn) l QA Manager at iContact. l Over 15 years of QA experience working in the following areas: Healthcare, HR, Agriculture and SaaS-based products. l Have a broad testing background that spans automation, data warehouses to web-based systems in a wide variety of technologies and testing techniques. l Heavy interest in Agile Testing Methodologies and direct experience leading Agile teams in the SCRUM Software Development Lifecycle (SDLC). l 5 years of experience with building automation frameworks Agenda l Introduction l The Communication Problem l What are Acceptance Tests? l Simple Example l Acceptance Testing with Cucumber Acceptance Criteria for Presentation Goal: Presenter would like to have a successful presentation Acceptance Criteria: 1. Verify the attendees stay awake 2. Verify the attendees are happy with the presentation The Communication Problem The Challenge: l Today, the most difficult problem in software development is knowing what to build, not how to build it. l Knowing what to build, requires knowing why it is being built. In other words, understanding the goal. The Communication Problem The Problem: l Stories typically concentrate on the what and the how, not the why. Goals and examples are usually missing. l Stories are usually a list of imperative requirements (acceptance criteria). l Imperative requirements are usually interpreted in subtly different ways by each person. If goals and examples are absent, misinterpretation is much more likely. The Communication Problem Example: l Requirement: Draw a star with 10 points. OR The Communication Problem Solution: Specification by Example l People understand requirements best using concrete examples. l Examples clear up ambiguity and misunderstandings. l Examples expose missing requirements. l Examples force everyone to think harder about a problem. The Communication Problem Specification by Example l This is the basis of acceptance testing and Behavior Driven Design (BDD). l It is critical that everyone participates in creating the examples so that the understanding is shared. l Ultimately, acceptance testing is less about the example itself and more about the conversation required to create the example. What does this have to do with automated testing? What are Automated Acceptance Tests? In a nutshell, automated acceptance tests are executable examples that define the system's specifications. What are Automated Acceptance Tests? Acceptance Tests are l Formal examples that define requirements. l Executable. l Self-explanatory and readable by all team members. l Tests of business rules. l A living specification of system behavior. What are Automated Acceptance Tests? Acceptance Tests do NOT l Test pieces of code in isolation. They are not a substitute for unit and integration tests. l Verify the entire system. l Verify all permutations of a piece of functionality. l Use mock objects except to remove a dependency on an external 3rd party. What are Automated Acceptance Tests? Benefits: l Higher quality b/c everyone shares the same understanding of the requirements and builds it right the first time. l Tests become the system's regression suite. l Create an objective verification of “done-ness” for a story. A story is done when all tests pass. l Create transparency into progress on a story. l Manual testers are part of the automation process. l Allows for more exploratory testing b/c happy path is automated Simple Example Acceptance Criteria: Transactions are rounded to the nearest cent. Original Value Rounded Value $0.021 $0.02 $0.025 $0.02 $0.029 $0.02 Simple Example Why was there confusion? The goal was missing... Goal: Create a currency conversion system. The Concrete Example clarified what was meant by the acceptance criteria. Refined Acceptance Criteria: Transactions are always rounded down to the nearest cent. Simple Example l This simple example is a REAL EXAMPLE in a system that was well-tested and approved by all parties. l Attacker was able to steal > $15,000 starting with a single cent using this process... l $0.01 → 0.0051 € rounded to 0.01 € l 0.01 € → $0.0196 rounded to $0.02 l A seemingly minor ambiguity in the requirements was very expensive. Take Aways l Concrete examples drive understanding and consensus. l To create good representative examples, everyone must understand the goal of the story. l For technical people to properly understand the goal, they must understand the business domain. What does Cucumber have to do with this? Cucumber • Cucumber lets software development teams describe how software should behave in plain text. The text is written in a business-readable domain-specific language and serves as documentation of requirements, automated regression tests and development- aid - all rolled into one format. • Cucumber works with Ruby, Java, .NET, Flex or web applications written in any language. It has been translated to over 40 spoken languages. Automated Testing Cucumber Framework Components: Cucumber (Framework) HTMLUnit (Browser Emulator) Celerity/Selinium/Watir (Browser Emulator API) Features (Tests written in sentence format) Step Definitions (Backend code to support Features) Automated Testing Linking it all together Command line – Cucumber Feature File - Step Definition File – Rake TAGS - Framework - Tags Sentence Matching Sentence @abc Terminal Browser Emulator Celerity/Selinium/Watir Result Output- – Execute Command Automated Testing Process Process: • Product Owner writes acceptance tests in feature file scenarios. • During grooming the team discusses the scenarios and asks questions. • Once story is committed to work the developers write tests and then code and then run tests. • QA expands the scenarios in the feature files. • When testing is complete PO acceptance is done by running the acceptance tests. Cucumber A Cucumber test consists of 2 components: 1. The Feature files which contain the test Scenarios. 2. The Step Definition files which contain the Ruby “glue” code that connects the tests to the application. Cucumber Feature File Overview: l Written in plain text l Define the test suite and contain the actual tests which are called Scenarios l Written collaboratively by everyone on the team (PO, QA, and Dev) l QA can run these manually. Cucumber Feature Files: l Suite is defined by keyword “Feature:” followed by a description. l Individual tests are defined by “Scenario:” followed by a unique name. Scenarios are concrete examples of how we want the software to behave. l Tests have 4 reserved words that begin each line: Given, When, Then, And. l “Background:” that is run before each test so that you don’t have to repeat steps. Cucumber @Cucumberpresentation Feature: User would like to have a successful presentation Tests the following acceptance criteria: 1. Verify the attendees are awake 2. Verify the attendees are happy with the presentation @Attendeeawake Scenario: Verify the attendees are awake Given that I am an attendee When I have drunk caffeine Then I will be able to listen to the presenter @Attendeeishappy Scenario: Verify Attendees are happy Given I am the presenter When I talk slow And my content is interesting Then the attendees will be happy Cucumber Wrap up: • Cucumber is a functional test automation tool for lean and agile teams. • It supports behaviour-driven development, specification by example and agile acceptance testing. • You can use it to automate functional validation in a form that is easily readable and understandable to business users, developers and testers. • It helps teams create executable specifications, that are a goal for development, acceptance criteria and functional regression checks for future changes. • Allows teams to create living documentation, a single authoritative source of information on system functionality that is always up-to- date. • Allows manual testers to be involved in automation, as well as, frees them up from testing the “happy path” to be able to real exploratory testing. Cuke4PHP QUESTIONS? .
Load more

Recommended publications
  • Agile Methodologies and Extreme Programming Corso Di Laurea
    Agile Methodologies and EXtreme Programming Lecturer: Giuseppe Santucci (Some slides taken from slideshare.net) Outline • Development Methodologies • Agile Development (12 Key Practices) • Extreme Programming (XP) – How does it work ? 2 What is a SE Methodology? • A SE methodology is a rigorously defined process (or set of practices) for creating software – A set of rules you have to follow – A set of conventions the organization decides to follow – A systematical, engineered approach for organizing software projects 3 Agile Development Agile Manifesto “Our highest priority is to satisfy the customer through early and continuous delivery of valuable software“ [Manifesto for Agile] 5 The agile spirit • Incremental – Working software over comprehensive documentation • Cooperation – Customer collaboration over contract negotiation • Straightforward – Individuals and interactions over processes and tools • Adaptive – Responding to change over following a plan 6 Agile Methodologies • eXtreme Programming (XP) • Scrum • Crystal family of methodologies • Feature-Driven Development (FDD) • Adaptive Software Development (ASD) • Dynamic System Development Model (DSDM) • Agile Unified Process (AUP) 7 The XP inventor: Kent Beck • eXtreme Programming – The most prominent agile development methodology Kent Beck 8 The 12 Key Practices 1. Metaphor 2. The Planning Game 3. Test-Driven Development 4. Pair Programming 5. Refactoring 6. Simple Design 7. Collective Ownership 8. Continuous Integration 9. On-site Customer 10. Small Releases 11. 40-Hour Workweek
    [Show full text]
  • Agile Practices in Commercial Saas Teams
    SAMINT-MILI 2036 Master’s Thesis 30 credits May 2020 Agile Practices in Commercial SaaS Teams A case study on the adoption of agile practices in non-software teams Bruno Petersen Master’s Programme in Industrial Management and Innovation Masterprogram i industriell ledning och innovation Abstract Agile Practices in Commercial SaaS Teams Bruno Petersen Faculty of Science and Technology Agile software development methods have seen great success in software Visiting address: Ångströmlaboratoriet teams. Research on the topic of adopting agile methods in development teams Lägerhyddsvägen 1 is extensive. In the literature key enabling factors are identified and numerous House 4, Level 0 benefits of agile ways of working are named. Less attention has been payed to Postal address: the non-software functions in software development organizations, though. Box 536 Moreover, little is known about how well the enabling factors and benefits for 751 21 Uppsala software teams translate to other teams in the organization. The goal of this Telephone: study is to evaluate what benefits agile methods provide to non-software teams, +46 (0)18 – 471 30 03 whether the enabling factors are similar and what the challenges and drawbacks Telefax: for adopting agile methods in commercial teams are. Using the case of the +46 (0)18 – 471 30 00 Swedish Software-as -a-Service company Funnel, which introduced agile Web page: practices into their commercial teams, these questions are tackled. The study http://www.teknik.uu.se/student-en/ finds that knowledge transfer and governance are core areas that need to be engaged in during the adoption process. With decisions being made more autonomously ensuring the exchange of relevant information is crucial.
    [Show full text]
  • Acceptance Testing Tools - a Different Perspective on Managing Requirements
    Acceptance Testing Tools - A different perspective on managing requirements Wolfgang Emmerich Professor of Distributed Computing University College London http://sse.cs.ucl.ac.uk Learning Objectives • Introduce the V-Model of quality assurance • Stress the importance of testing in terms of software engineering economics • Understand that acceptance tests are requirements specifications • Introduce acceptance and integration testing tools for Test Driven Development • Appreciate that automated acceptance tests are executable requirements specifications 2 V-Model in Distributed System Development Requirements Acceptance Test Software Integration Architecture Test Detailed System Design Test See: B. Boehm Guidelines for Verifying and Validating Software Unit Requirements and Design Code Specifications. Euro IFIP, P. A. Test Samet (editor), North-Holland Publishing Company, IFIP, 1979. 3 1 Traditional Software Development Requirements Acceptance Test Software Integration Architecture Test Detailed System Design Test Unit Code Test 4 Test Driven Development of Distributed Systems Use Cases/ User Stories Acceptance QoS Requirements Test Software Integration & Architecture System Test Detailed Unit Design Test These tests Code should be automated 5 Advantages of Test Driven Development • Early definition of acceptance tests reveals incomplete requirements • Early formalization of requirements into automated acceptance tests unearths ambiguities • Flaws in distributed software architectures (there often are many!) are discovered early • Unit tests become precise specifications • Early resolution improves productivity (see next slide) 6 2 Software Engineering Economics See: B. Boehm: Software Engineering Economics. Prentice Hall. 1981 7 An Example Consider an on-line car dealership User Story: • I first select a locale to determine the language shown at the user interface. I then select the SUV I want to buy.
    [Show full text]
  • Model-Based Fuzzing Using Symbolic Transition Systems
    Model-Based Fuzzing Using Symbolic Transition Systems Wouter Bohlken [email protected] January 31, 2021, 48 pages Academic supervisor: Dr. Ana-Maria Oprescu, [email protected] Daily supervisor: Dr. Machiel van der Bijl, [email protected] Host organisation: Axini, https://www.axini.com Universiteit van Amsterdam Faculteit der Natuurwetenschappen, Wiskunde en Informatica Master Software Engineering http://www.software-engineering-amsterdam.nl Abstract As software is getting more complex, the need for thorough testing increases at the same rate. Model- Based Testing (MBT) is a technique for thorough functional testing. Model-Based Testing uses a formal definition of a program and automatically extracts test cases. While MBT is useful for functional testing, non-functional security testing is not covered in this approach. Software vulnerabilities, when exploited, can lead to serious damage in industry. Finding flaws in software is a complicated, laborious, and ex- pensive task, therefore, automated security testing is of high relevance. Fuzzing is one of the most popular and effective techniques for automatically detecting vulnerabilities in software. Many differ- ent fuzzing approaches have been developed in recent years. Research has shown that there is no single fuzzer that works best on all types of software, and different fuzzers should be used for different purposes. In this research, we conducted a systematic review of state-of-the-art fuzzers and composed a list of candidate fuzzers that can be combined with MBT. We present two approaches on how to combine these two techniques: offline and online. An offline approach fully utilizes an existing fuzzer and automatically extracts relevant information from a model, which is then used for fuzzing.
    [Show full text]
  • Agile-Methodologies-And-Extreme
    Agile Development and Extreme Programming Rouhollah Rahmati & Sina Khankhajeh Department of Computer Engineering Sharif University of Technology Agenda • Development Methodologies • Agile Development • Extreme Programming (XP) • In Practise • Develop Your Own Methodology Development Methodologies What is a Methodology? • A methodology is a formalized process or set of practices for creating software • A set of rules you have to follow • A set of conventions the organization decides to follow • unlike method • which systematically details a given procedure or process • Software development methodology • framework acts as a basis for applying specific approaches to develop and maintain software The Waterfall Development Process The Waterfall Process • The traditional development process: Software Requirements Analysis Design Implementation • Or at worst … Testing maintenance • But this always ends up happening! Agile Development Agile Manifesto “Our highest priority is to satisfy the customer through early and continuous delivery of valuable software“ [Manifesto for Agile] principles underlie the Agile Manifesto • Welcome changing requirements, even late in development • Working software is delivered frequently (weeks rather than months) • Working software is the principal measure of progress • Close, daily co-operation between business people and developers principles underlie the Agile Manifesto (continued) • Face-to-face conversation is the best form of communication (co-location) • Projects are built around motivated individuals, who
    [Show full text]
  • An Analysis of Current Guidance in the Certification of Airborne Software
    An Analysis of Current Guidance in the Certification of Airborne Software by Ryan Erwin Berk B.S., Mathematics I Massachusetts Institute of Technology, 2002 B.S., Management Science I Massachusetts Institute of Technology, 2002 Submitted to the System Design and Management Program In Partial Fulfillment of the Requirements for the Degree of Master of Science in Engineering and Management at the ARCHIVES MASSACHUSETrS INS E. MASSACHUSETTS INSTITUTE OF TECHNOLOGY OF TECHNOLOGY June 2009 SEP 2 3 2009 © 2009 Ryan Erwin Berk LIBRARIES All Rights Reserved The author hereby grants to MIT permission to reproduce and to distribute publicly paper and electronic copies of this thesis document in whole or in part in any medium now known or hereafter created. Signature of Author Ryan Erwin Berk / System Design & Management Program May 8, 2009 Certified by _ Nancy Leveson Thesis Supervisor Professor of Aeronautics and Astronautics pm-m A 11A Professor of Engineering Systems Accepted by _ Patrick Hale Director System Design and Management Program This page is intentionally left blank. An Analysis of Current Guidance in the Certification of Airborne Software by Ryan Erwin Berk Submitted to the System Design and Management Program on May 8, 2009 in Partial Fulfillment of the Requirements for the Degree of Master of Science in Engineering and Management ABSTRACT The use of software in commercial aviation has expanded over the last two decades, moving from commercial passenger transport down into single-engine piston aircraft. The most comprehensive and recent official guidance on software certification guidelines was approved in 1992 as DO-178B, before the widespread use of object-oriented design and complex aircraft systems integration in general aviation (GA).
    [Show full text]
  • Acceptance Testing How Cslim and Fitnesse Can Help You Test Your Embedded System
    Acceptance Testing How CSlim and FitNesse Can Help You Test Your Embedded System Doug Bradbury Software Craftsman, 8th Light Tutorial Environment git clone git://github.com/dougbradbury/c_learning.git cd c_learning ./bootstrap.sh or with a live CD: cp -R cslim_agile_package c_clearning cd c_learning git pull ./bootstrap.sh Overview Talk w/ exercises: Acceptance Tests Tutorial: Writing Acceptance tests Tutorial: Fitnesse Tutorial: CSlim Talk: Embedded Systems Integration Bonus Topics Introductions Who are you? Where do you work? What experience do you have with ... embedded systems? acceptance testing? FitNesse and Slim? Objectives As a result of this course you will be able to: Understand the purposes of acceptance testing; Use acceptance tests to define and negotiate scope on embedded systems projects; Integrate a CSlim Server into your embedded systems; Objectives (cont) As a result of this course you will be able to: Add CSlim fixtures to your embedded system; Write Fitnesse tests to drive the execution of CSlim fixtures; Write and maintain suites of tests in a responsible manner. Points on a star How many points does this star have? Star Point Specification Points on a star are counted by the number of exterior points. Points on a star How many points does this star have? By Example 3 5 9 Points on a star Now, how many points does this star have? Robo-draw Pick a partner ... Acceptance Testing Collaboratively producing examples of what a piece of software is supposed to do Unit Tests help you build the code right. Acceptance Tests
    [Show full text]
  • Software Testing: Essential Phase of SDLC and a Comparative Study Of
    International Journal of System and Software Engineering Volume 5 Issue 2, December 2017 ISSN.: 2321-6107 Software Testing: Essential Phase of SDLC and a Comparative Study of Software Testing Techniques Sushma Malik Assistant Professor, Institute of Innovation in Technology and Management, Janak Puri, New Delhi, India. Email: [email protected] Abstract: Software Development Life-Cycle (SDLC) follows In the software development process, the problem (Software) the different activities that are used in the development of a can be dividing in the following activities [3]: software product. SDLC is also called the software process ∑ Understanding the problem and it is the lifeline of any Software Development Model. ∑ Decide a plan for the solution Software Processes decide the survival of a particular software development model in the market as well as in ∑ Coding for the designed solution software organization and Software testing is a process of ∑ Testing the definite program finding software bugs while executing a program so that we get the zero defect software. The main objective of software These activities may be very complex for large systems. So, testing is to evaluating the competence and usability of a each of the activity has to be broken into smaller sub-activities software. Software testing is an important part of the SDLC or steps. These steps are then handled effectively to produce a because through software testing getting the quality of the software project or system. The basic steps involved in software software. Lots of advancements have been done through project development are: various verification techniques, but still we need software to 1) Requirement Analysis and Specification: The goal of be fully tested before handed to the customer.
    [Show full text]
  • Continuous Quality and Testing to Accelerate Application Development
    Continuous Quality and Testing to Accelerate Application Development How to assess your current testing maturity level and practice continuous testing for DevOps Continuous Quality and Testing to Accelerate Application Development // 1 Table of Contents 03 Introduction 04 Why Is Continuous Quality and Testing Maturity Important to DevOps? 05 Continuous Testing Engineers Quality into DevOps 07 Best Practices for Well- Engineered Continuous Testing 08 Continuous Testing Maturity Levels Level 1: Chaos Level 2: Continuous Integration Level 3: Continuous Flow Level 4: Continuous Feedback Level 5: Continuous Improvement 12 Continuous Testing Maturity Assessment 13 How to Get Started with DevOps Testing? 14 Continuous Testing in the Cloud Choosing the right tools for Continuous Testing On-demand Development and Testing Environments with Infrastructure as Code The Right Tests at the Right Time 20 Get Started 20 Conclusion 21 About AWS Marketplace and DevOps Institute 21 Contributors Introduction A successful DevOps implementation reduces the bottlenecks related to testing. These bottlenecks include finding and setting up test environments, test configurations, and test results implementation. These issues are not industry specific. They can be experienced in manufacturing, service businesses, and governments alike. They can be reduced by having a thorough understanding and a disciplined, mature implementation of Continuous Testing and related recommended engineering practices. The best place to start addressing these challenges is having a good understanding of what Continuous Testing is. Marc Hornbeek, the author of Engineering DevOps, describes it as: “A quality assessment strategy in which most tests are automated and integrated as a core and essential part of DevOps. Continuous Testing is much more than simply ‘automating tests.’” In this whitepaper, we’ll address the best practices you can adopt for implementing Continuous Quality and Testing on the AWS Cloud environment in the context of the DevOps model.
    [Show full text]
  • Continuous Delivery
    Continuous Quality Improvement Vijay kumar Vankamamidi Joseph Eapen Ebin John Poovathany Delivery Quality Goal Faster Time to Market Reduce Risk Build software that is production ready at all times Frequent, low risk Fast feedback, Built-in Software visibility and Quality releases control 2 Agile development philosophy > The process for releasing/deploying software must be repeatable and reliable. > Build quality in! > Automate everything! > Done means “potentially shippable”. - Complete PSR > Everybody has responsibility for quality. > Improve continuously. 3 Continuous Quality Improvement People, Process & Systems Communities of practice for continuous learning (Design, Coding, Testing) Software Craftsmanship Product Business Functional B A / Developer Field / IT Ops Customer Director Analyst Architect Tester Product Design, Coding Customer Release and Customer Need Requirement Requirement High level Testing Director and testing Validation deployment Collection Analysis Design Validation Agile Methodologies DevOps ( Development, IT Operations, and Support) Standardize the frameworks Hygiene factors like Definition of Done Establish a standard Continuous Continuous Improvement culture Integration Framework 4 Agile Methodologies Focus on People & Process 5 Quality through Agile > Standardize the frameworks (Scrum, Kanban and Scrumban) - Bring in common understanding of Agile - Informal learning opportunities > Hygiene practices - Constructive partnership with customer • Product vision and Requirement clarity - Definition of Done • User
    [Show full text]
  • Stephan Goericke Editor the Future of Software Quality Assurance the Future of Software Quality Assurance Stephan Goericke Editor
    Stephan Goericke Editor The Future of Software Quality Assurance The Future of Software Quality Assurance Stephan Goericke Editor The Future of Software Quality Assurance Editor Stephan Goericke iSQI GmbH Potsdam Germany Translated from the Dutch Original book: ‘AGILE’, © 2018, Rini van Solingen & Manage- ment Impact – translation by tolingo GmbH, © 2019, Rini van Solingen ISBN 978-3-030-29508-0 ISBN 978-3-030-29509-7 (eBook) https://doi.org/10.1007/978-3-030-29509-7 This book is an open access publication. © The Editor(s) (if applicable) and the Author(s) 2020 Open Access This book is licensed under the terms of the Creative Commons Attribution 4.0 Inter- national License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence and indicate if changes were made. The images or other third party material in this book are included in the book’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the book’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use.
    [Show full text]
  • Testing Guidelines
    Guidance Testing Guidelines Purpose This document provides guidelines for testing changes to the BSC Software, Systems and Processes. It also provides a high level view of test procedures that are followed by various parties involved in testing. The management of testing and test deliverables is detailed in the Test Management procedure (Reference 1). The scope of testing for each software release is detailed in the relevant Release Test Strategy document, which is developed by ELEXON and the relevant Service Providers. The scope is agreed by all test participants and captured in the Test Strategy. This document is specifically written as a guideline for testing the BSC Systems. However, the principles described may be used when planning for testing of other software systems. Contents 1. Testing Process Overview ...................................................................................................... 2 2. Application Manager and Developer Testing Overview ........................................................ 4 3. Business Process Operator Testing Overview ....................................................................... 9 4. Terms Used ............................................................................................................................ 13 5. References ............................................................................................................................. 13 6. Appendix A – Test Results ...................................................................................................
    [Show full text]