Smart BDD Testing Using Cucumber and Jacoco
Total Page：16
File Type：pdf, Size：1020Kb
www.hcltech.com Smart BDD Testing Using Cucumber and JaCoCo Business assurance $ Testing AuthOr: Arish Arbab is a Software Engineer at HCL Singapore Pte Limited, having expertize on Agile GUI/API Automation methodologies. WHITEPAPER ApriL 2015 SMART BDD TESTING USING CUCUMBER AND JACOCO | APRIL 2015 TABLE OF CONTENTS INTRODUCTION 3 PROBLEM FACED 3 SOLUTION APPROACH 4 BENEFITS 6 IMPROVEMENTS 7 APPLICABILITY TO OTHER PROJECTS 7 UPCOMING FEATURES 7 REFERENCES 8 APPRECIATIONS RECEIVED 8 ABOUT HCL 9 © 2015, HCL TECHNOLOGIES. REPRODUCTION PROHIBITED. THIS DOCUMENT IS PROTECTED UNDER COPYRIGHT BY THE AUTHOR, ALL RIGHTS RESERVED. 2 SMART BDD TESTING USING CUCUMBER AND JACOCO | APRIL 2015 INTRODUCTION Behavioral Driven Development (BDD) testing uses natural language to describe the “desired behavior” of the system that can be understood by the developer, tester and the customer. It is a synthesis and refinement of practices stemming from TDD and ATDD. It describes behaviors in a single notation that is directly accessible to domain experts, testers and developers for improving communication. It focuses on implementing only those behaviors, which contribute most directly to the business outcomes for optimizing the scenarios. PROBLEM FACED By using the traditional automation approach, and if given a summary report of automated tests to the business analysts, then it is guaranteed that the report would be met with a blank expression. This makes it tricky to prove that the tests are correct— do they match the requirement and, if this changes, what tests need to change to reflect this? The whole idea behind BDD is to write tests in plain English, describing the behaviors of the thing that you are testing. The main advantage of this is that the tests reflect the user stories and the business requirements of the application. Therefore, the results generated can be read by a non-technical person, e.g., a project sponsor, a domain expert, a business analyst, and the tests can be validated against the requirements for better operational efficiency. To achieve the BDD approach, there is an open source tool—Cucumber—that parses the scenarios/tests that are written in Gherkin language (GWT). This tool also interacts with the API/Junit/Source code of an application, using its step definitions test code. The best option to get the business intent into our tests is to have clear deliverables and acceptance criteria in the user stories. These can be then translated into feature tests containing scenarios written in Gherkin Language. BDD is the “outside-in” software development methodology, which has changed the tester’s role dramatically in the recent years and bridges the communication gap between business and technology. This innovative approach has the potential to shape a new organization standard with new SDET roles. Problems faced while implementing the new framework: y Justifying the coverage done by this framework becomes difficult as the complexity of the application code increases y Exposing APIs/Source Code was difficult as developers disagreed y Converting the software requirement into GWT format at the later stages of the software development utilized more efforts y Requiring testers with good technical/coding knowledge y Extending the Junit by using the mocking techniques or spring-based test was a hindrance when the complexity of the code increased y Convincing the BTMs to review the scenarios on every day was difficult © 2015, HCL TECHNOLOGIES. REPRODUCTION PROHIBITED. THIS DOCUMENT IS PROTECTED UNDER COPYRIGHT BY THE AUTHOR, ALL RIGHTS RESERVED. 3 SMART BDD TESTING USING CUCUMBER AND JACOCO | APRIL 2015 SOLUTION APPROACH Traditionally, software requirement was written in simple English sentences, but to convert those requirements into GWT format required an extra effort that should be properly handled. As GWT approach results in better quality of product, requirement should be written in GWT format by the business analyst at the early stages of software development rather than writing the requirement in domain-specific language. Set of scenarios should depict a particular user story. Implementing this approach in a later stage may increase the cost of a project. Example of writing the scenarios in the abovementioned format would be: Traditional software requirement: As a user, I want to login to the application with valid credentials. Simple Cucumber Feature File in Gherkin Format: Feature: Login to the system, User should be allowed to login with valid credentials Scenario: Login Success Given The login page is opened When I input valid username and password in the textboxes And I click the login button Then I logged in successfully. Step definitions for matching scenarios: Similarly for “When” and “Then,” step definitions are created. Corresponding step definitions are then written, and the result is displayed in the above format that assures developer/tester/BA and customers that a particular software requirement has been successfully implemented. Common method of testing an application at the API level requires its API or web services. However, if the application has neither the API available nor the developers who want to create it for testing purposes, then the problem arises to test the code written by developer without any extra development effort. To overcome this problem, Junit classes that are written by developers can be used to call the respective service methods. However, if the unit testing is being done using mocking technique, then it would be difficult to test the actual functionality of the application. So, in order to use the Junit methods effectively, dependency injections techniques can be used at the testing end. Such usage would call the Junit method by passing a particular dependency, which, in turn, calls the real service method. © 2015, HCL TECHNOLOGIES. REPRODUCTION PROHIBITED. THIS DOCUMENT IS PROTECTED UNDER COPYRIGHT BY THE AUTHOR, ALL RIGHTS RESERVED. 4 SMART BDD TESTING USING CUCUMBER AND JACOCO | APRIL 2015 BDD approach—detailed explanation of using Cucumber by leveraging Junit The pictorial representation of Cucumber automation framework: App Service Methods Model Service Methods Junit Classes Junit Classes containing unit test methods Cucumber Step Definitions Cucumber Cucumber Main(Driver) Feature File(Scenarios) DB, Test Data Test Result And Reports Data(.xlsx) Database HTML Reports, Screen shots, logs of failed scenarios etc. Flow of the BDD framework using Cucumber is as follows- y The test data, which is to be used in the scenarios, is stored in Excel sheet or MySql Database. y The test data is then passed to feature file, containing scenarios written in GWT format. Scenarios are denoted by certain tags. y Driver class decides which scenario among the entire features files should be executed by proving a proper tag value. Before running a particular feature, dependency used in application is injected to avoid mocking unit testing framework. y Appropriate matching method defined in the Cucumber step definition file is called. y From the step definition, appropriate unit test method of the parameterized Junit class is called and, thus, test data is passed into it. y Unit test method, which, in turn, calls the real service method by passing the test data to it. y Response is then returned from the service method to unit test method, which, in turn, passes the information to Cucumber step definition file. y Expected response is validated against the returned (actual) response in the step definition. y Logs are created upon failure or success and an auto HTML report is created, depicting the passed and failed scenarios in appropriate colors. © 2015, HCL TECHNOLOGIES. REPRODUCTION PROHIBITED. THIS DOCUMENT IS PROTECTED UNDER COPYRIGHT BY THE AUTHOR, ALL RIGHTS RESERVED. 5 SMART BDD TESTING USING CUCUMBER AND JACOCO | APRIL 2015 Now, if we have the API available, then, instead of calling the Junit test method, the API method is called directly and rest of the flow remains the same. Then the question arises, “Whether or not we are covering all the areas of an application with our test as the scenarios written are very behavioral specific?” Here the need for the code coverage arises. JaCoCo is one of the tools, which can be integrated with Cucumber Test, and the code coverage report generated can be easily read by customers/developers/business analysts and testers. The traditional way of testing may work out with testers, having less technical knowledge. Nevertheless, as we can clearly point out by seeing the above solutions and exposure of code to tester. The need for a tester with good coding skills is prerequisite to implement this approach successfully. JaCoCo report Cucumber test execution can be integrated into CI (Jenkins) server and can be executed depending upon the need. Now, when we have everything in place— scenarios, test code and test results—it becomes difficult to convince the BTMs to review the scenarios on a daily basis. There is another open-source tool available that overcomes these difficulties: Pickles, open-source living document generator. This tool generates the document consisting of the scenarios, which can be easily reviewed by the customer as it supports advance search capabilities. BENEFITS Key Benefits y Cucumber and JaCoCo are open-source tools, which provide equal benefits that other paid tools provide. y Around 80% reduction in the test execution time as compared to manual testing. y Cucumber can be integrated with existing tools like Selenium, TestComplete and Q T P. y Quality of an application increased as the defect is found at an early stage of development. y Cucumber can be used to test REST and SOAP Web services, thereby, easily adjusting in traditional way of API automation. © 2015, HCL TECHNOLOGIES. REPRODUCTION PROHIBITED. THIS DOCUMENT IS PROTECTED UNDER COPYRIGHT BY THE AUTHOR, ALL RIGHTS RESERVED. 6 SMART BDD TESTING USING CUCUMBER AND JACOCO | APRIL 2015 y Unlike other API testing tools, it produces more detailed, easy-to-understand test execution report for client, developer, BA and tester regardless of their technical knowledge.