Agile Testing Practices Megan S. Sumrell Director of Transformation Services Valtech Introductions About us… Now about you… Your name Your company Your role Your experience with Agile or Scrum? Personal Expectations Agenda Introductions Agile Overview Traditional QA Teams Traditional Automation Approaches Role of an Agile Tester Testing Activities Refine Acceptance Criteria TDD Manual / Exploratory Testing Defect Management Documentation Performance Testing Regression Testing Agenda Continued Test Automation on Agile Teams Testing on a Greenfield Project Testing on a Legacy Application Estimation Sessions Sprint Planning Meetings Retrospectives Infrastructure Skills and Titles Closing Agile Overview Agile Manifesto "We are uncovering better ways of developing software by doing it and helping others do it. Through this work we have come to value: Individuals and interactions over processes and tools Working software over comprehensive documentation Customer collaboration over contract negotiation Responding to change over following a plan That is, while there is value in the items on the right, we value the items on the left more." Scrum Terms and Definitions User Story: high level requirements Product Backlog: list of prioritized user stories Sprint : one cycle or iteration (usually 2 or 4 weeks in length) Daily Stand-up: 15 minute meeting every day to review status Scrum Master: owns the Scrum process and removes impediments Product Owner: focused on ROI and owns priorities on the backlog Pigs and Chickens Traditional QA Teams How are you organized? When do you get involved in the project? What does your “test phase” look like? What testing challenges do you have? Traditional Test Automation Automation Challenges Cost of tools Hard to learn Can’t find time Maintenance UI dependent Only a few people can run them Traditional Test Pyramid UNIT TESTS Business Rules GUI TESTS Will these strategies work in an Agile environment? Food for Thought……. “Tests play several pivotal roles during the software development process. First, tests unambiguously communicate how things are supposed to work. Second, they provide feedback on whether the system actually works the way it is supposed to work. Third, tests provide the scaffolding that allows developers to make changes throughout the development process…” Lean Software Development – Mary and Tom Poppendieck Role of An Agile Tester Role of “QA” on an Agile Team Testing starts with the specification, not verification Focus on acceptance criteria Focus on automation, not manual testing User Stories are not done until the automated acceptance tests are passing Forces the team to think about testability at the beginning Testing Activities Testing Activities Refine acceptance criteria Test Driven Development Manual / Exploratory Testing Defect Management Documentation Performance Testing Regression Testing Refine Acceptance Criteria Requirements vs. Tests Refine Acceptance Tests •Who? – Analysts, Testers, Developers, PO •How? – Collaborate! – Build executable requirements if possible •Why? – Avoids misinterpretation of the requirements – Full transparency into the test cases – Sets expectations ATDD: Acceptance Test Driven Development Development is driven by examples Acceptance testing happens before development starts Business facing tests Represent workflows and business logic rules Some Questions to Ask……. What else do the developers need to know about this user story? What can go wrong during the story? What am I assuming? Are there circumstances when this story may behave differently? Workshop: Let’s write some acceptance tests…. User Story: As a mathematician, I want to be able to use the calculator to divide numbers and get the answer so that I can solve problems. Acceptance Tests: Verify that the division is correct Verify that the answer is in ##.## format Workshop - continued User Story: As a user, I want to search for a name so that I can find that person’s phone number. Acceptance Tests: Search by first or last name ‘Like’ match search Results are in first name, last name, phone number order Test Driven Development TDD: Test Driven Development TDD is not unit testing TDD is not writing tests where it is easy to do it TDD is not testing after you have coded TDD is not manually testing your code TDD IS ABOUT TESTING FIRST TDD: Test Driven Development TDD is not magic To be effective, tests must be: • Accurate • Appropriate • Sufficient TDD: Test Driven Development TDD is a discipline that developers must adopt Testers can help educate developers on how to write good tests Developers tend to test what works Testers tend to look for what breaks Write the Test testDivideByZero extends TestCase { Calculator calc = new Calculator(); try { calc.divide(6, 0); fail(“No exception was thrown”); } catch(MathException e) { assertEquals(“Cannot divide by zero”, e.message()); } } Write the Code Before: public int divide(int divisor, int dividend) { return divisor / dividend; } Write the Code After: public int divide(int dividend, int divsor) { if (divisor == 0) throw new MathException(“Cannot divide by zero”); return divisor / dividend; } Refactor Remove duplication Improve implementation Simplify Exploratory Testing Exploratory Testing •Who? –Testers •How? – Paired testing •Why? – Ensure all negative or unusual paths are covered – Paired testing adds value Session Based Exploratory Testing James and Jonathan Bach – 2000 Consists of the following: Charter Session Session Report Debrief “Exploratory testing is simultaneous learning, test design, and test execution.” -James Bach Workshop - Create Charters Think about buying something on-line. What charters would you create for exploratory testing on an on-line checkout system Defect Management Defect Management •Who? – Testers (sometimes Analysts) •How? – Verify that unit tests were written – Add automated tests for the future •Why? – Ensure that the defects were fixed – Automate tests to ensure it doesn’t break again Documentation Workshop: Documentation What do you document on your agile teams? What are your test artifacts? Eliminate Waste Ensure all artifacts are valuable Don’t create things people won’t use Don’t Repeat Yourself (DRY) If a test can take the replace of a requirements document, don’t write both Don’t do more than is necessary If a wireframe sketch is sufficient to describe a GUI, don’t spend the time and money to create a photo-realistic mock-up Performance Testing Performance Testing •Who? – Performance Testers •How? – Map test cases to performance requirements •Why? – Uncover performance issues early!!! Regression Testing How can you regression test every iteration? Automation (lots and lots of it!) Should happen every day Immediate feedback when code changes are made Test Automation on Agile Teams Agile Test Pyramid GUI TESTS Business Rules UNIT TESTS Automate Acceptance Tests •Who? – Testers and Developers •How? – Collaborate! – Work to automate defined acceptance tests •Why? – Allows everyone to track progress – Ensures requirements are met – Allows for fast, easy regression testing More Reasons Why We Tackle Automation this Way…. Test First Start automation without the UI Involves entire team in testing Involves customer in testing Visibility of results Avoid gold plating Leads to good architecture and design Tests are easy to read Collapse the V model Testing code is minimal And more……. Executable Requirements Think back to our requirement vs. test conversation….. Let’s see this in action FitNesse Unit vs. GUI vs. FitNesse GUI GUI specific TESTING FitNesse Customer Oriented Code oriented UNIT TESTING FIT TABLE FIXTURE SYSTEM FitNesse has several fixture types… Column - calculations Row/Array – data sets Do/Action - workflow Action Database And more…. Anatomy of a Column Fixture First Row Identifies the Test/Requirement. This is the name of the fixture (the class) Division Numerator Denominator Quotient? 10 2 5 100 2.5 40 Second Row is the inputs and outputs. Expected outputs have the ? Or () after them, inputs do not Division Numerator Denominator Quotient? 10 2 5 100 2.5 40 Each additional row is an instance of the test that explains the requirement we are testing Division Numerator Denominator Quotient? 10 2 5 100 2.5 40 Workshop: Write a Column Fixture User Story 1: As a mathematician, I want to be able to use the calculator to divide numbers and get the answer so that I can solve problems. Acceptance Tests: Verify that the division is correct Verify that the answer is in ##.## format Anatomy of a Row Fixture First Row Identifies the Other cells on the first row Test/Requirement. This is the represent inputs passed into name of the fixture (and the the fixture name of the class) ListofUsers Active FirstName LastName UserName John Smith Jsmith Beth Jones Bjones Suzy Plus splus The second row lists the data elements in the expected output ListofUsers Active FirstName LastName UserName John Smith Jsmith Beth Jones Bjones Suzy Plus splus The 3rd through nth rows list the data you expect the test to return ListofUsers Active FirstName LastName UserName John Smith Jsmith Beth Jones Bjones Suzy Plus splus Workshop: Write a Row Fixture User Story 3: As a user, I want to search for a name so that I can find their phone number. Acceptance Tests: Search by first or last name “Like” match search Results are in first name, last name, phone number Do Fixture Do fixtures should read like a story Used to test workflows Often tie to epics Each step in the do fixture likely has detailed tests elsewhere Let’s look at an example….