Agile Testing Practices

Megan S. Sumrell Director of Transformation Services Valtech Introductions

About us… Now about you… Your name Your company Your role Your experience with Agile or Scrum? Personal Expectations Agenda

Introductions Agile Overview Traditional QA Teams Traditional Automation Approaches Role of an Agile Tester Testing Activities Refine Acceptance Criteria TDD Manual / Exploratory Testing Defect Management Documentation Performance Testing Regression Testing Agenda Continued

Test Automation on Agile Teams Testing on a Greenfield Project Testing on a Legacy Application Estimation Sessions Sprint Planning Meetings Retrospectives Infrastructure Skills and Titles Closing Agile Overview Agile Manifesto

"We are uncovering better ways of developing software by doing it and helping others do it. Through this work we have come to value: Individuals and interactions over processes and tools Working software over comprehensive documentation Customer collaboration over contract negotiation Responding to change over following a plan That is, while there is value in the items on the right, we value the items on the left more." Scrum Terms and Definitions

User Story: high level requirements Product Backlog: list of prioritized user stories Sprint : one cycle or iteration (usually 2 or 4 weeks in length) Daily Stand-up: 15 minute meeting every day to review status Scrum Master: owns the Scrum process and removes impediments Product Owner: focused on ROI and owns priorities on the backlog Pigs and Chickens Traditional QA Teams How are you organized? When do you get involved in the project? What does your “test phase” look like? What testing challenges do you have? Traditional Test Automation Automation Challenges

Cost of tools Hard to learn Can’t find time Maintenance UI dependent Only a few people can run them Traditional Test Pyramid

UNIT TESTS

Business Rules

GUI TESTS Will these strategies work in an Agile environment? Food for Thought…….

“Tests play several pivotal roles during the software development process. First, tests unambiguously communicate how things are supposed to work. Second, they provide feedback on whether the system actually works the way it is supposed to work. Third, tests provide the scaffolding that allows developers to make changes throughout the development process…” Lean Software Development – Mary and Tom Poppendieck Role of An Agile Tester Role of “QA” on an Agile Team

Testing starts with the specification, not verification Focus on acceptance criteria Focus on automation, not manual testing User Stories are not done until the automated acceptance tests are passing Forces the team to think about testability at the beginning Testing Activities Testing Activities

Refine acceptance criteria Test Driven Development Manual / Exploratory Testing Defect Management Documentation Performance Testing Regression Testing Refine Acceptance Criteria Requirements vs. Tests Refine Acceptance Tests

•Who? – Analysts, Testers, Developers, PO •How? – Collaborate! – Build executable requirements if possible •Why? – Avoids misinterpretation of the requirements – Full transparency into the test cases – Sets expectations ATDD: Acceptance Test Driven Development Development is driven by examples Acceptance testing happens before development starts Business facing tests Represent workflows and business logic rules Some Questions to Ask…….

What else do the developers need to know about this user story? What can go wrong during the story? What am I assuming? Are there circumstances when this story may behave differently? Workshop: Let’s write some acceptance tests…. User Story: As a mathematician, I want to be able to use the calculator to divide numbers and get the answer so that I can solve problems.

Acceptance Tests: Verify that the division is correct Verify that the answer is in ##.## format Workshop - continued

User Story: As a user, I want to search for a name so that I can find that person’s phone number. Acceptance Tests: Search by first or last name ‘Like’ match search Results are in first name, last name, phone number order Test Driven Development TDD: Test Driven Development

TDD is not unit testing TDD is not writing tests where it is easy to do it TDD is not testing after you have coded TDD is not manually testing your code

TDD IS ABOUT TESTING FIRST TDD: Test Driven Development

TDD is not magic To be effective, tests must be: • Accurate • Appropriate • Sufficient TDD: Test Driven Development

TDD is a discipline that developers must adopt Testers can help educate developers on how to write good tests Developers tend to test what works Testers tend to look for what breaks

Write the Test testDivideByZero extends TestCase { Calculator calc = new Calculator(); try { calc.divide(6, 0); fail(“No exception was thrown”); } catch(MathException e) { assertEquals(“Cannot divide by zero”, e.message()); } } Write the Code

Before: public int divide(int divisor, int dividend) { return divisor / dividend; } Write the Code

After: public int divide(int dividend, int divsor) { if (divisor == 0) throw new MathException(“Cannot divide by zero”);

return divisor / dividend; } Refactor

Remove duplication Improve implementation Simplify

Exploratory Testing Exploratory Testing

•Who? –Testers •How? – Paired testing •Why? – Ensure all negative or unusual paths are covered – Paired testing adds value Session Based Exploratory Testing

James and Jonathan Bach – 2000 Consists of the following: Charter Session Session Report Debrief “Exploratory testing is simultaneous learning, test design, and test execution.” -James Bach Workshop - Create Charters

Think about buying something on-line. What charters would you create for exploratory testing on an on-line checkout system Defect Management Defect Management

•Who? – Testers (sometimes Analysts) •How? – Verify that unit tests were written – Add automated tests for the future •Why? – Ensure that the defects were fixed – Automate tests to ensure it doesn’t break again Documentation Workshop: Documentation

What do you document on your agile teams? What are your test artifacts? Eliminate Waste

Ensure all artifacts are valuable Don’t create things people won’t use Don’t Repeat Yourself (DRY) If a test can take the replace of a requirements document, don’t write both Don’t do more than is necessary If a wireframe sketch is sufficient to describe a GUI, don’t spend the time and money to create a photo-realistic mock-up Performance Testing Performance Testing

•Who? – Performance Testers •How? – Map test cases to performance requirements •Why? – Uncover performance issues early!!! Regression Testing How can you regression test every iteration? Automation (lots and lots of it!) Should happen every day Immediate feedback when code changes are made Test Automation on Agile Teams Agile Test Pyramid

GUI TESTS

Business Rules

UNIT TESTS Automate Acceptance Tests

•Who? – Testers and Developers •How? – Collaborate! – Work to automate defined acceptance tests •Why? – Allows everyone to track progress – Ensures requirements are met – Allows for fast, easy regression testing More Reasons Why We Tackle Automation this Way…. Test First Start automation without the UI Involves entire team in testing Involves customer in testing Visibility of results Avoid gold plating Leads to good architecture and design Tests are easy to read Collapse the V model Testing code is minimal And more……. Executable Requirements

Think back to our requirement vs. test conversation….. Let’s see this in action FitNesse Unit vs. GUI vs. FitNesse

GUI GUI specific TESTING

FitNesse Customer Oriented

Code oriented UNIT TESTING FIT TABLE FIXTURE SYSTEM FitNesse has several fixture types…

Column - calculations Row/Array – data sets Do/Action - workflow Action Database And more…. Anatomy of a Column Fixture

First Row Identifies the Test/Requirement. This is the name of the fixture (the class)

Division

Numerator Denominator Quotient?

10 2 5

100 2.5 40 Second Row is the inputs and outputs. Expected outputs have the ? Or () after them, inputs do not

Division

Numerator Denominator Quotient?

10 2 5

100 2.5 40 Each additional row is an instance of the test that explains the requirement we are testing

Division

Numerator Denominator Quotient?

10 2 5

100 2.5 40 Workshop: Write a Column Fixture

User Story 1: As a mathematician, I want to be able to use the calculator to divide numbers and get the answer so that I can solve problems. Acceptance Tests: Verify that the division is correct Verify that the answer is in ##.## format Anatomy of a Row Fixture

First Row Identifies the Other cells on the first row Test/Requirement. This is the represent inputs passed into name of the fixture (and the the fixture name of the class)

ListofUsers Active

FirstName LastName UserName

John Smith Jsmith

Beth Jones Bjones

Suzy Plus splus The second row lists the data elements in the expected output

ListofUsers Active

FirstName LastName UserName

John Smith Jsmith

Beth Jones Bjones

Suzy Plus splus The 3rd through nth rows list the data you expect the test to return

ListofUsers Active

FirstName LastName UserName

John Smith Jsmith

Beth Jones Bjones

Suzy Plus splus Workshop: Write a Row Fixture

User Story 3: As a user, I want to search for a name so that I can find their phone number. Acceptance Tests: Search by first or last name “Like” match search Results are in first name, last name, phone number Do Fixture

Do fixtures should read like a story Used to test workflows Often tie to epics Each step in the do fixture likely has detailed tests elsewhere Let’s look at an example…. Example of a Do Fixture

As a roulette player, I want to place multiple bets and have my winnings automatically calculated so that I can become a millionaire. To do this, a series of steps need to happen (that could each be a separate story)

PLACE BETS

SPIN WHEEL

CALCUALTE WINNINGS

FIXTURE NAME: defines the class

SIMPLE ACTIONS: method names that execute them

ROW FIXTURES: method returns a row fixture

Some Additional Considerations

Your automation should control your data Your automation should be quick Your automation should be machine agnostic Your automation should let you sleep at night Your test should run in any order Some Additional Considerations

Who will write the tests? Fitnesse is designed to permit Customers to write tests However, the Customer will probably need help from QA • Fitnesse Wiki syntax • Corner cases and unhappy paths Some Additional Considerations

Impact on Development The fixtures used in Fitnesse tests need internal implementations written in the same language as the code These internal fixtures are the “glue” that binds the test tables to the application’s implementation that is being tested This code must be written and maintained by developers Testing on a Greenfield Application SPRINT START SPRINT END

Define Acceptance Tests Automate Acceptance Tests

TDD (Test Driven Development)

Manual Testing

Verify Bug Fixes

Full Regression

Performance

Exploratory Testing on a Legacy Application Legacy Challenges

What challenges exist with Legacy Applications that make agile testing challenging? Strategy 1: Not recommended

SPRINT START SPRINT END

Define Acceptance Tests Automate Acceptance Tests Exploratory

TDD (Test Driven Development) Full Regression

Manual Testing Performance

Verify Bug Fixes Strategy 2: Hardening Sprint

SPRINT ONE SPRINT TWO SPRINT N (Hardening Sprint)

Automate Acceptance Tests Automate Acceptance Tests Full Regression

TDD TDD Exploratory

Manual Testing Manual Testing Performance

Verify Bug Fixes Verify Bug Fixes

Exploratory Exploratory Estimation Sessions Estimation Techniques

• Planning Poker

• Rock, Paper, Scissors

• Others? Detail an Accuracy

• Some believe that better estimates require more detailed analysis

– More detailed analysis takes additional time and resources • Others believe that the initial “gut” reaction is sufficient for Agile planning purposes

– Time required for additional analysis does not result in sufficiently better estimates to justify the added costs Detail and Accuracy

• Estimates can be like statistics

– Out of context, an individual estimate may have little meaning

– In the overall iteration, though, some estimates will be low and others will be high • In the end, they cancel each other out What are things testers often think about when estimating? Sprint Planning Meetings What are their Purpose?

Define the sprint backlog Define “DONE” Commit to the sprint Workshop: Let’s Create Tasks

User Story 1: As a mathematician, I want to be able to use the calculator to divide numbers and get the answer so that I can solve problems. Task Breakout

• Who creates tasks?

• Who estimates tasks?

• Who owns tasks? Definition of Done Retrospectives Retrospectives

Opportunity to review what worked and what didn’t Must be conducted in a non-threatening environment To be effective All participants must feel comfortable and willing to share Should be held at each iteration boundary Should result in tangible action items that are put into place immediately Workshop: Run a Retrospective Infrastructure Continuous Integration

“Continuous Integration is a software development practice where members of a team integrate their work frequently, usually each person integrates at least daily - leading to multiple integrations per day. Each integration is verified by an automated build (including test) to detect integration errors as quickly as possible.” – Martin Fowler Check-in Code

Publish Build Entire Results Project

Run Tests How Does CI Work?

Single Source Code Repository Build is Automated Build is self-testing Everyone commits every day The build is fast Visible results Automated deployment Test Environments?

Do you have an environment you have complete control over? Are builds pushed or pulled to it? How often do you get builds? Does it look like your production environment? Do you “own” it? Test Data

• Where do you get your test data? • How well do you know your test data? • Do you control it? • Can you re-set it? • Can you run the same script over and over again? Data Needs

Different testing activities require different data – one database will not suit all your needs. Manual / Exploratory Automation Performance Any others? Skills and Titles Titles….

QA E Sen ngin ior D eer Test evel Engineer oper

t lys na A r QA te loper es eve e T D ar QA A w rchit ft ect So Workshop: Hiring an Agile Tester

What skills would you want from your agile testers? Closing Agile Foundations

For more information,

Visit us on the web at http://www.valtech.us

Email us directly at [email protected]

Call us at (972) 789-1200

Fax us at (972) 789-1340

Write us (or just drop by) at Valtech 19111 North Dallas Pkwy, Suite 300 Dallas, TX 75287

© 2009 Valtech, Inc. All rights reserved. Sources

Mountain Goat Software (www.mountaingoatsoftware.com) “Testing Extreme Programming” by Lisa Crispin and Tip House “Test Driven .NET Development with FitNesse” by Gojko Adzic FitNesse.org “Agile Estimating and Planning” by Mike Cohn “User Stories Applied” by Mike Cohn “Lean Software Development” by Mary and Tom Poppendieck “Collaboration Explained” by Jean Tabaka