Agile Testing Practices

Home , Agile testing, Integration testing

Building Building quality in for faster releases TESTINGAGILE PRACTICES

Hosts: Hosts:

© 2014 Scrum Inc. Scrum framework, making us a laboratoryliving on the cutting edge of We run our services company using Scrum as the primary manageme • • • • through our comprehensive suite of support services: We also help companies achieve the full benefits of Scrum • • • We maintain the Scrum methodology by: co-creator of Scrum. We are based in Cambridge, MA. Scrum Inc. is the Agile leadership company of Dr. Jeff Sutherl

Publishing andPublishing new content development Coaching (hands-on support to Scrum teams) ScrumConsulting (linking and business strategy, customizing Scrum) Training (Scrum Master, Product Owner, Agile Leadership, onlin of industries, processes and business challenges Adapting the methodology to an ever-expanding set behavior Conducting original research on organizational Capturing and codifying evolving best practices 䇿

Find out more at :

www.scruminc.com Who We Are Dave Sharrock . and,

e courses, etc.) 䇾 Enterprise nt Joe Jusce

Modern test tools for your Weird/Backward/Deprecated/Unique envir Tools to get testing done quickly. Top Agile Testing Practices. Agile Testing Costs & Benefits. MIND BLOWING CONCLUSION: DevTestOps The mindset of managing a successful agile testing practice. to How Crush your Competitors Agile with Testing. Outsourced QA? TechnicalEliminating Debt Elegant with Testing. qualitybuilding in. The Business case and numbers for exploratory testing, manual testin • • • • • • • •

up. your code. You do all of this in a pair. When you do this, co you update your deployment environment, you deploy your tests, y check in your automated tests, you yourbuild deployment package and Software. Insurance. Unit Unit Testing, Integration Testing, Regression Testing, User Acceptance Te Test Driven Development, Test First Development, DevTest You yourbuild test, you yourbuild code, you integration test you Code Craftpersonship, shared ownership, elegance, qualit building Part of the Solution? Gated Check-in and Rapid Roll Back in h 24 times the cost to repair a bug that escaped the sprint. TDD in Hardware and Red Green Refactor. Cucumber and Ruby on the as400 tools with written by Agenda . Cheezy , mpetitors can’t keep mpetitors can’t DevTestOps ardware and , at Nationwide r code, you y in. ou deploy onment. g, and . its its tests, sting. sting.

using 30-50% (1000’s of tests) in about 18 months - went from <1% test coverage to getting on for this!) by PO group know, who thought there was ever a need for - eliminated need for post-release testing (I occurrence - eliminated hot fixes as a normal release weeks downtime (2 hours for DB updates) every 2 release every 90 days to 1 hours planned - move from 8-12 hours unplanned downtime per from team DoD Winning at Agile Testing: be2 and a concerted effort/understanding

shareholder gain. -NYSE Triggered immediate stock climb, peaking at 10% –WSJ. softwarebuilding while faster” Dev after test toois slow! Banned “to cut costs WSJ: Microsoft bansCEO test teams le‐development‐course‐as‐talk‐on‐job‐cuts‐loom/ o‐nadella‐outlines‐vision‐with‐more‐overhauls.html Source: NYSE via

© 2014 Scrum Inc. 6 5 4 3 2 1 DevTestOps Red, Green, Refactor TDD Gated Check-In, Rapid Roll Back Automate tests Test inside the Sprint Top Agile Testing Practices fastest companies. II. Observed norm I. Observed norm Observed in the releasing each in in competitive for teams markets. sprint. III.

Sprint 1

Source, Adapted From: Henrik Kniberg End users

Timeline v1.0.0

Sprint 2

v1.1.0

Sprint 1

Acceptance test team Acceptance Source, Adapted From: Henrik Kniberg Timeline v1.0.0

1.0 1.0 acceptance Sprint 2

End users Bug!

v1.0.1

test

v1.0.1

v1.1.0

make it happen. it make happen. let’sso over), carrying (work to you the tools give the sprint for failing reason #1 their as this have with We it, most work of we doing the teams aren’t you know Agile Agile Testing

Docker Power Tools FitNesse Cucumber • • • • • • • • •

Automate tests across thousands of test instances. Ultra compact VM files. 2 Possibly the most elegant, compact. 2 Collaborative and information sharing. Wiki driven ATD framework. Product Owners can use directly. Plain language DSL tool. PowerTools nd nd Tools to make this happen quickly gen Virtual Machine service. gen test framework.

owners to accompany new backlog stories. Cucumber experts scripts thewith writing product Venture company nowith testers. Debt as soon as found.is it Enabled by policy, early on, Remediate Technical the end of any sprint. CTO chiefis PO, and can hit the deploy button at you are done.” The PO says “Dev team, turn these green and Developers get a User Story and a red test. Central Desktop

and and specify inputs and outputs to hook into application Users/testers use Wiki or Excel to Developers generates classes (“fixtures”) Test specified in table format

Fitnesse

Tools: Fit and Cucumber TestAcceptance (ATDD) Development Driven 4 3 2 denominator

(Wiki front end) end) front (Wiki

24 returned 24 expected 25 4.2 5 quotient

• Cucumber page Ihome the And on am days 45 last the in activity account had I And have “xyx123” password with in I And logged am Recent Scenario: Account Activity I want to As a In toorder page Ihome the And on am d 45 last the in activity account no had I And have “xyx123” password with in I And logged am No Scenario: Recent Account Activity list of a 45 last the my see over activity I And should 45 days, please select a time frame to frame time a review” select please days, 45 message: a get I And should

Given When Given When Then Then Tool for natural language scenarios Registered User User Registered I am a registered user “Jsmith” user I registered a am check my recent activity activity my recent check I should see the “Account “Account the see I should Activity” Page I should see the “Select “Select the see I should Account Page Activity Period” I click the “Recent I “Recent the click Activity” button I click the “Recent I “Recent the click Activity” button I am a registered user “Jsmith” user I registered a am ensure my account is correct is correct my account ensure “You last the in activity no had

ays ays days days

Modern test tools are available for your weird/ by Jeff “ Jeff by on the as400 Ruby and Cucumber for Scrum Hardware. Hardware. for Scrum Arduino Test Fixture • • • •

backward/deprecated/unique environment. Ships same day. $24 USD. Test Logic onboard. Red Light, Green Light. Cheezy Sources: Naonwide's Applicaon Development Center, Dani ” Morgan, at Nationwide Insurance. : with : tools with elle Roecker, Team WIKISPEED.

defects by 40%! project management cost down 80%, and reduce to How improve from there? Half cost to deliver, Already 98% on time and on budget, CMMI level 5. Time to fix a bug. of them all: to How do that? By monitoring the metric that is king • •

resolution. performed root cause analysis and systemized the If time went over 2 hours, Scrum Masters immediately Monitored by control charts. • • •

automated test, and BUILD QUALITY IN. afford NOTCouldn’t to exploratory test, manual test, Measured bug fix time and it’s success, peer reviewed. paired to published 5 papers documenting this practice Carsten Jakobson, their internal process champion, has •

Sources: IEEE digital Library, “Scrum and CMMI Going From G Repeated at Palm on web operating system code. Systematic 24x less ood To Great”. Jeﬀ Sutherland and Carsten Jakobson if if resolved in sprint. .

impede Relationships are tense and resentful. Legacy systems and practices Launch a deep usability discipline. usability a discipline. deep Launch collaboration.Improve cross-competency goodbyeSay to waterfall. Adopt perspective. a product Start a technical debt management program. debt Startmanagement a technical Business users are patiencelosing IT old-school with culture. agility. Bottom AGILE line - GET

We’ll be done We’ll be by sprint 10! 6

Sprint

Source: Adapted from Henrik Kniberg

definitely bydone Sorry, we’re late! we’re Sorry, 11 by sprint 12!

We should 12

Um... we’re done we’re Um... when we’re we’re when done!

velocity No increasedtechnicaldebt • Definition ofDone

.... time bla

bla

Stop accumulating debt Slow down First step .... Source: Adapted from Henrik Kniberg Sustainable pace

Technical debtdecreased • Definition ofDone

.... bla Start repaying debt Slow down even more (optional) Second step

bla ....

Increasing pace!

that piece of the solution. refactored in place to (re)establish elegant architecture in Done. call our solution done it meetsuntil our Definition of don’t are robust! event or control programmatically (pinouts in hardware) placement are brittle, UI tests that call the button click Build Quality In. Ownership. Shared Elegance. improve the Definition of Done: Craftspersonship • • • • •

Testing Practice User Acceptance testing. regression testing, integration testing, Unit testing, User Acceptance Testing, UI tests that rely on mapsclick or control Done solutions are not just tested, but . Always inspired and to inspiring Test QA Dev’s doesn’t solution, we

half of the team went back to India. velocity. railway station signs at all Netherlands railway stations Defect rates are less than 1 per 1000 lines of code and steadily ge Done at the end of the Sprint means customer has accepted the code as r The customer completes acceptance testing on all features during each Spr Scrum teams run all XP practices inside the Scrum intensiveincluding After establishing local velocity at 5 times other waterfall ven An 8 person half-Dutch and half-Indian Scrum team started the pro ProRail rescued a failed waterfall project to abuild new scheduli

Case study: Building a new railway information a Building new Case study:

ProRail) OUTSOURCED QA?

dors on the project, the Indian ng system and automated

tting tting lower. ject and established local pair programming. eady for production. int. int.

using using FitNesse 95% of Bugs are found and fixed the within Sprint Regression tests are run continuously Developers write ‘fixtures’, link testswhich thewith production co Testers write functional tests thewhile user story beingis implemen Analysts, testers and developers define together how a user story wil

de ted l l be tested

If you are the team Scrum If Master: Scrum team the are you Owner: IfProduct team the are you Ifteam the are you Test XPor Lead champion: practices of Engineering: IfDirector the are you If CEO,Microsoft: the BAN are like you TEST TEAMS. Crush Your Competitors Agile with Testing • • • • • • • • • •

Measure time to time fix bug. a Measure In Place. Refactor includes of Done Definition Demand debt. technical increased no include estimates all Demand Write test. testa failing a as in tool backlog can who team your on of people number the visible Make club. book a Host interested. everyone with Share as DevTestOps ability own your Practice refactoring. radical and architecture elegant on up Read consu a in or bring yourself, team for each servers build Configure encourage and Identify Test XPas Leads ea in champions practices • • •

analysis and systematize the resolution. the resolution. systematize and analysis perfo Systematic, for immediately If a bug, Like 2 hours than more coverage. Make Automated Test team’s each measure visible, Coverage in increase Turn Gated Check-in. on

. dev , test, deploy. rm root cause rm root cause ltant to do it. to ltant do ch team. ch code code

@ScrumInc, @JoeJustice0, scrum and scrum inc. store to view upcoming topics. advance your scrum ourwith online courses. Visit the Scrumlab http://www.scruminc.com/category/blog/ www.scruminc.com/scrumlab articles, online courses, tools, and papers on all things scrum For up coming events, new content releases, and more!

Stay Connected

•

more complete model of progressing requirements acceptance Example model (sharper definition of done) The move toward Acceptance Test Driven Development requires a • Conditions ofConditions Satisfaction - broad terms

Acceptance Criteria - further refined •

• Examples - actual scenarios or data

Executable Examples - Executable Spec

Password: Email Address: Register to Password and Email your Enter It consists of specifying email (as User ID) and Password Suppose we are creating a registration function

• Password: Email Address: Register to Password and Email your Enter

business stakeholders Epic/Feature/Userinitial Story The PO definewould the conditions of acceptance in concert with ofConditions Satisfaction are high level criteria established with • In our previous example, the conditions of acceptance for the passwo

Ensure the password notis easy to crack.

rd be:would an

requirements: stakeholder to define these criteria Actual examples are the ultimate way to communicate The PO works aclosely with tester, developer, and business ajx972dab 1aaaaaaaa abcdefghi abc123 Password

Valid Valid Invalid Invalid Expected

character and one number one and character Your one at least must contain password long charcters 12 than more no and Your characters 8 at must least be password Message Expected

Why valid? Comment

•

then against:build stakeholder to define these criteria Acceptance Criteria specifies the details that the team can The PO works aclosely with tester, developer, and business Register to Password and Email your Enter Password: Email Address: • • • • • Following the example for Password:

Must be at least 8 characters and no more than 12 Etc. Etc. (there may be more criteria) Must contain at least one character Must contain at least one digit Must contain only alpha-numberics and the period

DevTestOps MIND BLOWING STEP NEXT

If you are the team Scrum If Master: Scrum team the are you Owner: IfProduct team the are you IfXPteam the are you champion: practices of Engineering: IfDirector the are you If CEO,Microsoft: the BAN are like you TEST TEAMS. • • • • • • • • • •

Measure time to time fix bug. a Measure In Place. Refactor includes of Done Definition Demand debt. technical increased no include estimates all Demand Write test. testa failing a as in tool backlog can who team your on of people number the visible Make club. book a Host interested. everyone with Share as DevTestOps ability own your Practice refactoring. radical and architecture elegant on up Read consu a in or bring yourself, team for each servers build Configure XP encourage and Identify team. each on champion practices • • •

Conclusions Conclusions . dev , test, deploy. rm root cause rm root cause ltant to do it. to ltant do code code

Case study: Building a new railway information a Building new Case study:

ProRail) OUTSOURCED QA?

dors on the project, the Indian ng system and automated

tting tting lower. ject and established local pair programming. eady for production. int. int.

New New Product Development: Known Product Development: • •

WIKISPEED Red/Green card example. How to How rescue a failed Scrum. • • •

than a sprint. Split until experiment can be run and tested in less Start epics with phrased as tests definition of done, DEPLOYABLE. Add testers to the Scrum team to meet the real Testing Implementation

• • • • 95% of defects found inside iteration are eliminated before the Defect rate gets lower and lower as code base increases in size

techniques onshore and offshore. They do the same thing in the same way. Indians say feelsit exactly the same in India as doesit in Amster SmartBoards, and wikis, other tools used to enhance communication Daily Skype video Scrum meeting of team across geographies. TDD, pair programming, continuous integration. Same tools and

end of the iteration

dam.

integration testing, Unit testing, User Acceptance Testing, User Acceptance testing. regression testing, Categories of Testing

elegance, qualitybuilding in. Code Craftpersonship, shared ownership, The mindset of managing a successful agile testing practice.

and Software. Gated Check-in and Rapid Roll Back in hardware Good Good Agility Starts Good Testingwith

Build Build Agile Agile Testing Testing? Deploy Deploy gile gile

Corroborated studies in the U.S. and Europe. the current sprint. It costs 24x as much to fix a bug that escapes The solution? Found at Palm and Systematic. The numbers for exploratory testing, manual •

only testers can move cards across backlog. time, implemented by testers devjoining team and Testing in the sprint saves 24 times the bug fix Sources: IEEE digital Library, “Scrum and CMMI Going From G ood To Great”. Jeﬀ Sutherland and Carsten Jakobson .