Agile Test Automation Strategy, V2.Pptx
Total Page:16
File Type:pdf, Size:1020Kb
MJ Half-day Tutorials 10/3/16 13:00 Test Automation Strategies for the Agile World Presented by: Bob Galen Velocity Partners Brought to you by: 350 Corporate Way, Suite 400, Orange Park, FL 32073 888---268---8770 ·· 904---278---0524 - [email protected] - http://www.starwest.techwell.com/ Bob Galen Velocity Partners An agile methodologist, practitioner, and coach, Bob Galen ([email protected]) helps guide companies in their adoption of Scrum and other agile methodologies and practices. Bob is a principal agile evangelist at Velocity Partners; president of RGCG; and frequent speaker on software development, project management, software testing, and team leadership. He is a Certified Scrum Coach, Certified Scrum Product Owner, and an active member of the Agile and Scrum Alliances. Bob published Scrum Product Ownership–Balancing Value from the Inside Out. Test Automation Strategies for the Agile World Bob Galen President & Principal Consultant RGCG, LLC [email protected] Introduction Bob Galen n Independent Agile Coach (CEC) at RGCG, LLC n Director, Agile Practices at n Somewhere ‘north’ of 30 years overall experience J n Wide variety of technical stacks and business domains n Developer first, then Project Management / Leadership, then Testing n Senior/Executive software development leadership for 20+ years n Practicing formal agility since 2000 n XP, Lean, Scrum, and Kanban experience n From Cary, North Carolina Bias Disclaimer: Agile is THE BEST Methodology for Software Development… However, NOT a Silver Bullet! Copyright © 2016 RGCG, LLC 2 1 Outline n Traditional Automation – Business Case & ROI n 3-Pillars n Agile Test Automation Pyramid n Agile Automation – Business Case & ROI n Implementation Strategy n Communication n Wrap-up Copyright © 2016 RGCG, LLC 3 Let’s start with… Traditional Automation Strategy n What are your current strategies towards: q Test Automation q Frameworks q Tooling q Maintenance q ROI q Team structure n Get together in “pairs” and chat about this for 20 minutes. List your approaches and tactics. n Then leverage SWOT or FFA analysis to capture your current position. Then we’ll gather your results… Copyright © 2016 RGCG, LLC 4 2 Let’s start with… Traditional Automation Strategy n SWOT – Often used to analyze strategic direction or positions. List the “position”, then identify aspects of: q Strengths q Weaknesses q Opportunities q Threats Often in Quadrants n Force Field Analysis – again, list a position q Forces (FOR) è q Forces (AGAINST) ç Copyright © 2016 RGCG, LLC 5 Traditional Automation Business Case & ROI Copyright © 2016 RGCG, LLC 6 3 It’s simple really… n Manual testing is: q Slow q Labor intensive q Not intellectually stimulating q Error prone q Iterative q Did I say slow? n And we’re testing software…right? q So we ought to be able to automate the testing. Copyright © 2016 RGCG, LLC 7 It’s simple really… n Automated tests are: q Fast q Low/no errors q Free after initial development q Run continuously q Did I say fast? q Did I say ‘free’? n And we’re testing software…right? q So we ought to be able to automate ALL of the testing. Copyright © 2016 RGCG, LLC 8 4 Test Automation ROI Sample Points – Manual Testing ü Capture size of the team ü Cost per tester – usually hourly ü # of Test cases to be run ü Time to run them ü Cyclical nature of the testing cycles (regression, integration, system, etc.) ü Configurations ü Number of tests run per release ü Number of releases per year n Do the math to come up with iterative testing run-time investments…and costs Copyright © 2016 RGCG, LLC 9 Test Automation ROI Sample Points – Automation Development n Then contrast that against the cost for automation: ü Time / cost to establish automation infrastructure ü Time / cost to establish automated tests ü Time / cost for ongoing maintenance and results analysis ü Cost for automation tooling – depends on ‘type’, ex: Keyword Driven ü Cost per automation developer vs. manual tester ü Automation replacement target for manual test cases n Do the simple, discrete math to come up with cost & time to replace manual testing run-time and ROI savings Copyright © 2016 RGCG, LLC 10 5 Traditional Business Case n Run more tests, faster and with fewer people (lower costs) n VALUE being driven by: Running tests! n Did I say with fewer people (testers)? q We can hire more developers with the cost savings! And get MORE done… q We can even reduce the size of the bathrooms J Copyright © 2016 RGCG, LLC 11 http://www.aspiresys.com/testautomationroi/index.php#ROI Copyright © 2016 RGCG, LLC 12 6 http://www.aspiresys.com/testautomationroi/index.php#ROI Copyright © 2016 RGCG, LLC 13 Sample of Test Automation ROI calculators n Aspire Systems seems to have pulled down their link, here are other samples… q http://sourceforge.net/projects/ta-roi/ q http://www.automatedtestinginstitute.com/home/index.php? option=com_content&view=article&id=58:roi- calculator&catid=37:calculators&Itemid=65 q http://www-01.ibm.com/software/rational/offerings/testing/roi/tool/ ROI_Rational.html Copyright © 2016 RGCG, LLC 14 7 But often there is underestimation of risks… So the ROI was always a ‘fuzzy’ estimate § Underestimating the nature of a software project § Architecture & design, complexity, risks, unknowns, ambiguity, etc. § Underestimating parallel software projects (product + automation) complexity § Integration, dependencies, coordination § Underestimating ongoing support & maintenance § Repairs & updates § Designing and adding new tests § Impact of new product technologies § Underestimating skill levels for automation development Copyright © 2016 RGCG, LLC 15 It’s also very dangerous! Will Robinson… § It’s commoditizing your testing activity. Could you imagine – § Measuring developer value by LOC…and doing ROI? § Measuring architectural integrity by complexity analysis…and doing ROI? § Measuring product owner value by number of backlog elements…and doing ROI? Based on some sort of automation introduction? Of course not. It’s insane. § Then why do we do it with tests…and testers? Copyright © 2016 RGCG, LLC 16 8 Traditional Test Automation Tools Just a quick stop… n We’re all using them, so they must be good and the approaches must be ‘correct’…right? n However, there are challenges— q Training q Size q UI-centric q COST q Can’t test everything (Back-end database code) q One-size fits all strategy q Limited developer / whole team usage Copyright © 2016 RGCG, LLC 17 Typical Deployment Level of effort MORE Typical Time & Budget Initial design Investment & Architecture LESS é Initial Test Level of Effort - Case Automation Investment ê Technology Disruption Ongoing Maintenance Copyright © 2016 RGCG, LLC 18 9 Now it may sound… n Like I’m opposed to test automation. Truly, I’m not. I think automation rocks! Or like I’m opposed to gaining a ‘return’, absolutely not. q Automation is clearly a foundation element for increased quality and going faster – releasing more often n But, this traditional model or approach of focusing on ROI…is sort of insane Copyright © 2016 RGCG, LLC 19 3-Pillars of Agile Quality & Testing Copyright © 2016 RGCG, LLC 20 10 3-Pillars Genesis n I’ve learned that “Balance” is important n A sad tale of: q Thousands of ATDD testing; Gherkin run amok q All of them are working; continuously testing; increasing “coverage’ and life is Good! n BUT q These same teams couldn’t write a cohesive User Story to save their life q So, where were the Acceptance Tests coming from? Copyright © 2016 RGCG, LLC 21 3-Pillars of Agile Quality Development & Test Software Testing Cross-Functional Team Automation Practices • Risk-based testing: • Pyramid-based Strategy: Functional & Non-Functional • Team-based Pairing (Unit + Cucumber + Selenium) • Test planning @ Release & • Stop-the-Line Mindset Sprint levels • Continuous Integration • Code Reviews & Standards • Exploratory Testing • Attack technical • Active Done-Ness infrastructure in the Backlog • Standards – checklists, templates, repositories • Aggressive Refactoring of • Visual Feedback – Technical Debt Dashboards • Balance across manual, exploratory & automation • User Stories, “3 Amigo” • Actively practice ATDD and based Conversations BDD • Whole Team Ownership of “Quality” • Knowing the Right Thing to Build; And Building it Right • Healthy – Agile Centric Metrics • Steering via: Center of Excellence or Community of Practice • Strategic balance across 3 Pillars; Assessment, Recalibration, and Continuous Improvement 22 Copyright © 2016 RGCG, LLC 11 Foundation of the 3-Pillars • Whole team view includes building it right, • Whole Team Ownership of everyone tests, everyone demo’s, etc. “Quality” • Focus on features/stories, confirmation, • Knowing the “Right” thing to conversation, and getting them staged Build AND Building it “Right” properly OVER testing • Healthy – Agile Centric Metrics • 4-tier metrics: Quality, Value, Prediction, Team • Steering Required – CoE or CoP • Agile strategies need light-handed “steering”; establish a CoE (heavier weight) or a CoP • Strategic balance across 3 (lightweight) Pillars; Assessment, Recalibration, and Continuous • Consider finding an assessment framework Improvement and then tying it to your strategy measurement, recalibration, and continuous improvement. • Make the foundation visible thru information radiators and metrics 23 Copyright © 2016 RGCG, LLC 3-Pillars of Agile Quality A central part of agile adoption is focusing on CI, 3- Development & tiered Automation development, and Dashboards to Test Automation begin incrementally building coverage for faster feedback