sign in sign up
<<
Home , Continuous testing, Functional testing, Integration testing, Test case

A CONFUSED TESTER IN AGILE WORLD … QA A LIABILITY OR AN ASSET

THIS IS A WORK OF FACTS & FINDINGS BASED ON TRUE STORIES OF ONE & MANY TESTERS !! J

Presented By Ashish Kumar, WHAT’S AHEAD • A STORY OF TESTING. • FROM THE MIND OF A CONFUSED TESTER. • FEW CASE STUDIES. • CHALLENGES IDENTIFIED. • SURVEY STUDIES. • GLOBAL RESPONSES. • SOLUTION APPROACH. • PRINCIPLES AND PRACTICES. • CONCLUSION & RECAP. • Q & A.

A STORY OF TESTING IN AGILE… HAVE YOU HEARD ANY OF THESE ??

• YOU DON’T NEED A DEDICATED TESTING TEAM ON YOUR AGILE TEAMS • IF WE HAVE BDD,ATDD,TDD,UI AUTOMATION , UNIT TEST >> WHAT IS THE NEED OF ?? • WE WANT 100% AUTOMATION IN THIS PROJECT • TESTING IS BECOMING BOTTLENECK AND REASON OF SPRINT FAILURE • REPEATING REGRESSION IS A BIG TASK AND AN OVERHEAD • MICROSOFT HAS NO TESTERS NOT EVEN GOOGLE, FACEBOOK AND CISCO • 15K+ DEVELOPERS /4K+ PROJECTS UNDER ACTIVE • IN A “MOBILE-FIRST AND CLOUD-FIRST WORLD.” DEVELOPMENT/50% CODE CHANGES PER MONTH. • THE EFFORT, KNOWN AS AGILE , • 5500+ SUBMISSION PER DAY ON AVERAGE IS DESIGNED TO LOWER COSTS AND HONE OPERATIONS AS THE COMPANY FOCUSES ON BUILDING CLOUD AND • 20+ SUSTAINED CODE CHANGES/MIN WITH 60+PEAKS MOBILE SOFTWARE, SAY ANALYSTS • 75+ MILLION TEST CASES RUN PER DAY. • MR. NADELLA TOLD BLOOMBERG THAT IT MAKES MORE • DEVELOPERS OWN TESTING AND DEVELOPERS OWN SENSE TO HAVE DEVELOPERS TEST & FIX BUGS INSTEAD OF QUALITY. SEPARATE TEAM OF TESTERS TO BUILD CLOUD SOFTWARE. • GOOGLE HAVE PEOPLE WHO COULD CODE AND WANTED • SUCH AN APPROACH, A DEPARTURE FROM THE TO APPLY THAT SKILL TO THE DEVELOPMENT OF TOOLS, COMPANY’S TRADITIONAL PRACTICE OF DIVIDING INFRASTRUCTURE, AND . ENGINEERING TEAMS. • “DEVELOPER SKILLS AND A TESTER MINDSET.” • WOULD MAKE MICROSOFT MORE EFFICIENT, ENABLING IT TO CUT COSTS WHILE BUILDING SOFTWARE FASTER, • GOOGLE PERFORMS A GREAT DEAL OF MANUAL TESTING, EXPERTS SAY. BOTH SCRIPTED AND EXPLORATORY, BUT EVEN THIS TESTING IS DONE UNDER THE WATCHFUL EYE OF

Source: AUTOMATION. Wall Street Journal: http://blogs.wsj.com/cio/2014/07/15/microsoG‐plots‐agile‐development‐course‐as‐talk‐on‐job‐cuts‐loom/ Mico J Tools for at Google Scale https://www.youtube.com/watch?v=KH2_sB1A6lA&feature=youtu.be How Google test Software :James W, Jason A, Jeff C

FROM THE MIND OF A CONFUSED TESTER • IS QA PART OF THE DEVELOPMENT TEAM? • CAN WE FIT QA IN THE SAME ITERATION AS DEVELOPMENT? • SHOULD I FOCUS ON MANUAL OR AUTOMATION • HOW CAN WE SCALE AGILE QA? • WHO DOES QA? • DOES QA COSTS MORE IN AGILE AS PRODUCT SEEMS TO CHANGE FROM SPRINT TO SPRINT? • DO WE NEED “TEST PLAN”? • ARE STORY ACCEPTANCE TESTS ENOUGH? • WHEN DO WE KNOW TESTING IS DONE? • WHO DEFINES TEST CASES? • DO WE NEED TO TRACK BUGS? TEST ENGINEERING @ GOOGLE – ITS NOT QA QA AND AGILE ARE INEXTRICABLY INTERTWINED….

•BUT QUITE OFTEN IN AGILE ORGANIZATIONS, THE ART OF QA IS NOT WELL UNDERSTOOD. •THE VERY ESSENCE OF AGILE DEVELOPMENT IS DELIVERING QUALITY WORKING SOFTWARE FREQUENTLY. ´IN AGILE PROJECTS, QA SHOULD BE EMBEDDED IN THE SCRUM TEAMS BECAUSE TESTING AND QUALITY IS NOT AN AFTERTHOUGHT. ´QUALITY SHOULD BE BAKED IN RIGHT FROM THE START.

Source : https://www.linkedin.com/pulse/agile-just-tech-elite-david-akka Case 1

Project Description Type: Enhancement and Maintenance Project ; Domain : Core banking Team Size: 40 ; With Agile : < 5 years

QA Roles No testers on Team

QA Approach 1. Whole Team Approach over Testing Departments and Independent Testing 2. Developers perform Automation and Cross developed verification. 3. TDD 4. Developers Develops Unit test Case > Story Development > Functional Automation Test Case> > Done

Challenges 1.Hiring testers who can code features is difficult; finding feature developers who can test is even more difficult. 2. Maintenance is a BIG challenge 3. Non- during sprint is a challenge

One Query Why Should we pay more for Manual testing The 'whole team' approach has helped in instilling sense of ‘Inclusiveness’ within the team. It has also helped in reducing delays & improved the overall team efficiency. It is been a paradigm shift for many. Case 2

Project Description Type: Development Project ; Domain : Finance With Agile : < 3+ years

QA Roles 1. Cross-Functional Team 2. Functional Tester Performing both the task of Manual validation and Automation QA Approach 1. ATDD. 2. Team together works on test scenarios > Dev – develop the stories ||QA – Develop test case > QA automate the test scenario || Developers pitch in for help > BA Validates > Done 3. Whole Team Approach , Developers also supports QA to perform Automation and Verification

Challenges 1. Shortened time for testing 2. Sub-standard delivery of few stories towards the end of sprint 3. Spill Over 4. Testing backlog creation One Query Why should we duplicate the effort by having separate roles as manual and automation testers.

We believe in spirit of agile, it was difficult to break the shackles of mindset and create an effective whole team approach. But it works wonder for us although we have lot of scope for improvements Case 3

Project Description Type: Mission Critical Products; With Agile : 5+ years. More inline with DAD approach

QA Roles • Manual testers as Part of scrum team ( Work as product experts) • Automation testers distributed among different teams • Field Engineers along with PO does UAT, Regulatory Testing etc..

QA Approach • Component to verify :- Hardware , Firmware, Application Software • Application :- Automate, Interface :- Automate, Portion of Firmware and H/W :- Automate • Unit and by developers. System Integration and by QA • Because of complex integration and system dependency, dedicated hardening sprints at the end Challenges • Sometime there is lag in automation. • Risk based testing as all configurations can’t be testing before release • Work load is uneven for Manual Test team. One Query Can we make the non-functional test also a part of sprint, if yes how ?

Quality is everyone responsibility. Agile has made it true. It's not Developers or QA who is owning but right from customer everyone is building Quality in the product Case 4

Project Description Type: Development Project Team Size: 45 ; With Agile : 1~2 years

QA Roles 1. Separate Testing team / Vendor for QA

QA Approach 1. Development Sprint and QA sprint are separate. 2. Both the Sprint have different sprint goals and deliverables. 3. QA Sprint always lag by one dev sprint 4. They work on current sprint test scenarios and verifying previous sprint deliverables. 5. All the QA activities Functional and Non-functional are taken care in QA sprint. Challenges 1. Teams working in Silos 2. The approach is very much waterfall 3. Defects and issues found is QA sprint are part of product backlog.

One Query Why do we need to release sub standard builds in every sprint.

We Effectively synchronized QA activity on a distributed development model with dedicated QA-Dev pairing. Did not reduce QA’s Importance to unit test dev’s substandard build. A Sneak Peak into the past..

https://www.scrumalliance.org/community/articles/2015/june/a-confused-tester-in-agile-world LETS IDENTIFY SOME MORE CHALLENGES CHALLENGES IDENTIFIED

• CHANGING / LAST MINUTE CHANGES • NOT ENOUGH INFORMATION ON THE STORY • • TECHNICAL SKILLS / TEST AUTOMATION • MULTIPLE BROWSERS / MULTIPLE DEVICES • COMMUNICATION :: “TO PRODUCE AND COMMUNICATE RELEVANT INFORMATION PROMPTLY” • FEAR TO LOSE IDENTITY • COLLABORATION :: “TO MAKE TESTING, DEVELOPMENT AND BUSINESS COLLABORATE” • HOW TO KEEP UP WITH THE PACE OF THE DEVELOPMENT? • HOW TO TEST EARLY BUT NOT DO ANTICIPATORY TEST DESIGN?

STUDY OF AGILE PRACTICES IMPLEMENTATION IN DISTRIBUTED SOFTWARE DEVELOPMENT – A REFERENCE

This is a research conducted by Manjunath M S Rao, Vijay Wade and M M Jha .

This Paper presents the results of a systematic study of implementation of agile practices, which covers the summary of most effectively implemented practices, most widely recommended practices and least implemented practices in Global (GSE).

The findings are based on the survey data collated from 22 agile practitioners from 14 different software organizations spread across the globe.

1 to 2 14%

2 to 5 years 57% FEW SAMPLE SURVEY QUESTIONS

1. ARE RELEASE BACKLOGS BUILT WITH THE INVOLVEMENT OF THE RELEVANT STAKEHOLDERS ( PRODUCT OWNER, SALES/MKT, PRODUCT MANAGER, ARCHITECTS, BUSINESS ANALYSTS, SYSTEM TESTING. ETC..)? 2. IS THE TEAM CROSS FUNCTIONAL AND INDEPENDENT TO DELIVER A FUNCTIONAL SOFTWARE(STORY) WITHIN A SPRINT ? 3. ARE RISKS AND ISSUES GETTING TRACKED WITHIN SPRINTS? 4. HAVE YOU IMPLEMENTED XP PRACTICES LIKE TEST DRIVEN DEVELOPMENT, PAIR PROGRAMMING ETC? PLEASE PROVIDE DETAILS IN REMARKS 5. DO YOU HAVE A SETUP TO HANDLE CONTINUOUS INTEGRATION AND DELIVERY TO MAINTAIN THE PACE OF DELIVERY?

6. ARE YOU USING AUTOMATION TO OPTIMIZE EFFORT AND TO IMPROVE PRODUCT QUALITY? 7. IS THE DELIVERABLE AT THE END OF THE SPRINT / ITERATION READY TO BE SHIPPED ?( IS THERE A SEPARATE TESTING PHASE OR DELIVERABLE FROM EACH SPRINT IS READY TO SHIP?)

8 8 6

IS QA AN ASSET ON YOUR TEAM ?? J J

Response Recommendaons 1.Always 1.Strongly recommended 2.Someme 2. Recommended

3.Not-Done 3.Not recommended

4.NA 4.Fine Tune RESPONSE AND RECOMMENDATION

Fine Tune 14%

Strongly Sometimes Recommended 36% 45% Recommended Always 41% 64% Source: http://cartoontester.blogspot.in/2010_01_01_archive.html If Time permits - Appendix

PRINCIPLES AND PRACTICES

• TESTING MOVES THE PROJECT FORWARD • TESTING IS NOT A PHASE……ON AGILE TEAMS, TESTING IS A WAY OF LIFE. CONTINUOUS TESTING IS THE ONLY WAY TO ENSURE CONTINUOUS PROGRESS. • EVERYONE TESTS – WHOLE TEAM APPROACH – COLLABORATION • SHORTENING FEEDBACK LOOPS • KEEP THE CODE CLEAN • LIGHTWEIGHT DOCUMENTATION • TEST-LAST V. TEST-DRIVEN Source: Quality Tree Software, Inc Source: Quality Tree Software, Inc Source: Quality Tree Software, Inc

Q1: In Agile Methodology, Some think that with focus more on complete automation and continuous integration, the job will become less important or obsolete. What's your Take on This ??

Source: https://www.linkedin.com/groups/961927/961927-6145911947582808065?trk=hb_ntf_COMMENTED_ON_GROUP_DISCUSSION_YOU_CREATED#commentID_6145995769821302784 https://www.linkedin.com/groups/49087/49087-6145911731643260933?trk=hb_ntf_LIKED_GROUP_DISCUSSION_YOU_CREATED https://www.linkedin.com/groups/99444/99444-6145911085917564931?trk=hb_ntf_LIKED_GROUP_DISCUSSION_YOU_CREATED

Q2 : Contrary to the synchronous activities of traditional waterfall project, agile expects development action to be performed in order they are needed more of asynchronous we can say. Now what are the different ways for a testing professional to engage EFFECTIVELY during a sprint before any feature has been built?

Source; https://www.linkedin.com/groups/55636/55636-6146759619357265922?trk=hb_ntf_LIKED_GROUP_DISCUSSION_YOU_CREATED Q3 : How may of you think that S/W quality is compromised due to shortened period allotted for testing with major emphasis on development. Also if you can please add reasons for this & correction measures.

Source; https://www.linkedin.com/groups/55636/55636-6146757659057020932?trk=hb_ntf_LIKED_GROUP_DISCUSSION_YOU_CREATED Q4: During My Pursuit of identifying the approach different teams are following for testing in an agile environment I came across some following cases, (The details given are brief) REFERENCES

https://www.scrumalliance.org/community/articles/2015/june/a-confused-tester-in-agile-world “Life is a sprint. If you start on the right note, you enjoy it till the end, and win at the last.”

@A_ashish_kumar

[email protected]