DOCSLIB.ORG

A Confused Tester in Agile World … Qa a Liability Or an Asset

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
A Confused Tester in Agile World … Qa a Liability Or an Asset A CONFUSED TESTER IN AGILE WORLD … QA A LIABILITY OR AN ASSET THIS IS A WORK OF FACTS & FINDINGS BASED ON TRUE STORIES OF ONE & MANY TESTERS !! J Presented By Ashish Kumar, WHAT’S AHEAD • A STORY OF TESTING. • FROM THE MIND OF A CONFUSED TESTER. • FEW CASE STUDIES. • CHALLENGES IDENTIFIED. • SURVEY STUDIES. • GLOBAL RESPONSES. • SOLUTION APPROACH. • PRINCIPLES AND PRACTICES. • CONCLUSION & RECAP. • Q & A. A STORY OF TESTING IN AGILE… HAVE YOU HEARD ANY OF THESE ?? • YOU DON’T NEED A DEDICATED SOFTWARE TESTING TEAM ON YOUR AGILE TEAMS • IF WE HAVE BDD,ATDD,TDD,UI AUTOMATION , UNIT TEST >> WHAT IS THE NEED OF MANUAL TESTING ?? • WE WANT 100% AUTOMATION IN THIS PROJECT • TESTING IS BECOMING BOTTLENECK AND REASON OF SPRINT FAILURE • REPEATING REGRESSION IS A BIG TASK AND AN OVERHEAD • MICROSOFT HAS NO TESTERS NOT EVEN GOOGLE, FACEBOOK AND CISCO • 15K+ DEVELOPERS /4K+ PROJECTS UNDER ACTIVE • IN A “MOBILE-FIRST AND CLOUD-FIRST WORLD.” DEVELOPMENT/50% CODE CHANGES PER MONTH. • THE EFFORT, KNOWN AS AGILE SOFTWARE DEVELOPMENT, • 5500+ SUBMISSION PER DAY ON AVERAGE IS DESIGNED TO LOWER COSTS AND HONE OPERATIONS AS THE COMPANY FOCUSES ON BUILDING CLOUD AND • 20+ SUSTAINED CODE CHANGES/MIN WITH 60+PEAKS MOBILE SOFTWARE, SAY ANALYSTS • 75+ MILLION TEST CASES RUN PER DAY. • MR. NADELLA TOLD BLOOMBERG THAT IT MAKES MORE • DEVELOPERS OWN TESTING AND DEVELOPERS OWN SENSE TO HAVE DEVELOPERS TEST & FIX BUGS INSTEAD OF QUALITY. SEPARATE TEAM OF TESTERS TO BUILD CLOUD SOFTWARE. • GOOGLE HAVE PEOPLE WHO COULD CODE AND WANTED • SUCH AN APPROACH, A DEPARTURE FROM THE TO APPLY THAT SKILL TO THE DEVELOPMENT OF TOOLS, COMPANY’S TRADITIONAL PRACTICE OF DIVIDING INFRASTRUCTURE, AND TEST AUTOMATION. ENGINEERING TEAMS. • “DEVELOPER SKILLS AND A TESTER MINDSET.” • WOULD MAKE MICROSOFT MORE EFFICIENT, ENABLING IT TO CUT COSTS WHILE BUILDING SOFTWARE FASTER, • GOOGLE PERFORMS A GREAT DEAL OF MANUAL TESTING, EXPERTS SAY. BOTH SCRIPTED AND EXPLORATORY, BUT EVEN THIS TESTING IS DONE UNDER THE WATCHFUL EYE OF Source: AUTOMATION. Wall Street Journal: http://blogs.wsj.com/cio/2014/07/15/microsoG‐plots‐agile‐development‐course‐as‐talk‐on‐job‐cuts‐loom/ Mico J Tools for Continuous Integration at Google Scale https://www.youtube.com/watch?v=KH2_sB1A6lA&feature=youtu.be How Google test Software :James W, Jason A, Jeff C FROM THE MIND OF A CONFUSED TESTER • IS QA PART OF THE DEVELOPMENT TEAM? • CAN WE FIT QA IN THE SAME ITERATION AS DEVELOPMENT? • SHOULD I FOCUS ON MANUAL OR AUTOMATION • HOW CAN WE SCALE AGILE QA? • WHO DOES QA? • DOES QA COSTS MORE IN AGILE AS PRODUCT SEEMS TO CHANGE FROM SPRINT TO SPRINT? • DO WE NEED “TEST PLAN”? • ARE STORY ACCEPTANCE TESTS ENOUGH? • WHEN DO WE KNOW TESTING IS DONE? • WHO DEFINES TEST CASES? • DO WE NEED TO TRACK BUGS? TEST ENGINEERING @ GOOGLE – ITS NOT QA QA AND AGILE ARE INEXTRICABLY INTERTWINED…. •BUT QUITE OFTEN IN AGILE ORGANIZATIONS, THE ART OF QA IS NOT WELL UNDERSTOOD. •THE VERY ESSENCE OF AGILE DEVELOPMENT IS DELIVERING QUALITY WORKING SOFTWARE FREQUENTLY. ´IN AGILE PROJECTS, QA SHOULD BE EMBEDDED IN THE SCRUM TEAMS BECAUSE TESTING AND QUALITY IS NOT AN AFTERTHOUGHT. ´QUALITY SHOULD BE BAKED IN RIGHT FROM THE START. Source : https://www.linkedin.com/pulse/agile-just-tech-elite-david-akka Case 1 Project Description Type: Enhancement and Maintenance Project ; Domain : Core banking Team Size: 40 ; With Agile : < 5 years QA Roles No testers on Team QA Approach 1. Whole Team Approach over Testing Departments and Independent Testing 2. Developers perform Automation and Cross developed verification. 3. TDD 4. Developers Develops Unit test Case > Story Development > Functional Automation Test Case> Exploratory testing > Done Challenges 1.Hiring testers who can code features is difficult; finding feature developers who can test is even more difficult. 2. Maintenance is a BIG challenge 3. Non-functional testing during sprint is a challenge One Query Why Should we pay more for Manual testing The 'whole team' approach has helped in instilling sense of ‘Inclusiveness’ within the team. It has also helped in reducing delays & improved the overall team efficiency. It is been a paradigm shift for many. Case 2 Project Description Type: Development Project ; Domain : Finance With Agile : < 3+ years QA Roles 1. Cross-Functional Team 2. Functional Tester Performing both the task of Manual validation and Automation QA Approach 1. ATDD. 2. Team together works on test scenarios > Dev – develop the stories ||QA – Develop test case > QA automate the test scenario || Developers pitch in for help > BA Validates > Done 3. Whole Team Approach , Developers also supports QA to perform Automation and Verification Challenges 1. Shortened time for testing 2. Sub-standard delivery of few stories towards the end of sprint 3. Spill Over 4. Testing backlog creation One Query Why should we duplicate the effort by having separate roles as manual and automation testers. We believe in spirit of agile, it was difficult to break the shackles of mindset and create an effective whole team approach. But it works wonder for us although we have lot of scope for improvements Case 3 Project Description Type: Mission Critical Products; With Agile : 5+ years. More inline with DAD approach QA Roles • Manual testers as Part of scrum team ( Work as product experts) • Automation testers distributed among different teams • Field Engineers along with PO does UAT, Regulatory Testing etc.. QA Approach • Component to verify :- Hardware , Firmware, Application Software • Application :- Automate, Interface :- Automate, Portion of Firmware and H/W :- Automate • Unit and Integration Testing by developers. System Integration and System Testing by QA • Because of complex integration and system dependency, dedicated hardening sprints at the end Challenges • Sometime there is lag in automation. • Risk based testing as all configurations can’t be testing before release • Work load is uneven for Manual Test team. One Query Can we make the non-functional test also a part of sprint, if yes how ? Quality is everyone responsibility. Agile has made it true. It's not Developers or QA who is owning but right from customer everyone is building Quality in the product Case 4 Project Description Type: Development Project Team Size: 45 ; With Agile : 1~2 years QA Roles 1. Separate Testing team / Vendor for QA QA Approach 1. Development Sprint and QA sprint are separate. 2. Both the Sprint have different sprint goals and deliverables. 3. QA Sprint always lag by one dev sprint 4. They work on current sprint test scenarios and verifying previous sprint deliverables. 5. All the QA activities Functional and Non-functional are taken care in QA sprint. Challenges 1. Teams working in Silos 2. The approach is very much waterfall 3. Defects and issues found is QA sprint are part of product backlog. One Query Why do we need to release sub standard builds in every sprint. We Effectively synchronized QA activity on a distributed development model with dedicated QA-Dev pairing. Did not reduce QA’s Importance to unit test dev’s substandard build. A Sneak Peak into the past.. https://www.scrumalliance.org/community/articles/2015/june/a-confused-tester-in-agile-world LETS IDENTIFY SOME MORE CHALLENGES CHALLENGES IDENTIFIED • CHANGING REQUIREMENTS / LAST MINUTE CHANGES • NOT ENOUGH INFORMATION ON THE STORY • CONTINUOUS TESTING • TECHNICAL SKILLS / TEST AUTOMATION • MULTIPLE BROWSERS / MULTIPLE DEVICES • COMMUNICATION :: “TO PRODUCE AND COMMUNICATE RELEVANT INFORMATION PROMPTLY” • FEAR TO LOSE IDENTITY • COLLABORATION :: “TO MAKE TESTING, DEVELOPMENT AND BUSINESS COLLABORATE” • HOW TO KEEP UP WITH THE PACE OF THE DEVELOPMENT? • HOW TO TEST EARLY BUT NOT DO ANTICIPATORY TEST DESIGN? STUDY OF AGILE PRACTICES IMPLEMENTATION IN DISTRIBUTED SOFTWARE DEVELOPMENT – A REFERENCE This is a research conducted by Manjunath M S Rao, Vijay Wade and M M Jha . This Paper presents the results of a systematic study of implementation of agile practices, which covers the summary of most effectively implemented practices, most widely recommended practices and least implemented practices in Global Software Engineering (GSE). The findings are based on the survey data collated from 22 agile practitioners from 14 different software organizations spread across the globe. 1 to 2 14% 2 to 5 years 57% FEW SAMPLE SURVEY QUESTIONS 1. ARE RELEASE BACKLOGS BUILT WITH THE INVOLVEMENT OF THE RELEVANT STAKEHOLDERS ( PRODUCT OWNER, SALES/MKT, PRODUCT MANAGER, ARCHITECTS, BUSINESS ANALYSTS, SYSTEM TESTING. ETC..)? 2. IS THE TEAM CROSS FUNCTIONAL AND INDEPENDENT TO DELIVER A FUNCTIONAL SOFTWARE(STORY) WITHIN A SPRINT ? 3. ARE RISKS AND ISSUES GETTING TRACKED WITHIN SPRINTS? 4. HAVE YOU IMPLEMENTED XP PRACTICES LIKE TEST DRIVEN DEVELOPMENT, PAIR PROGRAMMING ETC? PLEASE PROVIDE DETAILS IN REMARKS 5. DO YOU HAVE A SETUP TO HANDLE CONTINUOUS INTEGRATION AND DELIVERY TO MAINTAIN THE PACE OF DELIVERY? 6. ARE YOU USING AUTOMATION TO OPTIMIZE EFFORT AND TO IMPROVE PRODUCT QUALITY? 7. IS THE DELIVERABLE AT THE END OF THE SPRINT / ITERATION READY TO BE SHIPPED ?( IS THERE A SEPARATE TESTING PHASE OR DELIVERABLE FROM EACH SPRINT IS READY TO SHIP?) 8 8 6 IS QA AN ASSET ON YOUR TEAM ?? J J Response Recommenda0ons 1.Always 1.Strongly recommended 2.Someme 2. Recommended 3.Not-Done 3.Not recommended 4.NA 4.Fine Tune RESPONSE AND RECOMMENDATION Fine Tune 14% Strongly Sometimes Recommended 36% 45% Recommended Always 41% 64% Source: http://cartoontester.blogspot.in/2010_01_01_archive.html If Time permits - Appendix PRINCIPLES AND PRACTICES • TESTING MOVES THE PROJECT FORWARD • TESTING IS NOT A PHASE……ON AGILE TEAMS, TESTING IS A WAY OF LIFE. CONTINUOUS TESTING IS THE ONLY WAY TO ENSURE CONTINUOUS PROGRESS. • EVERYONE TESTS – WHOLE TEAM APPROACH – COLLABORATION • SHORTENING FEEDBACK LOOPS • KEEP THE CODE CLEAN • LIGHTWEIGHT DOCUMENTATION • TEST-LAST V. TEST-DRIVEN Source: Quality Tree Software, Inc Source: Quality Tree Software, Inc Source: Quality Tree Software, Inc Q1: In Agile Methodology, Some think that with focus more on complete automation and continuous integration, the software testing job will become less important or obsolete.
Load more

Recommended publications
  • Core Elements of Continuous Testing
    WHITE PAPER CORE ELEMENTS OF CONTINUOUS TESTING Today’s modern development disciplines -- whether Agile, Continuous Integration (CI) or Continuous Delivery (CD) -- have completely transformed how teams develop and deliver applications. Companies that need to compete in today’s fast-paced digital economy must also transform how they test. Successful teams know the secret sauce to delivering high quality digital experiences fast is continuous testing. This paper will define continuous testing, explain what it is, why it’s important, and the core elements and tactical changes development and QA teams need to make in order to succeed at this emerging practice. TABLE OF CONTENTS 3 What is Continuous Testing? 6 Tactical Engineering Considerations 3 Why Continuous Testing? 7 Benefits of Continuous Testing 4 Core Elements of Continuous Testing WHAT IS CONTINUOUS TESTING? Continuous testing is the practice of executing automated tests throughout the software development cycle. It’s more than just automated testing; it’s applying the right level of automation at each stage in the development process. Unlike legacy testing methods that occur at the end of the development cycle, continuous testing occurs at multiple stages, including development, integration, pre-release, and in production. Continuous testing ensures that bugs are caught and fixed far earlier in the development process, improving overall quality while saving significant time and money. WHY CONTINUOUS TESTING? Continuous testing is a critical requirement for organizations that are shifting left towards CI or CD, both modern development practices that ensure faster time to market. When automated testing is coupled with a CI server, tests can instantly be kicked off with every build, and alerts with passing or failing test results can be delivered directly to the development team in real time.
    [Show full text]
  • Studying the Feasibility and Importance of Software Testing: an Analysis
    Dr. S.S.Riaz Ahamed / Internatinal Journal of Engineering Science and Technology Vol.1(3), 2009, 119-128 STUDYING THE FEASIBILITY AND IMPORTANCE OF SOFTWARE TESTING: AN ANALYSIS Dr.S.S.Riaz Ahamed Principal, Sathak Institute of Technology, Ramanathapuram,India. Email:[email protected], [email protected] ABSTRACT Software testing is a critical element of software quality assurance and represents the ultimate review of specification, design and coding. Software testing is the process of testing the functionality and correctness of software by running it. Software testing is usually performed for one of two reasons: defect detection, and reliability estimation. The problem of applying software testing to defect detection is that software can only suggest the presence of flaws, not their absence (unless the testing is exhaustive). The problem of applying software testing to reliability estimation is that the input distribution used for selecting test cases may be flawed. The key to software testing is trying to find the modes of failure - something that requires exhaustively testing the code on all possible inputs. Software Testing, depending on the testing method employed, can be implemented at any time in the development process. Keywords: verification and validation (V & V) 1 INTRODUCTION Testing is a set of activities that could be planned ahead and conducted systematically. The main objective of testing is to find an error by executing a program. The objective of testing is to check whether the designed software meets the customer specification. The Testing should fulfill the following criteria: ¾ Test should begin at the module level and work “outward” toward the integration of the entire computer based system.
    [Show full text]
  • Customer Success Story
    Customer Success Story Interesting Dilemma, Critical Solution Lufthansa Cargo AG The purpose of Lufthansa Cargo AG’s SDB Lufthansa Cargo AG ordered the serves more than 500 destinations world- project was to provide consistent shipment development of SDB from Lufthansa data as an infrastructure for each phase of its Systems. However, functional and load wide with passenger and cargo aircraft shipping process. Consistent shipment data testing is performed at Lufthansa Cargo as well as trucking services. Lufthansa is is a prerequisite for Lufthansa Cargo AG to AG with a core team of six business one of the leaders in the international air efficiently and effectively plan and fulfill the analysts and technical architects, headed cargo industry, and prides itself on high transport of shipments. Without it, much is at by Project Manager, Michael Herrmann. stake. quality service. Herrmann determined that he had an In instances of irregularities caused by interesting dilemma: a need to develop inconsistent shipment data, they would central, stable, and optimal-performance experience additional costs due to extra services for different applications without handling efforts, additional work to correct affecting the various front ends that THE CHALLENGE accounting information, revenue loss, and were already in place or currently under poor feedback from customers. construction. Lufthansa owns and operates a fleet of 19 MD-11F aircrafts, and charters other freight- With such critical factors in mind, Lufthansa Functional testing needed to be performed Cargo AG determined that a well-tested API on services that were independent of any carrying planes. To continue its leadership was the best solution for its central shipment front ends, along with their related test in high quality air cargo services, Lufthansa database.
    [Show full text]
  • Drive Continuous Delivery with Continuous Testing
    I Don’t Believe Your Company Is Agile! Alex Martins CTO Continuous Testing CA Technologies October 2017 1 Why Many Companies Think They’re Agile… They moved some Dev projects from waterfall to agile They’re having daily standups They have a scrum master Product owner is part of the team They are all talking and walking agile… And are talking about Continuous Delivery BUT… QA is STILL a Bottleneck… Even in DevOps Shops A 2017 survey of self- …of delays were occurring at proclaimed DevOps 63% the Test/QA stage of the practitioners found that … cycle. “Where are the main hold-ups in the software production process?” 63% 32% 30% 16% 22% 21% 23% http://www.computing.co.uk/digital_assets/634fe325-aa28-41d5-8676-855b06567fe2/CTG-DevOps-Review-2017.pdf 3 Challenges to Achieving Continuous Delivery & Testing IDEA Requirements 64% of total defect cost originate in the requirements analysis and design phase1. ? ? of developers time is spent 80% of teams experience delays in development and QA Development 50% 3 finding and fixing defects2 due to unavailable dependencies X X Security 70x required manual pen 30% of teams only security scan 50% more time spent on security test scan cost vs. automation10 once per year9 defects in lower-performing teams8 X X X 70% of all 63% of testers admit they 50% of time 79% of teams face prohibitive QA / Testing testing is still can’t test across all the different spent looking for restrictions, time limits or access manual4 devices and OS versions5 test data6 fees on needed 3rd party services3 ! Release 57% are dissatisfied with the time it takes to deploy new features7 ! ! Ave.
    [Show full text]
  • Your Continuous Testing with Spirent's Automation Platforms
    STREAMLINE Your Continuous Testing With Spirent’s Automation Platforms DE RELE VE AS LO E P E T A ING T R G ES E T T S N U I O U IN T N E CO STAG Table of Contents Agile 3 What is Continuous Test (CT)? 4 What Hinders CT Implementation? 5 The Chasm 6 The Bridge 7 Spirent Velocity Framework 8 Lab as a Service (LaaS) 9 Test as a Service (TaaS) 10 CT Implementation Best Practices 11 Summary 12 2 of 12 Agile software development practices gained momentum in the late 90s. Agile emphasizes close collaboration between business stakeholders, the development team, and QA. This enabled faster software delivery, better quality and improved customer satisfaction. By employing DevOps practices, the pace and benefits are amplified. 3 of 12 What is Continuous Test (CT)? Continuous Test (CT) enables CT haed Cotuous terato ad Deery ee network testing to be more effectively performed by DEVELOP ITEATE STAGE ELEASE development teams by enabling them to take advantage of the QA team’s knowledge of real world customer use cases and environments. This is known as “shift left” because testing is moved earlier in the development cycle. With “shift left” tests are run as early as possible to accelerate understanding of AUTOMATE TESTS ORCHESTRATE TEST ENVIRONMENTSEXECUTE TESTS EARLIER problem areas in the code and where development attention is required. Why Do You Need CT? The combination of earlier and faster testing shortens time to release while improving quality and customer satisfaction. 4 of 12 What Hinders CT Implementation? Deficient or non existent Insufficient test Lack of test results Inadequate understanding of tools for creating resources for timely analysis tools hinder customer environments and automated tests test execution assessment of test results use cases by developers EW SOFTWAE DEVELOP ITEATE STAGE ELEASE SOFTWAE BILD ELEASE The promise of DevOps can’t be fully realized until Continuous Testing (CT) is factored in.
    [Show full text]
  • Leading Practice: Test Strategy and Approach in Agile Projects
    CA SERVICES | LEADING PRACTICE Leading Practice: Test Strategy and Approach in Agile Projects Abstract This document provides best practices on how to strategize testing CA Project and Portfolio Management (CA PPM) in an agile project. The document does not include specific test cases; the list of test cases and steps for each test case are provided in a separate document. This document should be used by the agile project team that is planning the testing activities, and by end users who perform user acceptance testing (UAT). Concepts Concept Description Test Approach Defines testing strategy, roles and responsibilities of various team members, and test types. Testing Environments Outlines which testing is carried out in which environment. Testing Automation and Tools Addresses test management and automation tools required for test execution. Risk Analysis Defines the approach for risk identification and plans to mitigate risks as well as a contingency plan. Test Planning and Execution Defines the approach to plan the test cases, test scripts, and execution. Review and Approval Lists individuals who should review, approve and sign off on test results. Test Approach The test approach defines testing strategy, roles and responsibilities of various team members, and the test types. The first step is to define the testing strategy. It should describe how and when the testing will be conducted, who will do the testing, the type of testing being conducted, features being tested, environment(s) where the testing takes place, what testing tools are used, and how are defects tracked and managed. The testing strategy should be prepared by the agile core team.
    [Show full text]
  • Functional Testing Functional Testing
    From Pressman, “Software Engineering – a practitionerʼs approach”, Chapter 14 and Pezze + Young, “Software Testing and Analysis”, Chapters 10-11 Today, weʼll talk about testing – how to test software. The question is: How do we design tests? And weʼll start with Functional Testing functional testing. Software Engineering Andreas Zeller • Saarland University 1 Functional testing is also called “black- box” testing, because we see the program as a black box – that is, we ignore how it is being written 2 in contrast to structural or “white-box” testing, where the program is the base. 3 If the program is not the base, then what is? Simple: itʼs the specification. 4 If the program is not the base, then what is? Simple: itʼs the specification. Testing Tactics Functional Structural “black box” “white box” • Tests based on spec • Tests based on code • Test covers as much • Test covers as much specified behavior implemented behavior as possible as possible 5 Why Functional? Functional Structural “black box” “white box” • Program code not necessary • Early functional test design has benefits reveals spec problems • assesses testability • gives additional explanation of spec • may even serve as spec, as in XP 6 Structural testing can not detect that some required feature is missing in the code Why Functional? Functional testing applies at all granularity levels (in contrast to structural testing, which only applies to Functional Structural unit and integration testing) “black box” “white box” • Best for missing logic defects Common problem: Some program logic was simply forgotten Structural testing would not focus on code that is not there • Applies at all granularity levels unit tests • integration tests • system tests • regression tests 7 2,510,588,971 years, 32 days, and 20 hours to be precise.
    [Show full text]
  • HP Functional Testing Software Data Sheet
    HP Functional Testing software Data sheet With HP Functional Testing you can automate functional and regression testing for every modern software application and environment, extend testing to a wider range of teams, and accelerate the testing process—so you can improve application quality and still make your market window. Simplifies test creation and HP Functional Testing makes it easy to insert, modify, data-drive, and remove test steps. It features: maintenance • Keyword capabilities: Using keywords, testers HP Functional Testing is advanced, automated can build test cases by capturing flows directly testing software for building functional and regression from the application screens and applying robust test suites. It captures, verifies, and replays user record/replay capturing technology. interactions automatically and helps testers quickly • Automatic updating: With new application builds, identify and report on application effects, while you only need to update one reference in the shared providing sophisticated functionality for tester repository and the update is propagated to all collaboration. The product includes HP QuickTest referencing tests. Professional and all of its add-ins. It is sold stand-alone or as part of the broader HP Unified • Easy data-driving: You can quickly data-drive any Functional Testing solution, which couples object definition, method, checkpoint, and output HP Functional Testing with HP Service Test to value through the integrated data table. address both GUI and non-GUI testing. • Timely advice: In
    [Show full text]
  • Continuous Testing Report 2019
    Continuous Testing Report In association with 2 Continuous Testing Report Contents Introduction 4 Executive summary 6 Current trends in continuous testing Test design 9 Functional and performance testing 13 Test data management 17 Test environment management 21 Test orchestration in the agile enterprise 24 Continuous testing: the road ahead 27 About the study 31 About the sponsors 37 3 Introduction Welcome dear readers. dependency on IT solutions today, with the integration of front-office and consumer-facing apps with back- Quality and testing approaches, methods, and office core systems, the leveraging of cloud and expertise have undergone radical changes over the microservices and the integration and use of IoT. And, last few years. Every organization today aspires on top of that, AI is emerging to make these solutions to deliver faster and more valuable IT solutions to autonomous and self learning. business and customers. To do this, they have been leveraging agile and DevOps methodologies and All this technology is delivered by different teams, using smarter automation technologies and as-a- many of which may not even be part of a single Service solutions to deliver IT faster and with greater company. flexibility. As we scramble to deliver innovative solutions for the At the same time, the IT landscape has also been newer, more complex IT landscape, there is, of course, growing in complexity. There is an increased a risk of failure. While some failures are inevitable and often provide a valuable learning opportunity (given a quick feedback loop), there are others that we must prevent from happening. Failures in core systems that seriously disrupt the business operations of an enterprise, failures that seriously impact a large number of clients and therefore jeopardize an organization’s reliability and brand perception, or failures in systems that cannot easily be rolled back all demand good testing of these systems before being deployed.
    [Show full text]
  • Devsecops DEVELOPMENT & DEVOPS INFRASTRUCTURE
    DevSecOps DEVELOPMENT & DEVOPS INFRASTRUCTURE CREATE SECURE APPLICATIONS PARASOFT’S APPROACH - BUILD SECURITY IN WITHOUT DISRUPTING THE Parasoft provides tools that help teams begin their security efforts as DEVELOPMENT PROCESS soon as the code is written, starting with static application security test- ing (SAST) via static code analysis, continuing through testing as part of Parasoft makes DevSecOps possible with API and the CI/CD system via dynamic application security testing (DAST) such functional testing, service virtualization, and the as functional testing, penetration testing, API testing, and supporting in- most complete support for important security stan- frastructure like service virtualization that enables security testing be- dards like CWE, OWASP, and CERT in the industry. fore the complete application is fully available. IMPLEMENT A SECURE CODING LIFECYCLE Relying on security specialists alone prevents the entire DevSecOps team from securing software and systems. Parasoft tooling enables the BENEFIT FROM THE team with security knowledge and training to reduce dependence on PARASOFT APPROACH security specialists alone. With a centralized SAST policy based on in- dustry standards, teams can leverage Parasoft’s comprehensive docs, examples, and embedded training while the code is being developed. ✓ Leverage your existing test efforts for Then, leverage existing functional/API tests to enhance the creation of security security tests – meaning less upfront cost, as well as less maintenance along the way. ✓ Combine quality and security to fully understand your software HARDEN THE CODE (“BUILD SECURITY IN”) Getting ahead of application security means moving beyond just test- ✓ Harden the code – don’t just look for ing into building secure software in the first place.
    [Show full text]
  • Continuous Quality and Testing to Accelerate Application Development
    Continuous Quality and Testing to Accelerate Application Development How to assess your current testing maturity level and practice continuous testing for DevOps Continuous Quality and Testing to Accelerate Application Development // 1 Table of Contents 03 Introduction 04 Why Is Continuous Quality and Testing Maturity Important to DevOps? 05 Continuous Testing Engineers Quality into DevOps 07 Best Practices for Well- Engineered Continuous Testing 08 Continuous Testing Maturity Levels Level 1: Chaos Level 2: Continuous Integration Level 3: Continuous Flow Level 4: Continuous Feedback Level 5: Continuous Improvement 12 Continuous Testing Maturity Assessment 13 How to Get Started with DevOps Testing? 14 Continuous Testing in the Cloud Choosing the right tools for Continuous Testing On-demand Development and Testing Environments with Infrastructure as Code The Right Tests at the Right Time 20 Get Started 20 Conclusion 21 About AWS Marketplace and DevOps Institute 21 Contributors Introduction A successful DevOps implementation reduces the bottlenecks related to testing. These bottlenecks include finding and setting up test environments, test configurations, and test results implementation. These issues are not industry specific. They can be experienced in manufacturing, service businesses, and governments alike. They can be reduced by having a thorough understanding and a disciplined, mature implementation of Continuous Testing and related recommended engineering practices. The best place to start addressing these challenges is having a good understanding of what Continuous Testing is. Marc Hornbeek, the author of Engineering DevOps, describes it as: “A quality assessment strategy in which most tests are automated and integrated as a core and essential part of DevOps. Continuous Testing is much more than simply ‘automating tests.’” In this whitepaper, we’ll address the best practices you can adopt for implementing Continuous Quality and Testing on the AWS Cloud environment in the context of the DevOps model.
    [Show full text]
  • Chap 3. Test Models and Strategies 3.5 Test Integration and Automation 1
    Chap 3. Test Models and Strategies 3.5 Test Integration and Automation 1. Introduction 2. Integration Testing 3. System Testing 4. Test Automation Appendix: JUnit Overview 1 1. Introduction -A system is composed of components. System of software components can be defined at any physical scope. Component System Typical intercomponent interfaces (locus of (Focus of Integration) (Scope of integration faults) Integration) Method Class Instance variables Intraclass messages Class Cluster Intraclass messages Cluster Subsystem Interclass messages Interpackage messages Subsystem System Inteprocess communication Remote procedure call ORB services, OS services -Integration test design is concerned with several primary questions: 1. Which components and interfaces should be exercised? 2. In what sequence will component interfaces be exercised? 2 3. Which test design technique should be used to exercise each interface? -Integration testing is a search for component faults that cause intercomponent failures. -System scope testing is a search for faults that lead to a failure to meet a system scope responsibility. ÷System scope testing cannot be done unless components interoperate sufficiently well to exercise system scope responsibilities. -Effective testing at system scope requires a concrete and testable system-level specification. ÷System test cases must be derived from some kind of functional specification. Traditionally, user documentation, product literature, line-item narrative requirements, and system scope models have been used. 3 2. Integration Testing -Unit testing focuses on individual components. Once faults in each component have been removed and the test cases do not reveal any new fault, components are ready to be integrated into larger subsystems. -Integration testing detects faults that have not been detected during unit testing, by focusing on small groups of components.
    [Show full text]