Continuous Testing Maturity Model

With 81% of organizations adopting DevOps and 91% Level 1: Initial adopting Agile, Continuous Testing is no longer optional. To At this initial level, the key metric is the number of test cases. meet business expectations, organizations must adopt an All test cases are designed based on tester intuition. Testing optimized Continuous Testing practice that provides real- is performed manually or partially automated with a script- time insight into the business risk of your release candidates. based approach (which results in a high rate of false positives This is not simply a matter of adding more automated tests. that require constant maintenance). Testers must manually It requires a deeper transformation that impacts people, ensure test data suitability (e.g., by localizing and refining test process, and technologies. data) and wait for dependencies to be provisioned in test Based on our experience guiding enterprise testing teams environments. Any API testing is the domain of developers. to optimized Continuous Testing, Tricentis has developed a Efficiency gain: 1.3X Continuous Testing Maturity Model. This model is designed to help you assess where your team stands today and understand what’s needed to progress from one level to the next. FACTSHEET CONTINUOUS TESTING MATURITY MODEL

Level 2: Aligned Level 5: Optimized

A risk assessment has been completed and Risk Coverage Comprehensive test automation has been established and is now the key metric of test case definition and execution. is supported by sophisticated, stateful service virtualiza- Test automation still focuses on the UI, but now uses Model- tion and test data generation/provisioning. Metrics are in Based Test Automation (MBTA), which significantly reduces place to monitor and continuously improve the effective- false positive rates and maintenance efforts. Since there is ness of the software testing process. Continuous Testing still no comprehensive test data management in place, auto- is fully integrated into Continuous Integration and the mation primarily focuses on new data object creation rather Continuous Delivery pipeline. The transformation into than complex administrative use cases. Efficiency gain: 3X “DevTestOps” via Process, People and Product is achieved. Efficiency gain: >20X Level 3: Managed

Session-based Exploratory Testing is introduced to expose Your Path to Continuous Testing risks that specification-based testing cannot find (e.g., in For a detailed assessment of where your testing process functionality implemented beyond the boundaries of the stands today and a map of your recommended path to Opti- specification). Additional test cases are defined via combina- mized Continuous Testing, contact your Tricentis representa- torial Test Case Design methodologies such as linear expan- tive or email [email protected]. sion. If functionality is exposed via APIs, API Testing is intro- The Tricentis Customer Success team will meet with you to duced at the tester level. MBTA-driven UI testing is extended review your existing practices, processes, technologies, test in areas where API testing is not applicable or effective. artifacts, and configurations. Based on those findings, they Test automation is introduced into Continuous Integration will help you understand your current maturity level, what you through initial integrations with build and deployment tools. can gain by advancing, and exactly what steps are involved in Efficiency gain: 6X attaining your goals.

Level 4: Mature Once you’ve committed to Continuous Testing, change is

Test Data Management now provides the test data needed to inevitable (as with any transformational initiative). To help you enable continuous, consistent test automation. Service Virtu- understand exactly what to change—and when—Tricentis alization ensures that testing can proceed even if dependent offers an automated Continuous Testing Assessment tool. components are unstable or unavailable. The introduction Through self-assessments as well as Tricentis-driven eval- of both TDM and service virtualization enables more sophis- uations, Tricentis provides continuous feedback on your ticated API testing, end-to-end testing, and continuous test Continuous Testing progress and outlines clear recommen- execution. Tests can now be executed continuously as part dations for your next steps. Leveraging KPIs derived from of the software delivery pipeline—providing instant feedback leading companies and industry best practices, we guide on the business risk associated with the software release you to achieve critical Continuous Testing milestones as- effi candidate. Efficiency gain: >10X ciently as possible.

* Efficiency gain is measured versus a “level 0” manual testing process in terms of risk’ coverage, cost, and time.

About Tricentis

Tricentis provides a Continuous Testing platform that accelerates testing to keep pace with Agile and DevOps. With the industry’s most innovative functional testing technologies, Tricentis breaks through the barriers experienced with conventional software testing tools - achieving test automation rates of over 90%.

Our 400+ customers include global names from the Top 500 brands such as A&E, Allianz, Deutsche Bank, HBO, JetBlue, Orange, Swiss Re, Telstra, Toyota, UBS, Vantiv, Virgin Airlines, and Zurich Insurance. Tricentis has offices in Austria, United States, Germany, Switzerland, UK, Netherlands, Poland, and Australia.

www.tricentis.com