Tricentis Continuous Testing Platform

Mike Schlabinger, VP Global Sales Enablement

1 What is Continuous Testing?

Continuous Testing is the process of executing automated tests as part of the software delivery pipeline in order to obtain feedback on the business risks associated with a software release as rapidly as possible.

• Right Feedback, Right Time, for the Right Stakeholder

• Can happen at any point in the SDLC

4 Tricentis Continuous Testing Platform

Agile Dev Testing Automated Distributed Continuous Testing Load Testing Scalable, in-sprint test Resilient regression testing Cloud-based performance labs management for open source across any architecture or at the fingertips of every test automation, exploratory application stack at the speed developer and tester for on- testing and BDD. of change. demand load testing.

Software Testing Reinvented for Agile and DevOps

5 Tricentis Continuous Testing Platform

Agile Dev Testing Automated Distributed Continuous Testing Load Testing Open source testing and agile Resilient regression testing at Cloud-based performance test management the speed of change testing labs on-demand

Software Testing Reinvented for Agile and DevOps

6 Tricentis Continuous Testing Platform

Tricentis Analytics Cross Project and Portfolio Visibility

7 Tricentis Continuous Testing Platform

Agile Dev Testing Automated Distributed Continuous Testing Load Testing

2x 2-3x 30-40% Risk Coverage Release Speed Cost Reduction

10x Performance Compared to other testing tools 8 Metrics Tricentis Analytics

Joint Analytics & Reporting Custom Dashboard

qTest Tosca Reporting Analytics Projects Projects Database Dashboard

Cross-Project Reporting

9 Integration Roadmap

1 Analytics 12.0, Oct 2018 Presented at Tricentis Accelerate Conference Unified Dashboard 1 2 in Vienna

Agile Dev Automated 12.1, Q1 2019 Public availability Testing Continuous Testing 3 Exploratory Testing 2 UI Automation Test Management 12.2, Q2 2019 Public availability Risk-based Testing and Manage Automate API Testing Manual Testing integrate Test Case Design shared qTest and Tosca Risk Based PrioritizationRisk Based Prioritization 3 Test Case Design Test Case Design Optimize Optimize 4

13.0, Q3/Q4 2019 Public availability Active Test Data ManagementActive Test Data Management 4 Active TDM shared qTest and Tosca Test Driven Service Virtualization Provide Provide

Internal, Subject to Change 10

Expansive Technology Support

OPEN OPEN SOURCE

RPG

SUPPORT

TECHNOLOGY

DEVOPS SUPPORT

11 Make open source tools more productive • Central,Software agile test management • OrchestrateTesting execution across tools • ConsolidatedReinvented analytics and reporting for DevOps

12 Language and legacy technology support Software RPG • SingleTesting interface for multiple languages • Model-Based Test Automation for rapid Reinventedcreation, management and for execution

• ConsolidatedDevOps analytics and reporting

13 Industry’s broadest support for PackagedSoftware Applications Testing • Business focused interface for ease of use Reinvented for • Model-Based Test Automation for rapid DevOpscreation, management and execution

• Consolidated analytics and reporting

14 Complete support for SAP Software SAP WinGUI SAP Fiori • Single,Testing business interface for all SAP UIs • Model-Based Test Automation for rapid Reinventedcreation, management and for execution

• ResilientDevOps end-to-end tests across technologies

15 Agile Dev Testing

• Agile Test Management by providing manual and automated test traceability to Jira issues,Software bugs, and releases in real-time • Shift QualityTesting Left with open source test automation management, test script scheduling,Reinvented and support for BDD for • PredictiveDevOps Test Analytics to gauge release readiness through executive dashboards & portfolio level reporting

17 Automated Continuous Testing

• Continuous Testing with model-based test automation and support for open source Softwaretools throughout the DevTest processTesting • Central Platform with seamless integrationReinvented into DevOps toolchains for for developers, testersDevOps and business users

• High Speed Releases by optimizing and orchestrating end-to-end tests across teams, projects and applications

18 Distributed Load Testing

• Easy Test Creation with ability to create tests inSoftware open source and proprietary tools for performanceTesting testers of all skill levels • Flexible, Massive Scale by leveraging the cloud Reinventedthrough shared or private AWSfor and Azure integrations that fits into your DevOpsDevOps pipeline

• Rich Bottleneck Analysis that highlights areas of risk for quick resolution of issues

19 PLAN DESIGN BUILD CONTINUOUS TESTING DEPLOY MONITOR

Plan Code & Test Design Version Build Provision Test in Sandbox Deploy to Monitor & Stage RPA

Integrate Integrate Integrate Integrate Report, Prioritize Explore Virtualize Distribute Automate & Integrate Design Manage Automate Execute Configure

23 A Sprint in a Tester’s Life

As a User I want to view customer data (UI) DEVs As UI-User I want to retrieve customer data via json service (Middleware) R Epic / User Stories created E P Load Test Exploratory Testing O PO User Stories (de-)refined R T Test Data Management Acceptance criteria (ATDD/BDD) aligned I N Model based UI Test, Web Desktop & Mobile Risk & TCD G

Model-based API Test GUI Load Test

Model based SV – Test QA

Refinement Sprint Provide Test Artifacts as “Self-Service“ in & across teams 24 About Tricentis

26 Top Analyst Recognitions

“Tricentis‘ Tosca Testsuite makes “It should be considered by enterprises that automation easy with model-based have struggled to make test automation automation. Via Tosca Testsuite, Tricentis work, and by those seeking to support agile provides top test automation and continuous automation practices.” optimization design capabilities, test asset reuse and combined automation.“

30 Dec 2018

Nov 2017 Dec 2015

Aug 2014

Jan 2011

31 Dec 2018

Nov 2017 Dec 2015

Aug 2014

“It should be considered by enterprises that have struggled to make test automation work, and by those seekingJan 2011 to support agile continuous automation practices.”

32 Awards & Accolades

#44

33 Agile testing for Continuous testing for Continuous Testing core insurance system, online banking system through a Digital TCoE, 4.5M lines of code all core systems

19K tests → 3.5K tests 2K tests → 0.6K tests 12K tests → 4.2K tests (-82%) (-70%) (-65%)

46% → 93% 30% → 85% 43% → 92% (2x) (2.8x) (2.2x)

4 weeks → 3 days 120 hours → 30 min 6 weeks → 5 days (9x) (240x) (8.4x)

34 Bringing standardization Providing cutting edge Moving from Legacy to 600+ agile teams with digital experiences for QC/ALM to modern agile 30+ automation tools 65 million active users practices

Clear visibility into manual Standardized all Best of breed approach for to automated progress for hardware and digital agile transform using Jira executives to see QA value app testing across qTest Software + qTest

Centralized reporting Launched PlayStation Store Migrated 25 large scale across 30+ automation by centralizing QA results projects including test cases, tools and frameworks with integrating to Jira plans, linked traceability

More than 400+ test Doubled QA licenses of Scalable users across automation agents qTest in 2nd year to multiple NetApp divisions connected for a single view support global teams in will millions of test results of application health Japan and California from home grown test 35 automation Transformed TCOE from Scaling extreme test Providing digital Legacy ALM to modern automation for API, SDK, customer experiences CI/CD model UI, & performance across web, and mobile

Reduce test maintenance Scaled monthly test Compliance through time through trend activity to 153 platforms centralizing QA results analysis tested, 422 builds & and integrating with Jira 300,000 test runs

Increased test automation Reduced a test run Decreased average to 83% reporting from hours to regression testing time minutes from 2 days to 2 hours

Substantially reduced Scaled number of test Scaled open source test critical production defects runs executed from 45 to automation from hundreds from 15 avg to 0 to thousands 7,000 per minute 36 Testing Present

Testers Coverage

100% Manual Testing 11 ? Manual Unknown There’s a way to do it

Test Cases Execution better. Find it! Project Charter

0% Automated 4755 10 Testing Manual Weeks

Customer Case Study Risk Coverage Optimization Test Data Management Automation (UI & API) & OSV Continuous Integration & Distributed Execution

Test Cases Testers Coverage Coverage Coverage Execution

Test Case Count 7%

92% 11 37? % 89% 53% 34 Automated Manual BusinessUnknown Risk Business Risk Business Risk Minutes

64% API Tests Smoke Testing Smoke Testing

Test Cases Test Cases Execution Execution Execution Execution

1193 4755 10 5 2.5 488 Manual Manual Weeks Weeks Weeks Hours

75% Redundancy 50% Effort Test Data MultipleSingle Agent Agents

Customer Case Study Risk Coverage Optimization Test Data Management Automation (UI & API) & OSV Continuous Integration & Distributed Execution

Test Cases Production Defects Coverage Coverage Execution

Test Case Count 7%

92% 89% 53% 34 Automated Business Risk Business Risk Minutes

64% API Tests Smoke Testing Smoke Testing 72% Test Cases Critical Defect Reduction Execution Execution Execution

1193 5 2.5 8 Manual Weeks Weeks Hours

Test Case Design & Exploratory Testing Multiple Agents

Customer Case Study

RMS Case Study

SAP Within five months a team of three implemented the core regression suite, including 333 business scenarios across various business functions such as Financial Processes (Accounts Payable, Accounts Receivable, General Ledger), Procurement, Controlling, HR, Project related Processes, and automatic Invoice Processing, covering 34,237 test actions covering 6,734 verification points. The total execution of the SAP automated regression suite takes less than 24 hours; this would take 518 hours to execute manually. By running the automation suite the total effort is reduced by 96%. Questions?