AI-powered test automation for the next generation of applications Table of contents 1. Executive summary 3 2. Why AI-powered test automation? 4 3. AI and machine learning in context: 5 what are neural networks? 4. Three generations of test automation: 9 script-based, model-based and AI-powered 5. Integrating Vision AI with model-based test automation 19 6. Tricentis platform: Vision AI, risk AI and self-healing AI 21 7. Glossary 22 AI-powered test automation for the next generation of applications EXECUTIVE SUMMARY Tricentis Tosca, powered by Vision AI is a new, next-generation technology using advanced machine learning and neural networks that elevates test automation to a new business discipline. Unlike first generation script-based technologies which produced fragile and costly maintenance, Tosca’s Vision AI can be combined with model-based test automation to deliver faster, more resilient test automation across new tiers of technology. Core business benefits of Vision AI together with model- based test automation are: • Lower cost of maintenance • Speed to market • Reduced risk • Release predictability This whitepaper describes how Vision AI sees and steers elements on virtually any technology – from cloud-native, enterprise apps, to simple designs and mockups. This allows tests to be maintained with zero code in a human-readable model that anyone can understand and adopt. As such, Tosca eliminates the maintenance issues inherent in traditional testing methods, allowing organizations to deliver blazing-fast software innovation across their entire digital landscape. WHY AI-POWERED TEST AUTOMATION? In the year 2018, Tricentis developed the hypothesis that if we are to truly shift testing to the left – that is, build automation before the UI is created, as well as create resilient automation for modern applications that get updated daily, it will require a dynamic approach that entirely separates an application’s technical layer from automating it. In other words, only if we completely remove the need for technical identifiers (such as Class, ID, XPath etc.) and replace it with visual identifiers driven by AI-powered automation, will it truly deliver resilient test automation that keeps up with perpetual technological change. Why did Tricentis take a new approach by adding AI-powered test automation and build it on top of model-based test automation approach? AI-powered test automation is our response to the rapid rise of modern applications that are built on new tiers of technology from cloud native applications, microservices to containers. These new applications get updated daily, and script-based testing approaches have been unable to keep up. At the same time, what we witness in our customer’s portfolios is a mixture of modern and legacy systems that are often integrated in complex ways. A small change in one system can have rippling effects in other integrated technologies. According to MuleSoft (2020), the average organization uses more than 900 applications today. And a single transaction can touch more than 82 types of technologies. The terrible truth is that very few companies have real end-to-end tests, and those that do – the tests are not running when they are most needed. UI automation also continues to be a bottleneck in quicker testing, and agile delivery. The vision of Behaviour Driven Development (BDD) was to enable automation to be written by the business prior to the product existing. This was a struggle due to the fact that the automation is only ever written after the User Interface (UI) of an application is built. As a result, UI automation continues to be slow, and being slow is costly. The sooner you find a bug, the cheaper it is to fix it. This is why it is essential to test as early as possible in the development lifecycle – before the UI is created. To keep up with an ever-changing application landscape, and enable organizations to cope with their modernization initiatives, at Tricentis, we present Tricentis Tosca powered by Vision AI, the next evolution in test automation. Vision AI is a patented deep learning technology that delivers stable, self-healing, platform agnostic UI automation. Together with Tosca’s model-based test automation and Vision AI, organizations can reduce modernization costs and efforts, as well as build resilient end-to-end testing that extends across platforms and technologies. tricentis.com © 2021 Tricentis USA Corp. All rights reserved | 4 AI AND MACHINE LEARNING IN CONTEXT: WHAT ARE NEURAL NETWORKS? Artificial intelligence (AI) as a concept is ubiquitous in technology, business, science, medicine, economics, and many more other fields. The word “AI” is so commonplace nowadays that readers are usually expected to understand from context what is meant. However, there are common misconceptions on AI and machine learning (ML) including algorithms, as well as ways in which ML technologies learn over time. These concepts have implications on what the AI technology can claim to do and which future challenges it can solve. Let’s clear some air and define some of these concepts below. Artificial Intelligence vs. Machine Learning Think of artificial intelligence (AI) as the umbrella term and machine learning (ML) as a sub-component of AI. Broadly speaking, artificial intelligence is anything that can perform intelligent tasks that would normally require a human to do. This loose definition makes AI applicable to all sorts of areas like big data analysis, predictive analysis, SQL queries analysis and more. Machine learning, on the other hand, is a system that “continuously learns” and “improves” over time given more “experience”. This is quite an important differentiator as over time, machine learning systems will become more future proof. Consider the following example: your task is to find the fastest route from A to B. tricentis.com © 2021 Tricentis USA Corp. All rights reserved | 5 A machine learning system would know the problem (A) and the result (B), but not the part in the middle, that is, how to get there. The part in the middle is what you are expecting the ML system to “learn”. What is the difference between ML and an algorithm? An “algorithm” works differently to ML as it involves expertise and knowledge being added into a system in order to solve a particular problem. In the example above, an expert would use an algorithm to instruct the system how to get from A to B. The system is therefore only as smart as the knowledge or expertise that we build into it. What are neural networks? Figuratively speaking, a neural network works like a randomly configured “brain”. It is composed of an interconnected web of neurons which talk to each other. Similar to the brain – as you grow, learn and expose yourself to more experience, the chemistry of the brain changes. Certain pathways reinforce and certain pathways die off. The goal of neural networks, therefore, is to mimic the brain – or as IBM coined it in 2001, model the behavior of a computing system against the human autonomic nervous system (ANS).2 How does a neural network learn? Consider the task of catching a ball. We get an input from our eyes and the motor neural response to our hand. With time, you learn to catch the ball more effectively because the pathways in between your neurons are either reinforced or weakened based on the feedback of the result. Artificial neural networks work in the same way. We train the system by feedbacking it with a set of problems and solutions, and punishing or rewarding the system if the response is correct or wrong. 2 In 2001, IBM coined the term „autonomic computing” and modelled their approach against the human autonomic nervous system (ANS). The ANS regulates critical bodily functions such as homeostasis without conscious involvement. Applying this idea to computing, systems should adapt to their environment through self-configuration, self-optimization, self-protection and self-healing. References from: https:// www.aitesting.org/ tricentis.com © 2021 Tricentis USA Corp. All rights reserved | 6 To train a machine learning system to correctly deal with problems or tasks, we need to feed it with lots of training data. Training data is usually a large set of examples, and these should include: • Examples of good results • Examples of bad results • The ability to update our system Have a look at these images: It’s not rocket science to figure out that these are images of airplanes, despite seeing some of them for the first time. Simiarly, we train the system to figure out that this is an airplane because we feed it with large amounts of human labelled data and punish or reward the system based on the output given. Labelling data is the process of naming each individual image with a label such as “text_field”, “search_ button”, “scroll_down_menu”. Finally, all of this labelled training data is passed onto a machine learning system and we say to that system to ‘train’. The ML system starts randomly guessing the correct label for each object. If it is correct, it remembers how the pathways were modelled. When it is wrong, it remodels those internal pathways and tries again. This process happens thousands of times until it can correctly identify most labels for objects. What are the different ways in which a machine learning system learns? ML can learn through various techniques, like supervised, unsupervised or reinforced learning. The process we explained above is referred to as supervised learning. This involves feeding the system with many examples, rewarding and punishing the system based on the output given until it eventually reproduces the correct output. The downside of this approach is that you need accurately labeled data, which is a labor-intensive process. In unsupervised learning, you do not require labelled data.