sign in sign up
<<
Home , Test automation, Test script

1

CEN 4072 Testing Team Essay: Fall 2020

Active learning project: automated testing using scripting

ABSTRACT

Debugging can be an extremely labor-intensive activity involving many repetitive and common operations. The ability to step through these recurring, often mundane, tasks is perfect for scripting programs. In order to fully leverage the capability of scripts, computer science professionals should first understand the benefits of using scripts for that purpose. Also, they need to appreciate the features of the different scripting languages and be familiar with different tools to aid in debugging activities using scripts.

This paper presents a basic understanding of these benefits and examines a few key features and tools. It also highlights a simple example of how a script might be used to debug a program. Specifically, this paper’s example looks at as a front-end web development testing framework.

KEYWORDS

Automation, Debugging, Scripts, Testing, Selenium

INTRODUCTION

Scripts play a key role in automation. Essentially, scripts are simply programming languages that execute tasks one after another. As a result, the nature of scripts has led it to become the most important tool for a programmer that seeks automating parts of their workflow. Similarly, a programmer’s job involves testing their creations extensively, so programmers will commonly use scripts to automate the debugging process. By using a testing suite, a programmer can execute hundreds of tests with one command, allowing for a much more efficient workflow.

Benefits of Using Scripts in Debugging

Fundamentally, script-based testing and debugging tackles the general resource intensiveness of . This aim should not be undervalued, as software testing is

CEN 4072 – Software Testing: Essay 2 perhaps the single most important sub-activity of the lifecycle. The advent of automated testing was a significant breakthrough in the field. It greatly reduced the effort involved by streamlining the test creation process (via frameworks like JUnit) and making large scale viable (Sommerville). The introduction of scripts has continued that evolution. The added benefits of scripts vary depending on the technique, language, and application used; these separate topics will be discussed later. For this section, we will outline the benefits of using scripts in the broadest sense of debugging and software testing. What follows is a non-exhaustive list of those benefits:

Presentation Layer Scripting: Scripting languages can simulate human behavior at both low and high levels of abstraction (mouse button clicks and GUI component events, respectively). As a result, it is always feasible to test a program at the presentation layer (Zeller). This is a great advantage under certain circumstances. For example, for software that is designed such that there is no separation between the presentation and functionality, there may be no other option for automating testing.

Flexibility in Test Design: The ability to simulate behavior at different abstraction levels already offers flexibility in test design at the presentation layer, but scripts can also automate the testing process at the functionality and unit program layers. This is accomplished at the functionality layer via interfaces (e.g., SOAP for web services) that enable communication between programs and scripting languages. And there is good reason to opt for this form of —it is more resilient to change. That is, changes to the user interface will not affect the script (Zeller). As for the unit layer, scripts typically cannot control unit tests directly. However, as scripts are tools of automation, they can execute the test suite and process its output. All these options allow the programmer to create test scripts that both suit their needs and the needs of the program.

Functional and Non-: Scripting languages are uniquely suited for both functional and non-functional testing. The explanation for this is twofold. For one, as previously stated, scripts can interact with functionality layer of a software—invoking actions and observing results—to determine if exhibited behavior fulfills the software requirements. This is functional testing, and scripts can perform this process as well as any other method. The second part of the explanation, however, is where testing can be taken a step further. Scripts can also interact with software at the system-level. Using a virtual machine, they can simulate hardware, send inputs, and inject faults (Zeller). This virtual test environment is ideal for testing a multitude of non-functional requirements (e.g., performance, compatibility, reliability).

Quick Test Creation (Record/Playback): Many scripting languages are compatible with tools that “record” interaction with software—that is, these tools automatically generate scripts that correspond to the user’s actions. These scripts can then be “played back” in the same way any other script is run. This is helpful tool for testing as it “enables users to start automating quickly as no planning is required, tester can just simply record any manual ” (Hanna et al.). For example, imagine a programmer has identified an action sequence

CEN 4072 – Software Testing: Essay 3 that results in some error. They can quickly generate a test script by recording the sequence. While debugging, this script will quickly verify whether the problem remains.

Cumulative Effects: This section began by stating that the use of scripts in software testing targets the activity’s high resource requirements. Ultimately, that is what these benefits add up to. They limit monetary costs, reduce time spent on testing, help increase test coverage, and more.

Features of the Script Language

With scripting established as an effective way of testing software, there are many practices that must be followed to maximize the effectiveness of automatic testing. Having failsafe protocols when testing software can save time and false failed tests. An easy failsafe protocol to follow would be to sync the software and the testing tool (Khalid). This ensures that previous tests do not have an impact on the newer tests. Another recommended practice that should be followed is into implement event handlers. Event handlers can reduce the number of troubleshooting problems, sync the software and testing tool, act as a failsafe and control the execution. An example of an event handler would be a try and catch statement. This allows a user to execute a block of code and if an error occurs in the try block, catch will return any errors that happen in the try block. Sometimes, software works perfectly in one environment and not others, so an easy way to combat this is to test in multiple environments as well as write down variables so that patterns and permutations can be easily identified. A checklist to follow every testing session is to specify the purpose of the software, test early, create test cases before implementing code, test often, test enough so that everything is covered and have other people test (Zeller).

Developing scripts which are efficient and effective at testing all aspects of the software can be a difficult task. In order to develop test scripts, having clear and simple to read scripts are essential. Testers must be able to easily understand the scripts. Documentation of the script and having clear code helps tremendously in optimizing the

CEN 4072 – Software Testing: Essay 4 testing process. To simplify the scripts, it is recommended to have smaller scripts, each with one specific purpose. All scripts should be maintainable, meaning they should be able to be updated even if there are changes in the software. The most important part of writing test scripts is to make sure they are well thought out. The script must cover every single potential path the user may take. This ensures that there are absolutely no errors within the software and checks even the most remote options. An easy way to test all paths is to draw out every single path in a diagram and check off each path after a test script is written (StarDust).

Essential Tools and Functions

Writing test scripts is very similar to writing programs themselves. They need to be debugged and tested themselves before they are put into practice. Due to the similarities of processes of writing in programs and scripts, a lot of the same principles are followed. When we write any code, it needs to have a specific purpose, and that purpose should be clear and documented. It’s often better to have a few specific scripts then to have only one script that potentially covers all the matters of an entire program. This is like writing functions. The more focused a function, the more flexible and reusable it becomes (Zeller). Scripts should also be tested early and often. It’s important to start running test cases as soon as one unit is implemented. The system should be assembled from those tested units. Also, tests should be run after any change has taken place on the system and certainly with each new system release. Test cases can even be written prior to a unit being implemented. That can help keep the unit focused and act as the unit’s specifications. There is no such thing as testing too much, but there is such a thing as not testing enough. Test cases should sufficiently cover all the code written with a multitude of a variety of inputs. And when you feel that the script (or scripts) you have written are complete, invite other testers to try and uncover any defects or weaknesses that you may have missed (Zeller). Scripting itself doesn’t come with just one way to accomplish the task. There are several different techniques that have different properties. Scripting can be linear, structured, shared, data-driven and keyword-driven. The linear technique records a series of testing instructions with manual user interaction and then saving the actions as a test script. This method gets automating started quickly, however is difficult to be maintained. There’s also structured scripting which divides the larger test scripts into smaller, more focused scripts. This allows for scripts to call each other. Data-driven scripting is when the test data is stored outside of test script and in a different location all together. This scripting technique is advantageous because tests that are similar, but with different input data can be added easily to the testing process. Lastly, keyword-driven scripting separates test data from scripts, but also separates predefined keywords. Using this type of scripting means that the tester needs only know what keywords are needed for a test and what input goes with those keywords. The cost of this type of implementation is very expensive regarding time and resources. Easily, the data-driven scripting technique is the most efficient and affordable of all the techniques (Hanna).

CEN 4072 – Software Testing: Essay 5

Figure 1. Illustrates the organization of data-driven scripting (Hanna).

CEN 4072 – Software Testing: Essay 6

Example

One could make their own automated tests with something as simple as Bash scripts, since all automated testing simply checks the output of what you have versus what you want. It is extremely popular to use testing frameworks since they come with many bells and whistles, including multiple language support and more in-depth testing procedures. Selenium is one such testing framework. Here is a simple example of Selenium utilizing Python’s unittest script to test a web driver:

Selenium serves as the most comprehensive front-end web development testing framework. The key feature is the ability to write tests in many languages (PHP, C#, Ruby, Python, , Java, etc.). The framework has many components, but the most useful component is the Selenium RC/WebDriver. These components test your browser running the code directly, which in the past, would have to be done by QA testers manually. In addition, Selenium has another feature called Selenium IDE which is a point and click style of that requires little programming knowledge, and more intuitive testing tools for beginners. Overall, automation testing frameworks have come a long way, and work as convenient time savers which allow the programmer to focus on development rather than .

CEN 4072 – Software Testing: Essay 7

REFERENCES:

Hanna, Milad, N. El-Haggar, M. Mostafa. 2014. A Review of Scripting Techniques Used in Automated Software Testing. International Journal of Advanced Computer Science and Applications. http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.428.7456&rep=rep1&ty pe=pdf

Khalid, Ali. 2015 – 2020. 7 lessons from debugging a test automation framework. TechBeacon. https://techbeacon.com/app-dev-testing/7-lessons-debugging-test- automation-framework

Python Package Index. 2018. Python bindings for Selenium. https://pypi.org/project/selenium/

Sommerville, Ian. 2020. Engineering Software Products: An Introduction to Modern . Pearson.

StarDust. 2020. How to Write a Test Script. https://www2.stardust-testing.com/en/how-to- write-a-test-script

Zeller, Andreas. 2009. Why Programs Fail: A Guide to Systematic Debugging. Morgan Kaufmann Publishers. https://atakua.org/old-wp/wp- content/uploads/2016/05/Andreas_Zeller_Why_Programs_Fail_Second_Edition_A_ Guide_to_Systematic_Debugging__2009.pdf

CEN 4072 – Software Testing: Essay