Automated Testing Using Scripting CEN 4072

Automated Testing Using Scripting

Team Essay: Fall 2020

Team Tyler

ABSTRACT

In the process of developing a software, testing is one of the most important aspects in order for a project to be successfully deployed without major bugs. However, testing becomes extremely hard when working with time constraints. The process of debugging manually could take as much time as the process to develop a functionality of the software. Therefore, implementing automated testing through scripting improves the time lapse for debugging and increases the accuracy in finding more errors due to its fastest capabilities to test the code. In addition, automated testing provides the programmer with the ability to create specific test cases that better suits the software being developed. Using techniques such as breaking the functionality of the project into many modules the developer can locate issues in a more effective way. Therefore, automated testing through scripts can be considered a tool that developers should implement into their testing process in order to facilitate the task and increase the chances of finding bugs.

KEYWORDS

Scripting, Testing, Automatization, Framework, Software.

INTRODUCTION

Test scripts are the basic building blocks of automation. A test script is written in a scripting language like Perl, Python, Ruby, etc. and contains a series of instructions that execute a test case and report the results. These instructions may behave reactively to the data output from the program under test or the script can simply utilize a loop to repeat the same instructions many times with different sets of data. How complex the script is depends entirely on the project at hand and what the developer wants to accomplish with each script. To help a developer make scripts for use in automated software testing, there are several techniques, called frameworks, that have been widely adopted. These include linear scripting, structured and shared, data driven, keyword driven, and several more. 2

BENEFITS OF USING SCRIPTS IN DEBUGGING

In order to achieve a software that runs smoothly and to reduce the chances of crashes and bugs, the implementation of scripts for debugging is essential. In comparison with manual debugging, scripts allow programmers to use automated testing in order to reduce the time ths tedious task usually takes. Debugging manually is a very slow and stressful process. Sometimes, this can result in a series of tests that get skipped or ignored. Scripts help to make the testing process less painful. They allow programs to create the cases in which each function of the program should pass and return a successful result. When using test cases the software is broken down into modules in order to create a flag where the errors are occurring. Programmers can directly reach that functionality and identify the bug faster. A good example of this strategy is already applied by many programers when testing a software and getting wrong or unexpected answers. The strategy consists of commenting out lines of the code and modifying it in order to identify the issue or what is breaking the execution of the program. In the case of automated testing, it will check for which step the execution results are not the expected for that particular module of the software. Automated testing and debugging go hand in hand in that automated testing is done to find bugs and errors and debugging is performed to fix the bugs. While they both focus on bugs, testing ends with bugs and debugging begins with them. The automated part of testing is optional, but it is very helpful when a test needs to be performed many times. Unfortunately, bugs are more elusive and while there may be situations in which we encounter the same bug over and over, it is still not yet practical to automate debugging. Additionally, there are times when automated testing fails to find a bug, despite this being its main purpose. If an error or a bug is outside of the framework under which the automated test checks things, then it can go unnoticed and later cause a problem. Manual testing however can be much more exploratory and uncover bugs that automated testing would miss. For this reason, manual testing is unlikely to go away anytime soon and, for many projects, is considered essential.

AUTOMATED TESTING VS MANUAL TESTING

There are two types of testing used when testing a system, automated and manual. They both serve different purposes but automated testing is the more convenient and powerful one for many situations. Automated testing is performed by a software tool that is prewritten and will perform certain actions to make sure the software runs properly, whereas manual testing is performed by a human from scratch, nothing is prewritten and every testing tool is to be implemented right then and there. By seeing their differences, we could see why one might opt for automated; as it is more convenient, faster, and more accurate.

Manual testing could not only take much more time, but it is very prone to human errors along the way. Also, as automated testing avoids things like human errors, it can save money for the developer because it avoids the need to repeat any tests for the code. The more you are obliged to repeat a certain test - which manual testers often do - the more time and money it 3

would cost, which is something we try to stay away from. Manual testing cannot test thousands of codes for the system as it is physically near impossible, and even if it weren't, it would take time that the company or developer cannot afford, whereas automated testing could easily do this as well as share those automated test codes with other developers.

However, there are some limitations to having automated testing, those of which where the tester has to be very proficient in using such testing tools because any complications or errors may lead to redoing the whole test script, which in result is very costly and defeats the purpose of automated testing. Another disadvantage would be if there are too many screens to be tested, major maintenance issues would prevail as it is difficult to keep track of bigger screens.

All in all, the pros to automated testing are shown to be superior to manual testing, and the cons are very limited compared to manual testing. This comes to show us how important and advantageous automated testing is in the world of testing scripts and to developers everywhere.

FEATURES OF THE SCRIPT LANGUAGE

Pytest is a widely used script language that can be used to create automated tests. To use it, one should start by creating a .py file and importing pytest and the files which we want to test. Additionally, if the test file contains some value fetched from a database, you also need to import SQlite3. The next step is to create a test class and define some methods in the class. In these methods, we can get a value by calling a function from the file imported and using an assert method to verify the output value. There are several assert methods available to use such as assertEqual(a,b), assertTrue(x), assertIs(a,b), etc. After these steps, we can run our testing code.

The above is an example for testing output. If the return value is equal to the assert value, the output will show PASSED. The percentage shows the process of testing. 4

If the return value is not equal to the assert value, the output shows Assertion Failed.

If all the tests passed, the output shows how many tests passed and the processing duration time.

After testing, we have to maintain the automation test. If there are new functionalities added to the files or some functionalities have changed, the automated test will need to be updated to adequately test the new changes. Further, maintenance is executed when new automation scripts are added and needs to be reviewed in order to improve the effectiveness of automated tests.

ESSENTIAL TOOLS AND FUNCTIONS

Automated testing has what is known as frameworks. These frameworks are like scaffolding to build your automated tests on. They provide a set of rules, standards, protocols, and guidelines that can be leveraged to more efficiently make automated tests for whatever software project a developer is working on. Test automation frameworks help significantly when working in a team, as the team can decide on a framework that best suits the project needs. This allows for better understandability, teamwork, maintenance, and easy reporting. Having a framework also helps if someone wants to come back to the code and reuse it. By knowing which framework was used, they can more easily understand how it works and how they can reuse it.

An example of one of these automated testing frameworks is the data driven testing framework. It is oriented towards the situation where a tester needs to test the same functionality 5

many times with different sets of data as it helps the user to separate the test data and the test script logic. The framework looks like this:

Software Testing Fundamentals

DETAILED APPLICATION EXAMPLE

Scripting plays a tremendous role in software testing; using scripts can be very beneficial compared to other ways. To learn more about the benefits of how scripting can improve software testing and debugging, let’s take a look at an example using Python. Below is a simple program in Python named Temperature.py. 6

The program Temperature.py asks the user to enter a temperature. Then, the user is asked what they want to convert the temperature to (Celsius or Fahrenheit). Finally, the program prints out what the temperature was converted to, as shown in the two examples below.

Celsius to Fahrenheit

Fahrenheit to Celsius

In order to test this program, PyUnit will be used. This module has a built-in testing framework and a test runner. PyUnit is the standard unit testing framework module for Python, it is described as the Python version of JUnit. Below is a script in order to test Temperature.py. 7

The script above tests the functions that convert a temperature in Celsius to Fahrenheit, or vice versa. The self.assertEqual function has a first parameter in the function one of the two temperature conversions with the temperature. The second parameter has a float number that should be equal to the return of the function in the first parameter. Below is what happens when the script is run.

Once run, if all tests are correct, then the tests are correct and will print “OK”. However, if we change the conversion functions in Temperature.py, then the tests will fail. Below is the changed Temperature.py file (that will calculate the wrong conversions for both functions) and the results of the test script (same test script as below). 8

Below is the test results for the test script

When the test script is run, there are errors on both tests. The error for each test function shows the one that failed first in the function. For example, the results of the test that was ran above shows the error for “self.assertEqual(CelsiusToFahrenheit(0), 32)” it expects the function CelsiusToFahrenheit(0) to be equal to 32, but the function returns 2.0, which is why we have on the next line “AssertionError: 2.0 !=32.” This means the function is wrong and needs to be debugged.

The test script is a small example, but it shows just how powerful scripting can be for software testing. Consider a larger program with many source files; it would be a nightmare to test all of the files. With test scripting, however, it can be easier to identify what needs to be tested in a program.

There is a lot of preparation work that happens before test scripting, and there needs to be documentation in order to keep track of the tests. Scripting tests is not an easy task, and it will take a long time, but it can be very beneficial. The most beneficial part about script tests is that the software is compared with the standards and specifications. Unscripted tests have software compared with the tester’s expectations. This means that script tests can better identify an issue with the program, because the script tests rely more on the standards and specifications required for the program. The issue with unscripted tests is that they rely on the tester’s expectations, which may not handle all of the standards and specifications needed for the software.

Overall, scripting can greatly improve software testing and the debugging process. One benefit is that there is a lot of preparation needed, meaning the tests are preplanned and can better find problems in the program, if there is one. Another benefit to scripting is that script tests has the software compared with the standards and specifications that are required for the software. This means that the test script tests all of the requirements for that software to work the way it is meant to work. Scripting is a very powerful tool when it comes to testing software, which is why scripting can significantly improve software testing.

Works Cited

pp_pankaj Check out this Author's contributed articles., et al. “Difference between Scripted and Unscripted Testing.” GeeksforGeeks, 6 Aug. 2019, www.geeksforgeeks.org/difference-between-scripted-and-unscripted-testing/.

“25.3. Unittest - Unit Testing Framework.” 25.3. Unittest - Unit Testing Framework - Python 2.7.18 Documentation, docs.python.org/2/library/unittest.html.

“Automation Testing Tutorial: What Is Automated Testing?” Guru99, www.guru99.com/automation-testing.html.

“What Is The Benefit of Test Automation and Why Should We Do It?” SmartBear, smartbear.com/solutions/automated-testing/.

Software Testing Genius. “Pros and Cons of Automated Testing.” Software Testing Genius, 8 Sept. 2018, www.softwaretestinggenius.com/pros-and-cons-of-automated- testing/.

Sanjeev KulkarniFollowTest Manager | QA/Testing Services | Banking and Finance Applications | Retail | CRM | SAP | Data Science EnthusiastLike31Comment5ShareLinkedInFacebookTwitter1, et al. “A Review of Scripting Techniques Used in Automated Software Testing.” LinkedIn, www.linkedin.com/pulse/review-scripting-techniques-used-automated-software-testing- kulkarni/.

“Software Testing Fundamentals (STF) !” SOFTWARE TESTING Fundamentals, 18 Sept. 2020, softwaretestingfundamentals.com/.