1 Analyzed Software Testing Techniques ~ Black-Box
Total Page:16
File Type:pdf, Size:1020Kb
1 ANALYZED SOFTWARE TESTING TECHNIQUES ~ BLACK-BOX TESTING, WHITE-BOX TESTING, GRAYBOX-TESTING COSC 5370 - ADVANCED SOFTWARE ENGINEERING TEXAS A&M UNIVERSITY – CORPUS CHRISTI SPRING 2014 BY MERT EVREN 2 Abstract ~ Testing is an important phase in software engineering Software testing methods are used for examination, verification and validation of the source code of a program. These methods are applied to software applications and their components to uncover hidden errors. Software testing includes experimentally and scientifically inspecting the correctness of the application. Software testing determines the applications quality by evaluating the capability of the application. The entire testing procedure should be well-structured. In this paper, three testing methodologies; black-box testing, white- box testing, and gray-box testing, are analyzed in detail. The remainder of this paper is structured as follows; section 2 describes black-box testing, white-box testing, and gray-box testing. Different testing techniques of each method has shown. These testing methods are analyzed by providing pros and cons of each. Section 3 presents possible improvement on testing methods. Lastly, section 4 concludes the paper. 3 1. INTRODUCTION Testing is an important phase in software engineering. “To develop high-quality software, it is essential to use software testing methods.” [1] Software testing is defined “as a process of accessing the functionality and correctness of a software through analysis.” [4] Software testing methods are used for examination, verification and validation of the source code of a program. These methods are applied to software applications and their components to uncover hidden errors. Software testing includes experimentally and scientifically inspecting the correctness of the application. Software testing determines the applications quality by evaluating the capability of the application. The entire testing procedure should be well-structured. “Testing process must balance the requirements, technical limitation and user expectation.” [7] Testing cost changes depends on the application and the type of test applied, however not testing the application can be even cost more. Testing is done in every stage of software life cycle. “Testing ensures that what you get in the end is what you wanted to build.” [7] In this paper, three testing methodologies; black-box testing, white-box testing, and gray-box testing, are analyzed in detail. The remainder of this paper is structured as follows; section 2 describes black-box testing, white-box testing, and gray-box testing. Different testing techniques of each method has shown. These testing methods are analyzed by providing pros and cons of each. Section 3 presents possible improvement on testing methods. Lastly, section 4 concludes the paper. 2. TESTING METHODS 2.1. BLACK-BOX TESTING Black box testing, sometimes named functional testing or behavioral testing, defined as “is a software testing techniques in which functionality of the software under test (SUT) is tested without looking at the internal code structure, implementation details and knowledge of internal paths of the software.” [2], mainly concentrates on testing whether or not the code does what it is 4 supposed to do developed from its functional requirements. Black box testing develops test cases by using the specifications of the program and it is done by using the output of the user interface as would an end-user. Black-box testing is not concerned with the process software takes to provide particular output, it is concerned with the output that results from the input. 2.1.1. BLACK-BOX TESTING TECHNIQUES Figure-1 - Black-Box Testing Techniques [5] There are many black-box testing techniques exists, as shown in Figure-1. Following are some black-box testing techniques. 2.1.1 EQUIVALENCE CLASS PARTITIONING This testing technique divides input values of a program unit into valid and invalid partitions where test cases derived from. 5 2.1.2 BOUNDARY VALUE ANALYSIS Boundary value analysis determines the boundaries of input values which includes error values and valid values. 2.1.3 FUZZY TESTING Fuzzy testing is used to discover implementation bugs by using “malformed / semi- malformed data injection in an automated or semi-automated fashion. Fuzzing is also used to test for security problems in software.” [2] 2.1.2. PROS Very efficient and suited for testing large code segments. Simply focuses on if valid and invalid inputs provide correct outputs. Black-box testing is quite easy, advantaging tester’s usage with no coding skills or the knowledge of software implementation. Once the specifications are done, test cases can be designed. Tester are only concerned with GUI, internal paths of the program are not required to be analyzed. “The test cases can show presence or absence of classes of errors.” [7] 2.1.3. CONS Since black-box testing is based on GUI, it makes it hard for maintaining script when the user interface constantly changes. Because black-box testing doesn’t look into the internal code, the program can never be fully tested. “Blind Coverage: cannot control targeting code segments or paths which may be more error prone than others.” [2] 6 “Only small numbers of possible input can actually be tested.”[7] The test may be already executed on the program by its programmer which causes redundant testing. 2.2. WHITE-BOX TESTING White-box testing digs deep into the internal code of software. [2] Describes why this testing method named white-box “This method is named so because the software program, in the eyes of the tester, is like a white/transparent box; inside which one clearly sees.” White box testing investigates the implementation of the components in the source code, in another word white-box testing allows testers to see what is happening inside of the software. “White box testing is considered as a security testing method that can be used to validate whether code implementation follows intended design, to validate implemented security functionality, and to uncover exploitable vulnerabilities.” [7] White box testing provides full understanding of the internal mechanism of the application. “In white box testing it is necessary for a tester to have full knowledge of source code” [4] White-box testing is very efficient on revealing errors. “Exhaustive correctness checking is performed, all code paths are executed to ensure quality software outcome with reduced defect rate.” [5] 2.2.1. WHITE-BOX TESTING TECHNIQUES There are many existing test techniques for white-box testing as show in Figure-2. Here are the some of the white-box testing techniques analyzed. 2.2.1.1. CONTROL FLOW TESTING Control-flow testing is an efficient techniques for most of the software. “It is a structural testing strategy that uses the program control flow as a model control flow and favors more but simpler paths over fewer but complicated path.”[4] 7 2.2.1.2. BASIC PATH TESTING Basic path testing confirms that each independent path in the application source code is taken in a predetermined order. “It allows the test case designer to produce a logical complexity measure of procedural design and use this measure as an approach for outlining a basic set of execution path” [2] 2.2.1.3. DATA FLOW TESTING Data flow testing focuses on how data communicates within the system. It is used on modules with nested if and loop statements to detect variable that are unused or not initialized. 2.2.1.4. CONDITION TESTING Condition testing concentrates on the logical conditions of the program methods. Figure-2 – White-Box Testing Techniques 8 2.2.2. PROS Allows internal code to be fully tested where objects are able to be identified programmatically. “All independent paths in a module will be exercised at least once.” [7] Looking deep into implementation in detail allows programming errors to be easily detected. Behavior of software modules can be uncovered since testing involves accessing to source code. 2.2.3. CONS “It is nearly impossible to look into every bit of code to find out hidden errors, which may create problems, resulting in failure of the application.”[7] Testers have to be familiar with testing tools. Testers needs to be skilled in coding languages. Because of these requirements, it becomes difficult to scale testers. White-box testing uses test scripts. Implementation of these test scripts are tied to the source code of the application. Maintenance becomes very hard when any change made to the code. 2.3. GRAY-BOX TESTING Gray-box testing, also called as translucent testing, is combination of white-box testing and black-box testing. Gray-box testing is defined as, “a technique of testing the application with limited knowledge of the internal working of an application and also has the knowledge of fundamental aspects of the system” [4] Documentation of the program needs to be provided to testers in order to execute the tests. “The aim of this testing is to search for the defects if any due to improper structure or improper usage of applications.” [2] “Grey box testing technique will 9 increase the testing coverage by allowing us to focus on all the layers of any complex system through the combination of all existing white box and black box testing.” [4] Figure 3 – Representation of Gray-box Testing [4] 2.3.1. GRAY-BOX TESTING TECHNIQUES 2.3.1.1. MATRIX TESTING Status report of the application is generated which includes the relationship between the test requirements and the test cases. This report allows testers to analyze which requirements are included in which test cases. 2.3.1.2. REGRESSION TESTING This test is applied after new changes are made, such as functional improvement, to the source code of the application “Its purpose is to determine if the change has regressed other aspects of the program.” [2] Regression testing is a regular step of the software implementation procedure.