ICSEA 2016 : The Eleventh International Conference on Software Engineering Advances Toward Automatic Performance Testing for REST-based Web Applications Chia Hung Kao Chun Cheng Lin Hsin Tse Lu Department of Applied Mathematics CloudCube Co., Ltd DATA, Institute for Information Industry National Taitung University Taipei, Taiwan Taipei, Taiwan Taitung, Taiwan Email: [email protected] Email: [email protected] Email: [email protected] Abstract—Nowadays, more and more web applications are devel- test execution. Two major software artifacts, including APIs oped to provide their services over the Internet. In addition to document and test cases, generated from the software de- functionalities, performance characteristics, including response velopment process are used in the proposed approach. APIs time, throughput, latency, stability etc., are key factors when document describes information about functionalities provided selecting appropriate web applications. However, complex archi- by specific web application. On the other hand, test cases tectures, fast changing requirements, strict quality criteria and depict the test scenarios designed by software testers. Through time to market pressure all impose difficulties and complexities on testing activities. Therefore, for software testers, how to the composition of necessary APIs, the test scenario can be evaluate and ensure performance characteristics systematically achieved for testing tasks. Based on APIs document and test and efficiently becomes a critical challenge. In this paper, an cases, the proposed approach uses NLP [4] to parse and match approach for automatic performance testing for Representational corresponding test cases and API, and then generate test scripts State Transfer (REST)-based web applications is introduced. for performance testing tool automatically. On the one hand, Based on Application Programming Interface (API) document it eases the burden of test scripts design, implementation and test cases, the proposed approach employs natural language and maintenance efforts on software testers. On the other processing (NLP) to parse, match and generate test scripts for hand, software testers can focus more on the design of test performance testing tool automatically. It not only eases the cases and the analysis of test results. Finally, it facilitates the burden of test scripts design, implementation and maintenance execution of performance testing tasks through automation. efforts on software testers, but also facilitates the execution of performance testing tasks. Thus, the performance characteristics of web applications can be identified efficiently for further actions on development, Keywords–Performance testing; web application; software test- operation and maintenance tasks. ing. The remainder of this paper is organized as follows. Section I. INTRODUCTION II reviews related studies. Section III describes the design of the proposed architecture and Section IV presents the usage Software companies and service providers develop more of the architecture. Finally, Section V presents conclusion and and more web applications to provide their services over the future works. Internet. In addition to functionalities, potential users will consider performance characteristics, including response time, II. RELATED WORK throughput, latency, stability, etc. when selecting appropriate In this Section, related studies about automatic test web applications for their tasks [1]. Nowadays, in order to cases generation are introduced. Nebut,´ Fleurey, Traon, and fulfill various functional and quality requirements from users, Jez´ equel´ [5] proposed an approach for automating the gener- the complexity of web applications is increasing dramatically. ation of test scenarios from use cases. Through the developed Multi-tier considerations, the composition of different soft- transition system, the approach synthesized and generated ware components, architecture concerns, distributed or cluster test cases based on use cases extended with contracts. Jiang designs, and data processing mechanisms, all impose design and Ding [6] also designed a framework for automatically and implementation complexities on web applications. Thus, generating test cases from textual use cases. By using use cases the difficulties of testing activities arise correspondingly [2]. in specific format, the framework built an activity table for Furthermore, fast changing requirements and time to market constructing EFSM (Extended Finite State Machine), which is pressure could worsen the situation. In such circumstances, the base for test cases generation. Lipka et al. [7] presented a software testers need to frequently create or refine test cases, method for semi-automated generation of test scenarios based redesign and implement corresponding test scripts, and execute on textual use cases. The method derived the scenarios from test scripts to acquire results for further actions. Therefore, how use cases with annotations and then generated the sequence of to evaluate and ensure the performance characteristics of web method invocations for testing. Landhaußer¨ and Genaid [8] applications systematically and efficiently is a critical issue for proposed the usage of ontology to present the relationship software testers [3]. among source code, natural language elements of user stories In this paper, an approach for automatic performance and test scripts. Through the ontology, the test steps and testing for REST-based web applications is introduced. It related artifacts (e.g., APIs and test scripts) can be identified aims to provide software testers with an integrated process and reused for testing new user stories. Wang et al. [9] from test cases design, automatic test scripts generation, to proposed an approach that automatically generates executable Copyright (c) IARIA, 2016. ISBN: 978-1-61208-498-5 68 ICSEA 2016 : The Eleventh International Conference on Software Engineering Advances Figure 1. Overview of the automatic performance testing architecture. system test cases from use case specifications and a domain • Test Case: Software testers are responsible for the model. NLP is also used in the approach to extract behavioral design, implementation and maintenance of test cases information from use cases for test automation. Chen and based on requirements and specifications. Generally, Miao [10] presented an automatic approach to generate test test cases include preconditions, test steps, postcondi- scripts automatically against JavaScript. The study parsed test tions, and expected results. Test cases can be preserved cases in XML format and generated Java code for Selenium and managed by test case management systems. In to execute testing tasks. addition, several test case management systems (e.g., To sum up, several studies discussed about the automatic Testopia [12]) provide APIs for external access of generation of test cases for testing tasks. Based on previous specific contents within test cases. In the architecture, studies, the gap between test scenarios and test scripts is the test case will be retrieved and analyzed by a test considered in this study through the matching of test cases case parser. and APIs document to achieve more automatic testing. In • Test Case Parser: The test case parser is responsible addition, the integrated environment and process for facilitating for analyzing the test steps written in test case. By automatic testing is designed and introduced in this paper. using NLP, major components can be analyzed and identified, including cardinal number (CD), singular III. ARCHITECTURE DESIGN noun (NN), plural noun (NNS), verb (VB), determiner Fig. 1 depicts the design of the automatic performance test- (DT), to (TO), etc. Basically, CD can be considered ing architecture. Major actors and components are described as the configurations (e.g., number of users and work- as follows. load) in the performance test case. NN and NNS could • Software Testers: Software testers are responsible be actors, specific objects and the system, respectively. for all the testing activities throughout the software Finally, VB may indicate specific operations of the development process. Generally, the testing activities system. The analysis result can be used to match include (1) identify test environment, (2) identify per- corresponding APIs and determine configurations in formance acceptance criteria, (3) plan and design tests, test scripts. (4) configure the test environment, (5) implement the • API Catalog: The API catalog helps software de- test design, (6) execute the test, and (7) analyze results, velopers to create, update, query and maintain API report and retest [11]. In the proposed approach, documents for software systems. Famous API cata- software testers can focus on the planning and the logs [13] or API management services [14] include design of test cases based on requirements or quality Swagger, WSO2 API Management, Apigee, 3Scale, criteria. The implementation and the execution of tests etc. It is anticipated that software developers update can be achieved automatically. API documents to the API catalog once changes • Software Developers: Based on specific software happen. Thus, the API information will be kept up to development process, software developers perform re- date with the committed code and the deployed test quirement analysis, design, implementation and main- target. tenance tasks to web applications. The implementation • API Parser: The API parser is responsible for ana- will be committed to code