MODA: Automated Test Generation for Database Applications Via Mock Objects
Total Page:16
File Type:pdf, Size:1020Kb
MODA: Automated Test Generation for Database Applications via Mock Objects Kunal Taneja1, Yi Zhang2, Tao Xie1 1Department of Computer Science, North Carolina State University, Raleigh, NC, 27695, USA 2Center for Devices and Radiological Health, US Food and Drug Administration, Silver Spring, MD, 20993, USA 1{ktaneja, txie}@ncsu.edu, [email protected] ABSTRACT sequences, it becomes critical to guarantee the deployment of cor- Software testing has been commonly used in assuring the quality rect and sound database applications in these systems. of database applications. It is often prohibitively expensive to man- Software (unit) testing has been widely used by developers to ually write quality tests for complex database applications. Auto- assess the quality of database applications. Given that manual test mated test generation techniques, such as Dynamic Symbolic Exe- generation can be labor-intensive, developers usually rely upon au- cution (DSE), have been proposed to reduce human efforts in test- tomatic techniques [5, 7] to create quality tests, i.e., tests that ex- ing database applications. However, such techniques have two ma- ercise comprehensive behaviors of the program under test. Unfor- jor limitations: (1) they assume that the database that the applica- tunately, such automatic techniques face two significant challenges tion under test interacts with is accessible, which may not always when applied to database applications. be true; and (2) they usually cannot create necessary database states First, automatic test generation usually requires executing as a part of the generated tests. database applications to collect necessary information. Executing To address the preceding limitations, we propose an approach a database application, however, may alter the data stored in its that applies DSE to generate tests for a database application. In- database back-end, which may not always be desirable for data pri- stead of using the actual database that the application interacts with, vacy and preservation reasons. Using a local test database, instead our approach produces and uses a mock database in test generation. of the original one, may address this issue. However, it is not al- A mock database mimics the behavior of an actual database by per- ways an easy task to set up such a local database and clone config- forming identical database operations on itself. We conducted two urations of the original database onto it. empirical evaluations on both a medical device and an open source Second, no matter what database (original or test) is used, a test software system to demonstrate that our approach can generate, generation tool has to bridge the semantic gap between database without producing false warnings, tests with higher code coverage applications and their database back-ends. That is, to comprehen- than conventional DSE-based techniques. sively cover a database application, tests produced for it should not only provide inputs to the application itself, but also prepare neces- Categories and Subject Descriptors sary states in its database back-end. Mocking techniques [6], and their generalized variant called Pa- D.2.3 [Software Engineering]: Coding Tools and Techniques; rameterized Mock Objects (PMO) [10], can be used to alleviate the D.2.5 [Software Engineering]: Testing and Debugging first challenge. Such techniques replace an actual database with a mock object, called mock database, which responds to actual General Terms database queries with either default values (as in mocking tech- Languages, Experimentation niques) or values customized to explore program paths in the appli- cation under test (as in PMO). Keywords However, using mock databases is not a silver bullet. A mock Test Generation, Database Application, Mock Object database usually does not reflect the genuine behavior of the actual database that it simulates. Instead, it simply responds to a database query with customized values, even if the query is not satisfiable or 1. INTRODUCTION the values are unrealistic. Moreover, a mock database is incapable Database applications, i.e., programs interacting with database of simulating how an actual database changes its states during sys- back-ends, play an increasingly important role in mission-critical tem execution. Consequently, running tests with a mock database systems that require reliable data storage and correct data access. may result in unexpected results or false warnings being generated Since a failure of mission-critical systems can cause disastrous con- (i.e., tests fail in test execution but should not in reality; see Section 3 for an example). In this paper, we address the preceding challenges and present an automatic test generation approach for database applications. Permission to make digital or hard copies of all or part of this work for Given a database application and the schema defining its database personal or classroom use is granted without fee provided that copies are back-end, our approach produces a parameterized mock database not made or distributed for profit or commercial advantage and that copies that complies with the schema and, at the same time, captures the bear this notice and the full citation on the first page. To copy otherwise, to behavior of the database back-end. In particular, such a mock republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. database replicates the effect of operations over an actual database ASE’10, September 20–24, 2010, Antwerp, Belgium. by performing the same operations on itself, so that the state of the Copyright 2010 ACM 978-1-4503-0116-9/10/09 ...$10.00. Figure 1: Architectural Overview of MODA actual database is faithfully mirrored and only feasible values are Mocking Database. The component first parses the input returned for database queries. schema1 in order to obtain the information about how the origi- Having a mock database produced, our approach uses Dynamic nal database is structured. The input schema typically prescribes Symbolic Execution (DSE) [5, 7] to generate inputs for testing the the set of tables T defined in the original database, every column target database application. DSE runs the application, using the Ci of each table ti 2 T , and constraints over Ci. Based on the mock database, with both random and symbolic inputs. The execu- parsed information, the component creates and initializes tables in tion of the application with symbolic inputs, as well as the subse- a mock database DS, so that DS shares the same data structure as quent solving of path constraints collected during symbolic execu- the original database. tion, helps to explore program paths in the application that are not Mocking Database Interfaces. Database applications typically covered by random inputs. interact with database back-ends through standardized interfaces. We have implemented a prototype tool, called MODA (Mock To make an application under test talk to a mock database, it is es- Object Based Test Generation for Database Applications), for our sential to equip the mock database with the same set of interfaces approach. In its current implementation, MODA incorporates as a standard database offers. As currently implemented, the Mock Pex [9], a tool from Microsoft Research, as its DSE engine. To Object Framework component creates a mocking counterpart for make Pex work with a mock database, our tool equips it with two each API class and method defined in the MySQL library [8], espe- capabilities: (1) a code transformation feature that redirects the ap- cially query execution APIs that forward SQL queries to a database plication under test to interact with the mock database; and (2) a for execution. Given a query execution API, MODA produces its feature to inject records into the mock database, so that preparing mocking version to do two things: intercept the type and parame- database states as a part of tests becomes possible. ter information of the query forwarded by the API, and execute the We have conducted two empirical evaluations to assess the ef- corresponding mock operations on the mock database (see below fectiveness of MODA: one on a real-world medical device and the on mocking database operations) with the same parameters. For other on an open source software system. Results of the evaluations other APIs in the MySQL library, such as those that facilitate the demonstrate that our approach can achieve higher code coverage, establishment of database connections or the creation of SQL com- with no false warnings, than conventional DSE-based techniques. mands, MODA either trivially mocks them as empty methods or replicates their functionalities according to their specifications. 2. APPROACH Mocking Database Operations. When mocking actual databases, conventional mocking or PMO techniques suffer from Given a database application and the schema of its database the fact that they can neither interpret SQL queries nor simulate back-end, MODA coordinates four of its components, namely how a database changes its states over program execution. As a re- Mock Object Framework Code Transformer Test Generator , , , and sult, these techniques may produce infeasible tests that are typically Test Transformer , to produce a test suite that systematically exer- difficult for developers to filter out. cises the target application. MODA addresses the preceding issue by performing the same Figure 1 presents an architectural view of MODA, where the operations over the mock database as the application under test in- Mock Object Framework component establishes a mock database structs its database back-end to do, so that the effects of such oper- Code Transformer that simulates the database back-end and the ations over the original database are faithfully reflected in the mock component pre-processes the target application such that all of its database. Typically, a database application requests its database to database interactions are replaced by interactions with the mock manipulate the stored data by issuing four types of queries, namely Test Generator database. The component of MODA serves as SELECT , INSERT , UP DAT E, and DELET E.