Convolutional Neural Networks over Control Flow Graphs for Software Defect Prediction Anh Viet Phan, Minh Le Nguyen Lam Thu Bui Japan Advanced Institute of Information Technology Le Quy Don Technical University Nomi, Japan 923-1211 Ha Noi, Vietnam Email: fanhphanviet,[email protected] Email: [email protected] Abstract—Existing defects in software components is unavoid- able and leads to not only a waste of time and money but also many serious consequences. To build predictive models, previous studies focus on manually extracting features or using tree representations of programs, and exploiting different machine learning algorithms. However, the performance of the models is not high since the existing features and tree structures often fail to capture the semantics of programs. To explore deeply programs’ semantics, this paper proposes to leverage precise graphs repre- senting program execution flows, and deep neural networks for automatically learning defect features. Firstly, control flow graphs are constructed from the assembly instructions obtained by compiling source code; we thereafter apply multi-view multi-layer (a) File 1.c directed graph-based convolutional neural networks (DGCNNs) (b) File 2.c to learn semantic features. The experiments on four real-world datasets show that our method significantly outperforms the Fig. 1: A motivating example baselines including several other deep learning approaches. Index Terms—Software Defect Prediction, Control Flow Graphs, Convolutional Neural Networks Recently, several software engineering problems have been I. INTRODUCTION successfully solved by exploiting tree representations of Software defect prediction has been one of the most attractive programs - the Abstract Syntax Trees (ASTs) [12]. In the research topics in the field of software engineering. According field of machine learning, the quality of input data directly to regular reports, semantic bugs result in an increase of affects the performance of learners. Regarding this, due to application costs, trouble to users, and even serious conse- containing rich information of programs, tree-based approaches quences during deployment. Thus, localizing and fixing defects have shown significant improvements in comparison with in early stages of software development are urgent requirements. previous research, especially software metrics-based. Mou Various approaches have been proposed to construct models et al. proposed a tree-based convolutional neural network to which are able to predict whether a source code contains extract structural information of ASTs for classifying programs defects. These studies can be divided into two directions: one by functionalities [15]. Wang et al. employed a deep belief is applying machine learning techniques on data of software network to automatically learn semantic features from AST metrics that are manually designed to extract features from tokens for defect prediction [23]. Kikuchi et. al measured the arXiv:1802.04986v1 [cs.SE] 14 Feb 2018 source code; the other is using programs’ tree representations similarities between tree structures for source code plagiarism and deep learning to automatically learn defect features. detection [10]. The traditional methods focus on designing and combining However, defect characteristics are deeply hidden in pro- features of programs. For product metrics, most of them are grams’ semantics and they only cause unexpected output in based on statistics on source code. For example, Halstead met- specific conditions [24]. Meanwhile, ASTs do not show the rics are computed from numbers of operators and operands [8]; execution process of programs; instead, they simply represent CK metrics are measured by function and inheritance counts the abstract syntactic structure of source code. Therefore, [6]; McCabe’s metric estimates the complexity of a program both software metrics and AST features may not reveal many by analyzing its control flow graph [13]. However, according types of defects in programs. For example, we consider the to many studies and surveys, the existing metrics often fail in procedures with the same name sumtoN in two C files File capturing the semantics of programs [9], [14]. As a result, 1.c and File 2.c (Fig. 1). Two procedures have a tiny although many efforts have been made such as adopting difference at line 7 File 1.c and line 15 in File 2.c. As robust learning algorithms and refining the data, the classifier can be seen a bug from File 2.c, the statement i=1 causes performance is not so high [4]. an infinite loop of the for statement in case N>=1. Whereas using RSM tool1 to extract the traditional metrics, their feature • The model implementation is released to motivate related vectors are exactly matching, since two procedures have the studies. same lines of code, programming tokens, etc. Similarly, parsing The remainder of the paper is organized as follows: Section II the procedures into ASTs using Pycparser2, their ASTs are explains step-by-step our approach for solving software defect identical. In other words, both metrics-based and tree-based prediction problem from processing data to adapting the approaches are not able to distinguish these programs. learning algorithm. The settings for conducting the experiments To explore more deeply programs’ semantics, this paper such as the datasets, the algorithm hyper-parameters, and proposes combining precise graphs which represent execution evaluation measures are indicated in Section III. We analyze flows of programs called Control Flow Graphs (CFGs), and a experimental results in Section IV, and conclude in Section VI. powerful graphical neural work. Regarding this, each source code is converted into an execution flow graph by two stages: II. THE PROPOSED APPROACH 1) compiling the source file to the assembly code, 2) generating This section formulates a new approach to software defect the CFG from the compiled code. Applying CFGs of assembly prediction by applying graph-based convolutional neural net- code is more beneficial than those of ASTs. Firstly, assembly works over control flow graphs of binary codes. Our proposed code contains atomic instructions and their CFGs indicate method is illustrated in Fig. 3 includes two steps: 1) generating step-by-step execution process of programs. After compiling CFGs which reveal the behavior of programs, and 2) applying a two source files (Fig. 1) we observed that i+=1 and i=1 are graphical model on CFG datasets. In the first step, to obtain the translated into different instructions. Secondly, assembly code graph representation of a program, the source code is compiled is refined because they are the products after AST processing of into an assembly code using g++ on Linux. The CFG thereafter the compiler. The compiler applies many techniques to analyze is constructed to describe the execution flows of the assembly and optimize the ASTs. For example, two statements int instructions. In the second step, we leverage a powerful deep n; n=4; in the main procedure of File 2.c are treated in neural network for directed labeled graphs, called the multi- a similar way with the statement int n = 4; and, the if view multi-layer convolutional neural network, to automatically statement can be removed since the value of n is identified. build predictive models based on CFG data. Meanwhile, ASTs just describe the syntactic structures of programs, and they may contain many redundant branches. A. Control Flow Graphs Assume that if the statement i=1 in procedure sumtoN of A control flow graph (CFG) is a directed graph, G = (V,E) File 2.c is changed to i+=1 then both programs are the where V is the set of vertices fv1; v2; :::; vng and E is the same. However, their AST structures have many differences set of directed edges f(vi; vj); (vk; vl); :::g [1]. In CFGs, each including the positions of subtrees of two functions main and vertex represents a basic block that is a linear sequence of sumtoN, the separation of declaration and assignment of the program instructions having one entry point (the first instruction variable n, and the function prototype of sumtoN in File executed) and one exit point (the last instruction executed); 2.c. and the directed edges show control flow paths. We thereafter leverage a directed graph-based convolutional This paper aims to formulate a method for detecting faulty neural network (DGCNN) on CFGs to automatically learn source code written in C language. Regarding this, CFGs of defect features. The advantage of the DGCNN is that it can assembly code, the final products after compiling the source treat large-scale graphs and process the complex information of code, server as the input to learn faulty features using machine vertices like CFGs. The experimental results on four real-world algorithms. Based on recent research, it has been proved that datasets show that applying DGCNN on CFGs significantly CFGs are successfully applied to various problems including outperforms baselines in terms of the different measures3. malware analysis [2], [3], software plagiarism [5], [22]. Since The main contributions of the paper can be summarized as semantic errors are revealed while programs are running, follows: analyzing the execution flows of the assembly instructions • Proposing an application of a graphical data structure may be helpful for distinguishing faulty patterns from non- namely Control Flow Graph (CFG) to software defect pre- faulty ones. diction and experimentally proving that leveraging CFGs Fig. 2 illustrates an example of the control flow graph is successful in building high-performance classifiers. constructed from an assembly code snippet, in which each • Presenting an algorithm for constructing Control Flow vertex corresponds to an instruction and a directed edge shows Graphs of assembly code. the execution path from an instruction to the other. The pseudo- • Formulating an end-to-end model for software defect code to generate CFGs is shown in Algorithm 1. The algorithm prediction, in which a multi-view multi-layer convolutional takes an assembly file as the input, and outputs the CFG. neural networks is adopted to automatically learn the Building the CFG from an assembly code includes two major defect features from CFGs.