The Thirtieth AAAI Conference on Innovative Applications of Artificial Intelligence (IAAI-18)
Discovering Program Topoi through Clustering
Carlo Ieva,1,2 Arnaud Gotlieb,1 Souhila Kaci,2 Nadjib Lazaar2 1SIMULA Research Laboratory, Oslo, Norway 2LIRMM, University of Montpellier, France {carlo, arnaud}@simula.no, {souhila.kaci, nadjib.lazaar}@lirmm.fr
Abstract functions entry points and the sorting criterion is based on the distance between common functions and entry points. Understanding source code of large open-source software The whole method can be summarized according to its three projects is very challenging when there is only little docu- mentation. New developers face the task of classifying a huge main steps: number of files and functions without any help. This paper 1. Preprocessing. Source code, with its comments, is parsed documents a novel approach to this problem, called FEAT, generating for every code unit (either functions in proce- that automatically extracts topoi from source code by using dural languages or methods in object oriented ones) a cor- hierarchical agglomerative clustering. Program topoi summa- responding textual document. Additionally, a graph rep- rize the main capabilities of a software system by present- ing to developers clustered lists of functions together with resentation of the caller-callee relationship (call graph) is an index of their relevant words. The clustering method used also created in this step. in FEAT exploits a new hybrid distance which combines 2. Clustering. Code units are grouped together by means of both textual and structural elements automatically extracted hierarchical agglomerative clustering (HAC). from source code and comments. The experimental evalu- ation of FEAT shows that this approach is suitable to un- 3. Entry Point Selection. Within the context of every cluster, derstand open-source software projects of size approaching code units are ranked and those placed in higher positions 2,000 functions and 150 files, which opens the door for its will constitute a program topos. deployment in the open-source community. The contribution of this paper is three-fold: Introduction 1. FEAT is a novel, fully automated approach for program topoi extraction based on clustering units directly from The development of large-scale open-source projects in- source code. To exploit HAC, we propose an original hy- volves many distinct developers contributing to the creation brid distance combining structural and semantic elements of large code repositories. As an example, the July 2017 re- of source code. HAC requires the selection of a partition lease of the Linux kernel (v. 4.12), amounting to almost 20 among all those produced along the clustering process, MLOC (lines of code), requested the effort of 329 develop- our approach makes use of a hybrid criterion based on ers marking a growth of 1 MLOC respect to its predeces- graph modularity (Donetti and Munoz˜ 2004) and textual sor. This figures make clear that, whenever a new developer coherence (Foltz, Kintsch, and Landauer 1998) to auto- wants to become a contributor, he/she has to face the prob- matically select the appropriate one; lem of understanding a huge amount of code, organized as an unclassified set of files and functions. 2. Clusters of code units need to be analyzed to extract pro- Organizing the code in a more abstract way, closer to hu- gram topoi. We define a set of structural elements ob- mans is an attempt that received interest from the Software tained from source code and use them to create an alter- Engineering community. Unfortunately, there is no recog- native representation of clusters of code units. Principal nized recipe or tool that can concretely provide any help in Component Analysis (PCA), with its capability of dealing dealing with large software repositories. with multi-dimensional data, provides us a way to mea- We propose an effective solution to this problem by auto- sure the distance of code units respect to an ideal entry matically extracting program topoi which are ordered lists of point. This distance is the basis for code units’ ranking function names associated with an index of relevant words. shown to end-users. How does the sorting take place? FEAT does not con- 3. We implemented FEAT on top of a general-purpose soft- sider all functions as equal: some of them are considered ware analysis platform and performed an experimental like a gateway to the implementation of high-level, user- study over some open-source software projects. During observable capabilities of a program. We call these special the evaluation we analyzed FEAT under several perspec- Copyright © 2018, Association for the Advancement of Artificial tives: the clustering step, effectiveness in topoi discovery Intelligence (www.aaai.org). All rights reserved. and, scalability of the approach.
7771 Related work ploits both text mining and code structure analysis to guide Our research spans the domain of program understanding the creation of clusters. focusing mainly on feature extraction (Rubin and Chechik We believe that there are several differences between 2013; Chen and Rajlich 2000; Marcus and Haiduc 2013) FEAT and the above mentioned approaches and that FEAT that aims to automatically discover the main characteristics fosters program understanding by creating a novel hybrid of a software system by analyzing source code or other arti- approach blending structural and semantic aspects of source facts. It must be distinguished from feature location, whose code. objective is to locate where and how these characteristics are implemented (Rubin and Chechik 2013). Feature lo- cation requires the user to provide an input query where Background the searched characteristic is already known, while feature extraction tries to automatically discover these characteris- Software Clustering tics. Since several years, software repository mining is con- sidered mainstream in feature extraction. However, we can Software clustering methodologies create group of entities, distinguish between software-repository mining approaches such as classes, functions, etc. of a software system in order dealing with software documentation only and, those deal- to ease the process of understanding the high-level structure ing with source code only. of a large and complex software system (Shtern and Tzerpos Mining Software Documentation. In (Dumitru et al. 2012). The basis for any cluster analysis to group entities is ), both text-mining techniques and flat clustering are used their set of attributes. to extract feature descriptors from user requirements kept The application of clustering to a software system re- in software repositories. By combining association-rules quires the identification of the entities which are the object mining and k-Nearest-Neighbour, the proposed approach of the grouping. Several artifacts can be chosen but the most makes recommendations on other feature descriptors to popular one is source code (Mitchell 2003). The selection strengthen an initial profile. More recently (McBurney, Liu, of entities is affected by the objective of the method. For and McMillan 2016) presented four automatic generators of program restructuring at a fine-grained level, function call list of features for software projects, which select English statements are chosen as entities (Xu et al. 2004) while for sentences that summarize features from the project docu- design recovery problems (Andritsos and Tzerpos 2005) en- mentation. tities are often software modules but also classes or routines. FEAT has two distinguishing elements w.r.t. these tech- Extracting facts from source code can be done follow- niques. Firstly, FEAT deals with both software documen- ing two different conceptual approaches: structural and se- tation and source code by applying at the same time code mantic. Structure-based, approaches rely on static relation- and text analysis techniques. Secondly, FEAT uses HAC as- ships among entities: variable references, procedure calls, suming that software functions are organized according to a inheritance etc. Semantic approaches take into account the certain (hidden) structure that can be automatically discov- domain knowledge information contained in source code ered. and extracted from comments and identifier names (Kuhn, Mining Source Code. (Linstead et al. 2007) proposes Ducasse, and Gˆırba 2005). The software clustering commu- dedicated probabilistic models based on code analysis us- nity widely adopts structure-based approaches but it has to ing Latent Dirichlet Allocation to discover features under the be noted that the output produced by semantic approaches form of so-called topics (main functions in code). (McMillan tends to be more meaningful. That is why some try to com- et al. 2012) present a source-code recommendation system bine the strengths of both methods like (Tzerpos and Holt for software reuse. Based on a feature model (a notion used 2000). in product-line engineering and software variability model- ing), the proposed system tries to match the description with The actual cluster creation is accomplished through a relevant features in order to recommends the reuse of exist- clustering algorithm. Clustering is the most common form ing source code from open-source repositories. (Abebe and of unsupervised learning and the key input to a clustering Tonella 2015) proposes natural language parsing to automat- algorithm is the distance measure. Different distance mea- ically extract an ontology from source code. Starting from a sures give rise to different clusterings. lightweight ontology (a.k.a, concept map), the authors de- There are two categories of hierarchical algorithms: ag- velop a more formal ontology based on axioms. glomerative (bottom-up) and divisive (top down). In soft- Conversely, FEAT is fully automated and does not re- ware clustering, according to (Kaufman and Rousseeuw quire any form of training or any additional modeling activ- 1990), divisive algorithms offer an advantage over agglom- ity (e.g., feature modeling). It uses an unsupervised machine erative ones because users are mostly interested in the struc- learning technique making its usage and application much ture revealed by the large clusters created during the early simpler. stages of the process. On the other hand, the way agglom- (Kuhn, Ducasse, and Gˆırba 2007) use clustering and LSI erative clustering proceeds towards large clusters may be (Latent Semantic Indexing) to assess the similarity between affected by unfortunate decisions made in the first steps. source artifacts and to create clusters according to their sim- Agglomerative hierarchical clustering are most widely used ilarity. The most relevant terms extracted from the LSI anal- however. This is because it is infeasible to consider all pos- ysis are reused for labeling the clusters. FEAT instead ex- sible divisions of the first large clusters (Wiggerts 1997).
7772 the twofold assumption lying behind the approach. Hence, if FEAT n Text 3 Modularity SW is a software system counting code units its represen- Clustering tation can be defined through the call graph of SW and the set of unit documents D as: Call Graph Coherence Definition 1. (FEAT model) SW CG, D
2 4 The first element (structural) of the pair is the call graph Clusters CG = (U, E) where U = {u1,u2, ··· ,un} is the set of units and E the set of edges representing the caller-callee C1 C2 C3 CnC relationships. The second one (semantic) is the set of unit- documents: D = {d1, d2, ··· , dn}. U1 U3 U4 The creation of the semantic part of FEAT requires that, Cluster Ranking U2 U5 U6 UnUn U7 for every unit, we extract the following elements: code unit 1 Entry Point Selection 5 names, variable names, string literals and, comments. All these elements contribute to the creation of a set of tex- 6 Topoi tual documents; one for each unit. This preliminary text, 1 where the original words’ order is ignored (bag of words), Source Code 1 Ua undergoes several transformations (tokenization, stop words 2 Ub removal, stemming, weighting) producing a unit-document m 3 Uc noted as du =[w1,w2, ··· ,wm] ∈ IR where wi is the weight associated to the ith-word.
Figure 1: FEAT process’s overview Distance between Code Units The key input to a clustering algorithm is the distance mea- sure allowing to group data points together. In FEAT we Call Graph have two distances, one for each of the two components included in Def.1. Let us begin with the distance over the A Call Graph (CG) is a convenient way of representing the structural part of SW. unit caller-callee relationship in a software program. Given Given the (undirected) call graph CG, the distance be- a program P composed of a collection of units {ui}i∈1...n, tween two code units ua and ub is computed by using the the CG of P is a graph where each node is associated with a |π u ,u | k u u u u length of a shortest path (noted as ( a b) = ) between unit i and there is an arc from i to j if and only if j is them: called by ui at some location in the source code. Note that ⎧ u u even though ui may call uj several times, there is only a ⎪0 if a = b ⎨ k−1 single arc from ui to uj in CG. u ,u 1−λ λi |π u ,u | k dCG( a b)=⎪ 1−λD if ( a b) = (1) ⎩⎪ i=0 FEAT Approach 1 if |π(ua,ub)| = ∞ Understanding a software system through its source code where D is the call graph diameter (the length of the longest can be pursued following two conceptual approaches: struc- shortest-path) and λ>1 is a parameter used to provide an tural and semantic. Structure-based approaches focus on the exponential growing of the distance. static relationships among entities while semantic ones in- The distance over the semantic part of the system’s repre- clude all aspects of a system’s domain knowledge that can sentation is computed through the angular distance defined be obtained from comments and identifier names (Shtern as: d · d and Tzerpos 2012). Discovering the main capabilities of a 2 ua ub dD(ua,ub)= arccos (2) software system can benefit from structural information that π du du can be used to identify a capability as a set of structurally a b close code units contributing to its implementation. On the Hybrid distance other hand, under the semantic view, those parts of a system Both dD and dCG are proper distance measures satisfying showing commonalities in terms of natural language words the three axioms: symmetry, positive definiteness and, tri- can be considered part of a system’s capability as well. angular inequality. On their basis we present a novel hybrid Structural and semantic approaches convey two different, distance with the objective of producing clusters whose ele- both valuable, perspectives of a system. FEAT combines ments show high internal cohesion under both perspectives: them in order to achieve a more accurate picture. FEAT, structural and semantic. This combination can mitigate some whose overview is depicted in Fig.1, is based on a three steps unwanted effects that may occur if we use only one distance. process: preprocessing (box noted 2), clustering (3) and, en- For instance, two units sharing many words, but not con- try point selection (5). The input to FEAT is a software sys- nected in the call graph (or very far from each other), would tem SW considered as its source code and comments (1). In the preprocessing step (2) FEAT parses source code and 1Quoted sequence of characters representing a string value i.e. comments creating a representation of SW which supports x="foo"
7773 be evaluated with high similarity if only dD be used, while, Raghavan, and Schutze¨ 2008). Following the idea of a com- without any kind of structural relationship, they cannot be- bination of structural and semantic aspects of source code, long to the same feature. Similarly, two close units in the we introduce our hybrid cutting criterion. call graph, but without any word in common, should not be Given a partition of vertices of a graph, modularity clustered together because we assume that elements of a ca- (Aaron, Newman, and Moore 2004) reflects the concentra- pability should share a common vocabulary. tion of edges within clusters compared with a random dis- The hybrid distance is defined as a linear combination of tribution of links between all nodes regardless of clusters. dD and dCG using a real number α ranging in [0, 1]: We use modularity to measure the division of a call graph in clusters having high inter-cluster sparsity and high intra- u ,u α u ,u − α u ,u dFEAT( a b)= dD( a b)+(1 )dCG( a b) (3) cluster density. Modularity is defined as: α The external parameter is used to tune the impact of one 1 kikj α Q(P)= Ai,j − δ(ci,cj) (5) distance value over the other. The choice of a value for 2m 2m depends on some characteristics of the code under analysis i,j like the quality of comments, naming conventions etc. More 2 where A is the adjacency matrix , ki (resp. kj) is the de- details about this will be provided in the experimental sec- i j c c i tion. gree of node (resp. ), i (resp. j) is the cluster of (resp. j), and m is the total number of edges. Function δ is the δ c ,c c c i j Distance over Clusters Kronecker delta: ( i j)=1iff i = j (nodes , are in the same cluster), 0 otherwise. High values of modularity In order to merge clusters, HAC requires to compute the dis- 1 (knowing that Q ∈ [− 2 , 1]) correspond to interesting parti- tance between sets of units. In FEAT we represent a set of tions of a call graph. units as its centroid hence, the distance between two clusters A measure adopted in natural language processing (NLP), corresponds to the distance of their centroids. Following the used for assessing how similar are the segments of a text, is concept of a hybrid representation of structural and seman- coherence (Foltz, Kintsch, and Landauer 1998). Coherence tic elements we define a hybrid centroid distance. Unit doc- is based on the measure of words overlapping. We consider uments lie in a Euclidean space then the centroid of a cluster all unit-documents belonging to a cluster as sections of a C μ C d1+d2+···+dn is D( )= |C| . Instead, the structural part of whole text. Developers, within the context of the units par- C is represented as a graph medoid. Medoids are represen- ticipating in the implementation of a system capability, use tatives of a discrete set of elements and in FEAT they are a consistent language revealed through the choice of names defined as: for variables, the text in comments, etc. So, while looking at clusters as they were textual documents, we want to find the Definition 2. (Graph Medoid) Let CG = (U, E) be a call C {u ,u ,...,u } |π u ,u | partition showing the highest coherence defined as: graph, = 1 2 m a cluster, ( a b) a shortest ⎛ ⎞ u u σ u |π u ,u | path between a and b, C( )= ( i ) , |C| |C| ∀ui∈C ⎝ 2 ⎠ 1 H(P)= 1 − dD(uk,uj) M = {u | arg min {σC(u)}} and, s = |C| min{σC(u)} |C|(|C| − 1) u∈U u∈U ∀C∈P k=1 j=k+1 then the graph medoid of C is: (6) Finally, we can define a new hybrid measure as the com-
2 bination of normalized coherence and modularity: μCG(C)=argmin (|π(ui,u)|−s) u∈M ∀ui∈C H(P) 2Q(P)+1 T (P)=α +(1− α) FEAT |P| (7) In other words, the graph medoid of C is the unit u ∈U 3 lying at the most central position w.r.t. all units in C. The cutting criterion is then determined as the HAC’s itera- C C Finally, the hybrid distance between two clusters i and j tion where the maximum value of TFEAT is reached. is: Algorithm 1 presents our clustering step with a hybrid d (Ci, Cj)=αdD (μD(Ci),μD(Cj)) representation of units. We use the priority queue version FEAT n2 n − α μ C ,μ C (4) of HAC having a time complexity of Θ( log( )). Alg.1 +(1 )dCG( CG( i) CG( j)) starts by considering each unit as a cluster at lines 4-6. It Eq.3 can now be seen as a special case of Eq.4 occurring computes the pairwise distances between clusters at lines 7- when |Ci| = |Cj| =1. 8. Then, it iteratively merges pair of clusters according to a minimal dFEAT distance value (lines 11-12) until either a Cutting Criterion partition is reduced to one cluster or the cutting criterion is reached (i.e., TFEAT value cannot be improved) (line 9). At HAC does not require a prespecified number of clusters, it each iteration, the algorithm updates the pairwise distances creates a hierarchy of them through an iterative process pro- Δ of the new partition (lines 14-15). At the end, the algo- ducing a new partition of clusters at every merging step. rithm returns a partition P of m clusters. How to choose a partition P = {C1, C2,...,Cm} among the 2 n − 1 possible ones (where n is the number of data points) Aij =1if there exists an edge between vertices i and i and depends on the objectives of the domain at hand (Manning, Aij =0otherwise
7774 Algorithm 1: Priority Queue HAC with hybrid cut- Definition 3. (Entry Point Ranking) Let C be a cluster of n ting criterion units, the ranking over its units is defined as:
1 In U = {u1,...,un}: n units; α ∈ [0, 1],τ <0; KC = {k1,k2, ··· ,kn | 2 Out P = {C1,...,Cm}: partition of m clusters; d(qC, vk1 ) ≤ d(qC, vk2 ) ≤···≤d(qC, vkn )} 3 P←∅; Δ ←∅; yp,yc ← 1, 0 ; 4 foreach ui ∈Udo Program Topoi 5 Ci ←{ui}; We can now provide a formal definition of a program topos. 6 P←P∪Ci; Definition 4. (Program Topos) Let ΔC be the set of dis- 7 foreach Ci, Cj ∈P: i
7775 The generation of any instance of the model is replicated Hexdump gEdit ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● 10 times and the results, shown in Tab.1, are averaged out. ● ) ) % % HAC merges units and clusters, driven by dFEAT, produc- ( ( ing partitions and, selects the partition where the maximum ccuracy ccuracy A A All value for TFEAT occurs. To compare FEAT’s value to brute ● Comments Code
T ● ● ● ● ● ● force approach ones, we provide an indicator called score. 0 20406080100 0 20406080100 T U 0.0 0.2 0.4 0.6 0.8 1.0 0.0 0.2 0.4 0.6 0.8 1.0 Let us call the set of TFEAT values of all partitions of α α generated through the brute force approach, then we define T −min(T ) Tscore = FEAT ∈ ● the cutting criterion score as: ● max(T )−min(T ) ● ● ● ●
) ) ●
% ● %
( ● ( ● ● , ●
[0 1] representing how close is the partition found from on on ● i i ● s s
i i ●
● FEAT to the optimal ones. rec rec P P Every line in Tab.1 contains the average values of 10 tests
● ● ● ● ● ● with the same input values (n, ρ, α =0.5). 0 5 10 15 20 25 0 5 10 15 20 25 0.0 0.2 0.4 0.6 0.8 1.0 0.0 0.2 0.4 0.6 0.8 1.0 α α
n ρ Tscore n ρ Tscore n ρ Tscore ● ● ● ● 0.1 0.887 0.1 0.880 0.1 0.903 ● ● ● ●
● ● ● ●
0.2 0.811 0.2 0.893 0.2 0.801 ) ● ) % % 0.3 0.759 0.3 0.822 0.3 0.848 ll( ● ● ● ll( eca eca 0.4 0.740 0.4 0.792 0.4 0.738 R R 0.5 0.731 0.5 0.750 0.5 0.671
● ● ● ● ● ●
10 11 12 0 20406080 0 20406080 0.6 0.762 0.6 0.766 0.6 0.795 0.0 0.2 0.4 0.6 0.8 1.0 0.0 0.2 0.4 0.6 0.8 1.0 0.7 0.780 0.7 0.817 0.7 0.778 α α 0.8 0.872 0.8 0.849 0.8 0.828 0.9 0.973 0.9 0.908 0.9 0.977 Figure 2: FEAT’s performance relative to the analysis of 0.813 0.831 0.815 Hexdump and gEdit while α and, extracted textual elements vary Table 1: Experimental results of the clustering step’s evalu- ation The evaluation proceeds by comparing the entry points auto- We observe that when HAC is driven by d and T FEAT FEAT matically extracted from FEAT with those in the oracle and it shows an interesting performance in finding partitions with computing accuracy, precision and, recall. The input vari- both high modularity and coherence. With a very low num- ables for the experiment are: α and some predefined set of ber of iterations, which in the worst case can be equal to textual elements. The ranges for these variables are: n − 1, FEAT shows an average Tscore > 0.8 in all combi- nations that we experimented. • α ∈ [0, 1] with increments of 0.1 Discovering Topoi through FEAT • Textual sets of elements: Code, Comments and, All. Code includes: identifiers and literals. Comments: just com- This section presents an experiment for the evaluation of ments are extracted. All: union of code and comments. FEAT as a technique for recovering the main capabilities of a software system. We selected two open-source softwares This experiment generated 60 tests whose results are sum- (more details about them can be found in Tab.4 lines 2 and marized in Fig.2. The first observation we can make is that 7): the cutting criterion drastically affects FEAT’s behaviour. As soon as α goes above a given value (α ≥ 0.8 for Hex- • Hexdump. It is an hexadecimal viewer, i.e. an application dump and α ≥ 0.6 for gEdit), the quality of classifications that displays binary data contained in files as readable se- drastically drops. The reason for this is that the clustering quences of characters. process, which for α ≥ 0.6 is mainly driven by semantic el- • gEdit. The default text editor of the GNOME desktop en- ements, is stopped too early creating clusters which are too vironment. small and, from the structural standpoint, too fragmented. To test the approach we created one oracle for each project When clusters are internally too sparse the entry point rank- by looking at the user’s manual and inspecting the source ing cannot be executed. Instead, balanced values of α ≈ 0.5 code. gEdit’s oracle contains 39 unit names selected as entry show the best performance for accuracy (≥ 80%) and recall points while Hexdump’s one contains 12 entry points. The (≈ 70%). value for β is 0.25 (see Def.4). The research questions we Let us summarize the results in the light of the research address here are: questions. (RQ2) A balanced combination of α ≈ 0.5 leads • RQ2: What is the impact of α on FEAT’s performance? to higher recall values for every set of extracted textual ele- ments. Accuracy and precision do not seem to be much af- • RQ3: How does the selection of textual elements (identi- fected by α.(RQ3) More textual elements, as in the All com- fier, comments, etc.) affect FEAT’s performance? bination, does not imply better performance. The blue line • RQ4: How useful are topoi to identify system’s capabili- in Fig.2 is related to Comments which obtains higher values ties? in almost all cases. The last research question (RQ4) deals
7776 Unit Name Index Project LOC #Unit #File Dict. ρ chooser, cmd, [...], dialog, document, file, 1 Linux FS EXT2 8,445 180 14 748 0.0201 gedit cmd file open folder, [...], open 2 Hexadec. Viewer 12,053 254 13 764 0.0091 [...], findit, found, [...], hit, [...], mask, pattern, GNU bc findit cb 3 12,851 215 20 723 0.0204 search Calculator 1.06 Intel Ethernet 4 30,499 581 16 1,479 0.0062 Table 2: Part of two program topoi obtained from the analy- Drivers and Util. sis of gEdit (first line) and Hexdump 5 Ultradefrag v7.0 34,637 1,112 74 1,874 0.0054 Zint Barcode 6 38,095 345 43 1,275 0.0134 Generator v2.3.0 Unit Name d(qC , vi) 1 gedit cmd search find {prev,next} 0.139 7 GNU Editor v3.2 42,718 1,370 59 1,048 0.0021 gedit cmd edit {delete,copy, 8 bash v1.0 70,955 1,477 128 2,216 0.0027 2 0.158 select all,cut,redo,undo} 9 Linux IPv4 84,606 2,216 127 3,211 0.0011 findit cb 0.302 x3270 Terminal 3 10 91,449 1,881 136 3,008 0.0025 find next cb 0.311 Emulator v3.5
Table 3: Extract of the rankings of two program topoi with Table 4: Open Source software projects used during the scal- distances between entry points and query vector ability evaluation with the usability of FEAT in a real setting. Tab.2 shows two (black) and the clustering time (blue). The difference be- entry points of two program topoi: the first from gEdit and tween black and blue curves is equal to the time needed the second from Hexdump. In the first line we have the open by the preprocessing step. In all cases the time of the en- file capability of the text editor while in the second one the try point selection step can be neglected. The right hand side find a pattern capability of Hexdump. Some of the words of Fig.3 contains the graphs about memory usage. In this contained in the indices provide useful clues about the con- case we plot only one graph per characteristic because no text and purpose of the capability accessed through the entry meaningful differences have been observed among the var- point to which they belong. ious values of α. Let us focus on the running time graphs. Another interesting point to highlight is shown in Tab.3 The analysis of running time leads to two main observations. α> where we have three groups of units taken from two topoi First, performance is negatively affected by values of 0 of the two projects. Every group lists units at the same (or which means that the computation of the textual distance quite similar) distance to the query vector. The interesting (Eq.2) and the coherence criterion (Eq.6) are demanding. In- thing is that the units in every group belong to the same deed the graph KWords/Time shows a steep increase around ≈ functional area and their relatedness can be revealed by the 2KWords. Furthermore, looking at the KUnit/Running RT ≈ reciprocal proximity in the PCA space. In the first group we Time graph for 2KUnit we have α=0 1400 s and RT ≈ have find related functionality (the suffixes to complete unit α=0.5 10 000 s. Second, the clustering step, whose n2 n names are between curly braces), in the second one some complexity is Θ( log( )), is the most costly one when α> α clipboard management units and, in the last one again find 0 while for =0running time is dominated by related functionality. We believe that these information can the preprocessing step. Density shows a negative correla- be profitably used for the discovery of system’s capabilities. tion w.r.t. both time and memory. Regarding the memory usage no clear patterns emerged, nevertheless we observe Scalability Evaluation of FEAT a fast growth in relationship with KLOC ≥ 70. In con- clusion, FEAT shows acceptable performance with projects The adoption of a new methodology is determined also by counting up to 1.3KUnit and dictionaries with size up to its scalability, then in this section we address the following ≈ 2KWordsproducing results in about 20 min, for greater research question: values of these characteristics the full hybrid approach be- • RQ5 What is the correlation between FEAT’s running comes rapidly too costly. time and memory usage w.r.t. software projects charac- As further work, we believe that improving the efficiency teristics such as: number of units, size of the dictionary, of distances’ computation over the semantic part of the LOC and, call graph density? source code, would ease the adoption of the tool and maxi- FEAT has been implemented on top of a software testing mize its impact. platform based on OSGi (Open Services Gateway initia- tive) and BPMN (Business Process Modeling Notation). The Conclusion hardware used for running the experiments is based on an FEAT is a novel approach for program understanding repre- Intel dual core i7-4510U CPU with 8GB RAM. senting a software system under both structural and seman- The last experiment, whose results are reported in Fig.3, tic perspectives. Program topoi are concrete and useful sum- involves the projects listed in Tab.4. On the left hand side maries of systems’ capabilities providing a valuable help in we have graphs reporting the running time (RT [s])ofthe several phases of the software life-cycle. FEAT is a fea- experiment while α varies in {0, 0.5, 1}. For every value of sible, unsupervised approach for automatically discovering α we have two curves: the total time employed by FEAT program topoi directly from source code.
7777 ● ● Donetti, L., and Munoz,˜ M. A. 2004. Detecting network communi- Total time Clustering time ● ties: a new systematic and efficient algorithm. Journal of Statistical =0.5
α ● Mechanics: Theory and Experiment. ●
● ● =1.0 Dumitru, H.; Gibiec, M.; Hariri, N.; Cleland-Huang, J.; Mobasher, α ●
● Memory (MB) ● B.; Castro-Herrera, C.; and Mirakhorli, M. On-demand feature rec- ● ●
Running Time (sec.) ● ● ● ● ● ommendations derived from mining public product descriptions. In ● ● ● α =0.0 ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ICSE’11, 181–190. ● ● ● 0 2000 4000 6000 8000 10000 100 200 300 400 500 20 40 60 80 20 40 60 80 Foltz, P. W.; Kintsch, W.; and Landauer, T. K. 1998. The Measure- KLOC KLOC ment of Textual Coherence with Latent Semantic Analysis. Dis- ● ●
● course Processes 25(2-3):285–307.
=0.5 α Gilbert, E. N. 1959. Random graphs. Ann. Math. Statist. ● ● ● ● 30(4):1141–1144. =1.0 α ●
● Memory (MB) Kaufman, L., and Rousseeuw, P. J. 1990. Finding Groups in Data: ● ● ●
Running Time (sec.) ● ● ● ● ● α An Introduction to Cluster Analysis. John Wiley. =0.0 ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● 0 2000 4000 6000 8000 10000
100 200 300 400 500 Kuhn, A.; Ducasse, S.; and Gˆırba, T. 2005. Enriching reverse 0.5 1.0 1.5 2.0 0.5 1.0 1.5 2.0 engineering with semantic clustering. In 12th Working Conference KUnit KUnit on Reverse Engineering,2005, 133–142. ● ●
● Kuhn, A.; Ducasse, S.; and Gˆırba, T. 2007. Semantic clustering: Identifying topics in source code. Information and Software Tech- ● ●
● ● α =0.5 nology 49(3):230–243.
● α =1.0 ●
Memory (MB) Linstead, E.; Rigor, P.; Bajracharya, S.; Lopes, C.; and Baldi, P. ● ● ●
Running Time (sec.) ● ● ● 2007. Mining concepts from code with probabilistic topic models. ● ● α =0.0 ● ● ● ● ● ● ● ● ● ● ● ● In Proc. of Automated Software Eng., 461. ● ● ● ● ● ●●● ● ● ● ● ● ● ● ●● 0 2000 4000 6000 8000 10000 100 200 300 400 500 1.0 1.5 2.0 2.5 3.0 1.0 1.5 2.0 2.5 3.0 Manning, C. D.; Raghavan, P.; and Schutze,¨ H. 2008. Introduc- Dictionary Size (KWords) Dictionary Size (KWords) tion to Information Retrieval. New York, NY, USA: Cambridge
● α ●
=0.5 University Press. ● α
=1.0 Marcus, A., and Haiduc, S. 2013. Text Retrieval Approaches for ● ● ● ● Concept Location in Source Code. Berlin, Heidelberg: Springer
● Berlin Heidelberg. 126–158. ● Memory (MB) α ● ●● =0.0 Running Time (sec.) ● ● ● McBurney, P. W.; Liu, C.; and McMillan, C. 2016. Automated fea- ●● ● ● ●● ● ● ● ● ture discovery via sentence selection and source code summariza- ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ●● ●● 0 2000 4000 6000 8000 10000 100 200 300 400 500 tion. Journal of Software: Evolution and Process 28(2):120–145. 0.005 0.010 0.015 0.020 0.005 0.010 0.015 0.020 CG density ρ CG density ρ McMillan, C.; Hariri, N.; Poshyvanyk, D.; and Cleland-Huang, J. 2012. Recommending Source Code for Use in Rapid Software Figure 3: Correlation between running time and memory us- Prototypes. In Proc. of Int. Conf. in Soft. Engineering (ICSE’12), age w.r.t. LOC, number of units, size of the dictionary and, 848–858. density of CG Mitchell, B. S. 2003. A heuristic approach to solving the software clustering problem. In 19th International Conference on Software Maintenance,2003, 285–288. Our plans for the deployment of FEAT include Software Rubin, J., and Chechik, M. 2013. A survey of feature location Heritage4(SH). SH is a public, large archive counting al- techniques. Domain Engineering 29–58. most 65M open-source software projects, we plan to inte- Shtern, M., and Tzerpos, V. 2012. Clustering Methodologies for grate FEAT with SH’s platform in order to provide a more Software Engineering. Advances in Software Engineering 2012:1– abstract approach for searching software artefacts, based on 18. program topoi, allowing the usage of natural text in queries. Siddharthan, A. 2002. Christopher d. manning and hinrich schutze. Foundations of Statistical Natural Language Processing. References MIT press, 2000. ISBN 0-262-13360-1, 620 pp. $64.95/£44.95 (cloth). Natural Language Engineering 8(1):91–92. Aaron, C.; Newman, M.; and Moore, C. 2004. Finding community Tzerpos, V., and Holt, R. C. 2000. ACDC: an algorithm for structure in very large networks. Physical Reviews E. 70. comprehension-driven clustering. In Proceedings of the Seventh Abebe, S. L., and Tonella, P. 2015. Extraction of domain concepts Working Conference on Reverse Engineering, 2000, 258–267. from the source code. Science of Computer Programming 98:680– Wiggerts, T. A. 1997. Using clustering algorithms in legacy sys- 706. tems remodularization. In 4th Working Conference on Reverse En- Andritsos, P., and Tzerpos, V. 2005. Information-theoretic soft. gineering, 1997, 33–43. clustering. IEEE Trans. Software Eng. 31(2):150–165. Xu, X.; Lung, C.; Zaman, M.; and Srinivasan, A. 2004. Program Chen, K., and Rajlich, V. 2000. Case study of feature location restructuring through clustering techniques. In 4th IEEE Interna- using dependence graph. In Program Comprehension, 2000. Pro- tional Workshop on Source Code Analysis and Manipulation, 2004, ceedings. 8th International Workshop, 241–247. 75–84.
4www.softwareheritage.org
7778