arXiv:cond-mat/0305575v2 [cond-mat.stat-mech] 13 Oct 2003 ieetfo te nierdsses n lsrin closer systems. and biological evolving systems, to ways engineered some other somewhat from systems em- software different an substantial makes evolvability as This on specificity phasis parameter. interfacial controlling target important that collab- and designs collective evolv- orative through with implemented evolvable, often highly once ability at and be functional to is highly organized important are both systems Especially software in that networks. fact perspectives the complex novel of some study suggest our- also the find they now But we centerpiece which selves. the in world as are right, information-based systems own the Software of their in system. important course form a those that of of graphs of description or networks skeletal collaborations define a to and used be interactions can pieces the levels of and libraries, many out files, etc.), at source up classes, subsystems built (subroutines, granularity is and of Software units interacting field. this many in relatively received attention have little date to which networks, complex en- changing of face 11]. the [10, in vironments adaptable and the and evolve, unravel robust form, to remain systems networked aims which which by networks”, principles sci- “complex to emerging of sim- an served into ence in have fields disparate discoveries found many These together those draw from networks. different random qualities, rather found ple “small-world” been be recently and can have “scale-free” which – many pathways [9] share inter- organisms metabolic to of 7], and variety [8], [6, a 3], yeast cinema in in [2, and proteins Web 5] of Wide [4, actions World science the – in [1], networks collaborations Internet these the of structures including and underlying grow Sur- the systems prisingly, organizations. networked complex intricately reveal society, to evolve human of artifacts eini eta lmn fsfwr construction, software of element central a is Design of class important another represent systems Software engineered the and nature of forms organic the both In tutr,fnto,adeovblt fsfwr collabo software of evolvability and function, structure, ASnmes 97.b 92.f 87.80.Vt 89.20.Ff, 89.75.Fb, findings. numbers: these PACS of wake robu the the network in of about discussed features questions are salient for ganization the design softwa of of object-oriented some model of simple captures a which d present and processes also topologies, I technological network practices. these other gineering of in measures identified several those foun present soft have to examined and similar have systems, I networks software networks, open-source an complex several of methods, within study classes, the in objects, vances modules, collaborating many otaesseseeg rmmr esrkst omintric form to keystrokes mere from emerge systems Software .INTRODUCTION I. onl hoyCne,Roe al onl nvriy It University, Cornell Hall, Rhodes Center, Theory Cornell otaessesa ope networks: complex as systems Software hitpe .Myers R. Christopher hs nomtoa tutrsmyntb sraiyap- readily as be networks, parent. not complex may of structures classes informational other whose forces organize the understand to us serve help that to tools mod- and provide metaphors, may els, systems software evolvable orga- of functional computing nization the adap- understanding autonomous broadly, in More of arise agents. systems that teams, self-organizing computations coupled emergent tive, loosely for also distributed, applica- but for by only developed not tions important increasingly software the is in Understanding form networks that design. organizations explicit structural lie large-scale of structures realm whose the outside organizations software emergent implementation, leads to But of adaptation and time. action extension, a composition, persistent at refactoring, and components of combined set the small a the addressing of the often interactions with most networks, explicitly software deal of structure methodologies design many and eto Ipoie summary. a biology. provides systems sug- VI and Section object- issues networks, systems, complex few software design, a among oriented ob- synergies highlights the the V by of of Section gested features model essential systems. simple the served of a some in present processes captures refactoring I discussed on which IV, based is systems section software others In evolving those by of III. work implications section Related engineering software observations. the of discusses and some networks, of complex connection studied structure recently their other the to describes examines networks, II collaboration those Section open-source existing systems. several and software from graphs graphs collaboration ex- call networks class subroutine collaboration specifically the here, of amined nature in the describes detail and design, more software of aspects relevant some h eane fti eto rvdsa vriwof overview an provides section this of remainder The oilgcl n ilgclsses I systems. biological and sociological, , esse vlto ae nrefactoring on based evolution system re tes vlaiiy eeeay n or- and degeneracy, evolvability, stness, hmt eelsaefe,small-world scale-free, reveal to them d susterrltosi osfwr en- software to relationship their iscuss uruie.Bidn nrcn ad- recent on Building subroutines. d bevdsses oeimplications Some systems. observed aeclaoaingah contained graphs collaboration ware t ucinlntok connecting networks functional ate aa Y14853 NY haca, aingraphs ration 2

A. Collaboration in software systems: function and disruption to the system as a whole. The need to ac- evolvability commodate change is a major driving force in software engineering; this might include changes in external user Software engineering aims to decompose complex com- requirements, underlying hardware platforms, forms of putational functionality into many separate but inter- input data, or types of algorithms used. In poorly struc- locking pieces. Rather than reimplementing similar com- tured code, modifying or adding a single feature may putations explicitly in every new context, programmers require updates to many files or subroutines, which can develop abstractions of functions and data types that can themselves then cascade throughout the system. To com- be used many times, in multiple contexts. The ability to bat this, many strategies have been developed to support reuse existing code can both speed the process of adding the simultaneous demands of function and evolvability, new functionality (since new code need not be written) so that code modifications remain localized. Many of and facilitate the process of modifying existing function- these strategies hinge on instituting sufficient decoupling ality (since changes can be made to a single reused piece between subsystems, which enables developers to avoid of code, rather than to multiple similar versions of code constraints and commitments that make it difficult to strewn throughout a large application). The process of change one part of a system without changing many oth- computation, therefore, involves collaboration: the dis- ers. tribution of responsibility for computation among multi- Design patterns [12], for example, are an important ple software elements, such as objects, classes, methods, class of strategies to support software system evolvabil- subroutines, modules, and components. These collabo- ity. These patterns are motifs describing the relation- rations allow elaborate computational tasks to be built ships among collaborating classes or objects in an object- up in a modular and hierarchical fashion, in loose anal- oriented (OO) software system which are effective at en- ogy with the way that sophisticated electronic circuits capsulating variability and change. Different patterns are built up from reusable, low-level components. identify different aspects of a system which are likely to Software collaborations imply dependency relation- change; those aspects might include how objects are cre- ships, in that some computational elements (e.g., classes ated, which objects need to communicate with one an- and subroutines) need others in order to carry out pieces other, or what particular algorithms are used to solve a of their appointed task. One goal of software design problem. The highly variable elements of a system are and development, therefore, is to construct an opti- then encapsulated behind generic interfaces or dedicated mal or near-optimal system of dependency relationships, objects that act as brokers to mediate computational ac- whereby core elements are reused in different contexts tivity, thereby decoupling objects so as to avoid excess de- to perform recurring fundamental tasks, with minimally pendencies that can inhibit evolvability. Design patterns constraining specializations added in higher functional and related techniques are typically applied at a small layers in order to build upon or combine those fundamen- scale, at the microstructural level describing interactions tal tasks. The utility of minimal specialization (typically among a few objects. An open question, therefore, is in the types of data being passed between subroutines how microstructural design methodologies conspire in the and methods) is tied to the goal of code reuse: software large to form macroscopic software structures. units that function only under highly specialized condi- Evolution and evolvability are of course central con- tions are generally less able to be broadly reused than cepts in the description of biological systems, different in those units that require only as much specialization as important ways from their meaning in software engineer- is required to implement a needed computation. Many ing. While software engineering involves intelligent ac- factors can influence the decomposition that is actually tion, biological evolution is blind, and does not. Nonethe- chosen in a particular software project, however. For ex- less, from a systems perspective, there may be forms of ample, the process of subdividing computations into min- network organization which support adaptations that are imal units and generating complexity by combining those applicable to both blind biological evolution and intelli- pieces does incur some overhead in computational perfor- gent software design. Furthermore, the software engi- mance. In scientific computing, the need for high perfor- neering community is increasingly recognizing the value mance has historically outweighed other design concerns, of prototyping and other forms of interactive, trial-by- leading to more coarse-grained decompositions of data error design, in an effort to “embrace change” rather and functionality that enable efficient numerical process- than struggle against the rapid pace of software evolu- ing on large chunks of data unencumbered by indirection tion [13]. Many of the software design patterns men- and function calls. Ongoing research in the field of scien- tioned above, which are now codified and part of a devel- tific computing aims to develop techniques for incorpo- oper’s standard repertoire, were initially emergent and rating advanced software design methodologies without recurring solutions that developers uncovered in their incurring excessive penalties in performance. quest to build flexible, reusable code that could operate Building up software to carry out complex tasks is only in rapidly changing environments. one goal, however. The resulting system must also be The distributed and collaborative nature of software evolvable, that is, transformable into a new system to ac- design is increasingly relevant at the social level of pro- complish new but related tasks without excessive cost or grammer interaction, as well. Many software projects 3

class A { begin as small efforts led by one or a few people, only to A grow into large activities involving many developers scat- // definition of class A tered around the globe, a transition that has been dubbed }; B A “from the cathedral to the bazaar” [14]. While organized class B { B design methodologies that support software growth and A* ab; change are useful even in the smallest projects, they be- // rest of definition of class B come especially important in distributed multi-developer C A }; efforts where many individuals may work only on small C B pieces of the overall system. The work presented here class C { C does not explicitly examine the effects of transformations A* ac; in the social infrastructure of developers, but examina- B* bc; tions of that sort could prove quite interesting. // rest of definition of class C D A }; D C class D: public C { B. Software collaboration graphs A* ad; D // rest of definition of class D The interactions among software components are mul- }; tidimensional and multifaceted, and any representation of a software system typically involves a slice or projec- FIG. 1: Sample class collaboration graph (right) representing tion through that complex space of interactions. Call relationship among classes A, B, C and D, as specified by graphs that describe the calling of subroutines or meth- C++ class definitions (left). Graph nodes are classes, and ods by one another have long been used to understand graph edges are directed collaboration relationships between the structure and execution of procedural activity in soft- classes. This definition of class collaboration involves both ware systems, while class and/or object collaboration di- aggregation relationships (B→A, C→A, C→B, and D→A) agrams are used to glean insight into the relationships and inheritance/subclassing relationships (D→C). among abstract data types in OO systems. Static graphs typically describe the set of interactions that are pos- sible, while dynamic graphs generated during program execution identify interactions that actually take place where one class is defined as a subclass of another – and under specific run-time conditions. This work addresses through aggregation – i.e., where one class is defined to static class collaboration graphs arising in OO systems hold an instance of another class. A simple illustrative and static call graphs arising in procedural systems, both example of such a graph is shown in Figure 1. The direc- of which can be parsed from source code. tion of class collaboration and subroutine calling follows standard software engineering convention reflecting the In OO systems, the definition of objects and their inter- flow of control in a system: an edge in a class collab- actions plays a central role. Objects represent data types oration graph is directed from class B to class A if B that are defined to extend the basic, native data types makes reference to A in its definition (either through in- provided by programming languages (integers, floating heritance or aggregation), and an edge in a call graph point numbers, characters, etc.), in order to develop more points from node g to f if subroutine g calls subroutine complex, application-specific, abstractions of data and f from within its scope. The definition of class collabo- their associated behaviors (which are implemented via ration used here allows one to decompose the full class methods). Typically, layers of objects are defined, build- collaboration graph into two separate subgraphs, the in- ing increasingly complex representations by aggregating heritance graph and the aggregation graph. simpler ones. An object representing a vector field, for example, may combine simpler objects representing vec- I have examined collaboration networks associated tors and spatial fields, which can themselves be defined with six different open-source software systems. These independently of each other. (A vector object, for exam- include class collaboration graphs for three OO sys- ple, might provide support for adding two vectors and tems written in C++, and call graphs for three pro- computing their dot product, whereas a spatial field ob- cedural systems written in C. The class collaboration ject might support a coordinate-based lookup to retrieve graphs are from: version 4.0 of the VTK visualization the value of the field at a specified location. Both vec- library [15]; the CVS snapshot dated 4/3/2002 of Digital tors and spatial fields might themselves be built up from Material (DM), a library for atomistic simulation of ma- even simpler objects, such as arrays.) Classes describe terials [16, 17]; and version 1.0.2 of the AbiWord word the form of objects in OO systems, and objects are in- processing program [18]. The call graphs are from: ver- stantiated at run time from their class descriptions. sion 2.4.19 of the kernel of the Linux operating system, Class collaboration is the process by which more com- version 3.23.32 of the MySQL relational database system, plex, multifunctional classes are built from simpler ones. and version 1.2.7 of the XMMS multimedia system. De- In this work, class collaboration is defined to include tails on the construction and/or origin of these networks the interaction of classes both through inheritance – i.e., is provided in the Appendix. 4

VTK DM AbiWord Linux MySQL XMMS (a) 788 187 1096 5420 1501 1097 (b) 1389 278 1857 11460 4245 1901 (c) 6 10 19 47 10 36 (d) 771 162 1035 5285 1480 971 (e) 1374 258 1798 11370 4231 1809 (f) 4 2 46 10 12 0 (g) 5 6 25 6 7 0 (h) 8 10 72 9 10 0 (i) 0.0165 0.0428 0.1332 0.0057 0.02 0.0

TABLE I: Connected component analysis for the six software systems of interest. Shown for each graph are: (a) number of nodes in each graph, (b) number of edges in each graph, (c) number of WCCs, (d) number of nodes in the largest WCC, (e) number of edges in the largest WCC, (f) number of SCCs, (g) number of nodes in the largest SCC, (h) number of edges in the largest SCC, and (i) the fraction of nodes belonging to any SCC. All six systems are characterized by a single dominant WCC, and little membership in SCCs.

II. RESULTS

A. Connected components

Connected components in a graph are those subgraphs for which all nodes in the subgraph are mutually reach- able by traversing edges in the subgraph. For a directed graph, one can define both weakly connected compo- nents (WCC) and strongly connected components (SCC). WCC are those connected components found in an undi- rected version of the graph (i.e., by treating all edges FIG. 2: Largest weakly connected component of the class as bi-directional), while SCC are those connected com- collaboration graph for the VTK system (layout courtesy of ponents mutually reachable by traversing directed edges. the Tulip graph visualization package). By definition, every node in a directed graph will be in some WCC; not all nodes, however, belong to an SCC. Connected component analysis reveals trends across since that would undermine the genericity inherent in the six systems of interest, which are summarized in those base classes. Similarly, one subroutine may call an- Table I. All six systems consist of a single dominant other as part of its execution, but typically, the reverse WCC, comprising a large fraction of the total nodes in is not true. the system (ranging from 86-99%), and a few (5-46) very small remaining WCCs. A picture of the largest WCC for the VTK system is shown in Figure 2. Conversely, B. Degree distributions few nodes belong to any SCC (less than about 4% in five of the six systems). The lack of strong membership Degree distributions, summarizing the connectivity of in SCCs is rather different from that found in other di- each node in a graph, are a feature that distinguish many rected complex networks, such as the WWW [3] and var- complex networks from simple random graphs. For each ious metabolic networks [9]. This difference is perhaps node i in a directed graph, there is both an in-degree in not surprising, given the nature of the software graphs ki , the number of incoming edges to node i, and an out- out under investigation, which largely reflect aggregation (of degree ki , the number of outgoing edges from node i. data in the case of the class collaboration graphs, or func- The in- and out-degree distributions P in(k) and P out(k) tion in the case of call graphs). SCCs reflect subgraphs indicate the probability of finding a node with a specified that are mutually reachable, but the hierarchical direc- in-degree or out-degree k, respectively, in a given graph. tionality of use implicit in software aggregation makes Many complex networks have recently been found to pos- mutually reachable clusters of this sort unlikely, and un- sess a “scale-free” degree distribution [19], indicating a desirable from a software development standpoint. While lack of characteristic scale (or degree) in the distribu- generic base classes are reachable from their derived sub- tion P (k). This implies that P (k) obeys a power law classes, for example, the opposite is generally not true, over an extended range of degrees k: P (k) ∼ k−γ , or 5

3 in (1.9) in (2.0) in (2.2) admittedly small, particularly for the out-degree distri- out (3.1) out (3.3) out (2.8) butions, which one might argue to be representative of exponential distributions. Power-law fits for all 12 distri- butions have been carried out over the regions for which 2 they exhibit scaling, and the values of the exponents γin

(k) out c and γ are shown in the legends in Fig. 3. N

10 Interestingly, the class collaboration graphs shown in

log Fig. 3 reveal a marked asymmetry between the in-degree 1 and out-degree distributions, whereas the call graphs do not. In the class collaboration graphs, the out-degree ex- (a) (b) (c) ponent appears to be significantly larger (γout ≈ 3) than the in-degree exponent (γin ≈ 2). For the procedural call 0 0 1 2 0 1 2 0 1 2 in out 4 graphs, γ ≈ γ ≈ 2.5. For both sets of graphs, the in (2.4) in (2.5) in (2.5) in-degree distributions tend to extend to higher k; that out (2.4) out (2.4) out (2.5) is, it is more likely to find a node with many incoming links than outgoing links. 3 As noted, the three class collaboration graphs also each contain an embedded inheritance graph. Since multiple (k)

c inheritance is commonly avoided (due to programming

N 2

10 difficulties that it introduces), the out-degree distribu-

log tions of the inheritance graphs are strongly peaked at k = 0 and k = 1. The in-degree distributions for each 1 of the three inheritance graphs, on the other hand, also exhibit rough power-law scaling, with exponents γ ≈ 2 (not shown). Therefore, insofar as the in-degree distri- (d) (e) (f) 0 butions are concerned, the structural forms of the overall 0 1 2 3 0 1 2 3 0 1 2 collaboration graphs mirror those of the embedded inher- log k log k log k 10 10 10 itance graphs. The heavy tail in the out-degree distribu- tion, however, arises entirely from non-inheritance-based FIG. 3: Cumulative in-degree and out-degree distributions associations of classes (such as the relationships B→A, Nc(k) for each of the six systems: (a) VTK, (b) Digital Mate- C→A, C→B, and D→A in Fig. 1). rial, (c) AbiWord, (d) Linux, (e) MySQL, (f) XMMS. Nc(k) is the number of nodes in a graph with degree greater than or Classes and subroutines with small out-degree are gen- equal to k. Histogram data for each distribution are shown erally simple, since they do not aggregate other elements. with symbols. Lines indicate power-law fits (straight lines on (If they became too complex, there would be a pressure to log-log scales) to histogram data in regions indicated; fits to break them up into simpler pieces and introduce outgo- in-degree distributions are represented by solid lines, and fits ing links.) Conversely, elements with large out-degree are to out-degree distributions by dashed lines. The legends show generally more complex because they aggregate behavior in out the values of the power-law exponents γ and γ for each from many others. Therefore, the existence of heavy- −γ+1 fit (where Nc(k) ∼ k ). tailed and/or scale-free out-degree distributions suggests a broad spectrum of complexities. On the other hand, classes and subroutines with large in-degree are – by def- perhaps a power law truncated by an exponential cutoff: inition – reused in many contexts, while those with small P (k) ∼ k−γe−k/kc . In contrast, a uniform random graph in-degree are not. Thus, the existence of heavy-tailed in- of N nodes and k¯ links on average per node has a de- degree distributions implies a broad spectrum of reuse. gree distribution with a characteristic scale k¯, with P (k) Less interesting, perhaps, are those many classes and sub- decaying exponentially away from k¯ [20]. routines that are neither heavily reused, nor heavily con- I have examined the in- and out-degree distributions structed from other elements. of the large dominant WCC for each of the six software Software engineering practice encourages reuse, that systems of interest. For each WCC, I have computed the is, large in-degree; so it is not surprising that the largest in unnormalized cumulative frequency distributions Nc (k) degrees in these systems are for incoming links. It is not out and Nc (k), where Nc(k) indicates the number of nodes obvious, however, why the class in- and out-degree distri- in a graph with degree greater than or equal to k, and butions should be characterized by quantitatively differ- have plotted the logarithms of these distributions in Fig- ent scaling exponents. The fact that the procedural call ure 3. (Nc(k) is an unnormalized integral of the probabil- graphs examined do not exhibit this in-out exponent in- −γ −γ+1 ity distribution P (k); for P (k) ∼ k , Nc(k) ∼ k .) equality also suggests further avenues of study: first and These distributions reveal a power-law scaling region foremost would be an examination of the call graphs as- (straight line on a log-log plot), followed by a faster de- sociated with the three OO systems studied, to ascertain cay at large k. The extents of the power-law regions are whether in-out asymmetry is a property of class collabo- 6

100 40 100 (a) (b) (c) tion between in- and out-degree for nodes with large de- gree in five of the six data sets. Including the entire set vtkObject 80 DMArray 80 of nodes for each data set (i.e., not selecting only those 30 nodes with k ≥ 10) reveals, however, almost no correla- 60 vtk- 60 tion in the call graph data (Linux, MySQL, XMMS), and Unstructured- weak positive correlation in the class collaboration data in Grid 20 k (VTK, DM, AbiWord). This anticorrelation of large in- 40 40 and out-degree implies that, for the most part, there is a

10 clear distinction between large-scale producers of infor- 20 20 mation (with high in-degree) and large-scale consumers (with high out-degree). Simple components tend to be

0 0 0 heavily reused, presumably because they are generic and 0 5 10 15 20 0 5 10 15 20 0 5 10 15 20 applicable to many different contexts, whereas complex 250 250 40 components do not, because they are highly specialized (d) (e) (f) and only applicable in limited contexts. 200 200 There are, however, outliers to that trend of separa- 30 tion; classes with both large in-degree and large out-

150 150 degree are evident in Fig. 4(a)-(c). These classes have both significant internal complexity (associated with ag- in 20 k gregating the behavior of several other classes), and sig- 100 100 nificant external responsibility. There is reason to expect that such classes could be problematic insofar as software 10 50 50 development is concerned. By way of informal case study, my experience with development of the Digital Material

0 0 0 (DM) system (Fig. 4(b)) confirms such a suspicion. The 0 5 10 15 20 25 30 0 10 20 30 40 50 0 10 20 30 40 DMArray class identified implements both array-like and k k k out out out tree-like functionalities, and is intended to be a primitive data structure for much of the numerical computation FIG. 4: Scatter plot of the number of incoming links versus in the system. Curiously, DMArray caused the most per- the number of outgoing links, for every node in each of the six sistent trouble within the DM development group. The systems under consideration: (a) VTK, (b) Digital Material, source of the difficulty largely centered around the con- (c) AbiWord, (d) Linux, (e) MySQL, and (f) XMMS. flicting roles that DMArray plays. On the one hand, the class is intended to be a primitive black box onto which more complicated functionality is to be layered, while at ration or of object-oriented systems more generally. the same time, it is itself a complicated datatype with substantial internal structure and external behavior. It would be interesting to know whether other items with C. Degree correlations large in- and out-degrees apparent in Fig. 4 have played similarly problematic roles in the development of those Correlations among degrees can also provide insight various systems. The complexity of software components into the structure of complex networks. The directedness with large in- and out-degrees is highlighted in the soft- of these graphs allows us to examine the relationship be- ware metrics literature, for example, by the “fan-in fan- tween in-degrees and out-degrees. Figure 4 shows this out complexity” metric [21], which states that complex- relationship for each of the six systems of interest, where ity of a code module is proportional to the square of the every node in each graph is represented by its (kout, kin) product of the fan-in (in-degree) and the fan-out (out- pair. degree) of the module. It is visually apparent in Fig. 4 that nodes with large Another measure of degree correlation is the mixing by out-degree generally have small in-degree, and those with degree of a graph [22]. This quantity measures the linear large in-degree have small out-degree. This trend can correlation of degrees over all edges of a graph, i.e., the be verified by evaluating the linear (Pearson) correlation correlation of degrees ki and kj for all sites i and j that coefficient between the sets {kin} and {kout} for those define an edge in a given graph, aggregated over all edges nodes with either a large in-degree or large out-degree in that graph. This reflects the tendency of nodes of sim- (or both); a threshold value of at least ten edges has ilar degree to be connected to one another. Most work been chosen as a filter, somewhat arbitrarily as a de- on mixing by degree has focused on undirected graphs or marcation between the dense core of low-degree nodes undirected versions of directed graphs, for which there and the sparser set of high-degree nodes. Table II(a) is a single correlation coefficient of interest: corr(ki, kj ), shows the correlation coefficient for each of the six soft- relating ki and kj for the node pair (i, j) that are linked ware systems studied, demonstrating a negative correla- by an edge in the graph. In a directed graph, there 7

VTK DM Abi Linux MySQL XMMS sumers described above. These results further emphasize (a) In/out that the directedness of these graphs is important. New- k ≥ 10 -0.48 0.01 -0.16 -0.18 -0.23 -0.75 man [22] has noted that sociological networks (e.g., scien- all k 0.09 0.10 0.18 -0.01 -0.03 -0.07 tific collaborations) tend to be assortative (positively cor- (b) Mixing related), but that technological and biological networks in – in 0.088 -0.043 0.065 -0.005 0.114 0.067 are generally disassortative. Clearly, for these software in – out -0.034 -0.010 0.083 -0.009 -0.067 -0.036 graphs, teasing out the degree correlations requires ex- out – in -0.169 0.020 0.042 -0.098 -0.101 -0.180 amining directed edges. out – out 0.137 0.098 0.111 0.014 0.179 0.093 undirected -0.194 -0.192 -0.084 -0.067 -0.083 -0.114 D. Clustering and hierarchical organization TABLE II: Degree correlation coefficients. (a) For each of the six software systems, correlation of kin and kout for each Clustering – the tendency of a node’s neighbors to graph node, for both the full data sets (“all k”), and for the re- be themselves neighbors in the graph – is a significant in out duced sets with k ≥ 10 or k ≥ 10. (b) For each of the six characteristic of small-world networks. While the results software systems, degree mixing coefficients, relating in- and above indicated the importance of graph directedness, out-degrees for each edge in the graphs. The left column in- clustering is typically measured on undirected graphs. dicates the specific directed degree correlation function. Also included is the correlation coefficient for an undirected version For such a graph, the clustering coefficient Ci of node i of each graph. is defined as Ci =2n/ki(ki − 1), where n is the number of pairs of neighbors of node i that are linked, and ki is the degree of node i. This quantity is simply the frac- tion of all possible pairs of neighbors of node i that are out out are four possible correlation coefficients: corr(ki , kj ), themselves linked in the graph. out in in out in in corr(ki , kj ), corr(ki , kj ), and corr(ki , kj ), where Recent work by Ravasz et al. [23, 24] suggests that the index i refers to the source node of the directed edge, degree-dependent clustering of the form C(k) ∼ k−1 and j refers to the destination node. Table II(b) shows is a signature of hierarchical organization in networks, the values of each of these correlation coefficients, for and can serve to distinguish hierarchical from non- each of the six graphs in question. Also computed is hierarchical scale-free networks. They also suggest that the degree mixing of the undirected version of each soft- hierarchical organization serves to resolve the apparent ware graph. The undirected versions all show a weak dilemma between scale-free degree distributions on the negative correlation (dissortativity) which suggests that one hand (which imply no characteristic scale of connec- nodes with similar degrees tend not to be connected to tivity) and modular structure on the other (which sug- each other. The directed graphs, however, tell a different gest connectivities at the scale of those consistent with story. There, we find – most significantly – a weak pos- modular units). Figure 5 plots, for each of the six soft- itive correlation (assortativity) among out-degrees, that ware graphs of interest, the degree-dependent clustering is, a tendency for nodes with similar out-degree to be C(k), defined as the average clustering C for nodes in the connected. There is a yet weaker positive correlation undirected graph with degree k. These data are roughly among in-degrees. While these correlations are rather consistent with those presented in refs. [23, 24], typically weak, their magnitudes are typical of those seen in a va- showing a flat C(k) for small k which rolls over to a k−1 riety of other complex networks [22]. tail at large k (more clearly defined for the larger graphs). −1 The weak positive assortativity among out-degrees The k tail is derived in refs. [23, 24] for a specifically seen in the software networks is due in part to the hierar- constructed hierarchical model; therefore, the existence −1 chical layering of functionality. For example, in the VTK of k scaling in real graphs would appear to be an indi- system, a complex aggregated class with large out-degree rect indicator of hierarchical organization. Nonetheless, such as the vtkUnstructuredGrid does not collaborate given the hierarchical nature of software design, further directly with very low-level objects (such as vtkObject); investigation of this sort of clustering in software graphs instead, it is built up out of collaborations with “mid- is warranted. Methods for extracting modules and sub- level” objects (such as vtkPolygon and vtkHexahedron), systems using the clustering data may also provide in- which themselves are aggregates built up from lower- sight into the organization of these systems [23]. level classes. Similarly, the weaker positive assortativ- ity among in-degrees for some of the graphs probably reflects the tendency for simpler objects with large in- E. Topology, complexity, and evolution degree to collaborate with each other at the base of a hierarchy of functional layers. In the undirected versions Software systems can be characterized by a variety of of these graphs, the negative correlation observed reflects metrics, which can be compared to the underlying col- in part the fact that nodes with a large out-degree are laboration network topology. The VTK system, in par- not linked to the nodes with a large in-degree, because ticular, has been developed in a manner that facilitates of the functional separation between producers and con- such measurements. For every class in the VTK system, 8

1 correlation with out-degree, and a weaker, negative cor- relation with in-degree. Each of these metrics reflects a different facet of class complexity (implementation size, interface size, revision rate), and we see that nodes with 0.1 large out-degrees tend to be more complex than those with large in-degrees, consistent with the scenario out- lined previously. C(k) The evolution of complex networks is a problem of 0.01 great interest, and the class revision history data for the VTK system provide some insight into the evolutionary processes of software development, which are summa- (a) (b) (c) rized in Figure 6. Figure 6(a) shows a comparison of the 0.001 collaboration graph degree distributions for the current 1 VTK system (circa 2002) with those for the system in its nascent state, at the end of January 1994 (the VTK “reptile brain”, so to speak). We see that the heavy- tailed collaborative structure of the system was in place from the outset, although the in-out asymmetry was less 0.1 pronounced. Figure 6(b) examines the coevolution of classes that collaborate in the VTK system. For every

C(k) edge in the VTK graph, the revision rates of the outgo- ing source node and the incoming destination node are 0.01 plotted against each other. From these data, we find an interesting, and perhaps unexpected, trend. For the en- tire dataset, there is a weak positive correlation among (d) (e) (f) revision rates for connected classes (correlation = 0.10), 0.001 1 10 100 1000 1 10 100 1000 1 10 100 1000 indicating a weak tendency for collaborating classes to k k k evolve at the same rate. But if we restrict our focus to only those classes with large revision rate, e.g., greater FIG. 5: Clustering coefficient C(k) vs. degree k for undi- than 30 revisions per year on average, we find a strong rected versions of each of the six software systems. C(k) is negative correlation (correlation = -0.72). This implies the average clustering C for all nodes in a graph with degree k. that classes that evolve most quickly tend not to interact −1 Also shown for each graph is k scaling (dotted line), sug- directly with each other. gestive of hierarchical organization. Graphs are: (a) VTK, (b) Digital Material, (c) AbiWord, (d) Linux, (e) MySQL, (f) The data in Table III reveal that, in the VTK system, XMMS. classes with large out-degree tend to evolve more rapidly than do classes with large in-degree. On the one hand, VTK class complexity measures in out one might imagine that the information consumers with large out-degree evolve more rapidly simply because they source file size -0.28 0.58 have greater implementation sizes and a larger number number of methods -0.26 0.61 average revision rate -0.28 0.68 of methods subject to revision. Alternatively, one might imagine that evolution rate is primarily governed by the TABLE III: For VTK only, correlation of various class metrics nature of connections to other classes: information pro- (source file size, number of methods, average revision rate) ducers with large in-degree are constrained to remain sta- with in- and out-degrees. tionary (since other classes depend on them), while infor- mation consumers are generally forced to evolve to keep pace with changes in all the other classes that they ag- I have calculated three quantities of interest: (a) total gregate but are generally unconstrained by large numbers source file size for each class; (b) total number of meth- of users. This latter scenario would suggest signatures of ods defined for each class (including inherited methods), coevolution, that is, correlation of evolution rates that and (c) average revision rate for each class over the life- are connected in the collaboration graph. time of the VTK project (average number of source file We saw in Fig. 6(b) that there was a weak coevolu- revisions per year, since initial commitment to the VTK tionary tendency on average for the entire VTK system, CVS source code repository). Details on these calcula- but a strong anticorrelation of coevolution among rapidly tions are presented in the Appendix. evolving classes. The rapidly evolving classes are pri- For each class, these three metrics can be related to marily (as seen in Table III) information consumers with the in- and out-degrees of that class; the linear correla- large out-degree, which are part of specialized functional tion of these metrics with degree is reported in Table III. subsystems rather than the more generic functional sub- We see that all three metrics have a strong and positive strate. The strong anticorrelation of interactions among 9

3 III. RELATED WORK in (2002) 2.5 out (2002) The fields of software metrics and reverse engineering in (Jan 1994) examine software systems in the aggregate, with signif- out (Jan 1994) 2 icant emphasis on measures of nodal degree in various (k)

c software graphs. By and large, however, their scale-free N 1.5

10 nature appears to have escaped notice within those com-

log munities. Distributions of component connectivity are 1 often summarized in terms of means and variances, which are poor characterizations of scale-free distributions. The 0.5 asymmetry between large in-degree and large out-degree (a) has long been identified by software engineers as an im- 0 portant element of design, since large out-degree indi- 0 0.5 1 1.5 2 cates significant code reuse while large in-degree indicates log k 10 excessive object complexity [26]. 50 The existence of scale-free, small-world networks in software graphs has been noted recently by a few groups, 40 however. Valverde and coworkers [27, 28] have exam- ined class collaboration graphs derived from the Java Development Framework 1.2 (JDK1.2) and the UbiSoft 30 ProRally 2002 game system. They have noted scale-free degree distributions and larger-than-random clustering characteristic of small worlds. Their work does not dis- 20 tinguish, however, between incoming and outgoing links in the software networks they have studied; that is, they 10 have formed undirected graphs by ignoring edge direc-

revision rate for incoming node tionality. With undirected graphs, they could not detect (b) either the asymmetry between the in-degree and out- 0 0 10 20 30 40 50 degree exponents evident in Fig. 3, or the anticorrelation revision rate for outgoing node of large in-degree and large out-degree seen in Fig. 4. Perhaps more importantly, ignoring the directionality of FIG. 6: Facets of the evolutionary history of the VTK sys- edges leads to different sorts of conclusions about the im- tem. (a) Comparison of the degree distributions for the VTK plications of software engineering guidelines. They claim system in its nascent state (class collaboration graph as of that software design is predicated in part on the “rule of Jan. 31, 1994), and for VTK version 4.0 (as presented in avoiding hubs (classes with large number of dependen- Fig. 3(a)). (b) Scatter plot of class revision rates for VTK cies, that is, large degree)” [27], which does not recog- collaboration graph. Each point represents an edge in the nize that large out-degree and large in-degree have very graph; the x- and y- coordinates of the point are given by the average revision rates of the outgoing source and incoming different software engineering implications. Valverde et destination node, respectively, for that edge. Note the anti- al. [27], however, do propose an interesting scenario by correlation of revision rates for those classes with large rate which minimization of development costs might lead to (e.g., greater than 30 per year). an optimal trade-off between developing a small num- ber of large, expensive components with few intercon- nections and a large number of small, inexpensive com- ponents with many interconnections. They suggest that only sub-optimal solutions can be found in large, complex systems, leading to scale-free/small-world behavior; this is an interesting conjecture deserving further study, per- haps within the context of synthetic models of software highly evolving classes may thus reflect a degree of mod- systems, such as the one that I will introduce below. ularity within the system, that is, the functional separa- Potanin et al. [29] have examined the structure of ob- tion of different specialized subsystems. Whatever weak ject graphs, representing run-time snapshots of object coevolution there is appears to be confined to the more interactions in several OO programs. Object graphs are generic substrate. Further work is needed to explore the dynamic, run-time analogs of the static class collab- the relationships among evolution, connectivity, and con- oration graphs studied here. Potanin et al. observed straint. Similar explorations are taking place within biol- power-law in-degree and out-degree distributions, noting ogy, to explore the relationship between network connec- a tendency for in-degree exponents to be clustered near tivity (e.g., graphs of protein interactions) and evolution 2.5, and out-degree exponents near 3, somewhat like the rate [25]. trend that I have observed for class collaborations in this 10 work. They also note the strong separation of large in- tem evolvability might alter the topological structure of degree and large out-degree, similar to that presented software collaboration graphs. An intriguing framework here in Fig. 4. Ultimately, developing a theory of the for addressing such a question, and for generating models relationships between static class graphs and dynamic of evolving software systems, is the set of processes col- object graphs might prove useful to the software engi- lectively known as refactoring [31], which aim to remove neering community. “bad smells” from code that inhibit evolvability (e.g., ex- Wheeldon and Counsell [30] have identified power-law tendability, modifiability, maintainability, and readabil- relationships in several OO measures, including inher- ity). Refactoring tends to encourage the development of itance and aggregation graph degrees, and numbers of smaller, more concise, single-purpose fragments of code methods, fields, and constructors defined by classes in (classes, methods, and subroutines) that can be reused OO systems. They treat inheritance and aggregation in a wider range of contexts, as opposed to larger, mul- as two separate types of class collaboration (which they tipurpose pieces of code that often contain duplicated are), while I have chosen to define collaboration more program logic. Large methods and classes are often bro- broadly to include both. They fit power laws to the ken up into collections of smaller ones, with appropriate full range of their distribution data, even when there indirection from the former to the latter, leading to the are apparent transitions between scaling behaviors (e.g., creation of more nodes and more edges in the resulting from power-law to exponential), making comparison with software graphs. Duplicated pieces of code are extracted the present work difficult. The numerical values of the from multiple locations in the source code, and local- degree-distribution exponents they quote (γ ≈ 1) are ized in a single place to which other pieces of code re- rather different from those found here and in the work of fer. (In some instances, however, a developer may deem Valverde et al. [27] (γ ≈ 2−3), but it is not clear whether such indirection excessive and not worth the overhead; or not they have corrected for the exponential bin sizes in those cases, refactoring techniques would suggest the that they use to deal with sparse statistics in the tails of removal of nodes and/or the collapsing of hierarchies.) the distributions. Many refactoring techniques can be cast in the language Both Potanin et al. and Wheeldon and Counsell of optimization, by minimizing (or altogether removing) make reference to the process of preferable attachment bad smells that pervade software. as a mechanism for generating scale-free networks, as Motivated by the basic observation that refactoring outlined by Barab´asi and Albert (BA) [7]. Preferen- improves code evolvability by reorganizing its internal tial attachment was originally proposed to describe the structure, I have implemented a simple model of an growth of the World Wide Web, but seems less well- evolving software system, based on a few refactoring suited to describe the growth of software systems, al- techniques. The model is overly simplified, insofar as though such attachment is indirectly related in that low- its treatment of software systems and practices is con- specificity “hubs” will generally attract more incoming cerned, and can never replicate the detailed structures of links if they present broadly reusable abstractions. But real software systems, which are certainly history- and the BA preferential attachment model is acknowledged to project-dependent. Nevertheless, some coarse features of be incapable of generating hierarchical structure, which the observed software networks can be replicated with is clearly relevant for software design, and which is ev- the simple refactoring model, which points the way to- ident in power-law, degree-dependent clustering such as ward more sophisticated models of this sort, as well as seen here in Fig. 5. further empirical study of actual software systems under- Several features of the present research – including going refactoring. examinations of degree-dependent clustering, correlation In the model, binary strings of arbitrary length (i.e., of network topology with various class complexity mea- strings composed of 0’s and 1’s) serve as proxies for the sures, and the evolutionary history of class collaborations subroutines in call graphs and the classes in class collab- – have not been explored in the other works cited above. oration graphs. No attempt is made here to distinguish Also, the present work introduces – in the following sec- between call graphs and class collaboration graphs, so tion – a model of software evolution based on a set of I will generically refer to these binary strings as “func- standard practices that captures some of the salient fea- tions” or “strings”, and to the network of their inter- tures of the observed software networks. actions as a “call graph”. Aggregation of functions is achieved through concatenation of strings (represented here via the addition operator). Therefore, a larger, IV. A REFACTORING-BASED MODEL OF more complex, string can be built up from smaller, sim- SOFTWARE EVOLUTION pler strings: e.g., 0110110010010 = 01101100+ 10010 = (01101 + 100) + (100 + 10). Such a concatenation also As suggested earlier, software systems have a complex has a call-graph-based interpretation: the original par- structure not only to support the implementation of com- ent node (0110110010010) has a link to each of the child plicated functionality, but also to allow for low-cost evolv- nodes from which it is composed (01101100 and 10010), ability. It is an interesting question to ask, therefore, as do the second-generation parent nodes to their chil- how software engineering practices used to enhance sys- dren. One can think of the original parent node (before 11

0110110010010 0110110010010 is chosen at random, and is removed from the system, such that its one parent node is linked directly to its two 01101100 10010 01101100 child nodes. This specification is admittedly arbitrary, and could be further parameterized; but the general pur- 01101100 10 01101100 10 pose of such a process is to remove functions that do not (a) (b) represent broadly useful abstractions (i.e., are not used by many parent functions in different contexts) and that FIG. 7: Processes implemented in the simple refactoring do little more than to simply aggregate a small number model. (a) Long strings are subdivided into two substrings, of other functions (i.e., two). with links made to existing node strings so as to avoid node duplication. (b) One-off functions are removed to avoid ex- In the version of the model studied to date, the evolu- cessive indirection; in this example, function 10010 from part tion process is begun by constructing a call graph consist- (a) is removed, and its parent is linked directly to its two ing of N0 uniformly random binary strings, each of length children (100 and 10). L0, with no function calls among them. One could think of this initial set as a group of long functions that are written explicitly in low-level code, with no subroutines decomposition) as a single long function which calls no defined to abstract subunits of the computations. As other functions; after decomposition, the original func- the refactoring process unfolds, these overly long func- tion consists only of calls to the new child nodes. (Ob- tions are decomposed into sets of smaller functions, with viously, such a decomposition is not unique, but that is links developing in the call graph, and with smaller func- true of software systems as well.) tions separated out for reuse by others. Initially, the In this model, three refactoring processes are imple- only active refactoring processes are long function de- mented within the framework of a Monte Carlo simula- composition and reuse of existing functions, since there tion, as schematically represented in Figure 7: (1) func- are no one-off functions to be removed at first. Over tions that are excessively long tend to be decomposed time, however, one-off functions become available for re- into a set of smaller functions; (2) functions that already moval. Such a system will eventually reach an asymp- exist are used by others, rather than having duplicated totic steady state where all possible refactorings have versions of the same function within the system; and been carried out, although the decomposition of strings (3) some functions that are deemed to support “exces- with length l

4 composed) or 2 (decomposed into two substrings). Alter- in 3 out native rules for decomposing large strings might produce (k) c heavy-tailed out-degree distributions even in the absence N 2

10 of one-off function removal, but further work is needed

log 1 to explore in detail the behaviors of models inspired by refactoring techniques. Those interested in the trade- (a) 0 0 1 2 3 offs between indirection and computational performance might be motivated to construct a variant of this model log10k 700 whereby long string decompositions incur a penalty cost 600 associated with performance degradation; other studies 500 might try to quantify the suboptimality suggested by 400 in Valverde et al. [27] which could be associated with frus- k 300 tration of the sort seen in other complex, disordered sys- 200 tems. 100 0 (b) 0 10 20 30 40 k out V. SOFTWARE SYSTEMS AND COMPLEX 1000 NETWORKS: THE IMPLICATIONS OF in OBJECT-ORIENTED DESIGN 100 out A. Robustness, fault tolerance, and evolvability 10 degree

1 Researchers grappling to understand the structure, (c) 1 10 100 1000 function and evolution of complex networks highlight ro- string length bustness as an important theme in complex networks. 0.1 Studies have indicated [8, 32] that one by-product of -1 k scale-free networks in certain systems is enhanced ro- bustness in the face of random node failure (although 0.01 with increased fragility to failures of highly connected C(k) nodes). Analyses that highlight the similarities between biological systems and engineering systems often empha- 0.001 size redundancy in networks as a mechanism for fault (d) 1 10 100 1000 k tolerance [33]. Software systems, on the other hand, are notoriously fragile, and at many scales (e.g., single FIG. 8: Simulation results for model of software evolution by point mutations at the level of a typographical error, refactoring: (a) in- and out-degree distributions, (b) scatter- inability to find a library needed for linking, cascading plot of in- vs. out-degree, (c) in- and out-degree versus string modifications that follow from changes to a single class). length, and (d) degree dependence of clustering, C(k) vs. k, Of course, there are specialized applications which abso- −1 also showing the k dependence suggestive of hierarchical lutely require fault tolerance or are capable of exploiting organization. Parameter values for the simulation were (see redundant resources in the case of network disruption text for discussion): N0 = 50, L0 = 1000,p = 0.7,l0 = 4,T = (e.g., in distributed processing applications). 1.0. The initial graph with 50 nodes and 0 edges evolved into By and large, however, the complexity that lurks a graph with 3721 nodes and 14335 edges. within software systems is not responsible for implement- ing fault tolerance and robust control, as is the case for many complex engineering systems. Rather, much degree/small out-degrees represent small, simple func- of the structural complexity of large software systems tions (short strings) that can be and, in fact, are reused in – and in object-oriented systems in particular – is to many contexts by many other larger functions; whereas support evolvability. The need to continually accommo- large out-degree/small in-degrees represent large, com- date and incorporate changes in the external environ- plex functions that aggregate many smaller functions ment (user requirements, hardware platforms, etc.) lead but which are themselves used in few contexts. Finally, to software designs that support modularity, decoupling, Fig. 8(d) reveals degree-dependent clustering similar to and encapsulation [12]. This connection between envi- that suggested by the hierarchical organization scenario ronmental changes and adaptation toward modular net- and as seen in the data in Fig. 5, although the source of work structure has been noted in several other contexts, the flattening of C(k) at large k is unknown. As noted, such as in models of biological evolution [34] and neural the large out-degree tail in Fig. 8(a) is generated by one- networks [35]. In contrast, very little of the complexity off function removal. Were this process not included, the inherent in complex engineering systems [33] is in place out-degree for every function would be either 0 (not de- to support evolvability and the construction of the next 13 generation system (despite the fact that design elements but this confuses duplicated code with redundant code; do get reused). duplicated code is not redundant unless it is embedded in Much of the evolvability that is organized within soft- the same computational context, in the same way that ware systems arises from carefully planned genericity and two identical resistors serving distinct roles in an elec- associated decoupling, using polymorphism and encapsu- trical circuit are not redundant to each other. They lation to negotiate the inherent trade-offs between speci- claim that there is no degeneracy in software because ficity and evolvability of interactions. While naive no- “degeneracy is very common [in] natural systems...but tions of object orientation suggest the proliferation of in- totally unknown with the context of technological evolu- creasingly deep class hierarchies that implement increas- tion”. They further note that “degeneracy is intimately ingly specialized objects, the software engineering com- associated with tinkering in evolution: it reflects the re- munity has learned that systems based on those sorts of use that is made of different parts of a system in order objects and interactions are often hard to modify [12]. to achieve similar functions.” I would argue that de- Design patterns aim to organize the interactions of ob- generacy is in fact quite common in some software sys- jects in such a way as to ensure sufficient specificity for tems, largely in the form of polymorphism, but perhaps regulation and control without unduly freezing a sys- through other mechanisms as well. Furthermore, gener- tem into commitments and constraints that are difficult alizing from other technological systems to software sys- to evolve. Viewed in this manner, design patterns are tems is problematic, in part because software is softer similar to biochemical processes such as regulated re- and more abstract than other engineered systems. And cruitment [36] that serve to ensure specificity through software design does itself involve a significant amount the cooperative action of several, more generic chemical of evolutionary “tinkering”, which is becomingly increas- constituents, rather than the specific action of a single, ingly recognized and formalized through processes such complex component. It may be that scale-free network as refactoring [31] and extreme programming [13]. topologies help to mediate the trade-offs between speci- ficity and evolvability, and present a mechanism for min- imizing constraint while ensuring the specificity required C. Motifs, patterns, and emergent computational for regulation and control. structures

There is growing interest in scanning large, emergent B. Degeneracy and redundancy networks to locate statistically significant, recurring mo- tifs, and ultimately identifying the functional significance The biological community has begun to make distinc- of those motifs [39, 40]. Information processing systems – tions between redundancy, involving the ability of identi- including gene transcription networks, neuronal systems, cal elements to perform identical functions, and degener- and electronic circuits – are seen to make use of recur- acy, involving the ability of different elements to perform ring motifs such as feed-forward loops and bi-fans (and similar (or perhaps identical) functions [37, 38]. This dis- in some cases, bi-parallel subgraphs) [40]. A prelimi- tinction emphasizes the role that degeneracy can play in nary examination of recurring motifs in the six software evolvability; whereas identical and redundant elements graphs studied here, using the motif finding algorithm are unable to provide any novel function in the face of of Alon and co-workers [40], identifies these same motifs changing environments, similar and degenerate elements as being prevalent, but further study is needed to exam- offer avenues for adaptation because they offer the poten- ine their significance. It remains to be seen, however, tial to provide different behaviors in different contexts. whether such techniques will be able to identify mean- Degeneracy in biological networks is in fact similar to ingful software motifs (e.g., design patterns) in all their polymorphism in object-oriented systems, in that differ- glory, given the relatively crude representations of soft- ent objects can substitute for one another to perform ware networks presented here. The software reverse en- structurally similar functions which nonetheless differ in gineering community is beginning to tackle the problem detail. This polymorphism imbues OO systems with of extracting complex design patterns from existing soft- evolvability by enabling them to be more easily adapted ware systems, but such work relies largely on detailed, to changing needs and environments. Tononi et al. [37] a priori specifications of the structure of those patterns have developed information-theoretic measures to quan- and more detailed class information than is contained in tify redundancy and degeneracy in neuronal networks, the graphs studied here [41]. An interesting challenge for and it is an interesting open question as to whether con- the software engineering community would be to develop nections between degeneracy and polymorphism might systems and algorithms capable of extracting important suggest novel ways of analyzing and interpreting software patterns and motifs from large software networks with- systems based on similar sorts of measures. out such detailed prior information. Such an effort would Curiously, Sol´e et al. [28] have commented that soft- not only be useful for software design and analysis, but ware collaboration networks have “a certain degree of might also help to guide the field of complex networks in [redundancy] but no [degeneracy]”. Their assertion of identifying functionally important motifs. redundancy is based on the existence of duplicated code, One other interesting connection between software and 14 complex networks involves the very notion of “software models aimed at uncovering large-scale structure from engineering”. As software systems move increasingly into small-scale process. Further archaeology of existing soft- the realm of the emergent and the unpredictable, a new ware systems would also help us to better understand notion of “software science” may unfold, emphasizing relationships among network structure, object complexi- fundamental phenomena to be explored, as opposed to ties, object interactions, development processes, and sys- cut and dried systems to be built. An interesting ques- tem evolution, and to unravel the differences between tion concerns the formation of structures akin to soft- class collaboration graphs and call graphs presented ware design patterns. In the same way that recurring here. Systems that have undergone large-scale refactor- spatial patterns (vortices, dislocations, fronts, and soli- ing could be mined to ascertain whether real-world refac- tons) can arise in physical systems under stress, it may toring processes change the nature of software graphs (as be that recurring functional patterns (adapters, factories, they have in my simple model system), and large open- mediators, and proxies) can arise in appropriately defined source development projects such as Linux and AbiWord computational systems driven far from equilibrium. that have made transitions from the “cathedral” to the “bazaar” [14] could be investigated for network-level sig- natures of such transitions. VI. SUMMARY AND CONCLUSIONS Combining insights from empirical studies of exist- ing systems with those gleaned from more abstracted In this paper, I have examined several aspects of soft- models of software systems – such as the refactoring ware collaboration networks, inspired by questions in model presented here – should be more fruitful than ei- complex networks, software engineering, and systems bi- ther approach in isolation. It would be interesting to ology. Not unlike findings by others, the software col- learn whether emergent, automatically generated com- laboration networks studied all exhibit scale-free and/or putational systems, such as those uncovered by ge- heavy-tailed degree distributions qualitatively similar to netic programming techniques [43, 44] or algorithmic those observed in recently studied biological and tech- chemistries [45], give rise to the sorts of topologies that nological networks. An examination of these software are observed here. It remains to be seen, however, systems reveals that the hierarchical nature of software whether practical insights into the design and develop- design has an impact on the overall network topology. ment of software can arise from the consideration of soft- Simpler, more generic classes and subroutines form the ware systems as complex networks, more broadly con- heavy tail of the in-degree distribution, and complex, strued. more specialized aggregates populate the heavy tail of Software systems present novel perspectives to the the out-degree distribution, with the two generally well study of complex networks. Software is designed to be separated from one another. While the process of aggre- both functional and evolvable, and those dual needs sug- gation facilitates the coregulation of many constituent gest particular forms of network organization. Whereas elements, such control is also constraining, and more dif- other complex networks emphasize redundancy to sup- ficult to evolve. Design patterns, polymorphism, refac- port fault tolerance, software networks highlight other toring, and related techniques aim to minimize specificity degrees of freedom that play a central role in supporting of interactions while still enabling specific control, and it evolvability, such as genericity, polymorphism, encapsu- may be that the scale-free nature of software collabo- lation, and collaboration. If those degrees of freedom are ration networks reflects these sorts of trade-offs in the relevant to the organization and evolution of biochemical large. networks, software systems may be useful in suggesting The work presented here highlights the need to pre- novel insights into collective biological function. serve edge directions in studies of directed software graphs, a fact that has long been recognized within the software engineering community. Edge directionality is Acknowledgments required to uncover several network features, such as: differences between in- and out-degree distributions, the This work is supported by the USDA Agricultural Re- anticorrelation between large in-degree and large out- search Service under project 1907-21000-009-04S (Spe- degree, and the positive assortative mixing among out- cific Cooperative Agreement 58-1907-9-017). Resources degrees. Software collaboration is inherently directed, of the Cornell Theory Center (CTC) were used in this and any attempt to explain network topologies from soft- research; the CTC receives funding from Cornell Univer- ware engineering principles or processes without recog- sity, New York State, federal agencies, foundations, and nizing that asymmetry will most likely fall short. corporate partners. I would like to thank David Schnei- More work is needed to better abstract and charac- der, Sam Cartinhour, Jim Sethna, Josh Waterfall, Kevin terize the software development process, and to uncover Brown, Jon Kleinberg, Shalev Itzkovitz, and Alex Iskold the implications of that process for large-scale network for useful conversations, Maria Nemchuk for writing a topology. Lehman’s laws of software evolution and their web-crawler to extract VTK CVS revision history data, associated interpretation in the context of feedback and Mel Gorman for making call graph data available and self-regulation [42] might form the basis of theories or clarifying its construction, and Uri Alon for making his 15 group’s motif finding tool available. I would also like Material, AbiWord, Linux, MySQL, XMMS, Doxygen, to thank the developers of the many software systems Graphviz, CVS, Python, kjbuckets, NumPy, Scientific that contributed to this effort: including: VTK, Digital Python, Grace, and Tulip.

[1] M. Faloutsos, P. Faloutsos, and C. Faloutsos, ACM free geometry in object-oriented programs (2003), see SIGCOMM Computer Communication Review 29, 251 www.mcs.vuw.ac.nz/∼alex/files/ (1999). PotaninNobleFreanBiddleCACM2003.pdf. [2] R. Albert, H. Jeong, and A.-L. Barab´asi, Nature 401, [30] R. Wheeldon and S. Counsell, Power law distributions in 130 (1999). class relationships (2003), arXiv:cs.SE/0305037. [3] A. Broder, R. Kumar, F. Maghoul, P. Raghavan, [31] M. Fowler, K. Beck, J. Brant, W. Opdyke, and S. Rajagopalan, R. Stata, A. Tomkins, and J. Wiener, D. Roberts, Refactoring: Improving the Design of Ex- WWW9/Computer Networks 33 (2000). isting Code (Addison-Wesley, Reading, MA, 1999). [4] M. E. J. Newman, Phys. Rev. E 64, 016131 (2001). [32] J. M. Carlson and J. C. Doyle, Proc. Nat. Acad. Sci. 99, [5] M. E. J. Newman, Phys. Rev. E 64, 016132 (2001). 2538 (2002). [6] D. J. Watts and S. H. Strogatz, Nature 393, 440 (1998). [33] M. C. Csete and J. C. Doyle, Science 295, 1664 (2002). [7] A.-L. Barab´asi and R. Albert, Science 286, 509 (1999). [34] H. Lipson, J. Pollack, and N. Suh, Evolution 56/8, 1549 [8] H. Jeong, S. Mason, A.-L. Barab´asi, and Z. N. Oltvai, (2002). Nature 411, 41 (2001). [35] A. Seth and G. Edelman, in Proceedings of the 23rd An- [9] H. Jeong, B. Tombor, R. Albert, Z. N. Oltvai, and A.-L. nual Conference of the Center for Nonlinear Dynamical Barab´asi, Nature 407, 651 (2000). Systems, Los Alamos National Laboratory, NM, USA. [10] S. H. Strogatz, Nature 410, 268 (2001). (2003). [11] A.-L. Barab´asi, Linked: The New Science of Networks [36] M. Ptashne and A. Gann, Genes & Signals (Cold Spring (Perseus Publishing, Cambridge, MA, 2002). Harbor Laboratory Press, Cold Spring Harbor, NY, [12] E. Gamma, R. Helm, R. Johnson, and J. Vlissides, 2002). Design Patterns: Elements of Reusable Object-Oriented [37] G. Tononi, O. Sporns, and G. M. Edelman, Proc. Nat. Software (Addison-Wesley, Reading, MA, 1994). Acad. Sci. USA 96, 3257 (1999). [13] K. Beck, Extreme Programming Explained: Embrace [38] G. M. Edelman and J. A. Gally, Proc. Nat. Acad. Sci. Change (Addison-Wesley, Boston, MA, 1999). USA 98, 13673 (2001). [14] E. Raymond, The Cathedral and the Bazaar: Musings on [39] S. Shen-Orr, R. Milo, S. Mangan, and U. Alon, Nature Linux and Open Source by an Accidental Revolutionary Genet. 31, 64 (2002). (O’Reilly & Associates, Sebastopol, CA, 1999). [40] R. Milo, S. Shen-Orr, S. Itzkovitz, N. Kashtan, [15] VTK, www.vtk.org/. D. Chklovskii, and U. Alon, Science 298, 824 (2002). [16] DigitalMaterial, www.tc.cornell.edu/Research/ [41] A. Asencio, S. Cardman, D. Harris, and E. Laderman, Multiscale/DigitalMaterial/. in Proceedings of the Ninth Working Conference on Re- [17] C. R. Myers, S. R. Arwade, E. Iesulauro, P. A. verse Engineering (IEEE Computer Society, Washing- Wawrzynek, M. Grigoriu, A. R. Ingraffea, P. R. Dawson, ton, D.C., 2002), pp. 87–96. M. P. Miller, and J. P. Sethna, Mat. Res. Soc. Symp. [42] M. Lehman, J. Ramil, P. Wernick, D. Perry, and Proc. 538, 508 (1999). W. Turski, in Proceedings of the Fourth International [18] AbiWord, www.abisource.com/. Software Metrics Symposium (METRICS 97) (IEEE [19] A.-L. Barab´asi, R. Albert, and H. Jeong, Physica A 281, Computer Society, Washington, D.C., 1997), p. 20. 69 (2000). [43] J. Koza, Genetic Programming: On the Programming [20] P. Erd¨os and A. R´enyi, Publ. Math. Inst. Hung. Acad. of Computers by Means of Natural Selection (The MIT Sci. 5, 17 (1960). Press, Cambridge, MA, 1992). [21] S. M. Henry and D. G. Kafura, IEEE Transactions on [44] J. Koza, Genetic programming II : automatic discovery Software Engineering 7(5), 510 (1981). of reusable programs (The MIT Press, Cambridge, MA, [22] M. E. J. Newman, Phys. Rev. Lett. 89, 208701 (2002). 1994). [23] E. Ravasz, A. Somera, D. Mongru, Z. Oltvai, and A.-L. [45] W. Fontana and L. Buss, in Boundaries and Barriers, Barab´asi, Science 297, 1551 (2002). edited by J. Casti and A. Karlqvist (Addison-Wesley, [24] E. Ravasz and A.-L. Barab´asi, Phys. Rev. E 67, 026112 1996), pp. 56–116. (2003). [46] Doxygen, www.doxygen.org/. [25] H. Fraser, A. Hirsh, L. Steinmetz, C. Scharfe, and [47] Graphviz, www.research.att.com/sw/tools/graphviz/. M. Feldman, Science 296, 750 (2002). [48] CodeViz, www.csn.ul.ie/∼mel/projects/codeviz/. [26] B. Henderson-Sellers, Object-Oriented Metrics: Measures [49] See, www.tc.cornell.edu/∼myers/Data/SoftwareGraphs. of Complexity (Prentice-Hall PTR, Upper Saddle Revier, [50] VTK-CVS-Repository, NJ, 1996). public.kitware.com/cgi-bin/cvsweb.cgi/. [27] S. Valverde, R. Ferrer-Cancho, and R. V. Sol´e, Europhys. Lett. 60, 512 (2002). [28] R. V. Sol´e, R. Ferrer, J. M. Montoya, and S. Valverde, Complexity 8, 20 (2002). [29] A. Potanin, J. Noble, M. Frean, and R. Biddle, Scale- 16

APPENDIX: MATERIALS AND METHODS ple can produce multiple classes emanating from a single pair of source files). They are excluded from the analysis The class collaboration graphs presented here (VTK, relating class metrics to graph degrees, although they do Digital Material, AbiWord) are all based on subgraphs contribute to the source file sizes of their parent classes generated by Doxygen [46], an automatic document gen- (introducing some error to those sizes). The computed eration tool that parses C++ header files to describe source code file size for each class is the sum of the total classes, their methods, inheritance, and collaborations. number of lines of code (including comments and blank Doxygen generates a set of files describing collaboration lines) in the two source files. The number of methods graphs in the “dot” format [47], and Doxygen’s definition defined for each class are derived by combining Doxygen of collaboration to include inheritance plus association information on class methods and the embedded inheri- was used for this particular study. Each class collabora- tance graph defined for each system. Doxygen documents tion graph is generated independently of all others, so all only those methods defined within a class, which does not the subgraphs must be assembled into a global collabo- include methods inherited from base classes. The inheri- ration graph. Some minor typographic changes to class tance graph is thus traced to add to this list of methods names were required to enable programs in the Graphviz defined for each class those public methods defined by its package to process the resulting graph files. base classes. Finally, because VTK has been developed The call graphs (Linux, MySQL, XMMS) were avail- within the framework of the CVS source code revision able for download on the Web as demonstration data system, information is available describing the revision associated with the CodeViz package, developed by Mel history of every source file in the system (and hence, for Gorman [48]. CodeViz includes patches to the gcc com- each class in the system, because of the strict mapping of piler that enable the extraction of static call graphs of classes to two source files). This CVS revision informa- functions and macros. As such, CodeViz does not in- tion is available on the Web [50], which can be crawled clude calls through function pointers, nor does it capture and parsed. Any change to either the header or imple- inline functions or naming collisions between functions mentation file of a class resulting in a new CVS version with the same name in multiple files. number for either of those files was counted as a revision Connected component analysis was done using the to the class; this, therefore, counts twice any change to ccomps and sccmap tools in the Graphviz package [47], a method signature that would necessitate an update to for weak and strong connected component analysis, re- both the header and implementation files, leading to an spectively. As noted, subsequent analyses were carried overestimation of rates, which may or may not be offset out on the single, dominant weak connected component by the fact that multiple source revisions could be swept found in each system. Graph data and associated infor- under a single update to the CVS system. From these mation are available online [49]. data, counts of the total number of revisions of each class The various class metrics for the VTK system pre- were generated, and divided by that class’s total lifetime sented in section II E were extracted as follows. Almost in the CVS repository, to arrive at an average revision every class in VTK is declared and defined in two sep- rate since inception (expressed as average number of revi- arate source files (header .h and implementation .cpp) sions per year). Classes that had been in the CVS repos- whose names correspond to the associated class. Two ex- itory for less than 3 million seconds (roughly 35 days) ceptions are inner classes (which are defined within their were excluded from the analysis, since their short life- parent source files, but which are identified by Doxygen time tended to introduce large errors in the calculation as unique classes), and templated classes (which in princi- of their revision rates.