Extracting Library-Based Object-Oriented Applications
Peter F. Sweeney Frank Tip IBM Thomas J. Watson Research Center IBM Thomas J. Watson Research Center P.O. Box 704, Yorktown Heights, NY 10598, USA P.O. Box 704, Yorktown Heights, NY 10598, USA pfs@ us.ibm.com [email protected]
ABSTRACT s Modem objec~oriented applications typically rely on one or more independently developed class libraries. In an increasingly popular model of software distribution, software is developed in one computing environment and de- With the advent of virtual machine technology~ library ployed in other environments by transfer over the internet. code is amenable to the same analyses as application code, because the same representation is used in each Extraction tools perform a static whole-program analysis to determine unused functionality in applications in order case. When an application is distributed separately from the libraries it depends upon, an extraction tool to reduce the time required to download applications. We have identified a number of scenarios where extraction tools needs to be aware of the boundary between the two. require information beyond what can be inferred through • Different kinds of software distributions (e.g.,complete static analysis: software distributions other than complete applications, web-based applications that execute in applications, the use of reflection, and situations where an the context of a browser, and extensible frameworks) application uses separately developed class libraries. This have different sets of entry points, and require the paper explores these issues, and introduces a modular spec- application extractor to make different assumptions ~cation language for expressing the information required about the deployment environment. In fact, the same for extraction. We implemented this language in the con- unit of software may even play different roles, depend- text of Jax, an industrial-strength application extractor for ing on the deployment scenario. Jaw, and present a small ease study in which different ex- traction scenarios are applied to a commercially available s The use of dynamic features such as reflection I poses library-based application. additional problems for extraction tools, because a static analysis alone is incapable of detern~inlng the program constructs that are used, and hence the program con- 1. INTRODUCTION structs that can be removed. In an increasingly popular software distribution model, software is developed in one computing environment, and • There are also some interesting interactions between deployed in other environments by transfer over the inter- the above issues. For example, consider a situation net. Because the time required to transfer an application is where an application A is to be distributed together generally proportional to the transferred number of bytes, with an independently developed class library L in it becomes important to make applications as small as pos- which reflection is used. In general, the use of re- sible. Application extractor8 are tools that reduce appli- flection in L may depend on the features in L that cation size by determining unused functionality that can be are used by A. We will discuss how this observation removed from the application without affecting program be- affects extraction. havior. Each of these issues requires information that cannot be ob- Previously~ extractors have been designed primarily with complete applications in mind. Such whole-application ex- tained using static analysis alone, and has to be provided to the extraction tool by the user. This paper explores the tractors require one to specify an application's entry point(s), above issues in detail, and provides a uniform solution in and rely on a static whole-program analysis to determine the form of a small, modular specification language MEL functionality that can be removed without affecting program (Modular Extraction Language) for prodding the informa- behavior. However, the extraction of software distributions other than complete applications raises several issues: tion required to extract various kinds of programs. MEL's features are essentially language-independent, with the ex- ception of some Java-speci~c syntax used to refer to program Permission to make digital or hard copies of all or part of this work for constructs such as classes, methods and fields. In order to personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advan- validate our approach, we implemented MEL in the context tage and that copies bear this notice and the full citation on the first page. of Jax, an industrial-strength application extractor for Jav~ To copy otherwise, to republish, to post on servers or to developed at IBM Research [18]. We discuss how several of redistribute to listS, requires prior specific permission and/or a fee. SIGSOFT 2000 (FSE-8) 11/00 San Diego, CA, USA IFor convenience, we will henceforth use the term 'Yel]ec- © 2000 ACM ISBN 1-58113-205-010010011._$5.00 tion" to refer to all mechanisms for loading and accessing program constructs by specifying their name as a string value, and for examining program structure.
98 the program transformations and optimizations performed for the extraction of A itself. If we want to accommodate by Jax are adapted to take into account MP.L scripts, and scenarios where v obtains a different version s of L, then the present a small case study in which different extraction sce- extractor should derived Aez t from A without making as- narios are applied to a commercially available library-based sumptions about the specific version of L that happens to Java application. be available in a's development environment. The standard The remainder of this paper is organized as follows. In Jav~ libraries are an obvious example of this situation. Section 2, we present the requirements on extraction tools We will now investigate the issues related to the use of in the presence of class library usage. Section 3 introduces reflection. In essence, reflection allows one to access a pro- a specification language for defining the extraction of vari- gram construct by specifying a run-time string value that ous kinds of library-based applications. Section 4 presents represents the construct's name, and to examine the struc- a mechanism for translating specifications to a small set of ture of the classes used in a program. Such features are assertions. Section 5 discusses an implementation of MEL, problematic for extraction tools because, in general, a static and reports on a small case study. Section 6 summarizes analysis cannot determine which program constructs are ac- related work, and Section 7 presents conclusions and direc- cessed using reflection, and should therefore not be removed tions for future work. or transformed. Thus, extractors require additional informa- tion from the user that specifies which program constructs 2. REQUIREMENTS are accessed using reflection. In our experience, determining the program constructs that may be accessed using reflec- In this section, we analyze a number of frequently occur- tion is a fairly easy task for a programmer familiar with the ring distribution scenarios, and determine what information code. However, it can be quite difficult to determine how is required by extraction tools beyond what can be obtained reflection is used in third-party libraries, especially if the through static analysis, source code for these libraries is unavailable. In the exam- ple of Figure 1, the extraction of ALezt from A and L by 2.1 Distribution scenarios application vendor a requires additional information about Figure I shows several distribution scenarios that may oc- the use of reflection in L. This can be difficult to determine cur in the presence of: a library vendor I responsible for cre- from distribution Lez t alone, because it does not contain the ating and distributing a class library L, an application ven- source code for the library. To complicate matters further, dor a responsible for creating and distributing an L-based the set of program constructs in L accessed using reflection application A, and two users, u and v, of application A. may depend on the features in L that are used by A. In It is reasonable to assume that library vendor l will want general, different L-based applications may cause different to make library L ms small as possible, in order to reduce usage of reflection within L. Our solution to these problems the download times experienced by customers, but also to (discussed in detail below) will he to have library vendor reduce the load of the server from which the library is down- l distribute a script along with Lez t that contains the in- loaded. Hence, I creates an eztracted version Lez t of L, and formation required to extract any L-based application. Our distributes Lez t instead of L. Clearly, Lez t should offer the scripts allow I to specify that a program construct is only ac- same functionality as L, but size-reducing optimizations can cessed using reflection under certain conditions (e.g., when still be applied to parts of L that are not exposed to users. a certain method is reachable). Application vendor a presumably downloads Lez t for use We have only discussed a few example distribution sce- during development of application A. When application A narios. Other likely scenarios include: is ready for distribution, there are two options, depending on whether or not a user already has the prerequisite library • Extracting a library together with multiple applica- L installed. Figure I shows a user u who does not have (the tions that use it. correct version of) L. Assuming that u does not expect to download or create other L-based applications, it is desir- • Extracting a library in the context of another library able for u to download a distribution ALez t that comprises that uses it. We believe that such situations, where the functionality of A and the parts of L used by A, but multiple layers of libraries exist and where only the that omits the parts of L that are not used by A. Because topmost layer is exposed to an application, is likely to applications typically use only a small part of the function- become increasingly common. ality of libraries they rely on 2, the removal of the parts of L not used by A is likely to significantly reduce the size of the 2.2 Roles of software units distribution. We will adopt the non-descriptive term so.f~are unit in There are also scenarios where it is preferable to keep the the sequel to denote any collection of classcs that constitutes distributions of L and A separate. Figure I shows another a logics] entity. Recall that there is no dl/~erence between user v of application A, who has downloaded Lax t directly code in a class library and code in an executable applica- from l, because he is pIRnnlug to deploy multiple applica- tion, and it is only the way in which software units are used tions that rely on the library. Because v already has Le= t, and composed that determines how extraction should be per- he only needs to download the application itselffrom vendor formed. In the remainder of this paper, the term role will he a. To this end, a creates an extracted version Aez t of A that used to refer to the way in which a software unit is used. We can be downloaded by v. It is important to realize that keep- will consider four roles that frequently occur in the context ing the distributions of A and L separate has repercussions of Java:
21n previous work on whole-application extraction [18], we 8Thls could either be an earlier version of L that was ob- reported that up to 60~ of the methods in several library- tained from library vendor l, or a completely different im- based benchmark applications is unreached. plementation of the library from a different vendor.
99 from another application vendor
Aext Bext ~ ALxt ,,,// L ~lext Lext i ....l ...... std. libraries std. vMlibraries std. vMlibraries std. libraries
VM VM
l: library a: application u: application v: application vendor vendor user #1 user #2
Figure 1: Illustration of different distribution scenarios.
• An application is an executable software unit with an In practice, there are very few situations where all classes external interface consisting of a single main() method. should be extracted. One can think of the structure of an It is assumed that classes in applications are not fur- application as "layered", with the bottom layer consisting ther extended by derivation after extraction. of the standard libraries, the middle layer consisting of class libraries built on top of the standard libraries (perhaps con- • An applet is an executable software unit that is exe- sisting of sublayers), and the topmost layer consisting of the cuted in the context of a browser. An applet extends application itself. It is usually the case that classes below class java,applet.Applet and its external interface a certain layer do not need to be extracted and shipped consist of a set of methods in java.applet.Applet because they are already available in the deployment en- that it overrides. It is assumed that classes in applets vironment. In particular, the standard class libraries are are not further extended by derivation after extraction. generally available and are usually excluded from the ex- • A lib~ry is not assumed to be executable by itself, but traction process. 4 It is important to realize that this is not is used as a building block by other units. Classes in merely an issue of avoiding redundant work and shipping libraries may be extended by derivation. The external redundant code, but potentially also one of correctness. If interface of a library consists of any method that has an application class contains a call to a method in the stan- public or protected access rights. dard libraries, jnllnlng that call on one platform may result in code that does not work on another platform. • A component is similar to a library in the sense that it is an incomplete program used as a building block by 2.4 Dealingwith dynamicfeatures other units. But, lm]ike a library, it is assumed that classes in a component cannot be extended by deriva- Java's reflection mechanism allows programs to do ear- tion. The external interface of a component contains ions forms of self-inspection. Figure 2 shows an exam- every method with public access rights. ple program that uses structural reflection (sometimes also referred to as introspection). In this program, the class Other roles such as JavaBeans [16] and servlets [7] can be that represents the type T of object t is retrieved us- modeled similarly. For example, in the case of JavaBeans, ing a call to method java.laug.0bject.getClassO, and all of the JavaBean's methods that may be invoked by client stored in variable c. The program then calls method applications are contained in its external interface. j ava. lang. Class. getDeclaredlqethods () to obtain a vec- 2.3 Specifying the extraction domain tor of object• representing the methods in T. For each method in this vector, the name is retrieved (by way of a call There is no distinction between classes in different soft- to java. lang. reflect.Hethod, getName 0), and printed to ware units at the language level. Consequently, it is nec- standard output. Hence, the program generates the follow- essary to specify the '%oundarles" between software units hag output: when performing extraction. In our approach, the user se- lects the set of classes that should be extracted, and worst- 4In the case of embedded systems and network PC's that case assumptions are made about the behavior of classes run a fixed set of applications it may be desirable to include that are not selected. the standard class libraries in the extraction domain.
100 import java.lo.*; import java.is.*; import java.lang.Class; import java.lans.Class; import java.lane.reflect.Method; public class Example2 { public class Examplel { public static void baz(String nama){ public static void main(String arts[l) { try { T t = new TO; Class c = Class.forName(nama); Class c = T.getClassO; Object o ~ c.ne.InstanceO ; Method[] methods ffi c.gstDeclaredNethodsO; I i ffi (I)o; for (int iffiO; i < methods.length; i++)~ i.zapO ; Method m = methods[i]; } StringmethodName = m.getNameO; catch (ClassNotFoundExcaption e){ System.out.prlntln(mothodNams); System.out.println("Error: " + } "Could not find " + name); } catch (IllegalAccessException e) }; System.out.println("Error: " + "Illegal acosss to " + name); } class T { catch (InstantiationExcsptlon s){
void ,oo0 ... System.out.println("Error: " + void bar(){ ); "Abstract " + name); } }; } );
Figure 2: A Java program that uses reflection. interface I { public void zapO; ); los bar Figure 3: A Java program that uses dynamic loading.
Clearly, program behavior depends on the presence and the name of the methods in class T, even though these meth- Java provides a mechanism for implementing methods in a ods are not invoked anywhere. It is obvious that the use platform-dependent way, typically using C. The mechanism of reflection in Figure 2 precludes program transformations works roughly as follows: The native keyword is used to such as the removal or renaming of methods in class T be- designate a method as being implemented in a different lan- cause such actions would affect program behavior. guage, and the corresponding method definition is provided Dynamic loading, another form of reflection, is a heavily- in an object file (e.g., a dynamically linked library) asso- used s mechanism for instructing a Java Virtual Machine to ciated with the Java application. The native code in the load a class X with a specified name s, and return an ob- object file may instantiate classes, invoke methods, and ac- ject c representing that class. Reflection can be applied to cess fields in the application. This obviously poses problems c to create X-objects on which methods can be invoked. for any program transformation that relies on accurate in- The crucial issue is that s is computed at run-~me. This formation about class instantiation and method invocation, implies that, in general, static analyses c~-ot determine because object code is notoriously hard to analyze. which classes are dynamically loaded,s It should be evident from the above examples that, with- Figure 3 shows a program fragment that exhibits a fairly out additional information, the use of reflection, dynamic typical use of dynamic loading. Class Example2 contains a loading, and native methods requires that ezt;remely con- method baz which takes a single argument of type String, servative assumptions be made during extraction: It would and dynamically loads a class with that name by on]ling essentially be impossible to remove, rename, or transform method java.lang.Class.forNamsO. A reference to the any program construct. The approach taken in this paper dynamically loaded class is stored in variable c. The pro- relies on the user to specify a list of program constructs gram then calls method j ava.lang.Class.newInstanceO (i.e., classes, methods, and fields) that are accessed using to create a new object of the dynamically loaded type, casts these mechanisms, and to make the appropriate worst-case it down to an interface type I, and calls method zap on assumptions about these constructs. the object. Observe that class instantiation (of the dynam- ically loaded class) and method invocation (of the default 2.5 Modeling different usage contexts constructor of that class) occur implicitly. This poses prob- Section 2.1 already alhided to issues related to the use of lems for optimizations such as dead method removal because third-party libraries in which reflection is used. In order to the analyses upon which these optimizations are based typi- create MEL scripts that are reusable in different contexts, it cally need to know which classes are instantiated, and which is often desirable to specify that a given program construct methods are invoked. is only accessed using reflection under certain conditions. To illustrate this issue, Figure 4 shows a small class library 5Nine of the thirteen benchmarks studied in [18] use dy- consisting of three classes L, M and N. Class L has two meth- namic loading. ods: f and 8. A call to method f results in the dynamic Sln some cases, the type of a dynamically loaded class can loading of class M, and a call to method 8 results in the dy- be inferred by constant propagation of the string literals that represent the class name. However, we have observed namic loading of class N. Note that a client that calls f but that these names are often read from filesor manipulated in not 8 will only access M, and a client that calls 8 but not f non-trivial ways. will only access N. A specification of the library's behavior
101 import java. lang. Class ; MELScript ::= Item. Item ::= DomainSpecifier I public class L { Statement I Import public static void fO{ DomainSpecifier ::= ClassPath I Include ... ClassPath ::= path
102 reflection is used in a specific method. path • • • include L Table 1 shows how statements are translated to assertions. library L The table contains a row for each type of MEL statement in preserve M when reached L.gO which the rightmost column shows the assertions generated preserve N when reached L.fO for that statement. The translation process for roles can be summarized as follows: Figure 7: Specification L.rael for the class library of Figure 4. Worst-case assumptions are made to determine a set of methods that can be invoked from outside the ex- path • •• traction domain. Each such method is assumed to be include A application A reached, and its identity is preserved to indicate that import L.mel external references may rely on its name and signa- ture. Different roles require different treatment. For Figure 8: Specification A.mel for the application of example, for applications, only the mainO method Figure 6. is referenced externally and needs to be added to the set. However, for classes that play a llbrary role all public and protected methods are added. Statement ::= Assertion ] ConditionalAssertion Assertion ::= SimpleAssertion Assertion ::= extendible
103 statement derived assertions application U preserveIdentity C' .... reached0.main(java.lang. String[]) preserveIdentityC.main(java.lang.Strin~[]) appletC instantiatedC' preserveIdentity C preserveIdentityC.mfor every C.~thatoverrides java.applet.Applet.~ reached (]'.m for every O.'m, that overrides java. applet .Applet.m component U prsserveIdentity 6." preserveIdentity C.m for every public method C.m reached C.m for every public method 0.~ preserveIdentity ~./for every public field C.f accessed C./for every public field C./ ~ibrary G' preserveIdent ity (~' extendible C reached C.m for every public or protected method C.m preserveIdentlty C.m for every public or protected method C.m overridable C.m for every public or protected instance method C.m accessed C.f for every public or protected field O.f preserveIdentlty C.f for every public or protected field C.f preserve G £nstantiated C when C is not an interface or an abstract class preserveIdentity preserve G.~ reached G.rn, preserveIdentity O.m preserve U.~ accessed ~.f preserveldentit~ O.f preserve G .hen reached D.n instantiatedG.hen reached D.n preserveIdentityCwhen reached D.n preserve U.r~when reached D.n reached G.~when reached D.n preserveIdentity C.~when reached D.n preserve G.~ when reached D.. accessed U.f when reached D.n preserveIdentityC.f when reached D.n
Table 1: Translation of statements into assertions. in the context of Jax [18].z The implementation also per- classes, and that has been demonstrated to scale well in mits users to specify MEL assertions directly, and has mech- practice [18]. RTA is most easily implemented as an it- anisms for specifying the name of the generated sip file, erative algorithm that uses three worklists cont~in~g (i) and for selectively disabling optimizations. Jax provides reached methods, (ii) reached call sitess, and (iii) instanti- two mechanisms to support MEL. In '%arch mode", a MEL ated classes. The worklist of reached methods is initialized script is read from a file, and the application is processed to contain the set of methods called from outside the ap- accordingly. A Graphical User Interface (GUI) that allows plication (e.g., an application's main() method), and the users to create MEL scripts interactively is also provided. other two worklists are initialized to the empty set. Then, We will discuss how a number of program transforma- following steps are performed repeatedly: tions and optimizations performed by Jax can be adapted to operate on various kinds of library-based applications by • The body of a reached method is scanned. Any call sites and class instantiations that were not previously talH,g into account MEL assertions. These optimizations encountered are added to the appropriate worklist. were originally presented as whole-programs optimizations, by ma]dng the "closed world" assumption that the entire • Each call to a method C.f is resolved with respect to program is available at analysis time. each instantiated class D, where D is a subclass of C. This involves performing a method lookup for f in S.1 Call graph construction class D. If the lookup resolves to a method that was Since all of the optimizations under consideration rely di- not previously reached, it is added to the workllst of rectly or indirectly on the construction of a call graph, we reached methods, and the call graph is updated with will first discuss how call graph construction algorithms can edges that reflect the flow of control between caller and he adapted to take into account MEL assertions. We will caUee. use Rapid Type Analysis (RTA) [5, 4], an efficientcall graph construction algorithm, as a specific example. Other call- This iterative process continues as long as additional meth- graph construction algorithms (see e.g., [9, 11, 19]) can be ods, call sites, and instantiated classes are found. In cases adapted similarly. where a class C in the extraction domain overrides a method RTA is a popular algorithm for constructing call graphs f in 8 class outside the extraction domain, we make the and devirtualizing call sites that only requires class hierar- worst-case assumption that there is a call to this method on chy information and global information about instantiated any object of any instantiated class. In order to adapt RTA to take into account MEL asser- 7 Version 6.0 of Jax (released in August 1999) supports MEL tions, we first need to adapt the initialization o£ the work- in its full generality, although the syntax of the MEL key- words in the system differs slightly from the syntax used in s Since all calls to any given method f are resolved similarly, this paper. any reasonable implementation combines them.
104 lists. The worklist of reached methods is initialized to con- simply be removed. Fields that are only written are also taln any method m for which an assertion reached m was removed along with the write-operations that access these generated. The worklist of reached call sites is initialized fields. Dead field removal can be adapted to handle MEL to contain the empty set. Finally, the worklist of instanti- assertions by considering a field £7.f to be read-accessed if ated classes is initialized to contain any class C for which there exists a accessed C.f assertion. Conditional accessed an assertion 2nstantiated C was generated. assertions can be treated in the same way as conditional Then, in the iterative part of the algorithm, we add the foL reached assertions. lowing additional steps, which are executed when a method m is added to the worklist of reached methods. 5.5 Name compression Name compression reduces application size by replacing • Whenever a method m is added to the worklist of reached methods for which an assertion instantial;ed the names of classes, methods, and fields with shorter names. The names of a class or field x can be changed if • is not C when reached m exists, class C is added to the worklist of instantiated classes if it does not already instantiated or accessed outside the extraction domain, re- occur in that list. spectively. The conditions under which methods can be re- named are a bit more complicated. Certain methods such as • Whenever a method m is added to the worldist of constructors, class initializers, and class finalizers cannot be reached methods for which an assertion reached m ~ renamed. A method that overrides a method outside the ex- when reached m exists, method m' is added to the traction domain cannot be renamed. Finally, if one method worklist of reached methods if it does not already oc- overrides another, both must be renamed correspondinglyJ ° cur in that list. In the presence of MEL assertions, a number of additional constraints have to be imposed on the renaming of program 5.2 Dead method removal constructs. Any program construct z for which there exists Dead Method Removal [18] is an optimization that re- an assertion preserveIdentity x cannot be renamed, any moves redundant method definitions. This optimization re- method m for which there exists an assertion overridable lies on the information gathered during call graph construc- m cannot be renamed, and any class c for which there exists tion to determine situations where a method can be removed an assertion extendible c cannot be renamed. completely, as well as situations where a method's body can be removed but where its signature needs to be retained. 5.6 Class hierarchy transformations The latter situation arises in the following eases: Removal of unused classes, and merging of a derived class • There is a reached call site that refers statically to an C with its base class B reduce application size. The lat- instance method C.m, but C.~r~ is not the target of any ter transformation involves moving the methods It and fields dynamic dispatch or direct call. from C' to B, and updating the references to these methods accordingly. The main benefit of class merging has to do • There is a class C that (i) contains an unreached method with the fact that in Java class files, each class is a self- C.m, and (ii) implements an interface r containing a contained unit with its own set of literals, referred to as its declaration I.m of the Same method that is called else- constant pool. Classes that are adjacent in the hierarchy where in the application. typically have many literals in common, and merging such Note that, in the latter case, method C.m cannot be removed classes reduces the duplication of literals across the different because the resulting class file would be syntactically invalid. class files. Class merging may also enable the transformation In both cases, no additional information is necessary beyond of polymorphic calls into direct method calls. Space limita- the information determined during call graph construction. tions do not permit a complete discussion of class merging here, and we refer the reader to [18, 20] for details. In order 5.3 Call devirtualization to take into account MEL assertions, any class C for which Call devirtualization [6, 3] transforms run-time dispatch there exists an assertion preserveIdentity C should not calls into direct calls. This transformation can be applied he removed, or merged into its base class. at a call site z that calls method C.m if (i) there is only one method that can be reached from z, and (ii) method 5.7 A Case Study C.m cannot be overridden after extraction of the applica- We now present a small case study in which different ex- tion. The first condition can be verified by inspection of the traction scenarios are applied to Cinderella 12, an interactive call graph, and the second condition is met if there is no as- geometry tool used for education and self-study in schools sertions overridable C.m or extendible C, where C.m is and universities. Cinderella consists of an application, which the method invoked at call site z. Other optimizations that can be used for constructing interactive geometry exercises, rely on closed-world assumptions (e.g., inl/ning [15] and call and an applet in which students can attempt to solve these devirtualization) can be adapted similarly. 1°Actually, the situation is slightly more complex. Consider a situation where a class C extends a class /3 and imple- 5.4 Dead field removal ments an interface I, and where a method f is declared in Dead field removal [17] removes fields that are not ac- l, defined in/3, hut not defined in C itself. Then, the occur- ceased, as well as fields that are wrlte-accessed but not read- rences of f in [ and /3 are related and should be renamed accessed. This optimization requires that the bodies of all correspondingly. reached methods are scanned for read and write operations 11A minor practical issue that comes up here is that construc- to fields, s Fields that are neither read nor written can tor methods need to be made unique. At the Java class file level, this can be accomplished by adding additional dummy ~This is most easily done during call graph construction arguments. when method bodies have to be traversed anyway. 12See wmw, cinderella, de.
105 [ distribution [[ zip file [ classes [ methods [ fields [ the standard libraries and based on the names of classes, Original 664,836 337 3057 2391 and a simple, low-level mechanism was provided to spec- ify that certain program constructs accessed using reflection [ Antlr 181,535 130 1369 677 should be preserved. As a result, Jax was only suitable for distribution scenarios in which an application is shipped by itself, or where an application and a library are extracted and shipped together. The benchmarks studied in [18] are Applet+Antlr 184,486 177 1355 842 all instances of one of these scenarios. The work in this Application 391,813 285 2490 1784 +Antlr paper was motimsted by our goal to accommodate other dis- Both + Antlr 403,327 293 2541 1797 tribution scenarios such as independently shipped libraries, and to unburden the developer of a library-based applica- tions from having to specify information (e.g.s the use of Table 2: Results of multiple distribution scenarios for reflection) about the library. "Cinderella". The extraction of applications was pioneered in the Smalltalk community, where it is usually referred to as exercises. Two interesting observations can be made about "packaging~ [12, 10, 14]. Smalltalk packaging tools typically Cinderella. First, the application and the applet are derived have mechanisms for excluding certain standard classes and from the same code base, which is contained in a single objects from consideration, and for forcing the inclusion of zip file. Second, Cinderella relies on a class library called objects and methods. While the latter mechanism is suf- "ANTLR" for parsing. ficient to handle programs that use reflection, we are not Table 2 shows different distribution scenarios for Cin- aware of any Smalltaik extractor that models different types derella. The first two rows, labeled Original and Antlr of applications, or that provides a feature to preserve certain (orig.) are concerned with the original distributions of program constructs conditionally. Cinderella and ANTLR, respectively. The columns of the ta- Agesen and Ungar [2, 1] describe an application extractor ble show the size of the zip file, and the numbers of classes, for the Self language that eliminates unused slots from ob- methods and fields, respectively. The next row, labeled jects (a slot corresponds to a method or field). In his PhD Antlr shows the result of extracting ANTLR as a stand-alone thesis [1, page 146], Agesen writes that there is no easy solu- library. The reduction in size was obtained by removing sev- tion to dealing with reflection other than "rewriting existing eral methods and fields that are cmly accessible inside the code on a case by case basis as is deemed necessary ~ and sug- library. The next three rows, labeled Applet, Application, gests "encouraging programmers writing new code to keep and Both shows the size of extracting the application, the ap- the limitations of extraction technology in mind". In con- plet, e ud their combination without ANTLR. Finally, the last trast, we allow the user to specify where reflection occurs, three rows, labeled Applet + Antlr, Applicatlon + Antlr, so that applications that use reflection can be extracted. and Both + Antlr show the results of extracting the appli- Chen et. al [8] describe Acacia, an extraction tool for cation, the applet, and their combination together with the C/C+-{- based on a repository that records several relation- parts of ANTLR that they use. ships between program entities. Several types of teachability The following observations can be made: analyses can be performed, including a forward reachability analysis for determining entities that are unused. Chen et • The applet's functionality is (roughly) a subset of the al. identify several issues that make extraction difficultsuch avplication's functionality, because adding the applet as the use of libraries for which code is unavailable, and to the distribution does not increase size by much. situations where functionality should be preserved because source modules are shared with other applications. Unlike • On the other hand, the size of the applet is significantly our work, Acacia is an analysis tool aimed at prodding in- smaller than the combined distribution. Hence, users formation to the user, and does not actually perform any that only require the applet will prefer the distribution program transformations such as de~d code elimination. A contaln~ng only the applet. number of issues that we study such as the use of reflection • From the fact that the distributions that include are not discussed, and no mechanism appears to be available ANTLR are not much bigger than the distributions for supplying additional information to the extractor. without ANTLR, we can infer that Cinderella uses only In the context of Java, we are aware of a number of several a small subset of ANTLR's functionality. commercially available extraction tools. DashO-Pro Is and Condensity 14 are tools with similar goals as Jax. We are • Extracting ANTLR by itselfresults in anontrivial (about not aware of any published work on the algorithms used by 20%) reduction of distribution size. This confirms that these tools, or on the internal architecture of these tools. extracting stand-alone class libraries is worthwhile. There is a large body of work on reverse engineering that attempts to extract designs or object models from applica- 6. RELATED WORK tions (see e.g., [13]). This work could benefit from appli- We will begin this section with a brief historical perspec- cation extraction technology because, by eliminating dead tive on this work. The approach taken in this paper was code, more precise designs could be extracted, and spurious motivated by our experiences with Jax, an application ex- relationships between classes or program constructs would tractor for Java [18]. Jax was developed as tool for extract- tSDashO-Pro is a trademark of preemptive Solutions, Inc. ing applications, and initially relied on ad-hoc solutions for See wee. preemptive, com. several of the problems we study in this paper. For exam- 14Condensity is a trademark of Plumb Design, Inc. See ple, there was a fixed '%oundary" between applications and ewe. condensity, com.
106 not appear in the extracted designs. Similar to application Eleventh Annual Conf. on Object-Oriented extractors, design extraction tools require additional infor- Programming Systems, Languages, and Applications mation from the user in the presence of dynamic language (OOPSLA '96) (San Jose, CA, 1996), pp. 324-341. SIGPLAN Notices 31(10). features such as reflection. [6] CALDER, B., AND GRUNWALD, D. Reducing indirect function call overhead in C++ programs. In Proc. of 7. CONCLUSIONS AND FUTURE WORK the ~lst Annual AGM Symposium on Principles of Programming Languages (1994), pp. 397-408. We have identified a number of situations where the ex- [7] CALLAWAY,D. R. Inside Servlets: Server-Side traction of software requires information that cannot be ob- Programming for the Java Platform. Addison-Wesley, tained by static analysis techniques alone. This includes 1999. software distributions other than complete applications, the [8] CHEN, Y.-F., GANSNER, E. R., AND KOUTSOFIOS, E. use of reflection in applications, and situations where library- A C++ data model supporting reachability analysis based applications are extracted and distributed separately. and dead code detection. IEEE Transactions on To address these issues, we have proposed a small, modu- Software Engineering ~4, 9 (Sept. 1998), 682-694. lar specification language, MEL, that allows one to specify [9] DEAN, J., GROVE, D., AND CHAMBERS, C. Optimization of object-oriented programs using static the information required for extraction in a uniform manner. class hierarchy analysis. In Proc. of the Ninth We have argued that the modular nature of MEL scripts al- European Conf. on Object-Oriented Programming lows for a useful separation of responsibilities: each module GOOP'95)(Aarhns, Denmark, Aug. 1995), of a MEL script can be written by a programmer who is fa- ~ • Olthoff, Ed., Springer-Verlag, pp• 77-101. miliar with the code, and extraction of an application that [I0] DIGITALK INC. Smalltalk/V for wln$~ Programming, relies on third-party libraries ouly requires a MEL script for 1993. Chapter 17: "Object Libraries and Library that library. We have discussed how several whole-program Builder. [II] DIWAN, A., MOSS, J. E. B., AND McKINLEY, K. S. transformations performed by extractors can be adapted to Simple and effective analysis of statically-typed various other kinds of software units by taking into account object-oriented programs. In Proc. of the Eleventh the information contained in MEL specifications• Our ap- Annual Conf. on Object-Oriented Programming proach was implemented in the context of Jax, an applica- Systems, Languages, and Applications (OOPSLA '96) tion extractor for Java [18], and we presented a small case (San Jose, CA, 1996), pp. 292-305. SIGPLAN Notices study that involves several realistic extraction scenarios. 31(10). We intend to support the extraction of other widely used [12] IBM CORPORATION. IBM Smalltalk User's Guide, version 3, release 0 ed., 1995. Chapter library types such as JavaBeans [16]. Other topics for ongo- 36: Introduction to Packaging, Chapter 37: "Simple ing research include adding more sophisticated conditional Paclmging, Chapter 38: "Advanced Packaging. features to MEL such as conditions based on paths in call [13] JACKSON, D., A~D WAINGOLD, A. Lightweight graphs, and boolean conjunction and disjunction of condi- extraction of object models from bytecode. In Proc. of tions. Furthermore, we are considering "safety" features the International Conf. on Soft'ware Engineering such as the insertion of run-time checks to ensure that the (ICSE '99) (Los Angeles, CA), 1999). information in a MEL script is correct and complete. [14] PARcPLAcE SYSTEMS. ParePlace Smalltalk, objectworks release 4.1 ed., 1992. Section 16: Deploying an Application, Section 28: Binary Object Acknowledgements Streaming Service. [15] SOHEDFLER, R. W. An an~ysis of ~nll-e substitution We are grateful to John Field, Harold Ossher, Gregor Snell- for a structured programming language. Commas. ing, and the anonymous FSE referees for comments on drafts ACM ~0, 9 (Sept. 1977), 647-654. of this paper. [16] SUN MICROSYSTEMS. JavaEeans, version 1.01 ed. 2550 Garcia Avenue, Mountain View, CA 94043, July 1997. [17] SWP.~.NEY, P. F., AND TIP, F. A study of dead data 8. REFERENCES members in C++ applications. In Proc. of the ACM [1] AGBSBN, O. Concrete Type Inference: Delivering SIGPLAN'98 Conf. on Programming Language Object-Oriented Applications. Phi) thesis, Stanford U., Desigen and Implementation [PLDI '98)(Montreal, December 1995. Appeared as Sun Microsystems Canada, June 1996), pp. 324-332. ACM SIGPLAN Laboratories Technical Report SMLI TR-96-52. Notices 33(6). [2] AGESEN, O., AND UNGAR, D. Siftingout the gold: [18] TIp, F., LAFFRA, C., SWEENw.Y, P. F., AND Delivering compact applications from an exploratory STREETER, D. Practical experience with an object-oriented programming environment. In Proc. of application extractor for Java. In Proc. of the the Ninth Annual Conf. on Object-Oriented Fourteenth Annual Conf. on Object-Oriented Programming Systems, Languages, and Applications Programming Systems, Languages, and Applications (OOPSLA '9~)(Portland, OR, 1994), pp. 355-370. (OOPSLA '99) (Denver, CO), 1999), pp. 292-305. ACM SIGPLAN Notices 29(10). SIGPLA N Notices 34(10). [3] AIGNBR, G., AND HOLZLE, U. Eliminating virtual [19] TIp, F., AND PALSBF-~G, I. Scalable function calls in C+q- programs. In Proc. of the Tenth propagation-based call graph construction algorithms. European Conf. on Object-Oriented Program In Proc. of the Fifteenth Annual Conf. on (Baooe'98) ((L~, Austria), July 1996), Object-Oriented Programming Systems, Languages, pp. 142-166. and Applications (OOFSLA '00) (Minneapolis, MN), [4] BACON, D. F. Fast and Effective Optimization of 2000). To appear. Statically Typed Object-Oriented Languages• PhD [20] Tin, F., AND SWEBNEY, P. F. Class hierarchy thesis, Computer Science Division, U. of California, specialization. In Proc. of the Eleventh Annual Conf. Berkeley, Dec. 1997. Report No. UCB/CSD-98-1017. on Object-Oriented Programming Systems, Languages, [5] BACON, D. F., AND SWBmNEY, P. F. Fast static and Applications (OOPSLA '97) (Atlanta, GA, 1997), analysis of C++ virtual function calls. In Proc. of the pp. 271-285. ACM SIGPLAN Notices 32(10).
107