Challenging Analytical Knowledge on Exception-Handling

Challenging Analytical Knowledge On Exception-Handling: An Empirical Study of 32 Java Software Packages Martin Monperrus, Maxence Germain de Montauzan, Benoit Cornu, Raphael Marvie, Romain Rouvoy

To cite this version:

Martin Monperrus, Maxence Germain de Montauzan, Benoit Cornu, Raphael Marvie, Romain Rouvoy. Challenging Analytical Knowledge On Exception-Handling: An Empirical Study of 32 Java Software Packages. [Technical Report] hal-01093908, Laboratoire d’Informatique Fondamentale de Lille. 2014. hal-01093908

HAL Id: hal-01093908 https://hal.inria.fr/hal-01093908 Submitted on 19 Dec 2014

HAL is a multi-disciplinary open access L’archive ouverte pluridisciplinaire HAL, est archive for the deposit and dissemination of sci- destinée au dépôt et à la diffusion de documents entific research documents, whether they are pub- scientifiques de niveau recherche, publiés ou non, lished or not. The documents may come from émanant des établissements d’enseignement et de teaching and research institutions in France or recherche français ou étrangers, des laboratoires abroad, or from public or private research centers. publics ou privés. Challenging Analytical Knowledge On Exception-Handling: An Empirical Study of of 32 Java Software Packages

Martin Monperrus Maxence Germain de Montauzan Benoit Cornu Raphael Marvie Romain Rouvoy

Technical Report, Inria, 2014

Abstract In this paper, we aim at contributing to the body of knowledge on exception-handling, by iden- In this paper, we aim at contributing to the body tifying the reasons behind known principles of ex- of knowledge on exception-handling. We take nei- ception handling design. We take neither an an- ther an analytical approach (“we think exception alytical approach (“we think exception handling is handling is good because X and Y”) nor an empir- good because X and Y ”) nor an empirical approach ical approach (“most developers do Z and T”). Our (“most developers do Z and T ”). Our idea is rather method is to compare analytical knowledge against to compare analytical knowledge against empiri- empirical one. We first review the literature to cal one. We hope that from this confrontation will find analytical knowledge on exception handling, emerge principles founded on analytical arguments we then set up a dataset of 32 Java software ap- and validated by empirical practices. plications and an experimental protocol to stati- We first review the literature to find analytical cally characterize and measure the exception han- knowledge on exception handling (Section 2), we dling design. We eventually compare our measures then set up a dataset of 32 Java software appli- against the claims on exception handling that au- cations and an experimental protocol to statically thors have made over time. Our results show that characterize and measure the exception handling some analytical principles for exception design do design (Section 3). We eventually compare our not support the empirical validation: 1) practi- measures against the claims on exception handling tioners violate the principle and 2) upon analysis, that authors have made over time. there are indeed very good use cases going against To sum up, our contributions are: this principle. This is in particular the case for • a survey of analytical knowledge on exception “Empty Catch Blocks are Bad” and “Do not Catch handling, Generic Exceptions”. • a set of empirical facts on the exception handling design of 32 Java libraries, • the validation or falsification of 6 important 1 Introduction claims about good exception handling.

C code handles errors using return codes. Java, as well as other most modern programming lan- 2 Analytical Knowledge About guages has a dedicated construct for errors: ex- Exception Handling ceptions. The core idea behind exception dates back to the 70ies [10] and its implementation in What is analytical knowledge about exception- Java with try/catch/finally is very close to the handling? It is knowledge that is derived from one of Modula designed in the 80ies [19]. However, what one thinks about exceptions from an analyt- there is still no established knowledge on what a ical point of view. Conversely, empirical knowledge good or a bad design of exception handling is. is knowledge that is derived from empirical obser- We can of course find authoritative experts vations on how practitioners use error and excep- who wrote a book chapter or an article on this tion handling. topic [2, 21]. However, many points on this topic In this section, we review what has been said are still debated [16] and you can find on open- by authoritative references about exception han- source forums lively debates (if not flame wars) dling design. We consider as “authoritative” the about exceptions. knowledge described in books, articles published

1 try{ 1 clean exceptional behavior, both in terms of excep- // code 2 tion types and in terms of system layers. All the } catch (IOException ex) { 3 throw new ServerConnectionProblemException( 4 authors agree on this point. Bloch states that this ex); as “higher layers should catch lower-level excep- } 5 tions and, in their place, throw exceptions that can Listing 1: Example of Exception be explained in terms of the higher-level abstrac- Wrapping/Recast in Java tion” [2]. Cwalina recommends “wrapping speciﬁc exceptions thrown from a lower layer in a more appropriate exception, if the lower layer exception in respectful venues and Internet posts authored does not make sense in the context of the higher- by senior respected engineers from known IT com- layer operation” [6]. Wirfs-Brock goes along the panies. Thanks to the Internet, many developers same line and advises to “Recast lower-level excep- assert analytical knowledge on exception handling tions to higher-level ones whenever you raise an based on their own experience1, we do not consider abstraction level” [21]. this non-authoritative knowledge. Exceptions often initiate a debugging session. We propose to classify the analytical knowledge For the sake of debugging, there are two important about exception usages into six categories: the de- guidelines: one should “specify the inner exception sign of catch sites (Section 2.1), the design of throw when wrapping exceptions” [6] as shown Listing 1; sites (Section 2.2), the design of exception classes also, the stack trace of the original exception— (Section 2.3), the design of try blocks (Section 2.4), of primary importance for debugging—should be the design of ﬁnally blocks (Section 2.4). They are never be destroyed (what McCune calls “destruc- ordered by frequency of use. tive wrapping” [14]).

2.1 Catching Exceptions 2.2 Throwing Exceptions When working with a programming language that The second basic activity when working with a offers exceptions, the first thing one learns about language with exception is to throw exception ob- the exception system is how to catch an exception. jects (instances of exception classes), usually when Indeed, many exceptions occurred at development error conditions are met. time, due to the program under construction be- First, let us recall the traditional way of han- ing temporarily incorrect. Those exceptions may dling errors in exception-less programming lan- be thrown by the runtime environment (e.g., a guage, such as standard C. It consists of return- NullPointerException in Java) or by libraries ing error codes or passing mutable error variables (e.g., an IllegalArgumentException). to store them. This has several drawback and ex- We have found many pieces of analytical knowl- ceptions have been invented to cope with them edge about catching exceptions. First, it is rec- (see [6] for a comprehensive discussion). However, ommended that, there should be no empty catch due to habits or education, many developers keep blocks ( “Don’t Catch and Ignore” [14], “Don’t ig- using error-codes even with languages with excep- nore exceptions” [2]). tions. Consequently, it has to be said to “report Second, the scope of the caught exception is execution failures by throwing exceptions” [6] and important, the scope is statically specified by to “use exceptions rather than return codes” [13]. the exception type (“catch (IOException e)”). However, exceptions can be very handy to have Many believe [14, 6] that one should not catch an agile control flow and some use them outside the most generic exception object (“Exception” exceptional cases. Some authors disagree and con- or even “Throwable” in Java): “Do not catch Ex- sider that exceptions should be used “only to signal ception” [14]. emergencies” [21]. Third, there is a known practice consisting of Second, there exist guidelines on the exception translating an exception type into another one, types to be thrown. Standard development li- that is more appropriate, as shown in Listing 1. braries already define a lot of exception classes. This is known as “exception translation” [2], “ex- It has been recommended to “favor the use of ception wrapping” [6] or “exception recast” [21]. standard exceptions” [2, 6]. For instance, Java’s The goal of exception wrapping is to provide a IllegalArgumentException is appropriate to sig- 1See for instance http://c2.com/cgi/wiki? nal that a parameter is incorrect. Also, it has been ExceptionPatterns (last visited: July 2, 2013) advocated not to throw completely generic excep-

2 tions [14]. Designing Finally Blocks Cwalina et al. value Another important point in the literature is finally blocks that can be found in some lan- about how to create a new exception. Cwalina guages, such as Java and .Net. They recommend et al. recommends “to consider using exception them especially for cleanup code [6]. However, builder methods” [6]. Wirfs-Brock warns from they assert that one should not explicitly throw repeatedly re-throwing the same exception [21]. exceptions from finally blocks [6]. When an ex- Both guidelines aim at facilitating debugging and ception is thrown in a finally block, the initial evolution. exception and its stack trace is lost forever and Finally, the guidelines on throwing exceptions this greatly hinders debugging. This also holds for can also be very technical as illustrated by the “return” statements in finally blocks. following recommendations of Cwalina et al.’s book [6]: 1. “Avoid explicitly throwing exceptions from finally blocks”; 2. “Do not have public mem- Designing Try Blocks There are very few dis- bers that can either throw or not based on some cussions on the how to design try blocks. The option”; 3. “Do not have public members that re- only notable guideline is by Google engineers, who turn exceptions as the return value or an out pa- recommend to “minimize the amount of code in a rameter”. try block” [17]. To sum up, much has been analytically said 2.3 Designing Exception Classes about exception design. Nevertheless, what does the reality of programs and programmers look like? Once a programmer is acquainted with throwing and catching exceptions, she may design her own exception classes. Let us now review the main guidelines on designing exception classes. 2.5 Relation between Analytical and First, Java, .Net and other languages provide Empirical Knowledge two kinds of exceptions: checked and unchecked exceptions. The checked exceptions are subject Analytical and empirical knowledge on exception- to compile-time verification. Unchecked are sim- handling are not completely independent. Analyt- ilar to those exceptions found in dynamic lan- ical knowledge about exception handling has an guages. We detail their semantics later (in Sec- impact on practice. tion 3.7). There is no consensus on when and how First, it gives the foundations of the exception- to choose between checked or unchecked excep- handling constructs and semantics of a program- tions. For example, some technology gurus, such ming language. Given their experience and analy- as Robert C. Martin (“Uncle Bob”), recommend to sis, the designers of a programming language make “use unchecked exceptions” [13]. design decisions on how exceptions are specified Second, thrown exceptions are meant to signal and handled (statically and dynamically). This an error and to trigger some kind or recovery (or has an impact on practice: due to the choices of to log for offline analysis). Both use cases require the language designers, some design may be im- detailed information. As such, it is recommended possible to implement given the constructs of the to add state to exception classes (e.g., using class language and the associated compilation errors. fields) [13, 21], which is summarized as “provide Second, thinkers derive guidelines from this context with exceptions” [13]. analytical knowledge about exception handling. Finally, exceptions in object-oriented languages Within a given programming language, those are fully fledged classes and can form exception hi- guidelines aim at helping practitioners to de- erarchies. As Jeff Atwood puts it [1], “designing sign and implement “good” exception handling exception hierarchies is tricky”. Wirfs-Brock pro- (“good” w.r.t. performance, understandability, poses to mitigate the problem by avoiding “declar- reuse, maintenance, etc.). The readers of those ing lots of exception classes” [21]. guidelines may follow them, which is an impact from the analysis to the practice. There may be 2.4 Other Guidelines a relation between the programming language design decisions and guidelines: some guidelines may The last guidelines to be discussed relate to the be workarounds for pitfalls of the exception sys- design of try and finally blocks. tem.

3 try { 1 2 s.update 3 () 4 ; 5 }catch ( 6 7 GeneralSecurityException 8 e 9 ){ 10 throw new DNSSECException( 11 12 e.toString 13 () 14 ) 15 ; 16 } 17 18 Listing 3: The XML tree extracted from Listing 2 using srcML

try{ 1 3.2 Experimental Protocol s.update(); } 2 catch (GeneralSecurityException e) { 3 We now present the process we follow to stat- throw new DNSSECException(e.toString()); }4 ically analyze the exception handling design of Listing 2: An excerpt of DNSJava that throws an Java libraries. First, the abstract syntax tree exception (AST) of a corpus of Java programs are extracted and persisted as XML trees. Second, those trees are queried and analyzed using the XQuery lan- 3 Empirical Findings on Excep- guage [3]. tion Handling 3.2.1 Dataset We now set up an experiment to observe, characterize, and measure the exception handling design We build a dataset of Java program as follows. in Java libraries. Our protocol consists in stati- The inclusion criteria is to maximize the diver- cally studying the source code of Java software. sity of application domains and development pro- We select 6 design principle for exception han- cesses. It goes from parts of the standard Java dling, devise measures for them and assess whether library (java-regexp, java-io) to database appli- those principles hold in practice. This process en- cations (h2) and desktop applications (columba). ables us to validate or invalidate analytical knowl- Some programs come from well-known and estab- edge. Furthermore, our experiments also confirm lished development organizations (Apache Com- some existing findings through replication. mons Collections, Apache Lucence) and other are more specific (FraSCAti [20] is a middleware plat- 3.1 Challenged Analytical Principles form developed in our software engineering research group for several years). There is a to- Based on our analysis of analytical knowledge tal of 32 Java libraries in our dataset. These (see 2), we select 6 analytical principles that are are argouml, avrora, batik, carol, columba, com- prominent in the literature: monscollections, DNSJava, fop, foray, FraSCAti, h2, 1. “Empty catch blocks are bad”, itext, java-regexp, java-util, jboss, jedit, jface, jhot- 2. “Reuse Standard Exceptions”, draw, jUnit, jython, log4j, lucene, org-eclipse-jdt- 3. “Define Exceptions With State”, core, org-eclipse-ui-workbench, pmd, rhino, Scarab, 4. “Use Checked Exceptions”, solr, Struts, sunflow, tomcat, xalan. This dataset is 5. “Consider Exception Builder Methods”, available as auxiliary material. 6. “Do not Catch Generic Exceptions”. We choose those principles because they are ei- 3.2.2 Implementation ther emphasized by different authors (e.g., “Empty catch blocks are bad”) or they resonate with our ex- Transformation of Java Source Code into perience and discussions with our fellow program- XML We extract the ASTs of java libraries using mers. srcML [5]. An entire program consisting of dozens

4 of Java files is transformed into a singe XML file. int port = 8080;// default value 1 In our dataset, those XML files contain between try{ 2 port = loadPortFromConfigFile(); 3 105, 000 and 4, 826, 000 nodes and weight between } catch (Exception ex) { 4 1, 172 and 52, 893 kilobytes. Listing 3 shows an // keeps the default value 5 } 6 excerpt of Java source code and the corresponding openSocket(port); 7 XML representation. The snippet contains a try Listing 4: Empty catch blocks to support setting and a catch block, both are converted into the appropriate XML tags. default values

Analysis of XML Abstract Syntax Trees To explore the exception handling design of real Java Strategy o = createDefaultStrategy(); 1 libraries, we query the XML abstract syntax trees try{ 3 using the XQuery language. This language, com- o = createBetterStrategy(); 4 } catch (Exception ex) { 5 bined with XPath selectors, enables to quickly ob- // keeps default strategy 6 tain interesting metrics. For instance, the number } 7 of catch blocks containing a throw statement is try{ 9 expressed as count(//catch//throw). Listing 2 o = createBestStrategy(); 10 } catch (Exception ex) { 11 would match this selector. // keeps default or better strategy 12 } 13 3.3 Overview of the Java Exception Listing 5: Empty catch blocks for implementing a System strategy design pattern Since we analyze Java libraries, this section briefly introduces the Java exception handling system. In Java, an exception is “thrown” (or “raised”, using several catch clauses, the developer can dis- we equate those terms) when a problem arises. tinguish the treatment to be applied upon error For example, a NullPointerException indicates depending on the type of exception thrown by the that an access is performed to a variable that system. does not contain any reference to a valid object, or IllegalArgumentException is thrown when a In addition, the keyword finally can be ap- method parameter is invalid. pended at the end of a try block (whether there Some exceptions are thrown by the is no, one or multiple catch blocks) to describe underlying Java Runtime Environment a body of statements that are always executed (JRE) (e.g., NullPointerException, (whether an exception is thrown or not). ArrayIndexOutOfBoundsException). The others exceptions are also explicitly thrown from the application code, by using the keyword throw. Note that developers can also man- 3.3.1 Presentation Template ually throw JRE exceptions (e.g., throw new NullPointerException(). There are two categories of exceptions in Java: For all the challenged design principles, we use the checked and unchecked. Checked exceptions re- same template for presentation: definition, analyt- quire to be declared in the associated method sig- ical knowledge, previous empirical knowledge, our nature if they can be thrown, but are not caught empirical results, discussion. within the body of this method. The idea of The definition explains the content and ratio- checked exception is to force the developers of nal of this principle. The analytical knowledge client code to handle foreseeable errors. summarizes the arguments presented in Section 2. Catching an exception is achieved by the pair The previous empirical knowledge gives empirical of keywords try and catch. The block associated figures about this claim (e.g., abundance), which to the try (a try block) describes a sequence of we found in previous papers. Then, we present instructions that can potentially throw an excep- out empirical results measured on the dataset pre- tion, while the block associated to the catch refers sented in 3.2.1. Finally, a discussion synthesizes to the instructions to be executed whenever a spe- the pros and cons of the principle and whether it cific exception is thrown within the try block. By has supported the confrontation with reality.

5 // in./org/eclipse/jface/viewers/deferred/ 1 Discussion According to Bloch’s statement, BackgroundContentProvider.java empty catch blocks should be rare. In practice, while(true){ 2 try{ 3 they often occur up to 1 catch block out of 3 // this is the main work 4 (for Eclipse JDT). This empirical reality suggests doSort(sortingProgressMonitor); 5 } catch (Exception ex) { 6 that there exist good reasons to write empty catch // ignore 7 blocks. To further understand the meaning and } 8 } 9 the relevance of empty catch blocks, we have ran- domly browsed dozens of empty catch blocks. We Listing 6: Excerpt of an empty catch block for now present the result of this grounded approach resilience in Eclipse JFace and discuss usages of empty catch blocks that we consider as proper design: falling back to default values (see Listing 4), trying different strategies 3.4 Challenging “Empty Catch Blocks (see Listing 5), and resilience (see Listing 6). are Bad” Falling back to a default value means that if the computation fails, there is a meaningful best-effort Definition An empty catch, as its name sug- value to continue the execution. There are many gests, contains no code. Upon execution, an empty variants of this, such as assigning the default value catch stops the propagation of an exception and inside the catch block or returning a default value leaves the program state as is. when in a method, Listing 4 presents such a catch block. Analytical Knowledge Bloch says “don’t ig- Trying different strategies consists in variations nore exception” (in the sense of not silencing ex- of the chain of responsibility design pattern where ceptions with empty catch blocks) [2, p. 258] and different alternatives are available for the same Tim McCune goes along the same line [15]. Bloch’s task. For instance, Listing 5 shows several state- arguments are sharp: “The advice in this item ap- ments that are chained in order to try several plies equally to checked and unchecked exceptions. strategies. The empty catch block that stops the Whether an exception represents a predictable ex- exception of a strategy enables one to keep the ceptional condition or a programming error, ignor- previously successfully computed object. ing it with an empty catch block will result in a pro- By resilience, we mean that an exception may gram that continues silently in the face of error. crash the current action being performed, but not ” However, Bloch may admit some empty catch the whole application. For instance, a request to blocks when they “contain a comment explaining server may fail, but should not crash the applica- why it is appropriate to ignore the exception.”. tion. An application of such a resilience scenario is shown in Listing 6. Similarly, in a loop over el- Previous Empirical Knowledge Reimer [18] ements, an exception may occur for one element observed that between 2% and 26% of catch blocks but should not crash the rest of the computation are empty in the JDK and in 6 closed-source appli- being performed (not presented for sake of space). cations. Fu and Ryder [7] confirmed that between In both cases, an empty catch block is relevant. 5% and 40% of catch blocks are empty handlers Empty catch blocks can be used to transform (out of 2099 catch blocks from 5 applications). exception-oriented error-handling into return- oriented error-handling as shown in Listing 7. Finally, there exists hidden empty catch Our Empirical Results In our dataset, all blocks—i.e., programming designs or idioms that (32/32) libraries contain empty catch blocks, up to are in essence equivalent to empty catch blocks. hundreds of times. For instance, Eclipse’s JDT-core For instance, using a new thread to perform an ac- has 423/1430 (30%) of them and Tomcat 309/2348 tion is equivalent to an empty catch: when an ex- (13%). This further confirms the aforementioned ception occurs, the newly created thread crashes, empirical results [18, 7]. We have refined the but not the whole application. Hence, applications measurement to count empty documented catch with many short-lived threads, such as GUIs with blocks (this measure has never been computed thread-based event handling, have a kind of built- before). For instance, iText contains 118 empty in resilience due to those many “implicit empty catch blocks, 43 of them being documented. Some catch blocks”. libraries document catch blocks very systematically, up to 100% (423/423 for JDT-core).

6 // in org/mozilla/javascript/Kit.java 1 Analytical Knowledge Cwalina and try{ 2 Abrams [6] promote reusing standard excep- return loader.loadClass(className); 3 } catch (ClassNotFoundException ex) { 4 tions. In particular, they advise to “consider } catch (SecurityException ex) { 5 throwing existing exceptions residing in the } catch (LinkageError ex) { 6 } catch (IllegalArgumentException e) { 7 System namespaces instead of creating custom // Thrown if className has incorrect 8 exception types (esp for usage errors)”. This characters } 9 means adopting, whenever it is considered as return null; 10 appropriated, the standard exceptions that are provided by the underlying system (e.g., the .Net Listing 7: Empty catch blocks in Rhino or Java runtime environments). According to can convert exception-oriented error-handling in them, developers should “create and throw custom return-oriented error-handling exceptions if [they] have an error condition that can be programmatically handled in a different way than any other existing exception. Otherwise, Empty catch blocks are much used in prac- [they should] throw one of the existing excep- tice. When the global application state is cor- tions.” This means that before creating her own rect, stopping the exception propagation is ap- exception, the developer should look at whether propriate and this is what empty catch blocks some the exception abstractions provided by the do. libraries at hand can be meaningfully applied. In any case, they advise to “not create and throw new exceptions just to have ýour team’s´ Bloch’s sharp reject of empty catch block was exception.” All these rules are acknowledged by not completely right with regards to empirical ev- Bloch in [2], who recommends to “favor the use idence: we would reformulate “don’t ignore excep- of standard exceptions.” Indirectly, by claiming to tions” as “consider using empty catch blocks when “avoid declaring lots of exception classes”, Wirfs- the global application state is correct”. To con- Brock [21] advises the developers to carefully clude, the empirical evidence has seriously chal- design custom exceptions, an objective that can lenged the analytical belief that “empty catch be achieved by reusing standard exceptions. blocks are bad”. We have presented a list of scenar- ios in which they are indeed relevant and meaning- Previous Empirical Knowledge We do not ful. have identified any empirical studies on throwing standard exceptions. 3.5 Challenging “Reuse Standard Ex- ceptions” Our Empirical Results In our dataset, all Definition A standard exception is an ex- (32/32) applications throw standard exceptions ception whose definition is provided by a within their code and for 22 of them (73%) the third-party software library, including the most thrown exception is a standard exception. standard runtime environment, such as the For instance, in iText, the most commonly thrown JDK in Java. Examples of standard excep- exception is an IllegalArgumentException tions include IllegalArgumentException and (168/881 thrown exceptions, 19%). More IllegalStateException. Such exceptions are specifically, one can observe that standard usually caught and handled by the applications exceptions like IllegalArgumentException, building on these libraries. IllegalStateException, and IOException are However, “reuse standard exceptions” actually all thrown by the libraries of our dataset. focuses on throwing exceptions and not catching them. Indeed, standard exceptions can also be Discussion Among the standard exceptions thrown by application code whenever their seman- that are reused, our experience with browsing tics matches the context of execution. For in- exception-handling code reveals specific cases. stance, in Java, when an incorrect parameter is First, the standard exceptions Exception and passed as an argument of a method call, it is mean- Throwable are used to report error messages to ingful to throw an IllegalArgumentException, as the end-user (see Listing 9). Second, the stan- shown in Listing 8, which reports such an example dard exception RuntimeException is mostly used from our dataset. to convert a checked exception into a unchecked

7 // in java/org/ow2/carol/jndi/ns/AbsRegistry. 1 For example, a IllegalParameterException can java be defined with a field that contains the name of public void setPort(int p){ 2 if (p <= 0) { 3 the incorrect argument. throw new IllegalArgumentException("The 4 number for the port is incorrect. It must bea value> 0. Value was’"+p Analytical Knowledge Martin says “provide +"’"); } 5 context with exceptions” [13, p. 107]. Wirfs-Brock this.port = p; 6 goes along the same line: “provide context along } 7 with an exception” [21]. Martin says that addi- Listing 8: Reuse of the standard exception tional data should always be present: “Each excep- IllegalArgumentException to capture non- tion that you throw should provide enough context authorized parameters. to determine the source and location of an error. In Java, one can get a stack trace from any exception; however, a stack trace can not tell you the // in src/java/org/apache/fop/render/rtf/ 1 rtflib/tools/BuilderContext.java intent of the operation that failed.” public void replaceContainer(RtfContainer oldC2 Bloch is more specific about the nature of the , RtfContainer newC) throws Exception { // treating the Stack asa Vector allows 3 additional information: “Include failure-capture such manipulations(yes,I hear you information in detail messages” [2, p. 254]. In par- screaming;-) ticular: “the detail message of an exception should final int index = containers.indexOf(oldC); 4 if (index < 0) { 5 contain the values of all parameters and fields that throw new Exception("container to replace 6 "contributed to the exception."” not found:" + oldC); } 7 As we can see, all authors agree that the stack containers.setElementAt(newC, index); 8 trace is insufficient and that additional data is re- } 9 quired. Listing 9: Exception used to report an error. Previous Empirical Knowledge We do not have found anything on this topic. one, we will come back on this point in 3.7.

Our Empirical Results In our experiment, we All in all, the design principle “Use Checked consider that an exception has a state if and only if Exceptions” is much applied in our dataset. it contains at least one additional significant field2. There is a clear match between analytical and In our dataset, there are 389 domain specific ex- empirical knowledge, which validates the prin- ceptions. Among them, we found 112 exceptions ciple. with additional fields. Also, we notice that in practice, many exception definitions duplicate already existing information. For example, 44 exceptions 3.6 Challenging “Define Exceptions have the cause exception as a field (already present With State” in Throwable), 2 exceptions duplicate the message (idem) and 1 explicitly defines its stack trace (ibi- Definition An exception with state is an excep- dem). All this data being already present in a ba- tion that contains additional data on the failure sic exception, these exceptions are not considered cause or the failure context. This additional data as exception with state. is usually encoded as fields in the exception object. On those 112 exceptions with state, 101 exceptions contain additional data directly related to the context where the failure occurs. For instance, // in src/org/argouml/util/MyTokenizer.java 1 public void putToken(String s) { 2 Listing 11 reports an excerpt of a stateful excep- if (s == null){ 3 tion: the immutable exception includes data about throw new NullPointerException("Cannot put4 a null token"); when a signature expired (field when) and when } 5 this signature has been called (field now). putToken = s; 6 } 7 Also, there are 11 exceptions that contain information on the type of error encoded as an error Listing 10: NullPointerException used to capture an illegal argument. 2The Java specific field serialVersionUID is not considered as an additional field

8 class SignatureExpiredException extends 1 class ClassFormatException extends Exception {1 DNSSECException { public static final int ErrBadMagic = 1; 2 /* When the signature expired*/ 2 public static final int ErrBadMinorVersion =3 private Date when; 3 2; /* When the verification was attempted*/ 4 public static final int ErrBadMajorVersion =4 private Date now; 5 3; ... 5 SignatureExpiredException(Date when, Date 7 public static final int 6 now ) { ErrInvalidMethodSignature = 28; super("signature expired"); 8 this.when = when; 9 private int errorCode; 8 this.now = now; 10 public ClassFormatException(int code) { 9 } 11 this.errorCode = code; 10 } 12 } 11

Listing 11: Domain exception with state from /** 13 DNSJava * @return int 14 */ 15 public int getErrorCode() { 16 return this.errorCode; 17 } 18 code value. Listing 12 shows a example of such an } 19 exception. Listing 12: Domain exception with error-code from jdt-core Discussion We observe that less than 30% domain-specific exceptions (112/389) define some state. This shows that the design principle “define nature. exceptions with state” does not hold systematically. Figure 1 shows the number of exceptions with state and the number of stateless exceptions for Analytical Knowledge The authors of the each project. Only 1 project (java-regexp) contains Java language specification [11] say that “Most 100% of stateful exceptions (in total 1 domain- user-defined exceptions should be checked excep- specific exception) where 8 projects do not contain tions. ” because “compile-time checking [..] aids any exception with state. The distribution of the programming in the large”. exceptions with state does not seem to be corre- Bloch [2] recommends to “use checked exceptions lated with the project size. For example, fop and for recoverable conditions and runtime exceptions java-util have the same number of domain-specific for programming errors” (item 40) and to “avoid exceptions (20 exceptions defined) but fop has only unnecessary use of checked exceptions” (item 41). 4 exceptions with state (20%) where java-util has Both recommendations are related to a known de- 12 (60%). The distribution does not seem either bate on first, whether Java checked exceptions are linked with the reputation of the organization be- good or not [16], and what should be the design hind each library. Indeed, the well-known and used principles to choose between one form and the tomcat has 15% of exception with state, the fa- other [9]. mous Eclipse’s jdt-core has 22% while the much We can list many other documents, either in fa- less known libraries and Avrora have respectively vor or against checked exceptions. Indeed, as the 57% and 68% of stateful exceptions. Java language designers themselves put it, checked These figures show that Bloch, Martin, and exceptions are sometimes considered as “an irrita- Wirfs-Brock’s analytical design principles do not tion to programmers” [11]. really find their ways in real projects. For unrecoverable checked exceptions, the programmers have 2 choices: either they write a fake 3.7 Challenging “Use Checked Excep- try-catch block to simulate a recovery block or tions” they declare the checked exception in the enclosing method signature. The former is a known practice Definition The Java language defines two kinds that consists of catching checked exceptions and of exceptions: checked and runtime exceptions wrapping them in runtime exceptions, as shown in (also called unchecked exceptions) [11, Chap- Listing 13. Let us call this implementation pattern ter 11]. The checked exceptions are subject to the “checked-runtime wrapping pattern”. compile-time verification. The verification ensures The other solution, explicitly declaring the ex- that raised checked exceptions are either handled ception that can be thrown is called the “excep- or explicitly declared in the enclosing method’s sig- tion declaration pattern”. When developers declare

9 ensmr hce xetosta utm ex- and total, runtime than ceptions In exceptions checked more exceptions. defines use runtime to prefer or dataset checked our from libraries software of exceptions. runtime and checked of usage Results Empirical Our thrown. is exception checked versus a thrown is where excep- exception runtime runtime are a there where that thrown: often shows more are also tions He there that exceptions. asserted and 1.4.1 are JDK the analyzed Knowledge Empirical Previous the of mechanism compiler. checking Java exception the disable sense. make grain fine would Java the that of think declaration exception, not checked exception do they of that type means it specific a declaring of Exception throws For Workaround Classical The Exceptions Checked 13: Listing } {...} try is IOException // { e) IOException ( catch iue1 ubro xetoswt tt n h ubro ttls xetosfrec project each for exceptions stateless of number the and state with exceptions of Number 1: Figure Rniexeto (e); RuntimeException new throw is,w aeloe twehrprogrammers whether at looked have we First, to recipes empirical somehow are patterns Both # Exceptions 50 checked 10 15 20 25 30 35 40 0 5 endrnieecpin and exceptions runtime defined

Struts foray 12

/ Java-regexp 32 Carol nmto intrsinstead signatures method in

ensmr utm excep- runtime more deﬁnes commonscollections log4j jhotdraw

ehv esrdthe measured have We Columba jface Scarab solr 150 3 jboss iiy[12] Kiniry ,

000 org-eclipse-ui-workbench

checked rhino 15 2 times

, jython / 650

32 argouml-app 1 4 3 2 10 jUnit bei tlatoemto.Temxmmvalue maximum IOException The for method. one is least at do in projects able declare All projects exist. All not in- does that catch an one ﬁle when when a instance, happens opens for It fails, operation JDK. put/ouput the of exception IOException in- Exception of throws number in maximum is The stances pattern. laration 27 for pattern Exception declaration exception Exception the of stances in- of 263 number maximum in The is stances pattern. this throwing once consists and pattern exception RuntimeException: checked this a the of catching version of basic most pattern. wrapping The checked-runtime the of stances ex- 9 the On and tions runtime). as Eclipse’s checked tremes, de- many remaining as (the clare exceptions checked than tions pmd utm xetosotof out exceptions runtime ial,w osdrtecnrt xml of example concrete the consider we Finally, / in- of number the measured have we Third, in- of number the measured have we Second,

32 h2 e utm exceptions. runtime new jedit IOException rjcsuea es neteecpindec- exception the once least at use projects Lucene FraSCAti 3

hti h ubro times of number the is that , batik sdcae ntemto signature. method the in declared is hce xetos and exceptions, checked pceLucene Apache . DNSJava : hc stems omnchecked common most the is which Scarab itext 4765 jdt-core fop ntersignature. their in

u to (up java-util where IOException 26 ehd elr throwing declare methods xalan deﬁnes / org-eclipse-jdt-core 32 h oebs throws base code the : 10 456 avrora 525 rjcsuea least at use projects tomcat oanexceptions. domain 19 ie in times

ehd declare methods lucene utm excep- runtime ob throw- be to Rhino withstate withoutstate Lucene throws defines ). Discussion Checked exceptions are used in // in DNSJava 1 practice: programmers indeed define and throw throw parseException(fullName,"Name too long" 2 ); checked exceptions. However, there are also two major pieces of evidence that checked exceptions // in Jython 4 throw makeException(t); 5 are questionable. First some projects deliberately only use run- // in Foray 7 throw unexpectedValue(value, fobj); 8 time exceptions. They do not seem to be written throw this.unexpectedRetrieval(); 9 by novice uneducated developers. In our dataset, // in H2 11 there are 6 software projects hosted by the Apache throw convertException(e); 12 foundation that prefer runtime exceptions. The throw throwUnsupportedExceptionForType("+"); 13 Apache foundation is known to gather very good throw getUnsupportedException(); 14 developers and to foster best-practices. Further- Listing 14: Examples of the Usage of the more, Eclipse jdt-core also much favors runtime ex- Exception Builder Pattern ceptions. This is very interesting: the developers of a Java compiler, who are aware of all subtleties of the language, who have handled thousands of 3.8 Challenging “Consider Exception compiler bug reports, prefer to only use runtime Builder Methods” exceptions. Definition An exception builder method is a helper method to obtain exception objects. It Our results challenge the statement that is a variant of the builder pattern [8]. In- “compile-time checking [..] aids programming stead of using a throw new A(...) the de- in the large”: many respectful developers do veloper invokes an helper in order to in- not think so. stantiate the exception to be thrown, e.g. throw helper.raiseException(...).

Second, we have shown many traces of Analytical Knowledge Kwalina recommends workarounds to trick the static exception check- “using exception builder methods.” The main rea- ing mechanism (in the form of the checked-runtime son is to avoid code bloat for the configuration of wrapping pattern), both in terms of number of the exception objects. projects using the pattern and total number of occurrences. This means something with respect Our results In our dataset, 19/32 libraries con- to library design. When one defines checked ex- tain usages of the exception builder pattern. Still, ceptions within one’s own project, one is the only some projects do not use this pattern at all, this is impacted by this design choice. When one de- the case for Apache Commons collections and java- clare checked exceptions in a library interface, all utils. Then, the frequency of usage ranges from ex- clients are impacted. Our empirical study gives ceptional (less than 1%) in projects such as Eclipse numerical values of this impact: there are hun- jdt-core, Jboss, or Tomcat and raises up to 60.5% dreds of times where developers catch a poten- in Java-regexp, 81.7% in Jython, and 81.8% in H2 tial checked exception declared by a library and code-bases. Jython and H2 are particularly inter- wrap them as a runtime. For instance, there is esting as they are following the design principle a total of 1034 catch elements of IOException very systematically. which re-throw a fresh runtime exception (in our dataset). If IOException had been a runtime exception, many workarounds and similarly irritat- Discussion We have identified 3 forms of using ing code would have been avoided. the exception builder method pattern. First, the object where the exception occurs em- beds the builder method. This follows the single There is a difference between inner- responsibility principle in the scope of the object. application checked exceptions and library A method centralizes the exception building mech- checked exceptions. The latter can induce anism for a given class. Examples from 4 different workarounds in thousands of methods of client projects are shown in Listing 14. code. Second, a utility class provides some static methods for exception building. This centralized

11 // Jython 1 but this is implementation-specific. In this paper, throw Py.TypeError("__dict__ must be set toa 2 when we use Exception, we refer to the conceptual Dictionary"+newDict.getClass().getName()) ; most generic kind of exception). Listing 15: Examples of the Usage of the Exception Builder Pattern as Static Method Analytical Knowledge McCune asserts “Don’t catch Exception”. Cwalina qualifies this asser- tion and adds a distinction between framework // Jython(info is the injected object) 1 code and application code: “Do not swallow errors throw info.unexpectedCall(args.length, true); 2 by catching non-specific exceptions in framework //DNS Java(st is the injected object) 4 code”, “Avoid swallowing errors by catching non- throw st.exception("unexpected tokens at end 5 specific exceptions in application code”. Here the of record"); key difference lies in the “do not” versus “avoid”. A Listing 16: Modularizing exception building using search on the Internet indicates that “Don’t catch dependency injection Exception” is a kind of urban legend, as witnessed by the wiki page of Ward Cunningham’s website3. class is accessible from anywhere in the project. Previous Empirical Knowledge Cabral and As shown in Listing 16, Jython uses this pattern Marques [4] give the percentage of generic catch for managing the building of Python exceptions. blocks (catch(Exception e)) for 4 kinds of sys- Third, an injected dependency provides some tems (libraries, server applications, server and exception related methods. This elegant design stand alone applications) in 2 different program- allows the replacement of the exception building ming languages (.Net and Java). According to mechanism by another one when necessary. In ad- their experiments, in .Net, the generic exception dition, a second benefit of this form is to permit type is the most common caught exception for all the use of exception mocks for testing purpose. 4 kinds of systems. It even accounts for more than To our knowledge, we are the first to provide 50% of all catch blocks in standalone applications. an empirical characterization of the usage of ex- In their dataset of Java applications, the generic ception builders. We note that projects that use exception types (Exception and Throwable) are exception builder method pattern do not limit its ranked #2 in the top caught exception list. implementation to a particular form, but mixes them. H2 uses (1) and (2), DNSJava uses (1) and Our Empirical Results In our dataset all 32 li- (3), Jython uses all three forms. braries catch at least once Exception. In addition, 25/32 catch at least once Throwable. They are no Not all projects use a form of exception lonely catch blocks written by novice developers builder, but when it is done, it gives birth to and uncaught by a code review or the continuous very advanced design. checking system: In Tomcat, they are 475 catch blocks (out of 2, 348, 20%) catching Exception. Tomcat is not an exception, there are 13 libraries for which we found more than 100 generic catch 3.9 Challenging “Do not Catch Generic blocks. Exceptions” Discussion Many developers write generic catch Definition In most mainstream object-oriented blocks. Qualifying this claim by our dataset, a programming languages, the exceptions use the stronger argument is that many respectful devel- same type system as classes. Consequently, one opers of high-quality open-source projects write can define a hierarchy of exceptions. This has generic catch blocks. The unqualified “Don’t catch very important consequence, when a try-catch exceptions” is very much challenged. If one trusts block declares to catch an exception type t, it so many expert developers, one accepts that catch- actually catches exception instances of all sub- ing generic exceptions is sometimes a good design classes of t. In other terms, when one writes principle. The question that arises is: when is it catch(Exception e), all exceptions are caught, good to catch the most generic type of exceptions? when one writes catch(FileNotFoundException Over the course of our experiments, we analyzed e), only specific exceptions are caught (In Java, the most generic type of exception is Throwable 3http://c2.com/cgi/wiki?DontCatchExceptions

12 many such generic catch blocks. We discovered // in Tomcat’s JspRuntimeLibrary.java 1 two main reasons to have catch generic exceptions. Object convert(String propertyName, String s, 2 Class t, Class propertyEditorClass) throws First, the resilience exception handling pattern JasperException { involves catching the most generic exception type. try{ 3 // 39 lines of code 4 We have presented and discussed this pattern } catch (Exception ex) { 5 above in Section 3.4. This pattern indeed involves throw new JasperException(ex); 6 } 7 a caught exception type that is generic. } 8 Second, catching Exception is useful for ensur- Listing 17: A Real World Example of the All-in- ing a strong and interesting exception contract. one Exception Contract. This method guarantees An exception contract is contract on a code ele- that if an exception is thrown, it will be of type ments related to exceptions. For instance, in Java JasperException and related languages, checked exceptions relates to a contract on methods: “no checked exceptions will ever be raised by calling this method”. First, some analytical principles for exception We discovered that catching Exception is nec- design do not support the empirical validation: essary to provide the following exception contract: 1) practitioners violate the principle and 2) upon “if an exception is thrown, it will be of type X”. analysis, there are indeed very good use cases go- We call this contract the “all-in-one exception con- ing against this principle. This is in particular the tract” (for referring to the idea that all exception case for “Empty Catch Blocks are Bad” and “Do types are translated in one single exception type). not Catch Generic Exceptions”. This contract can be implemented as follows: 1) Second, our study has reveals specific exception the complete method body is wrapped in a try- design patterns, such as resilience and the all-in- catch, 2) the catch block catches the most generic one exception contract. They all contribute to the exception type, 3) the catch block throws an ex- body of knowledge on exception handling. ception of type X (and optionally wraps the caught exception inside the thrown exception). For instance, Listing 17 shows a method of the References Apache Tomcat server which implements this contract. By construction, this method guarantees [1] J. Atwood. Creating more excep- that if an normal exception is thrown4, it will be tional exceptions. http://www. of type JasperException. codinghorror.com/blog/2004/10/ Note that this contract is different from the creating-more-exceptional-exceptions. , 2004. checked exception contract. Adding “throws html JasperException” says that “there exist situ- [2] J. Bloch. Effective Java. Addison-Wesley, ations where such a JasperException can be 2001. thrown”. On the contrary, a well-implemented all-in-one exception contract ensures that only [3] S. Boag, D. Chamberlin, M. F. Fernández, JasperException can be thrown. D. Florescu, J. Robie, and J. Siméon. Xquery To sum up, we empirically found two cases in 1.0: An xml query language. W3c working which catching the most generic exception type draft 04 april 2005, W3C, 2005. http://www. makes sense and contradicts the bold statement w3.org/TR/xquery/. that one should not catch Exception. To our [4] B. Cabral and P. Marques. Exception han- knowledge, we are the first to unveil this part of dling: A field study in java and. net. In exception handling design. ECOOP 2007–Object-Oriented Programming, pages 151–175. Springer, 2007. 4 Conclusion [5] M. L. Collard, H. H. Kagdi, and J. I. Maletic. An XML-Based Lightweight C++ Fact Ex- In this paper, we have studied the exception han- tractor. In Proceedings of the 11th IEEE In- dling design of 32 Java libraries. The outcome of ternational Workshop on Program Compre- this study is twofold. hension, pages 134–143, 2003.

4We explained above that we discard the language- [6] K. Cwalina and B. Abrams. Framework De- speciﬁc and conceptually irregular Java “Error” sign Guidelines: Conventions, Idioms, and

13 Patterns for Reuseable .NET Libraries. Mi- net/pub/a/today/2006/04/06/exception- crosoft .NET development series. Addison- handling-antipatterns. html, accessed, 2006. Wesley, 2008. [15] T. McCune. Exception-handling an- [7] C. Fu and B. G. Ryder. Exception-chain tipatterns. http://today.java. Analysis: Revealing Exception Handling Ar- net/pub/a/today/2006/04/06/ chitecture in Java Server Applications. In exception-handling-antipatterns.html, Proceedings of the International Conference 2006. on Software Engineering, 2007. [16] Oracle. Unchecked exceptions: The [8] E. Gamma, R. Helm, R. Johnson, and J. Vlis- controversy. http://docs.oracle.com/ sides. Design Patterns: Elements of Reusable javase/tutorial/essential/exceptions/ Object-Oriented Software. Addison-Wesley runtime.html. Professional Computing Series. Addison- Wesley Publishing Company, New York, NY, [17] A. Patel, A. Picard, E. Jhong, J. Hyl- 1995. ton, M. Smart, and M. Shields. Google python style guidelines. http: [9] B. Goetz. Java theory and prac- //google-styleguide.googlecode.com/ tice: The exceptions debate. http: svn/trunk/pyguide.html, Rev 2.54. //www.ibm.com/developerworks/java/ library/j-jtp05254/index.html, 2004. [18] D. Reimer and H. Srinivasan. Analyzing Ex- ception Usage in Large Java Applications. In [10] J. B. Goodenough. Exception handling: Is- Exception Handling in Object Oriented Sys- sues and a proposed notation. Commun. tems: Towards Emerging Application Areas ACM, 18(12):683–696, 1975. and New Programming Paradigms, pages 10– 19, July 2003. [11] J. Gosling, B. Joy, G. Steele, and G. Bracha. Java Language Specification. Addison-Wesley, [19] P. Rovner. Extending modula-2 to build 3rd edition, 2005. large, integrated systems. Software, IEEE, 3(6):46–57, 1986. [12] J. R. Kiniry. Exceptions in java and eiffel: Two extremes in exception design and appli- [20] L. Seinturier, P. Merle, R. Rouvoy, cation. In Advanced Topics in Exception Han- D. Romero, V. Schiavoni, and J.-B. Ste- dling Techniques, pages 288–300. Springer, fani. A Component-Based Middleware 2006. Platform for Reconfigurable Service-Oriented Architectures. Software: Practice and [13] R. C. Martin. Clean Code: A Handbook of Experience, 42(5):559–583, May 2012. Agile Software Craftsmanship. Prentice Hall PTR, Upper Saddle River, NJ, USA, 2008. [21] R. H. Wirfs-Brock. Toward Exception- Handling Best Practices and Patterns. IEEE [14] T. McCune. Exception handling an- software, 23(5):11–13, Sept. 2006. tipatterns. Online: http://today. java.