IEEE TRANSACTIONS ON SOFTWARE ENGINEERING 1 Class Schema Evolution for Persistent Object-Oriented Software: Model, Empirical Study, and Automated Support Marco Piccioni, Member, IEEE, Manuel Oriol, and Bertrand Meyer Abstract—With the wide support for object serialization in object-oriented programming languages, persistent objects have become common place and most large object-oriented software systems rely on extensive amounts of persistent data. Such systems also evolve over time. Retrieving previously persisted objects from classes whose schema has changed is however difficult, and may lead to invalidating the consistency of the application. The ESCHER framework addresses these issues through an IDE-integrated approach that handles class schema evolution by managing versions of the code and generating transformation functions automatically. The infrastructure also enforces class invariants to prevent the introduction of potentially corrupt objects. This article describes a model for class attribute changes, a measure for class evolution robustness, four empirical studies, and the design and implementation of the ESCHER system. Index Terms—versioning; persistence; serialization; object-oriented class schema evolution; IDE integration ✦ 1 INTRODUCTION scribes previous approaches to class schema evolu- ETWEEN the time an object-oriented program tion. Section 4 defines a formal model representing B writes objects to persistent storage and when atomic attribute changes through a set of transforma- another execution retrieves them, the program may tion functions, and suggests a definition of a measure have evolved; in particular, classes describing these for evolution robustness. The empirical studies in Sec- objects may have changed. The need to fit old objects tion 5 are essential to understand what kind of schema into new classes is the problem of schema evolution evolution actually happens in practice; without empir- for object-oriented software. While many solutions ical evidence, proposed solutions to the schema evo- have been suggested and are currently used, none has lution problem are bound to be off the mark. The first been to our knowledge generally accepted. study analyzes the evolution of serializable classes in The techniques used to cope in practice rely heavily five versions of the java.util package. It highlights on manual effort by the developers, who must exam- the acuteness of the problem by showing that 17.7% ine previous class versions and provide conversion of the changes affect the persistence of the serializa- code. This approach is not only tedious but a threat ble classes, and justifies the set of atomic attribute to software reliability, as it usually relies on tolerant changes considered in the model. The measure for arXiv:1103.0711v5 [cs.SE] 21 Jun 2012 retrieval algorithms that make questionable decisions evolution robustness is also calculated to provide fur- about the key issue: how to avoid accepting semanti- ther evidence. The second study covers, for balance, cally inconsistent objects into the retrieving system. the history of five versions of the Eiffel base library The purpose of the present work is to lay a solid classes. The findings confirm those of the first study foundation for a general, stable solution to the prob- and lead to a refined set of atomic attribute changes. lem of object-oriented class schema evolution. The Two more studies analyzing persistent classes in two contribution includes four major components: empir- well known software projects, the Apache Tomcat ical studies; a formal model, resulting both from the web server and the EiffelStudio IDE (development empirical studies and from object-oriented theory; a environment), lead to similar conclusions. The imple- measure of persistence evolution robustness and a mentation (Section 6) is the ESCHER tool, seamlessly schema evolution tool implementing the model. integrated into EiffelStudio and supporting develop- Section 2 presents a motivating example and the ers through the entire schema evolution process, from corresponding ESCHER tool support. Section 3 de- class release time to retrieval time. It includes version handling, code template generation for transformation • Marco Piccioni and Bertrand Meyer are with the Chair of Software functions and a retrieval algorithm preventing the Engineering, ETH Zurich, Switzerland. acceptance of inconsistent objects into the system. E-mail: [email protected], [email protected] • Manuel Oriol is with ABB Corporate Research, Industrial Software Section 7 evaluates ESCHER by analyzing which code Systems, Baden-D¨attwil, Switzerland and the University of York, UK. changes ESCHER actually detects in the four projects E-mail: [email protected], [email protected] mentioned above. It then analyzes the advantages and limitations of the contribution, drawing conclu- IEEE TRANSACTIONS ON SOFTWARE ENGINEERING 2 sions and suggesting future work. This paper extends is computed on-demand from the total amount of previous work from the authors [1] by adding three deposits and withdrawals, and there is an explicit new studies (on the EiffelBase library, on the Java class invariant capturing the bank account intended Apache Tomcat repository and on the EiffelStudio IDE semantics. This invariant is checked after the invoca- repository), by updating the model, by defining and tion of the constructor make of a bank account object, applying a measure for evolution robustness to all the and also before and after the invocation of any other classes in the java.util package, and by refining routine in the class. the tool implementation. class BANK ACCOUNT 2 A SCHEMA EVOLUTION EXAMPLE AND create make ITS HANDLING IN ESCHER feature −− Initialization This section presents an example scenario and shows make −− Create a bank account with an initial deposit. how the ESCHER tool copes with it. do balance := 1 end 2.1 A case of object-oriented schema evolution feature −− Status report balance : INTEGER Assume there is a software system storing objects of −− Account balance. type BANK ACCOUNT (see Figure 1). The balance info : INTEGER −− Some numeric information. feature −− Basic operations class deposit (sum: INTEGER) BANK ACCOUNT −− Add ‘sum’ to account. create make require sum non negative: sum > 0 feature −− Initialization do make balance := balance + sum −− Create a bank account with an initial deposit. ensure do balance correct : balance = old balance + sum tot deposits := 1 end end feature −− Status report withdraw (sum: INTEGER) balance : INTEGER −− Retrieve ‘sum’ from account. −− Account balance. require do sum non negative: sum > 0 Result := tot deposits − tot withdrawals has sufficient funds : sum < balance end do info : STRING balance := balance − sum −− Some numeric information. ensure feature −− Basic operations balance correct : balance = old balance − sum deposit (sum: INTEGER) end −− Add ‘sum’ to account. invariant require valid account : balance > 0 sum non negative: sum > 0 end do tot deposits := tot deposits + sum Fig. 2. BANK ACCOUNT, version 2. ensure balance correct : balance = old balance + sum end Version 1 then evolves into version 2 (Figure 2), in withdraw (sum: INTEGER) which the query balance becomes an attribute updated −− Retrieve ‘sum’ from account. every time a deposit or a withdraw takes place, require sum non negative: sum > 0 and the attribute info becomes an integer. While in has sufficient funds : sum < balance languages with a C-derived syntax like Java we would do need to modify client code to accommodate the fact tot withdrawals := tot withdrawals + sum ensure that a method becomes an attribute, here clients are balance correct : balance = old balance − sum not affected, because in Eiffel both attributes and end argument-less functions can be accessed in the same feature {NONE} −− Implementation tot deposits : INTEGER way from outside the class, providing uniform access −− Total amount deposited. to the class itself. Note also how the invariant is tot withdrawals : INTEGER now expressed in terms of the new attribute balance. −− Total amount withdrawn. invariant What happens when trying to retrieve an object stored valid account : tot deposits > tot withdrawals with version 1 of class BANK ACCOUNT into an end object of version 2 depends on the retrieval system. Silently accepting objects having the attribute balance Fig. 1. BANK ACCOUNT, version 1. initialized to the default is in this case wrong. To IEEE TRANSACTIONS ON SOFTWARE ENGINEERING 3 understand why, imagine developers forget to code an In Java a class can enable future serialization of appropriate transformation function assigning the dif- its instances by implementing the Serializable ference between tot deposits and tot withdrawals stored interface, or its descendant Externalizable [2], [3], in version 1 to the newly created balance attribute. [4]. Developers can provide custom deserialization Then balance will be zero for all retrieved objects, methods to help establishing the class invariant. which is hardly ideal. If we consider the Eiffel version The .NET framework, in a way similar to Java and 2 of class BANK ACCOUNT instead, at deserialization starting from version 2.0, provides Version Tolerant time there will be an invariant failure triggering an Serialization (VTS) [5]. In spite of the name, VTS does exception, because the value of balance is not positive not implement true version support. The serialized as the class invariant prescribes. object and all the referenced objects in the object