State Based Encapsulation for Modular Reasoning about Behavior-Preserving Refactorings Anindya Banerjee1 and David A. Naumann2 1 IMDEA Software Institute, Madrid, Spain 2 Stevens Institute of Technology, Hoboken NJ 07030 USA Abstract. A properly encapsulated data representation can be revised for refac- toring or other purposes without affecting the correctness of client programs and extensions of a class. But encapsulation is difficult to achieve in object-oriented programs owing to heap based structures and reentrant callbacks. This chapter shows that it is achieved by a discipline using assertions and auxiliary fields to manage invariants and transferrable ownership. The main result is representation independence: a rule for modular proof of equivalence of class implementations. 1 Introduction You are responsible for a library consisting of many Java classes. While fixing a bug or refactoring some classes, you revise the implementation of a certain class in a way that is intended not to change its observable behavior, e.g., an internal data structure is changed for reasons of performance. You are in no position to check, or even be aware of, the many applications that use the class via its instances or via instances of its sub- classes if any. In principle, the class could have a full functional specification. It would then suffice to prove that the new version meets the specification. In practice, full spec- ifications are rare. Nor is there a well established logic and method for modular reason- ing about the code of a class in terms of the specifications of the classes it uses, without regard to their implementations or the users of the class in question —though progress has been made, as reported in the companion chapters [29,55,60,38]. One problem is that encapsulation, crucial for modular reasoning about invariants, is difficult to achieve in programs that involve shared mutable objects and reentrant callbacks which violate simple layering of abstractions. Yet complicated heap structure and calling patterns are used, in well designed object-oriented programs, precisely for orderly composition of abstractions in terms of other abstractions. There is an alternative to verification with respect to a specification. One can attempt to prove that the revised version is behaviorally equivalent to the original. Of course their behavior is not identical, but at the level of abstraction of source code (e.g., modulo specific memory addresses), it may be possible to show equivalence of behavior. If any specifications are available they can be taken into account using assert statements. There is a standard technique for proving equivalence [36,46,27]: Define a coupling relation to connect the states of the two versions and prove that it has the simulation property, i.e., it holds initially and is preserved by parallel execution of the two ver- sions of each method. In most cases, one would want to define a local coupling relation 2 for a single pair of instances of the class, as methods act primarily on a target object (self) and the island [6], i.e., a group of objects that comprise its internal representation. An induced coupling for complete states is then obtained by a general construction. A language with good encapsulation should enjoy a representation independence prop- erty that says a simulation for the revised class induces a simulation for any program built using the class. Suitable couplings are the identity except inside the abstraction boundary and an identity extension lemma says simulation implies behavioral equiv- alence of two programs that differ only by revision of a class. This means that from a client’s point of view the behaviors of the two programs are the same. Again, such reasoning can be invalidated by heap sharing, which violates encapsulation of data, and by callbacks, which violate hierarchical control structure. There is a close connection between the equivalence problem and verification: ver- ification of object oriented code involves object invariants that constrain the internal state of an instance. Encapsulation involves defining the invariant in a way that protects it from outside interference so it holds globally provided it is preserved by the methods of the class of interest. Simulations are like invariants over two copies of the state space, and again modular reasoning requires that the coupling for a class be independent from outside interference. The main contribution of this chapter is a representation inde- pendence theorem using a state-based discipline for heap encapsulation and control of callbacks. Extant theories of data abstraction assume, in one way or another, a hierarchy of abstractions such that control does not reenter an encapsulation boundary while already executing inside it. It is commonplace in object oriented programs for a method m acting on some object o to invoke a method on some other object which in turn leads to invocation of some method on o —possibly m itself— while the initial invocation of m is in progress. This makes it difficult to reason about when an object’s invariant holds [38,47]; we give an example later. There is an analogous problem for reasoning with simulations. The first work on representation independence for programs with shared objects [6] provides an abstrac- tion theorem that deals with sharing and is sound for programs with reentrant callbacks —but it is not easy to apply in cases where reentrant callbacks are possible. The theo- rem allows the programmer to assume that all methods preserve the coupling relation when proving simulation, i.e., when reasoning about parallel execution of two versions of a method of the class of interest. This assumption is like verifying a procedure imple- mentation under the assumption that called procedures are correct. But the assumption that called methods preserve the coupling is of no use if the call is made in an uncoupled intermediate state. For the examples in [6], we resort to ad hoc reasoning for examples involving callbacks. In the “Boogie methodology” [11,43], reentrancy is managed using an explicit aux- iliary (or ghost) field inv to designate states in which an object invariant is to hold. The ghost field is extra instrumentation added to a program for reasoning purposes but does not influence data flow or control flow in the original program in any manner; thus the behavior of the annotated program is the same as that of the original program. Encap- sulation is achieved using a notion of ownership represented by an auxiliary mutable field own. This is more flexible than type-based static analyses because the ownership 3 invariant need only hold in certain flagged states. Heap encapsulation is achieved not by disallowing boundary-crossing pointers but by limiting, in a state-dependent way, their use. Reasoning hinges on a global program invariant that holds in all states, using inv fields to track which object invariants are temporarily not in force because control is within their encapsulation boundary. When inv holds, the object is said to be packed; a field may only be updated when the object is unpacked. In this chapter we adapt the inv=own discipline to proving class equivalence by sim- ulation. The inv fields make it possible for an induced coupling relation to hold at some pairs of intermediate states during parallel execution of two alternative implementa- tions. This means that the relation-preservation hypothesis of the abstraction theorem can be used at intermediate states even when the local coupling is not in force. So per- method modular reasoning is fully achieved. In large part the discipline is unchanged, as one would hope in keeping with the idea that a coupling is just an invariant over two parallel states. But we have to adapt some features in ways that make sense in terms of informal considerations of information hiding. The discipline imposes no con- trol on field reads, only writes, but for representation independence we need to control reads as well. The discipline also allows ownership transfer quite freely, though as we will discuss later, it is not trivial to design code that correctly performs transfers. For representation independence, the transfer of previously-encapsulated data to clients (an unusual form of controlled “rep exposure” [28]) is allowed but must occur only in the code of the encapsulating class; even then, it poses a difficult technical challenge. The significance of our adaptations is discussed in Section 8. A key insight is that, although transferring ownership and packing/unpacking in- volve only ghost fields that cannot affect program execution, it is useful to consider them to be observable. It is difficult to reason about two versions of a class, in a mod- ular way, if they differ in the way objects cross the encapsulation boundary or in the points at which methods assume the invariant is in force. The requisite similarity can be expressed using assert statements, so we can develop a theory based on this insight without the need to require that the class under revision has any specifications. The main contributions of this chapter are (a) formulation of a notion of instance- based coupling analogous to invariants in the inv=own discipline; (b) proof of a repre- sentation independence theorem for a language with inheritance and dynamic dispatch, recursive methods and callbacks, mutable objects, type casts, and recursive types; and (c) results on identity extension and use of the theorem to prove program equivalence. Together these constitute a rule by which the reasoner considers just the methods of the revised class and concludes that the two versions yield equivalent behavior for any program context. The theorem allows ownership transfers that cross encapsulation boundaries: from client to abstraction [28], between instances of the abstraction, and even from abstrac- tion to client [54,43,4].