Abstract Predicates and Mutable ADTs in Hoare Type Theory Aleksandar Nanevski1, Amal Ahmed2, Greg Morrisett1, and Lars Birkedal3 1 Harvard University faleks,[email protected] 2 Toyota Technological Institute at Chicago [email protected] 3 IT University of Copenhagen [email protected] Abstract. Hoare Type Theory (HTT) combines a dependently typed, higher-order language with monadically-encapsulated, stateful compu- tations. The type system incorporates pre- and post-conditions, in a fashion similar to Hoare and Separation Logic, so that programmers can modularly specify the requirements and effects of computations within types. This paper extends HTT with quantification over abstract predicates (i.e., higher-order logic), thus embedding into HTT the Extended Calcu- lus of Constructions. When combined with the Hoare-like specifications, abstract predicates provide a powerful way to define and encapsulate the invariants of private state that may be shared by several functions, but is not accessible to their clients. We demonstrate this power by sketch- ing a number of abstract data types that demand ownership of mutable memory, including an idealized custom memory manager. 1 Background Dependent types provide a powerful form of specification for higher-order, func- tional languages. For example, using dependency, we can specify the signature of an array subscript operation as sub : 8α.Πx:array α.Πy:fi:nat j i < x:sizeg.α, where the type of the second argument, y, refines the underlying type nat using a predicate that ensures that y is a valid index for the array x. Dependent types have long been used in the development of formal mathe- matics, but their use in practical programming languages has proven challenging. One of the main reasons is that the presence of any computational effects, in- cluding non-termination, exceptions, access to store, or I/O { all of which are indispensable in practical programming { can quickly render a dependent type system unsound. The problem can be addressed by severely restricting dependencies to only effect-free terms (as in for instance DML [30]). But the goal of our work is to try to realize the full power of dependent types for specification of effectful programs. To that end, we have been developing the foundations of a language that we call Hoare Type Theory or HTT [22], which we intend to be an expressive and explicitly annotated internal language, providing a semantic framework for elaborating more practical external languages. HTT starts with a pure, dependently typed core language and augments it with an indexed monadic type of the form fP gx:AfQg. This type encapsulates and describes effectful computations that may diverge or access a mutable store. The type can be read as a Hoare-like partial correctness specification, asserting that if the computation is run in a world satisfying the pre-condition P , then if it terminates, it will return a value x of type A and be in a world described by Q. Through Hoare types, the system can enforce soundness in the presence of effects. The Hoare type admits small footprints as in Separation Logic [26, 24], where the pre- and postconditions only describe the part of the store that the program actually uses; the unspecified part is automatically assumed invariant. Recently, several variants of Hoare Logic for higher-order, effectful languages have appeared. Yoshida, Honda and Berger [31, 4] define a logic for PCF with references, Krishnaswami [13] defines a Separation Logic for core ML extended with a monad, and Birkedal et al. [5] define a Higher-Order Separation Logic for reasoning about ADTs in first-order programs. However, we believe that HTT has several key advantages over these and other proposed logics. First, HTT supports strong (i.e., type-varying) updates of mutable locations, while the above program logics require that the types of memory locations are invari- ant. This restriction makes it difficult to model stateful protocols as in the Vault language [7], or low-level languages such as TAL [20] and Cyclone [12] where memory management is intended to be coded within the language. Second, none of these logics considers pointer arithmetic, nor source language features like type abstraction, modules, or dependent types, which we consider here. Third, and most significant, Hoare logics cannot really interact with the type systems of the underlying language, unlike HTT where specifications are integrated with types. In Hoare Logic, it is not possible to abstract over specifications in the source programs, aggregate the logical invariants of the data structures with the data itself, compute with such invariants, or nest the specifications into larger specifications or types. These features are essential ingredients for data abstrac- tion and information hiding, and, in fact, a number of works have been proposed towards integrating Hoare-like reasoning with type checking. Examples include tools and languages like Spec# [1], SPLint [9], ESC/Java [8], and JML [6]. There are several important outstanding problems in the design of such lan- guages for integrated programming and verification. As discussed in [6], for ex- ample: (1) It is desirable to use effectful code in the specifications, but most languages insist that specifications must be pure, in order to preserve sound- ness. Such a restriction frequently leads to implementing the same functionality twice { once purely for specification, and once impurely for execution. (2) Spec- ifications should be able to describe and control pointer aliasing. (3) It is tricky to define a useful notion of object or module invariant, primarily because of local state owned by the object. Most definitions are too restrictive to support some important programming patterns, for example, reentrant callbacks [2]. Our prior work on HTT [22] addresses the first two problems: (1) we allow effectful code in specifications by granting such code first-class status, via the monad for Hoare triples, and (2) we control pointer aliasing, by employing the 2 small footprint approach of Separation Logic. Both of these properties were discussed at the beginning of this section. The focus of this paper are extensions to HTT that enable us to also address problem (3), among others. In a language like HTT that integrates programming and verification, truly reusable program components (e.g., libraries of data types and first-class ob- jects) require that their internal invariants are appropriately abstracted. The component interfaces need to include not only abstract types, but also abstract specifications. Thus it is natural to extend HTT with support for abstraction over predicates (i.e., higher-order logic). More specifically, we describe a vari- ant of HTT that includes the Extended Calculus of Constructions (ECC) [14], modulo minor differences described in Section 5. This allows terms, types, and predicates to all be abstracted within terms, types, and predicates respectively. There are several benefits of this extension. First, higher-order logic can formulate almost any predicate that may be encountered during program veri- fication, including predicates defined by induction and coinduction. Second, we can reason within the system, about the equality of terms, types and predi- cates, including abstract types and abstract predicates. In the previous version of HTT [22], we could only reason about the equality of terms, whereas equality on types and predicates was a judgment (accessible to the typechecker), but not a proposition (accessible to the programmer). Internalized reasoning on types endows HTT with a form of first-class modules that can contain types, terms, and axioms. It is also important in order to fully support strong updates of loca- tions. Third, higher-order logic can define many constructs that, in the previous version, had to be primitive. For instance, the definition of heaps can now be encoded within the language, thus simplifying some aspects of the meta theory. Most importantly, however, abstraction over predicates suffices to represent the private state of functions or ADTs within the type system. Private state can be hidden from the clients by existentially abstracting over the state invariant. Thus, libraries for mutable state can provide precise specifications, yet have sufficient abstraction mechanisms that different implementations can share a common interface. Moreover, specifications may choose to reveal certain aspects of private state to the client, thus granting the client partial or complete access to, or even ownership of portions of the private state. We demonstrate these ideas with a few idealized examples including a module for memory allocation and deallocation. 2 Overview Similar to the modern monadic functional languages [19], HTT syntax splits into the pure and the impure fragment. The pure fragment contains higher-order functions and pairs, and the impure fragment contains the effectful commands for memory lookup and strong update (memory allocation and deallocation can be defined), as well as conditionals and recursion. The expressions from the effectful fragment can be coerced into the pure one by monadic encapsulation. The type constructors include the primitive types of booleans, natural num- bers and the unit type, the standard constructors Π and Σ for dependent prod- 3 ucts and sums, as well as Hoare types fP gx:AfQg, and subset types fx:A: P g. The Hoare type fP gx:AfQg is the monadic type which classifies effectful com- putations that may execute in any initial heap satisfying the assertion P , and either diverge, or terminate returning a value x:A and a final heap satisfying the assertion Q. The subset type fx:A: P g classifies all the elements of A that satisfy the predicate P . We adopt the standard convention and write A!B and A×B instead of Πx:A: B and Σx:A: B when B does not depend on x. The syntax of our extended HTT is presented in the following table. Types A; B; C ::= K j nat j bool j 1 j prop j mono j Πx:A: B j Σx:A: B j fP gx:AfQg j fx:A: P g Elim terms K; L ::= x j K N j fst K j snd K j out K j M : A Intro terms M; N; O ::= K j ( ) j λx.