Optimizing Higher-Order Pattern Unification

Optimizing Higher-Order Pattern Uniﬁcation

Brigitte Pientka and Frank Pfenning?

Department of Computer Science Carnegie Mellon University [email protected],[email protected] Submitted, January 2003

Abstract. We present an abstract view of existential variables in a dependently typed lambda-calculus based on modal type theory. This allows us to justify optimizations to pattern unification such as linearization, which eliminates many unnecessary occurs-checks. The presented modal framework explains a number of features of the current implementation of higher-order unification in Twelf and provides insight into several optimizations. Experimental results demonstrate significant performance improvement in many example applications of Twelf, including those in the area of proof-carrying code.

1 Introduction

Unification lies at the heart of automated reasoning systems, logic programming and rewrite systems. Thus its performance affects in a crucial way the global efficiency of each of these applications. This need for efficient unification algo- rithms has led to many investigations in the first-order setting. However, the efficient implementation of higher-order unification, especially for dependently typed λ-calculus, is still a central open problem limiting the potential impact of higher-order reasoning systems such as Twelf [15], Isabelle [12], or λProlog [9]. The most comprehensive study on efficient and robust implementation techniques for higher-order unification so far has been carried out by Nadathur and colleagues for the simply-typed λ-calculus in the programming language λProlog [7, 8]. The Teyjus compiler [10] embodies many of the insights found, in particular an adequate representation of lambda terms and mechanisms to delay and compose substitutions. Higher-order unification is implemented via Huet’s algorithm [5] and special mechanisms are molded into the WAM instruction set to support branching and carrying unification problems. To only perform an occurs-check when necessary, the compiler distinguishes between the first occurrence and subsequent occurrences of a variable and compiles them into different WAM instructions. While for the first occurrence of a variable the occurs-check may be omitted, full unification is used for all subsequent variables. This approach seems to work well in the simply-typed setting, however it is not clear how to generalize it to dependent types. ? This work has been partially supported by NSF Grant CCR-9988281 “Logical and Meta-Logical Frameworks” and by a Siebel Scholarship awarded to the first author. In this paper, we discuss the efficient implementation of higher-order pattern unification for the dependently typed lambda-calculus. Unlike Huet’s general higher-order unification algorithm which involves branching and backtracking, higher-order pattern unification [6, 13] is deterministic and decidable. An important step toward the efficient implementation of higher-order pattern unification was the development based on explicit substitutions and de Bruijn indices [3] for the simply-typed lambda-calculus. This allows a clear distinction between bound and existential variables and reduces the problem to essentially first-order unification. Although the use of de Bruijn indices leads to a simple formal system, the readability may be obstructed and critical principles are obfuscated by the technical notation. In addition, some techniques like pre-cooking of terms and optimizations such as lowering and grafting remain ad hoc. This makes it more difficult to transfer these optimizations to other calculi. We present an abstract view of existential variables in the dependently typed lambda-calculus based on modal type theory. Our calculus does not require de Bruijn indices, nor does it require closures M[σ] as first-class terms. This leads to a simple clean framework which allows us to explain a number of features of the current implementation of higher-order unification in Twelf [15] and provides insight into several optimizations. In the paper, we will particularly focus on one optimization called linearization, which eliminates many unnecessary occurs-checks. We have implemented this optimization of higher-order unification as part of the Twelf system. Experimental results demonstrate significant performance improvement, including those in the area of proof-carrying code. The paper is organized as follows: First we give some background on modal logic and modal type theory, and discuss its relation to the dependently typed lambda calculus (see Section 2). In Section 3 we discuss higher-order pattern unification. In particular, we focus on the optimization, called linearization, which eliminates unnecessary occurs-checks. Finally in Section 4 we discuss experimental results. Related work is discussed in Section 5.

2 A modal foundation for typed existential variables

2.1 Motivation We start by presenting a foundation for dependently typed existential variables based on modal logic. Following the methodology of Pfenning and Davies [14], we can assign constructive explanations to modal operators. A key characteristic of this view, is to distinguish between propositions that are true and propositions that are valid. A proposition is valid if its truth does not depend on the truth of any other propositions. This leads to the basic hypothetical judgment

A1 valid,...An valid; B1 true,...,Bm true ` C true.

Under the multiple-world interpretation of modal logic, C valid corresponds to C true in all reachable worlds. This means C true without any assumptions, except those that are assumed to be true in all worlds. We can generalize this

2 idea to also capture truth relative to a set of speciﬁed assumptions by writing C valid Ψ, where Ψ is the abbreviation for C1 true,...,Cn true. In terms of the multiple world semantics, this means that C is true in any world where C1 through Cn are all true and we say C is valid relative to the assumptions in Ψ. Hypotheses about relative validity are more complex now, so our general judgment form is

A1 valid Ψ1,...,An valid Ψn; B1 true,...,Bm true ` C true

While it is interesting to investigate this modal logic above in its own right, it does not come alive until we introduce proof terms. In this paper, we investigate the use of a modal proof term calculus as a foundation for existential variables. We will view existential variables u as modal variables of type A in a context Ψ while bound variables are treated as ordinary variables. This allows us to distinguish between existential variables u::(Ψ `A) for relative validity assumptions A valid Ψ declared in a modal context, and x:A for ordinary truth assumptions A true declared in an (ordinary) context. If we have an assumption A valid Ψ we can only conclude A true if we can verify all assumptions in Ψ.

∆, A valid Ψ, ∆0; Γ ` Ψ (∗) ∆, A valid Ψ, ∆0; Γ ` A true

In other words, if we know A true in Ψ, and all elements in Ψ can be veriﬁed from the assumptions in Γ , then we can conclude A true in Γ . As we will see in the next section, this transition from one context Ψ to another context Γ , can be achieved via a substitutions from Ψ to Γ .

2.2 Dependently typed lambda calculus based on modal logic

In this section, we introduce a dependently typed lambda calculus. Existential variables u are treated as modal variables and x denotes ordinary variables. c and a are constants, which are declared in a signature. This is a conservative extension of the LF [4] so we suppress some routine details such as signatures.

Kinds K ::= type | Πx:A. K Families A, B, C ::= a | AM | Πx:A1.A2 Objects M,N ::= c | x | u[σ] | λx:A. M | M1 M2 Substitutions σ, τ ::= · | σ, M/x Contexts Γ,Ψ ::= · | Γ, x:A Modal Contexts ∆ ::= · | ∆, u::(Ψ `A) Note that the substitution σ is part of the syntax of existential variables. This eliminates the need of pre-cooking [3] which raises existential variables to the correct context. The principal judgments are listed below. As usual, we omit similar judgments on types and kinds and all judgments concerning deﬁnitional equality.

3 ∆; Γ ` M : A Object M has type A ∆; Γ ` σ : Ψ Substitution σ matches context Ψ ` ∆ mctx ∆ is a valid modal context ∆ ` Ψ ctx Ψ is a valid context Note substitutions σ are deﬁned only on ordinary variables x and not modal variables u. We write idΓ for the identity substitution (x1/x1, . . . , xn/xn) for a context Γ = (·, x1:A1, . . . , xn:An). We will use π for a substitution which may permute the variables, i.e π = (xΦ(1)/x1, . . . , xΦ(n)/xn) where Φ is a total permutation deﬁned on the elements from a context Γ = (·, x1:A1, . . . , xn:An). We only consider well-typed substitutions, so π must respect possible dependencies in its domain. We also streamline the calculus slightly by always substituting simultaneously for all ordinary variables. This is not essential, but saves some tedium in relating simultaneous and iterated substitution. Moreover, it is also closer to the actual implementation where we use de Bruijn indices and postpone explicit substitutions. The typing rules are given in Figure 1.

0 ∆, u::(Ψ `A), ∆ ; Γ ` σ : Ψ 0 0 (∗) ∆; Γ, x:A, Γ ` x : A ∆, u::(Ψ `A), ∆ ; Γ ` u[σ]:[σ]A

∆; Γ, x:A1 ` M : A2 ∆; Γ ` M1 : Πx:A2.A1 ∆; Γ ` M2 : A2

∆; Γ ` λx. M : Πx:A1.A2 ∆; Γ ` M1 M2 :[idΓ ,M2/x]A1

∆; Γ ` σ : Ψ ∆; Γ ` M :[σ]A ∆; Γ ` (·):(·) ∆; Γ ` (σ, M/x):(Ψ, x:A)

` ∆ mctx ∆ ` Ψ ctx ∆; Ψ ` A : type ` (·) mctx ` (∆, u::(Ψ `A)) mctx

∆ ` Ψ ctx ∆; Ψ ` A : type ∆ ` (·) ctx ∆ ` (Ψ, x:A) ctx

Fig. 1. Typing rules for objects, substitutions, and context

Note that the rule for modal variables is the rule (*) presented in the previous section, annotated with proof terms and slightly generalized, because of the dependent type theory we are working in. This rule also justifies our implementation choice of using existential variables only in the form u[σ]. Our convention is that substitutions as defined operations on expressions are written in prefix notation [σ]P for an object, family, kind, or substitution P . These operations are capture-avoiding as usual. Moreover, we always assume that all free variables in P are declared in σ. Substitutions that are part of the

4 syntax are written in postﬁx notation, u[σ]. Note that such explicit substitutions occur only for variables u labeling relative validity assumptions. Substitutions are deﬁned in a standard manner. We omit the details at the level of types and kinds for the sake of brevity.

[σ]c = c [σ1, M/x, σ2]x = M [σ](u[τ]) = u[[σ]τ] [σ](N1 N2) = ([σ]N1) ([σ]N2) [σ](λy:A. N) = λy:[σ]A. [σ, y/y]N provided y not declared or free in σ

[σ](·) = (·) [σ](τ, N/y) = ([σ]τ, [σ]N/y) provided y not declared or free in σ

The side conditions can always be veriﬁed by (tacitly) renaming bound variables. We do not need an operation of applying a substitution σ to a context. The last principle makes it clear that [σ]τ corresponds to composition of substitutions, which is sometimes written as τ ◦ σ. The following substitution principles for substitutions σ hold. They are sug- gested by the modal interpretation and proved by simple structural inductions. We elide corresponding principles for families and kinds.

Theorem 1.

1. If ∆; Γ ` σ : Ψ and ∆; Ψ ` N : C then ∆; Γ ` [σ]N :[σ]C. 2. If ∆; Γ ` σ : Ψ and ∆; Ψ ` τ : Ψ 0 then ∆; Γ ` [σ]τ : Ψ 0. 3. [σ]([τ]M) = [[σ]τ]M and [σ]([τ]τ 0) = [[σ]τ]τ 0

A new and interesting operation arises from the substitution principles for relative validity. The new operation of substitution is compositional, but two interesting situations arise: when a variable u is encountered, and when we sub- stitute into a λ-abstraction. For sake of brevity, we only give the substitution on objects. [[M/u]]c = c [[M/u]]x = x [[M/u]](u[σ]) = [[[M/u]]σ]M [[M/u]](v[σ]) = v[[[M/u]]σ] for u 6= v [[M/u]](N1 N2) = ([[M/u]]N1) ([[M/u]]N2) [[M/u]](λy:A. N) = λy:[[M/u]]A. [[M/u]]N

We remark that the rule for substitution into a λ-abstraction does not require a side condition. This is because the object M is defined in a different context, which is accounted for by the explicit substitution stored at occurrences of u. This ultimately justifies implementing substitution for existential variables by mutation.

5 Finally, consider the case of substituting into a closure, which is the critical case of this deﬁnition.

[[M/u]](u[σ]) = [[[M/u]]σ]M

This is clearly well-founded, because σ is a subexpression (so [[M/u]]σ will ter- minate) and application of an ordinary substitution has been defined previously without reference to the new form of substitution. Using the given definitions, we can then show that the new substitution operation for relative validity satisfies the substitution principles. Again, this is motivated by the logical interpretation and follows by simple inductions after straightforward generalization to encompass all syntactic categories.

Theorem 2.

0 1. If ∆; Ψ ` M : A and ∆, u::(Ψ `A), ∆ ; Γ ` N : C then ∆, [[M/u]]∆0; [[M/u]]Γ ` [[M/u]]N : [[M/u]]C 0 0 2. If ∆; Ψ ` M : A and ∆, u::(Ψ `A), ∆ ; Γ ` τ : Ψ then ∆, [[M/u]]∆0; [[M/u]]Γ ` [[M/u]]τ : [[M/u]]Ψ 0 3. [[M/u]]([σ]P ) = [[[M/u]]σ]([[M/u]]P ) 4. [[M/u]]([[N/v]]P ) = [[[[M/u]]N/v]]([[M/u]]P ) if u 6= v and v not free in M

2.3 Normal Forms There are two notions of normal forms that are useful in the implementation. The ﬁrst corresponds to a β-normal form. We simultaneously deﬁne normal (U) and neutral (R) objects and normal substitutions η.

Normal Objects U ::= λx:A. U | R Neutral Objects R ::= c | x | u[η] | RU Normal Substitutions η ::= · | η, U/x We obtain the canonical objects (long βη-normal forms) by requiring normal objects of the form R to have base type (that is, not to have function type). In the implementation we use a stronger normal form where existential variables (represented here by modal variables) must also be of atomic type. This is accomplished by a technique called lowering. Lowering replaces a variable 0 u::(Ψ `Πx:A1.A2) by a new variable u :(Ψ, x:A1`A2). This process is repeated until all existential variables have a type of the form Ψ ` b N1 ...Nk. This operation has been proved correct for the simply-typed case by Dowek et al. [3], but remains somewhat mysterious. Here, it is justiﬁed by the modal substitution principle.

Lemma 1.

0 1. (Lowering) If ∆, u::(Ψ `Πx:A1.A2), ∆ ; Γ ` M : A 0 0− − − − then ∆, u ::(Ψ, x:A1`A2), ∆ ; Γ ` M : A − 0 where (P ) = [[(λx:A1.u [idΨ , x/x])/u]]P .

6 0 0 2. (Raising) If ∆, u ::(Ψ, x:A1`A2), ∆ ; Γ ` M : A 0+ + + + then ∆, u::(Ψ `Πx:A1.A2), ∆ ; Γ ` M : A + 0 where (P ) = [[(u[idΨ ] x)/u ]]P . 3. ()+ and ()− are inverse substitutions (modulo βη-conversion). Proof. Direct, by weakening and the modal substitution principle. For part (1) 0 0 we observe that ∆, u ::(Ψ, x:A1`A2); Ψ ` λx:A1.u [idΨ , x/x]: Πx:A1.A2. For part (2) we use instead that ∆, u::(Ψ `Πx:A1.A2); Ψ, x:A1 ` u[idΨ ] x : A2. Part (3) is direct by calculation. Since we can lower all modal variables, we can change the syntax of normal forms so that terms u[η] are also normal objects of base type, rather than neutral objects. This is, in fact, what we chose in the implementation.

2.4 Existential Variables As mentioned several times above, in the implementation the modal variables in ∆ are used to represent existential variables (also known as meta-variables), while the variables in Γ are universal variables (also known as parameters). Existential variables are created in an ambient context Ψ and then lowered. We do not explicitly maintain a context ∆ of these existential variables, but it is important that a proper order for them exists. Existential variables are created with a mutable reference, which is updated with an assignment when we need to carry out a substitution [[M/u]]. In certain operations, and particularly after type reconstruction, we need to abstract over the existential variables in a term. Since the LF type theory provides no means to quantify over u::(Ψ À) we raise such variables until they 0 0 have the form u ::(·À ). It turns out that in the context of type reconstruction we can now quantify over them as ordinary variables x0:A0. However, this is not satisfactory as this requires first raising the type of existential variables for abstraction, and later again lowering the type of existential variables during unification to undo the effect of raising. To efficiently treat existential variables, we would like to directly quantify over modal variables u. The judgmental reconstruction in terms of modal logic suggests two ways to 2 incorporate modal variables. One way is via a new quantifier Π u::(Ψ À1).A2, the other is via a general modal operator 2Ψ . Proof-theoretically, the former is slightly simpler, so we will pursue this here. The new operator then has the form 2 Π u::(Ψ À1).A2 and is defined by the following rules.

∆; Ψ ` A : type ∆, u::(Ψ `A); Γ ` B : type ∆; Γ ` Π 2u::(Ψ `A).B : type

∆, u::(Ψ À); Γ ` M : B ∆; Γ ` N : Π 2u::(Ψ À). B ∆; Ψ ` M : A 2 2 ∆; Γ ` λ u. M : Π u::(Ψ À).B ∆; Γ ` N 2 M : [[M/u]]B The main complication of this extension is that variables u can now be bound and substitution must be capture avoiding. In the present implementation, this is handled by de Bruijn indices.

7 3 Toward eﬃcient higher-order pattern uniﬁcation

3.1 Preliminaries In the following, we will consider the pattern fragment of the modal lambda- calculus. Higher-order patterns are terms where existential variables must be applied to distinct bound variables. This fragment was first identified by Miller [6] for the simply-typed lambda-calculus, and later extended by Pfenning [13] to the dependently typed and polymorphic case. We enforce that all terms are in normal form, and the type of existential variables has been lowered and is atomic. We call a normal term U an atomic pattern, if all the subterms of the form u[σ] are such that σ = y1/x1, . . . yk/xk where y1, . . . , yk are distinct bound variables. This is already implicitly assumed for x1, . . . , xk because all variables defined by a substitution must be distinct. Higher-order pattern unification can be done in two phases (see [13, 3] for a more detailed account). During the first phase, we decompose the terms until one of the two terms we unify is an existential variable u[σ]. This decomposition phase is straightforward and resembles first-order unification closely. During the second phase, we need to find an actual instantiation for the existential variable u. There are two main cases to distinguish: (1) when we unify two existential . variables, u[σ] = v[σ0], and when we unify an existential variable with another . . kind of term, u[σ] = M. The latter case is transformed into u = [σ]−1 M assuming u does not occur in M and all variables v[τ] are pruned so that the free variables in τ all occur in the image of σ (see [6, 3] for details). Note that we view [σ]−1 M as a new meta-level operation such as substitution, because it may be defined even if σ is not invertible in full generality. The main efficiency problem in pattern unification lies in treating this last case. In particular, we must traverse the term M. First of all, we must perform the occurs-check to prevent cyclic terms. Second, we may need to prune the substitutions associated with existential variables occurring in M. Third, we need to ensure that all bound variables occurring in M do occur in the range of σ, otherwise [σ]−1 M does not exist. To illustrate the problem, we give the definition for inverting substitutions: [σ]−1 c = c [σ]−1 x = y if x/y ∈ σ, undefined otherwise [σ]−1 (v[τ]) = v[[σ]−1 τ] [σ]−1 (RU) = ([σ]−1 R) ([σ]−1 U) [σ]−1 (λx:A. U) = λx:([σ]−1 A). [σ, x/x]−1 U if x not declared or free in [σ]−1 [σ]−1 (·) = (·)[σ]−1 (τ, U/x) = ([σ]−1 τ, [σ]−1 U/x) In the next section, we will show how linearization can be used to enforce the two criteria which eliminates the need to traverse M. First, we will enforce that all existential variables occur only once, thereby eliminating the occurs- check. Second, we will require that the substitution σ associated with existential variables is always a permutation π. This ensures that the substitutions are always invertible and eliminates the need for pruning.

8 3.2 Linearization One critical optimization in unification is to perform the occurs-check only when necessary. While unification with the occurs-check is at best linear in the sum of the sizes of the terms being unified, unification without the occurs-check is linear in the smallest term being unified. In fact the occurs-check can be omitted if the terms are linear, i.e., every existential variable occurs only once. Let us consider the following clause from a program which evaluates expressions from a small functional language Mini-ML. It says that functions evaluate to themselves. Using the introduced modal term language, this can be expressed as follows:

exp : type. lam :(exp → exp) → exp.

eval : exp → exp → type. ev lam : Π 2e::(y:exp`exp).eval (lam (λx:exp.e[x/y])) (lam (λx:exp.e[x/y])).

The existential variable e in the clause ev lam is quantiﬁed by Π 2e::(y:exp`exp). To enforce that every existential variable occurs only once, the clause head of ev lam can be translated into the linear type

2 0 2 Π e ::(z:expèxp).Π e::(y:expèxp). eval (lam (λx:exp.e[x/y])) (lam (λx:exp.e0[x/z])) together with the following variable definition ∀x:exp.e0[x/z] =D e[x/y] where e0 is a new existential variable. Then a constant time assignment algorithm can be used for assigning a linear clause head to a goal, and the variable definitions are solved by conventional unification. As a result, the occurs-check is only performed if necessary. In the dependently typed lambda-calculus, there are several difficulties in performing this optimization. First of all, all existential variables carry their context Ψ and type A. If we introduce a new existential variable, then the ques- tion arises what type should be assigned to it. As type inference is undecidable in the dependently typed case, this may be expensive. In general, we may even obtain a term which is not necessarily well-typed. Let us modify the previous example, and annotate the expressions with their type thus enforcing that any evaluation of a Mini-ML expression will be well- typed. tp : type. arrow : tp → tp → tp. exp : tp → type. 2 2 lam : Π t1::(·`tp).Π t2::(·`tp).(exp t1 → exp t2) → exp (t1 ⇒ t2). eval : Π 2t::(·`tp).exp t → exp t → type. 2 2 2 ev lam : Π t1::(·`tp).Π t2::(·`tp).Π e::(y:exp t1èxp t2). eval (arrow t1 t2)(lam t1 t2 (λx:exp t1.e[x/y])) (lam t1 t2 (λx:exp t1.e[x/y])).

9 During linearization, the clause head of ev lam will now be translated into

2 2 2 Π t1::(·`tp).Π t2::(·`tp).Π e::(y:exp t1èxp t2). 2 2 2 2 2 0 Π t3::(·`tp).Π t4::(·`tp).Π t5::(·`tp).Π t6::(·`tp).Π e ::(z:exp t1èxp t2). 0 eval (arrow t1 t2)(lam t3 t4 (λx:exp t1.e[x/y])) (lam t5 t6 (λx:exp t1.e [x/z])). and the following variable definitions

D D D D 0 D t1 = t3 ∧ t1 = t5 ∧ t2 = t4 ∧ t2 = t6 ∧ ∀x:exp t1.e [x/z] = e[x/y] Due to the linearization, the linear clause head is clearly not well-typed. However, it is well-typed modulo variable definitions. Therefore, it will be well- typed after all existential variables have been instantiated during assignment and the variable definitions have been solved. It would be interesting to accord first-class type-theoretic status to the variable definitions, but we leave this to future work, since the implementation treats them only in a very special manner explained in Section 3.3. Note that some of these variable definitions are in fact redundant, which is another orthogonal optimization (see [11] for an analysis on a fragment of LF). It is worth pointing out that this situation does not arise in the simply typed case. These considerations lead to the following description of variable definitions:

D Variable Deﬁnitions D ::= true | u[idΨ ] = U | D1 ∧ D2 | ∀x:A.D

The idea of factoring out duplicate existential variables can be generalized to replacing arbitrary subterms by new existential variables and creating variable definitions. In particular, the process of linearization also replaces any existential variables v[σ] where σ is not a permutation by a new variable u[idΨ ] and a D variable definition u[idΨ ] = v[σ]. The linearization itself is quite straightforward and we will omit the details here. In the actual implementation, we do not generate types A and contexts Ψ for the new, linearly occurring existential variables, but ensure that all such variables are in instantiated and disappear by the time the variable definitions have been solved.

3.3 Assignment for higher-order patterns In this section, we give a refinement of the general higher-order pattern unification which exploits the presented ideas. The algorithm proceeds in three phases. First we will unify a linear atomic higher-order pattern L with an object U. The following judgments capture the assignment between a linear atomic higher-order pattern L and a normal object U. We write θ for simultaneous substitutions [[U1/u1,...Un/un]] for existential variables which have straightforward definition and properties. . ∆; Γ ` L = U/(θ, E) assignment for normal objects 0 . 0 ∆; Γ L = R /(θ, E) assignment for neutral objects

10 where R is a linear neutral object. E denotes residual equations which may be generated during assignment. The assignment algorithm itself is given below. . ∆; Γ, x:A ` L = U/(θ, E) . lam . existsL ∆; Γ ` λx:A.L = λx:A.U/(θ, ∀x:A.E) ∆; Γ ` u[π] = U/([π]−1 U/u, true)

0 . ∆; Γ L = R/(θ, E) . coerce . . existsR ∆; Γ ` L0 = R/(θ, E) ∆; Γ ` L = u[σ]/(·,L = u[σ])

. const . var ∆; Γ x = x/(·; true) ∆; Γ c = c/(·; true) . . ∆; Γ L0 = R/(θ ; E ) ∆; Γ ` L = U/(θ ,E ) 1 1 2 2 app 0 . ∆; Γ L L = RU/(θ1 ∪ θ2; E1 ∧ E2) Note that we do not need to worry about capture in the rule lam, since existential variables and bound variables are defined in different contexts. In the rule app, we are allowed to union the two substitutions θ1 and θ2, as the linearity requirement ensures that the domains of both substitutions are disjoint. Note that the case for unifying an existential variable u[π] with another term U is now simpler and more efficient than in the general higher-order pattern case. In particular, it does not require a traversal of U (see rule existsL). Since the inverse of the substitution π can be computed directly and will be total, we know [π]−1 U exists and can simply generate a substitution [π]−1 U/u. Finally, we may need to postpone solving some unification problems and generate some residual equations if the non-linear term is an existential variable (see existsR). The result of the assignment algorithm is a substitution θ1 for the existential variables in L and potentially some residual equations E. In the second phase, we apply θ1 to the variable definitions D which were generated during linearization 0 of U and solve [[θ1]]D using conventional pattern unification. We only need to pay attention to the case where we unify an existential variable u[idΨ ] with another variable u0[σ]. In this case, we simply generate a substitution [[u0[σ]/u]], as the inverse of idΨ will be the identity substitution again. This ensures that all existential variables introduced during linearization, will be instantiated after assignment succeeds. As a final result of solving the variable definitions D, we obtain an additional substitution θ2. In the third phase, we solve the remaining residual equations E, which were generated during phase 1 under θ1 ◦ θ2.

4 Experiments

In this section, we discuss some experimental results with diﬀerent programs written in Twelf. All experiments are done on a machine with the following speciﬁcations: 1.60GHz Intel Pentium Processor, 256 KB cache. We are using SML of New Jersey 110.0.3 under Linux Red Hat 7.1. Times are measured in sec- onds. In the tables below, the column “opt” refers to the optimized version with

11 linearization and assignment, while the column “stand” refers to the standard implementation using general higher-order pattern uniﬁcation.

4.1 Higher-order logic programming

In this section, we present two experiments with higher-order logic programming. The ﬁrst one uses an implementation of a meta-interpreter for ordered linear logic by Polakow and Pfenning [16]. In the second experiment we evaluate our uniﬁcation algorithm using an implementation of foundational proof-carrying code developed at Princeton University [1].

Meta-interpreter for ordered linear logic

example opt stand speed-up variable def calls to assign trivial non-tr. fail success fail sqnt (bf) 0.84 2.09 149% 44% 46% 23% 77% sqnt (dfs) 0.93 2.35 152% 44% 47% 22% 78% sqnt (perm) 4.44 7.11 60% 44% 52% 20% 80% sqnt (rev) 1.21 1.70 40% 45% 48% 21% 79% sqnt (mergesort) 2.26 3.39 50% 46% 53% 20% 80%

As the results for the meta-interpreter demonstrate, the performance improvement ranges between 40% and 152%. Roughly, 45% of the time there were no variable definitions at all. From the non-trivial equations roughly 45% were not unifiable. This means overall, in approx. 20% - 30% of the cases the assignment algorithm succeeded and the failure of unification was delayed. It is worth noting that 77% to 80% of the calls to assignment presented in Section 3.3 fail immediately. Foundational proof-carrying code

example opt stand speed-up variable def calls to assign trivial non-tr. fail success fail inc 5.8 9.19 58% 64% 46% 18% 82% switch 36.00 49.69 38% 64% 48% 19% 81% mul2 5.51 9.520 72% 64% 46% 18% 82% div2 121.96 153.610 26% 63% 48% 20% 80% divx 333.69 1133.150 239% 63% 50% 21% 79% listsum 1073.33 ∞ ∞ 65% 45% 18% 82% polyc 2417.85 ∞ ∞ 65% 41% 17% 83% pack 197.07 1075.610 445% 66% 45% 19% 82%

In the table above we show the performance on programs from the proof- carrying code benchmark. Performance is improved by up to 445% and some examples are not executable without linearization and assignment. The results clearly demonstrate that an eﬃcient uniﬁcation algorithm is critical in large-scale examples.

12 4.2 Higher-order theorem proving Besides a logic programming engine, the Twelf system also provides a theorem prover which is based on iterative deepening search. In this section, we consider two examples, theorem proving in an intuitionist sequent calculus and theorem proving in the classical natural deduction calculus.

Proof search in the intuitionist sequent calculus

example opt stand speed-up variable def calls to assign trivial non-tr. fail success fail dist-1 53.00 57.11 8% 100% 0% 52% 48% distImp 0.40 0.44 10% 100% 0% 53% 47% pierce 1520.77 1563.35 3% 100% 0% 52% 48% trans 0.13 0.13 0% 100% 0% 53% 47%

As the results demonstrate, the performance of the theorem prover is not greatly influenced by the optimized unification algorithm. The main reason is that we have many dynamic assumptions, which need to be unified with the current goal. However, we use the standard higher-order pattern unification algorithm for this operation and use the optimized algorithm only for selecting a clause. For dynamic assumptions we cannot maintain the linearity requirement and linearizing the dynamic assumptions at run-time seems too expensive.

Proof search in NK (and, impl, neg)

example opt stand speed-up variable def calls to assign trivial non-tr. fail success fail andEﬀ1-nk 7.67 13.14 71% 100% 0% 80% 20% andEﬀ2-nk 3.86 6.58 70% 100% 0% 81% 19% assocAnd-nk 2.24 3.74 67% 100% 0% 81% 19% combS-nk 3.85 6.64 72% 100% 0% 81% 19%

The second example is theorem proving in the natural deduction calculus. In contrast to the previous experiments with the sequent calculus, there is a substantial performance improvement by approximately 70%. Although linear head compilation substantially improves performance, more optimizations, such as tabelling and indexing, are needed to solve more complex theorems.

5 Related Work

The language most closely related to our work, is λProlog. Two main different implementations of λProlog exist, Prolog/Mali and Teyjus. In Prolog/MALI implementation, the occurs-check is left out entirely[2]. While Teyjus [8, 7] also eliminates some unnecessary occurs-checks statically during compilation. How- ever, in addition to the presence of dependencies in Twelf, there are several other differences between our implementation and Teyjus. 1) Teyjus compiles first and

13 subsequent occurrences of existential variables into different instruction. There- fore, assignment and unification are freely mixed during the execution. This may lead to expensive failure in some cases, since unification is still called. In our approach, we perform a simple fast assignment check and delay unification entirely. As the experimental results demonstrate, only a small percentage of the cases fails after it already passed the assignment test and most cases benefit from a fast simple assignment check. 2) We always assume that the types of existential variables are lowered. This can be done at compile time and incurs no run-time overhead. In Huet’s unification algorithm, projection and imitation rules are applied at run-time to construct the correct prefixes of λ-abstractions. 3) Our approach can easily incorporate definitions and constraint domains. This is important since unifying definitions and constraint expressions may potentially be expensive. In fact, we generalize and extend the idea of linearization in the implementation and factor out not only duplicate existential variables but also any difficult sub-expressions such as definitions and constraint expressions. Therefore, our approach seems more general than the one adopted in Teyjus.

6 Conclusion

We have presented modal foundation for existential variables which underlies our higher-order pattern unification implementation in Twelf. This leads to a simple framework in which many optimizations such as lowering, grafting and linearization can be justified. As experiments show, performance is improved substantially. This is especially important in large-scale applications such as proof-carrying code and allows us explore the full potential of logical frameworks in real-world applications. In the future, we plan to investigate and implement further optimizations to reduce the performance gap between higher-order and first-order system. One optimization which is particularly important to sustain performance in large- scale examples is term indexing. However, indexing of higher-order terms is still an open problem. We believe the presented strategy is a suitable basis to adopt indexing techniques from the first-order setting for two reasons: 1) the presented strategy reduces the higher-order unification problem to a simple class of problems which is essentially first-order. 2) the experimental results show that many unification problems arising in practice fall into this class and can be solved by this strategy. This indicates that a term indexing algorithm based on this assignment strategy may also be effective in practice.

References

1. Andrew Appel. Foundational proof-carrying code. In J. Halpern, editor, Pro- ceedings of the 16th Annual Symposium on Logic in Computer Science (LICS’01), pages 247–256. IEEE Computer Society Press, June 2001. Invited Talk. 2. Pascal Brisset and Olivier Ridoux. Naive reverse can be linear. In Koichi Fu- rukawa, editor, International Conference on Logic Programming, pages 857–870, Paris, France, June 1991. MIT Press.

14 3. Gilles Dowek, Thérèse Hardin, Claude Kirchner, and Frank Pfenning. Unification via explicit substitutions: The case of higher-order patterns. In M. Maher, editor, Proceedings of the Joint International Conference and Symposium on Logic Programming, pages 259–273, Bonn, Germany, September 1996. MIT Press. 4. Robert Harper, Furio Honsell, and Gordon Plotkin. A framework for defining logics. Journal of the Association for Computing Machinery, 40(1):143–184, January 1993. 5. GérardHuet. A unification algorithm for typed λ-calculus. Theoretical Computer Science, 1:27–57, 1975. 6. Dale Miller. Unification of simply typed lambda-terms as logic programming. In Eighth International Logic Programming Conference, pages 255–269, Paris, France, June 1991. MIT Press. 7. Gopalan Nadathur. A treatment of higher-order features in logic programming. Technical Report draft, available upon request, Department of Computer Science and Engineering, University of Minnesota, January 2003. 8. Gopalan Nadathur, Bharat Jayaraman, and Debra Sue Wilson. Implementation considerations for higher-order features in logic programming. Technical Report CS-1993-16, Department of Computer Science, Duke University, June 1993. 9. Gopalan Nadathur and Dale Miller. An overview of λProlog. In Kenneth A. Bowen and Robert A. Kowalski, editors, Fifth International Logic Programming Conference, pages 810–827, Seattle, Washington, August 1988. MIT Press. 10. Gopalan Nadathur and Dustin J. Mitchell. System description: Teyjus—a compiler and abstract machine based implementation of lambda prolog. In H. Ganzinger, editor, Proceedings of the 16th International Conference on Automated Deduction (CADE-16), pages 287–291, Trento, Italy, July 1999. Springer-Verlag LNCS. 11. George C. Necula and Peter Lee. Efficient representation and validation of logical proofs. In Vaughan Pratt, editor, Proceedings of the 13th Annual Symposium on Logic in Computer Science (LICS’98), pages 93–104, Indianapolis, Indiana, June 1998. IEEE Computer Society Press. 12. Lawrence C. Paulson. Natural deduction as higher-order resolution. Journal of Logic Programming, 3:237–258, 1986. 13. Frank Pfenning. Unification and anti-unification in the Calculus of Constructions. In Sixth Annual IEEE Symposium on Logic in Computer Science, pages 74–85, Amsterdam, The Netherlands, July 1991. 14. Frank Pfenning and Rowan Davies. A judgmental reconstruction of modal logic. Mathematical Structures in Computer Science, 11:511–540, 2001. Notes to an invited talk at the Workshop on Intuitionistic Modal Logics and Applications (IMLA’99), Trento, Italy, July 1999. 15. Frank Pfenning and Carsten Schürmann. System description: Twelf — a meta- logical framework for deductive systems. In H. Ganzinger, editor, Proceedings of the 16th International Conference on Automated Deduction (CADE-16), pages 202–206, Trento, Italy, July 1999. Springer-Verlag, Lecture Notes in Artificial In- telligence (LNAI) 1632. 16. Jeff Polakow and Frank Pfenning. Ordered linear logic programming. Techni- cal Report CMU-CS-98-183, Department of Computer Science, Carnegie Mellon University, December 1998.