G22.2390-001 Logic in Computer Science Fall 2007 Lecture 8
1 Review
Last time
• Compactness
• Enumerability Theorem
• Definability of Models
• Finite Models
• Size of Models
2 Outline
• Theories
• Congruence Closure
• Interpretations Between Theories
Sources:
Sections 2.6 through 2.7 of Enderton.
Z. Manna and C. Zarba. Combining Decision Procedures. Draft available from http://theory.stanford.edu/∼zm/new-papers.html.
G. Nelson and D. Oppen. Fast Decision Procedures Based on Congruence Closure. JACM 27(2), 1980, pp. 356-364.
P. Downey, R. Sethi, and R. Tarjan. Variations on the Common Subexpression Problem. JACM 27(4), 1980, pp. 758-771.
3 Size of Models
The cardinality |L| of a language L is the least infinite cardinal greater than or equal to the number of symbols in the signature of L.
The cardinality |M| of a model M is the cardinality of its domain dom(M).
Lowenheim-Skolem¨ (LS) Theorem
Let Γ be a satisfiable set of formulas in a language L, then Γ is satisfiable in some model of cardinality κ ≤|L|.
Proof
By soundness, Γ is consistent, and is thus satisfiable by the model constructed in the proof of the completeness theorem. But the domain of that model is M/E which has cardinality ≤|M|, and |M| = |L|. 2
4 Size of Models
“Skolem’s paradox”
Let AST be your favorite set of axioms for set theory. If they are consistent, they have a model. Because the signature of the language of set theory is finite, there is a countable model. But we can prove, starting with the axioms of set theory, that there are “uncountably” many sets.
How is this possible?
5 Size of Models
“Skolem’s paradox”
Let AST be your favorite set of axioms for set theory. If they are consistent, they have a model. Because the signature of the language of set theory is finite, there is a countable model. But we can prove, starting with the axioms of set theory, that there are “uncountably” many sets.
How is this possible?
The answer is that in the countable model of set theory, things do not correspond to what we normally think of as the model of set theory. Thus, the model of the “natural numbers” in the countable model cannot be put in one-to-one correspondence with all of the elements of the model. But this does not mean that the size of the model is truly uncountable.
5-a Size of Models
LST Theorem
Let Γ be a set of formulas in a language of cardinality κ, and assume that Γ is satisfiable in some infinite model. Then for every cardinal λ ≥ κ, there is a model of cardinality λ in which Γ is satisfiable.
Proof
Let M be an infinite model where Γ is satisfiable. Expand the language by adding a set C of λ new constant symbols. Let ∆= {c1 6= c2 | c1,c2 are ′ distinct members of C}. Then, any finite subset Γ0 of Γ ∪ ∆ is satisfiable in M ′ where M is M extended to map all constants in Γ0 to different elements of M . By compactness, Γ ∪ ∆ is satisfiable. By the LS Theorem, there is a model of cardinality ≤ λ. But any model must have at least cardinality λ. Thus there is a model of cardinality λ. 2
6 Size of Models
Corollary
If Γ is a set of sentences in a countable language, then if Γ has some infinite model, it has models of every infinite cardinality. ′ Two models M and M with the same signature are elementarily equivalent ′ (M ≡ M ) iff for any sentence σ, |=M σ iff |=M ′ σ.
Corollary
If M is an infinite model for a countable language, then for any infinite cardinal λ, ′ ′ there is a model M of cardinality λ such that M ≡ M .
Proof
Let Γ be the set of all sentences true in M . By the corollary above, Γ has a ′ model M of cardinality λ. But note that for every sentence σ, either σ ∈ Γ or ′ ¬σ ∈ Γ (why?). Thus, M ≡ M . 2
7 Theories
Last time, we defined a theory as a set of first-order sentences.
For this lecture we will refine our definition to be a set of first-order sentences closed under logical implication.
Thus, T is a theory iff T is a set of sentences and if T |= σ, then σ ∈ T for every sentence σ.
Examples
• For a given signature, the smallest possible theory consists of exactly the valid sentences over that signature.
• The largest theory for a given signature is the set of all sentences. It is the only unsatisfiable theory. Why?
8 Theories
For a class K of models over a given signature Σ, define the theory of K as
Th K = {σ | σ is a Σ-sentence which is true in every model in K}.
Theorem
Th K is indeed a theory.
Proof
Suppose Th K|= σ. We know that |=M Th K for each M in K. It follows that |=M σ for each M in K, and thus σ ∈ Th K. 2
Suppose Γ is a set of sentences.
Define the set Cn Γ of consequences of Γ to be {σ | Γ |= σ}.
Then Cn Γ= Th Mod Γ.
9 Theories
A theory T is complete iff for every sentence σ, either σ ∈ T or (¬σ) ∈ T . Note that if M is a model, then Th {M} is complete. In fact, for a class K of models, Th K is complete iff any two members of K are elementarily equivalent. A theory T is axiomatizable iff there is a decidable set Γ of sentences such that T = Cn Γ. A theory T is finitely axiomatizable iff T = Cn Γ for some finite set Γ of sentences.
Theorem
If Cn Γ is finitely axiomatizable, then there is a finite Γ0 ⊆ Γ such that Cn Γ0 = Cn Γ.
Proof
If Cn Γ is finitely axiomatizable, then for some sentence τ , Cn Γ= Cn τ . Clearly, Γ |= τ . By compactness, we have that there exists Γ0 ⊆ Γ such that Γ0 |= τ . Thus, Cn τ ⊆ Cn Γ0 ⊆ Cn Γ, and since Cn Γ= Cn τ , it follows that Cn Γ0 = Cn Γ. 2 10 Theories
Using the above terminology, we can restate our earlier results as follows:
• An axiomatizable theory (in a reasonable language) is effectively enumerable. • A complete axiomatizable theory (in a reasonable language) is decidable.
Our results about theories can be summarized in the following diagram.
Decidable Finitely axiomatizable if complete
Effectively Enumerable Axiomatizable
11 Los-Vaught Test
For a theory T and a cardinal λ, say that T is λ-categorical iff all models of T having cardinality λ are isomorphic.
Theorem
Let T be a theory in a countable language such that • T is λ-categorical for some infinite cardinal λ. • All models of T are infinite. Then T is complete.
Proof ′ ′ It suffices to show that for any two models M and M of T , M ≡ M . Since M ′ and M are infinite, there exist (by LST) elementarily equivalent models of cardinality λ. But these models must be isomorphic, and by the homomorphism theorem, isomorphic models are elementarily equivalent. 2
12 Validity and Satisfiability Modulo Theories
Given a Σ-theory T ,a Σ-formula φ is
1. T -valid if |=M φ[s] for all models M of T and all variable assignments s.
2. T -satisfiable if there exists some model M of T and variable assignment s such that |=M φ[s].
3. T -unsatisfiable if 6|=M φ[s] for all models M of T and all variable assignments s.
The validity problem for T is the problem of deciding, for each Σ-formula φ, whether φ is T -valid.
The satisfiability problem for T is the problem of deciding, for each Σ-formula φ, whether φ is T -satisfiable.
Similarly, one can define the quantifier-free validity problem and the quantifier-free satisfiability problem for a Σ-theory T by restricting the formula φ to be quantifier-free.
13 Validity and Satisfiability Modulo Theories
A decision problem is decidable if there exists an effective procedure which always terminates with an answer for any given instance of the problem.
For example, the validity problem for a Σ-theory T is decidable if there exists an effective procedure for determining whether T |= φ for every Σ-formula φ.
Note that validity problems can always be reduced to satisfiability problems:
φ is T -valid iff ¬φ is T -unsatisfiable.
We will consider a few examples of theories which are of particular interest in verification applications.
14 The Theory TE of Equality
The theory TE of equality is the theory Cn ∅.
Note that the exact set of sentences in TE depends on the signature in question.
The theory does not restrict the possible values of symbols in any way. For this reason, it is sometimes called the theory of equality with uninterpreted functions (EUF).
The satisfiability problem for TE is just the satisfiability problem for first order logic, which is undecidable.
The satisfiability problem for conjunctions of literals in TE is decidable in polynomial time using congruence closure.
15 The Theory TZ of Integers
Let ΣZ be the signature (0, 1, +, −, ≤).
Let AZ be the standard model of the integers with domain Z.
Then TZ is defined to be Th AZ .
As showed by Presburger in 1929, the validity problem for TZ is decidable, but its complexity is triply-exponential.
The quantifier-free satisfiability problem for TZ is “only” NP-complete. × Let ΣZ be the same as ΣZ with the addition of the symbol × for multiplication, × × and define AZ and TZ in the obvious way. × The satisfiability problem for TZ is undecidable (a consequence of Godel’s¨ incompleteness theorem). × In fact, even the quantifier-free satisfiability problem for TZ is undecidable.
16 The Theory TR of Reals
Let ΣR be the signature (0, 1, +, −, ≤).
Let AR be the standard model of the reals with domain R.
Then TR is defined to be Th AR.
The satisfiability problem for TR is decidable, but the complexity is doubly-exponential.
The quantifier-free satisfiability problem for conjunctions of literals (atomic formulas or their negations) in TR is solvable in polynomial time, though exponential methods (like Simplex or Fourier-Motzkin) often perform better in practice. × Let ΣR be the same as ΣR with the addition of the symbol × for multiplication, × × and define AR and TR in the obvious way. × In contrast to the theory of integers, the satisfiability problem for TR is decidable.
17 The Theory TA of Arrays
Let ΣA be the signature (read , write ).
Let ΛA be the following axioms:
∀ a ∀ i ∀ v (read (write (a, i, v), i)= v) ∀ a ∀ i ∀ j ∀ v (i 6= j → read (write (a, i, v),j)= read (a, j)) ∀ a ∀ b ((∀ i (read (a, i)= read (b, i))) → a = b)
Then TA = Cn ΛA.
The satisfiability problem for TA is undecidable, but the quantifier-free satisfiability problem for TA is decidable (the problem is NP-complete).
18 Theories of Inductive Data Types
An inductive data type (IDT) defines one or more constructors, and possibly also selectors and testers.
Example: list of int
• Constructors: cons :(int, list) → list, null : list • Selectors: car : list → int, cdr : list → list • Testers: is cons, is null
The first order theory of a inductive data type associates a function symbol with each constructor and selector and a predicate symbol with each tester.
Example: ∀ x : list. (x = null ∨∃ y : int, z : list. x = cons(y, z))
For IDTs with a single constructor, a conjunction of literals is decidable in polynomial time.
For more general IDTs, the problem is NP-complete, but reasonbly efficient algorithms exist in practice.
19 Other Interesting Theories
Some other interesting theories include:
• Theories of bit-vectors
• Fragments of set theory
20 Congruence Closure
Let G =(V, E) be a directed graph such that for each vertex v in G, the successors of v are ordered.
Let C be any equivalence relation on V . ∗ The congruence closure C of C is the finest equivalence relation on V that contains C and satisfies the following property for all vertices v and w.
Let v and w have successors v1,...,vk and w1,...,wl respectively. ∗ ∗ If k = l and (vi, wi) ∈ C for 1 ≤ i ≤ k, then (v, w) ∈ C .
In other words, if the corresponding successors of v and w are equivalent under ∗ ∗ C , then v and w are themselves equivalent under C .
Often, the vertices are labeled by some labeling function λ. In this case, the property becomes: ∗ If λ(v)= λ(w) and if k = l and (vi, wi) ∈ C for 1 ≤ i ≤ k, then ∗ (v, w) ∈ C .
21 A Simple Algorithm
Let C0 = C and i = 0.
1. Number the equivalence classes in Ci consecutively from 1.
2. Let α assign to each vertex v the number α(v) of the equivalence class containing v.
3. For each vertex v construct a signature s(v)= λ(v)(α(v1),...,α(vk)), where v1,...,vk are the successors of v.
4. Group the vertices into classes of vertices having equal signatures.
5. Let Ci+1 be the finest equivalence relation on V such that two vertices equivalent under Ci or having the same signature are equivalent under Ci+1. ∗ 6. If Ci+1 = Ci, let C = Ci; otherwise increment i and repeat.
22 Congruence Closure and TE
Recall that TE is the empty theory with equality over some signature Σ containing only function symbols.
If Γ is a set of ground Σ-equalities and ∆ is a set of ground Σ-disequalities, then the satisfiability of Γ ∪ ∆ can be determined as follows.
• Let G be a graph which corresponds to the abstract syntax trees of terms in Γ ∪ ∆, and let vt denote the vertex of G associated with the term t.
• Let C be the equiavlence relation on the vertices of G induced by Γ. ∗ • Γ ∪ ∆ is satisfiable iff for each s 6= t ∈ ∆, (vs, vt) 6∈ C .
23 An Algorithm for TE union and find are abstract operations for manipulating equivalence classes. union(x, y) merges the equivalence classes of x and y.
find(x) returns a unique representative of the equivalence class of x. CC(Γ, ∆) Construct G(V, E) from terms in Γ and ∆.
while Γ 6= ∅ Remove some equality a = b from Γ; Merge(a, b);
if find(a)= find(b) for some a 6= b ∈ ∆ then
return false ;
return true ;
24 An Algorithm for TE
Merge(a, b)
if find(a)= find(b) then return; Let A be the set of all predecessors of all vertices equivalent to a; Let B be the set of all predecessors of all vertices equivalent to b; union(a, b);
foreach x ∈ A and y ∈ B
if signature(x)= signature(y) then Merge(x, y);
25 Congruence Closure
DST Algorithm
The Downey-Sethi-Tarjan Congruence Closure algorithm is more efficient. It makes use of some additional data structures and methods.
Additional Helper Methods • union(a, b) in this algorithm, the first argument always becomes the new equivalence class representative. • list(e) returns the list of vertices with at least one successor in equivalence class e. • enter(v) stores (v, signature(v)) in a signature table. • delete(v) removes (v, signature(v)) from the signature table if it is there. Note that this operation does not remove any other entry, even if it has the same signature as v. • query(v) if there is an entry (w, signature(w) in the signature table, and signature(w)= signature(v), then return w; otherwise, return ⊥.
26 DST Algorithm
CC(Γ, ∆) Construct G(V, E) from terms in Γ and ∆. Merge(Γ);
if find(a)= find(b) for some a 6= b ∈ ∆ then
return false ;
return true ;
27 DST Algorithm
Merge(combine)
pending := set of all vertices;
while pending 6= ∅
foreach v ∈ pending
if query(v)= ⊥ then enter(v);
else add (v, query(v)) to combine;
pending := ∅;
foreach (a, b) ∈ combine
if find(a) 6= find(b) then
if |list(find(a))| < |list(find(b))| then swap a and b;
foreach u ∈ list(find(b))
delete(u); add u to pending; union(find(a), find(b));
combine := ∅;
28 Interpretations Between Theories
Given two theories, T0 and T1, sometimes it is possible to show that one of the theories is at least as powerful as the other.
A simple case is when T0 and T1 are in the same language and one is a subset of the other.
We will consider more general cases in which the languages of the two theories differ.
29 Defining Functions
A common practice in mathematical reasoning is to introduce a new piece of notation, defining it in terms of a formula not containing the new notation.
Formally, suppose Σ is a signature and f is a function symbol not in Σ. Let Σ+ = Σ ∪ {f}. A function definition is a formula of the form:
∀ v1 ∀ v2 (fv1 = v2 ↔ φ) where φ is a Σ-formula in which only v1 and v2 may occur free. Let the above sentence be designated δ.
Theorem
The following are equivalent: 1. The definition is non-creative, i.e. for any Σ-sentence σ if T ∪ {δ}|= σ in Σ+, then T |= σ in Σ. 2. f is well-defined, i.e. the following sentence (which we designate ǫ) is in the theory T :
∀ v1 ∃! v2 φ. Note that ∃! means “there exists uniquely”. Technically it is an abbreviation for a more complicated formula which expresses both existence and uniqueness. 30 Defining Functions
Proof
To show that (1) implies (2), note that δ |= ǫ. Thus, if we take σ ≡ ǫ in (1), it follows that T |= ǫ.
To show that (2) implies (1), suppose that T |= ǫ. Let M be a Σ-model of T . For d ∈ dom(M), let F (d) be the unique e ∈ dom(M) such that M |= φ[[d, e]]. The existence of such an e for each d is ensured because T |= ǫ. Now let (M,F ) be the Σ+ model which agrees with M on all parameters except F and which assigns F to the symbol f . It is easy to see that (M,F ) is a model of δ.
Thus, if T ∪ {δ}|= σ, then (M,F ) |= σ. But since σ is a Σ-sentence, it follows that M |= σ. Since M was chosen arbitrarily, it follows that T |= σ. 2
31 Interpretations
There are more general ways in which one theory can be as strong as another theory in another language.
Example
Consider the theory of (N , 0,S) (natural numbers with 0 and successor) and on the other hand the theory of (Z, +, ×).
The second theory is at least as strong as the first. To show this, we make the following observations: • An integer is nonnegative iff it is the sum of four squares.
• The set {0} is defined by v1 + v1 = v1. • The successor relation is defined by ∀ z (z × z = z ∧ z + z 6= z → v1 + z = v2). Thus, for example, the sentence ∀ x Sx 6= 0 can be translated as
∀ x [∃ y1 ∃ y2 ∃ y3 ∃ y4 x = y1 × y1 + y2 × y2 + y3 × y3 + y4 × y4 → ¬∀ u (u + u = u →∀ v (∀ z (z × z = z ∧ z + z 6= z → x + z = v) → v = u))].
32 Interpretations
Suppose Σ0 and Σ1 are signatures and T1 is a Σ1-theory.
An interpretation π of Σ0 into T1 consists of the following three items.
1. a Σ1-formula π∀ in which at most v1 occurs free, such that (i) T1 |= ∃ v1 π∀.
2. a Σ1-formula πP for each n-ary predicate symbol P ∈ Σ0 in which at most the variables v1,...,vn occur free.
3. a Σ1-formula πf for each n-ary function symbol f ∈ Σ0 in which at most v1,...,vn, vn+1 occur free such that
(ii) T1 |= ∀ v1,...,vn(π∀(v1) →···→ π∀(vn)
→ ∃ x (π∀(x) ∧∀ vn+1 (πf (v1,...,vn+1) ↔ vn+1 = x))). For our previous example, we have
1. π∀(x)= ∃ y1 ∃ y2 ∃ y3 ∃ y4 x = y1 × y1 + y2 × y2 + y3 × y3 + y4 × y4
2. π0(x)= x + x = x
3. πS(x, y)= ∀ y (z × z ∧ z + z 6= z → x + z = y)
33 Interpretations
Assume that π is an interpretation and let M be a model of T1. π There is a natural way to extract from M a model M for Σ0. Let π • dom( M)= the set defined in M by π∀,
πM π • P = the relation defined in M by πP , restricted to dom( M). πM • f (a1,...,an)= the unique b such that M |= πf [[a1,...,an,b]], π where a1,...,an are in dom( M). π By condition (i), dom( πM) 6= ∅. By condition (ii), the definition of f M makes sense. −1 Define the set π [T1] of Σ0-sentences as π Th { M | M ∈ Mod T1}. −1 In other words, π [T1] is the set of all Σ0-sentences true in every model obtainable from a model of T1 as shown above.
34 Interpretations
Given a Σ0 formula φ and an interpretation π of Σ0 into T1, we can find a formula φπ which in some sense corresponds exactly to φ.
For an atomic formula α, we scan the formula from right to left. When a function symbol f is found, applied to arguments x1,...,xn, it is replaced by a new variable y and the atomic formula is prefixed by ∀ y (πf (x1,...,xn, y) →. We continue until there are no more function symbols. Finally, we replace the predicate symbol P (if it is not equality) by πP .
For example, π π (Pfgx) = ∀ y (πg(x, y) → (P fy) ) π = ∀ y (πg(x, y) →∀ z(πf (y, z) → (P z) ))
= ∀ y (πg(x, y) →∀ z(πf (y, z) → πP (z))).
The interpretation of non-atomic formulas are defined in the obvious way: (¬φ)π =(¬φπ), (φ → ψ)π =(φπ → ψπ), and π π (∀ xφ) = ∀ x (π∀(x) → φ ).
35 Interpretations
Lemma
Let π be an interpretation of Σ0 into T1 and M a model of T1. For any π Σ0-formula φ and any map s of the variables into dom( M), π |= πM φ[s] iff |=M φ [s].
The proof is by induction on φ and is omitted.
Corollary
−1 π For a Σ0-sentence σ, σ ∈ π [T1] iff σ ∈ T1.
An interpretation π of a theory T0 into a theory T1 is an interpretation π of the −1 signature of T0 into T1 such that T0 ⊆ π [T1]. −1 If T0 = π [T1], then π is said to be a faithful interpretation of T0 into T1. π For a faithful interpretation, we have σ ∈ T0 iff σ ∈ T1.
36