∃-InvSat (a.k.a. pp-definability) is co-NEXPTIME-complete Ross Willard University of Waterloo, Canada Dagstuhl Seminar 09441 October 30, 2009 Ross Willard (Waterloo) ∃-InvSat Dagstuhl 09441 1 / 18 Let Γ be a finite constraint language (on a domain D). Definition A pp-formula over Γ is a logical formula of the form ^ ∃y atomici (x, y) i where each atomici (x, y) is a constraint from Γ ∪ {=D }. Such formulas . are also called ∃-CNF (Γ) formulas. define implicit constraints of a CSP(Γ) instance. define conjunctive queries in database theory. Relations defined by such formulas are said to be . expressible by Γ (by CSP theorists). generated by Γ (by algebraists). Ross Willard (Waterloo) ∃-InvSat Dagstuhl 09441 2 / 18 ∃-InvSat is the following decision problem: Input: D – a finite domain Γ – a finite set of relations on D R – another relation on D. Question: is R pp-definable from Γ? We know that ∃-InvSat ... is at worst in co-NEXPTIME [from the Galois correspondence . ]. is locally (i.e., for each fixed Γ) in P when |D| = 2 [Dalmau ‘00]. is globally in P when |D| = 2 [Creignou, Kolaitis, Zanuttini ‘08]. Question: What is the exact complexity of ∃-InvSat in general? Ross Willard (Waterloo) ∃-InvSat Dagstuhl 09441 3 / 18 Theorem (W) ∃-InvSat is co-NEXPTIME-complete. Fine print In fact, there exists k > 2 such that ∃-InvSat restricted to k-element domains is co-NEXPTIME-complete. Remains hard even if, for some tuple d, we know that R ∪ {d} is pp-definable from Γ. Does not matter whether relations are represented as full truth tables or as lists of tuples. Outline of proof 1 Characterize “pp-definability from Γ” (in terms of polymorphisms). 2 Find a nice NEXPTIME-complete problem X. 3 Reduce X to ¬∃-InvSat (via polymorphisms). Ross Willard (Waterloo) ∃-InvSat Dagstuhl 09441 4 / 18 Step 1: Characterize pp-definability from Γ. Write B = (D; Γ). Fix n ≥ 1. Recipe: Let A = (A; ...) be any finite structure (of same type as B). n Fix an n-tuple c = (c1,..., cn) ∈ A . h Let HomA,B be the set of all homomorphisms A → B. n Collect all n-tuples (h(c1),..., h(cn)) ∈ D as h varies over HomA,B: h • (varies) c1 • . A c2 .. B = (D; Γ) • cn (fixed) Fact 1: {h(c): h ∈ HomA,B} is a typical relation pp-definable from Γ. Ross Willard (Waterloo) ∃-InvSat Dagstuhl 09441 5 / 18 Specialize by assuming A = Bm (for some m ≥ 1). h • (varies) m c1 • . B c2 .. B = (D; Γ) • cn (fixed) (Thus HomBm,B = {all m-ary polymorphisms of Γ}.) Define H(c) = {h(c): h ∈ HomBm,B} m P(c) = {pi (c) : 1 ≤ i ≤ m} ⊆ H(c) m where pi is the i-th dictator function of arity m. Fact 2: If P(c) ⊆ R ⊆ H(c), then R is pp-definable from Γ ⇔ R = H(c). Ross Willard (Waterloo) ∃-InvSat Dagstuhl 09441 6 / 18 Step 2: Find a nice NEXPTIME-complete problem. More precisely, a nice tiling problem. Roughly speaking, this involves: An unlimited supply of tiles, each having a tile type ∈ {t1,..., tk }. A positive integer N. One then attempts to cover an N × N grid with tiles, t4 t5 t5 t2 t1 t1 t3 t1 t2 t5 t4 t1 t4 t1 t4 t1 t4 t4 t5 t2 t4 t2 t5 t4 t1 t3 t4 t5 t5 t3 t4 t1 t5 t1 t5 t5 t5 t4 t5 t1 t5 t1 t5 t5 t5 t5 t1 t2 t2 t2 t3 t5 t3 t2 t1 t2 t3 t1 t1 t3 t1 t3 t1 t4 so that horizontally adjacent and vertically adjacent tiles satisfy some given constraints. Ross Willard (Waterloo) ∃-InvSat Dagstuhl 09441 7 / 18 More precisely: Definition Fix N ≥ 2. 1 A domino system is a finite relational structure D = (∆; H, V ) with H, V binary. (∆ = “tiles,” H = “horizontal,” V = “vertical.”) 2 [N] = {0, 1,..., N − 1}. 3 CN denotes the structure ([N] × [N]; ≺1, ≺2) where ≺1 = {(i, j), (i + 1, j): i, j ∈ [N], i < N − 1} ≺2 = {(i, j), (i, j + 1) : i, j ∈ [N], j < N − 1}. 4 An N × N tiling by D is a homomorphism τ : CN → D. m 5 Given w = (w0, w1,..., wm−1) ∈ ∆ with m ≤ N, we say that an N × N tiling τ satisfies initial condition w if τ(i, 0) = wi ∀i < m. Ross Willard (Waterloo) ∃-InvSat Dagstuhl 09441 8 / 18 Fix a domino system D = (∆; H, V ). ExpTile(D), the Exponential Tiling-by-D Problem, is: Input: w ∈ ∆m for some m ≥ 2. Question: does there exist a 2m × 2m tiling by D satisfying initial condition w? Fact 3: There exists D such that ExpTile(D) is NEXPTIME-complete. Fine print: Can even restrict to inputs where m is a power of 2. (Very nice.) Ross Willard (Waterloo) ∃-InvSat Dagstuhl 09441 9 / 18 Step 3. Reduce ExpTile(D) to ¬∃-InvSat. Fix an input w ∈ ∆m to ExpTile(D), m ≥ 2. (We must build D, Γ, R.) We’ll need a set D (small) on which to build a structure B = (D; Γ). We’ll encode a copy of [2m] × [2m] in Dm and a copy of ∆ in D. m We’ll identify auxiliary parameters ci ∈ D (1 ≤ i ≤ n) and > ∈ D. We’ll define Γ so that no polymorphism h : Bm → B can send c = (c1,..., cn) to t = (>,..., >), unless h sends “[2m] × [2m]” to “∆,” and m m h[2m]×[2m] is a 2 × 2 tiling by D satisfying initial condition w. Constraints: The number of relations in Γ must be O(mc ); The arity of each relation in Γ must all be O(log m). The number, n, of parameters ci must be O(log m). Ross Willard (Waterloo) ∃-InvSat Dagstuhl 09441 10 / 18 Recap: given input w ∈ ∆m to ExpTile(D), we’ll build h • > [2m] × [2m] (varies) Bm B = (D; Γ) ∆ c1 c2 cn • • ··· • t := (>,..., >) | {z } n=O(log m) n We’ll examine H(c) = {h(c): h ∈ HomBm,B} ⊆ D and its subset P(c) = {pi (c): i < m}. We’ll achieve: ∀h : Bm → B, h(c) = t ⇔ h encodes a [2m] × [2m] tiling by D satisfying i.c. w. (Conversely, ∃ a tiling ⇒ ∃ such h.) Hence: t ∈ H(c) ⇔ there exists a 2m × 2m tiling by D satisfying the input initial condition w. Ross Willard (Waterloo) ∃-InvSat Dagstuhl 09441 11 / 18 h • > [2m] × [2m] (varies) Bm B = (D; Γ) ∆ c1 c2 cn • • ··· • t := (>,..., >) | {z } n=O(log m) Moreover, We’ll find that there is an easily constructed n-ary relation R (not depending on w), satisfying the following: t 6∈ R. R ∪ {t} is (easily) pp-definable from Γ. P(c) ⊆ R ⊆ H(c) ⊆ R ∪ {t} Hence R ∪ {t} if ∃ 2m × 2m tiling by D with i.c. w H(c) = R otherwise. R is pp-df/Γ ⇔ there does not exist such a tiling. Ross Willard (Waterloo) ∃-InvSat Dagstuhl 09441 12 / 18 Details 1. Construct D. 00 01 10 11 ∞ > ⊥ D = a b ∆ 0 1 2. Encode ∆ and > in D. X 3. Encode [2m] × [2m] in Dm. (Assume m = 8.) To encode e.g. (53, 188) we do: 53 = 10101100 (least significant bit at left) 188 = 00111101 . (53, 188) = (10, 00, 11, 01, 11, 11, 00, 01) ∈ D8. Ross Willard (Waterloo) ∃-InvSat Dagstuhl 09441 13 / 18 m 4. Define auxiliary parameters ci ∈ D . We’ll use log2 m + 1 of them. When m = 8: c0 = (0, 1, 0, 1, 0, 1, 0, 1) c1 = (0, 0, 1, 1, 0, 0, 1, 1) c2 = (0, 0, 0, 0, 1, 1, 1, 1) c3 = (b, b, a, b, a, a, a, b). (Rule: c3(j) = a iff c0(j), c1(j), c2(j) contains a subsequence 0,1.) h • > [28] × [28] (varies) D8 D ∆ c0 c1 c2 c3 • • • • Next: add structure (Γ) to impose “tiling requirement” on h. Ross Willard (Waterloo) ∃-InvSat Dagstuhl 09441 14 / 18 5. Enforcing the horizontal adjacency constraints. Example (m = 8): if h : B8 → B and . (151, 54) = (10, 11, 11, 00, 11, 01, 00, 01) x . (152, 54) = (00, 01, 01, 10, 11, 01, 00, 01) y h c0 = ( 0 , 1 , 0 , 1 , 0 , 1 , 0 , 1 ) −→ > c1 = ( 0 , 0 , 1 , 1 , 0 , 0 , 1 , 1 ) > c2 = ( 0 , 0 , 0 , 0 , 1 , 1 , 1 , 1 ) > c3 = ( b , b , a , b , a , a , a , b ) > then we want this to imply x, y ∈ ∆ and moreover (x, y) ∈ H. Note that the “carry” in 151 + 1 = 152 occurs in column 4. We can build a 6-ary relation H4 which will enforce this implication for all horizontally adjacent pairs u, v ∈ [28] × [28] where the carry in the x increment occurs in column 4. Ross Willard (Waterloo) ∃-InvSat Dagstuhl 09441 15 / 18 Here it is: H4 = {(1y, 0y, c0(j), c1(j), c2(j), c3(j)) : y ∈ {0, 1}, 1 ≤ j < 4} ∪ {(0y, 1y, 1, 1, 0, b): y ∈ {0, 1}} ∪ {(xy, xy, c0(j), c1(j), c2(j), c3(j)) : x, y ∈ {0, 1}, 4 < j ≤ 8} ∪ {(x, y, >, >, >, >): x, y ∈ ∆ and (x, y) ∈ H} 2 4 ∪ {(x, y, b0, b1, b2, b3) ∈ ∆ × {>, ⊥} : ⊥ ∈ {b0, b2, b2, b3}} ∪ {(∞, ∞, ∞, ∞, ∞, ∞)}. Ross Willard (Waterloo) ∃-InvSat Dagstuhl 09441 16 / 18 Similarly, for each i we build a relation Hi to recognize carries at coordinate i. Do the same thing for vertical adjacencies. Ultimately, Γ will include: m relations H1,..., Hm of arity log2 m + 3 for horizontal constraints. m relations V1,..., Vm of arity log2 m + 3 for vertical constraints. w w m relations I1 ,..., Im of arity log2 m + 2 for the initial condition w. R will be the n := (log2 m + 1)-ary relation R = P(c) ∪ {>, ⊥}n \{(>, >,..., >)} ∪ {(∞, ∞,..., ∞)}.