Symmetry of Information and Nonuniform Lower Bounds

Home , Advice (complexity)

Symmetry of information and nonuniform lower bounds

Sylvain Perifel

March 7, 2007 Outline

1. Introduction and notations

2. Advices of size nc

3. Symmetry of information

4. Polynomial-size advices — nonuniform

— uniform

I P/poly: set of languages recognized by a family of polynomial-size boolean circuits (gates ∧, ∨ and ¬, one circuit per input length)

I Open question: EXP ⊂ P/poly?

I Main result: polynomial-time symmetry of information implies EXP 6⊂ P/poly.

Two complexity classes

I EXP: set of languages recognized in exponential time by a deterministic Turing machine

nk EXP = ∪k≥0DTIME(2 ). — uniform

— nonuniform

I Open question: EXP ⊂ P/poly?

I Main result: polynomial-time symmetry of information implies EXP 6⊂ P/poly.

Two complexity classes

I EXP: set of languages recognized in exponential time by a deterministic Turing machine

nk EXP = ∪k≥0DTIME(2 ).

I P/poly: set of languages recognized by a family of polynomial-size boolean circuits (gates ∧, ∨ and ¬, one circuit per input length) I Main result: polynomial-time symmetry of information implies EXP 6⊂ P/poly.

Two complexity classes

I EXP: set of languages recognized in exponential time by a deterministic Turing machine — uniform

nk EXP = ∪k≥0DTIME(2 ).

I P/poly: set of languages recognized by a family of polynomial-size boolean circuits (gates ∧, ∨ and ¬, one circuit per input length) — nonuniform

I Open question: EXP ⊂ P/poly? Two complexity classes

I EXP: set of languages recognized in exponential time by a deterministic Turing machine — uniform

nk EXP = ∪k≥0DTIME(2 ).

I P/poly: set of languages recognized by a family of polynomial-size boolean circuits (gates ∧, ∨ and ¬, one circuit per input length) — nonuniform

I Open question: EXP ⊂ P/poly?

I Main result: polynomial-time symmetry of information implies EXP 6⊂ P/poly. I EXP 6⊆ P (time hierarchy theorem).

I Space complexity version:

PSPACE ⊂ NC/poly?

I Even the question “EXP ⊂ L/poly?” is open.

Remarks

I EXP 6= P/poly (there are undecidable languages in P/poly). I Space complexity version:

PSPACE ⊂ NC/poly?

I Even the question “EXP ⊂ L/poly?” is open.

Remarks

I EXP 6= P/poly (there are undecidable languages in P/poly).

I EXP 6⊆ P (time hierarchy theorem). I Even the question “EXP ⊂ L/poly?” is open.

Remarks

I EXP 6= P/poly (there are undecidable languages in P/poly).

I EXP 6⊆ P (time hierarchy theorem).

I Space complexity version:

PSPACE ⊂ NC/poly? Remarks

I EXP 6= P/poly (there are undecidable languages in P/poly).

I EXP 6⊆ P (time hierarchy theorem).

I Space complexity version:

PSPACE ⊂ NC/poly?

I Even the question “EXP ⊂ L/poly?” is open. I If C is a complexity class and a : N → N a function, then C/a(n) is the set of languages A such that there exists B ∈ C and a function c : N → {0, 1}∗ satisfying:

I ∀n, |c(n)| ≤ a(n); ∗ I ∀x ∈ {0, 1} , x ∈ A ⇐⇒ (x, c(|x|)) ∈ B.

I “The class C is helped by the advice c(|x|)” (the same for all words of each length).

Advices

I A Turing machine can be helped by an advice (one word given for all inputs of same size). I “The class C is helped by the advice c(|x|)” (the same for all words of each length).

Advices

I A Turing machine can be helped by an advice (one word given for all inputs of same size).

I If C is a complexity class and a : N → N a function, then C/a(n) is the set of languages A such that there exists B ∈ C and a function c : N → {0, 1}∗ satisfying:

I ∀n, |c(n)| ≤ a(n); ∗ I ∀x ∈ {0, 1} , x ∈ A ⇐⇒ (x, c(|x|)) ∈ B. Advices

I A Turing machine can be helped by an advice (one word given for all inputs of same size).

I If C is a complexity class and a : N → N a function, then C/a(n) is the set of languages A such that there exists B ∈ C and a function c : N → {0, 1}∗ satisfying:

I ∀n, |c(n)| ≤ a(n); ∗ I ∀x ∈ {0, 1} , x ∈ A ⇐⇒ (x, c(|x|)) ∈ B.

I “The class C is helped by the advice c(|x|)” (the same for all words of each length). P({0, 1}∗).

I Even P/1 contains undecidable languages. . .

k I P/poly = ∪k≥0P/n (polynomial-size advice).

I P/poly: conversion advice ←→ boolean circuit.

Advices (continued)

n I P/2 = ? I Even P/1 contains undecidable languages. . .

k I P/poly = ∪k≥0P/n (polynomial-size advice).

I P/poly: conversion advice ←→ boolean circuit.

Advices (continued)

n ∗ I P/2 = P({0, 1} ). k I P/poly = ∪k≥0P/n (polynomial-size advice).

I P/poly: conversion advice ←→ boolean circuit.

Advices (continued)

n ∗ I P/2 = P({0, 1} ).

I Even P/1 contains undecidable languages. . . Advices (continued)

n ∗ I P/2 = P({0, 1} ).

I Even P/1 contains undecidable languages. . .

k I P/poly = ∪k≥0P/n (polynomial-size advice).

I P/poly: conversion advice ←→ boolean circuit. NP I Kannan 1982: NEXP 6⊂ P/poly;

I Schn¨oning1985: EXPSPACE 6⊂ P/poly.

c I Homer & Mocas 1995: ∀c > 0, EXP 6⊂ P/n .

I Here: symmetry of information (SI) ⇒ EXP 6⊂ P/poly;

I Lee & Romashchenko 2004: (SI) ⇒ EXP 6⊆ BPP (remark: BPP ⊂ P/poly, Adleman 1978).

The question “EXP ⊂ P/poly?”

I Simple diagonalization fails (too many circuits). c I Homer & Mocas 1995: ∀c > 0, EXP 6⊂ P/n .

I Here: symmetry of information (SI) ⇒ EXP 6⊂ P/poly;

I Lee & Romashchenko 2004: (SI) ⇒ EXP 6⊆ BPP (remark: BPP ⊂ P/poly, Adleman 1978).

The question “EXP ⊂ P/poly?”

I Simple diagonalization fails (too many circuits).

NP I Kannan 1982: NEXP 6⊂ P/poly;

I Schn¨oning1985: EXPSPACE 6⊂ P/poly. I Here: symmetry of information (SI) ⇒ EXP 6⊂ P/poly;

I Lee & Romashchenko 2004: (SI) ⇒ EXP 6⊆ BPP (remark: BPP ⊂ P/poly, Adleman 1978).

The question “EXP ⊂ P/poly?”

I Simple diagonalization fails (too many circuits).

NP I Kannan 1982: NEXP 6⊂ P/poly;

I Schn¨oning1985: EXPSPACE 6⊂ P/poly.

c I Homer & Mocas 1995: ∀c > 0, EXP 6⊂ P/n . The question “EXP ⊂ P/poly?”

I Simple diagonalization fails (too many circuits).

NP I Kannan 1982: NEXP 6⊂ P/poly;

I Schn¨oning1985: EXPSPACE 6⊂ P/poly.

c I Homer & Mocas 1995: ∀c > 0, EXP 6⊂ P/n .

I Here: symmetry of information (SI) ⇒ EXP 6⊂ P/poly;

I Lee & Romashchenko 2004: (SI) ⇒ EXP 6⊆ BPP (remark: BPP ⊂ P/poly, Adleman 1978). Lemma c If A ∈ P/n then there exists a constant k and a family (pn) of programs of size k + nc such that

I U(pn, x) = 1 iﬀ x ∈ A;

I U(pn, x) works in polynomial time.

Proof.

By deﬁnition, x ∈ A ⇐⇒ (x, c(|x|)) ∈ B. Then pn is merely the concatenation of the program for B and of c(n).

Advices of size nc

n I Words of {0, 1} are ordered lexicographically x1 < x2 < ··· < x2n .

I We ﬁx an “eﬃcient” universal Turing machine U. Proof.

By deﬁnition, x ∈ A ⇐⇒ (x, c(|x|)) ∈ B. Then pn is merely the concatenation of the program for B and of c(n).

Advices of size nc

n I Words of {0, 1} are ordered lexicographically x1 < x2 < ··· < x2n .

I We ﬁx an “eﬃcient” universal Turing machine U.

Lemma c If A ∈ P/n then there exists a constant k and a family (pn) of programs of size k + nc such that

I U(pn, x) = 1 iﬀ x ∈ A;

I U(pn, x) works in polynomial time. Advices of size nc

n I Words of {0, 1} are ordered lexicographically x1 < x2 < ··· < x2n .

I We ﬁx an “eﬃcient” universal Turing machine U.

Lemma c If A ∈ P/n then there exists a constant k and a family (pn) of programs of size k + nc such that

I U(pn, x) = 1 iﬀ x ∈ A;

I U(pn, x) works in polynomial time.

Proof.

By definition, x ∈ A ⇐⇒ (x, c(|x|)) ∈ B. Then pn is merely the concatenation of the program for B and of c(n). Proof. We build A by input sizes and word by word. Let t(n) = 2n1+c1c2 and a(n) = n + nc2 . Let us fix n and define A=n:

for at least half of the programs p of size ≤ a(n), x1 ∈ A ⇐⇒ t(n) U (p, x1) = 0.

(at least half of the programs give a wrong answer for x1).

Let V1 be the set of programs giving the right answer for x1.

Advices of size nc (continued)

Proposition

For all constants c1, c2 ≥ 1, there exists a sparse language A in 1+c c c DTIME(2n 1 2 ) but not in DTIME(2n 1 )/nc2 . Let V1 be the set of programs giving the right answer for x1.

Advices of size nc (continued)

Proposition

For all constants c1, c2 ≥ 1, there exists a sparse language A in 1+c c c DTIME(2n 1 2 ) but not in DTIME(2n 1 )/nc2 .

Proof. We build A by input sizes and word by word. Let t(n) = 2n1+c1c2 and a(n) = n + nc2 . Let us ﬁx n and deﬁne A=n:

for at least half of the programs p of size ≤ a(n), x1 ∈ A ⇐⇒ t(n) U (p, x1) = 0.

(at least half of the programs give a wrong answer for x1). Advices of size nc (continued)

Proposition

For all constants c1, c2 ≥ 1, there exists a sparse language A in 1+c c c DTIME(2n 1 2 ) but not in DTIME(2n 1 )/nc2 .

Proof. We build A by input sizes and word by word. Let t(n) = 2n1+c1c2 and a(n) = n + nc2 . Let us ﬁx n and deﬁne A=n:

for at least half of the programs p of size ≤ a(n), x1 ∈ A ⇐⇒ t(n) U (p, x1) = 0.

(at least half of the programs give a wrong answer for x1).

Let V1 be the set of programs giving the right answer for x1. and so on. . .

for at least half of the programs p ∈ Vk−1, xk ∈ A ⇐⇒ t(n) U (p, xk ) = 0. c The process stops when Vk is empty, that is, for k = n + n 2 . We decide that xj 6∈ A for j > k.

Advices of size nc (proof continued)

for at least half of the programs p ∈ V1, x2 ∈ A ⇐⇒ t(n) U (p, x2) = 0.

(at least half of the remaining programs are wrong on x2). for at least half of the programs p ∈ Vk−1, xk ∈ A ⇐⇒ t(n) U (p, xk ) = 0. c The process stops when Vk is empty, that is, for k = n + n 2 . We decide that xj 6∈ A for j > k.

Advices of size nc (proof continued)

for at least half of the programs p ∈ V1, x2 ∈ A ⇐⇒ t(n) U (p, x2) = 0.

(at least half of the remaining programs are wrong on x2).

and so on. . . Advices of size nc (proof continued)

for at least half of the programs p ∈ V1, x2 ∈ A ⇐⇒ t(n) U (p, x2) = 0.

(at least half of the remaining programs are wrong on x2).

and so on. . .

for at least half of the programs p ∈ Vk−1, xk ∈ A ⇐⇒ t(n) U (p, xk ) = 0. c The process stops when Vk is empty, that is, for k = n + n 2 . We decide that xj 6∈ A for j > k. nc1 c I A 6∈ DTIME(2 )/n 2 : any program with any advice makes at least one mistake;

n1+c1c2 I A ∈ DTIME(2 ).

Advices of size nc (proof continued)

c I A is sparse (at most n + n 2 elements of size n); n1+c1c2 I A ∈ DTIME(2 ).

Advices of size nc (proof continued)

c I A is sparse (at most n + n 2 elements of size n);

nc1 c I A 6∈ DTIME(2 )/n 2 : any program with any advice makes at least one mistake; Advices of size nc (proof continued)

c I A is sparse (at most n + n 2 elements of size n);

nc1 c I A 6∈ DTIME(2 )/n 2 : any program with any advice makes at least one mistake;

n1+c1c2 I A ∈ DTIME(2 ). Corollary For all k, PP 6⊂ DTIME(nk )/(n − log n).

Some consequences

Corollary For all constant c > 0, EXP 6⊂ P/nc and k c PSPACE 6⊂ (∪k DSPACE(log n)/n ). Some consequences

Corollary For all constant c > 0, EXP 6⊂ P/nc and k c PSPACE 6⊂ (∪k DSPACE(log n)/n ).

Corollary For all k, PP 6⊂ DTIME(nk )/(n − log n). I Resource-bounded Kolmogorov complexity: U is required to run within a time bound t

C t (x|y) = min{|p| : U t (p, y) = x}.

I Symmetry of information: C(x, y) ' C(x) + C(y|x). ≤: easy direction ≥: hard direction.

I Polynomial-time symmetry of information: easy direction still holds; hard direction is open! (true if P = NP, Longpr´e& Watanabe 1995).

Kolmogorov complexity

I Plain Kolmogorov complexity: C(x|y) = min{|p| : U(p, y) = x}. I Symmetry of information: C(x, y) ' C(x) + C(y|x). ≤: easy direction ≥: hard direction.

I Polynomial-time symmetry of information: easy direction still holds; hard direction is open! (true if P = NP, Longpr´e& Watanabe 1995).

Kolmogorov complexity

I Plain Kolmogorov complexity: C(x|y) = min{|p| : U(p, y) = x}.

I Resource-bounded Kolmogorov complexity: U is required to run within a time bound t

C t (x|y) = min{|p| : U t (p, y) = x}. I Polynomial-time symmetry of information: easy direction still holds; hard direction is open! (true if P = NP, Longpr´e& Watanabe 1995).

Kolmogorov complexity

I Plain Kolmogorov complexity: C(x|y) = min{|p| : U(p, y) = x}.

I Resource-bounded Kolmogorov complexity: U is required to run within a time bound t

C t (x|y) = min{|p| : U t (p, y) = x}.

I Symmetry of information: C(x, y) ' C(x) + C(y|x). ≤: easy direction ≥: hard direction. Kolmogorov complexity

I Plain Kolmogorov complexity: C(x|y) = min{|p| : U(p, y) = x}.

I Resource-bounded Kolmogorov complexity: U is required to run within a time bound t

C t (x|y) = min{|p| : U t (p, y) = x}.

I Symmetry of information: C(x, y) ' C(x) + C(y|x). ≤: easy direction ≥: hard direction.

I Polynomial-time symmetry of information: easy direction still holds; hard direction is open! (true if P = NP, Longpr´e& Watanabe 1995). Remark: stronger version than the usual one p(n)q(n) instead of q(p(n)).

Symmetry of information

Hypothesis (SI)

There exists a polynomial q such that for all p and all words x, y, z of size |x| + |y| + |z| = n:

C p(n)(x, y|z) ≥ C p(n)q(n)(x|z) + C p(n)q(n)(y|x, z) − O(log n). Symmetry of information

Hypothesis (SI)

There exists a polynomial q such that for all p and all words x, y, z of size |x| + |y| + |z| = n:

C p(n)(x, y|z) ≥ C p(n)q(n)(x|z) + C p(n)q(n)(y|x, z) − O(log n).

Remark: stronger version than the usual one p(n)q(n) instead of q(p(n)). Proof.

Show by induction on n that ∀z, if (∀j, C(uj |u1,..., uj−1, z) ≥ k) then C(u1,..., un|z) ≥ nk − (n − 1)O(log m).

t tq(m) C (u1,..., un|z) ≥ C (u1,..., un/2|z)+ tq(m) C (un/2+1,..., un|u1,..., un/2, z) − O(log m).

Iterations of (SI)

Lemma Suppose (SI) holds.

Let u1,..., un be words of size s and let z be another word. Let m = ns + |z|. Suppose there exists k such that for all j ≤ n,

tq(m)log n C (uj |u1,..., uj−1, z) ≥ k.

t Then C (u1,..., un|z) ≥ nk − (n − 1)O(log m). Iterations of (SI)

Lemma Suppose (SI) holds.

Let u1,..., un be words of size s and let z be another word. Let m = ns + |z|. Suppose there exists k such that for all j ≤ n,

tq(m)log n C (uj |u1,..., uj−1, z) ≥ k.

t Then C (u1,..., un|z) ≥ nk − (n − 1)O(log m).

Proof.

Show by induction on n that ∀z, if (∀j, C(uj |u1,..., uj−1, z) ≥ k) then C(u1,..., un|z) ≥ nk − (n − 1)O(log m).

t tq(m) C (u1,..., un|z) ≥ C (u1,..., un/2|z)+ tq(m) C (un/2+1,..., un|u1,..., un/2, z) − O(log m). Proof. If A ∈ DTIME(r(n))/a(n) then χn[1..i] is computed in time ir(n) with a program of size a(n) + O(1).

Links Kolmogorov/nonuniform complexity

Characteristic string χn ∈ {0, 1}2n of A=n:

n =n χi = 1 ⇐⇒ xi ∈ A .

Lemma Suppose that there exist inﬁnitely many n and 1 ≤ i ≤ 2n satisfying

C ir(n)(χn[1..i]) > n + a(n).

Then A 6∈ DTIME(r(n))/a(n). Links Kolmogorov/nonuniform complexity

Characteristic string χn ∈ {0, 1}2n of A=n:

n =n χi = 1 ⇐⇒ xi ∈ A .

Lemma Suppose that there exist inﬁnitely many n and 1 ≤ i ≤ 2n satisfying

C ir(n)(χn[1..i]) > n + a(n).

Then A 6∈ DTIME(r(n))/a(n).

Proof. If A ∈ DTIME(r(n))/a(n) then χn[1..i] is computed in time ir(n) with a program of size a(n) + O(1). I In EXP, impossible to diagonalize over all advices of polynomial size

I → we cut the advices into blocks of size n and diagonalize over these blocks;

I then we “glue” these blocks back thanks to (SI).

Polynomial-size advices — the idea

n I U will return χ instead of recognizing each word. I → we cut the advices into blocks of size n and diagonalize over these blocks;

I then we “glue” these blocks back thanks to (SI).

Polynomial-size advices — the idea

n I U will return χ instead of recognizing each word.

I In EXP, impossible to diagonalize over all advices of polynomial size I then we “glue” these blocks back thanks to (SI).

Polynomial-size advices — the idea

n I U will return χ instead of recognizing each word.

I In EXP, impossible to diagonalize over all advices of polynomial size

I → we cut the advices into blocks of size n and diagonalize over these blocks; Polynomial-size advices — the idea

n I U will return χ instead of recognizing each word.

I In EXP, impossible to diagonalize over all advices of polynomial size

I → we cut the advices into blocks of size n and diagonalize over these blocks;

I then we “glue” these blocks back thanks to (SI). Proof. 3 We build A by input sizes and word by word. Let t(n) = nO(log n). Let us ﬁx n and deﬁne A=n:

for at least half of the programs p of size ≤ n, x ∈ A ⇐⇒ 1 the ﬁrst bit of U t(n)(p) is 0.

(at least half of the programs give a wrong answer for x1).

Let V1 be the set of programs giving the right answer for x1.

Main result

Theorem If (SI) holds, then EXP 6⊂ P/poly. Let V1 be the set of programs giving the right answer for x1.

Main result

Theorem If (SI) holds, then EXP 6⊂ P/poly.

Proof. 3 We build A by input sizes and word by word. Let t(n) = nO(log n). Let us ﬁx n and deﬁne A=n:

for at least half of the programs p of size ≤ n, x ∈ A ⇐⇒ 1 the ﬁrst bit of U t(n)(p) is 0.

(at least half of the programs give a wrong answer for x1). Main result

Theorem If (SI) holds, then EXP 6⊂ P/poly.

Proof. 3 We build A by input sizes and word by word. Let t(n) = nO(log n). Let us ﬁx n and deﬁne A=n:

for at least half of the programs p of size ≤ n, x ∈ A ⇐⇒ 1 the ﬁrst bit of U t(n)(p) is 0.

(at least half of the programs give a wrong answer for x1).

Let V1 be the set of programs giving the right answer for x1. We call u(1) the n ﬁrst bits of the characteristic string of A=n just deﬁned. Then:

for at least half of the programs p of size ≤ n, x ∈ A ⇐⇒ n+1 the ﬁrst bit of U t(n)(p, u(1)) is 0.

(at least half of the programs are wrong on xn+1, even with the advice u(1)).

Proof continued

We go on like this as before, discarding half of the remaining programs at each step:

for at least half of the programs p ∈ V , x ∈ A ⇐⇒ n−1 n the n-th bit of U t(n)(p) is 0. Proof continued

We go on like this as before, discarding half of the remaining programs at each step:

for at least half of the programs p ∈ V , x ∈ A ⇐⇒ n−1 n the n-th bit of U t(n)(p) is 0.

We call u(1) the n ﬁrst bits of the characteristic string of A=n just deﬁned. Then:

for at least half of the programs p of size ≤ n, x ∈ A ⇐⇒ n+1 the ﬁrst bit of U t(n)(p, u(1)) is 0.

(at least half of the programs are wrong on xn+1, even with the advice u(1)). And so on, until the next segment u(2) of size n is deﬁned. Then:

for at least half of the programs p of size ≤ n, x ∈ A ⇐⇒ 2n+1 the ﬁrst bit of U t(n)(p, u(1), u(2)) is 0.

(at least half of the programs make a wrong answer for x2n+1, even with the advice u(1), u(2)).

Proof continued

Keep going on: call V1 the set of programs that where right at the preceding step.

for at least half of the programs p ∈ V , x ∈ A ⇐⇒ 1 n+2 the second bit of U t(n)(p, u(1)) is 0. Proof continued

Keep going on: call V1 the set of programs that where right at the preceding step.

for at least half of the programs p ∈ V , x ∈ A ⇐⇒ 1 n+2 the second bit of U t(n)(p, u(1)) is 0.

And so on, until the next segment u(2) of size n is deﬁned. Then:

for at least half of the programs p of size ≤ n, x ∈ A ⇐⇒ 2n+1 the ﬁrst bit of U t(n)(p, u(1), u(2)) is 0.

(at least half of the programs make a wrong answer for x2n+1, even with the advice u(1), u(2)). Proof continued

log n =n We deﬁne n segments of size n and decide that xj 6∈ A for j > n × nlog n.

I A 6∈ P/poly because for all j, C t(n)(u(j)|u(1),..., u(j−1)) ≥ n − 1. Thus by iteratively applying (SI), C t (χn[1..n1+log n]) ≥ nlog n.

I A ∈ EXP. Conclusion

I Good idea to study (SI): if true, then EXP 6⊂ P/poly; if false, then P 6= NP...

I What about the usual version of (SI) (with time bound q(p(n)) instead of q(n)p(n))?

I Hope: unconditionnal results by using CAMD (a version of Kolmogorov complexity based on the class AM). Outline

1. Introduction and notations

2. Advices of size nc

3. Symmetry of information

4. Polynomial-size advices