Probabilistic Computation
Lecture 13 Understanding BPP
1 Recap
2 Recap
Probabilistic computation
2 Recap
Probabilistic computation
NTM (on “random certificates”) for L:
2 Recap
Probabilistic computation
NTM (on “random certificates”) for L:
Pr[M(x)=yes]:
2 Recap
Probabilistic computation
NTM (on “random certificates”) for L: x∉L x∈L Pr[M(x)=yes]:
2 Recap
Probabilistic computation
NTM (on “random certificates”) for L: x∉L x∈L Pr[M(x)=yes]: x∉L x∈L PTM for L: Pr[yes]:
2 Recap
Probabilistic computation
NTM (on “random certificates”) for L: x∉L x∈L Pr[M(x)=yes]: x∉L x∈L PTM for L: Pr[yes]: x∉L x∈L BPTM for L: Pr[yes]:
2 Recap
Probabilistic computation
NTM (on “random certificates”) for L: x∉L x∈L Pr[M(x)=yes]: x∉L x∈L PTM for L: Pr[yes]: x∉L x∈L BPTM for L: Pr[yes]: x∉L x∈L RTM for L: Pr[yes]:
2 Recap
3 Recap PP, RP, co-RP, BPP
3 Recap PP, RP, co-RP, BPP
PP too powerful: NP ⊆ PP
3 Recap PP, RP, co-RP, BPP
PP too powerful: NP ⊆ PP
RP, BPP, with bounded gap
3 Recap PP, RP, co-RP, BPP
PP too powerful: NP ⊆ PP
RP, BPP, with bounded gap
Gap can be boosted from 1/poly to 1-1/exp
3 Recap PP, RP, co-RP, BPP
PP too powerful: NP ⊆ PP
RP, BPP, with bounded gap
Gap can be boosted from 1/poly to 1-1/exp
A realistic/useful computational model
3 Recap PP, RP, co-RP, BPP
PP too powerful: NP ⊆ PP
RP, BPP, with bounded gap
Gap can be boosted from 1/poly to 1-1/exp
A realistic/useful computational model
To d a y:
3 Recap PP, RP, co-RP, BPP
PP too powerful: NP ⊆ PP
RP, BPP, with bounded gap
Gap can be boosted from 1/poly to 1-1/exp
A realistic/useful computational model
To d a y:
NP ⊈ BPP, unless PH collapses
3 Recap PP, RP, co-RP, BPP
PP too powerful: NP ⊆ PP
RP, BPP, with bounded gap
Gap can be boosted from 1/poly to 1-1/exp
A realistic/useful computational model
To d a y:
NP ⊈ BPP, unless PH collapses
P P BPP ⊆ Σ2 ∩ Π2
3 BPP vs. NP
4 BPP vs. NP
Can randomized algorithms efficiently decide all NP problems?
4 BPP vs. NP
Can randomized algorithms efficiently decide all NP problems?
P Unlikely: NP ⊆ BPP ⇒ PH = Σ2
4 BPP vs. NP
Can randomized algorithms efficiently decide all NP problems?
P Unlikely: NP ⊆ BPP ⇒ PH = Σ2
Will show BPP ⊆ P/poly
4 BPP vs. NP
Can randomized algorithms efficiently decide all NP problems?
P Unlikely: NP ⊆ BPP ⇒ PH = Σ2
Will show BPP ⊆ P/poly
Then NP ⊆ BPP ⇒ NP ⊆ P/poly
4 BPP vs. NP
Can randomized algorithms efficiently decide all NP problems?
P Unlikely: NP ⊆ BPP ⇒ PH = Σ2
Will show BPP ⊆ P/poly
Then NP ⊆ BPP ⇒ NP ⊆ P/poly
P ⇒ PH = Σ2
4 BPP ⊆ P/poly
5 BPP ⊆ P/poly
If error probability is sufficiently small, will show there should be at least one random tape which works for all 2n inputs of length n
5 BPP ⊆ P/poly
If error probability is sufficiently small, will show r x there should be at least one ჲ ჳ ჳ ჲ ჲ ჲ random tape which works for ჲ ჲ ჲ ჲ ჲ ჳ all 2n inputs of length n ჲ ჲ ჲ ჲ ჲ ჲ ჲ ჲ ჳ ჲ ჲ ჲ ჲ ჲ ჲ ჲ ჳ ჲ ჳ ჳ ჲ ჲ ჲ ჲ ჳ ჲ ჲ ჲ ჲ ჲ ჲ ჲ ჲ ჳ ჲ ჲ ჲ ჲ ჲ ჲ ჳ ჲ ჲ ჲ ჲ ჳ ჲ ჲ ჲ ჲ ჲ ჲ ჲ ჲ ჲ ჲ ჲ ჲ ჲ ჳ ჲ ჲ ჲ ჲ ჲ ჲ 5 BPP ⊆ P/poly
If error probability is sufficiently small, will show r x there should be at least one ჲ ჳ ჳ ჲ ჲ ჲ random tape which works for ჲ ჲ ჲ ჲ ჲ ჳ all 2n inputs of length n ჲ ჲ ჲ ჲ ჲ ჲ ჲ ჲ ჳ ჲ ჲ ჲ Then, can give that ჲ ჲ ჲ ჲ ჳ ჲ random tape as advice ჳ ჳ ჲ ჲ ჲ ჲ ჳ ჲ ჲ ჲ ჲ ჲ ჲ ჲ ჲ ჳ ჲ ჲ ჲ ჲ ჲ ჲ ჳ ჲ ჲ ჲ ჲ ჳ ჲ ჲ ჲ ჲ ჲ ჲ ჲ ჲ ჲ ჲ ჲ ჲ ჲ ჳ ჲ ჲ ჲ ჲ ჲ ჲ 5 BPP ⊆ P/poly
If error probability is sufficiently small, will show r x there should be at least one ჲ ჳ ჳ ჲ ჲ ჲ random tape which works for ჲ ჲ ჲ ჲ ჲ ჳ all 2n inputs of length n ჲ ჲ ჲ ჲ ჲ ჲ ჲ ჲ ჳ ჲ ჲ ჲ Then, can give that ჲ ჲ ჲ ჲ ჳ ჲ random tape as advice ჳ ჳ ჲ ჲ ჲ ჲ ჳ ჲ ჲ ჲ ჲ ჲ One such random tape if ჲ ჲ ჲ ჳ ჲ ჲ average (over x) error ჲ ჲ ჲ ჲ ჳ ჲ -n probability is less than 2 ჲ ჲ ჲ ჳ ჲ ჲ ჲ ჲ ჲ ჲ ჲ ჲ ჲ ჲ ჲ ჲ ჲ ჳ ჲ ჲ ჲ ჲ ჲ ჲ 5 BPP ⊆ P/poly
If error probability is sufficiently small, will show r x there should be at least one ჲ ჳ ჳ ჲ ჲ ჲ random tape which works for ჲ ჲ ჲ ჲ ჲ ჳ all 2n inputs of length n ჲ ჲ ჲ ჲ ჲ ჲ ჲ ჲ ჳ ჲ ჲ ჲ Then, can give that ჲ ჲ ჲ ჲ ჳ ჲ random tape as advice ჳ ჳ ჲ ჲ ჲ ჲ ჳ ჲ ჲ ჲ ჲ ჲ One such random tape if ჲ ჲ ჲ ჳ ჲ ჲ average (over x) error ჲ ჲ ჲ ჲ ჳ ჲ -n probability is less than 2 ჲ ჲ ჲ ჳ ჲ ჲ ჲ ჲ ჲ ჲ ჲ ჲ BPP: can make worst ჲ ჲ ჲ ჲ ჲ ჳ error probability < 2-n ჲ ჲ ჲ ჲ ჲ ჲ 5 BPP vs. PH
6 BPP vs. PH
P BPP ⊆ Σ2
6 BPP vs. PH
P BPP ⊆ Σ2
P P So BPP ⊆ Σ2 ∩ Π2
6 P BPP ⊆ Σ2
7 P BPP ⊆ Σ2
x∈L: “for almost all” r, M(x,r)=yes
7 P BPP ⊆ Σ2
x∈L: “for almost all” r, M(x,r)=yes x∉L: M(x,r)=yes for very few r
7 P BPP ⊆ Σ2
x∈L: “for almost all” r, M(x,r)=yes x∉L: M(x,r)=yes for very few r
L = { x| for almost all r, M(x,r)=yes }
7 P BPP ⊆ Σ2
x∈L: “for almost all” r, M(x,r)=yes x∉L: M(x,r)=yes for very few r
L = { x| for almost all r, M(x,r)=yes }
If it were “for all”, in coNP
7 P BPP ⊆ Σ2
x∈L: “for almost all” r, M(x,r)=yes x∉L: M(x,r)=yes for very few r
L = { x| for almost all r, M(x,r)=yes }
If it were “for all”, in coNP
L = { x| ∃a small “neighborhood”, ∀z, for some r “near” z, M(x,r)=yes }
7 P BPP ⊆ Σ2
x∈L: “for almost all” r, M(x,r)=yes x∉L: M(x,r)=yes for very few r
L = { x| for almost all r, M(x,r)=yes }
If it were “for all”, in coNP
L = { x| ∃a small “neighborhood”, ∀z, for some r “near” z, M(x,r)=yes }
Note: Neighborhood of z is small (polynomially large), so can go through all of them in polynomial time
7 P BPP ⊆ Σ2
Space of random tapes = {0,1}m
Yes x = {r| M(x,r)=yes }
8 P BPP ⊆ Σ2 -n m x∈L: |Yesx|>(1-2 )2
Space of random tapes = {0,1}m
Yes x = {r| M(x,r)=yes }
8 P BPP ⊆ Σ2 -n m -n m x∈L: |Yesx|>(1-2 )2 x∉L: |Yesx|<2 2
Space of random tapes = {0,1}m
Yes x = {r| M(x,r)=yes }
8 P BPP ⊆ Σ2 -n m -n m x∈L: |Yesx|>(1-2 )2 x∉L: |Yesx|<2 2
Space of random tapes = {0,1}m
Yes x = {r| M(x,r)=yes } x∈L: Will show that there exist a small set of shifts of
Yes x that cover all z
8 P BPP ⊆ Σ2 -n m -n m x∈L: |Yesx|>(1-2 )2 x∉L: |Yesx|<2 2
Space of random tapes = {0,1}m
Yes x = {r| M(x,r)=yes } x∈L: Will show that there exist a small set of shifts of
Yes x that cover all z
If z is a shift of r ∈ Yesx, r is in the neighborhood of z
8 P BPP ⊆ Σ2 -n m -n m x∈L: |Yesx|>(1-2 )2 x∉L: |Yesx|<2 2
Space of random tapes = {0,1}m
Yes x = {r| M(x,r)=yes } x∈L: Will show that there exist a small set of shifts of
Yes x that cover all z
If z is a shift of r ∈ Yesx, r is in the neighborhood of z
x∉L: Yesx very small, so its few shifts cover only a small region
8 P BPP ⊆ Σ2
9 P BPP ⊆ Σ2
“A small set of shifts”: P = {u1,u2,...,uk}
9 P BPP ⊆ Σ2
“A small set of shifts”: P = {u1,u2,...,uk}
P(r)={ r⊕u1,r⊕u2,...,r⊕uk} where r, ui are m-bit strings, and k is “small” (poly(n))
9 P BPP ⊆ Σ2
“A small set of shifts”: P = {u1,u2,...,uk}
P(r)={ r⊕u1,r⊕u2,...,r⊕uk} where r, ui are m-bit strings, and k is “small” (poly(n))
m For each x∈L, does there exist a P s.t. P(Yesx) := ∪r∈Yes(x) P(r) = {0,1} ?
9 P BPP ⊆ Σ2
“A small set of shifts”: P = {u1,u2,...,uk}
P(r)={ r⊕u1,r⊕u2,...,r⊕uk} where r, ui are m-bit strings, and k is “small” (poly(n))
m For each x∈L, does there exist a P s.t. P(Yesx) := ∪r∈Yes(x) P(r) = {0,1} ?
m Yes! For all large S (like Yesx) can indeed find a P s.t. P(S) = {0,1}
9 P BPP ⊆ Σ2
“A small set of shifts”: P = {u1,u2,...,uk}
P(r)={ r⊕u1,r⊕u2,...,r⊕uk} where r, ui are m-bit strings, and k is “small” (poly(n))
m For each x∈L, does there exist a P s.t. P(Yesx) := ∪r∈Yes(x) P(r) = {0,1} ?
m Yes! For all large S (like Yesx) can indeed find a P s.t. P(S) = {0,1}
In fact, most P work (if k big enough)!
9 P BPP ⊆ Σ2
10 P BPP ⊆ Σ2
Probabilistic Method (finding hay in haystack)
10 P BPP ⊆ Σ2
Probabilistic Method (finding hay in haystack)
To prove ∃P with some property
10 P BPP ⊆ Σ2
Probabilistic Method (finding hay in haystack)
To prove ∃P with some property
Define a probability distribution over all candidate P’s and prove that the property holds with positive probability (often even close to one)
10 P BPP ⊆ Σ2
Probabilistic Method (finding hay in haystack)
To prove ∃P with some property
Define a probability distribution over all candidate P’s and prove that the property holds with positive probability (often even close to one)
Distribution s.t. easy to prove positive probability of property holding
10 P BPP ⊆ Σ2
11 P BPP ⊆ Σ2
Probabilistic method to find P = {u1,u2,...,uk}, s.t. for all large S, P(S) = {0,1}m
11 P BPP ⊆ Σ2
Probabilistic method to find P = {u1,u2,...,uk}, s.t. for all large S, P(S) = {0,1}m
Distribution over P’s: randomized experiment to generate P
11 P BPP ⊆ Σ2
Probabilistic method to find P = {u1,u2,...,uk}, s.t. for all large S, P(S) = {0,1}m
Distribution over P’s: randomized experiment to generate P
Pick each ui independently, and uniformly at random from {0,1}m
11 P BPP ⊆ Σ2
Probabilistic method to find P = {u1,u2,...,uk}, s.t. for all large S, P(S) = {0,1}m
Distribution over P’s: randomized experiment to generate P
Pick each ui independently, and uniformly at random from {0,1}m
m Pr(over P)[P(S) ! {0,1} ] = Pr(over P)[∃z z∉P(S)]
11 P BPP ⊆ Σ2
Probabilistic method to find P = {u1,u2,...,uk}, s.t. for all large S, P(S) = {0,1}m
Distribution over P’s: randomized experiment to generate P
Pick each ui independently, and uniformly at random from {0,1}m
m Pr(over P)[P(S) ! {0,1} ] = Pr(over P)[∃z z∉P(S)] " Σz Pr(over P)[z∉P(S)]
11 P BPP ⊆ Σ2
Probabilistic method to find P = {u1,u2,...,uk}, s.t. for all large S, P(S) = {0,1}m
Distribution over P’s: randomized experiment to generate P
Pick each ui independently, and uniformly at random from {0,1}m
m Pr(over P)[P(S) ! {0,1} ] = Pr(over P)[∃z z∉P(S)] " Σz Pr(over P)[z∉P(S)] = Σz Pr(over u1..uk)[∀i z⊕ui ∉ S]
11 P BPP ⊆ Σ2
Probabilistic method to find P = {u1,u2,...,uk}, s.t. for all large S, P(S) = {0,1}m
Distribution over P’s: randomized experiment to generate P
Pick each ui independently, and uniformly at random from {0,1}m
m Pr(over P)[P(S) ! {0,1} ] = Pr(over P)[∃z z∉P(S)] " Σz Pr(over P)[z∉P(S)] = Σz Pr(over u1..uk)[∀i z⊕ui ∉ S]
= Σz Πi Pr (over ui)[z⊕ui ∉ S]
11 P BPP ⊆ Σ2
Probabilistic method to find P = {u1,u2,...,uk}, s.t. for all large S, P(S) = {0,1}m
Distribution over P’s: randomized experiment to generate P
Pick each ui independently, and uniformly at random from {0,1}m
m Pr(over P)[P(S) ! {0,1} ] = Pr(over P)[∃z z∉P(S)] " Σz Pr(over P)[z∉P(S)] = Σz Pr(over u1..uk)[∀i z⊕ui ∉ S]
= Σz Πi Pr (over ui)[z⊕ui ∉ S] = Σz Πi Pr (over ui)[ui ∉ z⊕S]
11 P BPP ⊆ Σ2
Probabilistic method to find P = {u1,u2,...,uk}, s.t. for all large S, P(S) = {0,1}m
Distribution over P’s: randomized experiment to generate P
Pick each ui independently, and uniformly at random from {0,1}m
m Pr(over P)[P(S) ! {0,1} ] = Pr(over P)[∃z z∉P(S)] " Σz Pr(over P)[z∉P(S)] = Σz Pr(over u1..uk)[∀i z⊕ui ∉ S]
= Σz Πi Pr (over ui)[z⊕ui ∉ S] = Σz Πi Pr (over ui)[ui ∉ z⊕S] c m = Σz Πi (|S |/2 )
11 P BPP ⊆ Σ2
Probabilistic method to find P = {u1,u2,...,uk}, s.t. for all large S, P(S) = {0,1}m
Distribution over P’s: randomized experiment to generate P
Pick each ui independently, and uniformly at random from {0,1}m
m Pr(over P)[P(S) ! {0,1} ] = Pr(over P)[∃z z∉P(S)] " Σz Pr(over P)[z∉P(S)] = Σz Pr(over u1..uk)[∀i z⊕ui ∉ S]
= Σz Πi Pr (over ui)[z⊕ui ∉ S] = Σz Πi Pr (over ui)[ui ∉ z⊕S] c m -n = Σz Πi (|S |/2 ) < Σz Πi 2
11 P BPP ⊆ Σ2
Probabilistic method to find P = {u1,u2,...,uk}, s.t. for all large S, P(S) = {0,1}m
Distribution over P’s: randomized experiment to generate P
Pick each ui independently, and uniformly at random from {0,1}m
m Pr(over P)[P(S) ! {0,1} ] = Pr(over P)[∃z z∉P(S)] " Σz Pr(over P)[z∉P(S)] = Σz Pr(over u1..uk)[∀i z⊕ui ∉ S]
= Σz Πi Pr (over ui)[z⊕ui ∉ S] = Σz Πi Pr (over ui)[ui ∉ z⊕S] c m -n m -n k = Σz Πi (|S |/2 ) < Σz Πi 2 = 2 .(2 ) = 1
11 P BPP ⊆ Σ2
Probabilistic method to find P = {u1,u2,...,uk}, s.t. for all large S, P(S) = {0,1}m
Distribution over P’s: randomized experiment to generate P
Pick each ui independently, and uniformly at random from {0,1}m
m Pr(over P)[P(S) ! {0,1} ] = Pr(over P)[∃z z∉P(S)] " Σz Pr(over P)[z∉P(S)] = Σz Pr(over u1..uk)[∀i z⊕ui ∉ S]
= Σz Πi Pr (over ui)[z⊕ui ∉ S] = Σz Πi Pr (over ui)[ui ∉ z⊕S] c m -n m -n k = Σz Πi (|S |/2 ) < Σz Πi 2 = 2 .(2 ) = 1
So (with |S|>(1-2-n)2m and k=m/n), ∃P, P(S) = {0,1}m
11 P BPP ⊆ Σ2 -n m -n m x∈L: |Yesx|>(1-2 )2 x∉L: |Yesx|<2 2
Space of random strings = {0,1}m
Yes x = {r| M(x,r)=yes }
12 P BPP ⊆ Σ2 -n m -n m x∈L: |Yesx|>(1-2 )2 x∉L: |Yesx|<2 2
Space of random strings = {0,1}m
Yes x = {r| M(x,r)=yes }
m For each x∈L, ∃P (of size k=m/n) s.t. P(Yesx)={0,1}
12 P BPP ⊆ Σ2 -n m -n m x∈L: |Yesx|>(1-2 )2 x∉L: |Yesx|<2 2
Space of random strings = {0,1}m
Yes x = {r| M(x,r)=yes }
m For each x∈L, ∃P (of size k=m/n) s.t. P(Yesx)={0,1}
m For each x∉L, P(Yesx) ⊊ {0,1}
12 P BPP ⊆ Σ2 -n m -n m x∈L: |Yesx|>(1-2 )2 x∉L: |Yesx|<2 2
Space of random strings = {0,1}m
Yes x = {r| M(x,r)=yes }
m For each x∈L, ∃P (of size k=m/n) s.t. P(Yesx)={0,1}
m For each x∉L, P(Yesx) ⊊ {0,1}
-n m m | P(Yesx) | " k| Yesx | = (m/n) 2 2 < 2
12 P BPP ⊆ Σ2 -n m -n m x∈L: |Yesx|>(1-2 )2 x∉L: |Yesx|<2 2
Space of random strings = {0,1}m
Yes x = {r| M(x,r)=yes }
m For each x∈L, ∃P (of size k=m/n) s.t. P(Yesx)={0,1}
m For each x∉L, P(Yesx) ⊊ {0,1}
-n m m | P(Yesx) | " k| Yesx | = (m/n) 2 2 < 2
L = { x| ∃P ∀z for some r∈P-1(z) M(x,r)=yes }
12 BPP-Complete Problem?
13 BPP-Complete Problem?
Not known!
13 BPP-Complete Problem?
Not known!
L = { (M,x,1t) | M(x)=yes in time t with probability > 2/3} ?
13 BPP-Complete Problem?
Not known!
L = { (M,x,1t) | M(x)=yes in time t with probability > 2/3} ?
Is indeed BPP-Hard
13 BPP-Complete Problem?
Not known!
L = { (M,x,1t) | M(x)=yes in time t with probability > 2/3} ?
Is indeed BPP-Hard
But in BPP?
13 BPP-Complete Problem?
Not known!
L = { (M,x,1t) | M(x)=yes in time t with probability > 2/3} ?
Is indeed BPP-Hard
But in BPP?
Just run M(x) for t steps and accept if it accepts?
13 BPP-Complete Problem?
Not known!
L = { (M,x,1t) | M(x)=yes in time t with probability > 2/3} ?
Is indeed BPP-Hard
But in BPP?
Just run M(x) for t steps and accept if it accepts?
If (M,x,1t) in L, we will indeed accept with prob. > 2/3
13 BPP-Complete Problem?
Not known!
L = { (M,x,1t) | M(x)=yes in time t with probability > 2/3} ?
Is indeed BPP-Hard
But in BPP?
Just run M(x) for t steps and accept if it accepts?
If (M,x,1t) in L, we will indeed accept with prob. > 2/3
But M may not have a bounded gap. Then, if (M,x,1t) not in L, we may accept with prob. very close to 2/3.
13 BPTIME-Hierarchy Theorem?
14 BPTIME-Hierarchy Theorem?
BPTIME(n) ⊊ BPTIME(n100)?
14 BPTIME-Hierarchy Theorem?
BPTIME(n) ⊊ BPTIME(n100)?
Not known!
14 BPTIME-Hierarchy Theorem?
BPTIME(n) ⊊ BPTIME(n100)?
Not known!
But is true for BPTIME(T)/1
14 To d a y
15 To d a y
Probabilistic computation
15 To d a y
Probabilistic computation
P BPP ⊆ P/poly (so if NP ⊆ BPP, then PH=Σ2 )
15 To d a y
Probabilistic computation
P BPP ⊆ P/poly (so if NP ⊆ BPP, then PH=Σ2 )
P P BPP ⊆ Σ2 ∩ Π2
15 To d a y
Probabilistic computation
P BPP ⊆ P/poly (so if NP ⊆ BPP, then PH=Σ2 )
P P BPP ⊆ Σ2 ∩ Π2
Coming up
15 To d a y
Probabilistic computation
P BPP ⊆ P/poly (so if NP ⊆ BPP, then PH=Σ2 )
P P BPP ⊆ Σ2 ∩ Π2
Coming up
Basic randomized algorithmic techniques
15 To d a y
Probabilistic computation
P BPP ⊆ P/poly (so if NP ⊆ BPP, then PH=Σ2 )
P P BPP ⊆ Σ2 ∩ Π2
Coming up
Basic randomized algorithmic techniques
Saving on randomness
15