Computational Complexity
Computational Complexity
Lecture 0
1 Computation
2 Computation
A paradigm of modern science
2 Computation
A paradigm of modern science
Theory of computation/computational complexity is to computer science what theoretical physics is to electronics
2 Computational Complexity
3 Computational Complexity
Computation:
3 Computational Complexity
Computation:
Problems to be solved
3 Computational Complexity
Computation:
Problems to be solved
Algorithms to solve them
3 Computational Complexity
Computation:
Problems to be solved
Algorithms to solve them
in various models of computation
3 Computational Complexity
Computation:
Problems to be solved
Algorithms to solve them
in various models of computation
Complexity of a problem (in a comp. model)
3 Computational Complexity
Computation:
Problems to be solved
Algorithms to solve them
in various models of computation
Complexity of a problem (in a comp. model)
How much “resource” is sufficient/necessary
3 Computational Complexity
in w.r.t of Models of Complexity Problems computation measures
4 Problems
5 Problems
Input represented as (say) a binary string
5 Problems
Input represented as (say) a binary string
Given input, find a “satisfactory output”
5 Problems
Input represented as (say) a binary string
Given input, find a “satisfactory output”
Function evaluation: only one correct output
5 Problems
Input represented as (say) a binary string
Given input, find a “satisfactory output”
Function evaluation: only one correct output
Approximate evaluation
5 Problems
Input represented as (say) a binary string
Given input, find a “satisfactory output”
Function evaluation: only one correct output
Approximate evaluation
Search problem: find one of many (if any)
5 Problems
Input represented as (say) a binary string
Given input, find a “satisfactory output”
Function evaluation: only one correct output
Approximate evaluation
Search problem: find one of many (if any)
Decision problem: find out if any
5 Problems
Input represented as (say) a binary string
Given input, find a “satisfactory output”
Function evaluation: only one correct output
Approximate evaluation
Search problem: find one of many (if any)
Decision problem: find out if any
A Boolean function evaluation (TRUE/FALSE)
5 Decision Problems
6 Decision Problems
Evaluate a Boolean function (TRUE/FALSE)
6 Decision Problems
Evaluate a Boolean function (TRUE/FALSE)
i.e., Decide if input has some property
6 Decision Problems
Evaluate a Boolean function (TRUE/FALSE)
i.e., Decide if input has some property
Language
6 Decision Problems
Evaluate a Boolean function (TRUE/FALSE)
i.e., Decide if input has some property
Language
Set of inputs with a particular property
6 Decision Problems
Evaluate a Boolean function (TRUE/FALSE)
i.e., Decide if input has some property
Language
Set of inputs with a particular property
e.g. L = {x | x has equal number of 0s and 1s}
6 Decision Problems
Evaluate a Boolean function (TRUE/FALSE)
i.e., Decide if input has some property
Language
Set of inputs with a particular property
e.g. L = {x | x has equal number of 0s and 1s}
Decision problem: given input, decide if it is in L
6 Complexity of Languages
7 Complexity of Languages
Some languages are “simpler” than others
7 Complexity of Languages
Some languages are “simpler” than others
L1 = {x | x starts with 0}
7 Complexity of Languages
Some languages are “simpler” than others
L1 = {x | x starts with 0}
L2 = {x | x has equal number of 0s and 1s}
7 Complexity of Languages
Some languages are “simpler” than others
L1 = {x | x starts with 0}
L2 = {x | x has equal number of 0s and 1s}
Simpler in what way?
7 Complexity of Languages
Some languages are “simpler” than others
L1 = {x | x starts with 0}
L2 = {x | x has equal number of 0s and 1s}
Simpler in what way?
Fewer calculations, less memory, need not read all input, can do in an FSM
7 Complexity of Languages
Some languages are “simpler” than others
L1 = {x | x starts with 0}
L2 = {x | x has equal number of 0s and 1s}
Simpler in what way?
Fewer calculations, less memory, need not read all input, can do in an FSM Flying Spaghetti Monster?
7 Complexity of Languages
8 Complexity of Languages
Relating complexities of problems
8 Complexity of Languages
Relating complexities of problems
Mo = {x | x has more 0s than 1s}
8 Complexity of Languages
Relating complexities of problems
Mo = {x | x has more 0s than 1s}
Eq = {x | x has equal number of 0s and 1s}
8 Complexity of Languages
Relating complexities of problems
Mo = {x | x has more 0s than 1s}
Eq = {x | x has equal number of 0s and 1s}
Eq(x):
8 Complexity of Languages
Relating complexities of problems
Mo = {x | x has more 0s than 1s}
Eq = {x | x has equal number of 0s and 1s}
Eq(x):
if (Mo(x0) == TRUE and Mo(x) == FALSE) then TRUE; else FALSE
8 Complexity of Languages
Relating complexities of problems
Mo = {x | x has more 0s than 1s}
Eq = {x | x has equal number of 0s and 1s}
Eq(x):
if (Mo(x0) == TRUE and Mo(x) == FALSE) then TRUE; else FALSE
Eq is not (much) more complex than Mo. i.e., Mo is at least (almost) as complex as Eq.
8 Complexity of Languages
Relating complexities of problems
Mo = {x | x has more 0s than 1s}
Eq = {x | x has equal number of 0s and 1s}
Eq(x): Eq reduces to Mo
if (Mo(x0) == TRUE and Mo(x) == FALSE) then TRUE; else FALSE
Eq is not (much) more complex than Mo. i.e., Mo is at least (almost) as complex as Eq.
8 Models of Computation
9 Models of Computation
FSM, PDA, TM
9 Models of Computation
FSM, PDA, TM
Variations: Non-deterministic, probabilistic. Other models: quantum computation
9 Models of Computation
FSM, PDA, TM
Variations: Non-deterministic, probabilistic. Other models: quantum computation
Church-Turing thesis: TM is as “powerful” as it gets
9 Models of Computation
FSM, PDA, TM
Variations: Non-deterministic, probabilistic. Other models: quantum computation
Church-Turing thesis: TM is as “powerful” as it gets
Not enough TMs (algorithms/programs) to solve all decision problems!
9 Models of Computation
FSM, PDA, TM
Variations: Non-deterministic, probabilistic. Other models: quantum computation
Church-Turing thesis: TM is as “powerful” as it gets
Not enough TMs (algorithms/programs) to solve all decision problems!
Non-uniform computation: circuit families
9 Complexity Measures
10 Complexity Measures
Number of computational steps, amount of memory, circuit size/depth, ...
10 Complexity Measures
Number of computational steps, amount of memory, circuit size/depth, ...
Exact numbers very much dependent on exact specification of the model (e.g. no. of tapes in TM)
10 Complexity Measures
Number of computational steps, amount of memory, circuit size/depth, ...
Exact numbers very much dependent on exact specification of the model (e.g. no. of tapes in TM)
But “broad trends” robust
10 Complexity Measures
Number of computational steps, amount of memory, circuit size/depth, ...
Exact numbers very much dependent on exact specification of the model (e.g. no. of tapes in TM)
But “broad trends” robust
10 Complexity Measures
Number of computational steps, amount of memory, circuit size/depth, ...
Exact numbers very much dependent on exact specification of the model (e.g. no. of tapes in TM)
But “broad trends” robust
Trends: asymptotic
10 Complexity Measures
Number of computational steps, amount of memory, circuit size/depth, ...
Exact numbers very much dependent on exact specification of the model (e.g. no. of tapes in TM)
But “broad trends” robust
Trends: asymptotic
Broad: Log, Poly, Exp
10 Complexity Measures
Number of computational steps, amount of memory, circuit size/depth, ...
Exact numbers very much dependent on exact specification of the model (e.g. no. of tapes in TM)
But “broad trends” robust
Trends: asymptotic
Broad: Log, Poly, Exp Log Poly Exp
10 Complexity Theory
11 Complexity Theory
Understand complexity of problems (i.e., how much resource used by best algorithm for it)
11 Complexity Theory
Understand complexity of problems (i.e., how much resource used by best algorithm for it)
Relate problems to each other [Reduce]
11 Complexity Theory
Understand complexity of problems (i.e., how much resource used by best algorithm for it)
Relate problems to each other [Reduce]
Relate computational models/complexity measures to each other [Simulate]
11 Complexity Theory
Understand complexity of problems (i.e., how much resource used by best algorithm for it)
Relate problems to each other [Reduce]
Relate computational models/complexity measures to each other [Simulate]
Calculate complexity of problems
11 Complexity Classes
12 Complexity Classes
Collect (decision) problems with similar complexity into classes
12 Complexity Classes
Collect (decision) problems with similar complexity into classes
12 Complexity Classes
Collect (decision) problems with similar complexity into classes P
12 Complexity Classes
Collect (decision) problems with similar complexity into classes NP P
12 Complexity Classes
Collect (decision) problems with similar complexity IP into classes NP P
12 Complexity Classes
Collect (decision) problems with PSPACE similar complexity IP into classes NP P
12 Complexity Classes
Collect (decision) problems with PSPACE similar complexity IP into classes NP P
BPP
12 Complexity Classes
Collect (decision) problems with PSPACE similar complexity IP into classes NP P Relate classes to each other BPP
12 Complexity IP=PSPACE Classes
Collect (decision) problems with PSPACE similar complexity IP into classes NP P Relate classes to each other BPP
12 Complexity IP=PSPACE Classes P vs NP ?
Collect (decision) problems with PSPACE similar complexity IP into classes NP P Relate classes to each other BPP
12 Complexity IP=PSPACE P vs NP ? Classes P vs BPP ?
Collect (decision) problems with PSPACE similar complexity IP into classes NP P Relate classes to each other BPP
12 Complexity IP=PSPACE P vs NP ? Classes P vs BPP ?
Collect (decision) problems with PSPACE similar complexity IP into classes NP P Relate classes to each other BPP Hundreds of classes!
12 Complexity Zoo!
Collect (decision) problems with similar complexity into classes
Relate classes to each other
Hundreds of classes!
13 ALL
AH
RE
R
PR
ELEMENTARY
Complexity NEEE P-Sel
NEEXP EEE
EEXP EESPACE
MIP_{EXP} EXPSPACE
IP_{EXP} PEXP EXPH
SEH NEXP^{NP} NEE
EXP^{NP} AM_{EXP}
NEXP/poly MA_{EXP} BPEE
Zoo! EXP/poly NEXP BPEXP EE +EXP
EXP QRG ESPACE
RG Almost-PSPACE QPSPACE
PSPACE Coh
PL_{infty} CH
MP^{#P} AvgE Collect (decision) P^{PP} EH PP/poly P^{#P[1]}
BP.PP MP
PH SF_4 problems with Sigma_3P AmpMP QMIP QMIP_{le} QMIP_{ne} SQG Delta_3P
BQP/qpoly MIP* QIP MIP RG[1] Sigma_2P BPP^{NP}
BQP/mpoly NE/poly XOR-MIP*[2,1] QMA(2) QIP[2] IP RP^{NP} frIP similar complexity BQP/poly NP/poly QSZK QAM AM[polylog] compIP ZPP^{NP} QS_2P PP SF_3 (NP-cap-coNP)/poly BQP/qlog DQP CZK AM S_2P P^{QMA} A_0PP SF_2
P/poly BQP/mlog N.NISZK SZK QMA Delta_2P BPP_{path} APP Check
BPP//log BQP/log YQP NIQSZK NISZK_h QCMA SBP MA_E P^{NP[log^2]} AWPP C_=P into classes IC[log,poly] BPP/rlog BQP NE NISZK MA WAPP BPE P^{NP[log]} WPP BPP/mlog HeurBPP YPP N.BPP PZK AmpP-BQP RPE BPQP UE BH ModP
NP/log BPP/log AVBPP FH TreeBQP ZPE BH_2 LWPP Mod_5P +P Mod_3P
Nearly-P NP/one P/log BPP E US RP^{PromiseUP} SPP NT*
AvgP NP RQP SUBEXP P^{FewP} NT
YP compNP RBQP ZQP QP Few EP
ZBQP RP EQP betaP QPLIN FewP UAP
Relate classes to QNC ZPP Q beta_2P UP
P-Close RNC HalfP NLINSPACE
P NLIN polyL
NC LIN SC
each other QCFL AL NC^2
+L/poly +SAC^1 L^{DET} AC^1
NL/poly +L SAC^1 PL CSL
L/poly C_=L 1NAuxPDA^p GCSL
NL SPL CFL BPL
LFew RL DCFL Hundreds of FewL LogFew FewUL FOLL
UL R_HL
QNC^1 L classes! TC^0/poly NC^1 PBP QACC^0 TC^0 REG
PT_1 QAC^0 MAC^0 ACC^0 (k>=5)-PBP
PL_1 AC^0/poly AC^0[2] MAJORITY 4-PBP
SPARSE QNC_f^0 AC^0 3-PBP
TALLY QNC^0 SAC^0 +SAC^0 2-PBP
NC^0 PARITY
NONE
13 Complexity QMA(2) QIP[2] IP QSZK QAM AM[polylog] comp
Zoo! DQP CZK AM
Collect (decision) N.NISZK SZK QMA problems with NIQSZK NISZK_h QCMA SBP MA similar complexity into classes NE NISZK MA WAPP
Relate classes to N.BPP PZK AmpP-BQP each other FH TreeBQP Hundreds of BPP classes!
NP RQP
YP compNP RBQP ZQP
13 Complexity in various settings
14 Complexity in various settings
With various models of computation: decision trees, interactive settings, probabilistic computation
14 Complexity in various settings
With various models of computation: decision trees, interactive settings, probabilistic computation
Various measures: depth, width, amount of communication, number of rounds, amount of randomness, amount of non-uniformity, ...
14 Complexity in various settings
With various models of computation: decision trees, interactive settings, probabilistic computation
Various measures: depth, width, amount of communication, number of rounds, amount of randomness, amount of non-uniformity, ...
Various connections: time vs. space, randomness vs. hardness
14 Cryptography
15 Cryptography Need to prove that a scheme is secure (according to some definition)
15 Cryptography Need to prove that a scheme is secure (according to some definition)
i.e., breaking security has high complexity
15 Cryptography Need to prove that a scheme is secure (according to some definition)
i.e., breaking security has high complexity
Reductions: if you could break my scheme’s security efficiently, I can solve a hard problem almost as efficiently
15 Cryptography Need to prove that a scheme is secure (according to some definition)
i.e., breaking security has high complexity
Reductions: if you could break my scheme’s security efficiently, I can solve a hard problem almost as efficiently
Hard problems: almost all instances hard
15 Cryptography Need to prove that a scheme is secure (according to some definition)
i.e., breaking security has high complexity
Reductions: if you could break my scheme’s security efficiently, I can solve a hard problem almost as efficiently
Hard problems: almost all instances hard
For most keys scheme should be secure
15 Cryptography Need to prove that a scheme is secure (according to some definition)
i.e., breaking security has high complexity
Reductions: if you could break my scheme’s security efficiently, I can solve a hard problem almost as efficiently
Hard problems: almost all instances hard
For most keys scheme should be secure
Contrast with Information Theory: limited cryptography if no computational constraints on adversary
15 Computational Complexity in Economics/Games
16 Computational Complexity in Economics/Games
Traditionally, no computational constraints for strategizing players
16 Computational Complexity in Economics/Games
Traditionally, no computational constraints for strategizing players
But many problems in game-theory are computationally hard
16 Computational Complexity in Economics/Games
Traditionally, no computational constraints for strategizing players
But many problems in game-theory are computationally hard
Rational players may not be able to play optimally
16 Computational Complexity in Economics/Games
Traditionally, no computational constraints for strategizing players
But many problems in game-theory are computationally hard
Rational players may not be able to play optimally
Suggests the need to redesign markets/financial instruments to be computationally tractable
16 Computational Complexity in Economics/Games
Traditionally, no computational constraints for strategizing players
But many problems in game-theory are computationally hard
Rational players may not be able to play optimally
Suggests the need to redesign markets/financial instruments to be computationally tractable
Recent results...
16 All that and much more..
17 All that and much more..
Welcome to CS 579!
17 All that and much more..
Welcome to CS 579! Tex t b o o k : www.cs.princeton.edu/theory/complexity/
17 All that and much more..
Welcome to CS 579! Tex t b o o k : www.cs.princeton.edu/theory/complexity/ About 5 assignments, a class test, and a reading project
17 All that and much more..
Welcome to CS 579! Tex t b o o k : www.cs.princeton.edu/theory/complexity/ About 5 assignments, a class test, and a reading project Office hours: Wednesdays 2:00 to 3:00 (tentative)
17