The Complexity of Gradient Descent: CLS = PPAD ∩ PLS ALEXANDROS HOLLENDER JOINT WORK WITH JOHN FEARNLEY, PAUL GOLDBERG AND RAHUL SAVANI Some interesting computational problems Some interesting computational problems NASH: Find a mixed Nash equilibrium of a game. Some interesting computational problems NASH: Find a mixed Nash equilibrium of a game. FACTORING: Find a prime factor of a number 푛 ≥ 2. Some interesting computational problems NASH: Find a mixed Nash equilibrium of a game. FACTORING: Find a prime factor of a number 푛 ≥ 2. BROUWER: Find a fixpoint of a continuous function 푓: 0,1 3→ [0,1]3. Some interesting computational problems NASH: Find a mixed Nash equilibrium of a game. FACTORING: Find a prime factor of a number 푛 ≥ 2. BROUWER: Find a fixpoint of a continuous function 푓: 0,1 3→ [0,1]3. CONTRACTION: Find the unique fixpoint of a contraction 푓: 0,1 푛→ [0,1]푛. Some interesting computational problems NASH: Find a mixed Nash equilibrium of a game. FACTORING: Find a prime factor of a number 푛 ≥ 2. BROUWER: Find a fixpoint of a continuous function 푓: 0,1 3→ [0,1]3. CONTRACTION: Find the unique fixpoint of a contraction 푓: 0,1 푛→ [0,1]푛. PURE-CONGESTION: Find a pure Nash equilibrium of a congestion game. Some interesting computational problems What do these problems have in common? NASH: Find a mixed Nash equilibrium of a game. FACTORING: Find a prime factor of a number 푛 ≥ 2. BROUWER: Find a fixpoint of a continuous function 푓: 0,1 3→ [0,1]3. CONTRACTION: Find the unique fixpoint of a contraction 푓: 0,1 푛→ [0,1]푛. PURE-CONGESTION: Find a pure Nash equilibrium of a congestion game. Some interesting computational problems What do these problems have in common? NASH: Find a mixed Nash equilibrium of a game. FACTORING: They are NP Total Search (TFNP) problems! Find a prime factor of a number 푛 ≥ 2. • Total: there is always a solution BROUWER: • NP: it is easy to verify solutions Find a fixpoint of a continuous function 푓: 0,1 3→ [0,1]3. CONTRACTION: Find the unique fixpoint of a contraction 푓: 0,1 푛→ [0,1]푛. PURE-CONGESTION: Find a pure Nash equilibrium of a congestion game. Some interesting computational problems What do these problems have in common? NASH: Find a mixed Nash equilibrium of a game. FACTORING: They are NP Total Search (TFNP) problems! Find a prime factor of a number 푛 ≥ 2. • Total: there is always a solution BROUWER: • NP: it is easy to verify solutions Find a fixpoint of a continuous function 푓: 0,1 3→ [0,1]3. CONTRACTION: Find the unique fixpoint of a contraction 푓: 0,1 푛→ [0,1]푛. Can a TFNP problem be NP-hard? PURE-CONGESTION: Find a pure Nash equilibrium of a congestion game. Some interesting computational problems What do these problems have in common? NASH: Find a mixed Nash equilibrium of a game. FACTORING: They are NP Total Search (TFNP) problems! Find a prime factor of a number 푛 ≥ 2. • Total: there is always a solution BROUWER: • NP: it is easy to verify solutions Find a fixpoint of a continuous function 푓: 0,1 3→ [0,1]3. CONTRACTION: Find the unique fixpoint of a contraction 푓: 0,1 푛→ [0,1]푛. Can a TFNP problem be NP-hard? PURE-CONGESTION: Find a pure Nash equilibrium of a congestion game. Not unless co-NP = NP… The class TFNP [Megiddo-Papadimitriou, 1991] Total NP search problems: • “search” : looking for a solution, not just YES or NO • “NP”: any solution can be checked efficiently • “total”: there always exists at least one solution The class TFNP [Megiddo-Papadimitriou, 1991] Total NP search problems: • “search” : looking for a solution, not just YES or NO • “NP”: any solution can be checked efficiently • “total”: there always exists at least one solution TFNP lies between P and NP (search versions) The class TFNP [Megiddo-Papadimitriou, 1991] Total NP search problems: • “search” : looking for a solution, not just YES or NO • “NP”: any solution can be checked efficiently • “total”: there always exists at least one solution How do we show that a TFNP-problem is hard: The class TFNP [Megiddo-Papadimitriou, 1991] Total NP search problems: • “search” : looking for a solution, not just YES or NO • “NP”: any solution can be checked efficiently • “total”: there always exists at least one solution How do we show that a TFNP-problem is hard: ▪ No TFNP-problem can be NP-hard, unless NP = coNP… The class TFNP [Megiddo-Papadimitriou, 1991] Total NP search problems: • “search” : looking for a solution, not just YES or NO • “NP”: any solution can be checked efficiently • “total”: there always exists at least one solution How do we show that a TFNP-problem is hard: ▪ No TFNP-problem can be NP-hard, unless NP = coNP… 3-SAT ≤ NASH ⇒ certificate for unsatisfiable 3-SAT formulas The class TFNP [Megiddo-Papadimitriou, 1991] Total NP search problems: • “search” : looking for a solution, not just YES or NO • “NP”: any solution can be checked efficiently • “total”: there always exists at least one solution How do we show that a TFNP-problem is hard: ▪ No TFNP-problem can be NP-hard, unless NP = coNP… The class TFNP [Megiddo-Papadimitriou, 1991] Total NP search problems: • “search” : looking for a solution, not just YES or NO • “NP”: any solution can be checked efficiently • “total”: there always exists at least one solution How do we show that a TFNP-problem is hard: ▪ No TFNP-problem can be NP-hard, unless NP = coNP… ▪ Believed that no TFNP-complete problems exists… The TFNP landscape TFNP P The TFNP landscape Pigeonhole Principle TFNP PPP P The TFNP landscape Pigeonhole Principle Parity Argument TFNP Borsuk-Ulam PPA PPP P The TFNP landscape Pigeonhole Principle Parity Argument TFNP Borsuk-Ulam PPA PPP PLS Local Search Argument P PURE-CONGESTION LOCAL-MAX-CUT The TFNP landscape Pigeonhole Principle Parity Argument TFNP Borsuk-Ulam PPA PPP PPAD PLS Local Search Argument Directed Graph Argument P PURE-CONGESTION NASH LOCAL-MAX-CUT BROUWER The TFNP landscape Pigeonhole Principle Parity Argument TFNP Borsuk-Ulam PPA FACTORING PPP PPAD PLS Local Search Argument Directed Graph Argument P PURE-CONGESTION NASH LOCAL-MAX-CUT BROUWER TFNP subclasses What reasons are there to believe that PPAD ≠ P, PLS ≠ P, etc? TFNP subclasses What reasons are there to believe that PPAD ≠ P, PLS ≠ P, etc? • many seemingly hard problems lie in PPAD, PLS etc… TFNP subclasses What reasons are there to believe that PPAD ≠ P, PLS ≠ P, etc? • many seemingly hard problems lie in PPAD, PLS etc… • oracle separations between the classes (in particular PPAD ≠ PLS) TFNP subclasses What reasons are there to believe that PPAD ≠ P, PLS ≠ P, etc? • many seemingly hard problems lie in PPAD, PLS etc… • oracle separations between the classes (in particular PPAD ≠ PLS) • hard under cryptographic assumptions TFNP PPAD PLS NASH LOCAL-MAX-CUT BROUWER PURE-CONGESTION P TFNP PPAD PLS NASH LOCAL-MAX-CUT BROUWER PURE-CONGESTION PPAD ∩ PLS P TFNP PPAD PLS NASH LOCAL-MAX-CUT BROUWER PURE-CONGESTION PPAD ∩ PLS CONTRACTION P TFNP PPAD PLS NASH LOCAL-MAX-CUT BROUWER PURE-CONGESTION PPAD ∩ PLS CONTRACTION MIXED-CONGESTION P TFNP PPAD PLS NASH LOCAL-MAX-CUT BROUWER PURE-CONGESTION PPAD ∩ PLS CONTRACTION MIXED-CONGESTION P-LCP P TFNP PPAD PLS NASH LOCAL-MAX-CUT BROUWER PURE-CONGESTION PPAD ∩ PLS CONTRACTION MIXED-CONGESTION P-LCP TARSKI P TFNP PPAD PLS NASH LOCAL-MAX-CUT BROUWER PURE-CONGESTION PPAD ∩ PLS CONTRACTION MIXED-CONGESTION P-LCP TARSKI SSGs P PPAD ∩ PLS seems unnatural… PPAD ∩ PLS seems unnatural… Problem 퐴 : PPAD-complete Problem 퐵 : PLS-complete PPAD ∩ PLS seems unnatural… Problem 퐴 : PPAD-complete Problem 퐵 : PLS-complete EITHER-SOLUTION(푨,푩): Input: instance 퐼퐴 of 퐴, instance 퐼퐵 of 퐵 Goal: find a solution of 퐼퐴, or a solution of 퐼퐵 PPAD ∩ PLS seems unnatural… Problem 퐴 : PPAD-complete Problem 퐵 : PLS-complete EITHER-SOLUTION(푨,푩): Input: instance 퐼퐴 of 퐴, instance 퐼퐵 of 퐵 Goal: find a solution of 퐼퐴, or a solution of 퐼퐵 → EITHER-SOLUTION(푨,푩) is (PPAD ∩ PLS)-complete! PPAD ∩ PLS seems unnatural… BROUWER: Input: a continuous function 푓: 0,1 푛→ [0,1]푛 Goal: find a fixpoint 푥 푓 푥 = 푥 PPAD ∩ PLS seems unnatural… BROUWER: Input: a continuous function 푓: 0,1 푛→ [0,1]푛, precision 휀 > 0 Goal: find an approximate fixpoint 푥 푓 푥 − 푥 ≤ 휀 PPAD ∩ PLS seems unnatural… BROUWER: Input: a continuous function 푓: 0,1 푛→ [0,1]푛, precision 휀 > 0 Goal: find an approximate fixpoint 푥 푓 푥 − 푥 ≤ 휀 REAL-LOCAL-OPT: Input: - a continuous function 푝: [0,1]푛→ 0,1 - a (possibly non-continuous) function 푔: 0,1 푛→ [0,1]푛 PPAD ∩ PLS seems unnatural… BROUWER: Input: a continuous function 푓: 0,1 푛→ [0,1]푛, precision 휀 > 0 Goal: find an approximate fixpoint 푥 푓 푥 − 푥 ≤ 휀 REAL-LOCAL-OPT: Input: - a continuous function 푝: [0,1]푛→ 0,1 - a (possibly non-continuous) function 푔: 0,1 푛→ [0,1]푛 Goal: find a local minimum of 푝 with respect to 푔 푝 푔 푥 ≥ 푝 푥 PPAD ∩ PLS seems unnatural… BROUWER: Input: a continuous function 푓: 0,1 푛→ [0,1]푛, precision 휀 > 0 Goal: find an approximate fixpoint 푥 푓 푥 − 푥 ≤ 휀 REAL-LOCAL-OPT: Input: - a continuous function 푝: [0,1]푛→ 0,1 - a (possibly non-continuous) function 푔: 0,1 푛→ [0,1]푛 Goal: find a local minimum of 푝 with respect to 푔 푝 푔 푥 ≥ 푝 푥 − 휀 PPAD ∩ PLS seems unnatural… BROUWER: Input: a continuous function 푓: 0,1 푛→ [0,1]푛, precision 휀 > 0 Goal: find an approximate fixpoint 푥 푓 푥 − 푥 ≤ 휀 REAL-LOCAL-OPT: Input: - a continuous function 푝: [0,1]푛→ 0,1 - a (possibly non-continuous) function 푔: 0,1 푛→ [0,1]푛 Goal: find a local minimum of 푝 with respect to 푔 푝 푔 푥 ≥ 푝 푥 − 휀 → EITHER-SOLUTION(BROUWER,LOCAL-OPT) is (PPAD ∩ PLS)-complete.