fifty shades of adaptivity (in property testing) An Adaptivity Hierarchy Theorem for Property Testing Clément Canonne (Columbia University) July 9, 2017 Joint work with Tom Gur (Weizmann Institute UC Berkeley) “property testing?” approximate, randomized decision algorithms that make queries ∙ Big object: too big ∙ Expensive access: pricey data ∙ “Model selection”: many options ∙ Good Enough: a priori knowledge Need to infer information – one bit – from the data: quickly, or with very few lookups. why? Sublinear, 2 randomized decision algorithms that make queries ∙ Big object: too big ∙ Expensive access: pricey data ∙ “Model selection”: many options ∙ Good Enough: a priori knowledge Need to infer information – one bit – from the data: quickly, or with very few lookups. why? Sublinear, approximate, 2 decision algorithms that make queries ∙ Big object: too big ∙ Expensive access: pricey data ∙ “Model selection”: many options ∙ Good Enough: a priori knowledge Need to infer information – one bit – from the data: quickly, or with very few lookups. why? Sublinear, approximate, randomized 2 ∙ Big object: too big ∙ Expensive access: pricey data ∙ “Model selection”: many options ∙ Good Enough: a priori knowledge Need to infer information – one bit – from the data: quickly, or with very few lookups. why? Sublinear, approximate, randomized decision algorithms that make queries 2 ∙ Expensive access: pricey data ∙ “Model selection”: many options ∙ Good Enough: a priori knowledge Need to infer information – one bit – from the data: quickly, or with very few lookups. why? Sublinear, approximate, randomized decision algorithms that make queries ∙ Big object: too big 2 ∙ “Model selection”: many options ∙ Good Enough: a priori knowledge Need to infer information – one bit – from the data: quickly, or with very few lookups. why? Sublinear, approximate, randomized decision algorithms that make queries ∙ Big object: too big ∙ Expensive access: pricey data 2 Need to infer information – one bit – from the data: quickly, or with very few lookups. why? Sublinear, approximate, randomized decision algorithms that make queries ∙ Big object: too big ∙ Expensive access: pricey data ∙ “Model selection”: many options ∙ Good Enough: a priori knowledge 2 why? Sublinear, approximate, randomized decision algorithms that make queries ∙ Big object: too big ∙ Expensive access: pricey data ∙ “Model selection”: many options ∙ Good Enough: a priori knowledge Need to infer information – one bit – from the data: quickly, or with very few lookups. 2 3 Must decide: x 2 P, or d(x; P) > "? (and be correct on any x with probability at least 2=3) how? Known space (say, f0; 1gN) Property P ⊆ f0; 1gN) Query (oracle) access to unknown x 2 f0; 1gN Proximity parameter " 2 (0; 1] 4 , or d(x; P) > "? (and be correct on any x with probability at least 2=3) how? Known space (say, f0; 1gN) Property P ⊆ f0; 1gN) Query (oracle) access to unknown x 2 f0; 1gN Proximity parameter " 2 (0; 1] Must decide: x 2 P 4 (and be correct on any x with probability at least 2=3) how? Known space (say, f0; 1gN) Property P ⊆ f0; 1gN) Query (oracle) access to unknown x 2 f0; 1gN Proximity parameter " 2 (0; 1] Must decide: x 2 P, or d(x; P) > "? 4 how? Known space (say, f0; 1gN) Property P ⊆ f0; 1gN) Query (oracle) access to unknown x 2 f0; 1gN Proximity parameter " 2 (0; 1] Must decide: x 2 P, or d(x; P) > "? (and be correct on any x with probability at least 2=3) 4 in an (egg)shell. how? Property Testing: 5 in an (egg)shell. how? Property Testing: 5 how? Property Testing: in an (egg)shell. 5 query-based vs. sample-based, uniform vs. distribution-free, adaptive vs. non-adaptive it’s complicated Many flavors… … one-sided vs. two-sided, 6 uniform vs. distribution-free, adaptive vs. non-adaptive it’s complicated Many flavors… … one-sided vs. two-sided, query-based vs. sample-based, 6 adaptive vs. non-adaptive it’s complicated Many flavors… … one-sided vs. two-sided, query-based vs. sample-based, uniform vs. distribution-free, 6 it’s complicated Many flavors… … one-sided vs. two-sided, query-based vs. sample-based, uniform vs. distribution-free, adaptive vs. non-adaptive 6 adaptivity our focus: adaptivity Non-adaptive algorithm Makes all its queries upfront: Q ⊆ [N] = Q("; r) = fi1;:::; iqg Adaptive algorithm Each query can depend arbitrarily on the previous answers: 8 some observations Dense graph model At most a quadratic gap between adaptive and non-adaptive algorithms: q vs. 2q2 [AFKS00, GT03],[GR11] Boolean functions At most an exponential gap between adaptive and non-adaptive algorithms: q vs. 2q Bounded-degree graph model p Everything is possible: O(1) vs. Ω( n).[RS06] 9 But Many parallel queries can beat few sequential ones. Understanding the benefits and tradeoffs of adaptivity is crucial. why should we care? Of course Fewer queries is always better. 10 Understanding the benefits and tradeoffs of adaptivity is crucial. why should we care? Of course Fewer queries is always better. But Many parallel queries can beat few sequential ones. 10 why should we care? Of course Fewer queries is always better. But Many parallel queries can beat few sequential ones. Understanding the benefits and tradeoffs of adaptivity is crucial. 10 (and what does “amount of adaptivity” even mean?) this work A closer look Does the power of testing algorithms smoothly grow with the “amount of adaptivity?” 11 this work A closer look Does the power of testing algorithms smoothly grow with the “amount of adaptivity?” (and what does “amount of adaptivity” even mean?) 11 (i) Query Generation: The algorithm proceeds in k + 1 rounds, such that at (`);1 (`);jQ`j round ` ≥ 0, it produces a set of queries Q` := fx ;:::; x g ⊆ Ω, based on its own internal randomness and the answers to the previous sets of queries Q0;:::; Q`−1, and receives (`);1 (`);jQ`j f(Q`) = ff(x );:::; f(x )g; (ii) Completeness: If f 2 P, then it outputs accept with probability 2=3; (iii) Soundness: If dist(f; P) > ", then it outputs reject with probability 2=3. The queryP complexity q of the tester is the total number of queries made to k j j f, i.e., q = `=0 Q` . coming up with a definition Definition (Round-Adaptive Testing Algorithms) Let Ω be a domain of size n, and k; q ≤ n. A randomized algorithm is said to be a (k; q)-round-adaptive tester for P ⊆ 2Ω, if, on input " 2 (0; 1] and granted query access to f:Ω ! f0; 1g: 12 (ii) Completeness: If f 2 P, then it outputs accept with probability 2=3; (iii) Soundness: If dist(f; P) > ", then it outputs reject with probability 2=3. The queryP complexity q of the tester is the total number of queries made to k j j f, i.e., q = `=0 Q` . coming up with a definition Definition (Round-Adaptive Testing Algorithms) Let Ω be a domain of size n, and k; q ≤ n. A randomized algorithm is said to be a (k; q)-round-adaptive tester for P ⊆ 2Ω, if, on input " 2 (0; 1] and granted query access to f:Ω ! f0; 1g: (i) Query Generation: The algorithm proceeds in k + 1 rounds, such that at (`);1 (`);jQ`j round ` ≥ 0, it produces a set of queries Q` := fx ;:::; x g ⊆ Ω, based on its own internal randomness and the answers to the previous sets of queries Q0;:::; Q`−1, and receives (`);1 (`);jQ`j f(Q`) = ff(x );:::; f(x )g; 12 The queryP complexity q of the tester is the total number of queries made to k j j f, i.e., q = `=0 Q` . coming up with a definition Definition (Round-Adaptive Testing Algorithms) Let Ω be a domain of size n, and k; q ≤ n. A randomized algorithm is said to be a (k; q)-round-adaptive tester for P ⊆ 2Ω, if, on input " 2 (0; 1] and granted query access to f:Ω ! f0; 1g: (i) Query Generation: The algorithm proceeds in k + 1 rounds, such that at (`);1 (`);jQ`j round ` ≥ 0, it produces a set of queries Q` := fx ;:::; x g ⊆ Ω, based on its own internal randomness and the answers to the previous sets of queries Q0;:::; Q`−1, and receives (`);1 (`);jQ`j f(Q`) = ff(x );:::; f(x )g; (ii) Completeness: If f 2 P, then it outputs accept with probability 2=3; (iii) Soundness: If dist(f; P) > ", then it outputs reject with probability 2=3. 12 coming up with a definition Definition (Round-Adaptive Testing Algorithms) Let Ω be a domain of size n, and k; q ≤ n. A randomized algorithm is said to be a (k; q)-round-adaptive tester for P ⊆ 2Ω, if, on input " 2 (0; 1] and granted query access to f:Ω ! f0; 1g: (i) Query Generation: The algorithm proceeds in k + 1 rounds, such that at (`);1 (`);jQ`j round ` ≥ 0, it produces a set of queries Q` := fx ;:::; x g ⊆ Ω, based on its own internal randomness and the answers to the previous sets of queries Q0;:::; Q`−1, and receives (`);1 (`);jQ`j f(Q`) = ff(x );:::; f(x )g; (ii) Completeness: If f 2 P, then it outputs accept with probability 2=3; (iii) Soundness: If dist(f; P) > ", then it outputs reject with probability 2=3. The queryP complexity q of the tester is the total number of queries made to k j j f, i.e., q = `=0 Q` . 12 that was a mouthful, but… (i can’t draw) 13 ∙ Probability amplification ∙ Similar in spirit to… some remarks ∙ Other possible choices: e.g., tail-adaptive 14 ∙ Similar in spirit to… some remarks ∙ Other possible choices: e.g., tail-adaptive ∙ Probability amplification 14 some remarks ∙ Other possible choices: e.g., tail-adaptive ∙ Probability amplification ∙ Similar in spirit to… 14 we have a definition… … now, what do we do with it? Does the power of testing algorithms smoothly grow with the “amount of adaptivity” number of rounds of adaptivity? 15 our results Theorem (Hierarchy Theorem I) 0:33 For every n 2 N and 0 ≤ k ≤ n there is a property Pn;k of strings over Fn such that: (i) there exists a k-round-adaptive tester for Pn;k with query complexity O~(k), yet ~ 2 (ii) any (k − 1)-round-adaptive tester for Pn;k must make Ω(n=k ) queries.