Lecture 17: the Strong Exponential Time Hypothesis 1 Introduction 2 ETH, SETH, and the Sparsification Lemma

CS 354: Unfulfilled Algorithmic Fantasies May 29, 2019 Lecture 17: The Strong Exponential Time Hypothesis Lecturer: Joshua Brakensiek Scribe: Weiyun Ma

1 Introduction

We start with a recap of where we are. One of the biggest hardness assumption in history is P ̸= NP, which came around in the 1970s. This assumption is useful for a number of hardness results, but often people need to make stronger assumptions. For example, in the last few weeks of this course, we have used average-case hardness assumptions (e.g. hardness of random 3-SAT or planted clique) to obtain distributional results. Another kind of results have implications on hardness of approximation. For instance, the PCP Theorem is known to be equivalent to solving some approximate 3-SAT is hard unless P=NP. We have also seen conjectures in this category that are still open, even assuming P ̸= NP, such as the Unique Games Conjecture and the 2-to-2 Conjecture. Today, we are going to focus on hardness assumptions on time complexity. For example, for 3-SAT, P ̸= NP only implies that it cannot be solved in polynomial time. On the other hand, the best known algorithms run in exponential time in 푛 (the number of variables), which motivates the hardness assumption that better algorithms do not exist. This leads to conjectures known as the Exponential Time Hypothesis (ETH) and a stronger version called the Strong Exponential Time Hypothesis (SETH). Informally, ETH says that 3-SAT cannot be solved in 2표(푛) time, and SETH says that 푘-SAT needs roughly 2푛 time for large 푘. We will discuss the connections between ETH and SETH and derive a few hardness results assuming SETH.

2 ETH, SETH, and the Sparsification Lemma

Throughout the rest of the lecture, we use the following notation:

∙ 푘 is the width of a SAT formula. We assume 푘 is constant.

∙ 푛 is the number of variables.

∙ 푚 is the number of clauses. We assume 푚 ∼ poly(푛).

We deﬁne 훿푛 푠푘 = inf{훿 : 푘-SAT can be solved in 2 poly(푚) time}.

Observe that 푠2 = 0 since 2-SAT is in 푃 . For 푘 = 3, the following algorithm shows that 푠3 < 1: We start with a clause in the formula, say involving variables 푥1, 푥2, 푥3, and branch on the 7 possible assignments of 푥1, 푥2, 푥3 that make the clause true. We substitute the partial assignment to the rest of the formula, ﬁnd the next clause with three unassigned variables, and branch again on the 7 possible assignments of the three variables there. Repeating this process, we are left with an instance of 2-SAT in the end, which can be solved in polynomial time. Note that in the original

1 푛 formula, there are at most clauses such that no two of them involve a common variable. Thus 3 the running time of the algorithm is at most 7푛/3poly(푚) = 2(log2 7)푛/3poly(푚). This means that log 7 푠 ≤ 2 < 1. 3 3 Formally, ETH is the following hardness assumption, ﬁrst proposed by Impagliazzo and Paturi in 2001 [1]:

Assumption 2.1 (ETH [1]). There exists 푘 such that 푠푘 > 0. That is, solving 푘-SAT needs exponential time for some 푘. Impagliazzo and Paturi [1] were able to show the following result:

Proposition 2.2 ([1]). ETH (2.1) is equivalent to 푠3 > 0. The implication (⇐) is obvious. Here is an attempt to prove the implication (⇒): We can turn any 푘-SAT formula into a 3-SAT formula by introducing new variables. For example, the satisﬁability of a clause (푥1 ∨ · · · ∨ 푥푘) is equivalent to the satisﬁability of

(푥1 ∨ 푥2 ∨ 푦1) ∧ (푦1 ∨ 푥3 ∨ 푦2) ∧ (푦2 ∨ 푥4 ∨ 푦3) ∧ · · · ∧ (푦푘−3 ∨ 푥푘−1 ∨ 푥푘).

Starting with a 푘-SAT formula with 푛 variables and 푚 clauses, we then obtain a 3-SAT formula 표(푁) with 푁 = 푛 + 푚(푘 − 3) variables. Suppose 푠3 = 0 and we can solve the 3-SAT formula in 2 time. This implies that we can solve the original 푘-SAT formula in 2표(푛+푚(푘−3)) time. However, since 푚 can be 휔(푛), this is not enough for us to conclude that the 푘-SAT formula is solvable in 표(푛) 2 and derive a contradiction to 푠푘 > 0. The above argument would work if we have 푁 = 푂(푛). This is where the following lemma kicks in:

Lemma 2.3 (Sparsiﬁcation Lemma [1]). Let 푓 be a 푘-SAT formula 푓. For any 휖 > 0, there exists 휖푛 an algorithm that outputs formulae 푓1, . . . , 푓2휖푛 in 2 time such that:

1. Each 푓푖 is a 푘-SAT formula on the same variables as 푓.

2. There exists a constant 푐 = 푐(휖, 푘) such that any variable 푥푗 appears at most 푐 times in any formula 푓푖.

3. 푓 is satisfiable if and only if at least one 푓푖 is satisfiable. Let us use Lemma 2.3 to prove the implication (⇒) of Proposition 2.2. For a 푘-SAT formula 푓, we sparsify it into the 푓푖’s using Lemma 2.3. We can then turn each 푓푖 into a 3-SAT formula 푔푖 using the above method. Observe that since each variable appears at most a constant number of times in 푓푖, the number 푁푖 of variables in 푔푖 is at most linear in 푛. Now suppose 푠3 = 0 and we 표(푁푖) 표(푛) 휖푛 표(푛) can solve each 푔푖 in 2 = 2 time. Then, the total time for solving 푓 is 2 2 . Since 휖 can be taken to be arbitrarily small, it follows that 푠푘 = 0, which is a contradiction. For the rest of the lecture, we will focus on SETH, which is also proposed in [1].

Assumption 2.4 (SETH [1]). lim 푠푘 = 1. 푘→∞

2 In fact, our current best algorithms for 푘-SAT give us Θ(1) 푠 ≤ 1 − . 푘 푘 SETH roughly conjectures that no algorithms can beat this bound other than possibly reducing the 푘 factor to log 푘, etc.

3 Orthogonal Vectors

We now discuss some examples that use SETH (2.4) to derive hardness results for problems that are solvable in polynomial time (!). The first such problem is Orthogonal Vectors (OV): given two sets 퐴, 퐵 of {0, 1}-vectors in a 푑-dimensional space, do there exist 푎 ∈ 퐴, 푏 ∈ 퐵 such that ⟨푎, 푏⟩ = 0? For simplicity, we assume that |퐴| = |퐵| = 푁, and 푑 = Θ(log2 푁). (In fact, it is sufficient to assume 푑 = 휔(log 푁); otherwise the sets 퐴, 퐵 would either have repetitions or be the whole space {0, 1}푑 of vectors.) We have a 푂(푁 2푑) baseline algorithm that checks the inner products of all possible pairs of vectors. The following result of Williams [3] shows that, if we assume SETH, then we cannot find algorithms with much better time complexity: Theorem 3.1 ([3]). Assume SETH (2.4). Then for all 휖 > 0, solving OV needs Ω(푁 2−휖) time. Proof. The idea is to reduce 푘-SAT to OV and use the hardness assumption SETH on 푘-SAT to show that OV is also hard. We fix 푘 and take a 푘-SAT formula 푓 on 푛 variables 푥1, . . . , 푥푛 with 푛/2 푚 푚 clauses 퐶1, . . . , 퐶푚. We will define an injective map 푎 : {0, 1} → {0, 1} which embeds all 푚 possible assignments of the variables 푥1, . . . , 푥푛/2 as a set 퐴 of vectors in {0, 1} ; similarly, we will define an injective map 푏 : {0, 1}푛/2 → {0, 1}푚 which embeds all possible assignments of the 푚 variables 푥푛/2+1, . . . , 푥푛 as a set 퐵 of vectors in {0, 1} . Most importantly, we will make the maps 푎 and 푏 satisfy the following condition: ⟨푎(훼), 푏(훽)⟩ = 0 ⇔ partial assignments 훼 and 훽 together satisfy 푓, (1) which reduces the 푘-SAT instance 푓 to an instance of OV. Let us first explain why condition (1) allows us to prove the theorem. Let 푁 = 2푛/2 = |퐴| = |퐵| and 푑 = 푚 ≈ 푛 ≈ log 푁. (Here, we only consider the case where 푚 = Ω(푛). For the case 푚 = 표(푛) there are much faster algorithms for 푘-SAT.) For 휖 > 0, suppose we have an 푂(푁 2−휖) algorithm for 푂푉 . The above reduction then implies that 푘-SAT can be solved in 푂(2(2−휖)푛/2) = 푂(2(1−휖/2)푛) 휖 time. Since 푘 does not depend on 휖, we have shown that 푠 < 1 − for all 푘. This contradicts 푘 2 SETH (2.4). It remains to define the embeddings 푎 and 푏 that satisfy condition (1). For all 푖 = 1, . . . , 푚, we set {︃ 0 if 퐶푖 is satisfied by the partial assignment 훼, 푎(훼)푖 = 1 otherwise; {︃ 0 if 퐶푖 is satisfied by the partial assignment 훽, 푏(훽)푖 = 1 otherwise.

Now ⟨푎(훼), 푏(훽)⟩ = 0 if and only if every clause 퐶푖 is satisﬁed by either 훼 or 훽, which is equivalent to that 훼 and 훽 together satisfy 푓.

3 4 Nearest Neighbor

Now we turn to the geometric problem of the Nearest Neighbor (NN): for a set of points 퐴 in some space and a query point 푏, find the point 푎 in 퐴 that is closest to 푏 and the distance between 푎 and 푏. We might have a set 퐵 of query points to be processed. We will discuss two settings of NN. In the offline, NN the algorithm is given both the initial set 퐴 and the query points 퐵 at once and asked to answer all queries in 퐵. In the online NN, the algorithm is given the set 퐴 at first to preprocess and then asked to answer incoming queries one at a time as fast as possible. The offline NN problem is related to the simpler problem of finding the Bichromatic Closest Pair (BCP): given two sets 퐴 and 퐵 of points in some space (as above), find 푎 ∈ 퐴 and 푏 ∈ 퐵 such that ‖푎 − 푏‖ is as small as possible. (We will mostly be working with the hypercube {0, 1}푑 endowed with the ℓ1-norm.) Note that solving offline NN in particular solves BCP. Thus any lower bound on the complexity of BCP is a lower bound on the complexity of offline NN. Suppose |퐴| = |퐵| = 푁. The following result of Williams [3] gives a lower bound on the time complexity of BCP assuming SETH:

Theorem 4.1 ([3]). Assume SETH (2.4). Then for all 휖 > 0, solving BCP needs Ω(푁 2−휖) time.

As a consequence, solving oﬄine NN also needs Ω(푁 2−휖) time if we assume SETH.

Proof. The idea is to reduce OV to BCP and use the hardness of OV assuming SETH (see Theorem 3.1). Take two sets of vectors 퐴, 퐵 ⊂ {0, 1}푑 with equal size 푁 = |퐴| = |퐵|. We assume that 푑 = 푂(log2 푁). We have the following relation: for 푎, 푏 ∈ {0, 1}푑,

‖푎 − 푏‖1 = ‖푎‖1 + ‖푏‖1 − 2⟨푎, 푏⟩. (2)

For 푏 ∈ {0, 1}푑, denote ¯푏 = 1¯ − 푏, where 1¯ is the vector whose entries are all 1. Using (2), we have that for 푎, 푏 ∈ {0, 1}푑,

‖푎 − ¯푏‖1 = ‖푎‖1 + ‖¯푏‖1 − 2⟨푎, ¯푏⟩ = ‖푎‖1 + ‖¯푏‖1 − 2(‖푎‖1 − ⟨푎, 푏⟩) = ‖¯푏‖1 − ‖푎‖1 + 2⟨푎, 푏⟩. (3)

Thus, if we partition 퐴 and 퐵 into the subsets

퐴푖 = {푎 ∈ 퐴 : ‖푎‖1 = 푖}, 퐵푖 = {푏 ∈ 퐵 : ‖¯푏‖1 = 푖} and set 퐵¯푖 = {¯푏 : 푏 ∈ 퐵푖},

(3) then implies that for any 푎 ∈ 퐴푖 and ¯푏 ∈ 퐵¯푗, we have

‖푎 − ¯푏‖1 = 푗 − 푖 + 2⟨푎, 푏⟩ ≥ 푗 − 푖. (4)

Here, the inequality follows from the nonnegativity of the inner product, and equality holds if and only if ⟨푎, 푏⟩ = 0. Therefore, to ﬁnd 푎 ∈ 퐴, 푏 ∈ 퐵 with ⟨푎, 푏⟩ = 0, we can run BCP on the pairs of sets 퐴푖 and 퐵¯푗 for all 푖, 푗 ∈ {0, . . . , 푑} and check whether equality in (4) is ever achieved among the norm- minimizing pairs of vectors. If we have an 표(푁 2−휖) algorithm for 퐵퐶푃 , the above gives us an 표((푑 + 1)2푁 2−휖) = 표(푁 2−휖 log4 푁) algorithm for 푂푉 , which contradicts Theorem 3.1.

4 Theorem 4.1 can be used to deduce the following hardness result for online NN, due to Vas- silevska Williams and Williams [4] (proof as presented in Rubinstein [2]).

Theorem 4.2 ([4]). Assume SETH (2.4). Let 훿, 푐 > 0. Assume an algorithm is allowed 푂(푁 푐) time to preprocess 퐴. It still needs Ω(푁 1−훿) time to answer each online NN query 푏 ∈ 퐵.

Proof. The idea is to reduce offline NN to online NN and use the hardness of offline NN assuming 1 SETH (see the comment after Theorem 4.1). Take sets 퐴, 퐵 as before. Pick the parameter 훾 = . 2푐 1−훾 훾 We partition 퐴 into 푁 subsets 퐴1, . . . , 퐴푁 1−훾 such that each 퐴푖 has 푁 elements. We can 훾 푐 1/2 3/2−훾 preprocess each 퐴푖 in 푂((푁 ) ) = 푂(푁 ) time, and the total preprocessing time is 푂(푁 ). Now for each 푏 ∈ 퐵, we can treat it as an online query for each 퐴푖 and take the best result. Suppose we have an online NN algorithm that runs in 표(푁 1−훿) time per query. Then the above gives us an offline NN algorithm that has runtime

푂(푁 3/2−훾) + 표(푁 · 푁 1−훾 · (푁 훾)1−훿) = 표(푁 2−훿훾), which contradicts the hardness of oﬄine NN assuming SETH.

References

[1] Russell Impagliazzo and Ramamohan Paturi. On the complexity of k-sat. Journal of Computer and System Sciences, 62(2):367–375, 2001.

[2] Aviad Rubinstein. Hardness of approximate nearest neighbor search. In Proceedings of the 50th Annual ACM SIGACT Symposium on Theory of Computing, STOC 2018, pages 1260–1268, New York, NY, USA, 2018. ACM.

[3] Ryan Williams. A new algorithm for optimal 2-constraint satisfaction and its implications. Theoretical Computer Science, 348(2):357–365, 2005.

[4] Virginia Vassilevska Williams and Ryan Williams. Subcubic equivalences between path, ma- trix and triangle problems. 2010 IEEE 51st Annual Symposium on Foundations of Computer Science, pages 645–654, 2010.