Applied Logic Exercises - Complexity of satisfiability Marcin Szczuka Institute of Informatics, The University of Warsaw Monographic lecture, Spring semester 2018/2019 Marcin Szczuka (MIMUW) Applied Logic 2019 1 / 39 Plan wykładu 1 Computational compexity Time complexity Space complexity Complexity in non-deterministic case 2 Complexity classes Complexity class PSPACE Complexity class PTIME Complexity class NPTIME 3 NP-complete problems Polynomial problem reduction NP completeness SAT problem and Cook’s theorem Other versions of SAT NP-hardness and MAX-SAT Marcin Szczuka (MIMUW) Applied Logic 2019 2 / 39 cost function We will use the Turing Machine model (sometimes multi-tape) as our basic model of computation. Using the Church’s Thesis we will assume that it coresponds to “normal” algorithms. Notational convention Let f : N 7! N be a non-decreasing natural function. We state that function g : N 7! N is of order f(n), denoted by O(f(n)) if for f(n) 6= 0 exist a finite limit: g(n) lim n!1 f(n) Note that with this convention we are always looking at the fastest growing component of the function g. Marcin Szczuka (MIMUW) Applied Logic 2019 3 / 39 Plan wykładu 1 Computational compexity Time complexity Space complexity Complexity in non-deterministic case 2 Complexity classes Complexity class PSPACE Complexity class PTIME Complexity class NPTIME 3 NP-complete problems Polynomial problem reduction NP completeness SAT problem and Cook’s theorem Other versions of SAT NP-hardness and MAX-SAT Marcin Szczuka (MIMUW) Applied Logic 2019 4 / 39 Time complexity Class TIME(:) Let f(n) be a non-decreasing natural function. By TIME(f(n)) we denote a class of decision problems (languages) πL such, that there exist constants a; b 2 R and a Turing Machine M for which M can verify if w 2 L using no more than af(n) + b computational steps, regardless of the choice of w as long as jwj = n. In the definition above: 1 We mostly care of the fastest growing component. 2 We disregard the “overhead” associated with, e.g, data re-coding. This is only represented by the constant b. 3 We assume that one computational step takes one unit of time, i.e., the time of computation is equivalent to the number of steps. Marcin Szczuka (MIMUW) Applied Logic 2019 5 / 39 Plan wykładu 1 Computational compexity Time complexity Space complexity Complexity in non-deterministic case 2 Complexity classes Complexity class PSPACE Complexity class PTIME Complexity class NPTIME 3 NP-complete problems Polynomial problem reduction NP completeness SAT problem and Cook’s theorem Other versions of SAT NP-hardness and MAX-SAT Marcin Szczuka (MIMUW) Applied Logic 2019 6 / 39 Space (memory) complexity Class SPACE(:) Let f(n) be a non-decreasing natural function. By SPACE(f(n)) we denote a class of decision problems (languages) πL such, that there exist constants a; b 2 R and a Turing Machine M for which M can verify if w 2 L using no more than af(n) + b different positions (cells) on the tape, regardless of the choice of w as long as jwj = n. In the definition above: 1 We mostly care of the fastest growing component. 2 We disregard the “overhead” associated with, e.g, storage of the input. 3 We assume that in one computational step we can “occupy” no more than one unit of memory (one cell on tape). Marcin Szczuka (MIMUW) Applied Logic 2019 7 / 39 Plan wykładu 1 Computational compexity Time complexity Space complexity Complexity in non-deterministic case 2 Complexity classes Complexity class PSPACE Complexity class PTIME Complexity class NPTIME 3 NP-complete problems Polynomial problem reduction NP completeness SAT problem and Cook’s theorem Other versions of SAT NP-hardness and MAX-SAT Marcin Szczuka (MIMUW) Applied Logic 2019 8 / 39 Non-deterministic Turing Machine Non-deterministic Turing Machine Non-deterministic Turing Machine (NTM) is a tuple M = (Q; Σ; Γ; δ; q0; B; F ) where: Q – finite set of states, including the initial state q0; Γ – finite set of tape symbols, including the empty symbol B (Blank) and input symbols; Σ – set of input symbols, such that B2 = Σ and Σ ⊂ Γ; F ⊂ Q – set of terminal states; δ – transition relation δ ⊂ Q × Γ × Q × Γ × fL; Rg, where L i R correspond to head movements (Left/Right). An NTM in a given configuration may have more than one possibility of action for a given state and symbol on the tape. Marcin Szczuka (MIMUW) Applied Logic 2019 9 / 39 Properties of NTM In case od deterministic TMs we can talk about a step going from 0 configuration (q1; t1; u) to (q2; t2; u ), given that σ(q1; t1) = (q2; t2; h) and information u0 on tape is a result of applying h to u. In NTM we can talk about a admissible step going from configuration 0 (q1; t1; u) to (q2; t2; u ) if (q1; t1; q2; t2; h) 2 δ. Language of NTM A non-deterministic Turing Machine M decides(induces) the language L = L(M) ⊂ Σ∗, if for each w 2 L(M) there exists a finite sequence of admissible steps in M that ends in a terminal state and each transition in this sequence complies with relation δM for M. Non-deterministic Turing Machine M accepts a word from language L(M) if there exists at least one finite path of computation in the computation tree for this word that is admissible and ends in a terminal state. How to select the right path in the computation tree?! Marcin Szczuka (MIMUW) Applied Logic 2019 10 / 39 Non-deterministic time complexity Class NTIME(:) Let f(n) be a non-decreasing natural function. By NTIME(f(n)) we denote a class of decision problems (languages) πL such, that there exist constants a; b 2 R and a non-deterministic Turing Machine M for which every computation path of M that can verify if w 2 L contains no more than af(n) + b steps, regardless of the choice of w as long as jwj = n. In the definition above: 1 We consider the longest admissible computation path. 2 We disregard of the size of (possibly enormous) computation tree. 3 Wea assume that a “Ferry Godmother” (an oracle) shows us which path to take. Marcin Szczuka (MIMUW) Applied Logic 2019 11 / 39 Plan wykładu 1 Computational compexity Time complexity Space complexity Complexity in non-deterministic case 2 Complexity classes Complexity class PSPACE Complexity class PTIME Complexity class NPTIME 3 NP-complete problems Polynomial problem reduction NP completeness SAT problem and Cook’s theorem Other versions of SAT NP-hardness and MAX-SAT Marcin Szczuka (MIMUW) Applied Logic 2019 12 / 39 Complexity classes In theory (and practice) of complexity theory we are usually most interested in classes (types) of problems that have specific degree of hardness (complexity). Most commonly considered classes: Problems with constant complexity, especially w.r.t. memory. Linear time (and/or space) compelxity. Log-linear problems (n log n), e.g, sorting algorithms. Polynomial problems – deterministic and non-deterministic. The last of these classes (polynomial) will be of special interest to us, as it has many properties with practical ramifications. Marcin Szczuka (MIMUW) Applied Logic 2019 13 / 39 Plan wykładu 1 Computational compexity Time complexity Space complexity Complexity in non-deterministic case 2 Complexity classes Complexity class PSPACE Complexity class PTIME Complexity class NPTIME 3 NP-complete problems Polynomial problem reduction NP completeness SAT problem and Cook’s theorem Other versions of SAT NP-hardness and MAX-SAT Marcin Szczuka (MIMUW) Applied Logic 2019 14 / 39 Problems of polynomial memory cost Note, that if a problem is in SPACE(nk) then there exists an algorithm that solves it using a memory proportional to nk. Hence, the estimation for complexity of our problem is given by a polynomial of the degree at most k. Complexity class PSPACE Polynomial space (memory) problems (denote by PSPACE) are a class of decision problems (languages) πL such, that for each L 2 πL exists k 2 N for which L 2 SPACE(nk). Hence: [ PSPACE = SPACE(nk) k2N In non-determininistic case definition of NPSPACE class has more caveats. It is, however, possible to prove that PSPACE = NPSPACE. The proof is non-trivial. Marcin Szczuka (MIMUW) Applied Logic 2019 15 / 39 Plan wykładu 1 Computational compexity Time complexity Space complexity Complexity in non-deterministic case 2 Complexity classes Complexity class PSPACE Complexity class PTIME Complexity class NPTIME 3 NP-complete problems Polynomial problem reduction NP completeness SAT problem and Cook’s theorem Other versions of SAT NP-hardness and MAX-SAT Marcin Szczuka (MIMUW) Applied Logic 2019 16 / 39 Problems of polynomial (time) cost Just like in the case of memory, the class of problems with polynomial time cost will be of special interest to us. Complexity class P = PTIME Polynomial (time cost) problems, traditionally denoted by P = PTIME, are a class of decision problems (languages) πL such, that for each L 2 πL k exists k 2 N for which L 2 TIME(n ). Hence: [ k P = PTIME = TIME(n ) k2N Note, that by assuming that in a single computational step only one memory cell can be altered we obtain: P = PTIME ( PSPACE Marcin Szczuka (MIMUW) Applied Logic 2019 17 / 39 Plan wykładu 1 Computational compexity Time complexity Space complexity Complexity in non-deterministic case 2 Complexity classes Complexity class PSPACE Complexity class PTIME Complexity class NPTIME 3 NP-complete problems Polynomial problem reduction NP completeness SAT problem and Cook’s theorem Other versions of SAT NP-hardness and MAX-SAT Marcin Szczuka (MIMUW) Applied Logic 2019 18 / 39 Problems of non-deterministic polynomial (time) cost The polynomial time cost will be of special interest to us in non-deterministic case, just as it was in deterministic situation.