05‐09‐2015 PARALLEL AND DISTRIBUTED ALGORITHMS BY DEBDEEP MUKHOPADHYAY AND ABHISHEK SOMANI http://cse.iitkgp.ac.in/~debdeep/courses_iitkgp/PAlgo/index.htm THE EULER TOUR TECHNIQUE: EVALUATION OF TREE FUNCTIONS 1 05‐09‐2015 AN OVERVIEW When is a problem parallelizable? The complexity classes P and NC and the notion of P-completeness. The Circuit Value Problem is P-complete. WHEN IS A PROBLEM PARALLELIZABLE? A problem is parallelizable if it can be solved very fast with a reasonable number of processors. By very fast, we mean polylogarithmic time, i.e., O(logk n) time for some fixed constant k and input size n. Another requirement is that the number of processors used should be a polynomial in the input size, i.e., O(nc) for a fixed constant c. 2 05‐09‐2015 THE CLASS NC We now discuss the complexity class NC. This is Nick’s class in honour of Nick Pippenger. The class NC consists of all the problems that can be solved by polylogarithmic-time algorithms on a PRAM, using a polynomial number of processors. THE CLASS NC The class NC is defined to be the set of all language L such that: on all inputs of length n, L can be recognized by a PRAM algorithm running in O(logk n) time. 3 05‐09‐2015 P AND NC Let P be the class of all languages that can be recognized by a deterministic Turing machine within a polynomial number of steps. P can be defined as the class of all problems that can be solved on a RAM in polynomial time. P AND NC Obviously, NC P since we can take any NC algorithm and convert it into a polynomial time sequential algorithm. However, it is not known whether P NC. Most people believe that P NC. 4 05‐09‐2015 P AND NC If P NC, then there must be problems in P that cannot be solved efficiently in parallel. The class of P-complete problems are most likely candidates of problems that are not in NC. P AND NC The situation is similar to the relationship between the class NP and P. If someone can prove that there is a polynomial-time algorithm for at least one NP-complete problem, then there are polynomial time algorithms for all NP- complete problems. 5 05‐09‐2015 REDUCIBILITY Similarly, if there is a polylogarithmic-time algorithm for at least one P-complete problem, then there are polylogarithmic- time algorithms for all P-complete problems. How do we prove a problem to be P- complete? REDUCIBILITY Let L1 and L2 be two languages. The language L1 is NC-reducible to the language L2 if: there exists an NC-algorithm that transforms an arbitrary input u1 for L1 into an input u2 for L2 such that: u1 L1 if and only if u2 L2. 6 05‐09‐2015 THE NOTION OF P- COMPLETENESS A language L is P-complete if: 1. L P, and 2. every language in P is NC-reducible to P. The next question is: Is there a P-complete problem? THE CIRCUIT VALUE PROBLEM (CVP) We are given a Boolean circuit consisting of NOT gates and two-valued AND and OR gates. The problem is to determine the output of the Boolean circuit for a given set of inputs. More precisely, our Boolean circuit C is specified by a sequence of gates C = < g1, g2, …, gn >. 7 05‐09‐2015 THE CIRCUIT VALUE PROBLEM (CVP) Each gi is: •either an input, or •gi = gj gk, or gi = gj gk , or gi = gj for j, k < i. CVP IS P-COMPLETE We need to prove that: CVP is in P, and An arbitrary language L P is NC-reducible to CVP. CVP is clearly in P. Given an n-gate CVP, we can evaluate the gates sequentially in O(n) time by evaluating g1, g2, …, gn. 8 05‐09‐2015 CVP IS P-COMPLETE We next show that an arbitrary language L P is NC-reducible to CVP. We provide an NC algorithm that takes an arbitrary instance IL of L, and maps IL into an instance ICVP of CVP such that, IL has a yes answer if and only if ICVP has a value 1. TURING MACHINE 9 05‐09‐2015 TURING MACHINE Since L P , there exists a one-tape deterministic Turing machine M that accepts L in polynomial time T(n), for an input size n. We can assume that the input bits appear in n consecutive tape cells numbered 1 to n and all other cells are initially blank. The tape head is initially scanning cell 1 and after T(n) steps, the output is written into cell 1. TURING MACHINE Let Q = { q1,…, qs } be the set of states of M and q1 is the initial state. Let = { a1,…, am } be the tape alphabet. The state transition function is given by: : Q Q {L , R } 10 05‐09‐2015 TURING MACHINE We will show how to construct a Boolean circuit C such that the value of C is 1 if and only if M accepts L, i.e., cell 1 contains 1 at time T(n). Note that, M can only access T(n) cells in T(n) time. Hence, all cell indices i will be in the range 1 i T(n). All time steps are within the range 0 t T(n). There are only s states in M hence, all states qj are within the range 1 j s. SOME TOOLS The circuit C will be defined by the following Boolean functions: H(i , t), such that H(i , t) = 1 if and only if the head scans cell i at time t. C(i , j, t), such that C(i , j, t) = 1 if and only if cell i contains character aj at time t. S(k , t), such that S(k , t) = 1 if and only if the state of M is qk at time t. 11 05‐09‐2015 SOME TOOLS Our circuit C will consist of T(n) levels. For each time step of the Turing machine M, one level of our circuit will emulate the behavior of M. Level l will contain the gates computing the Boolean function H(i , l), C(i , j, l), S(k , l) for 1 i T(n). Note that, we don’t know exactly how M recognizes L, but in our circuit C, we capture all possible steps of M at every level. SOME TOOLS The number of gates at every level is T(n). We will first specify H(i , 0), C(i , j, 0), S(k , 0). These are the inputs to our circuit. We will then show how to express H(i , t + 1), C(i , j, t + 1), S(k , t + 1) in terms of H(*, t), C(*, *, t), S(*, t). This will give us a correct description of the circuit C. 12 05‐09‐2015 INPUT GATES FOR OUR CIRCUIT For t = 0, the tape head is scanning cell 1, the input bits are stored in cell 1 to n and M is in state q1. H(i , 0)= 1if i = 1, 0 otherwise. Since, the head is scanning cell 1. C(i , j, 0) = 1 if cell i contains initially aj. S(k , 0) = 1, if k = 1, 0 otherwise. GATES AT A GENERAL LEVEL Let us consider constructing H(i , t +1) when we know H(*, t), C(*, *, t), S(*, t). H(i , t +1) = 1 if and only if the head scans cell i at time t +1. Since the Turing machine M moves its head only one cell at each time step, It must be either on cell i - 1 or on cell i + 1 at time t. Let us consider the case when the head is at cell i -1at time t 13 05‐09‐2015 GATES AT A GENERAL LEVEL H(i , t +1) = 1 if and only if the head moves right and goes to cell i at time t +1. Suppose the content of cell i -1is aj and the Turing machine M is in state qk. Hence, to move the head to cell i, our transition function should be such that: (qk, aj) = ( , , R). That is, if the current state is qk and cell i -1 contains character aj, the head moves right. GATES AT A GENERAL LEVEL Suppose, IR is the set of all pairs {(k, j): (qk, aj) = ( , , R)}. We do not know the exact movement of the head of M when M recognizes L. So, we have to take into account all possible state transitions that may result in the head moving to cell i at time t + 1. Consider a pair (k, j) IR. 14 05‐09‐2015 GATES AT A GENERAL LEVEL For the pair (k, j) IR, the action of the Turing machine M can be written as: C(i -1, j, t) S(k , t) Hence, if the head of M is at cell i –1 cell i – 1 contains the character aj, and M is in state qk, The head of M will move to the cell i if and only if: H(i -1, t) (C(i -1, j, t) S(k , t)). GATES AT A GENERAL LEVEL But we do not know the state of M or the content of cell i -1at time t. Hence, we have to consider all possible states and all possible characters in cell i -1. We can write this as: This takes into account all possible characters aj in cell i - 1 and all possible states qk of M so that the head moves to cell i.