Computability and computational complexity
Lecture 1: Introduction
Ion Petre Computer Science, Åbo Akademi University
Fall 2015 http://users.abo.fi/ipetre/computability/
May 21, 2018 http://users.abo.fi/ipetre/computability/ 1 What is computer science?
n Hugely successful discipline, best seen from the impact of computers and computer technologies in everyday life
q computers control aircrafts, nuclear powerplants, chemical plants
q computers help run companies, guide investments
q many medical procedures cannot be executed without computers; intelligent body implants interacting with the organism
q computers execute tasks with great precision: intricate simulations, data mining, robot-aided manufacturing, integrated circuit design
q handle huge amounts of data, process them, extract information
q help in education
q provide entertainment
q help us stay connected
May 21, 2018 http://users.abo.fi/ipetre/computability/ 2 What is computer science
n Profound influence on many other disciplines
q computational physics
q computational mathematics
q computational linguistics
q computational and algorithmic music
q computational intelligence
q computational philosophy
q computational neuroscience
q computational economics
q …
May 21, 2018 http://users.abo.fi/ipetre/computability/ 3 Computer Science as a scientific discipline
n While answering to challenges from technology and from other sciences, Computer Science also developed as a fundamental discipline
q Developed its own language and tools to reason about computing
q Independently of the physical implementation of computers
q Identified several great challenges stemming from within Computer Science itself; some received great recognition from the scientific community n P=NP? one of the jewels of Computer Science
q Clay Mathematics Institute: http://www.claymath.org/millennium/P_vs_NP/
q Great company, see e.g. the Riemann hypothesis (1859)
q A problem about the inherent limits of computing
May 21, 2018 http://users.abo.fi/ipetre/computability/ 4 Algorithms
n Long studied in mathematics
q The oldest known algorithm is Euclid’s algorithm to compute the greatest common divisor – more than 2000 years old
q Term “algorithm” comes from the Persian mathematician Abu Abd-allah Mohammad ibn Musa al-Khwarizmi (ca. 780 – ca. 850; Latin version of his name Algoritmi)
n developed the discrete algorithmic rules (still used today) to compute additions, subtractions, multiplications and divisions of decimal numbers
Source: Wikipedia
May 21, 2018 http://users.abo.fi/ipetre/computability/ 5 Theory of algorithms: three directions
q focuses on the study of undecidability
q started with Gödel’s proof that there are algorithmically undecidable problems
n destroyed David Hilbert’s (1862-1943) dream that all “reasonably defined” problems are in principle algorithmically decidable n Design and analysis of algorithms (not covered in this course)
q grew in connection with the developing of computer technologies
q focuses on developing fast algorithms, methods for analyzing them n Computational complexity
q focuses on how many resources (e.g., time and space) are needed to solve a given problem (not an algorithm)
q complexity classes
May 21, 2018 http://users.abo.fi/ipetre/computability/ 6 Focus of the course
n Central concept in the course: computational problems q Seen from the point of view of applications: solve them (algorithms) q Seen from the point of view of computability: study their properties n Reduce them to another computational problem n Establish their complexity class n Degrees of complexity n Focus in this course on the inherent limitations of the world of computing q Things that computers really cannot do q Things that computers really cannot do efficiently q What (not) to do in practice with a problem of high computational complexity
May 21, 2018 http://users.abo.fi/ipetre/computability/ 7 Content of the course
n Problems and algorithms n coNP n Turing machines n Randomized computation n Computability n Approximability n Boolean logic n On P vs NP n Relations between complexity n Parallel computation classes n Polynomial space n Relations and completeness n Beyond Turing n NP-complete problems
May 21, 2018 http://users.abo.fi/ipetre/computability/ 8 Learning objectives
n Master the main computational complexity classes, their underlying models of computation, and relationships.
n Understand the concept of reductions and its role in classifying problems by their computational complexity.
n Understand the concept of “unsolvable/undecidable”
n Understand the concept of universal machines
n Understand the meaning, the significance and the implications of the P vs. NP problem
n Become familiar with the concepts of randomized, approximation, and parallel algorithms and become aware of the related complexity classes.
May 21, 2018 http://users.abo.fi/ipetre/computability/ 9 Administrative details
n Webpage: http://users.abo.fi/ipetre/computability/ n Email address: [email protected] n Teaching assistant: Diana-Elena Gratie [email protected]
n Lectures: Mondays 10.15-11.45 and Wednesdays 10.15-11.45 (Catbert, B3028, ICT House) n Exercise sessions: Mondays 13.30-15.00 (Catbert, B3028, ICT House) n Exam: 30 points for maximum mark, 15 to pass q Exam dates: 18.12.2015, 8.1.2016
n Course book: Christos H. Papadimitriou, Computational complexity. Addison-Wesley, 1994.
n Other useful books: q O. Goldreich. Computational complexity: a conceptual perspective. Cambridge University Press, 2008 q M. Sipser. Introduction to the theory of computation. PWS Pub co, 1996. q O. Goldreich. P, NP, and NP-completeness. Cambridge University Press, 2010. q D. Harel. Computers ltd. What they really can't do. Oxford University Press, 2000.
n Prerequisites: formal languages and automata, algorithmics
May 21, 2018 http://users.abo.fi/ipetre/computability/ 10 Problems vs. algorithms
n We focus in this course on the computational complexity of problems, not of algorithms n Question: what is a “problem”? n Introduce here three simple examples
q REACHABILITY
q MAX FLOW
q TSP
q through them introduce a few key definitions and concepts
May 21, 2018 http://users.abo.fi/ipetre/computability/ 11 Example 1: REACHABILITY
n REACHABILITY
q Instance: a finite directed graph G=(V,E) and two nodes 1,n ∈ V
q Question: Is there a path in G from 1 to n?
q Example:
1 4
5
2 3
May 21, 2018 http://users.abo.fi/ipetre/computability/ 12 Algorithm for REACHABILITY
S={1}; mark node 1 while S≠Ø do choose a node i from S and remove it from S; for all edges (i,j)∈E do if j is unmarked, then mark j and add it to S endif endfor endwhile if n is marked, then return “yes: there is a path from 1 to n” else return “no: there is no path from 1 to n” endif
May 21, 2018 http://users.abo.fi/ipetre/computability/ 13 REACHABILITY
n The algorithm is correct q prove by induction on the number of nodes of the graph that there is a path from 1 to n if and only if n is marked at the end of the algorithm n Unclear points in the algorithm q how is the graph represented in the algorithm? n we discuss later that this depends on our model of an algorithm n assume here we have the adjacency matrix q how do we choose element i from S n different strategies will lead to different graph traversal strategies q if S is a queue: breadth-first traversal q if S is a stack: depth-first traversal q how efficient is the algorithm n note that each edge is considered once n In terms of edges O(|E|) steps n in terms of nodes: O(|V|2) steps
May 21, 2018 http://users.abo.fi/ipetre/computability/ 14 Measure of algorithm efficiency
n Typically measured by the rate of growth of the running time (or memory consumption)
q how does the running time grow when the input size grows n The O-notation most used here
q multiplicative and additive constants ignored
May 21, 2018 http://users.abo.fi/ipetre/computability/ 15 O-notation
Let f,g:N→N n f(n)=O(g(n)) if f(n) ≤ C·g(n) for all n≥n0 for two constants n0 and C n f(n)=Ω(g(n)) if g(n)=O(f(n))
q Equivalently: f(n) ≥ C·g(n) for all n≥n0 for two constants n0 and C n f(n)=Q(g(n)) if f(n)=O(g(n)) and g(n)=O(f(n))
q k1·g(n)≤f(n) ≤ k2·g(n), for all n≥n0, for some constants k1, k2, n0 n Example: Let p(n) a polynomial of degree k k q p(n)= Q(n ) n n q for any constant c>1, p(n)=O(c ), but p(n)≠Ω(c )
n In other words: any polynomial grows strictly slower than any exponential k q Consequence: log n=O(n)
May 21, 2018 http://users.abo.fi/ipetre/computability/ 16 Efficiently solvable vs. intractable problems
n Throughout this course we consider that:
q a problem is efficiently solvable if there is an algorithm solving the problem such that the rate of growth of its solution time is polynomial, i.e., it is O(nd), for some d
q a problem is intractable when it has no polynomial time algorithm
May 21, 2018 http://users.abo.fi/ipetre/computability/ 17 Discussion
n Polynomial time bounds as a definition for efficiently solvable problems q there exist efficient computations that are not polynomial n for example O(2n/100) q there exist polynomial computations that are not efficient n for example O(n100) q Argument: any polynomial rate will eventually be overcome by any exponential one; the latter rate is only better for a finite number of instances n Our definitions only consider worst-case complexity q the exponential worst-case performance of an algorithm may be due to a statistically insignificant (or uninteresting) fraction of the inputs; might perform for most inputs q more informative would be the expected performance of an algorithm, rather than its worst-case one q Argument: the input distribution of a problem is rarely available in practice
“Adopting polynomial time worst-case performance as our criterion of efficiency results in an elegant and useful theory that says something meaningful about practical computation, and would be impossible without this simplification” Papadimitriou, 1994
May 21, 2018 http://users.abo.fi/ipetre/computability/ 18 Example 2: MAX FLOW
MAX FLOW n Instance: Network N=(V,E,s,t,c), where
q (V,E) is a directed graph, s,tÎV (source/sink)
q no incoming edges into s, no outgoing edges from t
q for each edge (i,j)∈E, c(i,j) is a non-negative integer (the capacity of the edge)
q flow of N is a function over non-negative integers f(i,j)≤c(i,j) such that for each node other than s and t, the sum of flows of its incoming edges is equal to the sum of the flow of its outgoing edges
q the value of the flow is the sum of the flows of the outgoing edges of s (the same as the sum of the flows of the incoming edges in t) n Question: What is the largest possible value for a flow in N?
May 21, 2018 http://users.abo.fi/ipetre/computability/ 19 MAX FLOW – an example
1
2 4
Network N s 2 t
3 3 1
A flow of value 3 Increasing the flow to 4 0/1 1/1 0/2 1/2 2/4 3/4
s 2/2 t s 2/2 t
3/3 1/3 3/3 1/3
1/1 1/1
May 21, 2018 http://users.abo.fi/ipetre/computability/ 20 Discussion
n MAX FLOW q this is not a decision problem as REACHABILITY q it is an optimization problem q it can be made into a decision problem: is there a flow of value at least K, for some constant K n Historical details q proposed in 1954 by T.E.Harris as a simplified model of Soviet railway traffic flow q First solution (exponential time): Ford-Fulkerson algorithm, 1955 q First polynomial algorithm: Edmonds, Karp, 1972 n Nice example of a problem where, once the exponential barrier was broken, better and better polynomial time algorithms were developed 5 3 q O(n ), …, O(n ), … n Note: there is a positive flow iff t is reachable from s
May 21, 2018 http://users.abo.fi/ipetre/computability/ 21 MAX FLOW
n A polynomial-time algorithm for deciding whether a given flow f is optimal or not
q For simplicity, we assume that between no two vertices are there edges of opposing orientation
q Assume there is a bigger flow f’ and define ∆f(i,j)=f’(i,j)-f(i,j)
q Because of the flow equalities in each node, for both f and f’, we get that ∆f also satisfies the equalities, i.e., it is a flow as well
n some of the values ∆f(i,j) might be negative; they can be thought of as a flow in the reverse direction; in this case |∆f(i,j)|≤f(i,j) (because f’(i,j)≥0)
n for the values ∆f(i,j) that are positive, we have that ∆f(i,j)≤c(i,j)-f(i,j) (because f’(i,j)≤c(i,j))
May 21, 2018 http://users.abo.fi/ipetre/computability/ 22 MAX FLOW
n A polynomial-time algorithm for deciding whether a given flow f is optimal or not (continued from previous slide)
q Based on this idea, for a given flow f we build the following derived network N(f)=(V,E’,s,t,c’), where
n E’=E U {(i,j): (j,i)∈E, f(j,i)>0}
n c’(i,j)=c(i,j)-f(i,j) if (i,j)∈E
n c’(i,j)=f(j,i) if (i,j)∈E’-E
n Key idea: f is optimum for N if and only if there is no positive flow in N(f) q Note: s is not a source in N(f); define the flow of N(f) as the sum of the flows on the outgoing edges from s minus the sum of the incoming edges into s
n Note: there is a positive flow if and only of t is reachable from s q à REACHABILITY
May 21, 2018 http://users.abo.fi/ipetre/computability/ 23 MAX FLOW – an example
1
2 4
Network N s 2 t
3 3 1
A flow of value 3 Network N(f) 0/1 1 2 0/2 2/4 2 2 s 2/2 t s 2 t 1 3/3 1/3 3 2 1/1 1
May 21, 2018 http://users.abo.fi/ipetre/computability/ 24 MAX FLOW - a polynomial algorithm
n Start with the everywhere-zero flow in N n Repeatedly build N(f) and find a path from s to t in N(f)
q If there is one, find the smallest capacity of any edge on the path and add this number to the value of f on all edges of this path
q If there is no path then f is maximum and we stop n Complexity of the algorithm: 2 q Each iteration takes O(n ) – solving REACHABILITY
q There can only be n·C stages, where C is the maximum capacity of any edge in the network
n each iteration increases the flow with at least 1 (work over integers) n maximum flow cannot be more than n·C because there are at most n edges going out of the source 3 q Complexity: O(n ·C)
May 21, 2018 http://users.abo.fi/ipetre/computability/ 25 Example 2.1: Bipartite matching
MATCHING n A bipartite graph is a triple B=(U,V,E), where
U={u1, u2,…, un} is a set of nodes called boys and V={v1, v2,…, vn} is a set of girls. EÍUxV is the set of edges n A perfect matching is a set MÍE of n edges such that for any two edges (u,v),(u’,v’)ÎM we have u¹u’ and v¹v’
q no two edges are adjacent to the same boy, no two edges are adjacent to the same girl n Question: Given a bipartite graph, does it have a perfect matching?
May 21, 2018 http://users.abo.fi/ipetre/computability/ 26 A key tool: problem reductions
n A reduction is an algorithm that solves problem A by transforming any instance x of A into an equivalent instance R(x) of a previously solved problem B n An efficient algorithm for B plus an efficient reduction of A to B provides an efficient algorithm for A
Algorithm for A:
Input x R(x) Answer Reduction Algorithm R for B
May 21, 2018 http://users.abo.fi/ipetre/computability/ 27 Reduction of MATCHING to MAX FLOW
s t
n Build a network from the given bipartite graph as suggested above, where all capacities are equal to 1 n The bipartite graph has a perfect matching if and only if its associated network has a flow of value n q our polynomial solution for MAX FLOW yields a polynomial solution for MATCHING
May 21, 2018 http://users.abo.fi/ipetre/computability/ 28 Example 3: Traveling salesman problem
TSP n Instance: Given n cities 1,2,…,n and a nonnegative integer distance
di,j between any two cities i and j (with di,j=dj,i) n Question: what is the shortest tour of the cities?
q In other words: find a permutation p of the cities such that Si dp(i),p(i+1) is minimal (where p(n+1)=1) n The formulation above: optimization problem n TSP as a decision problem: is there a tour of length at most B (budget)?
May 21, 2018 http://users.abo.fi/ipetre/computability/ 29 Discussion
n Naïve solution for TSP: enumerate all possible tours (permutations), compute their cost, pick the best
q Impractical: O(n!) tours n There are no polynomial-time algorithms known for TSP
q Not known whether there are polynomial algorithms for TSP
q Closely related to the P¹NP? problem
May 21, 2018 http://users.abo.fi/ipetre/computability/ 30 Learning objectives for this lecture
n Growth rate, O-notation n The concepts of decision problem, optimization problem n Problem reductions
May 21, 2018 http://users.abo.fi/ipetre/computability/ 31