Complexity Theory

IE 661: Scheduling Theory Fall 2003 Satyaki Ghosh Dastidar Outline z Goals z Computation of Problems { Concepts and Definitions z Complexity { Classes and Problems z Polynomial Time Reductions { Examples and Proofs z Summary

University at Buffalo Department of Industrial Engineering 2 Goals of Complexity Theory z To provide a method of quantifying problem difficulty in an absolute sense. z To provide a method comparing the relative difficulty of two different problems. z To be able to rigorously define the meaning of efficient algorithm. (e.g. Time complexity analysis of an algorithm).

University at Buffalo Department of Industrial Engineering 3 Computation of Problems

Concepts and Definitions Problems and Instances

A problem or model is an infinite family of instances whose objective function and constraints have a specific structure. An instance is obtained by specifying values for the various problem parameters.

Measurement of Difficulty Instance z Running time (Measure the total number of elementary operations).

Problem z Best case (No guarantee about the difficulty of a given instance). z Average case (Specifies a probability distribution on the instances). z Worst case (Addresses these problems and is usually easier to analyze).

University at Buffalo Department of Industrial Engineering 5 Time Complexity

Θ-notation (asymptotic tight bound)  fn( ) : there exist positive constants cc12 , , and n 0 such that Θ=(())gn   0≤≤≤cg12 ( n ) f ( n ) cg ( n ) for all n ≥ n 0 O-notation (asymptotic upper bound)

 fn( ) : there exist positive constants c and n0 such that Ogn(())=   0≤≤fn ( ) cgn ( ) for all n ≥ n0 Ω-notation (asymptotic lower bound)  fn( ) : there exist positive constants c and n0 such that Ω=(())gn   0≤≤cg ( n ) f ( n ) for all n ≥ n0 o-notation (asymptotic “loose” upper bound)  fn( ) : for any positive constant c> 0, there exists a constant ogn(())=   nfncgnnn00>≤<≥0 such that 0 ( ) ( ) for all

University at Buffalo Department of Industrial Engineering 6 …Time Complexity (contd.)

ω-notation (asymptotic “loose” lower bound)  fn( ) : for any positive constant c> 0, there exists a constant ω(())gn =   ncgnfnnn00>≤<≥0 such that 0 ( ) ( ) for all

cgn2 () cg()n

f ()n f ()n f ()n

cg() n cg1 () n

n n n n0 n0 n0 f ()n =Θ (())g n f ()nO= (())g n f ()n = Ω (())g n

University at Buffalo Department of Industrial Engineering 7 Algorithm Types z Polynomial Time Algorithm: An algorithm whose running time is bounded by a polynomial function is called a polynomial time algorithm. Example: Shortest path problem with nonnegative weights. Running Time: O(n2) z Exponential Time Algorithm: An algorithm that is bounded by an exponential function is called an exponential time algorithm. Example: Check every number of n digits to find a solution. Running Time: O(10n) z Pseudopolynomial Time Algorithm: A pseudopolynomial time algorithm is one that is polynomial in the length of the data when encoded in unary. Example: Integer Knapsack Problem. Running time: O(nb)

University at Buffalo Department of Industrial Engineering 8 Turing Machine z A Turing machine is an abstract representation of a computing device. The behavior of a TM is completely determined by: • The state the machine is in, • The number on the square it is scanning, and • A table of instructions or the transition table. “A function is computable if it can be computed by a Turing Machine.” - Church-Turing Hypothesis

University at Buffalo Department of Industrial Engineering 9 Finite State Machine

State Read Write Move Next State

0 0 L S1 S1 B 1 L S2

1 B R S1

0 1 R S2 S2 B 0 R S2

1 1 L S1

State Transition Table for a Turing Transition State Diagram for Machine Turing Machine

University at Buffalo Department of Industrial Engineering 10 Decision Problem

Decision problems are those that have a TRUE/FALSE answer. z SATISFIABILITY: Given a set of variables and a collection of clauses defined over the variables, is there an assignment of values to the variables for which each of the clauses is true? Example: Consider the expression

()()()()xxxxxxxxxxxxxxxx1432124323155142+++ +++ +++ +++

It can be easily verified that the assignment x1=0, x2=0, x3=0, x4=0, and x5=0 gives a truth assignment to each one of the four clauses.

University at Buffalo Department of Industrial Engineering 11 Decision Problems and Reductions z For every optimization problem there is a corresponding decision problem.

Example: Fm||Cmax minimize makespan (optmization). Is there a schedule with a makespan ≤ z ? (decision). Problem Reduction: Problem P reduces to problem P′ if for any instance of P an equivalent instance of P′ can be constructed. Polynomial Reducibility: Problem P polynomially reduces to problem P′ if a polynomial time algorithm for P′ implies polynomial time algorithm for P. P∝P′

University at Buffalo Department of Industrial Engineering 12 Complexity

Classes and Problems Complexity Classes z Definition: (Class P) The class P contains all decision problems for which there exists a Turing machine algorithm that leads to the right “yes/no” answer in a number of steps bounded by a polynomial in the length of the encoding. z Definition: (Class NP) The class NP contains all decision problems for which, given a proper guess, there exists a polynomial time “proof” or “certificate” C that can verify if the guess is the right “yes/no” answer.

NP

P

A tentative view of the world of NP University at Buffalo Department of Industrial Engineering 14 … Complexity Classes (contd.) z Definition: (Class co-P) The class co-P contains all decision problems for which there exists a polynomial time algorithm that can determine what all “yes/no” answers are incorrect. z Definition: (Class co-NP) The class co-NP contains all decision problems such that there exists a polynomial time “proof” or “certificate” C that can verify if the problem does not have the right “yes/no” answer.

co-NP NP P

A view of the world of NP and co-NP

University at Buffalo Department of Industrial Engineering 15 Important Results z P = co-P z NP ≠ co-NP z P ≠ NP It turns out that almost all interesting problems lie in NP and P is the set of easy problems. So are all interesting problems easy, i.e. do we have P = NP? This is the main open question in Computer Science. It is like other great questions z Is there intelligent life in the universe? z What is the meaning of life? z Will you get a job when you graduate?

University at Buffalo Department of Industrial Engineering 16 NP-Complete Problems z Definition: (NP-complete) A decision problem D is said to be NP-complete if DNP and, for all other decision problems D′NP, there exists a polynomial transformation from D′ to D, i.e., D′∝ D. Assumption: P≠NP. Result: If any single NP-complete problem can be solved in polynomial time, then all problems in NP can be solved. Cook’s Theorem NP co-“NP-complete” A problem is NP-complete if: P (i) The problem is in NP “NP-complete” co-NP (ii) All other problems in NP polynomially transforms into The world of NP, revisited the above problem. University at Buffalo Department of Industrial Engineering 17 NP-Hard Problems z Definition: (NP-hard) A decision problem whether a member of NP or not, to which we can transform a NP-complete problem is at least as hard as the NP-complete problem. Such a decision problem is called NP-hard. Example:

KTH LARGEST SUBSET A∈{,aa ,… a } : Given a set 12 t ,ba ≤ ∑ j , jA∈ andk ≤ 2,||A do there exist at least K distinct subsets where ′ A ∈{SS12,,… SK } and A′ ⊆ A such that ∑ Sbj ≤ ? jA∈ ′

University at Buffalo Department of Industrial Engineering 18 Six Basic NP-Complete Problems

z 3-SATISFIABILITY: Given a collection C = {c1, c2, …, cm} of clauses on a finite set U of variables such that |ci|=3 for 1≤i≤m, is there a truth assignment for U that satisfies all the clauses in C? z 3-DIMENSIONAL MATCHING: Given a set M ⊆×× WXY , where W, X, and Y are disjoint sets having the same number q of elements, does M contain a matching, i.e., a subset / / / z M ⊆ M such that | M | = q and no two elements of M agree in any coordinate? 1 t a z PARTITION: Given positive integers a1, … , at and b = ∑ j , 2 j=1 do there exist two disjoint subsets S1 and S2 such that ∑ abj = for i= 1, 2 ? jS∈ i University at Buffalo Department of Industrial Engineering 19 …Six Basic Problems (contd.)

z VERTEX COVER: Given a graph G=(V,E) and a positive integer K ≤ |V|, is there a vertex cover of size K or less for G, i.e., a subset VV/ ⊆ such that | V /| ≤ K and, for each edge {,}uv∈ E ,at least one of u and v belongs to V /?

z HAMILTONIAN CIRCUIT: For a graph G = (N, A) with node set N and arc set A, does there exist a circuit (or tour) that connects all the N nodes exactly once?

z CLIQUE: For a graph G = (N, A) with node set N and arc set A, does there exist a clique of size c? i.e., does there exist a setNN* ⊂ , consisting of c nodes such that for each distinct pair of nodes uv, ∈ N ,* the arc {u,v} is an element of A?

University at Buffalo Department of Industrial Engineering 20 Transformation Topology

SATISFIABILITY

3-SAT

3DM VC

PARTITION HC CLIQUE

Diagram of the sequence of transformations used to prove that the six basic problems are NP-complete. Problems of which the complexity is established through a reduction from PARTITION typically have pseudopolynomial time algorithms and are therefore NP-hard in the ordinary sense.

University at Buffalo Department of Industrial Engineering 21 Other Popular Problems

z 3-PARTITION: Given positive integers a1, … , a3t and b with bb 3t … <

University at Buffalo Department of Industrial Engineering 22 Polynomial Time Reductions

Examples and Proofs Dealing with Hard Problems

You: Give up!

Boss: Fires you!

University at Buffalo Department of Industrial Engineering 24 Still Dealing!!

You: Challenge Boss!

Boss: Asks for proof! You: Cannot prove! Boss: Gives you a rise?…..very unlikely!

University at Buffalo Department of Industrial Engineering 25 Better Strategy

You: Prove that the problem is “hard” and that everyone else has failed.

Boss: At least he gets no benefit out of firing you!

University at Buffalo Department of Industrial Engineering 26 Problem Reduction – Example 1 z KNAPSACK PROBLEM KNAPSACK problem is equivalent to the scheduling problem

1|dj=d|∑wjUj. The value d refers to size of the knapsack and the jobs are the items that have to be put into the knapsack.

The size of the item j is pj and the weight (value) of the item j is wj. It can be shown that PARTITION reduces to KNAPSACK by taking ntp==, jjjj aw, = a, 11tt dabzab====∑∑jj, . 22jj==11 It can be verified that there exists a schedule with an objective 1 n value ≤ ∑ wj iff there exists a solution for the PARTITION 2 j=1 problem.

University at Buffalo Department of Industrial Engineering 27 Problem Reduction – Example 2

z MINIMIZE MAKESPAN ON PARALLEL MACHINES (P2||Cmax)

Consider P2||Cmax. It can be shown that PARTITION reduces to this problem by taking ntp= , jjjj== aw, a, 1 t zab==∑ j . 2 j=1 It is trivial to verify that there exists a schedule with an objective 1 n value ≤ ∑ p j iff there exists a solution for the PARTITION 2 j=1 problem.

University at Buffalo Department of Industrial Engineering 28 Problem Reduction – Example 3 z MINIMIZE MAKESPAN IN A JOB SHOP

Consider J2|recrc, prmp|Cmax. It can be shown that 3-PARTITION reduces to J2|recrc, prmp|Cmax by taking the following transform- ation. If the number of jobs be n, take

n= 3t+1, p1j=p2j=aj, for j=1, …3t. Each of these 3t jobs has to be processed on machine 1 and then on machine 2. These 3t jobs do not recirculate. The last job, job 3t+1, has to start its processing on machine 2 and then alternate between machines 1 and 2. It has to be processes in this way t times on machine 2 and t times on machine 1, and each of these 2t processing times = b. For a schedule to have a makespan Cmax=2tb, this last job has to be scheduled without preemption. The remaining slots can be filled without idle times by jobs 1, ..., 3t iff 3-PARTITION has a solution.

University at Buffalo Department of Industrial Engineering 29 Problem Reduction – Example 4 z SEQUENCE-DEPENDENT SETUP TIMES Consider the TRAVELING SALESMAN PROBLEM (TSP) or in

scheduling terms 1|sjk|Cmax problem. That the HAMILTONIAN CIRCUIT (HC) can be reduced to 1|sjk|Cmax can be shown as follows. Let each node in a HC correspond to a city in a TSP. Let the distance between two cities equal 1 if there exists an arc between two corresponding nodes in the HC. Let the distance between two cites be 2 if such an arc does not exist. The bound on the objective is equal to the number of nodes in the HC. It is easy to show that the two problems are equivalent.

University at Buffalo Department of Industrial Engineering 30 Summary Observation z Present research is in the boundary of polynomial time problems and NP-hard problems. z If a problem is NP-complete (or NP-hard), there is no polynomial time algorithm that solves it unless P=NP. (No pseudopolynomial time algorithms for strong NP-complete problems).

University at Buffalo Department of Industrial Engineering 32 Why all these analyses? z Determine the boundary of polynomial time problems and NP- hard problems. z For which decision problems do algorithms exist? z Develop better algorithms in cryptography.

University at Buffalo Department of Industrial Engineering 33 Beyond NP-completeness z Try to prove that P=NP (AMS will give one million dollars). z Randomized Algorithms. z Approximation Algorithms. z Heuristics.

University at Buffalo Department of Industrial Engineering 34