sign in sign up
<<
Home , Approximation algorithm, APX

1 6117CIT - Adv Topics in Computing Sci at Nathan

Algorithms

The intelligence behind the hardware

Outline ! Approximation • The class APX ! Some complexity classes, like PTAS and FPTAS ! Illustration of some PTAS

! Based on • P. Schuurman and G. Woeginger (2001), Approximation Schemes - A Tutorial. • M. Mastrolilli course notes

2 © V. Estivill-Castro

The class APX ! (an abbreviation of "approximable") . ! The set of NP optimization problems that allow polynomial-time approximation algorithms with approximation ratio bounded by a constant (or constant-factor approximation algorithms for short). ! Problems in this class have efficient algorithms that can find an answer within some fixed percentage of the optimal answer. ! An approximation is called a !- for some constant ! if it can be proven that the solution that the algorithm finds is at most ! times worse than the optimal solution. 3 © V. Estivill-Castro

1 !Review from week 2

! The problem and traveling salesman problem with triangle inequality each have simple 2- approximation algorithms. ! The traveling salesman problem with arbitrary edge-lengths can not be approximated with approximation ratio bounded by a constant as long as the Hamiltonian- path problem can not be solved in polynomial time.

4 © V. Estivill-Castro

Alternative view on PTAS ! If there is a polynomial-time algorithm to solve a problem within every fixed percentage (one algorithm for each percentage), then the problem is said to have a polynomial-time approximation scheme (PTAS) • Unless P=NP, it can be shown that there are problems that are in APX but not in PTAS; that is, problems that can be approximated within some constant factor, but not every constant factor.

5 © V. Estivill-Castro

More on APX

! A problem is said to be APX-hard if there is a PTAS from every problem in APX to that problem, ! A problem is APX-complete if the problem is APX-hard and also in APX. ! As a consequence of PTAS ! APX, no APX-hard problem is in PTAS.

6 © V. Estivill-Castro

2 Focus on Optimization Problems

! Notation • We use I for an instance of an optimization problem • We use |I|=n, for the length of the input instance • We use Opt(I) for the value of the optimal solution ! We focus on minimization problems in this lecture, but concepts are symmetric for maximization problems.

7 © V. Estivill-Castro

!- Approximation Algorithm We denote A(I) = solution value.

An algorithm is an “!-Approximation Algorithm” if A(I) " ! Opt(I) for all instances and the running time is polynomial in |I|

! = worst-case approximation ratio ! >= 1 Good: ! close to 1

8 © V. Estivill-Castro

PTAS: Polynomial Time Approximation Scheme

This is a family {A#}#>0 of (1+#)-approximation algorithms with running time polynomial in |I|

•As observed in last lecture, the scheme fits the

definition if the running time is exponential in 1/# :

e.g. O(|I|1/#) NOTE: FPTAS: Fully PTAS running time also polynomial in 1/# : e.g. O(|I|/#3)

9 © V. Estivill-Castro

3 Strongly and Weakly NP-hard

! If a problem is NP-hard even if the input is encoded in unary, then it is called strongly NP-hard = NP-hard in the strong sense = unary NP-hard ! If a problem is polynomially solvable under a unary encoding, then it is solvable in pseudo-polynomial time.

! NP-Hard in the strong sense is contained within NP-Hard in the weak sense

10 © V. Estivill-Castro

Complexity Classes Relationships

NP Pseudo-Poly APX

PTAS FPTAS P

11 © V. Estivill-Castro

Some Known Approximation Algorithms Non-constant worst-case ratio • O(n1/2-e) • Total flow time O(n1/2) • Set covering O(log n) •Vertex cover O(log n) Constant worst-case ratio • TSP with triangle-inequalities 3/2 • Max Sat 1.2987

PTAS • Bin packing

FPTAS • Makespan on 2 machines 12 © V. Estivill-Castro

4 The first Approximation Algorithm (Graham ‘66)

Makespan minimization on identical machines (stronglyN-hard) identical machines

jobs with lengths

Objective, smallest

13 © V. Estivill-Castro

Algorithm: List-Scheduling (LS)

LS: schedule jobs in any given order to the first available (i.e. idle) machine

List:

14 © V. Estivill-Castro

Analysis of LS-Algorithm

! Define the lower bound

• LB=max {max pj; ! pj/m} ! Starting time of the FINAL job

• sf : starting time of the job that completes last ! Observation (result by LS-Algorithm): LS = sf+pf

! Let Ei be the completion (end) time of machine Mi

15 © V. Estivill-Castro

5 LS Analysis (cont) ! LS places the last job in the machine that is mostly available

• sf ! Ei (for all other machines i!f)

• sf = Ef - pf ! This implies (summing for each machine) that

• m sf ! ["i=1 (Ei)] - pf

• sf ! (1/m)(["i=1 Ei]-pf ) =(1/m)([" pj] -pf ) ! But LS • Cmax =sf+pf !(1/m) " pj + pf (1-(1/m)) ! Thus LS • Cmax ! [2- (1/m)]Opt

16 © V. Estivill-Castro

LS: Analysis

Theorem: LS is a (2-1/m)-approximation algorithm.

The approximation ratio is tight.

Example: p1 = p2 = 1 and p3 = 2

M M1 J J 1 J1 1 2 M J M2 J2 J3 2 3 C =2 Cmax=3 max 17 © V. Estivill-Castro

Linear Programming based approximation algorithms

ILP LP IDEA relax

poly time difficult

Opt OptLP

round to integral values A(I) # $ Opt

18 © V. Estivill-Castro

6 Example: R2||Cmax

R2||Cmax : Makespan minimization on 2 unrelated machines (weakly NP-hard) I: 2 unrelated machines n jobs

Job j has length

• p1j on machine M1 • p2j on machine M2

M1

M2

Cmax

19 © V. Estivill-Castro

Integer Linear Program (ILP)

We will encode by xij the fact that job j is placed in machine i Then the ILP looks as follows

• Minimize Cmax • Subject to

• x1j + x2j =1, for j=1,…n (each jobs is assigned once) n • !j=1 p1j x1j " Cmax n • !j=1 p1j x1j " Cmax

• x1j , x2j {0, 1} (it most be on one machine or the other) j=1,…,n

20 © V. Estivill-Castro

Linear Program (LP) Relaxation

We will encode by xij the fact that job j is placed in machine i Then the ILP looks as follows

• Minimize Cmax • Subject to

• x1j + x2j =1, for j=1,…n (each jobs is assigned once) n • !j=1 p1j x1j " Cmax n • !j=1 p1j x1j " Cmax

• x1j , x2j 0 (it most be on some machine or split) j=1,…,n

21 © V. Estivill-Castro

7 Analysis of the number of fractional jobs

! Known: a basic optimal LP solution has the property that the number of variables that get positive values is at most the number of rows in the constraint matrix • Thus, there are at most n+2 variables with positive values

! Since Cmax is always positive, at most n+1 of the xij variables are positive

• We reduce the value Cmax of if we make any pair of variables ofr the same job to zero ! Every job has at least one positive variable associated with it

• Because x1j + x2j =1 ! CONCLUSION: At most 1 (ONE) job has been split onto two machines

22 © V. Estivill-Castro

Rounding

J5: fractional M 1 J1 J4 J5

M2 J2 J3 J6 J5 OptLPOpt

M 1 J1 J4

M2 J2 J3 J6 J5

J5 Opt ROUNDINGOpt 23 © V. Estivill-Castro

How to get a PTAS

Output A(I): Input I Algorithm A feasible sol. for I

Add structure IDEA: Add more structure (depending on ) as , additional structure 0 Compare as

24 © V. Estivill-Castro

8 Structuring the Input

! I I#

difficult poly time

Opt Opt#

back in poly time A(I) " (1+!) Opt

25 © V. Estivill-Castro

ExampleP2||Cmax

P2||Cmax Makespan minimization on 2 identical machines (weakly NP-hard)

I: 2 identical machines n jobs with lengths

p1, p2, …, pn M1

M2 C Lower bound is again max

LB=max {max pj; # pj/2} Thus LB " Opt " 2 LB 26 © V. Estivill-Castro

How to round the input

! I I#

! # pj > ! LB pj := pj “big” $S/ (! LB)% jobs of length ! LB pj " ! LB ! where S= #small pj “small”

27 © V. Estivill-Castro

9 Analysis of Rounded Instance I Recall that ! " 2

•How many big jobs ? • a big job has # •This implies big " 2/# •How many conglomerates jobs ? ow man small os •A conglomerate of small jobs has # •This implies conglomerates " 2/#

LEMMA: The rounded instance has a constant(#) number of jobs. COROLARY: We can find its optimal solution in constant time!! PROOF: Use exhaustive search 28 © V. Estivill-Castro

Back to a feasible solution

1 i i … # # … 2 i i # # #

t

# # # # # small "#

Sum of the small 29 © V. Estivill-Castro

Back to a feasible solution (ctd)

1 i i … # # … 2 i i # # #

t

1 i i … # # … 2 i i # # #

t Cma " t # " 1# t

30 © V. Estivill-Castro

10 How much error is introduced?

# Cmax ! Opt + " LB Cmax ! Opt + 2" LB Opt# ! Opt + " LB ! (1+2") Opt#

Wait till next slide

31 © V. Estivill-Castro

pt vs. pt

(Case 1:LUCKY) M1 big big ... big

# Opt ! Opt M2 big ... "LB "LB

(Case 2:Optimal solution here has M1 big big ... "LB to be as good as LS) "LB M2 big big ...

# LS Opt ! Cmax =sf+pf !(1/m) # pj + pf (1-(1/m)) Since m=2, and (1/m) # pj ! Opt # and pf /2 ! "LB, we have Opt ! Opt + "LB 32 © V. Estivill-Castro

Structuring the execution of an algorithm

IDEA: take an exact but slow algorithm A and interact with it while it is working. Clean- part of its memory. As a result the algorithm becomes faster (less data to process) and generates incorrect output.

IDEAL CASE: the becomes polynomial and the incorrect output is a good approximation

Compare: vs. branch & bound

33 © V. Estivill-Castro

11 Example: R2||Cmax • Exact Algorithm (Dynamic Program)

•Encode a partial schedule for the first k jobs by a vector (state) [!, "]

• ! = total processing time on M1 • " = total processing time on M2

• Assign job one by one and update the states appropriately

34 © V. Estivill-Castro

Dynamic Program

1. S0 #{ [0,0] } 2. For k # 1 to n do

3. Sk # {} 4. For all [!, "] in Sk-1 do 5. add the two states

6. [! + p1k, "] and 7. [!, " + p2k] to Sk 8. End For

9. End For

10. Output min{ max{!, "} | [!, "] in Sn }

35 © V. Estivill-Castro

(Perturbed) Dynamic Program

1. S0 #{ [0,0] } 2. For k # 1 to n do

3. Sk # {} 4. For all [!, "] in Sk-1 do 5. add the two states

6. [! + p1k, "] and 7. [!, " + p2k] to Sk 8. End For

9. CLEAN UP Sk 10.End For

11.Output min{ max{!, "} | [!, "] in Sn }

36 © V. Estivill-Castro

12 State Space Sk

i i+1 i+2 i+3 i+4

P := j max{p1j, p2j} State space in PxP box

:= (1+)1/n , 2, 3,...

37 © V. Estivill-Castro

Clean up Sk

i i+1 i+2 i+3 i+4

P := j max{p1j, p2j} State space in PxP box

:= (1+)1/n , 2, 3,...

38 © V. Estivill-Castro

Running Time

The running time is n times the number of boxes. The number of boxes is at most (b+1)2 where b = P Therefore b=log(P) / log() Since = (1+)1/n then b n log(P) /log(1+) n log(P) (1+)/

Polynomial on the size of the unary encoding 39 © V. Estivill-Castro

13 How much error is introduced by “clean up”?

Clean up “replaces” state !" by another state !" in the same box: !/# $ ! $ !# "/# $ " $ "# Therefore, (derivations can show) error # There are n “clean up” phases, and each clean up phase has error #

Overall error $ #n = 1%

40 © V. Estivill-Castro

41

In-Approximability Techniques

e ipe he

MAIeN IDiEA: ence a T Prove that the existence of a polynomial time approximation algorithm for problem p with worst case guarantee better than ! implies the existence of an exact polynomial time algorithm for an NP-hard problem q (which implies P=NP)

42 © V. Estivill-Castro

14 Other general techniques

For FPTAS: • Strong NP-hardness • implies no FPTAS

For PTAS: • Gap-technique ==> • implies no PTAS

• MAX-SNP-hardness • Implies no PTAS

43 © V. Estivill-Castro

Example to disproof existence of an FPTAS (unless P=NP)

Makespan minimization on identical machines

identical machines jobs with lengths

PROOF: Assume are integers encoded in unary (e.g., write:

• Known: is NP-hard even under unary encoding

! polynomial in size of input " and is an integer in

44 © V. Estivill-Castro

Continuation (no FPTAS)

• • ( + )

" # / " / This gives • implies • implies

45 © V. Estivill-Castro

15 The “gap technique”

IDEA: Let f be a minimization with integral objective function values (costs). Let g be a fixed integer. Assume that the problem of deciding whether an instance of p has a feasible solution with cost at most g is NP-hard. Then p does not have an approximation algorithm with ratio ! < (g + 1)/g, unless P=NP.

46 © V. Estivill-Castro

The Gap Technique (ctd)

hard to decide if the cost is

possible costs of p REDUCTION: Assume for any instance it exists an a-approximation algorithm with . • If , then If >= , then >= 47 © V. Estivill-Castro

Example

Problem (bin packing or scheduling with deadline)

Input: jobs with lengths , 2, , and a hard deadline

Goal: find the minimum number of machines on which all jobs can be completed before their deadline .

48 © V. Estivill-Castro

16 Example (ctd)

THEOREM. Deciding if is -hard PROOF: (use partition).

Gap Technique

For the no -approximation algorithm is possible with (that is, no PTAS!), unless

49 © V. Estivill-Castro

50 © V. Estivill-Castro

17