Load Balancing Load Balancing: Example

Problem Example Input: m identical machines, n jobs with ith job having processing Consider 6 jobs whose processing times is given as follows time ti Goal: Schedule jobs to computers such that Jobs 1 2 3 4 5 6 ti 2 3 4 6 2 2 I Jobs run contiguously on a machine I A machine processes only one job a time Consider the following schedule on 3 machines I Makespan or maximum load on any machine is minimized 3 6 2 5 Definition 1 4 Let A(i) be the set of jobs assigned to machine i. The load on i is T = t . i j A(i) j The loads are: T = 8, T = 5, and T = 6. So makespan of The makespan∈ of A is T = max T 1 2 3 P i i schedule is 8

Load Balancing is NP-Complete Greedy Algorithm

1. Consider the jobs in some fixed order 2. Assign job j to the machine with lowest load so far

Decision version: given n, m and t1, t2,..., tn and a target T , is Example there a schedule with makespan at most T ? Jobs 1 2 3 4 5 6 Problem is NP-Complete: reduce from Subset Sum. ti 2 3 4 6 2 2

3 6 2 5 1 4 Putting it together Optimality

for each machine i Ti = 0 (* initially no load *) A(i) = (* initially no jobs *) ∅ for each job j Problem Let i be machine with smallest load Is the greedy algorithm optimal? No! For example, on A(i) = A(i) j (* schedule j on i *) ∪ { } Ti = Ti + tj (* compute new load *) Jobs 1 2 3 4 5 6 ti 2 3 4 6 2 2

Running Time the greedy algorithm gives schedule with makespan 8, but optimal is 7 I First loop takes O(m) time In fact, the load balancing problem is NP-complete. I Second loop has O(n) iterations

I Body of loop takes O(log m) time using a priority heap

I Total time is O(n log m + m)

Quality of Solution Bounding the Optimal Value

Lemma T max t , where T is the optimal makespan Theorem (Graham 1966) ∗ ≥ j j ∗ The makespan of the schedule output by the greedy algorithm is at Proof. most 2 times the optimal make span. In other words, the greedy Some machine will run the job with maximum processing time algorithm is a 2-approximation. Lemma Challenge T 1 t , where T is the optimal makespan ∗ ≥ m j j ∗ How do we compare the output of the greedy algorithm with the Proof. P optimal? How do we get the value of the optimal solution?

I We will obtain bounds on the optimal value I Total processing time is j tj 1 I Some machine must do atP least m (or average) of the total work Analysis of Greedy Algorithm Tightness of Analysis

Theorem Proposition The greedy algorithm is a 2-approximation The analysis of the greedy algorithm is tight, i.e., there is an example on which the greedy schedule has twice the optimal Proof. makespan Let machine i have the maximum load Ti , and let j be the last job scheduled on machine i Proof. Consider problem of m(m 1) jobs with processing time 1 and the I At the time j was scheduled, machine i must have had the − last job with processing time m. least load; load on i before assigning job j is T t i − j Greedy schedule: distribute first the m(m 1) jobs equally among I Since i has the least load, we know T t T , for all k. − i − j ≤ k the m machines, and then schedule the last job on machine 1, Thus, m(T t ) T i − j ≤ k k giving a makespan of (m 1) + m = 2m 1 1 1 − − I But Tk = t`. So Ti tj Tk = t` T ∗ The optimal schedule: run last job on machine 1, and then k ` P − ≤ m k m ` ≤ I Finally, T = (T t ) + t T + T = 2T distribute remaining jobs equally among m 1 machines, giving a P i Pi j j ∗ P∗ ∗ P − − ≤ makespan of m

Improved Greedy Algorithm Technical Lemma

Modified Greedy Lemma Sort the jobs in descending order of processing time, and process If there are more than m jobs then T ∗ 2tm+1 jobs using greedy algorithm ≥ Proof. for each machine i Consider the first m + 1 jobs Ti = 0 (* initially no load *) A(i) = (* initially no jobs *) I Two of them must be scheduled on same machine, by ∅ for each job j in descending order of processing time pigeon-hole principle Let i be machine with smallest load A(i) = A(i) j (* schedule j on i *) I Both jobs have processing time at least tm+1, since we ∪ { } Ti = Ti + tj (* compute new load *) consider jobs according to processing time, and this proves the lemma Analysis of Modified Greedy Tightness of Analysis

Theorem The modified greedy algorithm is a 3/2-approximation

Proof. Theorem (Graham) Once again let i be the machine with highest load, and let j be the Modified greedy is a 4/3-approximation last job scheduled The 4/3-analysis is tight. I If machine i has only one job then schedule is optimal

I If i has at least 2 jobs, then it must be the case that 1 j m + 1. This means tj tm+1 2 T ∗ ≥ ≤ ≤1 I Thus, T = (T t ) + t T + T i i − j j ≤ ∗ 2 ∗

(Weighted) Set Cover Problem Greedy Rule

Input Given a set U of n elements, a collection S1, S2,... Sm of subsets of U, with weights wi Goal Find a collection of these sets S whose union is C i I Pick the next set in the cover to be the one that makes “most equal to U and such that i wi is minimized. progress” towards the goal ∈C P I Covers many (uncovered) elements Example I Has a small weight Let U = 1, 2, 3, 4, 5, 6, 7, 8 , with { } I If R is the set of elements that aren’t covered as yet, add set wi Si to the cover, if it minimizes the quantity S = 1 w = 1 S = 2 w = 1 Si R 1 { } 1 2 { } 2 | ∩ | S = 3, 4 w = 1 S = 5, 6, 7, 8 w = 1 3 { } 3 4 { } 4 S = 1, 3, 5, 7 w = 1 +  S = 2, 4, 6, 8 w = 1 +  5 { } 5 6 { } 6 S , S is a set cover of weight 2 + 2 { 5 6} Greedy Algorithm Example: Greedy Algorithm

Initially R = U and C = 1 +  1 +  ∅ 1 1 while R = 6 ∅ let Si be the set that minimizes wi / Si R Example | ∩ | C = C i Let U = 1, 2, 3, 4, 5, 6, 7, 8 , with ∪ { } 1 R = R Si { } \ return C S = 1 S = 2 1 { } 2 { } S = 3, 4 S = 5, 6, 7, 8 3 { } 4 { } Running Time 1 S = 1, 3, 5, 7 S = 2, 4, 6, 8 5 { } 6 { } I Main loop iterates for O(n) time, where U = n | | w1 = w2 = w3 = w4 = 1 and w5 = w6 = 1 +  I Minimum Si can be found in O(log m) time, using a priority Greedy Algorithm first picks S4, then S3, and heap, where there are m sets in set cover instance finally S1 and S2 I Total time is O(n log m)

Cost for covering an element Cost of element: Example

Example Let U = 1, 2, 3, 4, 5, 6, 7, 8 , with Definition { } 1 2 k I Suppose the greedy algorithm selects sets S , S ,... S (in S = 1 S = 2 1 { } 2 { } that order) to form the set cover. S = 3, 4 S = 5, 6, 7, 8 3 { } 4 { } i S = 1, 3, 5, 7 S = 2, 4, 6, 8 I Consider an element s that is first covered when S is picked 5 { } 6 { } i I Let R be the set of elements that are uncovered when S is w1 = w2 = w3 = w4 = 1 and w5 = w6 = 1 + . The greedy picked algorithm picks S4, S3, S2, S1 in that order. The costs of elements i i i I The cost of covering s is c = w(S )/ S R , where w(S ) is are given as follows s | ∩ | weight of the set Si c1 = 1 c2 = 1 c3 = 1/2 c4 = 1/2 c5 = 1/4 c6 = 1/4 c7 = 1/4 c8 = 1/4 Costs of Covers and Elements Bounding costs of sets

Lemma For every set Sk , s S cs H( Sk ) wk , where ∈ k ≤ | | · H(n) = n 1 = Θ(ln n) Proposition i=1 i P If is the set cover computed by the greedy algorithm then C Proof. P i wi = s U cs ∈C ∈ Let Sk = s1,... sd , where si is covered before sj if i j Proof left as exercise { } ≤ P P I Consider the iteration when sj is covered; at this time Main Idea R sj , sj+1,... sd s S cs ⊇ { } Upper bound the ratio ∈ k , i.e., to say “to cover a lot of cost, wk I Average progress cost of Sk is wk / Sk R wk /(d j + 1) you need to use a lot ofP weight” | ∩ | ≤ − I Suppose sj gets covered because Si is selected by greedy wi wk wk algorithm. Then, csj = S R S R d j+1 | i ∩ | ≤ | k ∩ | ≤ − d d w I Hence, c = c i = H(d)w s Sk s j=1 sj j=1 d j+1 k ∈ ≤ − P P P

Analysis of the Greedy Algorithm Tightness of Analysis

Theorem The greedy algorithm for set cover is a H(d∗)-approximation, where d ∗ = maxi Si Analysis Tight? | | Proof. Does the Greedy Algorithm give better approximation guarantees? No! Let ∗ be the optimal set cover, and the set cover computed by C C Consider a generalization of the set cover example. Each column greedy algorithm k 1 1 has 2 − elements, and there are two sets consisting of a column I By previous lemma we know wi cs and so we H(d∗) s Si i each with weight 1 + . Additionally there are log n sets of ≥1 ∈ have w ∗ = i wi i H(d ) s S csi increasing size of weight 1. The greedy algorithm will pick these ∈C∗ ≥ ∈C∗ ∗ ∈Pi log n sets given weight log n, while the best cover has weight 2 + 2 I Further, iP s S csP s U cs P ∈C∗ ∈ i ≥ ∈ 1 I Thus, w ∗ = wi cs P iP∗ iP ∗ H(d∗) s Si i 1 ∈C 1 ≥ ∈C ∈ ≥ H(d ) s U cs = H(d ) i wi ∗ ∈ P ∗ P∈C P P P Best Algorithm for Set Cover

Theorem If P = NP then no polynomial time algorithm can achieve a better 6 than H(n) approximation. Proof beyond the scope of this course.