Arxiv:Cs/0205036V1 [Cs.DS] 18 May 2002 Introduction 1 Games

arXiv:cs/0205036v1 [cs.DS] 18 May 2002 Introduction 1 games. algo- matrix for parallel strategies a sparse find including to problems, rithm general obliv- covering solve give and We approximately problems. packing that of algorithms class rounding new bottle- a ious this on Avoiding bear probabilist to brings methods and program. algorithms efficient linear more yields the neck solving bottleneck the first avoids — that of rounding rounding randomized oblivious of called variant technique a new a introduce We Abstract onigyed h etapoiainkonb any by [3]. known tech- algorithm randomized approximation time basic polynomial best problems, few the NP-hard a yields many just rounding for on Yet relying of simple, niques. method analy- often the probabilistic are by The ses derandomized [17]. be probabilities in can conditional algorithm; it randomized a cases yields most This non-zero op- with the solution. approximates that, timal solution shows rounded One depending the probability probability, part. with fractional down the or coordinate on up each solution rounds the randomly of resulting and the dis- program, solves program, linear constraints, linear integrality of integer the design an regards formulates as the problem one for NP-hard Typically, 1] an [20, method algorithms. probabilistic approximation a is [18] rbblte sue o odrnoieprs,btto conditional but se, of per method derandomize to The not used solution. is probabilities optimal the essentially are of that pes- ap- [17] finding estimators (2) probabilities, simistic conditional solution; of approximate method the randomly an ply that yields smaller-than- show units) to (1) integer (possibly solution some- follows: optimal an can as rounding bottleneck avoided this that be solu- show a times We find to round. program to linear a tion solve first has they algorithms that rounding been randomized of drawback main a il J094 ato hsrsac a oewieat while done sup- was and research 14853 this NY Ithaca of University, by Part Cornell ported ORIE, 07974. of School NJ Hill, ray [email protected] ∗ TTBl as m D15 0 onanAe,Mur- Ave., Mountain 600 2D-145, rm. Labs, Bell AT&T admzdRounding: Randomized biiu Rounding: Oblivious v ads S Y rn D-179.E-mail: DDM-9157199. grant PYI NSF Tardos’ Eva ´ admzdRudn ihu ovn h ierProgram Linear the Solving without Rounding Randomized . ()SxhAMSA ypsu nDsrt loihs(SODA Algorithms Discrete on Symposium ACM-SIAM Sixth ((c) eadmzdo o,a not, or Derandomized admzdrounding Randomized independent elE Young E. Neal ic 1 civ h independence. the achieve fteeg and edge the of h egho neg siiily1adi utpidby multiplied is and 1 initially is where edge that path, 1+ an shortest of algorithms a length yields along the flow was it augment works). it related repeatedly flow, and times [4] multicommodity of to spirit For number in similar the assigned is (this to is chosen set proportional each cover, weight final the a obtain To chosen. is ieysuid(see studied sively strategies games. near-optimal zero-sum sparse, for finding for algorithm lel e egt hr h egto neeeti ntal 1 initially 1 is by element multiplied an is of and weight the largest where the weight, have elements net the whose For set a chooses 14]. repeatedly [10, algorithm cover fractional set rounding greedy oblivious the problem, cover yields set (integer) the For some width. rela- for large effective also of as not problems are are they algorithms although simple, Our tively a relaxation. shows and techniques approach Lagrangian probabilistic Our between connection obtain- rounding. strong those randomized to by can dual comparable able they produce solutions (5) they integer and provide (4) near-optimality, sub- prove it, approximate) that over (possibly solutions a optimizing by for given routine be cov- to or set packing the ering Lagrangian- allow they they are (3) (1) they algorithms, (2) relaxation features: width, the using following on the similarly problems depend share ours algo- these and Their rithms approached directly. techniques [16] Lagrangian-relaxation Tardos and loihsadmksi plcbefrinteger for applicable efficient allows it more makes This and and simpler algorithms give solution. to optimal rounding the randomized with- computing solution first approximate out the find algorithms sulting oeigpolm orsodn oitgradnon- and small integer of programs to linear integer corresponding and packing we problems general this, covering for demonstrate algorithms To approximation give programming. linear integer ǫc/c eeaie akn n Covering: and Packing Generalized akn n oeigpolm aebe exten- been have problems covering and Packing lvro biiu onigAlgorithms: Rounding Oblivious of Flavor ( ∗ e ahtm h dei sd( used is edge the time each ) e oe rbe,i ilsa loih that algorithm an yields it problem, cover set c stemnmmeg capacity). edge minimum the is § ) o xml,Pokn Shmoys, Plotkin, example, For 2). − ǫ ahtm e otiigit containing set a time each 95)) width nldn paral- a including , c ( e stecapacity the is ) and h re- The non- 2 Young

ω2 ln(m) Problem Definitions: Let P be a convex set in generalized packing: IRn and let f be a linear function (not nec. homogenous) 2ǫ2 m from P to IR . The width of P with respect to f is (1 + ǫ)ω ln(m) packing: ω = maxj,x fj(x) L, where L = minj,x fj (x). λ∗b(ǫ) The generalized− packing problem is to compute λ∗ = ω ln(m) minx∈P maxj fj(x). The packing problem occurs when covering: f is non-negative on P . The covering problem is to λ∗b( ǫ) ∗ − compute λ = maxx∈P minj fj (x), assuming f is non- b(ǫ) := (1 + ǫ) ln(1 + ǫ) ǫ; 2 2− negative. (This is equivalent to the generalized packing b( ǫ) > ǫ >b(ǫ) > 2ǫ . problem with the restriction that f is non-positive.) − 2 4.2+ǫ Our algorithms assume an optimization oracle for P and f — given non-negative y R , the oracle Table 1: Number of iterations. Each iteration requires ∈ m returns x and f(x), where x minimizes j yjfj (x). O(log m) time and O(m) operations (on an EREW- (This models, e.g., packing source-sink pathsP subject to PRAM), plus one oracle call. For an explicitly given edge constraints; in this case the oracle would compute problem (no oracle), each iteration requires O(log nm) a shortest path for given non-negative edge lengths.) time and O(nm) operations. For covering, the oracle must maximize the sum. Quality of Solutions: Given the oracle, n, m, ω, L, and ǫ > 0, our algorithms return ǫ-approximate Each iteration chooses the best pure strategy given solutions. For generalized packing, ǫ is the additive that the opponent plays the mixed strategy represented ∗ by y. The final solution returned is a mixed strategy error with respect to λ . For packing and covering, the 2 ω ln m error is a factor of 1 ǫ. that plays uniformly from 2ǫ2 pure strategies, one Complexity: Table± 1 shows the number of itera- for each iteration. (The opponent has m pure strategies; tions required and the complexity per iteration. In that ω is the maximum minus the minimum payoff.) The caption, “explicitly given” means that f(x) = Ax + b, existence of such sparse, near-optimal strategies was where A and b are, respectively, an explicitly given ma- shown probabilistically [2, 13]; our existence proof of trix and vector, while P = x IRn : x 0; x =1 . the approximate solution for generalized packing is a { ∈ ≥ i } Granularity: The oracle is called onceP in each generalization of the proof in [13]. iteration of the algorithm; the algorithm returns the average of the solutions returned by the oracle. Thus, 2 Related Work the granularity of the final solution is the granularity Plotkin, Shmoys, and Tardos [16] (generalizing a series of the solutions returned by the oracle, divided by the of works on multicommodity flow [19, 11, 12]) gave ap- number of iterations. For the abstract problems we proximation algorithms for general packing and covering consider, this can provide integer solutions comparable problems similar to those we consider. For these ab- to those obtainable by other techniques. stract problems, their results are comparable to those Dual Solutions: Our algorithms maintain a dual in this paper, but for many problems their results are solution, represented by a vector y, initially uniform. stronger. Most importantly, they give techniques for re- In each iteration, each yj is multiplied by a factor ducing the effective width of a linear program and tech- depending on fj (x) where x is the solution returned niques for problems (such as concurrent multicommod- by the oracle (e.g., for packing, yj is multiplied by ity flow) when the packing or covering set is a Cartesian 1+ ǫfj(x)/ω). The average over all iterations of the product. values of these dual solutions is ǫ-optimal with respect Luby and Nisan [15] give a parallel approximation to the value of the final (primal) solution. algorithm for positive linear programming — the special Sparse Strategies for Zero-Sum Games: The cases of linear programming of the form maxx c x : explicitly given general packing problem generalizes the Ax b; x 0 (a packing problem), or the{ · dual ≤ ≥ T } problem of finding near-optimal strategies for zero-sum miny b y : A y c; y 0 (a covering problem), matrix games: P is the set of mixed strategies for one where{ A·, b, and c have≥ non-negative≥ } coefficients. Here player, fj(x) is the expected payoff if the player plays A, b, and c are explicitly given. according to x and the opponent plays the pure strategy Previous algorithms applicable to zero-sum games j, and λ∗ is the value of the game. An approximate either required the solution of a linear program [8] or solution is a mixed strategy x guaranteeing an expected did not provide sparse strategies [5, 6, 15]. payoff within an additive ǫ of optimal. Randomized Rounding without Solving the Linear Program 3

3 Introductory Example: Set Cover Achieving Obliviousness: Because an uncovered ∗ To introduce oblivious rounding, we give a simple element that occurs in several sets in C contributes less example. The set cover problem is the following: to Φ, the above algorithm depends on the number of times each element is covered by C∗. This is counter- given a family of sets = S1,...,Sm , with each F { } intuitive, in that the only aspect of C∗ used in the Si 1, 2,...,n , a set cover C is a sub-family such ⊆ { } proof was x∗/ C∗ 1 1/ C∗ . Replacing each that every element j = 1,...,n is in some set in C. Si6∋j i | |≤ − | | The problem is to find a cover C that is not much correspondingP term in Φ yields ∗ larger than C , a minimum-cardinality cover. We derive s−|C| ∗ 1 an algorithm that, without knowing C , emulates a (3.2) Φ(˜ C)= 1 . random experiment that draws sets randomly from C∗. − C∗ jX6∈∪C | | The algorithm finds a cover of size at most C∗ ln n . ⌈| | ⌉ Φ˜ is a pessimistic estimator. More importantly, among 3.1 Existence: Let s = C∗ ln n . Consider draw- collections of sets of the same size, Φ˜ is uniformly pro- ⌈| | ⌉ ing s sets uniformly at random from C∗. What is the portional to the number of uncovered elements in the expected number of elements left uncovered? For any set. Thus, the algorithm that uses Φ˜ instead of Φ given element x X, the probability that it is not does not depend on C∗, it simply chooses each set ∈ covered in a given round is at most 1 1/ C∗ , be- to minimize the number of elements remaining uncov- − | | cause it is in at least one set in C∗. Thus the ex- ered. Nonetheless, it is guaranteed to keep up with the pected number of elements left uncovered is at most random experiment, finding a cover within C∗ ln n n(1 1/ C∗ )s

4 Algorithm for Generalized Packing The proof of Lemma (4.1) bounds the probability of Fix an instance (P,f,L,ω,ǫ) of the generalized packing failure by a sum of probabilities, each of which is problem. We consider randomly rounding an optimal bounded by an expected value (4.3) in Hoeﬀding’s proof. solution to obtain an ǫ-approximate solution; we then Thus (when L = 0 and ω = 1), the proof bounds the derive the algorithm that ﬁnds such a solution. probability of failure by the expected value of

∗ ∗ 1+ αfj (x) 4.1 Existence: Let λ and x be an optimal solution. ∗ , (1 + α)λ +ǫ Let S be a multiset obtained by repeatedly choosing Xj xY∈S random elements of P , where each random element is chosen from a distribution over P with n-dimensional the expectation of which is less than m/ exp(2 S ǫ2). | | mean x∗. Letx ¯ be the average of the points in S. The conditional expectation of the sum given T S is ⊆ Lemma 4.1. The probability that x¯ is not an ǫ- s−|T | 2 ∗ 2|S|ǫ 1+ αfj (x) 1+ αfj (x )] approximate solution is less than m/ exp 2 . ∗ ∗ ω (1 + α)λ +ǫ · (1 + α)λ +ǫ Xj xY∈T Proof. Without loss of generality, assume L = 0 and ω = 1. Otherwise take f(x) f(x)−L and ǫ ǫ/ω. where s is the desired size of S. To obtain the ← ω ← The convexity of P ensures thatx ¯ P . For pessimistic estimator for the algorithm, replace each ∈ ∗ ∗ each j, fj(¯x) = fj (x)/ S , which is the average f (x ) by the upper bound λ : x∈S | | j of S independentP random variables in [0, 1]. Since | | ∗ ∗ ∗ s−|T | E[fj(x)] = fj (x ) λ , by Hoeffding’s bound [7], 1+ αfj (x) 1+ αλ ∗ ≤ 2 ∗ ∗ Pr[fj (¯x) λ + ǫ] is less than 1/ exp(2 S ǫ ). Since (1 + α)λ +ǫ · (1 + α)λ +ǫ j ranges from≥ 1 to m, the result follows. | | Xj xY∈T ⋄ When s is large enough that m/ exp(2 S ǫ2) 1, this 4.2 Construction: As in the set cover example, quantity is a pessimistic estimator: (a)| it| is≤ an upper our algorithm mimics the random experiment. Each bound on the conditional probability of failure, (b) it round it adds an element to S to minimize a pessimistic is initially less than 1, and (c) some x can always be estimator. This pessimistic estimator is implicit in the added to S without increasing it. Properties (a) and existence proof. To find it, we need the inequalities that (b) follow from the choice of s and the inequalities in prove (a simplified version of) Hoeffding’s bound: the proof of Hoeffding’s lemma. Property (c) follows Lemma 4.2. ([7]) Let X = Xi be the sum of s from the derivation, as explained for the set cover ex- independent random variables inP[0, 1], with E(Xi) µi ample. Among multisets of a given size, this pessimistic 2≤ and µi = µ. Then Pr[X µ + sǫ] < 1/ exp(2sǫ ). estimator is uniformly proportional to ≥ Proof.P Let α = e4ǫ 1. − 1+ αfj (x). Pr Xi µ + sǫ Xj xY∈T h X ≥ i (1 + α)Xi Thus, to augment a given multiset T , the algorithm = Pr µ +ǫ 1 (1 + α) i ≥ adds the element x minimizing yjfj (x), where yj = Yi j x∈T 1+αfj(x). This, accountingP for the normalization 1+ αXi (4.3) E LQ= 0 and ω = 1, is the algorithm in Figure 1. ≤ (1 + α)µi+ǫ Yi 1+ αE(X ) 5 Packing and Covering Algorithms = i (1 + α)µi +ǫ We derive the packing algorithm analogously. Fix an Yi instance (P,f,ω,ǫ) of the packing problem. Let λ∗, 1+ αµi x∗, S andx ¯ be as for Lemma 4.1. Note that, for this ≤ (1 + α)µi+ǫ Yi problem, an ǫ-approximate solution is an x P with 2 ∈ 2ǫ f(x) (1 + ǫ)λ∗. < 1/e . ≤ Yi 5.1 Existence: The second step follows from (1 + α)z 1 + αz for 0 z 1 and Markov’s inequality. The≤ last step uses 2 Lemma 5.1. The probability that x¯ is not an ǫ- ≤ ≤ z+ǫ 2ǫ 4ǫ ∗ 1+ αz < (1 + α) /e for ǫ > 0, α = e 1, and |S|b(ǫ)λ approximate solution is less than m/ exp . z 0. − ω ≥ ⋄ h i Randomized Rounding without Solving the Linear Program 5

Find-Generalized-Packing(P,f,L,ω,ǫ) Find-Packing-Given-s (P,f,ǫ,ω,s) ǫ 4ǫ ln m 1. ǫ ; α e 1; S ; s 2 1. S ω 2ǫ ← {} 2. y ← 1 (j←=1,...,m− ) ← {} ← 2. y 1 (j =1,...,m) j j ← 3. repeat← 3. repeat 4. choose x P to minimize y f (x) 4. choose x P to minimize yjfj (x) ∈ j j j ∈ j 5. S S x P 5. S S x P ← ∪{ } fj (x)−L ← ∪{ } fj (x) 6. y y 1+ α (j =1,...,m) 6. yj yj 1+ ǫ (j =1,...,m) j ← j · ω ← · ω 7. until S s 7. until S s | |≥1 return| |≥1 8. return |S| x∈S x 8. |S| x∈S x P P Figure 1: Algorithm for generalized packing Figure 2: Algorithm for packing, given s. To obtain covering algorithm, negate ǫ and change “minimize” to “maximize”. Proof. Without loss of generality, assume ω = 1. Oth- erwise take f(x) f(x)/ω and λ∗ λ∗/ω. The convexity← of P ensures that← x ¯ P . For each corresponding to (5.4). The expectation given T S is ∈ ⊆ j, fj (¯x) = x∈S fj (x)/ S , which is the average of ∗ s−|T | | | 1+ ǫfj(x) 1+ ǫfj(x ) S independentP random variables in [0, 1], each with ∗ ∗ , | | ∗ ∗ (1 + ǫ)(1+ǫ)λ · (1 + ǫ)(1+ǫ)λ expectation fj (x ) λ . By Raghavan’s bound [17], Xj xY∈T ∗ ≤ ∗ Pr[fj(¯x) (1+ǫ)λ ] is less than 1/ exp[ S b(ǫ)λ ]. Since j ranges≥ from 1 to m, the result follows.| | where s is the desired size of S. When s is large enough ⋄ that m/ exp[ S b(ǫ)λ∗] 1, replacing f (x∗) by λ∗ gives | | ≤ j 5.2 Construction: Here is Raghavan’s proof: a pessimistic estimator. Among multisets T of the same size, the pessimistic estimator is proportional to Lemma 5.2. ([17]) Let X = Xi be the sum of 1+ ǫf (x). independent random variables inP[0, 1] with E(Xi) µi j ≤ Xj xY∈T and µi = µ> 0. ThenP Pr[X (1 + ǫ)µ] < 1/ exp[b(ǫ)µ]. ≥ Thus, to augment a given multiset T , the algorithm adds the element x minimizing y f (x), where y = Proof. j j j j x∈T 1+ǫfj(x). This, accountingP for the normalization to the case ω = 1, gives the algorithm in Figure 2. Pr[X (1 + ǫ)µ] Q ≥ This algorithm assumes s is given. We remove this (1 + ǫ)Xi = Pr 1 requirement in Section 6. (1 + ǫ)(1+ǫ)µi ≥ Yi 5.3 Covering Algorithm. The covering algorithm 1+ ǫXi (5.4) E is described in Figure 2. Its derivation is analogous ≤ (1 + ǫ)(1+ǫ)µi Yi to that of the packing algorithm. Fix an instance 1+ ǫ E(Xi) (P,f,ω,ǫ) of the approximate covering problem. Let = ∗ ∗ (1 + ǫ)(1+ǫ)µi λ , x , S andx ¯ be as for Lemma 4.1. Note that, for Yi this problem, λ∗ = min f (x∗) and an ǫ-approximate eǫµi j j < solution x P satisﬁes f(x) (1 ǫ)λ∗. (1 + ǫ)(1+ǫ)µi ∈ ≥ − Yi Lemma 5.3. The probability that x¯ is not an ǫ- The last line equals 1/ exp[b(ǫ)µ]. The second step uses approximate solution is less than m/ exp[ S b(ǫ)λ∗/ω]. | | (1+α)z 1+αz for 0 z 1 and Markov’s inequality. The last≤ uses (X ) ≤µ and≤ 1+ z ez, which is strict E i i We omit the proof, which is essentially the same as for if z = 0. ≤ ≤ 6 ⋄ packing, except it is based on the following variant of Raghavan’s bound: Thus (assuming ω = 1), the proof of Lemma 5.1 bounds the probability of failure by the expectation of Lemma 5.4. ([17]) Let X = Xi be the sum of independent random variables inP[0, 1] with E(Xi) µi 1+ ǫfj(x) ≥ and µi = µ> 0. (1+ǫ)λ∗ (1 + ǫ) ThenP Pr[X (1 ǫ)µ] < 1/ exp[b( ǫ)µ]. Xj xY∈S ≤ − − 6 Young

We omit the derivation of the algorithm, noting only The maximum value is at least the arithmetic mean, that the proof of Lemma 5.3 implicitly bounds the which is at least the harmonic mean, so at least one of k−1 probability of failure by the expectation of these simple dual solutions has value above ln n .

1 ǫfj(x) − ∗ . 6.2 Generalized Packing Dual: The vector y (1 ǫ)(1−ǫ)λ Xj xY∈S − maintained by the generalized packing algorithm represents a dual solution. At the start of a given iteration, 6 Dual Solutions the value of the dual solution associated with y is

Our algorithms implicitly find good approximate solu- minx∈P yjfj (x) (6.5) j . tions to the underlying dual linear programs. The ar- Py gument that the algorithm “keeps up” with the random j j P rounding of an unknown optimal solution implicitly uses (Since y 0, a simple argument shows this is a lower ≥ ∗ a dual solution to bound the optimal at each iteration. bound on λ = minx∈P maxj fj (x).) The value of the solution generated by the algorithm Notation: During the course of the algorithm, let thus converges not only to the value of the optimum, x¯ denote the current solution x∈S x/ S represented ¯ | | but also to the average of the values of these dual solu- by S. Let λ denote maxj fj (¯x).P Let¯v denote the average tions. The basic principle in each case is similar to that of the values of the dual solutions for the previous for set cover, which we give first for illustration. iterations. Let v(x, y) denote j yj fj (x)/ j yj . Lemma 6.2. The generalizedP packing algorithmP main- 6.1 Set Cover Dual: The dual problem is to assign tains the invariant non-negative weights to the elements so that the net ¯ weight assigned to the elements in any set is at most (1 + α)|S|(λ−L)/ω (1 + α)|S|((¯v−L)/ω+ǫ)m/ exp(2 S ǫ2). one. The value of the dual solution is the net weight ≤ | | assigned. At the start of a given iteration, suppose r elements Proof. WLOG, assume L = 0 and ω = 1. We show remain uncovered, and let d denote the largest number that j yj is at least the left-hand side and at most the in any set in . Then assigning each uncovered element right-hand side. The first part follows from the same F P a weight of 1/d yields a dual solution of value v = r/d. sequence of inequalities that was used in 4.2 to derive During the course of the algorithm, letv ¯ denote the the (numerator of the) pessimistic estimator:§ harmonic mean of the dual solutions corresponding to |S|λ¯ |S|f (¯x) the iterations so far. (1 + α) (1 + α) j ≤ Xj Lemma 6.1. The set cover algorithm maintains the = (1 + α)fj (x) invariant that the number of elements not covered by Xj xY∈S the current partial cover C is less than n/ exp( C /v¯). | | 1+ αfj (x). ˜ ≤ The proof is essentially the same as the proof that Φ Xj xY∈S is a pessimistic estimator, except the values of the dual Since y = 1+ αf (x), the first part follows. solutions take the place of C∗ . j x∈S j | | For theQ second part, we first note the role of the Proof. In an iteration where the dual solution has value dual solution in each iteration: given the current x r/d, the number of uncovered elements decreases from and y, the iteration increases the quantity j yj by a r to r d = r(1 1/v) < re−1/v. By induction on factor of 1 + αv(x, y). (This follows fromP inspection the iterations,− the− algorithm maintains the invariant of the algorithm and the definition of v(x, y).) Next that the number of uncovered elements is less than we apply the sequence of inequalities that bounded n/ exp 1/v where v is the value of the dual the pessimistic estimator below 1 in 4.2: By the ℓ ℓ ℓ § solution P corresponding to the ℓth iteration and ℓ ranges last inequality in Hoeffding’s bound (Lemma 4.2), 1 + 1 v(x,y)+ǫ 2 over the iterations so far. Note thatv ¯ = C / . αv(x, y) (1 + α) / exp(2ǫ ). Let vℓ denote the | | vℓ ⋄ ≤ P value of v(x, y) at the ℓth iteration (for 1 ℓ S ). Before the last iteration at least one element is left, By induction on the iterations ≤ ≤ | | so at that point n/ exp((k 1)/v¯) > 1. Thus, − (vℓ+ǫ) 2 y m(1 + α) ℓ / exp(2 S ǫ ). Corollary 6.1. The harmonic mean of the values of j ≤ | | Xj P the dual solutions over the first k 1 iterations is larger than k−1 , where k is the size of the− final cover. Since S v¯ = v , this gives the result. ln n | | ℓ ℓ ⋄ P Randomized Rounding without Solving the Linear Program 7

Find-Packing(P,f,ǫ,ω) ω ln m Corollary 6.4. After λ∗b(−ǫ) iterations of the cov- 1. S ; yj 1 (j =1,...,m) ¯ ← {} ← ering algorithm, λ (1 ǫ)¯v, that is, the primal and 2. repeat average dual values≥ diﬀer− by at most a factor of 1 ǫ. 3. choose x P to minimize v = y f (x) − ∈ j j j 4. S S x P The algorithm is described in Figure 3. ← ∪{ } 5. y y 1+ ǫ fj (x) (j =1,...,m) j ← j · ω 6. V max(V,v/ yj) ← j 7. Fj [( S 1)FPj + fj(x)]/ S (j =1,...,m) ← | |− | | 8. λ¯ maxj Fj 9. until←λ¯ (1 + ǫ)V 10. return ≤ x/ S x∈S | | P Figure 3: Algorithm for packing. To obtain covering algorithm, negate ǫ’s and change each “max” to “min”, “minimize” to “maximize”, and “ ”to “ ”. ≤ ≥

2 ω ln m Corollary 6.2. After 2ǫ2 iterations of the generalized packing algorithm, λ¯ v¯+ǫ. That is, the primal and average dual values diﬀer≤ by at most ǫ.

6.3 Packing Dual: The packing and covering algorithms also generate implicit dual solutions whose average values converge to the primal value. Let λ¯ andv ¯ be deﬁned as for the generalized packing dual.

Lemma 6.3. The packing algorithm maintains the invariant that

¯ (1 + ǫ)|S|λ/ω meǫ|S|v/ω¯ . ≤

We omit this and subsequent proofs in this section, since they are similar to that of Lemma 6.2.

(1+ǫ)ω ln m Corollary 6.3. After λ∗b(ǫ) iterations of the packing algorithm, λ¯ (1 + ǫ)¯v. That is, the primal and average dual values≤ diﬀer by at most a factor of 1+ ǫ.

Our ﬁnal packing algorithm detects convergence by comparing the primal value to the best dual value so far. The algorithm is shown in Figure 3. The algorithm maintains f(¯x) (in the variable F ) instead ofx ¯.

6.4 Covering Dual:

Lemma 6.4. The covering algorithm maintains the invariant that

¯ (1 ǫ)|S|λ/ω me−ǫ|S|v/ω¯ . − ≤ 8 Young

7 Using an Approximate Oracle 8 Integer Packing and Covering

If the subroutine for computing minx∈P j yj fj(x) re- The packing and covering algorithms in Figure 3, as turns only an approximate minimizer x, ourP algorithms they stand, do not allow explicit control over the gran- still work well. The degree of approximation (absolute ularity of the final solution. Because the number of and/or relative) of the subroutine carries over into the iterations can be less than the upper bound, the algo- performance guarantee of the algorithm. For covering, rithms only guarantee a lower bound on the granularity. it can also affect the convergence rate (and therefore the Of course, the lower bound is the difficult part, so it is granularity). not surprising that exact control over the granularity We model the error by assuming that, given y, the can be obtained. In this section, we discuss briefly how oracle returns an x such that to modify those algorithms to find, e.g., an integer solution. (7.6) v(x, y) (1 + δ1) min v(x, y)+ δ2 For simplicity, we consider a particular case of ≤ x∈P integer packing. Fix an instance of the packing problem where v(x, y) = y f (x)/ y , δ 0 denotes the (P,f,ω,ǫ). Let λ∗ and x∗ be an optimal solution. In j j j j j 1 ≥ relative error andPδ2 0 denotesP the absolute error. We addition, let V P be the extreme points on the ≥ ⊂ call this a (δ1,δ2)-approximate oracle. (For covering, boundary of P (if P is a polytope, V is its vertex set). the notion of approximation is defined analogously.) We assume that the oracle returns only elements of V . In each iteration, y still represents a dual solution. The integer packing problem is to compute a maximum Since x is only an approximate minimizer, the value of cardinality multiset S V such that f (x) 1. ⊆ x∈S j ≤ the dual solution is no longer v(x, y), but it is at least Note that, for any such S, S P 1/λ∗ because 2 | |≤⌊ ⌋ v(x,y)−δ . Still usingv ¯ to denote the average of the f(¯x) 1/ S , wherex ¯ = x/ S . An ǫ-approximate 1+δ1 ≤ | | x∈S | | values of the dual solutions for the previous iterations, integer solution is a set PS such that S 1/λ∗ and | |≥⌊ ⌋ definev ˜ to be the average of the corresponding v(x, y)’s. fj (x) 1+ ǫ. x∈S ≤ Lemmas 6.2, 6.3, and 6.4 go through directly provided P Let S be a multiset obtained by repeatedly choosing “˜v” is substituted for “¯v”. From the (modified) lemmas, random elements of V , where each random element is by the same reasoning that gives the corollaries to those chosen from a distribution on V with mean x∗. (Such lemmas, together with the fact thatv ˜ (1 + δ1)¯v + δ2, distributions exist because P is the convex closure of we get the following propositions. ≤ V .) Lemma 8.1. When S 1/λ∗, Proposition 7.1. Suppose the generalized packing al- | |≤ gorithm uses a (δ1,δ2)-approximate oracle. After 2 ω ln m ¯ Pr ( j) fj(x) 1+ ǫ < m/ exp[b(ǫ)/ω]. 2ǫ2 iterations, λ v˜ + ǫ (1 + δ1)¯v + δ2 + ǫ. h ∃ ≥ i l m ≤ ≤ xX∈S The proof is essentially the same as that of Lemma 5.1, except 1/ S replaces λ∗. Proposition 7.2. Suppose the packing algorithm uses | | (1+ǫ)ω ln m A corollary to the lemma is that, provided a (δ1,δ2)-approximate oracle. After λ∗b(ǫ) itera- m/ exp[b(ǫ)/ω] 1, there exists an ǫ-approximate inte- l m ≤ tions, λ¯ (1 + ǫ)˜v (1 + ǫ)(1 + δ1)¯v +(1+ ǫ)δ2. ger solution. The corresponding algorithm is the same ≤ ≤ as the basic packing algorithm, except the termination For covering,v ˜ (1 δ1)¯v δ2. condition is different. The algorithm terminates when ≥ − − adding another element would cause f (x) > 1+ǫ Proposition 7.3. Suppose the covering algorithm uses x∈S j for some j. Because the algorithm keeps up with a (δ ,δ )-approximate oracle. After P 1 2 the random process, the resulting set has size at least 1/λ∗ . ω ln m ⌊ ⌋ ∗ Complexity and Performance Guarantee: [(1 δ1)λ δ2]b( ǫ) − − − The algorithm is given in Figure 4. Note that 1/λ∗ ∗ ⌊ ⌋≤ iterations, λ¯ (1 ǫ)˜v (1 ǫ)(1 δ1)¯v (1 ǫ)δ2. S (1 + ǫ)/λ , so the number of iterations in ≥ − ≥ − − − − this| | ≤ case ⌊ is at most⌋ (1 + ǫ)/λ∗. For the condition These results hold for the algorithms without mod- m/ exp[b(ǫ)/ω] 1, it suffices that, for instance, ǫ ification. In particular, V in the packing algorithm in 2 max(ω ln m, √≤ω ln m). ≥ Figure 3 equals the best v(x, y) seen so far, which is at Covering: The same techniques apply for integer leastv ˜, so is guaranteed to be within a 1+ ǫ factor of λ¯ covering. For covering, define an ǫ-approximate integer within the required number of rounds. solution to be a set S such that S 1/λ∗ and | | ≤ ⌈ ⌉ Randomized Rounding without Solving the Linear Program 9

Find-Integer-Packing (P,f,ǫ,ω) question is whether oblivious rounding can be applied assumption: m/ exp[b(ǫ)/ω] 1. to such bounds to optimize more general functions. 1. S ; y 1, F 0 (j =1≤ ,...,m) ← {} j ← j ← 2. repeat References 3. choose x P to minimize y f (x) ∈ j j j 4. Fj Fj + fj (x) (j =1,...,mP ) ← 5. if maxj Fj > 1+ ǫ return S [1] Noga Alon and Joel H. Spencer. The Probabilistic 6. S S x Method. John Wiley and Sons, New York, 1992. ← ∪{ } [2] Ingo Althöfer. On sparse approximations to random- 7. y y 1+ ǫ fj (x) (j =1,...,m) j ← j · ω ized strategies and convex combinations. Linear Alge- bra and its Applications, 199, March 1994. Figure 4: Algorithm for integer packing. To obtain [3] Dimitris Bertsimas and Rakesh Vohra. Linear programming relaxations, approximation algorithms and covering algorithm, negate ǫ’s and change “max” to randomization; a unified view of covering problems. “min”, “minimize” to “maximize”, and “>” to “<”. Draft, January 1994. [4] H. Brönnimann and Michael T. Goodrich. Almost f (x) 1 ǫ. (Many variations are possible.) optimal set covers in bounded VC-dimension. In x∈S j Proc. of the 10th Annual Symposium on Computational Let S be a random≥ − multiset as above. P Geometry, 1994. Lemma 8.2. When S 1/λ∗, [5] M. D. Grigoriadis and L. G. Kachiyan. Approximate | |≥ Solution of Matrix Games in Parallel, pages 129–136. Elsevier Science Publishers B.V., 1992. Also available Pr ( j) fj (x) 1 ǫ < m/ exp[b( ǫ)/ω]. ∃ ≤ − − as TR-91-73 from DIMACS. h xX∈S i [6] M. D. Grigoriadis and L. G. Kachiyan. A sublinear- The resulting algorithm is described in Figure 4. The time randomized approximation algorithm for ma- number of iterations in this case is at most 1/λ∗ . trix games. Technical Report LCSR-TR-222, Rut- For the condition m/ exp[b( ǫ)/ω] 1, it suffices⌈ that⌉ gers University Computer Science Department, New ǫ √2ω ln m. − ≤ Brunswick, NJ, April 1994. ≥ [7] Wassily Hoeffding. Probability inequalities for sums 9 Conclusion of bounded random variables. American Statistical Journal, pages 13–30, March 1963. Partial derandomization: The point of oblivious [8] Thomas Hofmeister and Hanno Lefmann. Com- rounding is not derandomization per se, but to achieve puting sparse approximations deterministically. independence from the unknown aspects of the optimal Unpublished manuscript, Dortmund, Germany. solution. For some random rounding schemes, some of hofmeist,[email protected], the parameters of the random process are known; these 1994. can be left in the algorithm. For instance, in concur- [9] Joseph JáJá. Introduction to Parallel Algorithms. rent multicommodity flow, the relative amount of flow Addison-Wesley Publishing Company, Inc., 1992. of each commodity is known. A natural randomized [10] David S. Johnson. Approximation algorithms for combinatorial problems. Journal of Computer and rounding scheme is to choose a commodity with prob- System Sciences, 9:256–278, 1974. ability proportional to its (known) demand, and then [11] P. Klein, S. Plotkin, C. Stein, and E. Tardos. Faster to choose a flow path among paths for that commodity approximation algorithms for the unit capacity con- with probability proportional to its (unknown) weight in current flow problem with applications to routing and the optimal flow. Applying oblivious rounding to only finding sparse cuts. SIAM Journal on Computing, the second random choice gives a randomized algorithm 23(3):466–487, June 1994. in the style of [16]. [12] T. Leighton, F. Makedon, S. Plotkin, C. Stein, E. Tar- Mixed bounds: Each of the random analyses in dos, and S. Tragoudas. Fast approximation algorithms this paper employed a single type of probabilistic bound. for multicommodity flow problems. In Proc. of the 23rd This is not a limitation of the technique. Oblivious Ann. ACM Symp. on Theory of Computing, pages 101– rounding can be applied to analyses using, e.g., sums of 111, 1991. [13] Richard J. Lipton and Neal E. Young. Simple strate- probabilities bounded by Raghavan’s bounds, Hoeffd- gies for large zero-sum games with applications to com- ing’s bound, and Markov’s inequality. This is relatively plexity theory. In Proc. of the 26th Ann. ACM Symp. straightforward, if technically more tedious. on Theory of Computing, 1994. To appear. More general functions: Chernoff-type bounds [14] LászlóLovász. On the ratio of optimal integral and exist for more general classes of functions than linear fractional covers. Discrete Mathematics, 13:383–390, functions (e.g., Azuma’s inequality [1]). A natural 1975. 10 Young

[15] Michael Luby and Noam Nisan. A parallel approximation algorithm for positive linear programming. In Proc. of the 25th Ann. ACM Symp. on Theory of Com- puting, pages 448–457, 1993. [16] Serge Plotkin, David Shmoys, and Eva´ Tardos. Fast approximation algorithms for fractional packing and covering problems. In Proc. of the 32nd IEEE Annual Symp. on Foundation of Computer Science, pages 495– 504, 1991. [17] Prabhakar Raghavan. Probabilistic construction of de- terministic algorithms approximating packing integer programs. Journal of Computer and System Sciences, 37(2):130–143, October 1988. [18] Prabhakar Raghavan and C. Thompson. Randomized rounding: A technique for provably good algorithms and algorithmic proofs. Combinatorica, 7:365–374, 1987. [19] F. Shahroki and D. W. Matula. The maximum concurrent ﬂow problem. Journal of the ACM, 37:318–334, 1990. [20] Joel H. Spencer. Ten Lectures on the Probabilistic Method. Society for Industrial and Applied Mathemat- ics, 3600 University City Science Center, Philadelphia, PA 19104-2688, 1987.