Solving Divide-And-Conquer Recurrences
Total Page:16
File Type:pdf, Size:1020Kb
Solving Divide-and-Conquer Recurrences Victor Adamchik A divide-and-conquer algorithm consists of three steps: • dividing a problem into smaller subproblems • solving (recursively) each subproblem • then combining solutions to subproblems to get solution to original problem We use recurrences to analyze the running time of such algorithms. Suppose Tn is the number of steps in the worst case needed to solve the problem of size n. Let us split a problem into a ¥ 1 subproblems, each of which is of the input size n where b > 1. b Observe, that the number of subproblems a is not necessarily equal to b. The total number of steps Tn is obtained by all steps needed to solve smaller subproblems Tn b plus the ê number needed to combine solutions into a final one. The following equation is called divide-and-conquer recurrence relation Tn = a Tn b + f n ê H L As an example, consider the mergesort: -divide the input in half -recursively sort the two halves -combine the two sorted subsequences by merging them. Let T n be worst-case runtime on a sequence of n keys: H L If n = 1, then T n = Q 1 constant time H L H L If n > 1, then T n = 2 T n 2 + Q n H L H ê L H L here Q(n) is time to do the merge. Then 15-451: Algorithm Design and Analysis 2 Tn = 2 Tn 2 + Q n ê H L Other examples of divide and conquer algorithms: quicksort, integer multiplication, matrix multiplication, fast Fourier trnsform, finding conver hull and more. There are several techniques of solving such recurrence equations: • the iteration method • the tree method • the master-theorem method • guess-and-verify ü Tree method We could visualize the recursion as a tree, where each node represents a recursive call. The root is the initial call. Leaves correspond to the exit condition. We can often solve the recurrence by looking at the structure of the tree. To illustrate, we take this example n T n = 2 T + n2 H L K 2 O T 1 = 1 H L Here is a recursion tree that diagrams the recursive function calls T(n) T(n/2) T(n/2) T(n/4) T(n/4) T(n/4) T(n/4) ……………… T(1) T(1) Using a recursion tree we can model the time of a recursive execution by writing the size of the problem in each node. 3 Using a recursion tree we can model the time of a recursive execution by writing the size of the problem in each node. The last level corresponds to the initial condition of the recurrence. Since the work at each leaf is constant, the total work at all leaves is equal to the number of leaves, which is h log n 2 = 2 2 = n To find the total time (for the whole tree), we must add up all the terms -1+log n 1 1 1 2 1 k T n = n + n2 1 + + + + ... = n + n2 H L ‚ 2 4 8 k=0 2 The sum is easily computed by means of the geometric series h xh+1 - 1 xk = , x ∫ 1 ‚ - k=0 x 1 This yeilds 15-451: Algorithm Design and Analysis 4 T n = 2 n2 - 2 n + n = 2 n2 - n H L Check with Mathematica RSolve A9T@nD == 2 T@n ê 2D + n2, T@1D ä 1=, T@nD, nE 88T@nD → n H−1 + 2 nL<< Example. Solve the recurrence n T n = 3 T + n H L K 4 O The work at all levels is 3 9 n 1 + + + ... 4 16 log 4 n Since the height is log 4 n, the tree has 3 leaves. Hence, the total work is given by -1+log n 4 3 k log n T n = n + 3 4 T 1 H L ‚ H L k=0 4 By means of the geometric series and taking into account log n log 3 3 4 = n 4 the above sum yields log 3 log 3 T n = 4 n - 4 n 4 + n 4 T 1 = O n H L H L H L ü The Master Theorem The master theorem solves recurrences of the form n T n = a T + f n H L b H L for a wide variety of function f n and a ¥ 1, b > 1. In this section we will outline the H L 5 H L main idea. Here is the recursive tree for the above equation log b n It is easy to see that the tree has a leaves. Indeed, since the height is log b n, and the tree branching factor is a, the number of leaves is log a n 1 h log n log b log b log a a = a b = a a = n a = n b Summing up values at each level, gives n n 2 log a T n = f n + a f + a f + ... + n b T 1 H L H L b b2 H L Therefore, the solution is -1+log n b n log a k T n = n b T 1 + a f H L H L ‚ k k=0 b log a Now we need to compare the asymptotic behavior of f n with n b . There are three H L possible cases. log a log a Q n b if f n = O n b I M H L I M log n k+1 log a k T n = Q n b log n if f n = Q n b log n , k ¥ 0 H L I M H L I M log a Q f n if f n = W n b H H LL H L I M The following examples demonstrate the theorem. Case 1 . T n = 4 T n + n H L I 2 M log a log 4 2 2 We have f n = n and n b = n 2 = n , therefore f n = O n . Then the solution is H L H L I M T n = Q n2 by case 1. H L I M Case 2 . T n = 4 T n + n2 H L I 2 M In this case f n = n2 and f n = Q n2 . Then T n = Q n2 log n by case 2. H L H L I M H L I M Case 3 . T n = 4 T n + n3 H L I 2 M 3 log a 2 3 In this case f n = n and f n = W n b = W n . Then T n = Q n by case 3. H L H L I M I M H L I M 15-451: Algorithm Design and Analysis 6 Karatsuba Algorithm ü Multiplication of large integers The brute force approach ("grammar school" method) 1 2 3 4 5 ------ 6 1 5 4 9 2 --------- 5 5 3 5 We say that multiplication of two n-digits integers has time complexity at worst O n2 . I M We develop an algorithm that has better asymptotic complexity. The idea is based on divide-and-conquer technique. Consider the above integers and split each of them in two parts 123 = 12 * 10 + 3 45 = 4 * 10 + 5 and then multiply them: 123*45 = (12*10 + 3)(4*10 + 5) = 12 ∗ 4 ∗ 10 2 + H12 ∗ 5 + 4 ∗ 3L ∗ 10 + 3 ∗ 5 In general, the integer which has n digits can be represented as num = x * 10 m + y where n m = floor K 2 O n x = ceiling K 2 O n y = floor K 2 O Example, 154 517 766 = 15 451 ∗ 10 4 + 7766 Consider two n-digits numbers 7 p num 1 = x1 * 10 + x0 p num 2 = y1 * 10 + y0 Their product is num * num = x * y * 10 2 p + x * y + x * y * 10 p + x * y 1 2 1 1 H 1 0 0 1L 0 0 Just looking at this general formula you can say that just instead of one multiplication we have 4. Where is the advantage? numbers x1, x0 and y1, y0 have twice less digits. ü The worst-case complexity Let T n denote the number of digit multiplications needed to multiply two n-digits num- H L bers. The recurrence (since the algorithm does 4 multiplications on each step) T n = 4 T n + O n , T c = 1 H L I 2 M H L H L Note, we ignore multiplications by a base!!! Its solution is given by log n 2 T n = 4 2 = n H L The algorithm is still quadratic! ü The Karatsuba Algorithm 1962, Anatolii Karatsuba, Russia. num * num = x * y * 10 2 p + x * y + x * y * 10 p + x * y 1 2 1 1 H 1 0 0 1L 0 0 The goal is to decrease the number of multiplications from 4 to 3. We can do this by observing that x + x * y + y = x * y + x * y + x * y + x * y H 1 0L H 1 0L 1 1 0 0 H 1 0 0 1L It follows that num * num = x * y * 10 2 p + x + x * y + y - x * y - x * y * 10 p + x * y 1 2 1 1 J H 1 0L H 1 0L 1 1 0 0 N 0 0 and it is only 3 multiplications (see it ?). The total number of multiplications is given by (we ignore multiplications by a base) T n = 3 T n + O n , T(c) = 1 H L I 2 M H L Its solution is 15-451: Algorithm Design and Analysis 8 log n log 3 1.58 ..