12 Solving Recurrence Relations FACT

Home , Recurrence relation

12 Solving Recurrence Relations FACT. (L k1)(L k2) . . . (L kn) annihilates all se- − − i − i i quences of the form c1k + c2k + . . . + cnk . h 1 2 ni Recurrence relations are perhaps the most important tool in the analysis of algorithms. We have encountered sev- What if k = ℓ? To answer this question, we consider eral methods that can sometimes be used to solve such (L k)2 iki = (L k) (i + 1)ki+1 iki+1 relations, such as guessing the solution and proving it by − h i − h − i = (L k) ki+1 induction, or developing the relation into a sum for which − h i we find a closed form expression. We now describe a new = 0 . h i method to solve recurrence relations and use it to settle More generally, we have the remaining open question in the analysis of Fibonacci heaps. FACT. (L k)n annihilates all sequences of the form p(i)k−i , with p(i) a polynomial of degree n 1. h i − Annihilation of sequences. Suppose we are given an in- finite sequence of numbers, A = a0,a1,a2,... . We can Since operators annihilate only certain types of sequences, multiply with a constant, shift toh the left and addi another we can determinethe sequenceif we knowthe annihilating sequence: operator. The general method works in five steps:

kA = ka0,ka1,ka2,... , 1. Write down the annihilator for the recurrence. h i LA = a1,a2,a3,... , 2. Factor the annihilator. h i A + B = a0 + b0,a1 + b1,a2 + b2,... . 3. Determine what sequence each factor annihilates. h i As an example, consider the sequence of powers of two, 4. Put the sequences together. i ai = 2 . Multiplying with 2 and shifting to the left give 5. Solve for the constants of the solution by using initial the same result. Therefore, conditions.

LA 2A = 0, 0, 0,... . − h i Fibonacci numbers. We putthemethodto a testbycon- We write LA 2A = (L 2)A and think of L 2 as an sidering the Fibonacci numbers deﬁned recursively as fol- − − − operator that annihilates the sequence of powers of 2. In lows: general, L k annihilates any sequence of the form cki . What does−L k do to other sequences A = cℓi ,h wheni F0 = 0, − h i ℓ = k? F1 = 1, 6 Fj = Fj−1 + Fj−2, for j 2. (L k)A = cℓ,cℓ2,cℓ3,... ck,ckℓ,ckℓ2,... ≥ − h i − h i = (ℓ k) c,cℓ,cℓ2,... Writing a few of the initial numbers, we get the sequence − h i 2 = (ℓ k)A. 0, 1, 1, 2, 3, 5, 8,... . We notice that L L 1 annihi- − latesh the sequence becausei − −

We see that the operator L k annihilates only one type 2 2 − (L L 1) Fj = L Fj L Fj Fj of sequence and multiplies other similar sequences by a − − h i h i− h i − h i constant. = Fj+2 Fj+1 Fj h i − h i − h i = 0 . h i Multiple operators. Instead of just one, we can ap- If we factor the operator into its roots, we get ply several operators to a sequence. We may multiply 2 with two constants, , multiply and shift, L L 1 = (L ϕ)(L ϕ), k(ℓA) = (kℓ)A − − − − L(kA) = k(LA), and shift twice, L(LA)=L2A. For where example, (L k)(L ℓ) annihilates all sequences of the − − form cki + dℓi , where we assume k = ℓ. Indeed, L k 1+ √5 annihilatesh cki i and leaves behind (ℓ6 k)dℓi , which− is ϕ = = 1.618 ..., h i h − i 2 annihilated by L ℓ. Furthermore, (L k)(L ℓ) anni- 1 √5 hilates no other sequences.− More generally,− we− have ϕ = 1 ϕ = − = 0.618 .... − 2 −

41 The first rootis knownas the golden ratio because it repre- For larger j, we get sj from sj−1 by adding the size of a sents the aspect ratio of a rectangular piece of paper from minimum tree with root degree j 2, which is sj−2. Hence − which we may remove a square to leave a smaller rect- sj = sj−1 + sj−2, which is the same recurrence relation angular piece of the same ratio: ϕ :1=1: ϕ 1. that defines the Fibonacci numbers. The initial values are − Thus we know that (L ϕ)(L ϕ) annihilates Fj and shifted two positions so we get sj = Fj+2, as claimed. this means that the -th− Fibonacci− number is ofh thei form j Consider a Fibonacci heap with n nodes and let ν be a F = cϕj + c ϕj . We get the constant factors from the j node with maximum degree D = D(n). The Size Lemma initial conditions: implies n FD+2. The Fibonacci number with index D +2 is roughly≥ ϕD+2/√5. Because ϕD+2 < √5, we F0 = 0= c + c, have F1 = 1= cϕ + c ϕ. 1 n ϕD+2 1. Solving the two linear equations in two unknowns, we get ≥ √5 − c =1/√5 and c = 1/√5. This implies that − After rearranging the terms and taking the logarithm to the j j 1 1+ √5 1 1 √5 base ϕ, we get Fj = − . √5 2 ! − √5 2 ! D log √5(n + 1) 2. ≤ ϕ − From this viewpoint, it seems surprising that turns out Fj Recall that logϕ x = log2 x/ log2 ϕ and use the calculator to be an integer for all j. Note that ϕ > 1 and ϕ < 1. to verify that log ϕ = 0.694 ... > 0.5 and log √5 = | | j | | 2 ϕ It follows that for growing exponent j, ϕ goes to infinity 1.672 ...< 2. Hence j and ϕ goes to zero. This implies that Fj is approximately ϕj /√5, and that this approximation becomes more and log2(n + 1) D + logϕ √5 2 more accurate as j grows. ≤ log2 ϕ − < 2log2(n + 1). Maximum degree. Recall that D(n) is the maximum possible degree of any one node in a Fibonacci heap of Non-homogeneous terms. We now return to the anni- size n. We need two easy facts about the kind of trees that hilation method for solving recurrence relations and con- arise in Fibonacci heaps in order to show that D(n) is at sider most logarithmic in n. Let ν be a node of degree j, and let µ1, µ2,...,µj be its children ordered by the time they aj = aj−1 + aj−2 +1. were linked to ν. This is similar to the recurrence that defines Fibonacci numbers and describes the minimum number of nodes in DEGREE LEMMA. The degreeof µi is at least i 2. − an AVL tree, also known as height-balanced tree. It is de- fined by the requirement that the height of the two sub- PROOF. Recall that nodes are linked only during the deletemin operation. Right before the linking happens, the trees of a node differ by at most 1. The smallest tree two nodes are roots and have the same degree. It follows of height j thus consists of the root, a subtree of height and another subtree of height . We refer to the that the degree of µ was at least i 1 at the time it was j 1 j 2 i terms− involving as the homogeneous− terms of the re- linked to ν. The degree of µ might− have been even higher ai i lation and the others as the non-homogeneous terms. We because it is possible that ν lost some of the older children know that L2 L 1 annihilates the homogeneous part, after µi had been linked. After being linked, µi may have − − lost at most one of its children, for else it would have been aj = aj−1 + aj−2. If we apply it to the entire relation we get cut. Its degree is therefore at least i 2, as claimed. − 2 (L L 1) aj = aj+2 aj+1 aj SIZE LEMMA. The number of descendents of ν (includ- − − h i h i − h i − h i = 1, 1,... . ing ν) is at least Fj+2. h i The remaining sequence of 1s is annihilated by L 1. − PROOF. Let sj be the minimum number of descendents a In other words, (L ϕ)(L ϕ)(L 1) annihilates aj − j − j ′−j h i node of degree j can have. We have s0 = 1 and s1 = 2. implying that aj = cϕ + c ϕ + c 1 . It remains to find

42 the constants, which we get from the boundary conditions The Master Theorem. It is sometimes more convenient a0 =1, a1 =2 and a2 =4: to look up the solution to a recurrence relation than play- ing with different techniques to see whether any one can ′ c + c + c = 1, make it to yield. Such a cookbook method for recurrence ′ ϕc + ϕ c + c = 2, relations of the form ϕ2c + ϕ2c + c′ = 4. T (n) = aT (n/b)+ f(n) Noting that ϕ2 = ϕ +1, ϕ2 = ϕ +1, and ϕ ϕ = √5 we get c =(5+2√5)/5, c = (5 2√5)/5, and− c′ = 1. is provided by the following theorem. Here we assume − − that a 1 and b > 1 are constants and that f is a well- The minimum number of nodes of a height-j AVL tree is ≥ therefore roughly the constant c times ϕj . Conversely, the behaved positive function. maximum height of an AVL tree with n = cϕj nodes is MASTER THEOREM. Define and let be an roughly j = logϕ(n/c)=1.440 . . . log2 n + O(1). In c = logb a ε words, the height-balancing condition· implies logarithmic arbitrarily small positive constant. Then height. O(nc) if f(n)= O(nc−ε), T (n) = O(nc log n) if f(n)= O(nc),  c+ε Transformations. We extend the set of recurrences we  O(f(n)) if f(n)=Ω(n ). can solve by employing transformations that produce relations amenable to the annihilation method. We demon- The last of the three cases also requires a usually satis- strate this by considering mergesort, which is another fied technical condition, namely that af(n/b) < δf(n) divide-and-conquer algorithm that can be used to sort a for some constant δ strictly less than 1. For example, this list of n items: condition is satisfied in T (n)=2T (n/2)+ n2 which implies T (n)= O(n2). Step 1. Recursively sort the left half of the list. As another example consider the relation T (n) = Step 2. Recursively sort the right half of the list. 2T (n/2) + n that describes the running time of merge- c sort. We have c = log2 2=1 and f(n) = n = O(n ). Step 3. Merge the two sorted lists by simultaneously The middle case of the Master Theorem applies and we scanning both from beginning to end. get T (n)= O(n log n), as before.

The running time is described by the solution to the recurrence

T (1) = 1, T (n) = 2T (n/2)+ n.

We have no way to work with terms like T (n/2) yet. However, we can transform the recurrence into a more i i manageable form. Deﬁning n = 2 and ti = T (2 ) we get

t0 = 1, i ti = 2ti−1 +2 .

The homogeneous part is annihilated by L 2. Similarly, non-homogeneous part is annihilated by L− 2. Hence, 2 − (L 2) annihilates the entire relation and we get ti = (ci+−c)2i. Expressed in the original notation we thus have T (n) = (c log2 n + c)n = O(n log n). This result is of course no surprise and reconﬁrms what we learned earlier about sorting.