Recurrence Relations Acknowledgements

• The first 12 slides are from the course taught by Prof. Yun Peng at UMBC. – http://www.csee.umbc.edu/~ypeng/S03203/lecture-notes/Ch06.ppt Recursive Algorithms: Analysis

• We have already learnt to analyze the running time of iterative algorithms by considering the number of instructions executed in the Word RAM Model. • In case of Recursive algorithms, it is not easy to compute the number of instructions by looking at code. – The number of instructions in one instance of function call depends on the number of instructions executed when recursive calls are made. – Hence to analyze recursive algorithms, we require more sophisticated techniques • Specifically, we need to solve recurrence relations Recurrence Relations

• A recurrence relation for the sequence {an} is an equation that expresses an in terms of one or more of the previous terms of the sequence, namely, a0, a1, …, an-1, for all integers n with n  n0, where n0 is a nonnegative integer.

• A sequence is called a solution of a recurrence relation if its terms satisfy the recurrence relation. Recurrence Relations

• In other words, a recurrence relation is like a recursively defined sequence, but without specifying any initial values (initial conditions).

• Therefore, the same recurrence relation can have (and usually has) multiple solutions.

• If both the initial conditions and the recurrence relation are specified, then the sequence is uniquely determined. Recurrence Relations

• Example: Consider the recurrence relation

an = 2an-1 – an-2 for n = 2, 3, 4, …

• Is the sequence {an} with an=3n a solution of this recurrence relation?

• For n  2 we see that 2an-1 – an-2 = 2(3(n – 1)) – 3(n – 2) = 3n = an.

• Therefore, {an} with an=3n is a solution of the recurrence relation.

• Is the sequence {an} with an=5 a solution of the same recurrence relation?

• For n  2 we see that 2an-1 – an-2 = 2*5 - 5 = 5 = an.

• Therefore, {an} with an=5 is also a solution of the recurrence relation. Modeling with Recurrence Relations

• Example: • Someone deposits 10,000 in a savings account at a bank yielding 5% per year with interest compounded annually. How much money will be in the account after 30 years? • Solution:

• Let Pn denote the amount in the account after n years.

• How can we determine Pn on the basis of Pn-1? Modeling with Recurrence Relations

• We can derive the following recurrence relation:

• Pn = Pn-1 + 0.05Pn-1 = 1.05Pn-1.

• The initial condition is P0 = 10,000. • Then we have:

• P1 = 1.05P0 2 • P2 = 1.05P1 = (1.05) P0 3 • P3 = 1.05P2 = (1.05) P0 • … n • Pn = 1.05Pn-1 = (1.05) P0

• We now have a formula to calculate Pn for any natural number n and can avoid the iteration. Modeling with Recurrence Relations

• Let us use this formula to find P30 under the initial condition P0 = 10,000:

30 • P30 = (1.05) * 10,000 = 43,219.42

• After 30 years, the account contains Rs. 43,219.42. Modeling with Recurrence Relations

• Another example:

• Let an denote the number of bit strings of length n that do not have two consecutive 0s (“valid strings”). Find a recurrence relation and

give initial conditions for the sequence {an}.

• Solution: • Idea: The number of valid strings equals the number of valid strings ending with a 0 plus the number of valid strings ending with a 1. Modeling with Recurrence Relations

• Let us assume that n  3, so that the string contains at least 3 bits.

• Let us further assume that we know the number an-1 of valid strings of length (n – 1). • Then how many valid strings of length n are there, if the string ends with a 1?

– There are an-1 such strings, namely the set of valid strings of length (n – 1) with a 1 appended to them.

• Note: Whenever we append a 1 to a valid string, that string remains valid. Modeling with Recurrence Relations

• Now we need to know: How many valid strings of length n are there, if the string ends with a 0? • Valid strings of length n ending with a 0 must have a 1 as their (n – 1)th bit (otherwise they would end with 00 and would not be valid). • And what is the number of valid strings of length (n – 1) that end with a 1?

– We already know that there are an-1 strings of length n that end with a 1.

• Therefore, there are an-2 strings of length (n – 1) that end with a 1. Modeling with Recurrence Relations

• So there are an-2 valid strings of length n that end with a 0 (all valid strings of length (n – 2) with 10 appended to them).

• As we said before, the number of valid strings is the number of valid strings ending with a 0 plus the number of valid strings ending with a 1.

• That gives us the following recurrence relation:

an = an-1 + an-2 Modeling with Recurrence Relations

• What are the initial conditions?

• a1 = 2 (0 and 1)

• a2 = 3 (01, 10, and 11)

• a3 = a2 + a1 = 3 + 2 = 5

• a4 = a3 + a2 = 5 + 3 = 8

• a5 = a4 + a3 = 8 + 5 = 13 • …

• This sequence satisfies the same recurrence relation as the Fibonacci sequence.

• Since a1 = f3 and a2 = f4, we have an = fn+2. Example: Factorial

• Recall the factorial function: 1 if n= 1 n! = n.(n-1) ! if n > 1

• Consider the following (recursive) algorithm for computing n! int Factorial (int n) if (n=1) or (n=0) return 1 else return n  Factorial (n-1) Factorial: Analysis

• How many multiplications T(x) does factorial function perform? • When n=1 or n = 0 we don’t perform any • Otherwise, we perform one explicitly in the code, and some number of multiplications are performed in the recursive call to Factorial (n-1) • The number of multiplications can be expressed as a formula (similar to the definition of n!) • T(0) = 0 • T(n) = 1 + T(n-1) • To know the time complexity of factorial function, we will have to solve this recurrence relation. Factorial complexity

T(n) = T(n – 1) + 1, T(0) = 0

T(n) = T(n – 1 ) + 1 T(n – 1 ) = T(n – 2 ) + 1 = [T(n – 2 ) + 1 ] + 1 = T(n – 2 ) + 2 T(n – 2 ) = T(n – 3 ) + 1 = T(n – 3 ) + 3 ...... = T(n – k ) + k To use Boundary condition let T(n – k ) = T(0) k = n T(n) = 0 + n = n T(n) = O(n) Recurrence for Linear Search

• Problem: Given an array containing n elements, search if a value x is present in the array. • Linear Search: Examine each element of the array one by one. Total number of operations T(n) to handle n elements can be broken down in two parts. • One part is to examine the first element ( 1 operation), and the other part is to examine remaining elements which requires T(n – 1 ) operations. • We can write a recurrence relation for T(n) T(n) = T(n-1) + 1 Recurrence for Binary Search

• Problem: Given an array containing n elements, search if a value x is present in the array. • Binary Search: Examine the middle element of the array ( 1 operation), and then examine either the left half or the right half of the array which requires T(n/2) operations. • Thus the recurrence relation for T(n) takes the form T(n) = T(n/2) + 1 Binary Search Complexity

T(n) = T(n/2) + 1, T(1) = 1 T(n/2) = T(n/4) + 1 T(n) = T(n/2) + 1 T(n/4) = T(n/8) + 1 = [T(n/4) + 1 ] + 1 = [T(n/8) + 1 ] + 2 = T(n/8) + 3 = T(n/23) + 3 ...... = T(n/2k) + k

To use Boundary condition let us try to bring it to form T(1) Binary Search Complexity

let T(n/2k) = T(1) n/2k = 1 2k = n Take log of both sides k log 2 = log n k = log n Substituting back in T(n) =T(n/2k) + k T(n) = T(1) + log n = 1 + log n = O(log n) Solving Recurrence Relations - Iteration method

• Steps: ▪ Expand the recurrence ▪ Express the expansion as a summation by plugging the recurrence back into itself until you see a pattern. ▪ Evaluate the summation • In evaluating the summation one of the following formulae may be used: • Arithmetic series: •Special Cases of Geometric Series:

• Geometric Series: Solving Recurrence Relations - Iteration method

• Harmonic Series:

• Others: Analysis of Towers of Hanoi Algorithm

public static void hanoi(int n, char from, char to, char temp){ if (n == 1) Move (from, to); else{ hanoi(n - 1, from, temp, to); Move (from, to); hanoi(n - 1, temp, to, from); } }

• The recurrence relation for the running time of the method hanoi is: T(n) = a if n = 1 T(n) = 2T(n - 1) + b if n > 1 Analysis of Towers of Hanoi Algorithm

Expanding: T(n) = 2T(n – 1) + b = 2[2T(n – 2) + b] + b = 22 T(n – 2) + 2b + b = 22 [2T(n – 3) + b] + 2b + b = 23 T(n – 3) + 22b + 2b + b = 23 [2T(n – 4) + b] + 22b + 2b + b = 24 T(n – 4) + 23 b + 22b + 21b + 20b = …… = 2k T(n – k) + b[2k- 1 + 2k– 2 + . . . 21 + 20]

When k = n – 1, we have:

Therefore, The method hanoi is O(2n)