Recursive Algorithms Recursion A recursive algorithm is one in which objects are defined in terms of other objects of the same type. Slides by Christopher M. Bourke Instructor: Berthe Y. Choueiry Advantages: I Simplicity of code I Easy to understand Fall 2007 Disadvantages: I Memory I Speed Computer Science & Engineering 235 Introduction to Discrete Mathematics I Possibly redundant work Sections 7.1 - 7.2 of Rosen Tail recursion offers a solution to the memory problem, but really, [email protected] do we need recursion? Recursive Algorithms Motivating Example Analysis Factorial Recall the factorial function. 1 if n = 1 n! = n · (n − 1)! if n > 1 Consider the following (recursive) algorithm for computing n!: We’ve already seen how to analyze the running time of algorithms. However, to analyze recursive algorithms, we require more Algorithm (Factorial) sophisticated techniques. Specifically, we study how to define & solve recurrence relations. Input : n ∈ N Output : n! 1 if n = 1 then 2 return 1 3 end 4 else 5 return Factorial(n − 1) × n 6 end Motivating Example Recurrence Relations I Factorial - Analysis? Definition How many multiplications M(n) does Factorial perform? Definition I When n = 1 we don’t perform any. A recurrence relation for a sequence {an} is an equation that I Otherwise we perform 1. expresses an in terms of one or more of the previous terms in the Plus how ever many multiplications we perform in the I sequence, recursive call, Factorial(n − 1). a0, a1, . , an−1 I This can be expressed as a formula (similar to the definition of n!. for all integers n ≥ n0 where n0 is a nonnegative integer. M(0) = 0 A sequence is called a solution of a recurrence relation if its terms M(n) = 1 + M(n − 1) satisfy the recurrence relation. I This is known as a recurrence relation. Recurrence Relations II Recurrence Relations III Definition Definition Example Consider the recurrence relation: a = 2a − a . n n−1 n−2 The Fibonacci numbers are defined by the recurrence, It has the following sequences an as solutions: F (n) = F (n − 1) + F (n − 2) 1. an = 3n, F (1) = 1 2. an = n + 1, and F (0) = 1 3. an = 5. The solution to the Fibonacci recurrence is Initial conditions + recurrence relation uniquely determine the √ !n √ !n 1 1 + 5 1 1 − 5 sequence. fn = √ − √ 5 2 5 2 (your book derives this solution). Recurrence Relations IV Recurrence Relations V Definition Definition The initial conditions specify the value of the first few necessary More generally, recurrences can have the form terms in the sequence. In the Fibonacci numbers we needed two initial conditions, F (0) = F (1) = 1 since F (n) was defined by the T (n) = αT (n − β) + f(n),T (δ) = c two previous terms in the sequence. Initial conditions are also known as boundary conditions (as or n opposed to the general conditions). T (n) = αT + f(n),T (δ) = c β From now on, we will use the subscript notation, so the Fibonacci numbers are Note that it may be necessary to define several T (δ), initial fn = fn−1 + fn−2 conditions. f1 = 1 f0 = 1 Recurrence Relations VI Solving Recurrences Definition Recurrence relations have two parts: recursive terms and non-recursive terms. There are several methods for solving recurrences. T (n) = 2T (n − 2) + n2 − 10 | {z } | {z } I Characteristic Equations recursive non-recrusive I Forward Substitution Recursive terms come from when an algorithm calls itself. I Backward Substitution Recurrence Trees Non-recursive terms correspond to the “non-recursive” cost of the I algorithm—work the algorithm performs within a function. I Maple! We’ll see some examples later. First, we need to know how to solve recurrences. Linear Homogeneous Recurrences Linear Homogeneous Recurrences Examples Definition A linear homogeneous recurrence relation of degree k with Examples constant coefficients is a recurrence relation of the form The Fibonacci sequence is a linear homogeneous recurrence an = c1an−1 + c2an−2 + ··· + ckan−k relation. As are the following. a = 4a + 5a + 7a with c1, . , ck ∈ R, ck 6= 0. n n−1 n−2 n−3 an = 2an−2 + 4an−4 + 8an−8 Linear: RHS is a sum of multiples of previous terms of the I How many initial conditions do we need to specify for these? As sequence (linear combination of previous terms). The many as the degree, k = 3, 8 respectively. coefficients are all constants (not functions depending on n). Homogeneous: no terms occur that are not multiples of the I So, how do we solve linear homogeneous recurrences? aj’s. I Degree k: an is expressed in terms of k terms of the sequence. Solving Linear Homogeneous Recurrences I Solving Linear Homogeneous Recurrences II n We want a solution of the form an = r where r is some (real) constant. We observe that a = rn is a solution to a linear homogeneous k k−1 k−2 n r − c1r − c2r − · · · − ck−1r − ck = 0 recurrence if and only if This is called the characteristic equation of the recurrence relation. rn = c rn−1 + c rn−2 + ··· + c rn−k 1 2 k The roots of this polynomial are called the characteristic roots of the recurrence relation. They can be used to find solutions (if they We can now divide both sides by rn−k, collect terms, and we get a exist) to the recurrence relation. We will consider several cases. k-degree polynomial. k k−1 k−2 r − c1r − c2r − · · · − ck−1r − ck = 0 Second Order Linear Homogeneous Recurrences Second Order Linear Homogeneous Recurrences Example A second order linear homogeneous recurrence is a recurrence of the form Example an = c1an−1 + c2an−2 Find a solution to an = 5an−1 − 6an−2 Theorem (Theorem 1, p462) with initial conditions a0 = 1, a1 = 4 2 Let c1, c2 ∈ R and suppose that r − c1r − c2 = 0 is the characteristic polynomial of a 2nd order linear homogeneous I The characteristic polynomial is 1 recurrence which has two distinct roots, r1, r2. r2 − 5r + 6 Then {an} is a solution if and only if n n I Using the quadratic formula (or common sense), the root can an = α1r1 + α2r2 be found; r2 − 5r + 6 = (r − 2)(r − 3) for n = 0, 1, 2,... where α1, α2 are constants dependent upon the initial conditions. so r1 = 2, r2 = 3 1we discuss how to handle this situation later. Second Order Linear Homogeneous Recurrences Second Order Linear Homogeneous Recurrences Example Continued Example Continued I Solving for α1 = (1 − α2) in (1), we can plug it into the I Using the 2nd-order theorem, we have a solution, second. n n 4 = 2α1 + 3α2 an = α1(2 ) + α2(3 ) 4 = 2(1 − α2) + 3α2 4 = 2 − 2α + 3α I Now we can plug in the two initial conditions to get a system 2 2 of linear equations. 2 = α2 0 0 I Substituting back into (1), we get a0 = α1(2) + α2(3) 1 1 a1 = α1(2) + α2(3) α1 = −1 I Putting it all back together, we have 1 = α1 + α2 (1) n n 4 = 2α1 + 3α2 (2) an = α1(2 ) + α2(3 ) = −1 · 2n + 2 · 3n Second Order Linear Homogeneous Recurrences Single Root Case Another Example Recall that we can only apply the first theorem if the roots are distinct, i.e. r1 6= r2. Example If the roots are not distinct (r1 = r2), we say that one Solve the recurrence characteristic root has multiplicity two. In this case we have to apply a different theorem. an = −2an−1 + 15an−2 Theorem (Theorem 2, p464) with initial conditions a0 = 0, a1 = 1. 2 Let c1, c2 ∈ R with c2 6= 0. Suppose that r − c1r − c2 = 0 has If we did it right, we have only one distinct root, r0. Then {an} is a solution to an = c1an−1 + c2an−2 if and only if 1 n 1 n an = (3) − (−5) 8 8 n n an = α1r0 + α2nr0 How can we check ourselves? for n = 0, 1, 2,... where α1, α2 are constants depending upon the initial conditions. Single Root Case Single Root Case Example Example Example I By Theorem 2, we have that the solution is of the form What is the solution to the recurrence relation n n an = α14 + α2n4 an = 8an−1 − 16an−2 I Using the initial conditions, we get a system of equations; with initial conditions a0 = 1, a1 = 7? a0 = 1 = α1 a1 = 7 = 4α1 + 4α2 I The characteristic polynomial is 3 I Solving the second, we get that α2 = 4 r2 − 8r + 16 I And so the solution is Factoring gives us 3 I a = 4n + n4n n 4 r2 − 8r + 16 = (r − 4)(r − 4) I We should check ourselves... so r0 = 4 General Linear Homogeneous Recurrences General Linear Homogeneous Recurrences I Distinct Roots Theorem (Theorem 3, p465) There is a straightforward generalization of these cases to higher Let c1, . , ck ∈ R. Suppose that the characteristic equation order linear homogeneous recurrences. rk − c rk−1 − · · · − c r − c = 0 Essentially, we simply define higher degree polynomials. 1 k−1 k The roots of these polynomials lead to a general solution. has k distinct roots, r1, . , rk. Then a sequence {an} is a solution of the recurrence relation The general solution contains coefficients that depend only on the initial conditions. an = c1an−1 + c2an−2 + ··· + ckan−k In the general case, however, the coefficients form a system of if and only if linear equalities. n n n an = α1r1 + α2r2 + ··· + αkrk for n = 0, 1, 2,..., where α1, α2, .