Recursive Algorithms A recursive algorithm is one in which objects are defined in terms Recurrences of other objects of the same type. Computer Science & Engineering 235: Discrete Mathematics Advantages: I Simplicity of code Christopher M. Bourke I Easy to understand [email protected] Disadvantages: I Memory I Speed I Possibly redundant work Tail recursion offers a solution to the memory problem, but really, do we need recursion? Recursive Algorithms Motivating Example Analysis Factorial Recall the factorial function. 1 if n = 1 n! = n (n 1)! if n > 1 · − Consider the following (recursive) algorithm for computing n!: We’ve already seen how to analyze the running time of algorithms. However, to analyze recursive algorithms, we require more Algorithm (Factorial) sophisticated techniques. Input : n N Specifically, we study how to define & solve recurrence relations. ∈ Output : n! 1 if n = 1 then 2 return 1 3 end 4 else 5 return Factorial(n 1) n − × 6 end Motivating Example Recurrence RelationsI Factorial - Analysis? Definition How many multiplications M(n) does Factorial perform? Definition I When n = 1 we don’t perform any. A recurrence relation for a sequence a is an equation that I Otherwise we perform 1. { n} expresses an in terms of one or more of the previous terms in the I Plus how ever many multiplications we perform in the sequence, recursive call, Factorial(n 1). − a0, a1, . , an 1 − I This can be expressed as a formula (similar to the definition of for all integers n n where n is a nonnegative integer. n!. ≥ 0 0 M(0) = 0 A sequence is called a solution of a recurrence relation if its terms M(n) = 1 + M(n 1) − satisfy the recurrence relation. I This is known as a recurrence relation. Recurrence RelationsII Recurrence Relations III Definition Definition Example More generally, recurrences can have the form The Fibonacci numbers are defined by the recurrence, F (n) = F (n 1) + F (n 2) T (n) = αT (n β) + f(n),T (δ) = c − − − F (1) = 1 F (0) = 1 or n T (n) = αT + f(n),T (δ) = c β The solution to the Fibonacci recurrence is n n Note that it may be necessary to define several T (δ), initial 1 1 + √5 1 1 √5 fn = − conditions. √5 2 ! − √5 2 ! (your book derives this solution). Recurrence RelationsIV Recurrence RelationsV Definition Definition The initial conditions specify the value of the first few necessary Recurrence relations have two parts: recursive terms and terms in the sequence. In the Fibonacci numbers we needed two non-recursive terms. initial conditions, F (0) = F (1) = 1 since F (n) was defined by the two previous terms in the sequence. T (n) = 2T (n 2) + n2 10 − − Initial conditions are also known as boundary conditions (as recursive non-recrusive opposed to the general conditions). | {z } | {z } From now on, we will use the subscript notation, so the Fibonacci Recursive terms come from when an algorithm calls itself. numbers are Non-recursive terms correspond to the “non-recursive” cost of the fn = fn 1 + fn 2 − − algorithm—work the algorithm performs within a function. f1 = 1 f0 = 1 We’ll see some examples later. First, we need to know how to solve recurrences. Solving Recurrences Linear Homogeneous Recurrences There are several methods for solving recurrences. Definition I Characteristic Equations A linear homogeneous recurrence relation of degree k with I Forward Substitution constant coefficients is a recurrence relation of the form I Backward Substitution an = c1an 1 + c2an 2 + + ckan k I Recurrence Trees − − ··· − I Maple! with c1, . , ck R, ck = 0. ∈ 6 Linear Homogeneous Recurrences Solving Linear Homogeneous RecurrencesI Examples n Examples We want a solution of the form an = r where r is some (real) constant. n The Fibonacci sequence is a linear homogeneous recurrence We observe that an = r is a solution to a linear homogeneous relation. As are the following. recurrence if and only if n n 1 n 2 n k an = 4an 1 + 5an 2 + 7an 3 r = c r − + c r − + + c r − − − − 1 2 ··· k an = 2an 2 + 4an 4 + 8an 8 − − − n k We can now divide both sides by r − , collect terms, and we get a How many initial conditions do we need to specify for these? As k-degree polynomial. many as the degree, k = 3, 8 respectively. k k 1 k 2 r c1r − c2r − ck 1r ck = 0 So, how do we solve linear homogeneous recurrences? − − − · · · − − − Solving Linear Homogeneous RecurrencesII Second Order Linear Homogeneous Recurrences A second order linear homogeneous recurrence is a recurrence of the form an = c1an 1 + c2an 2 − − k k 1 k 2 r c1r − c2r − ck 1r ck = 0 − − − · · · − − − Theorem (Theorem 1, p414) This is called the characteristic equation of the recurrence relation. 2 Let c1, c2 R and suppose that r c1r c2 = 0 is the ∈ − − The roots of this polynomial are called the characteristic roots of characteristic polynomial of a 2nd order linear homogeneous 1 the recurrence relation. They can be used to find solutions (if they recurrence which has two distinct roots, r1, r2. exist) to the recurrence relation. We will consider several cases. Then a is a solution if and only if { n} n n an = α1r1 + α2r2 for n = 0, 1, 2,... where α1, α2 are constants dependent upon the initial conditions. 1we discuss how to handle this situation later. Second Order Linear Homogeneous Recurrences Second Order Linear Homogeneous Recurrences Example Example Continued Example I Using the 2nd-order theorem, we have a solution, Find a solution to n n an = α1(2 ) + α2(3 ) an = 5an 1 6an 2 − − − I Now we can plug in the two initial conditions to get a system with initial conditions a0 = 1, a1 = 4 of linear equations. 0 0 I The characteristic polynomial is a0 = α1(2) + α2(3) 1 1 a1 = α1(2) + α2(3) r2 5r + 6 − I Using the quadratic formula (or common sense), the root can 1 = α1 + α2 (1) be found; 4 = 2α1 + 3α2 (2) r2 5r + 6 = (r 2)(r 3) − − − so r1 = 2, r2 = 3 Second Order Linear Homogeneous Recurrences Second Order Linear Homogeneous Recurrences Example Continued Another Example I Solving for α = (1 α ) in (1), we can plug it into the 1 − 2 second. Example 4 = 2α1 + 3α2 4 = 2(1 α2) + 3α2 Solve the recurrence − 4 = 2 2α + 3α − 2 2 2 = α2 an = 2an 1 + 15an 2 − − − I Substituting back into (1), we get with initial conditions a0 = 0, a1 = 1. α1 = 1 − If we did it right, we have I Putting it all back together, we have 1 n 1 n an = (3) ( 5) n n 8 8 an = α1(2 ) + α2(3 ) − − = 1 2n + 2 3n − · · How can we check ourselves? Single Root Case Single Root Case Example Recall that we can only apply the first theorem if the roots are Example distinct, i.e. r = r . 1 6 2 What is the solution to the recurrence relation If the roots are not distinct (r1 = r2), we say that one characteristic root has multiplicity two. In this case we have to an = 8an 1 16an 2 apply a different theorem. − − − with initial conditions a0 = 1, a1 = 7? Theorem (Theorem 2, p416) 2 I The characteristic polynomial is Let c1, c2 R with c2 = 0. Suppose that r c1r c2 = 0 has ∈ 6 − − only one distinct root, r . Then a is a solution to 0 { n} r2 8r + 16 an = c1an 1 + c2an 2 if and only if − − − n n I Factoring gives us an = α1r0 + α2nr0 r2 8r + 16 = (r 4)(r 4) for n = 0, 1, 2,... where α1, α2 are constants depending upon the − − − initial conditions. so r0 = 4 Single Root Case General Linear Homogeneous Recurrences Example I By Theorem 2, we have that the solution is of the form n n There is a straightforward generalization of these cases to higher an = α14 + α2n4 order linear homogeneous recurrences. I Using the initial conditions, we get a system of equations; Essentially, we simply define higher degree polynomials. a0 = 1 = α1 The roots of these polynomials lead to a general solution. a1 = 7 = 4α1 + 4α2 The general solution contains coefficients that depend only on the 3 initial conditions. I Solving the second, we get that α2 = 4 I And so the solution is In the general case, however, the coefficients form a system of 3 linear inequalities. a = 4n + n4n n 4 I We should check ourselves... General Linear Homogeneous RecurrencesI General Linear Homogeneous Recurrences Distinct Roots Any Multiplicity Theorem (Theorem 3, p417) Let c1, . , ck R. Suppose that the characteristic equation ∈ k k 1 Theorem (Theorem 4, p418) r c1r − ck 1r ck = 0 − − · · · − − − has k distinct roots, r , . , r . Then a sequence a is a solution Let c1, . , ck R. Suppose that the characteristic equation 1 k { n} ∈ of the recurrence relation k k 1 r c1r − ck 1r ck = 0 − − · · · − − − an = c1an 1 + c2an 2 + + ckan k − − ··· − has t distinct roots, r1, . , rt with multiplicities m1, . , mt. if and only if a = α rn + α rn + + α rn n 1 1 2 2 ··· k k for n = 0, 1, 2,..., where α1, α2, . , αk are constants. General Linear Homogeneous Recurrences Linear Nonhomogeneous Recurrences Any Multiplicity Theorem (Continued) Then a sequence an is a solution of the recurrence relation For recursive algorithms, cost functions are often not homogenous { } because there is usually a non-recursive cost depending on the an = c1an 1 + c2an 2 + + ckan k input size. − − ··· − if and only if Such a recurrence relation is called a linear nonhomogeneous recurrence relation.