Recurrence Relations Many algorithms particularly divide and conquer al gorithms have time complexities which are naturally modeled byrecurrence relations Arecurrence relation is an equation which is dened in termsofitself Why are recurrences go o d things Many natural functions are easily expressed as re currences a a a a n pol y nomial n n n n a a a a exponential n n n a n a a a n weir d f unction n n n It is often easy to nd a recurrence as the solution of a counting problem Solving the recurrence can be done for many sp ecial cases as we will see although it is somewhat of an art Recursion is Mathematical Induction In b oth wehave general and b oundary conditions with the general condition breaking the problem into smaller and smaller pieces The initial orboundary condition terminate the recur sion As we will see induction provides a useful to ol to solve recurrences guess a solution and prove it byinduction T T T n n n T n Guess what the solution is Prove T n by induction n Show that the basis is true T Nowassumetruefor T n Using this assumption show n n T T n n Solving Recurrences No general pro cedure forsolving recurrence relations is known which is why it is an art My approach is Realize that linear nite history constant co ecient recurrences always can be solved Check out any combinatorics ordierential equations book fora pro cedure Consider a a a a a n n n It has history degree and co ecients of and Thus it can b e solved mechanically Pro ceed Find the characteristic equation eg Solve to get ro ots which app earinthe exp onents Take care of rep eated ro ots and inhomogeneous parts Find the constants to nish the job p p p p n n a n Systems like Mathematica and Maple have packages for doing this Guess a solution and prove by induction To guess the solution play around with small values for insight Note that you can do inductive pro ofs with the bigOs notations just b e sure you use it right Example Show that T n c n lg n forlarge enough c and n Assumethat it is true for n then T n cbnc lgbncn cn lgbnc n dropping o ors makes it bigger cnlg n lg n log of division cn lg n cn n cn lg n whenever c Starting with basis cases T T lets us complete the pro of for c Try backsubstituting until you know what is going on Also known as the iteration metho d Plug the recur rence back into itself until you see a pattern Example T nT bncn Try backsubstituting T n n bnc T bnc n bnc bnc Tbnc n bnc bnc T bnc n The term should nowbeobvious Although there are only log n termsbefore wegetto T it do esnt hurt to sum them all since this is a fast growing geometric series i X log n T T n n i T nn onO n Recursion Trees Drawing a picture of the backsubstitution pro cess gives you a idea of what is going on We must keep track of two things the size of the remaining argument to the recurrence and the additive stu to b e accumulated during this call Example T nT n n T(n) 2 2 n n T(n/2) T(n/2) 2 2 2 (n/2) (n/2) n/2 2 2 2 2 2 T(n/4) T(n/4) T(n/4) T(n/4) (n/4) (n/4) (n/4) (n/4) n/4 The remaining arguments are on the left the additive termsonthe right Although this tree has height lg n the total sum at each level decreases geometricallyso X X i i T n n n n i i The recursion tree framework made this much easier to see than with algebraic backsubstitution See if you can use the Master theorem to provide an instant asymptotic solution The Master Theorem Leta andbbecon stants let f nbeafunction and let T nbedened on the nonnegative integers by the recurrence T naT nbf n where we interpret nb to mean either bnbc or dnbe Then T ncan b e b ounded asymptotically as follows log a b If f n O n for some constant log a b then T nO n log a log a b b If f nn then T nn lg n log a b If f n n for some constant and if af nb cf n for some constant c and all suciently large nthenT nf n Examples of the Master Theorem Which case of the Master Theorem applies T nT n n Reading from the equation a b and f nn log Is n O n O n Yes so case applies and T nO n T nT n n Reading from the equation a b and f nn log Is n O n O n No if but it is true if socase applies and T nn log n T nT n n Reading from the equation a b and f nn log Is n n n Yes for so case might apply Is n c n Yes for c so there exists a c to sat isfy the regularitycondition so case applies and T nn Why should the Master Theorem be true Consider T naT nbf n Supp ose f n is small enough Say f nie T naT nb Then wehave a recursion tree where the only contri bution is at the leaves l There will b e log n levels with a leaves at level l b log n log a b b T na n Theorem in CLR 0 0 0 11 11 so long as f n is small enough that it is dwarfed by this wehave case of the Master Theorem Supp ose fn is large enough If wedraw the recursion tree for T naT nbf n T(n) f(n) T(n/b)... ... T(n/b) f(n/b) ... ... f(n/b) If f n is a big enough function the one top call can be bigger than the sum of all the little calls Example f nn n n n In fact this holds unless a In case of the Master Theorem the additive term dominates In case b oth parts contribute equallywhichis why the log p ops up It is usually what wewant to have happ en in a divide and conquer algorithm Famous Algorithms and their Recurrence Matrix Multiplication The standard matrix multiplication algorithm for two n n matrices is O n 23 13 18 23 2 3 4 34 3 4 5 18 25 32 4 5 23 32 41 Strassen discovered a divideandconquer algorithm which takes T nT n O n time lg Since O n dwarfs O n case of the master the orem applies and T nO n This has b een improved bymore and more compli cated recurrences until the current b est in O n Polygon Triangulation Given a polygon in the plane add diagonals so that each face is a triangle None of the diagonals are allowed to cross Triangulation is an important rst step in many geo metric algorithms The simplest algorithm might be to try each pair of p oints and check if they see each other If so add the diagonal and recur on b oth halves fora total of O n However Chazelle gave an algorithm which runs in p T nT nO n time Since n O n bycase of the Master TheoremChazelles algorithm is linear ie T nO n Sorting The classic divide and conquer recurrence is Merge sorts T nT n O n which divides the data into equalsized halves and sp ends linear timemerging the halves after they are sorted log Since n O n O n but not n O n Case of the Master Theorem applies and T n O n log n In case the divide and merge steps balance out p er fectlyasweusually hop e forfrom a divideandconquer algorithm Mergesort Animations XEROX FROM SEDGEWICK Approaches to Algorithms Design Incremental Job is partly done do a little more rep eat until done Agood example of this approach is insertion sort DivideandConquer Arecursive technique Divide problem into subproblemsofthe same kind For subproblemsthatare really small trivial solve them directly Else solve them recursively con quer Combine subproblem solutions to solve the whole thing combine Agood example of this approach is Mergesort.