Recurrence Relations

Many algorithms particularly divide and conquer al

gorithms have time complexities which are naturally

modeled byrecurrence relations

Arecurrence relation is an equation which is dened in

termsofitself

Why are recurrences go o d things

Many natural functions are easily expressed as re

currences

a a a a n pol y nomial

n n

n

n

a a a a exponential

n n

n

a n a a a n weir d f unction

n n

n

It is often easy to nd a recurrence as the solution

of a counting problem Solving the recurrence can

be done for many sp ecial cases as we will see

although it is somewhat of an art

Recursion is Mathematical

Induction

In b oth wehave general and b oundary conditions with

the general condition breaking the problem into smaller

and smaller pieces

The initial orboundary condition terminate the recur

sion

As we will see induction provides a useful to ol to solve

recurrences guess a solution and prove it byinduction

T T T

n

n

n

T

n

Guess what the solution is

Prove T n by induction

n

Show that the basis is true T

Nowassumetruefor T

n

Using this assumption show

n n

T T

n

n

Solving Recurrences

No general pro cedure forsolving recurrence relations is

known which is why it is an art My approach is

Realize that linear nite history

constant co ecient recurrences

always can be solved

Check out any combinatorics ordierential equations

book fora pro cedure

Consider a a a a a

n

n n

It has history degree and co ecients of

and Thus it can b e solved mechanically Pro ceed

Find the characteristic equation eg

Solve to get ro ots which app earinthe exp onents

Take care of rep eated ro ots and inhomogeneous

parts

Find the constants to nish the job

p p p p

n n

a

n

Systems like Mathematica and Maple have packages

for doing this

Guess a solution and prove by

induction

To guess the solution play around with small values

for insight

Note that you can do inductive pro ofs with the bigOs

notations just b e sure you use it right

Example Show that T n c n lg n forlarge enough

c and n

Assumethat it is true for n then

T n cbnc lgbncn

cn lgbnc n dropping o ors makes it bigger

cnlg n lg n log of division

cn lg n cn n

cn lg n whenever c

Starting with basis cases T T lets us

complete the pro of for c

Try backsubstituting until you

know what is going on

Also known as the iteration metho d Plug the recur

rence back into itself until you see a pattern

Example T nT bncn Try backsubstituting

T n n bnc T bnc

n bnc bnc Tbnc

n bnc bnc T bnc

n

The term should nowbeobvious

Although there are only log n termsbefore wegetto

T it do esnt hurt to sum them all since this is a

fast growing geometric series

i

X

log

n T T n n

i

T nn onO n

Recursion Trees

Drawing a picture of the backsubstitution pro cess gives

you a idea of what is going on

We must keep track of two things the size of

the remaining argument to the recurrence and the

additive stu to b e accumulated during this call

Example T nT n n

T(n)

2 2 n n

T(n/2) T(n/2) 2 2 2 (n/2) (n/2) n/2

2 2 2 2 2

T(n/4) T(n/4) T(n/4) T(n/4) (n/4) (n/4) (n/4) (n/4) n/4

The remaining arguments are on the left the additive

termsonthe right

Although this tree has height lg n the total sum at

each level decreases geometricallyso

X X

i i

T n n n n

i i

The recursion tree framework made this much easier

to see than with algebraic backsubstitution

See if you can use the Master

theorem to provide an instant

asymptotic solution

The Master Theorem Leta andbbecon

stants let f nbeafunction and let T nbedened

on the nonnegative integers by the recurrence

T naT nbf n

where we interpret nb to mean either bnbc or dnbe

Then T ncan b e b ounded asymptotically as follows

log a

b

If f n O n for some constant

log a

b

then T nO n

log a log a

b b

If f nn then T nn lg n

log a

b

If f n n for some constant

and if af nb cf n for some constant c

and all suciently large nthenT nf n

Examples of the Master

Theorem

Which case of the Master Theorem applies

T nT n n

Reading from the equation a b and

f nn

log

Is n O n O n

Yes so case applies and T nO n

T nT n n

Reading from the equation a b and

f nn

log

Is n O n O n

No if but it is true if socase applies

and T nn log n

T nT n n

Reading from the equation a b and

f nn

log

Is n n n

Yes for so case might apply

Is n c n

Yes for c so there exists a c to sat

isfy the regularitycondition so case applies and

T nn

Why should the Master

Theorem be true

Consider T naT nbf n

Supp ose f n is small enough

Say f nie T naT nb

Then wehave a recursion tree where the only contri

bution is at the leaves

l

There will b e log n levels with a leaves at level l

b

log n log a

b b

T na n Theorem in CLR

0

0 0

11 11

so long as f n is small enough that it is dwarfed by

this wehave case of the Master Theorem

Supp ose fn is large enough

If wedraw the recursion tree for T naT nbf n

T(n) f(n)

T(n/b)...... T(n/b) f(n/b) ...... f(n/b)

If f n is a big enough function the one top call can

be bigger than the sum of all the little calls

Example f nn n n n In

fact this holds unless a

In case of the Master Theorem the additive term

dominates

In case b oth parts contribute equallywhichis why

the log p ops up It is usually what wewant to have

happ en in a divide and conquer algorithm

Famous Algorithms and their

Recurrence

Matrix Multiplication

The standard matrix multiplication algorithm for two

n n matrices is O n

23 13 18 23 2 3 4 34 3 4 5 18 25 32

4 5 23 32 41

Strassen discovered a divideandconquer algorithm which

takes T nT n O n time

lg

Since O n dwarfs O n case of the master the

orem applies and T nO n

This has b een improved bymore and more compli

cated recurrences until the current b est in O n

Polygon Triangulation

Given a polygon in the plane add diagonals so that

each face is a triangle None of the diagonals are allowed

to cross

Triangulation is an important rst step in many geo

metric algorithms

The simplest algorithm might be to try each pair of

p oints and check if they see each other If so add the

diagonal and recur on b oth halves fora total of O n

However Chazelle gave an algorithm which runs in

p

T nT nO n time Since n O n

bycase of the Master TheoremChazelles algorithm

is linear ie T nO n

Sorting

The classic divide and conquer recurrence is Merge

sorts T nT n O n which divides the data

into equalsized halves and sp ends linear timemerging

the halves after they are sorted

log

Since n O n O n but not n O n

Case of the Master Theorem applies and T n

O n log n

In case the divide and merge steps balance out p er

fectlyasweusually hop e forfrom a divideandconquer

algorithm

Mergesort Animations

XEROX FROM SEDGEWICK

Approaches to Algorithms

Design

Incremental

Job is partly done do a little more rep eat until done

Agood example of this approach is insertion sort

DivideandConquer

Arecursive technique

Divide problem into subproblemsofthe same kind

For subproblemsthatare really small trivial solve

them directly Else solve them recursively con

quer

Combine subproblem solutions to solve the whole

thing combine

Agood example of this approach is Mergesort