Orthogonal Polynomials in Stein's Method

Orthogonal Polynomials in Steins Metho d

Wim Schoutens

KatholiekeUniversiteit Leuven

Department of Mathematics

Celestijnenlaan B

B Leuven

Belgium

Abstract

We discuss Steins metho d for Pearsons and Ords family of distributions We giveasys

tematic treatment including the Stein equation its solution and smo othness conditions Akey

role in the analysis is played by the classical orthogonal p olynomials

AMS Subject Classication J H

Keywords Steins Method Orthogonal Polynomials Pearsons Class Ords

Class Approximation Distributions Markov Processes

Postdo ctoral Researcher of the Fund for Scientic Research Flanders Belgium

Running Head Steins Method Orthogonal Polynomials

Mailing Address Wim Schoutens Katholieke Universiteit Leuven Department of Math

ematics Celestijnenlaan B B Leuven Belgium

Email WimSchoutenswiskuleuvenacbe

Intro duction

Steins Metho d provides a way of nding approximations to the distribution say of a random

variable which at the same time gives estimates of the approximation error involved The strenghts

of the metho d are that it can b e applied in many circumstances in which dep endence plays a part In

essence the metho d is based on a dening equation or equivalently an op erator of the distribution

and a related Stein equation Up to now is was not clear which equation to take One could

think at a lot of equations We show how for a broad class of distributions there is one equation

who has a sp ecial role We give a systematic treatment including the Stein equation its solution

and smo othness conditions

A key to ol in Steins theory is the generator metho d develop ed by Barb our Barb our

suggested employing for the op erator of the Stein equation the generator of a Markov pro cess In

us lo oks for a dening equation for which is related to the generator this generator metho d one th

of a Markov pro cess For a given distribution there may be various Markov pro cesses who t in

Barb ours metho d However up to now it was still not clear which Markov pro cess to take to

obtain good results We show how for a broad class of distributions there is a sp ecial Markov

pro cess a birth and death pro cess or a diusion which takes a leading role in the analysis

Furthermore a key role is played by the classical orthogonal p olynomials P Diaconis and S

Zab ell already mentioned this connection It turns out that the dening op erator is based on a

hyp ergeometric dierence or dierential equation which lies at the heart of the classical orthogonal

p olynomials Furthermore the sp ectral representation of the transition probabilities of the Markov

pro cess involved will b e in terms of orthogonal p olynomials closely related to the distribution to b e

approximated This systematic treatment together with the intro duction of orthogonal p olynomials

in the analysis seems to b e new Furthermore some earlier uncovered examples like the Beta the

Students t and the Hyp ergeometric distribution are nowworked out

Prelimaries

Birth and Death Pro cesses and Diusions

A birth and death process fX t g is a Markov pro cess on the state space S f g with

stationary transition probabilities ie P t PrX j jX i i j S is not dep ending on

ij ts s

s and with innitesimal generator A given by

Af i f i f i f i i S

i i i i

for all b ounded realvalued functions f B S and where we take for i not on the

i i

b oundary of S On the b oundary of S we must have We will always work with

i i

S N f g in which case we set or take S f Ng with N a p ositive

in teger and in which case Furthermore we do not allow the existence of an

absorbing state ie a b oundary state with birth parameter and death parameter equal to zero

The parameters and are called resp ectivelythebirth and death rates It can b e shown that

i i

the limits lim P tp j S exist and are indep endentof the initial state i It turns out

t ij j

j S where j S nfg that the p are given by p

j j j j j j

k S

and and In order that the sequence fp g denes a distribution we must have

P P

p We say that fp g is the limiting stationary distribution If then clearly

k k

then all p are zero and we do not have a limiting stationary distribution

In the analysis of birth and death pro cesses a prominent role is played by a sequence of

p olynomials fQ xn S g called birthdeath polynomials They are determined uniquely by

the recurrence relation

xQ x Q x Q x Q x n S

n n n n n n n n

together with Q xandQ x Karlin and McGregor proved that the transition

function P can b e represented as

e Q xQ xdx i j S t P t

i j ij j

where is a p ositive Borel measure with total mass and with supp ort on the nonnegative real

axis is called the spectral measure of P Taking t in one easily sees that the p olynomials

fQ xn S g are orthogonal with resp ect to In the examples we will encounter a variety of

birth and death pro cesses Other examples can be found in the litterature see for example

and

Another class of Markov pro cesses will app ear also in the analysis Diusions with state space

S a b a b We refer to for a general intro duction Supp ose A is the

generator of the diusion In a clear pro of is given of the fact that A is of the form

Af xxf x xf x

where x is called the drift co ecient and x the diusion co ecient We will highlight

the sp ectral representation for some of diusion pro cesses in the examples see also

Steins Metho d

Normal Approximation and Poisson Approximation

In Stein published a metho d to prove Normal approximation It is based on the fact

that a random variable Z has a Standard Normal distribution N if and only if for all dieren

tiable functions f suchthat E jf X j whereX has a Standard Normal distribution N

Z E f Z Zf

Hence it seems reasonable that if E f W WfW is small for a large class of functions

f then the distribution of W is close to the Standard Normal distribution Supp ose we wish to

estimate the dierence b etween the exp ectation of a smo oth function h with resp ect to the random

variable W and E hZ where Z has a Standard Normal distribution Stein showed that for

any smo oth realvalued b ounded function h there is a function f f solving the now called Stein

equation for the Standard Normal

f x xf xhx E hZ

with Z a Standard Normal random variable The unique b ounded solution of the ab ove equation

is given by

f xexpx hy E hZ exp y dy

Then we estimate

E f W Wf W

and hence E h W E hZ The next step is to show that the quantity is small In order to

do this we will use the structure of W For instance it mightbethat W is a sum of indep endent

random variables In addition we will use some smo othness conditions on f Stein showed the

following inequalities

jjf jj jjh E hZ jj

jjf jj suph inf h

jjf jj jjh jj

where jj jj denotes the supremum norm

In this way we can bound the distance of W from the Normal in terms of a test function h

the immediate b ound of the distance is one of the key advantages of Steins metho d compared to

moment generating functions or characteristic functions

Chen applied Steins idea in the context of Poisson approximation The Stein equation for

the Poisson distribution Pisnow a dierence equation

x xf x hx E hZ f

where h is a b ounded realvalued function dened on the set of the nonnegativeintegers and Z has a

Poisson distribution P The choice of the left hand side of equation is based on the fact that a

random variable W on the set of the nonnegativeintegers has a Poisson distribution with parameter

if and only if for all b ounded realvalued functions f on the integers E f W WfW

The solution of the Stein equation for the Poisson distribution Pisgiven by

x hk E hZ f xx

This solution is the unique except at x b ounded solution the value f x for negative x do es

not enter into consideration and is conventionally taken to be zero In one nds the following

estimates of the smo othness for f by an analytic argument

jjf jj jjhjj min andjjf jj jjhjj min

h h

where f xf x f x

General Approach

For an arbitrary distribution the general pro cedure is Find a go o d characterization of the desired

distribution in terms of an equation that is of the t yp e

Z is a rv with distribution if and only if E Af Z

for all smo oth functions f where A is an op erator asso ciated with the distribution Thus in

the standard normal case Af xf x xf x x R We will call such an op erator a Stein

operator Next assume Z to have distribution and consider the Stein equation

hx E hZ Af x

or anyrandomvariable W For every smo oth h nd a corresp onding solution f of this equation F

E hW E hZ E Af W Hence to estimate the proximityof W and Z it is sucientto

estimate E Af W for all p ossible solutions of

However in this pro cedure it is not completely clear which characterizing equation for the

distribution to cho ose one could think of a whole set of p ossible equations The aim is to b e able

to solve for a suciently large class of functions htoobtainconvergence in a known top ology

Barb ours Generator Metho d

Akey to ol in Steins theory is the generator metho d develop ed by Barb our Replacing f by f

in the Stein equation for the Standard Normal gives f x xf x hx E hZ If we

equation can be rewritten as A f hx E hZ The key set A f xf x xf x this

advantage is that A is also the generator of a Markov pro cess the OrnsteinUhlenbeck pro cess

with Standard Normal stationary distribution

If we replace f byf f x f x in the Stein equation for the Poisson distribution

wegetf x xf xxf x hx E hZ If weset A f xf x

xf xxf x this equation can b e rewritten as A f xhx E hZ Again we see that

A is a generator of a Markov pro cess an immigrationdeath pro cess with stationary distribution

the Poisson distribution Indeed from we see that A is the generator of a birth and death

pro cess with constant birth or immigration rate and linear death rate i

i i

This also works for a broad class of other distributions Barb our suggested employing for

an op erator the generator of a Markov pro cess So for a random variable Z with distribution

we are lo oking for an op erator A such that E Af Z and for a Markov pro cess fX t g

with generator A and with unique stationary distribution We will call such an op erator A a

SteinMarkov operator for The asso ciated equation will b e called the SteinMarkov equation

Af x hx E hZ

This metho d will b e in the following called the generator method

However for a given distribution there may be various op erators A and Markov pro cesses

with as stationary distributions We will provide a general pro cedure to obtain for a large class

of distributions one such pro cess

In this framework for a b ounded function h the solution to the SteinMarkov equation may

b e given by f x T hx E hZ dt where Z has distribution X is a Markov pro cess

t t

with generator A and stationary distribution and T hxE hX jX x

t t

Stein Op erators and SteinMarkov Op erators

We summarize some SteinMarkov op erators A and Stein op erators A for some wellknown distri

For more details see butions in the next tables where we set q p

and references cited therein

Table Stein op erators

Name Notation Af x

Normal N f x xf x

Poisson P f x x f xxf x

Gamma Ga xf xa xf x

Pascal Pa x f x x xf xxf x

Binomial BinN p pN xf x pN xqxf xqxf x

Table SteinMarkov op erators

Name Notation Af x

Normal N f x xf x

Poisson P f x xf x

x Gamma Ga xf xa xf

Pascal Pa x f x xf x

Binomial BinN p pN xf x qxf x

Note that the SteinMarkov and the Stein op erators are of the form

Af xsxf x xf xandAf xsxf x xf x

in the continuous case and of the form

Af x sxrf x xf x

sx xf x sx xf xsxf x

Af x sxrf x xf xsx xf x sxf x

in the discrete case where the sx and x are p olynomials of degree at most two and one

resp ectively and rf xf x f x Furthermore the ab ove distributions satisfy equations

with the same ingredients sx and x In the continuous case the densityor weight function

x of the distribution satises the dierential equation sxx xx and in the discrete

case the probabilities PrZ xp satisfy the dierence equation sxp xp

x x x

This brings us to the Pearson class of continuous distributions and Ords family of discrete

distributions

Steins Metho d for Pearsons and Ords Family

In K Pearson intro duced his famous family of frequency curves The elements of this family

arise by considering the p ossible solutions to the dierential equation

q xx x a x

b b x b x px

There are in essence ve basic solutions dep ending on whether the p olynomial px in the de

nominator is constant linear or quadratic and in the latter case on whether the discriminant

D b b b of px is p ositive negative or zero It is easy to show that the Pearson family

is closed under translation and scale change Thus the study of the family can be reduced to

dierential equations that result after an ane transformation of the indep endentvariable

If degpx then x can be reduced after change of variable to a Standard Normal

density

If degpx then the resulting solution maybeseento b e the family of Gamma distri

butions

If degpx and D then the densityisof the form xCx expx where

C is the appropriate normalizing constant

If degpx and D then the density x can be brought into the form x

C x exp arctan x where again C is the appropriate normalizing constant in

particular the tdistributions are a rescaled subfamily of this class

If degpx and D the density x can b e broughtinto the form xCx

x where C is the appropriate normalizing constant the Beta densities clearly b elong to

this class

In what follows we will supp ose that in the continuous case we have a distribution on an

interval a b with a and b p ossible innite with a second moment a distribution function F x

and a density function x but we will nd it more convenient to work with an equivalentform

of the dierential equation We assume that our density function x satises

sxx xx

for some p olynomials sx of degree at most two and x of exact degree one The equivalence

andq x x s x between and can easily b e seen by setting pxsx

Furthermore wewillmake the following assumptions on sx

sx axb and sasb if a b is nite

y dy we have that x is not a constant and is a Note that b ecause x and

decreasing linear function Indeed supp ose it was nonconstant and increasing and denote the only

zero of xby l then wewould haveforxl

Z Z

x x

sy y dy sxx y

y dy y dy

x x x

a a

which is imp ossible For a similar reason x can not b e constant

The only zero of x l say is just E Z where Z has distribution This can be seen by

calculating

Z Z

b b

E Z y y dy sy y dy sy y j

a a

Ords family comprises all the discrete distributions that satisfy

p p p a a x q x

x x x

p p b b x b x px

x x

where p PrZ x and x takes values in S fa a b bg with a b p ossible innite

and where we set for convenience p for x S

So we supp ose that we have a discrete distribution on S with a nite second moment but

also here we prefer to work with an equivalent form of the dierence equation We assume

that our probabilities p satisfy

sxp xp

x x

for some p olynomials sx of degree at most two and x of exact degree one The equivalence

between and can easily b e seen by using sxp sx p p sx and setting

x x x

pxsx and q x x sx In this way we can also rewrite the dierence equation

p sx x

p sx

Furthermore wewillmake the following assumptions on sx

sa if a is nite sx ax b

Note again that b ecause p and p that x is not a constant and is a decreasing

x i

linear function and that the only zero of x l say is just the mean of the distribution For a

complete description of Ords familywe refer to

We start with a characterization of a distribution with density x satisfying We set

C equal to the set of all real b ounded piecewise continuous functions on the interval a b and set

C equal to the set of all real continuous and piecewise continuously dierentiable functions f on

the interval a b for which the function g z jsz f z j j z f z j is b ounded We have the

following theorem

Theorem Suppose we have a random variable X on a b with density function x and nite

second moment and that x satises Then x x if and only if for al l functions

f C E sX f X X f X

Pro of First assume X has density function x Then

Z Z

b b

E sX f X X f X f xsxxdx f x xxdx

a a

Z Z

b b

f xsxx dx f x xxdx f xsxxj

a a

Z Z

b b

xdx f x xxdx f x x

a a

Conversely supp ose wehave a random variable X on a b with density function x and nite

second momentsuch that for all functions f C E sX f X X f X Then

E sX f X X f X sxf x xf x xdx

Z Z

b b

xf x xdx sx x f xdx f xxsxj

a a

Z Z

b b

sx x f xdx xf x x dx

a a

R R

b b

But this means that for all functions f C sx x f xdx x xf xdx So x

a a

satises the dierential equation sx x xx which uniquely denes the density x

In conclusion wehave xx

In the discrete case wehave a similar characterization of a distribution with probabilities p

satisfying and We set C equal to the set of all realvalued functions f on the integers

such that f is zero outside S and the function g x jsxrf xj j xf xj is b ounded and

where rf xf x f x We have the following theorem we omit the prove of it b ecause

it is completly along the same lines as the pro of of the continuous version

Theorem Suppose we have a discrete random variable X on the set S with probabilities PrX

xp and nite second moment and that p satises Then p p if and only if for al l

x x x x

X functions f C E sX rf X X f

In Steins metho d we wish to estimate the dierence between the exp ectation of a function

h C with resp ect to a continuous random variable W and E hZ where Z has distribution

To do this we solve rst the socalled Stein equation for the distribution

sxf x xf xhx E hZ

The solution of this Stein equation is given in the next prop osition

Prop osition The Stein equation for the distribution and a function h C has as solution

f x hy E hZ y dy

sxx

hy E hZ y dy

sxx

when axb and f elsewhere This f belongs to C

h h

Pro of First note that

hx E hZ sxx

f hy E hZ y dy x

sxx sx

xx hx E hZ

hy E hZ y dy

sxx sx

x hx E hZ

hy E hZ y dy

sx x sx

Next we just substitute the prop osed solution into the left hand side of the Stein equation

This gives

sxf x xf x hy E hZ y dy hx E hZ

sxx

hy E hZ y dy

sxx

hx E hZ

The second expression for f follows from the fact

Z Z

b x

hy E hZ y dy hy E hZ y dy

x a

To prove that for h C we have f C we need only show that g x jsxf xj

f xj axb is b ounded Wehaveforxl j x

g x jsxf xj j xf xj

hy E hZ y dy hx E hZ

sxx

hy E hZ y dy

sxx

jjhx E hZ jj

y y dy jjhx E hZ jj

sxx

jjhy E hZ jj

y y dy

sxx

jjhx E hZ jj

where jjf xjj sup jf xj A similar result for x l follows from This proves our

axb

prop osition

In the discrete case we wish to estimate the dierence between the exp ectation of a b ounded

function h with resp ect to a random variable W and E hZ where Z has distribution To do

this we solve rst the socalled Stein equation for the discrete distribution

sxrf x xf xhx E hZ

This Stein equation is solved in the next prop osition The pro of is completely of the same structure

as in the continuous case

Prop osition The Stein equation for the distribution and a bounded function h has as

solution

x b

X X

hi E hZ p hi E hZ p f x

i i

sx p sx p

x x

ia ix

when a xb and f elsewhere Furthermore this f belongs to C

h h

Again supp ose rst we are in the continuous setting The next step is to estimate

E sW f W W f W

and hence E hW E hZ To show the quantity in is small it is necessary to use the

structure of W In addition we might require certain smo othness conditions on f which would

translate into smo othness conditions on h by the following lemma

is the only zero of We will need the following Set as b efore l E Z and remember that l

p ositive constant

M maxF l F l

l sl

Lemma Suppose h C and let f be the solution of the Stein equation given by Then

jjf xjj M jjhx E hZ jj

Pro of Note that x is p ositive and decreasing in a l and negative and decreasing in l b So

wehave for xl

Z Z Z

x x x

sxx y

y dy sy y dy F x y dy

x x x

a a a

Similarlyfor xlwehave

sxx

F x y dy

Now for x l

y dy

jf xj jjhx E hZ jj hy E hZ y dy

sxx sxx

Similarly for x l

y dy

jf xj hy E hZ y dy jjhx E hZ jj

sxx sxx

R R

x b

Next we prove that the expressions y dy sxx and y dy sxx of and

a x

attain there maximum at x l Toshow this we calculate

y dy

sxx xsx sx

and

y dy

sxx xsx sx

Next we use and to obtain

R R

x b

y dy y dy

a x

forx l and forx l

sxx sxx

In conclusion wehave jjf xjj M jjh E hZ jj

In the discrete setting we wish to estimate

E sW rf W W f W

h h

and hence E hW E hZ We again might require certain smo othness conditions on f which

would translate into smo othness conditions on h by the following lemma Let xcx d c

and l dc b e the only zero of Nowwe need the following p ositive constant

bl c

X X

M max p p

i i

p sbl c

bl c

ibl c

Lemma Suppose h is a bounded function and let f be the solution of the Stein equation given

by Then jjf xjj M jjhx E hZ jj where jjf xjj sup jf xj

axb

The pro of is complete analoguous to the continuous case

Another inequality for distributions in Pearsons class is given in the next lemma

Lemma Suppose h C and let f be the solution of the Stein equation given by Then

jjf xjj jjsxjj jjhx E hZ jj

Pro of Because

hx E hZ x

hy E hZ y dy f x

sx x sx

x hx E hZ

hy E hZ y dy

sx x sx

wehave for x l

xF x

jf xjjjhx E hZ jj jjhx E hZ jj

sx x sx sx

where we used for the last inequality

Similarly for x l wehave

x F x

jf xjjjhx E hZ jj jjhx E hZ jj

sx x sx sx

This ends the pro of

Aslightly dierentversion for Ords class is given in the next lemma

Lemma Suppose h is a bounded function and let f be the solution of the Stein equation given

by Then

jjrf xjj max sup sx jjhx E hZ jj

j aj

axb

Pro of For x a and x bwehave

ha E hZ p ha E hZ jjhx E hZ jj

jrf aj

sa p a j aj

hb E hZ

jrf bj jf b j sup sxjjhx E hZ jj

h h

axb

For axb the pro of is completely similar as in the continuous case

Orthogonal Polynomials and Barb ours Markov Pro cess

After having considered the close relation b etween the dening dierence and dierential equations

of the distributions involved and their SteinMarkov op erators we bring into the analysis some

related orthogonal p olynomials The key link in the continuous case will b e the dierential

equation of hyp ergeometric typ e which is satised by the classical orthogonal p olynomials of a

y xy y where sx and x are p olynomials of at most continuous variable sx

second and rst degree resp ectively and is a constant

In the discrete case the link will be the dierence equation of hyp ergeometric typ e which is

satised by the classical orthogonal p olynomials of a discrete variable sxry x xy x

y x where sxand xare again p olynomials of at most second and rst degree resp ec

tively and is a constant

Let Q x be the orthogonal p olynomials of degree n with resp ect to the distribution then

the Q x satisfy suchanequationofhyp ergeometric typ e for some sp ecic constants But

n n

this means that wehave

AQ x Q x

n n n

In this waywe can formally solve the SteinMarkov equation

Af hx E hZ

with the aid of orthogonal p olynomials Let F xPrZ xtheinvolved distribution function

and d Q x dx with S the supp ort of Supp ose hx E hZ a Q x

n n n n

where we can determine the a by

a Q xhx E hZ dF xd n

n n

hx E hZ dF x But then for a given h the solution of is Note that a Q x

given by

f x Q x

Indeed wehave

X X X

a a

n n

Af xA a Q xhx E hZ Q x AQ x

n n n n

n n

n n n

Another place where the orthogonal p olynomials app ear is in Barb ours op erator metho d Recall

that we are considering some distribution continuous or discrete together with a SteinMarkov

op erator A of a Markov pro cess X say

In the discrete case the op erator A has the form

Af x sxrf x xf x

xf xsxf x sx xf x sx

which is the op erator of a birth and death pro cess with birth and death rates sn

nand sn resp ectivelyif

n n n

The orthogonal p olynomials Q xof satisfy

AQ x sx xQ x sx xQ xsxQ x

n n n n

Q x

n n

Supp ose we have a duality relation of the form Q xQ and that Q is a p olynomial

n x n x

of degree x Then can b e written as

Q sx xQ sx xQ sxQ

x n x n x n x n n

x and n we clearly see that this results in a three term recurrence equation Interchanging the role of

Q sn nQ sn nQ snQ

x n x n x n x n x

By Favards Theorem the Q must be orthogonal p olynomials with resp ect to some distribution

say Furthermore note that these p olynomials are the birthdeath p olynomials of the birth and

death pro cess X According to the Karlin and McGregor sp ectral representation wehave

P t PrX j jX i e Q Q dF y

ij t j i y j y

where and j and F x is the distribution function

j j j

Note that this distribution is com The stationary distribution is given by r

i i

pletely dened by the fraction of successive probabilities

r i i

i i

r i

i i

Comparing this with we see that the stationary distribution is indeed our starting distribution

In the examples wewillwork out this pro cedure for some wellknown discrete distributions

In the continuous case the op erator A has the form Af x sxf x xf x which

is the op erator of a diusion with drift co ecient x x and diusion co ecient x

sx Recall that the p olynomials y x which are orthogonal with resp ect to satisfy

y x n As in the discrete case the orthogonal p olynomials involved are Ay x

n n n

eigenfunctions and app ear in the sp ectral representation as shown in the examples section

Examples

The Standard Normal distribution N and the OrnsteinUhlenb eck pro cess The Normal

distribution Nm with mean m R and variance has a density function x m

expx m x R Clearly we have x m x m m xx

q xpx and thus sxpx and xq xs xm x So the Stein equation for

the Nm distribution is given by f xm xf xhx E hZ The Stein op erator

is given by Af x f xm xf x and M So for the Standard Normal

distribution N Lemma recovers the rst b ound in This case was the starting p ointof

Steins theory Supp ose we have x x x and x sx then we have

p p

nH x where the op erator A is given by Af f x xf x This is the AH x

n n

generator of the the Ornstein Uhlenbeck Process The sp ectral representation for the transition

densityisgiven by

p p

H y pt x y e H x

n n

are where H x is the Hermite polynomial of degree n The Hermite p olynomials H x

n n

orthogonal with resp ect to the Standard Normal distribution we started with

The Gamma distribution Gr and the Laguerre diusion The Gamma distribution

r x r

Gr with r has a density function x r e x r x Clearly

and thus sx px x and we have x rx r r xx q xpx

x q xs x r x So the Stein equation for the Gr distribution is given by

xf xr xf xhx E hZ and Af xf xr xf x which is the generator

of the socalled Laguerre diusion and has a sp ectrale representation given by

r r y

y e n

nt r r

pt x y e L xL y

n n

r n r

with L the Laguerre p olynomial of degree n It is no coincidence that also here the orthogonal

p olynomials involved are orthogonal with resp ect to the Gamma distribution we started with

The Beta distribution B and the Jacobi diusion The Beta distribution B

on with parameters has a density function x x x B

x Clearly wehave

x q x x

x xx px

and thus sxpx xx and xq xs x x So the Stein equation

for the B distribution is given by

x xf x xf xhx E hZ

This Stein equation seems to be new Supp ose we have x x x and

x xx where x and Then we encounter the generator of the Jacobi

xxf x xf x In this case the sp ectral expansion is in diusion Af

terms of the Jacobi polynomials P x P x

y y

nn t

e P x P y pt x y

n n n

where

B n nn

n n

and B is the Beta function

The Students tdistribution t with n f g degrees of freedom has a density function

n x

x n x R

n n n

Clearly wehave

n xn q x x n

x n x n px

and thus clearly sxpxx n and xq xs xn nx This means

that the Stein equation for the t distribution is given by

x n

f x xf xhx E hZ

n n

This Stein equation seems to b e new Note that Lemma gives us a useful b ound on f namely

jjf jj jjh E hZ jj

The Poisson distribution P is given by the probabilities p e x x

f g An easy calculation gives sxx and x x So the Stein op erator for the Pois

son distribution P is given by Af xxrf x xf xf x xf x which is the

same as in This case was studied by and many others The Poisson distribution

P has a SteinMarkov op erator A given by Af xf x x f xxf x

This is the op erator of a birth and death pro cess on f g with birth and death rates

and n n resp ectively This birth and death pro cess is the immigrationdeath pro cess with

a constant immigration rate and unit per capita death rate The birthdeath p olynomials

Q x for this pro cess are recursively dened by the relations

xQ xQ x nQ xnQ x

n n n n

together with Q x and Q x The p olynomials whichare orthogonal with resp ect to

which satisfy the following the Poisson distribution P are the Charlier p olynomials C x

equation of hyp ergeometric typ e

nC x C x nC x nC x

n n n n

and are selfdual ie C x C n Using this duality relation we obtain the three term

n x

recurrence relation of the Charlier p olynomials

C n nC n nC n C n n

x x x x

But this is after interchanging the role of x and n exactly of the same form as so we conclude

that Q xC x In this way using Karlin and McGregors sp ectral representation we

n n

can express the transition probabilities of our pro cess X as

j x

P t PrX j jX i e C x C x e

ij t i j

j x

The Binomial distribution BinN p on f Ng with parameter p is given

x N x

by the probabilities p p q x f Ng where q p Here sx qx

and x pN x So the Stein op erator for the BinN p distribution is given by Af x

qxrf xpN xf x pN xf x qxf x The Binomial distribution BinN p

p has a SteinMarkov op erator Agiven by

Af x pN xf x pN xqxf xqxf x

where q p This is the op erator of a birth and death pro cess on f Ng with birth

and death rates pN n and qn n N resp ectively also called the Ehrenfest

n n

Mo del In this case the birthdeath p olynomials Q x are recursively dened by

xQ xpN nQ x pN nqnQ xqnQ x

n n n n

x together with Q x and Q

The p olynomials which are orthogonal with resp ect to the Binomial distribution BinN p are

the Krawtchouk p olynomials K x N p n N They satisfy the following equation of

hyp ergeometric typ e

nK x N ppN nK x N p pN nqnK x N pqnK x N p

n n n n

and are selfdual ie K x N p K n N p This duality relation leads to the three term

n x

recurrence relation of the Krawtchouk p olynomials

nK n pN nK n N p pN nqnK n N pqnK n N p

x x x x

After interchanging the role of x and nwe conclude that Q xK x N p In this way using

n n

Karlin and McGregors sp ectral representation we can express the transition probabilities of

our pro cess X as

N N

j j xt x N x

P t p q e K x N pK x N p p q

ij i j

j x

and is given by p The Pascal distribution Pa with parameters

x f g

So sxx and x x The Stein op erator for the Pa distribution is thus

given by Af x xrf x xf x xf x xf x The Pascal

distribution Pa has a SteinMarkov op erator Agiven by

Af xx f x x xf xxf x

which is the op erator of the ab ove describ ed linear birth and death pro cess on f g with birth

and death rates n and n n resp ectively The birthdeath p olynomials

n n

involved are dened by

xQ xn Q x n nQ xnQ x

n n n n

together with Q x and Q x

The Meixner p olynomials M x n are orthogonal with resp ect to the Pascal

distribution Pa they satisfy the following equation of hyp ergeometric typ e

x n nM x nM x nM x n M

n n n n

and are selfdual ie M x M n Using this duality relation we obtain the three

n x

term recurrence relation of the Meixner p olynomials

nM n n M n n nM n nM n

x x x x

Interchanging the role of x and n we conclude that Q x M x With the sp ectral

n n

representation we express the transition probabilities of X as

xt x

P t e M x M x x

ij i j x

The Hyp ergeometric distribution Hyp N with parameters N N and N a non

negative integer is given by p x f Ng An easy calculation gives

x N x N

x N x and x N x So the Stein op erator for the Hyp N sx

distribution is given by

Af x x N xrf xN xf x

N x xf x x N xf x

This Stein equation seems to b e new The Hyp ergeometric distribution Hyp N has a Stein

Marko v op erator A given by

Af x xf x N x xx N xf xx N xf x

This is the op erator of a birth and death pro cess on f Ng with quadratic birth and death

rates

N n nand n N n n N

n n

resp ectively whih was studied in The birthdeath p olynomials involved are recursiv ely dened

by the relations

xQ xN n nQ x

n n

N n nn N nQ xn N nQ x

n n

together with Q x and Q x The Hahn polynomials Q x N

are orthogonal with resp ect to Hyp ergeometric distribution and satisfy the following equation of

hyp ergeometric typ e

nn Q x N

N x xQ x N

N x xxx N Q x N

N xx N Q x

Furthermore wehave the duality relation

Q x NR N

n x n

where the R are the Dual Hahn p olynomials and n In what follows wewill

x n

often write for notational convenience R instead of R N

x n x n

Using this duality relation we obtain the three term recurrence relation of the Dual Hahn

p olynomials

R N x xR

n x n x n

N x xxx N R xx N R

x n x n

But this is after interchanging the role of x and n of the same form as so we conclude that

Q xR x N Finally using Karlin and McGregors sp ectral representation

n n

we can express the transition probabilities of our pro cess X as

N N

N N x

x x

j N j

P t e R R

ij i x j x

x x

x N

Acknowledgements

The author thanks JLTeugels for his useful comments and N Bingham for having a lo ok at an

early draft

References

WJ Anderson ContinuousTime Markov Chains An ApplicationsOriented Approach

SpringerVerlag NewYork

AD Barb our Equilibrium distributions for Markov p opulation pro cesses Adv Appl Prob

AD Barb our Steins metho d and Poisson pro cess convergence In A celebration of Applied

Probability Editor J Gani J Appl Prob A

AD Barb our Steins metho d for diusion approximations Probab Theory Related Fields

AD Barb our and GK Eagleson Poisson approximation for some statistics based on ex

changeable trials Adv Appl Prob

AD Barb our L Holst and S Janson Poisson Approximation Clarendon Oxford

RN Bhattacharya and EC Waymire Sto chastic Pro cesses with Applications John Wiley

Sons New York

LHY Chen Poisson approximation for dep endent trials Ann Probab

TS Chihara An Intro duction to Orthogonal Polynomials Gordon and Breach New York

LondonParis

H Dette On a generalization of the Ehrenfest urn mo del J Appl Prob

P Diaconis and S Zab ell Closed form summation for classical distributions variations on a

theme of de Moivre Statistical Science

GK Eagleson A duality relation for discrete orthogonal systems Studia Scientiarum Math

ematicarum Hungarica

MEH Ismail J Letessier and G Valent Linear birth and death mo dels and asso ciated

Laguerre and Meixner p olynomials Journal of Approximation Theory

NL Johnson S Kotz and AW Kemp Univariate Discrete Distributions nd edition

WileyNewYork

S Karlin and JL McGregor The dierential equations of birthanddeath pro cesses and the

Stieltjes moment problem Trans Amer Math Soc

S Karlin and JL McGregor The classication of birth and death pro cesses Trans Amer

Math Soc

S Karlin and JL McGregor Linear growth birth and death pro cesses J Math Mech

S Karlin and JL McGregor Random walks Il linois J Math

S Karlin and JL McGregor Ehrenfest Urn Mo dels J Appl Prob

S Karlin and HM Taylor A First Course in Sto chastic Pro cesses nd edition Academic

Press New York

S Karlin and HM Taylor A Second Course in Sto chastic Pro cesses Academic Press New

York

R Ko ekoek and RF Swarttouw The Askeyscheme of hyp ergeometric orthogonal p olyno

mials and its q analogue Rep ort Delft UniversityofTechnology

W Ledermann and GEH Reuter Sp ectral theory for the dierential equations of simple

birth and death pro cesses Philos Trans Roy Soc London Ser A

HM Luk Steins Metho d for the Gamma Distribution and Related Statistical Applications

PhDThesis University of Southern California Los Angeles USA

AF Nikiforov SK Suslov and VB Uvarov Classical Orthogonal Polynomials of a Discrete

Variable SpringerVerlag BerlinHeidelb erg

PR Parthasarathy RB Lenin W Schoutens and W Van Assche A birth and death pro cess

related to the RogersRamanujan Continued Fraction J Math Anal Appl

W Schoutens Sto chastic Pro cesses in the Askey Scheme Do ctoral Thesis KULeuven

Leuven

C Stein A bound for the error in the normal approximation to the distribution of a sum

of dep endent random variables Proc Sixth Berkeley Symp Math Statist Probab

Univ California Press Berkeley

C Stein Approximate Computation of Exp ectations IMS Lecture Notes Monograph Se

ries Volume IMS Hayw ard California

W Van Assche PR Parthasarathy and RB Lenin Sp ectral representation of certain nite

birth and death pro cesses The Mathematical Scientist to appear