Chapter 2 Continuous Distributions

Home , Beta distribution, Gamma distribution, Weibull distribution

RS – 2 – Continuous Distributions

Continuous random variables: Review • For a continuous random variable X the probability distribution is described by the probability density function f(x), which has the following properties : 1. f(x) ≥ 0

 2.  fxdx  1.  b 3. P aXb fxdx . a b  fxdx  1. PaXb fxdx .  a 

1 RS – 2 – Continuous Distributions

Continuous random variables: Review

• Continuous distributions allow for a more elegant mathematical treatment.

• They are especially useful to approximate discrete distributions. Continuous distributions are used in this way in most economics and finance applications, both in the construction of models and in applying statistical techniques.

• Most used continuous distributions: -Uniform -Normal - Exponential -Weibull - Gamma

The Uniform distribution from a to b

Abraham de Moivre (1667-1754)

2 RS – 2 – Continuous Distributions

Uniform Distribution

• A random variable, X, is said to have a Uniform distribution from a to b if X is a continuous RV with probability density function f(x):  1  axb  fx  ba  0otherwise • PDF: Uniform distribution (from a to b)

0.4

0.3

0.2

0.1

0 051015

Uniform Distribution: CDF

• The CDF, F(x), of the uniform distribution from a to b:

 0 x  a   xa 0.4 Fx PX x a  x b f x ba  1 x  b 0.3 

0.2

0.1 Fx 

0 051015x a b

3 RS – 2 – Continuous Distributions

Uniform Distribution function: Graph of F(x)

 0 x  a   xa F xPXx axb  ba  1 x  b

1 Fx

0.5

0 051015 a b x

Uniform Distribution: Comments

• When a=0 and b=1, the distribution is called standard uniform distribution, which is a special case of the Beta distribution, with parameters (1,1)

• The uniform distribution is not commonly found in nature (but very common in casinos!). Because of its simplicity, it is used in theoretical work (for example, signals/private information are draw from a UD). It is used as prior distribution for binomial proportions in Bayesian statistics.

• It is also particularly useful for sampling from arbitrary distributions. A general method is the inverse transform sampling method, which uses the CDF of the target RV. This method is very useful in theoretical work. Since simulations using this method require inverting the CDF of the target variable, alternative methods have been devised for the cases where the CDF is not known in closed form.

4 RS – 2 – Continuous Distributions

The Normal distribution (Also called Gaussian or Laplacian)

Abraham de Moivre (1667-1754)

Carl Friedrich Gauss (1777–1855) Pierre-Simon, m. de Laplace (1749–1827)

The Normal distribution: PDF

A random variable, X, is said to have a normal distribution with mean  and standard deviation  if X is a continuous random variable with probability density function f(x):

2 1  x   fx e2  2 2

5 RS – 2 – Continuous Distributions

Some characteristics of the Normal distribution

1. Location of maximum point of f(x). parameter 

2 1  x fx e2  2 2 x 2  1 2  2 1 fx  e2  x 0 2 

This equality holds when x.

Therefore, the point  is an extremum point of f(x). (In this case a maximum)

Some characteristics of the Normal distribution

2. Inflection point of f(x).

x   2 1 d  fx  e2  2  x  2 3 dx

x   2 x   2 112 ex2  2  e2  2 2253 

2 x   2  1  2 x   e 2  10 3  2 2 

2 x   x   if  1 o r  1 i.e. x   2 Thus the points  – ,  +  are points of inflection of f(x).

6 RS – 2 – Continuous Distributions

Some characteristics of the Normal distribution

3. The pdf f(x) integrates to 1.  2 1  x  f xdx e2 2 dx 1   2 Proof:  2 1  x To evaluate  edx2 2  2 x   1 Make the substitution z  dz dx   When xz ,  and when xz  ,  .

2 zz22  111 x   edx2  2  edz22  edz 222    

Some characteristics of the Normal distribution

 z2  Consider evaluating  edzc2   Note:

zz22  u 2 2   cedzedzedzedu2 22 z 2          22 zu22  zu   e22 e dudz  e2 dudz    

Make the change to polar coordinates (R, ): z = R sin() and u = R cos()

7 RS – 2 – Continuous Distributions

Some characteristics of the Normal distribution 222 z Hence R zuand tan   u or 22  1 z R zuand   tan  u Using the following result

2  f z,sin,cos u dzdu  f R  R  RdRd   00

22  zu  cedudz2  2   2 222R   2  2  R eRdRde  2 d12 d    00 00 0

Some characteristics of the Normal distribution

 z 2  and cedz 2 2 

2 z 2 x   11 2 1 edz2 e2 dz 22 

8 RS – 2 – Continuous Distributions

Normal distribution: Comments

• The normal distribution is often used to describe or approximate any variable that tends to cluster around the mean. It is the most assumed distribution in economics and finance: rates of return, growth rates, IQ scores, observational errors, etc.

• The central limit theorem (CLT) provides a justification for the normality assumption when n is large.

• The normal distribution is often named after Carl Gauss, who introduced it formally in 1809 to justify the method of least squares. One year later, Pierre Simon (marquis de Laplace) proved the first version of the CLT. ( In some European countries it is called Laplacian.)

Notation: PDF: x ~ N(μ,σ2) and CDF: Φ(x)

The Exponential distribution

Georges Teissier (1900-1972, France) P. V. Sukhatme (1911-1997, India)

9 RS – 2 – Continuous Distributions

Exponential distribution: Derivation • Consider a continuous RV, X with the following properties: 1. P[X ≥ 0] = 1, and 2. P[X ≥ a + b] = P[X ≥ a] P[X ≥ b] for all a > 0, b > 0.

Let X= lifetime of an object. Then, (1) and (2) are reasonable to assume if X = lifetime of an object that does not age. Or (1) and (2) are reasonable for an object that lives a long time and a and b are small.

• The second property implies: PX  a b PX a bX a PX b PX a   Given the object has lived to age a, the probability that it lives b additional units is the same as if it was born at age a. Past history has no effect on x.

Exponential distribution: Derivation

Let F(x) = P[X ≤ x] and G(x) = P[X ≥ x]. Since X is a continuous RV  G(x) = 1 – F(x)

The two properties can be written in terms of G(x): 1. G(0) = 1, and 2. G(a + b) = G(a) G(b) for all a > 0, b > 0.

We can show that the only continuous function, G(x), that satisfies (1) and (2) is the exponential function

10 RS – 2 – Continuous Distributions

Exponential distribution: Derivation

From property 2 we can conclude

Using induction

n Hence putting a = 1. Gn  G 1 

n 1 Also putting a = 1/n. 1 1 n GG1   n      or GGn    1  Finally putting a = 1/m.

n n 1 n GGn 1  G11mm  G mm 

Exponential distribution: Derivation

Since G(x) is continuous  G(x) = [ G(1)]x for all x ≥ 0.

Recall: 0≤G(1)≤1 and G(0)=1, G(∞)=0. If G(1) = 0 then G(x) = 0 for all x > 0 and G(0) = 0 if G is continuous. (a contradiction) If G(1) = 1 then G(x) = 1 for all x > 0 and G(∞) = 1 if G is continuous. (a contradiction)  G(1) ≠ 0, 1 and 0 < G(1) < 1 Let = - ln(G(1)), then G(1) = e-

x  x x Thus Gx G  1  e e

11 RS – 2 – Continuous Distributions

Exponential distribution: Derivation

 00x  Finally Fx 1 Gx     x 10 ex

To find the density of X we use: ex x  0 fx Fx    00x 

A continuous random variable with this density function is said to have the exponential distribution with parameter .

Graphs of f(x) and F(x)

1 f(x)

0.5

0 -20246

1 F(x)

0.5

0 -2 0 2 4 6

12 RS – 2 – Continuous Distributions

Exponential distribution: Alternative Derivation

Let F (x ) = P[X ≤ x] be the CDF of the exponential RV, X. Then, P[X ≥ 0] = 1 implies that F(0) = 0.

Also P[x ≤ X ≤ x + dx|X ≥ x] = λ dx implies

P xX xdx PxXxdxXx   PX x Fx  dxFx   dx 1  Fx Fx dxFx  dFx  or   1 F x  dx dx  

Exponential distribution: Alternative Derivation dF Let’s solve the ODE for the unknown F:   1 F dx 1 1 dF  dx or dF  dx 1  F  1  F  and  ln 1 Fxc  ln 1 F  xc  and 1Fex cxc or FFx    1 e 

Now using the fact that F(0) = 0. Fe0 1cc  0 implies e  1 and c  0 Thus F xe  1x and fxFxe     x

This shows that X has an exponential distribution with parameter .

13 RS – 2 – Continuous Distributions

Exponential distribution: Comments

• The exponential distribution is often used to model the failure time of manufactured items in production lines. Typical example is light bulbs.

•If X denotes the time to failure of a light bulb, then the exponential distribution says that the probability of survival of the bulb decays exponentially fast. More precisely: P(X > x) = λ exp(-λx)

Note: The bigger the value of λ, the faster the decay. This indicates that for large λ¸ the average time of failure of the bulb is smaller. This is indeed true. We will show later that E(X) = 1/λ.

We say X has an exponential distribution with parameter .

Exponential distribution: Comments

• The exponential distribution is also used a distribution for waiting times between events occurring uniformly in time. Typical example is stock trading intervals.

• An interesting feature of the exponential distribution is its memoryless property –the distribution “forgets” the past.

14 RS – 2 – Continuous Distributions

The Weibull distribution A model for the lifetime of objects that do age.

Waloddi Weibull (Sweden, 1887-1979)

The Weibulll distribution

Recall the properties of continuous RV, X, that lead to the Exponential distribution. namely 1. P[X ≥ 0] = 1, and 2. P[x ≤ X ≤ x + dx|X ≥ x] = dx for all x > 0 and small dx.

Suppose that the second property is replaced by: 2. P[x ≤ X ≤ x + dx|X ≥ x] = (x)dx = x –1dx for all x > 0 and small dx.

Again, let X= lifetime of an object. Now, (2) implies that the object does age. If it has lived up to time x, the chance that it dies in the small interval x to x + dx depends both on the length of that interval, dx, and its age x.

15 RS – 2 – Continuous Distributions

Derivation of the Weibulll distribution

A continuous random variable, X satisfies the following properties: 1. P[X ≥ 0] = 1, and 2. P[x ≤ X ≤ x + dx|X ≥ x] = x -1dx for all x > 0 and small dx.

Let F (x ) = P[X ≤ x] be the CDF of the random variable, X. Then P[X ≥ 0] = 1 implies that F(0) = 0. Also P[x ≤ X ≤ x + dx|X ≥ x] = x-1dx implies Px  X x dx Px X x dxX  x  PX x Fx  dxFx    x  1dx 1 Fx Fx dxFx  dFx  or  x  1  1 Fx dx dx  

Derivation of the Weibulll distribution dF We can solve the ODE, for the unknown F:   x  1 1 F dx 1 1 dF  x 1 dx or dF  x 1 dx 1  F 1  F  x   and  ln 1 Fc ln 1 F xc   

  xcxc 1Fe or Fx 1 e

again F(0) = 0 implies c = 0.

  x  Thus Fx 1 e   x  and fx F  x x 1 e x  0

16 RS – 2 – Continuous Distributions

Derivation of the Weibulll distribution   x  and fx F  x x 1 e x  0 F(X) is called the Weibull distribution with parameters .

Special cases: When β = 1, we have the exponential distribution. When β = 2, it mimics the Rayleigh distribution (used in telecoms) When β = 3.5, it is similar to the normal distribution.

Popular Application: Time to failure Let X measure time-to-failure and follow the Weibull distribution. Then, the failure rate is proportional to a power of time. The shape parameter, β, is that power plus one: β<1 (“infant mortality phenomenon”); β=1 (constant failure); and β>1 (aging).

The Weibull density, f(x)

  0.7 ( = 0.9, = 2)

0.6 ( = 0.7,  = 2) 0.5

0.4 ( = 0.5,  = 2) 0.3

0.2

0.1

0 012345

17 RS – 2 – Continuous Distributions

The Gamma distribution An important family of distributions

Karl Pearson ( 1857–1936) Pierre-Simon, m. de Laplace (1749–1827)

18 RS – 2 – Continuous Distributions

The Gamma Function, (x) An important function in mathematics, due to Euler (1729) to interpolate factorials. Current formulation and notation due to Legendre. The Gamma function is defined for x ≥ 0 by  x  ueduxu1 0

uex1 u

 x

Properties of the Gamma Function, (x) 1. (1) = 1   11eduuu  e  e e0  0  0 2. (x + 1) = x(x) x > 0.

  x 1 uexu du  ue xu du 0 0 We will use integration by parts to evaluate uexu du uxu e du wdv wv  vdw where wux and dveduuxu , hence dwxudxve 1 , Thus uexu du ue xu  x u x1 e u du

19 RS – 2 – Continuous Distributions

Properties of the Gamma Function, (x)

  and uexu du ue xu  x u x1 e u du 0 00 or x 1 0 xx 

3. (n) = (n - 1)! for any positive integer n using xxx 1  and  1  1 21111!  32222!  433323!  M

Properties of the Gamma Function, (x)

1 4. 2    1 x uxu1 e du thus 1 u2 e  u du  2 00 Recall the density for the normal distribution x 2 1  fx e2 2 2 If   and   1 then

2 1  x f xe 2 2

222 112xxx Now 1=edx222 2 edx edx  2200

20 RS – 2 – Continuous Distributions

Properties of the Gamma Function, (x)

 2  x  x2 2 Make the substitution u  . Thus  edx = 2 0 2 1  2 1 1 u Hence x  2u2 and du xdx or dx du du . x 2 Also when xu 0, then  0,

1 2  2   x u 1 = edxe2 u dx 1  2 22200

1 and  2 = .

Properties of the Gamma Function, (x)

2!n 5. n 1  2 n!22n

Using x 1xx  .

3 111 2222  =.

53331 22222  =.

755531 222222  =.

21nn11 21 53 222222n  = L  2!nn  2!   2!2nnnn !22 n

21 RS – 2 – Continuous Distributions

Graph: The Gamma Function

25 (x) 20

0 012345 x

The Gamma distribution

Let the continuous random variable X have density function:

   1 x  xe x 0 fx      00x  where  > 0.

Then, X is said to have a Gamma distribution with parameters  and ., denoted as X ~ Gamma(, or Γ(, 

We can think of the Gamma distribution as a sum of  (independent) exponential distributions

22 RS – 2 – Continuous Distributions

Graph: The gamma distribution

0.4 ( = 2,  = 0.9)

0.3 ( = 2,  = 0.6)

( = 3,  = 0.6) 0.2

0.1

0 0246810

The Gamma Distribution: Comments 1. The set of gamma distributions is a family of distributions (parameterized by  and ).  f (x)  x 1ex x  0 () 2. Contained within this family are other distributions: a. The Exponential distribution – when  = 1, the gamma distribution becomes the exponential distribution with parameter . It is used to model time to failure or waiting times between events occurring uniformly in time.

b.The Chi-square distribution – when  = /2 and  = ½, the gamma distribution becomes the chi- square (2) distribution with  degrees of freedom. Later, we will see that sum of squares of independent N(0,1) variates have a chi-square distribution, with degrees of freedom given by the number of independent terms in the sum of squares.

23 RS – 2 – Continuous Distributions

The Gamma Distribution: Comments

The Exponential distribution:

  ex  x  0 fx   00x 

The Chi-square, 2, distribution with  degrees of freedom (or just χ):

  1 2   1 1 x  2 xe22 x 0 fx     2   00x 

  2 x  1 22  xe x 0  2    2  2   00x 

The Gamma Distribution: Comments

Graph: The  distribution ( = 4) 0.2

( = 5)

0.1 ( = 6)

0 0481216

24 RS – 2 – Continuous Distributions

The Gamma Distribution: Comments 3. Convergence to a Normal distribution. When α is large, the gamma distribution converges to a normal distribution with μ = αλ-1 and σ2 = αλ-2.

4. Summation property. If xi ~ Γ( , and independent  Σi xi ~ Γ(Σi  , 5. Scaling. If x ~ Γ(,   kx ~ Γ(, k  6. Typically used to model waiting times. 7. Other related distributions: a. Inverse Gamma distribution If X has a Γ(α, λ) distribution, then 1/X has an inverse-gamma distribution with parameters α and λ-1. b. The Beta distribution

If X and Y are independently distributed Γ(α1, λ) and Γ(α2, λ) respectively, then X/(X + Y) has a beta distribution with parameters α1 and α2.

The Gamma Distribution: Application of 2 properties - N

2 • Suppose xi ~ 1 –or xi ~ Γ(½, ½). Suppose they are i.i.d. We want to know the distribution of the sum of Σi xi = x1 +...+ xN.

Summation property. If xi ~ iid Γ(,   Σi xi ~ Γ(Σi , 

2 Then, Σi xi ~ Γ(Σi ½, ½Γ(N/2, ½or N .

2 Note: The degrees of freedom tell us the number of independent 1 variables we are adding.

If interested in the mean of the xi‘s, then using the scaling property: _ N x N 1 x  i ~ ( , ) i1 N 2 2N

25 RS – 2 – Continuous Distributions

The Gamma Distribution: Application of properties - Exponential

Suppose xi ~Exp()–or xi ~ Γ(1, . Suppose they are i.i.d.

The parameter = 1/λ can be estimated by ̅, where _ N x x  i i1 N We want to derive the distribution of .

is a scaled sum of n independent Γ(1, ) RV. That is,

~ Γ(N, /N)