Continuous RVs Uniform Distribution, from first principals

Lets define a continuous that has some constant value c between a and b where a ≤ b. What is the pdf? Lecture 13: Continuous Random Variables ( c if a ≤ x ≤ b f (x) = 0 otherwise Statistics 104 Z ∞ f (x) = 1 −∞ Colin Rundel Z b c = 1 a February 29, 2012 b cx|a = 1 cb − ca = 1 1 c = b − a ( 1 if a ≤ x ≤ b f (x) = b−a 0 otherwise

Statistics 104 (Colin Rundel) Lecture 13 February 29, 2012 1 / 20

Continuous RVs Continuous RVs Uniform Distribution - CDF Uniform Distribution - E(X ), Var(X )

( 1 if a ≤ x ≤ b Z b 2 b f (x) = b−a x x 0 otherwise E(X ) = dx = a b − a 2(b − a) a b2 − a2 b + a = = Z x 2(b − a) 2 P(X ≤ x) = F (x) = f (x) dx −∞ x Z 1 Z b 2 3 b = dt 2 x x E(X ) = dx = a b − a b − a 3(b − a) x a a t 3 3 2 2 = b − a b + ba + a b − a = = a 3(b − a) 3 x − a = b − a b2 + ba + a2 (b + a)2 Var(X ) = E(X 2) − E(X )2 = −  3 4 0 if x < a 2 2 2 2 2 2 2  4b + 4ba + 4a 3b + 6ab + 3a b − 2ba + a (b − a) F (x) = x−a = − = = b−a if a ≤ x ≤ b 12 12 12 12 1 if x > b

Statistics 104 (Colin Rundel) Lecture 13 February 29, 2012 2 / 20 Statistics 104 (Colin Rundel) Lecture 13 February 29, 2012 3 / 20 Continuous RVs Continuous RVs Uniform Distribution - mgf, raw moments Probability density function weirdness

Let X = U2 where U ∼ Unif(0, 1) what are the cdf and pdf of X ? M (t) = E(etX )  X 0 if x < 0 b √ √ Z b etx etx 2 = dx = F (x) = P(X ≤ x) = P(U ≤ x) = P(U ≤ x) = x if 0 ≤ x ≤ 1  a b − a t(b − a) a 1 if x > 1 etb − eta = 0 if x < 0 t(b − a) d  √1 f (x) = F (x) = 2 x if 0 ≤ x ≤ 1 dx  0 n 0 if x > 1 µn = E(X ) Z b n n+1 b x x What is the value of f (x) when x = 0 + ? What is P(X = 0 + )? = dx = a b − a n(b − a) a bn+1 − an+1 = n(b − a)

Statistics 104 (Colin Rundel) Lecture 13 February 29, 2012 4 / 20 Statistics 104 (Colin Rundel) Lecture 13 February 29, 2012 5 / 20

Continuous RVs Continuous RVs Triangle Distribution Triangle Distribution - CDF

The figure below describes a continuous uniform 2d probability distribution, Let X between the distribution of x-coordinates between −a

a=0.5 and a. 1.0 a=1 a=2 a=3 a=4 Find the pdf, cdf, E(X ), Var(X ). 0.8 0.6 F(x) c 0.4 0.2 0.0

−4 −2 0 2 4

x

-a a Statistics 104 (Colin Rundel) Lecture 13 February 29, 2012 6 / 20 Statistics 104 (Colin Rundel) Lecture 13 February 29, 2012 7 / 20 Continuous RVs Continuous RVs Exponential Distribution, cont.

Let X be a random variable that reflects the time between events which occur continuously with a given rate λ, X ∼ Exp(λ) Probability density function: Cumulative distribution function: f (x|λ) = λe−λx P(X ≤ x) = F (x|λ) = 1 − e−λx

 t −1  λ  M (t) = 1 − = X λ λ − t

E(X ) = λ−1 Var(X ) = λ−2 log 2 (X ) = λ Memoryless property - P(X > s + t|X > s) = P(X > t)

Statistics 104 (Colin Rundel) Lecture 13 February 29, 2012 8 / 20 Statistics 104 (Colin Rundel) Lecture 13 February 29, 2012 9 / 20

Continuous RVs Continuous RVs Minimum of Independent Exponential RVs Minimum of Independent Exponential RVs, cont.

So far we have just discussed simple functions of random variables With a little bit of thought you should be able to see that X > a can only involving things like addition, subtraction, multiplication, etc. In many occur if both X1 > a and X2 > a therefore practical applications we may want to know something like the following: P(X > a) = P(X1 > a and X2 > a) = P(X1 > a)P(X2 > a) = [1 − P(X1 ≤ a)][1 − P(X2 ≤ a)] There are two busses whose arrival times have independent exponential = [1 − (1 − e−λ1a)][1 − (1 − e−λ2a)] distribution with rates λ and λ , what is the distribution of the time I 1 2 = e−λ1ae−λ2a have to wait until one of the two busses arrives? 1 − P(X ≤ a) = e−(λ1+λ2)a P(X ≤ a) = 1 − e−(λ1+λ2)a In this example we have the arrival time of bus one given by X1 ∼ Exp(λ1) and the arrival time of bus two given by X1 ∼ Exp(λ1) and my waiting which is the CDF for an Exponential distribution with λ = λ1 + λ2. time is given by X where X = min(X1, X2). What is the distribution of X? Therefore, X ∼ Exp(λ1 + λ2).

This result can easily be generalized such that if X1,..., Xn are independent and exponentially distributed with λ1, . . . , λn then

min(X1,..., Xn) ∼ Exp(λ1 + ··· + λn)

Statistics 104 (Colin Rundel) Lecture 13 February 29, 2012 10 / 20 Statistics 104 (Colin Rundel) Lecture 13 February 29, 2012 11 / 20 Continuous RVs Continuous RVs Raw Moments of Exponential Distribution Generalizing the Factorial

We know we can find E(X n) using the moment generating function but We have just shown the following that when X ∼ Exp(λ): for some distributions we can find a simpler result. n! Assume that n ≥ 1 then for X ∼ Exp(λ) E(X n) = λn Z ∞ Z ∞ E[X n] = X nf (x) dx = X nλe−λx dx Let set λ = 1 and define an new value α = n + 1 −∞ 0 ∞ Z ∞ λe−λx = −x ne−λx − nX n−1 dx E(X α−1) = (α − 1)! 0 −λ 0 Z ∞ n Z ∞ λe−λx n α−1 −x = (0 − 0) + X n−1 dx = E(X n−1) x e dx = (α − 1)! λ 0 λ 0 Z ∞ As we know that E(X 0) = 1 then we can use the recurrence relationship Γ(α) ≡ xα−1e−x dx = (α − 1)! to see that 0 n! n Using a tradition definition of the factorial it only makes sense when n ∈ E(X ) = n N λ but we can use this new definition of the gamma function Γ(α) for any + Integration by parts: R f (x)g 0(x)dx = f (x)g(x) − R f 0(x)g(x)dx or R u dv = uv − R v du α ∈ R

Statistics 104 (Colin Rundel) Lecture 13 February 29, 2012 12 / 20 Statistics 104 (Colin Rundel) Lecture 13 February 29, 2012 13 / 20

Continuous RVs Continuous RVs Negative Generalizing the Negative Binomial Distribution

Let X be a random variable reflecting the total number of successes before We have previously shown a generalization of the factorial function to the th + the r failure where each trial is an independent Bernoulli trial with p postive real numbers (R ), if we replace the factorial function in the probability of success. Then the probability of k successes is given by the Negative Binomial with the gamma function we can extend the Negative Binomial distribution, X ∼ NB(r, p) distribution to positive real values of r.

! ! k + r − 1 k + r − 1 P(X = k) = f (k|r, p) = pk (1 − p)r P(X = k) = f (k|r, p) = pk (1 − p)r k k  1 − p r (k + r − 1)! M (t) = = pk (1 − p)r X 1 − pet k!(r − 1)! Γ(k + r) = pk (1 − p)r rp k!Γ(r) E(X ) = 1 − p rp In which case the distribution is still discrete, but both parameters are now Var(X ) = (1 − p)2 continuous. This variant of the Negative Binomial does not have a natural interpretation as it does not make sense to have 2.537 failures. iid Alternative view - X = Z1 + Z2 + ··· + Zr where Z1,..., Zr ∼ Geo(p).

Statistics 104 (Colin Rundel) Lecture 13 February 29, 2012 14 / 20 Statistics 104 (Colin Rundel) Lecture 13 February 29, 2012 15 / 20 Continuous RVs Continuous RVs Gamma/ Distribution - CDF Gamma/Erlang Distribution - pdf

Imagine instead of finding the time until an event occurs we instead want ∞ to find the distribution for the time until the nth event. d d X e−λt (λt)j f (t) = F (t) = dt dt j! j=n Let Tn denote the time at which the nth event occurs, then ∞ iid X  e−λt (λt)j−1 e−λt (λt)j  Tn = X1 + ··· + Xn where X1,..., Xn ∼ Exp(λ). Let N(t) be the number = λj − λ j! j! of events that have occured at time t. j=n ∞ ∞ X e−λt (λt)j−1 X e−λt (λt)j = λ − λ (j − 1)! j! j=n j=n ∞ ∞ e−λt (λt)j−1 X e−λt (λt)j−1 X e−λt (λt)j F (t) = P(Tn ≤ t) = P(N(t) ≥ n) = λ + λ − λ ∞ (j − 1)! (j − 1)! j! X j=n+1 j=n = P(N(t) = j) −λt n−1 ∞ −λt j ∞ −λt j e (λt) X e (λt) X e (λt) = λ + λ − λ j=n (n − 1)! j! j! j=n j=n ∞ −λt j X e (λt) e−λt λntn−1 e−λt λntn−1 = = = j! (n − 1)! Γ(n) j=n

Statistics 104 (Colin Rundel) Lecture 13 February 29, 2012 16 / 20 Statistics 104 (Colin Rundel) Lecture 13 February 29, 2012 17 / 20

Continuous RVs Continuous RVs Erlang Distribution

Let X reflect the time until the nth event occurs when the events occur We can generalize the Erlang distribution by using the gamma function according to a Poisson process with rate λ, X ∼ Er(n, λ) instead of the factorial function, we also reparameterize using θ = 1/λ, X ∼ Gamma(n, θ). e−λx λnx n−1 f (x|n, λ) = (n − 1)! e−x/θx n−1 ∞ f (x|n, λ) = X e−λx (λx)j θnΓ(n) F (x|n, λ) = R x −t/θ n−1 j! e t dt γ(n, x/θ) j=n F (x|n, λ) = 0 = θnΓ(n) Γ(n)

 λ n MX (t) =  1 n λ − t M (t) = X 1 − θt

E(X ) = n/λ E(X ) = nθ Var(X ) = n/λ2 Var(X ) = nθ2

Statistics 104 (Colin Rundel) Lecture 13 February 29, 2012 18 / 20 Statistics 104 (Colin Rundel) Lecture 13 February 29, 2012 19 / 20 Continuous RVs Gamma Distribution, cont.

Probability density function: Cumulative distribution function:

Statistics 104 (Colin Rundel) Lecture 13 February 29, 2012 20 / 20