http://statwww.epfl.ch 4. Continuous Random Variables 4.1: Definition. Density and distribution functions. Examples: uniform, exponential, Laplace, gamma. Expectation, variance. Quantiles. 4.2: New random variables from old. 4.3: Normal distribution. Use of normal tables. Continuity correction. Normal approximation to binomial distribution. 4.4: Moment generating functions. 4.5: Mixture distributions. References: Ross (Chapter 4); Ben Arous notes (IV.1, IV.3–IV.6). Exercises: 79–88, 91–93, 107, 108, of Recueil d’exercices. Probabilite´ et Statistique I — Chapter 4 1 http://statwww.epfl.ch Petit Vocabulaire Probabiliste Mathematics English Fran¸cais P(A | B) probabilityof A given B la probabilit´ede A sachant B independence ind´ependance (mutually) independent events les ´ev´enements (mutuellement) ind´ependants pairwise independent events les ´ev´enements ind´ependants deux `adeux conditionally independent events les ´ev´enements conditionellement ind´ependants X,Y,... randomvariable unevariableal´eatoire I indicator random variable une variable indicatrice fX probability mass/density function fonction de masse/fonction de densit´e FX probability distribution function fonction de r´epartition E(X) expected value/expectation of X l’esp´erance de X E(Xr) rth moment of X ri`eme moment de X E(X | B) conditional expectation of X given B l’esp´erance conditionelle de X, sachant B var(X) varianceof X la variance de X MX (t) moment generating function of X, or la fonction g´en´eratrices des moments the Laplace transform of fX (x) ou la transform´ee de Laplace de fX (x) Probabilite´ et Statistique I — Chapter 4 2 http://statwww.epfl.ch 4.1 Continuous Random Variables Up to now we have supposed that the support of X is countable, so X is a discrete random variable. Now consider what happens when D = {x ∈ R : X(ω)= x, ω ∈ Ω} is uncountable. Note that this implies that Ω itself is uncountable. Example 4.1: The time to the end of the lecture lies in (0, 45)min.• Example 4.2: Our (height, weight) pairs lie in (0, ∞)2. • Definition: Let X be a random variable. Its cumulative distribution function (CDF) (fonction de r´epartition) is FX (x)=P(X ≤ x)=P(Ax), x ∈ R, where Ax is the event {ω : X(ω) ≤ x}, for x ∈ R. Probabilite´ et Statistique I — Chapter 4 3 http://statwww.epfl.ch Recall the following properties of FX : Theorem : Let (Ω, F, P) be a probability space and X : Ω 7→ R a random variable. Its cumulative distribution function FX satisfies: (a) limx→−∞ FX (x)=0; (b) limx→∞ FX (x)=1; (c) FX is non-decreasing, that is, FX (x) ≤ FX (y) whenever x ≤ y; (d) FX is continuous to the right, that is, lim FX (x + t)= FX (x), x ∈ R; t↓0 (e) P(X > x) = 1 − FX (x); (f) if x < y, then P(x<X ≤ y)= FX (y) − FX (x). • Probabilite´ et Statistique I — Chapter 4 4 http://statwww.epfl.ch Definition: A random variable X is continuous if there exists a function fX (x), called the probability density function (la densit´e) of X, such that x P(X ≤ x)= FX (x)= fX (u) du, x ∈ R. Z−∞ ∞ The properties of FX imply (i) fX (x) ≥ 0, and (ii) −∞ fX (x) dx = 1. Note: The fundamental theorem of calculus gives R dF (x) f (x)= X . X dx y R Note: As P(x<X ≤ y)= x fX (u) du when x < y, for any x ∈ , R y x P(X = x) = lim P(x<X ≤ y) = lim fX (u) du = fX (u) du = 0. y↓x y↓x Zx Zx Note: If X is discrete, then its pmf fX (x) is also called its density. Probabilite´ et Statistique I — Chapter 4 5 http://statwww.epfl.ch Some Examples Example 4.3 (Uniform distribution): The random variable U with density function 1 , a<u<b, f(u)= b−a a<b, 0, otherwise, is called a uniform random variable. We write U ∼ U(a, b). • Example 4.4 (Exponential distribution): The random variable X with density function λe−λx, x> 0, f(x)= λ> 0, 0, otherwise, is called an exponential random variable with rate λ. We write X ∼ exp(λ). Establish the lack of memory property for X, that P(X > x + t | X > t)=P(X > x) for t,x > 0. • Probabilite´ et Statistique I — Chapter 4 6 http://statwww.epfl.ch Example 4.5 (Laplace distribution): The random variable X with density function λ f(x)= e−λ|x−η|, x ∈ R, η ∈ R,λ> 0, 2 is called a Laplace (or sometimes a double exponential) random variable. • Example 4.6 (Gamma distribution): The random variable X with density function α α−1 λ x e−λx, x> 0, f(x)= Γ(α) λ,α > 0, 0, otherwise, is called a gamma random variable with shape parameter α and rate ∞ α−1 −u parameter λ. Here Γ(α)= 0 u e du is the gamma function. α Note that setting = 1 yieldsR the exponential density. • Probabilite´ et Statistique I — Chapter 4 7 http://statwww.epfl.ch exp(1) Gamma, shape=5,rate=3 f(x) f(x) 0.0 0.4 0.8 0.0 0.4 0.8 −2 0 2 4 6 8 −2 0 2 4 6 8 x x Gamma, shape=0.5,rate=0.5 Gamma, shape=8,rate=2 f(x) f(x) 0.0 0.4 0.8 0.0 0.4 0.8 −2 0 2 4 6 8 −2 0 2 4 6 8 x x Probabilite´ et Statistique I — Chapter 4 8 http://statwww.epfl.ch Moments of Continuous Random Variables Definition: Let g(x) be a real-valued function and X a continuous random variable with density function fX (x). Then the expectation of g(X) is defined to be ∞ E{g(X)} = g(x)fX (x) dx, Z−∞ provided E{|g(X)|} < ∞. In particular the mean and variance of X are ∞ ∞ 2 E(X)= xfX (x) dx, var(X)= {x − E(X)} fX (x) dx. Z−∞ Z−∞ Example 4.7: Compute the mean and variance of (a) the U(a, b), (b) the exp(λ), (c) the Laplace, and (d) the gamma distributions. • Probabilite´ et Statistique I — Chapter 4 9 http://statwww.epfl.ch Quantiles Definition: Let 0 <p< 1. The p quantile of distribution function F (x) is defined as xp = inf{x : F (x) ≥ p}. For most continuous random variables, xp is unique and is found as −1 −1 xp = F (p), where F is the inverse function of F . In particular, the 0.5 quantile is called the median of F . Example 4.8 (Uniform distribution): Let U ∼ U(0, 1). Show that xp = p. • Example 4.9 (Exponential distribution): Let X ∼ exp(λ). −1 Show that xp = −λ log(1 − p). • Exercise: Find the quantiles of the Laplace distribution. • Probabilite´ et Statistique I — Chapter 4 10 http://statwww.epfl.ch 4.2 New Random Variables From Old Often in practice we consider Y = g(X), where g is a known function, and want to find FY (y) and fY (y). Theorem : Let Y = g(X) be a random variable. Then f (x) dx, X continuous, Ay X FY (y)=P(Y ≤ y)= fX (x), X discrete, ( R x∈Ay P where Ay = {x ∈ R : g(x) ≤ y}. When g is monotone increasing and has inverse function g−1, we have dg−1(y) F (y)= F {g−1(y)}, f (y)= f {g−1(y)}, Y X Y dy X with a similar result if g is monotone decreasing. • Probabilite´ et Statistique I — Chapter 4 11 http://statwww.epfl.ch β Example 4.10: Let Y = X , where X ∼ exp(λ). Find FY (y) and fY (y). • Example 4.11: Let Y = dXe, where X ∼ exp(λ) (thus Y is the smallest integer no smaller than X). Find FY (y) and fY (y). • Example 4.12: Let Y = − log(1 − U), where U ∼ U(0, 1). Find FY (y) and fY (y). Find also the density and distribution functions of W = − log U. Explain. • Example 4.13: Let X1 and X2 be the results when two fair dice are rolled independently. Find the distribution of X1 − X2. • Example 4.14: Let a, b be constants. Find the distribution and density functions of Y = a + bX in terms of FX , fX . • Probabilite´ et Statistique I — Chapter 4 12 http://statwww.epfl.ch 4.3 Normal Distribution Definition: A random variable X with density function 1 (x − µ)2 f(x)= exp − , x ∈ R, µ ∈ R,σ> 0, (2π)1/2σ 2σ2 is a normal random variable with mean µ and variance σ2: we write X ∼ N(µ, σ2). When µ = 0, σ2 = 1, the corresponding random variable Z is 2 standard normal, Z ∼ N(0, 1), with density φ(z)=(2π)−1/2e−z /2, for z ∈ R. The corresponding cumulative distribution function is x x 1 2 P(Z ≤ x)=Φ(x)= φ(z) dz = e−z /2 dz. (2π)1/2 Z−∞ Z−∞ This integral is tabulated in the formulaire and can be obtained electronically. Probabilite´ et Statistique I — Chapter 4 13 http://statwww.epfl.ch Standard Normal Density Function N(0,1) density phi(x) 0.0 0.1 0.2 0.3 0.4 −3 −2 −1 0 1 2 3 x Probabilite´ et Statistique I — Chapter 4 14 http://statwww.epfl.ch Properties of the Normal Distribution Theorem : The density function φ(z), cumulative distribution function Φ(z), and quantiles zp of Z ∼ N(0, 1) satisfy: (a) the density is symmetric about z = 0, φ(z)= φ(−z) for all z ∈ R; (b) P(Z ≤ z)=Φ(z) = 1 − Φ(z) = 1 − P(Z ≥ z), for all z ∈ R; (c) the standard normal quantiles zp satisfy zp = −z1−p, for all 0 <p< 1; (d) zrφ(z) → 0 as z → ±∞, for all r> 0; (e) φ0(z)= −zφ(z), φ00(z)=(z2 − 1)φ(z), etc.