http://statwww.epfl.ch 4. Continuous Random Variables
4.1: Definition. Density and distribution functions. Examples: uniform, exponential, Laplace, gamma. Expectation, variance. Quantiles. 4.2: New random variables from old. 4.3: Normal distribution. Use of normal tables. Continuity correction. Normal approximation to binomial distribution. 4.4: Moment generating functions. 4.5: Mixture distributions. References: Ross (Chapter 4); Ben Arous notes (IV.1, IV.3–IV.6). Exercises: 79–88, 91–93, 107, 108, of Recueil d’exercices.
Probabilite´ et Statistique I — Chapter 4 1 http://statwww.epfl.ch
Petit Vocabulaire Probabiliste
Mathematics English Fran¸cais P(A | B) probabilityof A given B la probabilit´ede A sachant B independence ind´ependance (mutually) independent events les ´ev´enements (mutuellement) ind´ependants pairwise independent events les ´ev´enements ind´ependants deux `adeux conditionally independent events les ´ev´enements conditionellement ind´ependants
X,Y,... randomvariable unevariableal´eatoire I indicator random variable une variable indicatrice fX probability mass/density function fonction de masse/fonction de densit´e FX probability distribution function fonction de r´epartition
E(X) expected value/expectation of X l’esp´erance de X E(Xr) rth moment of X ri`eme moment de X E(X | B) conditional expectation of X given B l’esp´erance conditionelle de X, sachant B var(X) varianceof X la variance de X MX (t) moment generating function of X, or la fonction g´en´eratrices des moments the Laplace transform of fX (x) ou la transform´ee de Laplace de fX (x)
Probabilite´ et Statistique I — Chapter 4 2 http://statwww.epfl.ch
4.1 Continuous Random Variables
Up to now we have supposed that the support of X is countable, so X is a discrete random variable. Now consider what happens when D = {x ∈ R : X(ω)= x, ω ∈ Ω} is uncountable. Note that this implies that Ω itself is uncountable. Example 4.1: The time to the end of the lecture lies in (0, 45)min.•
Example 4.2: Our (height, weight) pairs lie in (0, ∞)2. •
Definition: Let X be a random variable. Its cumulative distribution function (CDF) (fonction de r´epartition) is
FX (x)=P(X ≤ x)=P(Ax), x ∈ R, where Ax is the event {ω : X(ω) ≤ x}, for x ∈ R.
Probabilite´ et Statistique I — Chapter 4 3 http://statwww.epfl.ch
Recall the following properties of FX : Theorem : Let (Ω, F, P) be a probability space and X : Ω 7→ R a random variable. Its cumulative distribution function FX satisfies:
(a) limx→−∞ FX (x)=0;
(b) limx→∞ FX (x)=1;
(c) FX is non-decreasing, that is, FX (x) ≤ FX (y) whenever x ≤ y;
(d) FX is continuous to the right, that is,
lim FX (x + t)= FX (x), x ∈ R; t↓0
(e) P(X > x) = 1 − FX (x);
(f) if x < y, then P(x Probabilite´ et Statistique I — Chapter 4 4 http://statwww.epfl.ch Definition: A random variable X is continuous if there exists a function fX (x), called the probability density function (la densit´e) of X, such that x P(X ≤ x)= FX (x)= fX (u) du, x ∈ R. Z−∞ ∞ The properties of FX imply (i) fX (x) ≥ 0, and (ii) −∞ fX (x) dx = 1. Note: The fundamental theorem of calculus gives R dF (x) f (x)= X . X dx y R Note: As P(x Note: If X is discrete, then its pmf fX (x) is also called its density. Probabilite´ et Statistique I — Chapter 4 5 http://statwww.epfl.ch Some Examples Example 4.3 (Uniform distribution): The random variable U with density function 1 , a Probabilite´ et Statistique I — Chapter 4 6 http://statwww.epfl.ch Example 4.5 (Laplace distribution): The random variable X with density function λ f(x)= e−λ|x−η|, x ∈ R, η ∈ R,λ> 0, 2 is called a Laplace (or sometimes a double exponential) random variable. • Example 4.6 (Gamma distribution): The random variable X with density function α α−1 λ x e−λx, x> 0, f(x)= Γ(α) λ,α > 0, 0, otherwise, is called a gamma random variable with shape parameter α and rate ∞ α−1 −u parameter λ. Here Γ(α)= 0 u e du is the gamma function. α Note that setting = 1 yieldsR the exponential density. • Probabilite´ et Statistique I — Chapter 4 7 http://statwww.epfl.ch exp(1) Gamma, shape=5,rate=3 f(x) f(x) 0.0 0.4 0.8 0.0 0.4 0.8 −2 0 2 4 6 8 −2 0 2 4 6 8 x x Gamma, shape=0.5,rate=0.5 Gamma, shape=8,rate=2 f(x) f(x) 0.0 0.4 0.8 0.0 0.4 0.8 −2 0 2 4 6 8 −2 0 2 4 6 8 x x Probabilite´ et Statistique I — Chapter 4 8 http://statwww.epfl.ch Moments of Continuous Random Variables Definition: Let g(x) be a real-valued function and X a continuous random variable with density function fX (x). Then the expectation of g(X) is defined to be ∞ E{g(X)} = g(x)fX (x) dx, Z−∞ provided E{|g(X)|} < ∞. In particular the mean and variance of X are ∞ ∞ 2 E(X)= xfX (x) dx, var(X)= {x − E(X)} fX (x) dx. Z−∞ Z−∞ Example 4.7: Compute the mean and variance of (a) the U(a, b), (b) the exp(λ), (c) the Laplace, and (d) the gamma distributions. • Probabilite´ et Statistique I — Chapter 4 9 http://statwww.epfl.ch Quantiles Definition: Let 0 xp = inf{x : F (x) ≥ p}. For most continuous random variables, xp is unique and is found as −1 −1 xp = F (p), where F is the inverse function of F . In particular, the 0.5 quantile is called the median of F . Example 4.8 (Uniform distribution): Let U ∼ U(0, 1). Show that xp = p. • Example 4.9 (Exponential distribution): Let X ∼ exp(λ). −1 Show that xp = −λ log(1 − p). • Exercise: Find the quantiles of the Laplace distribution. • Probabilite´ et Statistique I — Chapter 4 10 http://statwww.epfl.ch 4.2 New Random Variables From Old Often in practice we consider Y = g(X), where g is a known function, and want to find FY (y) and fY (y). Theorem : Let Y = g(X) be a random variable. Then f (x) dx, X continuous, Ay X FY (y)=P(Y ≤ y)= fX (x),X discrete, ( R x∈Ay P where Ay = {x ∈ R : g(x) ≤ y}. When g is monotone increasing and has inverse function g−1, we have dg−1(y) F (y)= F {g−1(y)}, f (y)= f {g−1(y)}, Y X Y dy X with a similar result if g is monotone decreasing. • Probabilite´ et Statistique I — Chapter 4 11 http://statwww.epfl.ch β Example 4.10: Let Y = X , where X ∼ exp(λ). Find FY (y) and fY (y). • Example 4.11: Let Y = dXe, where X ∼ exp(λ) (thus Y is the smallest integer no smaller than X). Find FY (y) and fY (y). • Example 4.12: Let Y = − log(1 − U), where U ∼ U(0, 1). Find FY (y) and fY (y). Find also the density and distribution functions of W = − log U. Explain. • Example 4.13: Let X1 and X2 be the results when two fair dice are rolled independently. Find the distribution of X1 − X2. • Example 4.14: Let a, b be constants. Find the distribution and density functions of Y = a + bX in terms of FX , fX . • Probabilite´ et Statistique I — Chapter 4 12 http://statwww.epfl.ch 4.3 Normal Distribution Definition: A random variable X with density function 1 (x − µ)2 f(x)= exp − , x ∈ R, µ ∈ R,σ> 0, (2π)1/2σ 2σ2 is a normal random variable with mean µ and variance σ2: we write X ∼ N(µ, σ2). When µ = 0, σ2 = 1, the corresponding random variable Z is 2 standard normal, Z ∼ N(0, 1), with density φ(z)=(2π)−1/2e−z /2, for z ∈ R. The corresponding cumulative distribution function is x x 1 2 P(Z ≤ x)=Φ(x)= φ(z) dz = e−z /2 dz. (2π)1/2 Z−∞ Z−∞ This integral is tabulated in the formulaire and can be obtained electronically. Probabilite´ et Statistique I — Chapter 4 13 http://statwww.epfl.ch Standard Normal Density Function N(0,1) density phi(x) 0.0 0.1 0.2 0.3 0.4 −3 −2 −1 0 1 2 3 x Probabilite´ et Statistique I — Chapter 4 14 http://statwww.epfl.ch Properties of the Normal Distribution Theorem : The density function φ(z), cumulative distribution function Φ(z), and quantiles zp of Z ∼ N(0, 1) satisfy: (a) the density is symmetric about z = 0, φ(z)= φ(−z) for all z ∈ R; (b) P(Z ≤ z)=Φ(z) = 1 − Φ(z) = 1 − P(Z ≥ z), for all z ∈ R; (c) the standard normal quantiles zp satisfy zp = −z1−p, for all 0 Probabilite´ et Statistique I — Chapter 4 15 http://statwww.epfl.ch Example 4.15: Show that the mean and variance of X ∼ N(µ, σ2) are indeed µ and σ2. • Example 4.16: Find the p quantile of Y = µ + σZ, where Z ∼ N(0, 1). • Example 4.17: Find the distribution and density functions of Y = |Z| and W = Z2, where Z ∼ N(0, 1). • Example 4.18: Find P(Z ≤−2), P(Z ≤ 0.5), P(−2 P(Z ≤ 1.75), z0.05, z0.95, z0.5, z0.8, and z0.15. • Note: The next page gives an extract from the tables showing the function Φ(z) in the Formulaire. Probabilite´ et Statistique I — Chapter 4 16 http://statwww.epfl.ch z 0 1 2 3 4 5 6 7 8 9 0.0 .50000 .50399 .50798 .51197 .51595 .51994 .52392 .52790 .53188 .53586 0.1 .53983 .54380 .54776 .55172 .55567 .55962 .56356 .56750 .57142 .57535 0.2 .57926 .58317 .58706 .59095 .59483 .59871 .60257 .60642 .61026 .61409 0.3 .61791 .62172 .62552 .62930 .63307 .63683 .64058 .64431 .64803 .65173 0.4 .65542 .65910 .66276 .66640 .67003 .67364 .67724 .68082 .68439 .68793 0.5 .69146 .69497 .69847 .70194 .70540 .70884 .71226 .71566 .71904 .72240 0.6 .72575 .72907 .73237 .73565 .73891 .74215 .74537 .74857 .75175 .75490 0.7 .75804 .76115 .76424 .76730 .77035 .77337 .77637 .77935 .78230 .78524 0.8 .78814 .79103 .79389 .79673 .79955 .80234 .80511 .80785 .81057 .81327 0.9 .81594 .81859 .82121 .82381 .82639 .82894 .83147 .83398 .83646 .83891 1.0 .84134 .84375 .84614 .84850 .85083 .85314 .85543 .85769 .85993 .86214 1.1 .86433 .86650 .86864 .87076 .87286 .87493 .87698 .87900 .88100 .88298 1.2 .88493 .88686 .88877 .89065 .89251 .89435 .89617 .89796 .89973 .90147 1.3 .90320 .90490 .90658 .90824 .90988 .91149 .91309 .91466 .91621 .91774 1.4 .91924 .92073 .92220 .92364 .92507 .92647 .92786 .92922 .93056 .93189 1.5 .93319 .93448 .93574 .93699 .93822 .93943 .94062 .94179 .94295 .94408 1.6 .94520 .94630 .94738 .94845 .94950 .95053 .95154 .95254 .95352 .95449 1.7 .95543 .95637 .95728 .95818 .95907 .95994 .96080 .96164 .96246 .96327 1.8 .96407 .96485 .96562 .96638 .96712 .96784 .96856 .96926 .96995 .97062 1.9 .97128 .97193 .97257 .97320 .97381 .97441 .97500 .97558 .97615 .97670 2.0 .97725 .97778 .97831 .97882 .97932 .97982 .98030 .98077 .98124 .98169 Probabilite´ et Statistique I — Chapter 4 17 http://statwww.epfl.ch Normal Approximation to Binomial Distribution Before computers were widespread, one use of the normal distribution was as an approximation to the binomial distribution. Theorem (de Moivre–Laplace): Let Xn ∼ B(n, p), where 2 0 This gives an approximation for the probability that Xn ≤ r: Xn − µn r − µn . r − µn P(X ≤ r) = P ≤ = Φ . n σ σ σ n n n In practice this should be used only when min{np, n(1 − p)}≥ 5. Probabilite´ et Statistique I — Chapter 4 18 http://statwww.epfl.ch Normal and Poisson Approximations to Binomial B(16, 0.5) and Normal approximation B(16, 0.1) and Normal approximation density density 0.00 0.20 0.00 0.20 0 5 10 15 0 5 10 15 r r B(16, 0.5) and Poisson approximation B(16, 0.1) and Poisson approximation density density 0.00 0.20 0.00 0.20 0 5 10 15 0 5 10 15 r r Probabilite´ et Statistique I — Chapter 4 19 http://statwww.epfl.ch Continuity Correction A better approximation to P(Xn ≤ r) is given by replacing r by 1 1 r + 2 ; the 2 is known as a continuity correction. Binomial(15, 0.4) and Normal approximation Density 0.00 0.05 0.10 0.15 0.20 0 5 10 15 x Example 4.19: Let X ∼ B(15, 0.4). Compute exact and approximate values of P(X ≤ r) for r = 1, 8, 10, with and without continuity correction. Comment. • Probabilite´ et Statistique I — Chapter 4 20 http://statwww.epfl.ch 4.4 Moment Generating Functions Recall that the moment generating function of a random variable X is defined as MX (t) = E{exp(tX)}, for t ∈ R such that MX (t) < ∞. MX (t) is also called the Laplace transform of fX (x). Example 4.20: Find MX (t) when X ∼ exp(λ). • Example 4.21: Find the moment generating function of the Laplace distribution. • 2 Example 4.22: Find MX (t) when X ∼ N(µ, σ ). • Example 4.23: Let X ∼ exp(λ). Find the moment generating functions of Y = 2X, of X conditional on the event X < a, and of W = min(X, 3). • Probabilite´ et Statistique I — Chapter 4 21 http://statwww.epfl.ch 4.5 Mixture Distributions In practice random variables are almost always either discrete or continuous. Exceptions can arise, however. Example 4.24 (Petrol): Describe the distribution of the money spent by motorists buying petrol at an automate. • Example 4.25 (Mixture): Let X1 ∼ Geom(p) and X2 ∼ exp(λ). Suppose that X = X1 with probability γ and X = X2 with probability 1 − γ. Find FX , fX , E(X) and var(X). • Probabilite´ et Statistique I — Chapter 4 22