Probability STAT 416 Spring 2007 4 Jointly distributed random variables 1. Introduction 2. Independent Random Variables 3. Transformations 4. Covariance, Correlation 5. Conditional Distributions 6. Bivariate Normal Distribution 1 4.1 Introduction Computation of probabilities with more than one random variable two random variables . bivariate two or more random variables. multivariate Concepts: ² Joint distribution function ² purely discrete: Joint probability function ² purely continuous: Joint density 2 Joint distribution function Bivariate case: Random variables X and Y Define joint distribution function as F (x; y) := P (X · x; Y · y); ¡1 < x; y < 1 Bivariate distribution thus fully specified P (x1<X·x2; y1<Y ·y2) = F (x2; y2)¡F (x1; y2)¡F (x2; y1)+F (x1; y1) for x1 < x2 and y1 < y2 Marginal distribution: FX (x) := P (X · x) = F (x; 1) Idea: P (X · x) = P (X · x; Y < 1) = lim F (x; y) y!1 Analogous FY (y) := P (Y · y) = F (1; y) 3 Bivariate continuous random variable X and Y jointly continuous if there exists joint density function: @2 f(x; y) = F (x; y) @x @y joint distribution function then obtained by integration Zb Za F (a; b) = f(x; y) dxdy y=¡1 x=¡1 Density of marginal distribution X obtained by integration over ­Y : Z1 fX (x) = f(x; y) dy y=¡1 4 Example: Bivariate uniform distribution X and Y uniformly distributed on [0; 1] £ [0; 1] ) density f(x; y) = 1; 0 · x; y · 1: Joint distribution function Zb Za F (a; b) = f(x; y) dxdy = a b; 0 · a; b · 1: y=0 x=0 Density of marginal distribution: Z1 fX (x) = f(x; y) dy = 1; 0 · x · 1 y=¡1 i.e. density of univariate uniform distribution 5 Exercise: Bivariate continuous distribution Let joint density of X and Y be given as 8 < ax2y2; 0 < x < y < 1 f(x; y) = : 0; else 1. Determine the constant a 2. Determine the CDF F (x; y) 3. Compute the probability P (X < 1=2; Y > 2=3) 4. Compute the probability P (X < 2; Y > 2=3) 5. Compute the probability P (X < 2; Y > 3) Hint: Be careful with the domain of integration 6 S.R.: Example 1c Let joint density of X and Y be given as 8 < e¡(x+y); 0 < x < 1; 0 < y < 1 f(x; y) = : 0; else Find the density function of the random variable X=Y Again most important to consider the correct domain of integration Z1 Zay 1 P (X=Y · a) = e¡(x+y) dx dy = 1 ¡ a + 1 y=0 x=0 (More details are given in the book) 7 Bivariate discrete random variable X and Y both discrete Define joint probability function p(x; y) = P (X = x; Y = y) Naturally we have p(x; y) = F (x; y) ¡ F (x¡; y) ¡ F (x; y¡) + F (x¡; y¡) Obtain probability function of X by summation over ­Y : X pX (x) = P (X = x) = p(x; y) ­Y 8 Example Bowl with 3 red, 4 white and 5 blue balls; draw 3 balls randomly without replacement X . number of red drawn balls Y . number of white drawn balls ¡3¢¡4¢¡5¢ ¡12¢ e. g.: p(0; 1) = P (0R; 1W; 2B) = 0 1 2 = 3 = 40=220 j i 0 1 2 3 pX 0 10=220 40=220 30=220 4=220 84=220 1 30=220 60=220 18=220 0 108=220 2 15=220 12=220 0 0 27=220 3 1=220 0 0 0 1=220 pY 56=220 112=220 48=220 4=220 1 9 Multivariate random variable More than two random variables joint distribution function for n rabdom variables F (x1; : : : ; xn) = P (X1 · x1;:::;Xn · xn) Discrete: Joint probability function: p(x1; : : : ; xn) = P (X1 = x1;:::;Xn = xn) Marginals again by summation over all components that are not of interest, e.g. X X pX1 (x1) = ¢ ¢ ¢ p(x1; : : : ; xn) x22­2 xn2­n 10 Multinomial distribution one of the most important multivariate discrete distribution n independent experiments with r possible results, each with probability p1; : : : pr Xi . number of experiments with ith result, then n! n1 nr P (X1 = n1;:::;Xr = nr) = p ¢ ¢ ¢ p n1!¢¢¢nr ! 1 r Pr whenever i=1 ni = n Generalization of binomial distribution (r = 2) Exercise: Poker Dice (throw 5 dices) Compute probability of a (High) straight, Four of a kind, Full House 11 4.2 Independent random variables Two random variables X and Y are called independent if for all events A and B P (X 2 A; Y 2 B) = P (X 2 A)P (Y 2 B) Information on X does not give any information on Y X and Y independent if and only if P (X · a; Y · b) = P (X · a)P (Y · b) i.e. F (a; b) = FX (a) FY (b) for all a; b. Also equivalent with f(x; y) = fX (x) fY (y) in the continuous case and with p(x; y) = pX (x) pY (y) in the discrete case for all x; y 12 Example: continuous Looking back at the example 8 < ax2y2; 0 < x < y < 1 f(x; y) = : 0; else Are the random variables X and Y with joint density as specified above independent? What changes if we look at 8 < ax2y2; 0 < x < 1; 0 < y < 1 f(x; y) = : 0; else 13 Example Z »P(¸) . number of persons which enter a bar p . percentage of male visitors X; Y . number of male and female visitors respectively X; Y are independently Poisson distributed with parameters p¸ and q¸ respectively Solution: Law of total probability: P (X =i; Y =j) = P (X =i; Y =jjX + Y =i + j)P (X + Y =i + j) by definition: ¡ ¢ i+j i j P (X =i; Y =jjX + Y =i + j) = i p q ¡¸ ¸i+j P (X + Y =i + j) = e (i+j)! Together: i i j ¡¸ (¸p) j ¡¸p (¸p) ¡¸q (¸q) P (X =i; Y =j) = e i!j! (¸q) = e i! e j! 14 Example: Two dices X; Y . uniformly distributed on f1;:::; 6g 1 Due to independence we have p(x; y) = pX (x) pY (y) = 36 Distribution function: FX (x) = FY (x) = i=6, if x · i < x + 1 and 0 < x < 7 ij F (x; y) = FX (x)FY (y) = 36 ; x · i < x + 1; y · j < y + 1 Which distribution has X + Y ? P (X + Y = 2) = p(1; 1) = 1=36 P (X + Y = 3) = p(1; 2) + p(2; 1) = 2=36 P (X + Y = 4) = p(1; 3) + p(2; 2) + p(3; 1) = 3=36 P (X + Y = k) = p(1; k ¡ 1) + p(2; k ¡ 2) + ¢ ¢ ¢ + p(k ¡ 1; 1) 15 Sum of two independent distributions Sum of random variables itself a random variable Computation of distribution via convolution Discrete random variables: X X P (X + Y = k) = pX (x)pY (y) = pX (k ¡ y)pY (y) x+y=k ­Y Continuous random variables: Z1 fX+Y (x) = fX (x ¡ y)fY (y)dy y=¡1 Exercise: X »P(¸1) and Y »P(¸2) independent ) X + Y »P(¸1 + ¸1) 16 Example of convolution: Continuous case X, Y independent,uniformly distributed on [0; 1] i.e. f(x; y) = 1; (x; y) 2 [0; 1] £ [0; 1] fX (x) = 1; 0 · x · 1; fY (y) = 1; 0 · y · 1 Computation of density of Z := X + Y Z1 fZ (x) = fX (x ¡ y)fY (y)dy y=¡1 8 Rx > <> dy = x; 0 < x · 1 = y=0 > R1 :> dy = 2 ¡ x; 1 < x · 2 y=x¡1 Reason: fY (y) = 1 if 0 · y · 1 fX (x ¡ y) = 1 if 0 · x ¡ y · 1 , y · x · y + 1 17 Another example of convolution X, Y indep., ¡¡distributed with parameters t1; t2 and same ¸ ¸e¡¸x(¸x)t1¡1 ¸e¡¸y (¸y)t2¡1 fX (x) = ; fY (y) = ; x; y ¸ 0; ¡(t1) ¡(t2) Z1 fZ (x) = fX (x ¡ y)fY (y)dy y=¡1 Zx ¸e¡¸(x¡y)(¸(x ¡ y))t1¡1 ¸e¡¸y(¸y)t2¡1 = dy ¡(t1) ¡(t2) y=0 Zx ¸t1+t2 e¡¸x = (x ¡ y)t1¡1yt2¡1dy ¡(t1)¡(t2) y=0 ¯ ¯ ¯ ¯ ¯ y = xz ¯ ¸e¡¸x(¸x)t1+t2¡1 = ¯ ¯ = ¯ dy = xdz ¯ ¡(t1 + t2) 18 4.3 Transformations As in the univariate case one considers also in the multivariate case transformations of random variables We restrict ourselves to the bivariate case X1 and X2 jointly distributed with density fX1;X2 Y1 = g1(X1;X2) and Y2 = g2(X1;X2) Major question: What is the joint density fY1;Y2 ? 19 Density of Transformation Assumptions: ² y1 = g1(x1; x2) and y2 = g2(x1; x2) can be uniquely solved for x1 and x2, say x1 = h1(y1; y2) and x2 = h2(y1; y2) 1 ² g1 and g2 are C such that ¯ ¯ ¯ ¯ ¯ @g1 @g1 ¯ ¯ @x1 @x2 ¯ J(x1; x2) = ¯ ¯ 6= 0 ¯ @g2 @g2 ¯ @x1 @x2 Under these conditions we have ¡1 fY1;Y2 (y1; y2) = fX1;X2 (x1; x2)jJ(x1; x2)j Proof: Calculus Note the similarity to the one-dimensional case 20 Examples ² Sheldon Ross: Example 7a Y1 = X1 + X2 Y2 = X1 ¡ X2 ² Sheldon Ross: Example 7c X and Y independent gamma random variables U = X + Y V = X=(X + Y ) 21 Expectation of bivariate RV X and Y discrete with joint probability p(x; y). Like in the one-dimensional case: P P E(g(X; Y )) = g(x; y)p(x; y) x2­X y2­Y Exercise: Let X and Y eyes of two fair dice, independently thrown Compute expectation of the difference jX ¡ Y j 22 Expectation of bivariate RV X and Y continuous with joint density f(x; y).