Joint Distribution of two or More Random Variables • Sometimes more than one measurement (r.v.) is taken on each member of the sample space. In cases like this there will be a few random variables defined on the same probability space and we would like to explore their joint distribution. • Joint behavior of 2 random variable (continuous or discrete), X and Y determined by their joint cumulative distribution function F xXY, y( , ) = P( X≤ ,. x≤ ) Y y • n – imensional cased F x x,..., = P X≤ ,,. x≤ X x XX,...,1 n ( 1 n ) ( 1 1 K n n) week 7 1 Discrete case • Suppose X, Y are discrete random variables defined on the same probability space. •The joint probability mass function of 2 discrete random variables X and Y is the function pX,Y(x,y) defined for all pairs of real numbers x and y by p x yXY, ( , P) = X( = x and= ) Y y • For a joint pmf pX,Y(x,y) we must have: pX,Y(x,y) ≥ 0 for all values of x,y and ∑∑pXY, (, x) y= 1 xy week 7 2 Example for illustration • Toss a coin 3 times. Define, X: number of heads on 1st toss, Y: total number of heads. • The sample space is Ω ={TTT, TTH, THT, HTT, THH, HTH, HHT, HHH}. • We display the joint distribution of X and Y in the following table • Can we recover the probability mass function for X and Y from the joint table? • To find the probability mass function of X we sum the appropriate rows of the table of the joint probability function. • Similarly, to find the mass function for Y we sum the appropriate columns. week 7 3 Marginal Probability Function e hmarginal •Tprobability mass function for X is pX x( ) = ∑ pXY, ( , x) y y e hmarginal •Tprobability mass function for Y is pY y()= ∑ pXY, ( , x) y x • Case of several discrete random variables is analogous. • If X1,…,Xm e random variables onare discret the same sample space with joint probability function p x x,... = P X= ,..., x= X x XX,...1 n ( 1 m ) ( 1 1 m m) The marginal probability function for X1 is p x()= p( ,... x) x X1 1 ∑ XX,...1 n 1 m x2 ,..., m x • he 2-dimentional marginal probability function for TX1 and X2 is p x( x, ) = p,( x , x ,..., x) x XX1 2 1 2 ∑ XX,...1 n 1 2 3 m x3 ,..., m x week 7 4 Example • Roll a die twice. Let X: number of 1’s and Y: total of the 2 die. There is no available form of the joint mass function for X, Y. We display the joint distribution of X and Y with the following table. • The marginal probability mass function of X and Y are •Find P(X ≤ 1 and Y ≤ 4) week 7 5 Independence of random variables • Recall the definition of random variable X: mapping from Ω to R such that ∈{}ω ΩX: ()ω x =∈ Fforx ∈ R . • By definition of F, this implies that (X > 1.4) is and event and for the discrete case (X = 2) is an event. • In generalXAXA () ∈ = { ω : ( ω ) ∈ } is an event for any set A that is formed by taking unions / complements / intersections of intervals from R. • Definition Random variables X and Y are independent if the events XA ( ∈ ) and YB ( ∈ ) are independent. week 7 6 Theorem • Two discrete random variables X and Y with joint pmf pX,Y(x,y) and marginal mass function pX(x) and pY(y), are independent if and only if p xXY, ( y, ) = pXY( ) x( p) y • Proof: • Question: Back to the rolling die 2 times example, are X and Y independent? week 7 7 Conditional Probability on a joint discrete distribution • pmfiven the jointG f oX and Yto find , we want P X( x= and= ) Y y P X()= x| = Y = y P() Y= y and P X( x= and= ) Y y P Y()= y| = X = x P() X= x • Back to the example on slide 5, PYX()=2 | = 1= 0 2 1 PYX=()3 | = 1 =36 = 10 5 36 M PYX()=12 | = 1= 0 • hese 11 probabilities give the conditional pmfT f oY given X = 1. week 7 8 Definition r oX, •FY discrete random variables with joint pmf pX,Y(x,y) and marginal mass function pX(x) and pY(y). If x is a number such that pX(x) > 0, then the onal pmfconditi of Y given X = x is p( x, ) y p y() x| = p (| y= X ) = XY x, YX| YX| p() x • Is this a valid pmf? X • imilarly, the conditional pmfS f oX given Y = y is pXY, () x, y p xXY| () y| = pXY| (| x= Y ) = y pY () y • ote, from the above Nconditional pmf e wget p xXY, y( , ) = pXY| ( x| ) yY ( ) p y ng both sides over all possible values of iSummY we get p xX ( ) p= ∑ XY x, )( , y= ∑ XY p| )(| xY ( y) p y y y This is an extremely useful application of the law of total probability. f I : e tX, Y oare independent •N random variables then PX|Y(x|y) = PX(x). week 7 9 Example • uppose we roll a fair die; whateverS number comes up we toss a coin that many times. What is the distriber of heads?bution of the num Let X = number of heads, Y = number on die. We know that 1 p,() y= 1,2,3,4,5,6 y = Y 6 Want to find pX(x). • he coTnditional probability function of X given Y = y is given by 2 ⎛ y⎞⎛ 1 ⎞ pXY| () x| y= ⎜ ⎟⎜ ⎟ ⎝ x ⎠⎝ 2 ⎠ for x = 0, 1, …, y. • y the Law of Total Probability we have B y 6 ⎛ y⎞⎛ 1 ⎞ 1 p xX ()= ∑ pXY| ( x| ) yY () p= ∑ y⎜ ⎟⎜ ⎟ ⋅ y y= x⎝ x ⎠⎝ 2 ⎠ 6 Possible values of x: 0,1,2,…,6. week 7 01 Example • Interested in the 1st and 2nd success in repeated Bernoulli trails with probability of success p on any trail. Let X = number of failures before 1st success and Y = number of failures before 2nd success (including those before the 1st success). • The marginal pmf of X and Y are p x()= P ( X = ) x = x pq for x = 0, 1, 2, …. •and X 2 y p yY=() P Y ( = ) y =( 1 y) + pfor q y = 0, 1, 2, …. • The joint mass function, pX,Y(x,y) , where x, y are non-negative integers and x ≤ y is the probability that the 1st success occurred after x failure and the 2nd success after y – x more failures (after 1st success), e.g. 2 5 pXY, ()5,2 = P(FFSFFFS) =p q • The joint pmf is then given by ⎧p2 qy 0≤x ≤ y pXY, () x, y= ⎨ ⎩ 0 otherwise • Find the marginal probability functions of X, Y. Are X, Y independent? • What is the conditional probability function of X given Y = 5 week 7 11 Some Notes on Multiple Integration b • ecall: simple integral f x () R dx cab be regarded as the limit as N Æ ∞ of ∫a the Riemann sum of the form N ∑f() xiΔ i x i=1 th where ∆xi is the length of the i subinterval of some partition of [a, b]. • y analogy, a double integral B f∫∫ x( , y) dA D where D is a finite region in the xyit as -plane, can be regarded as the lim N Æ ∞ of the formof the Riemann sum N N f x∑ y()i, iΔ A i ∑ = f ()i x, iΔΔ y i i x y i=1 i=1 th where Ai is the area of the i rectangle in a decomposition of D into rectangles. week 7 21 Evaluation of Double Integrals • Idea – evaluate as an iterated integral. Restrict D now to a region such that any vertical line cuts its boundary in 2 points. So we can describe D by yL(x) ≤ y ≤ yU(x) and xL ≤ x ≤ xU • Theorem Let D be the subset of R2 defined by yL(x) ≤ y ≤ yU(x) and xL ≤ x ≤ xU If f(x,y) is continuous on D then xU yU f x(), y dA= f⎡ x () y, dy⎤ dx ∫∫ ∫∫x y L ⎢ L ⎥ D ⎣ ⎦ • Comment: When evaluating a double integral, the shape of the region or the form of f (x,y) may dictate that you interchange the role of x and y and integrate first w.r.t x and then y. week 7 13 Examples 1) Evaluate x ∫∫ ⋅ ydA where D is the triangle with vertices (0,0), (2,0) and (0,1). D 2) Evaluatexy ∫∫ x 2 + y 2 where dA D is the region in the 1st quadrate bounded D by y = 0, y = x and x2 + y2 = 1. 3) Integratef x () , y = − x xe −2 y over the set of points in the positive quadrate for which y > 2x. week 7 41 The Joint Distribution of two Continuous R.V’s • tioniDefin Random variables X and Y are (jointly) continuous if there is a non-negative function fX,Y(x,y) such that P X Y, ∈ A = f x, y dxdy (( ) ) ∫∫ XY, ( ) A for any “reasonable” 2-dimensional set A. • fX,Y(x,y) is called a joint density function for (X, Y). • In particular , if A = {(X, Y): X ≤ x, Y ≤ xjoint CDF of }, the X,Y is xy F xXY y, (), = fXY, u ),, v ( du dv ∫∫−∞∞ − • rom Fundamental Theorem of Calculus we have F ∂ 2 ∂ 2 f() x, y= F() x, = y F() x, y XY. ∂x ∂ y XY, ∂y ∂ x XY, week 7 51 Properties of joint density function •fXY, () x, y≥ 0 for allx , y∈ R • t’s integral over IR2 is ∞ ∞ f xXY, () y, dxdy= 1 −∫∫ ∞− ∞ week 7 61 Example • Consider the following bivariate density function ⎧12 ⎪ x()2 + xy 0≤x ≤ 1 , 0 ≤ y ≤ 1 fXY, () x, y= ⎨ 7 ⎩⎪ 0 otherwise • Check if it’s a valid density function.