Joint Distributions of 2 Or More Random Variables

Joint Distribution of two or More Random Variables

• Sometimes more than one measurement (r.v.) is taken on each member of the sample space. In cases like this there will be a few random variables defined on the same probability space and we would like to explore their joint distribution.

• Joint behavior of 2 random variable (continuous or discrete), X and Y determined by their joint cumulative distribution function

,YX ( , )= ( ≤ ≤ yYxXPyxF ).,

• n – dimensional case ( ,..., )= ( ≤ ≤ xXxXPxxF ).,, 1 XX n 1,..., n 11 K nn

week 7 1 Discrete case

• Suppose X, Y are discrete random variables defined on the same probability space.

•The joint probability mass function of 2 discrete random variables X and Y is the function pX,Y(x,y) defined for all pairs of real numbers x and y by

,YX ( , )= ( = = yYandxXPyxp )

• For a joint pmf pX,Y(x,y) we must have: pX,Y(x,y) ≥ 0 for all values of x,y and

∑∑ ,YX ( yxp )= 1, xy

week 7 2 Example for illustration

• Toss a coin 3 times. Define, X: number of heads on 1st toss, Y: total number of heads. • The sample space is Ω ={TTT, TTH, THT, HTT, THH, HTH, HHT, HHH}. • We display the joint distribution of X and Y in the following table

• Can we recover the probability mass function for X and Y from the joint table?

• To find the probability mass function of X we sum the appropriate rows of the table of the joint probability function. • Similarly, to find the mass function for Y we sum the appropriate columns. week 7 3 Marginal Probability Function

•The marginal probability mass function for X is

X ( )= ∑ ,YX ( , yxpxp ) y •The marginal probability mass function for Y is

Y ()= ∑ ,YX ( , yxpyp ) x • Case of several discrete random variables is analogous.

• If X1,…,Xm are discrete random variables on the same sample space with joint probability function ( ,... )= ( = ,..., = xXxXPxxp ) 1 XX n 1,... m 11 mm

The marginal probability function for X1 is ()= ( ,...xxpxp ) X1 1 ∑ 1 XX n 1,... m 2 ,...,xx m

• The 2-dimentional marginal probability function for X1 and X2 is ( , )= ( ,...,,, xxxxpxxp ) XX 21 21 ∑ 1 XX n 321,... m 3 ,...,xx m week 7 4 Example

• Roll a die twice. Let X: number of 1’s and Y: total of the 2 die. There is no available form of the joint mass function for X, Y. We display the joint distribution of X and Y with the following table.

• The marginal probability mass function of X and Y are

•Find P(X ≤ 1 and Y ≤ 4)

week 7 5 Independence of random variables

• Recall the definition of random variable X: mapping from Ω to R such that {}ω : ()ω =Ω∈ ∈ FxX for ∈ Rx . • By definition of F, this implies that (X > 1.4) is and event and for the discrete case (X = 2) is an event. • In general () ∈ = { ω : ( ω ) ∈ AXAX } is an event for any set A that is formed by taking unions / complements / intersections of intervals from R.

• Definition Random variables X and Y are independent if the events ( ∈ AX ) and ( ∈ BY ) are independent.

week 7 6 Theorem

• Two discrete random variables X and Y with joint pmf pX,Y(x,y) and marginal mass function pX(x) and pY(y), are independent if and only if

,YX ( , )= ( ) YX (ypxpyxp ) • Proof:

• Question: Back to the rolling die 2 times example, are X and Y independent?

week 7 7 Conditional Probability on a joint discrete distribution

• These 11 probabilities give the conditional pmf of Y given X = 1.

week 7 8 Definition

•For X, Y discrete random variables with joint pmf pX,Y(x,y) and marginal mass function pX(x) and pY(y). If x is a number such that pX(x) > 0, then the conditional pmf of Y given X = x is ( , yxp ) ()| = | (xXypxyp == ,YX ) |XY |XY ()xp • Is this a valid pmf? X

• Similarly, the conditional pmf of X given Y = y is

,YX (), yxp |YX ()| = |YX | (yYxpyxp == ) Y ()yp • Note, from the above conditional pmf we get

,YX ( , )= |YX ( | ) Y (ypyxpyxp ) Summing both sides over all possible values of Y we get

X ( )= ∑ ,YX ( , )()= ∑ |YX | Y (ypyxpyxpxp ) y y This is an extremely useful application of the law of total probability.

•Note: If X, Y are independent random variables then PX|Y(x|y) = PX(x). week 7 9 Example

• Suppose we roll a fair die; whatever number comes up we toss a coin that many times. What is the distribution of the number of heads? Let X = number of heads, Y = number on die. We know that 1 () yyp == 6,5,4,3,2,1, Y 6

Want to find pX(x). • The conditional probability function of X given Y = y is given by 2 ⎛ y⎞⎛ 1 ⎞ |YX ()| yxp = ⎜ ⎟⎜ ⎟ ⎝ x ⎠⎝ 2 ⎠ for x = 0, 1, …, y. • By the Law of Total Probability we have y 6 ⎛ y⎞⎛ 1 ⎞ 1 X ()= ∑ |YX | (Y ypyxpxp )= ∑ ()⎜ ⎟⎜ ⎟ ⋅ y =xy ⎝ x ⎠⎝ 2 ⎠ 6 Possible values of x: 0,1,2,…,6.

week 7 10 Example • Interested in the 1st and 2nd success in repeated Bernoulli trails with probability of success p on any trail. Let X = number of failures before 1st success and Y = number of failures before 2nd success (including those before the 1st success). • The marginal pmf of X and Y are () (=== pqxXPxp )x for x = 0, 1, 2, …. •and X 2 y Y () (( )+=== 1) qpyyYPyp for y = 0, 1, 2, ….

• The joint mass function, pX,Y(x,y) , where x, y are non-negative integers and x ≤ y is the probability that the 1st success occurred after x failure and the 2nd success after y – x more failures (after 1st success), e.g. 52 p ,YX ()5,2 = P(FFSFFFS)= qp • The joint pmf is then given by ⎧ 2qp y 0 ≤≤ yx ,YX (), yxp = ⎨ ⎩ 0 otherwise • Find the marginal probability functions of X, Y. Are X, Y independent? • What is the conditional probability function of X given Y = 5 week 7 11 Some Notes on Multiple Integration

b • Recall: simple integral () dxxf cab be regarded as the limit as N Æ ∞ of ∫a the Riemann sum of the form N ∑ ()Δxxf ii i=1 th where ∆xi is the length of the i subinterval of some partition of [a, b]. • By analogy, a double integral ∫∫ ( , )dAyxf D where D is a finite region in the xy-plane, can be regarded as the limit as N Æ ∞ of the Riemann sum of the form N N ∑ (), iii =Δ ∑ , ()ΔΔ yxyxfAyxf iiii i=1 i=1

th where Ai is the area of the i rectangle in a decomposition of D into rectangles. week 7 12 Evaluation of Double Integrals

• Idea – evaluate as an iterated integral. Restrict D now to a region such that any vertical line cuts its boundary in 2 points. So we can describe D by

yL(x) ≤ y ≤ yU(x) and xL ≤ x ≤ xU

• Theorem

Let D be the subset of R2 defined by yL(x) ≤ y ≤ yU(x) and xL ≤ x ≤ xU If f(x,y) is continuous on D then x y (), dAyxf = U ⎡ U , ()⎤dxdyyxf ∫∫ ∫∫x y L ⎢ L ⎥ D ⎣ ⎦

• Comment: When evaluating a double integral, the shape of the region or the form of f (x,y) may dictate that you interchange the role of x and y and integrate first w.r.t x and then y.

week 7 13 Examples

1) Evaluate ∫∫ ⋅ ydAx where D is the triangle with vertices (0,0), (2,0) and (0,1). D

2) Evaluate ∫∫ + 22 dAyxxy where D is the region in the 1st quadrate bounded D by y = 0, y = x and x2 + y2 = 1.

3) Integrate () , = xeyxf −− 2 yx over the set of points in the positive quadrate for which y > 2x.

week 7 14 The Joint Distribution of two Continuous R.V’s

• Definition Random variables X and Y are (jointly) continuous if there is a non-negative function fX,Y(x,y) such that (( , ) )=∈ ( , )dxdyyxfAYXP ∫∫ ,YX A for any “reasonable” 2-dimensional set A.

• fX,Y(x,y) is called a joint density function for (X, Y). • In particular , if A = {(X, Y): X ≤ x, Y ≤ x}, the joint CDF of X,Y is xy ,YX (), = ,YX (),, dvduvufyxF ∫∫−∞∞ −

• From Fundamental Theorem of Calculus we have ∂ 2 ∂ 2 (), yxf = (), yxF = (), yxF .YX ∂∂ yx ,YX ∂∂ xy ,YX

week 7 15 Properties of joint density function

• ,YX ()yxf ≥ 0, for all , ∈ Ryx

• It’s integral over R2 is

∞ ∞ ,YX ()dxdyyxf = 1, ∫∫∞− ∞−

week 7 16 Example

• Consider the following bivariate density function

⎧12 ⎪ ()2 + xyx yx ≤≤≤≤ 10,10 ,YX (), yxf = ⎨ 7 ⎩⎪ 0 otherwise

• Check if it’s a valid density function.

• Compute P(X > Y).

week 7 17 Properties of Joint Distribution Function

2 For random variables X, Y , FX,Y : R Æ [0,1] given by FX,Y (x,y) = P(X ≤ x,Y ≤ x)

• ,YX ()yxF = 0,lim x ∞−→ y ∞−→

• ,YX ()yxF = 1,lim x ∞→ y ∞→

• FX,Y (x,y) is non-decreasing in each variable i.e.

YX (), 11, ≤ YX ( , yxFyxF 22, ) if x1 ≤ x2 and y1 ≤ y2 .

•a,YX (),lim = Y (yFyxF ) nd ,YX ( ,lim )= X (xFyxF ) x ∞→ y ∞→

week 7 18 Marginal Densities and Distribution Functions

• The marginal (cumulative) distribution function of X is x ∞ X () (=≤= ),YX , dyduyufxXPxF ( ) ∫∫∞− ∞− • The marginal density of X is then ∞ ' X ()X ()== ,YX , dyyxfxFxf ( ) ∫ ∞− • Similarly the marginal density of Y is ∞ Y ()= ,YX , (dxyxfyf ) ∫ ∞−

week 7 19 Example

• Consider the following bivariate density function ⎧6xy2 yx ≤≤≤≤ 10,10 ,YX (), yxf = ⎨ ⎩ 0 otherwise • Check if it is a valid density function.

• Find the joint CDF of (X, Y) and compute P(X ≤ ½,Y ≤ ½).

• Compute P(X ≤ 2 ,Y ≤ ½).

• Find the marginal densities of X and Y.

week 7 20 Generalization to higher dimensions

Suppose X, Y, Z are jointly continuous random variables with density f(x,y,z), then

• Marginal density of X is given by:

∞ ∞ X ()= ,, (dydzzyxfxf ) ∫∫∞− ∞−

• Marginal density of X, Y is given by : ∞ ,YX (), = ,, (dzzyxfyxf ) ∫ ∞−

week 7 21 Example

Given the following joint CDF of X, Y

2222 ,YX (), −+= yxxyyxyxF yx ≤≤≤≤ 10,10

• Find the joint density of X, Y.

• Find the marginal densities of X and Y and identify them.

week 7 22 Example

Consider the joint density ⎧λ2e−λ y 0 ≤≤ yx ,YX (), yxf = ⎨ ⎩ 0 otherwise where λ is a positive parameter.

• Check if it is a valid density.

• Find the marginal densities of X and Y and identify them.

week 7 23