Jointly distributed random variables Sum of independent random variables Generating functions Fundamental Tools - Probability Theory III MSc Financial Mathematics The University of Warwick September 26, 2018 MSc Financial Mathematics Fundamental Tools - Probability Theory III 1 / 20 Jointly distributed random variables Joint distribution functions Sum of independent random variables Conditional distributions Generating functions Independent random variables Joint distribution function For two random variables X and Y , we define their joint distribution function by FXY (x; y) = P(X 6 x; Y 6 y); −∞ < x; y < 1: Two important classes: X and Y are jointly discrete if both X and Y are discrete. It is convenient to work with their joint probability mass function pXY (x; y) = P(X = x; Y = y): X and Y are jointly continuous if there exists a non-negative 2 function fXY : R ! R such that Z u=x Z v=y P(X 6 x; Y 6 y) = FXY (x; y) = fXY (u; v)dvdu: u=−∞ v=−∞ We call fXY the joint probability density function of X and Y . MSc Financial Mathematics Fundamental Tools - Probability Theory III 2 / 20 Jointly distributed random variables Joint distribution functions Sum of independent random variables Conditional distributions Generating functions Independent random variables Recovering marginal distribution Given a joint probability distribution/mass/density function of (X ; Y ), we can recover the the corresponding marginal characteristics of X as follows: Marginal distribution function FX (x) = P(X 6 x) = P(X 6 x; Y < 1) = FXY (x; 1): Marginal probability mass function (if (X ; Y ) are jointly discrete) X pX (x) = P(X = x) = P(X = x; Y 2 SY ) = pXY (x; y): y Marginal probability density function (if (X ; Y ) are jointly continuous) d d Z 1 fX (x) = FX (x) = FXY (x; 1) = fXY (x; y)dy: dx dx −∞ MSc Financial Mathematics Fundamental Tools - Probability Theory III 3 / 20 Jointly distributed random variables Joint distribution functions Sum of independent random variables Conditional distributions Generating functions Independent random variables Working with jointly continuous random variables From definition, the joint distribution and density function of (X ; Y ) are related via d 2 f (x; y) = F (x; y): XY dxdy XY In univariate case, we compute probabilities involving a continuous random variable via simple integration: Z P(X 2 A) = f (x)dx: A In bivariate case, probabilities are computed via double integration ZZ P((X ; Y ) 2 A) = fXY (x; y)dxdy: A and the calculation is not necessarily straightforward. MSc Financial Mathematics Fundamental Tools - Probability Theory III 4 / 20 Jointly distributed random variables Joint distribution functions Sum of independent random variables Conditional distributions Generating functions Independent random variables Example 1 Let (X ; Y ) be a pair of jointly continuous random variables with joint density function f (x; y) = 1 on (x; y) 2 [0; 1]2 (and is zero elsewhere). Find P(X − Y < 1=2). MSc Financial Mathematics Fundamental Tools - Probability Theory III 5 / 20 Jointly distributed random variables Joint distribution functions Sum of independent random variables Conditional distributions Generating functions Independent random variables Example 2 Let (X ; Y ) be a pair of jointly continuous random variables with joint density function f (x; y) = e−y on 0 < x < y < 1 (and is zero elsewhere). Verify that the given f is a well-defined joint density function. Find the marginal density function of X . MSc Financial Mathematics Fundamental Tools - Probability Theory III 6 / 20 Jointly distributed random variables Joint distribution functions Sum of independent random variables Conditional distributions Generating functions Independent random variables Expected value involving joint distribution Let (X ; Y ) be a pair of random variables. For a given function g(·; ·), the expected value of the random variable g(X ; Y ) is given by (P x;y g(x; y)pXY (x; y); E(g(X ; Y )) = R 1 R 1 −∞ −∞ g(x; y)fXY (x; y)dxdy: Two important specifications of g(·; ·): Set g(x; y) = x + y. One could obtain E(X + Y ) = E(X ) + E(Y ). Set g(x; y) = xy. This leads to computation of the covariance measure between X and Y defined by Cov(X ; Y ) = E(XY ) − E(X )E(Y ) and correlation measure defined by Cov(X ; Y ) corr(X ; Y ) = : pvar(X ) var(Y ) MSc Financial Mathematics Fundamental Tools - Probability Theory III 7 / 20 Jointly distributed random variables Joint distribution functions Sum of independent random variables Conditional distributions Generating functions Independent random variables Conditional distributions If (X ; Y ) are jointly discrete, the conditional probability mass function of X given Y = y is P(X = x; Y = y) pXY (x; y) pX jY (xjy) = = : P(Y = y) pY (y) If (X ; Y ) are jointly continuous, the conditional probability density function of X given Y = y is fXY (x; y) fX jY (xjy) = fY (y) which can be interpreted as follows: P(x 6 X 6 x + δx; y 6 Y 6 y + δy) P(x 6 X 6 x + δxjy 6 Y 6 y + δy) = P(y 6 Y 6 y + δy) fXY (x; y)δxδy ≈ = fX jY (xjy)δx: fY (y)δy MSc Financial Mathematics Fundamental Tools - Probability Theory III 8 / 20 Jointly distributed random variables Joint distribution functions Sum of independent random variables Conditional distributions Generating functions Independent random variables Conditional probability and expectation With conditional probability mass/density function, we can work out the conditional probability and expectation as follows: Conditional probability: (P x2A pX jY (xjy); P(X 2 AjY = y) = R A fX jY (xjy)dx: Conditional expectation: (P x xpX jY (xjy); E(X jY = y) = R 1 −∞ xfX jY (xjy)dx: MSc Financial Mathematics Fundamental Tools - Probability Theory III 9 / 20 Jointly distributed random variables Joint distribution functions Sum of independent random variables Conditional distributions Generating functions Independent random variables Example 3 e−x=y e−y Let the joint density function of X and Y be fXY (x; y) = y on 0 < x; y < 1 (and zero elsewhere). Find the conditional density fX jY , and compute P(X > 1jY = y) and E(X jY = y). MSc Financial Mathematics Fundamental Tools - Probability Theory III 10 / 20 Jointly distributed random variables Joint distribution functions Sum of independent random variables Conditional distributions Generating functions Independent random variables Independent random variables We say that X and Y are independent random variables if P(X 2 A; Y 2 B) = P(X 2 A)P(X 2 B) for any Borel set A; B 2 B(R). The following are equivalent conditions for X and Y being independent: E(f (X )g(Y )) = E(f (X ))E(g(Y )) for all functions f ; g. pXY (x; y) = pX (x)pY (y) or equivalently pX jY (xjy) = pX (x) in case (X ; Y ) are jointly discrete. fXY (x; y) = fX (x)fY (y) or equivalently fX jY (xjy) = fX (x) in case (X ; Y ) are jointly continuous. MSc Financial Mathematics Fundamental Tools - Probability Theory III 11 / 20 Jointly distributed random variables Joint distribution functions Sum of independent random variables Conditional distributions Generating functions Independent random variables Independence and zero correlation If X and Y are independent, then E(XY ) = E(X )E(Y ). This leads to: 1 Cov(X ; Y ) = 0, which also implies the correlation between X and Y is zero. 2 var(X + Y ) = var(X ) + var(Y ). The reverse is not true in general. The most importantly, zero correlation does not imply independence. See problem sheet. MSc Financial Mathematics Fundamental Tools - Probability Theory III 12 / 20 Jointly distributed random variables Sum of independent random variables Generating functions Sum of independent random variables We are interested in the following question: Suppose X and Y are two independent random variables. What is the distribution of X + Y then? The procedure is easier in the discrete case Example: Suppose X ∼ Poi(λ1) and Y ∼ Poi(λ2). MSc Financial Mathematics Fundamental Tools - Probability Theory III 13 / 20 Jointly distributed random variables Sum of independent random variables Generating functions Sum of independent random variables Assume (X ; Y ) are jointly continuous and let Z = X + Y . Then the CDF of Z is ZZ FZ (z) = P(Z 6 z) = P(X + Y 6 z) = fX (x)fY (y)dxdy x+y6z Z y=1 Z x=z−y = fX (x)dx fY (y)dy y=−∞ x=−∞ Z y=1 = FX (z − y)fY (y)dy: y=−∞ Differentiation w.r.t z gives the density of Z as Z y=1 fZ (z) = fX (z − y)fY (y)dy: y=−∞ MSc Financial Mathematics Fundamental Tools - Probability Theory III 14 / 20 Jointly distributed random variables Sum of independent random variables Generating functions Example Let X and Y be two independent U[0; 1] random variables. Find the density function of Z = X + Y . MSc Financial Mathematics Fundamental Tools - Probability Theory III 15 / 20 Jointly distributed random variables Probability generating functions Sum of independent random variables Moment generating functions Generating functions Probability generating function Probability generating function is only defined for a discrete random variable X taking values in non-negative integers f0; 1; 2; :::g. It is defined as 1 X X k GX (s) = E(s ) = s pX (k): k=0 View GX as a Taylor's expansion in s: 2 GX (s) = pX (0) + pX (1)s + pX (2)s + ··· (n) GX (0) We could then deduce pX (n) = n! , i.e. GX uniquely determines the pmf of X . In other words, if the probability generating functions of X and Y are equal, then X and Y have the same distribution. If X and Y are independent, X Y X Y GX +Y (s) = E(s s ) = E(s )E(s ) = GX (s)GY (s): Hence we can study the distribution of X + Y via GX (s)GY (s).