F(X,Y) Is the Joint Pdf of X and Y

1 2

INDEPENDENCE, COVARIANCE AND 2. (Still assuming Y is independent of X) Let h(y) be CORRELATION the common pdf of the conditional distributions Y|X. Then for every x, Independence: f (x, y) Intuitive idea of "Y is independent of X": The h(y) = f(y|x) = f (x) , distribution of Y doesn't depend on the value of X. X where f(x,y) is the joint pdf of X and Y. In terms of the conditional pdf's: "f(y|x) doesn't depend on x." Therefore

Caution: "Y is not independent of X" means simply f(x,y) = h(y) fX(x) that the distribution of Y may vary as X varies. It # doesn't mean that Y is a function of X. f (x, y)dx fY(y) = $"# X, Y

If Y is independent of X, then: # h(y) f (x) dx = $"# X 1. µx = E(Y|X = x) does not depend on x. # f (x) dx Question: Is the converse true? That is, if E(Y|X = x) = h(y) $"# X = h(y) = f(y|x) does not depend on x, can we conclude that Y is independent of X? In other words: If Y is independent of X, then the conditional distributions of Y given X are the same as the marginal distribution of Y.

! !

! 3 4

3. Now (still assuming Y is independent of X) we Summary: The following conditions are all have equivalent:

f (x, y) 1. X and Y are independent.

fY(y) = f(y|x) = f (x) , X 2. f (x,y) = f (y)f (x) so X,Y Y X

f (y)f (x) = f(x,y). Y X 3. The conditional distribution of Y|(X = x) is

independent of x. In other words: If Y is independent of X, then the joint

distribution of X and Y is the product of the marginal 4. The conditional distribution of X|(Y = y) is distributions of X and Y. independent of y.

Exercise: The converse of this last statement is true. 5. f(y|x) = f (y) for all y. That is: If the joint distribution of X and Y is the Y

product of the marginal distributions of X and Y, 6. f(x|y) = f (x) for all x. then Y is independent of X. X

Additional property of independent random Observe: The condition f (y)f (x) = f(x,y) is Y X variables: If X and Y are independent, then E(XY) = symmetric in X and Y. Thus (3) and its converse E(X)E(Y). (Proof might be homework.) imply that : Y is independent of X if and only if X is independent of Y. So it makes sense to say "X and Y are independent." Covariance: For random variables X and Y,

Cov(X,Y) = E([X - E(X)][Y - E(Y)])

Comments:

! 5 6

• Cov (capital C) " population cov (or Cov-hat) " sample. 1. Cov(X, X) = Cov is a parameter " population cov is a statistic " calculated from the sample. 2. Cov (Y, X) =

• Compare and contrast with definition of Var(X). 3. Cov (X, Y) = E([X - E(X)][Y - E(Y)]) =

• If X and Y both tend to be on the same side of their respective means (i.e., both greater than or both less than their means), then [X - E(X)][Y - E(Y)] tends to be positive, so Cov(X,Y) is positive. Similarly, if X and Y tend to be on opposite sides of their respective means, then Cov(X,Y) is negative. If there is no trend of either sort, then Cov(X,Y) should be zero. Thus covariance roughly In words … measures the extent of a "positive" or "negative" trend in the joint distribution of X and Y.

• Units of Cov(X,Y)? (Compare with the alternate formula for Var(X).)

4. Consequence: If X and Y are independent, then:

Cov(X,Y) =

Note: Converse false. (Future homework.)

Properties: 5. Cov(cX, Y) =

! ! ! ! 7 8

Bounds on Covariance Cov(X, cY) =

!X = population standard deviation Var(X) of X. 6. Cov(a + X,Y) = (Do not confuse with sample standard deviation Cov(X, a +Y) = = s or s.d. or "ˆ )

7. Cov(X + Y, Z)) = !Y defined similarly.

Cov(X, Y + Z) = ! X Y Consider the new random variable + . "X "Y 8. Var(X + Y) = Since Variance is always " 0,

# X Y & % ( (*) 0 ! Var% + ( $ "X "Y ' X Y X Y , Consequence: If X and Y are independent, then = Var( ) + Var( ) + 2Cov( ) "X "Y "X "Y 1 1 2 Var(X + Y) = = 2 Var(X) + 2 Var(Y) + Cov(X,Y) "x "Y "X"Y

= (converse false!) Cov(X,Y ) When else might this be true? = 2[1 + ]. ! "X"Y

Therefore:

! ! ! ! ! ! !

! 9 10

Cov(X,Y ) Equality in (**) # equality in (*) -- i.e, (**) " -1 "X"Y # X Y & Equivalently: Cov(X, Y) " -"X"Y . % ( Var% + ( = 0 # $ "X "Y ' $ X Y ' & ) Looking at Var& # ) similarly shows: X Y % "X "Y ( + " " is constant -- X Y Cov(X,Y ) say, (***) ! 1 X Y "X"Y + = c. "X "Y Equivalently: Cov(X, Y) ! "X"Y . X Y (Note that c must be the mean of + , "X "Y

(details left to the student) µX µY which is + ) "X "Y Combining (**) and (***): This in turn is equivalent to Cov(X,Y ) $ X ' & ) ! 1 Y = "Y & # + c) " X"Y % "X (

Equivalently: |Cov(X, Y)| !"X"Y or

#Y Y = " X + #Y c # X

This says: The pairs (X,Y) lie on a line with negative sl!o pe (namely, -!Y/!X)

! (Converse is also true -- details left to the student.)

! ! !

! ! ! !

! !

! ! ! 11 12

#Y Properties of $: Note: the line has slope " and # X "Y • Negative $ indicates a tendency for the variables X y-intercept µX + µY . "X and Y to co-vary negatively.

Cov(X,Y ) • Positive $ indicates a tendency for the variables X Similarly, = +1 exactly when the pairs "X"Y and Y to co-vary positively. (X,Y) lie on a line with positive slope. • -1 ! $ ! 1 Correlation: The (population) correlation coefficient of the random variables X and Y is • $ = -1 # all pairs (X,Y) lie on a straight line with negative slope. Cov(X,Y ) "X,Y = . #X#Y • $ = 1 # all pairs (X,Y) lie on a straight line with positive slope. Note: • $ for short. • Units of $ ? • $ is a parameter (refers to the population). • Do not confuse with the sample correlation • $ is the Covariance of the standardized random coefficient (usually called r): a statistic (calculated X "µX Y "µY variables and . (Details left to from the sample). # X #Y student.)

• $ = 0 # Cov(X,Y) = 0.

! !

! ! 13 14

Uncorrelated variables: Sample variance, covariance, and correlation

X and Y are uncorrelated means $X,Y = 0 (or (x1,y1), (x2,y2), … , (xn,yn) sample of data from the equivalently, Cov(X,Y) = 0). joint distribution of X and Y

Examples: Sample covariance:

1. X and Y are independent % uncorrelated. (Why?) cov(x,y) (or )

2. X uniform on the interval [-1, 1]. 1 n 2 #(xi " x )(yi " y ) Y = X . = n "1 X and Y are uncorrelated (details homework) i=1

X and Y not independent. (E(Y|X) not constant) Sample correlation coefficient

cov( x, y) $ is a measure of the degree of a “overall” r (or "ˆ ) = . nonconstant linear relationship between X and Y. sd(x)sd(y) Example 2 shows: Two variables can have a strong nonlinear relationship and still be uncorrelated. • Estimates of the corresponding population parameters. • Analogous properties.