Wishart Distribution Wishart Distribution

I Consider as usual a random sample x1, x2,..., xN from Np(µ, Σ).

I Recall: the sample covariance matrix S satisﬁes 1 S = A, N − 1 where N X 0 A = (xα − x¯)(xα − x¯) α=1

Statistics 784 NC STATE UNIVERSITY 1 / 13 Multivariate Analysis Wishart Distribution

I Write  0  x1 0  x2  X =    .   .  0 xN Then I  0  (x1 − x¯) 0  (x2 − x¯)  1 X =   = I − 1 10 X dev  .  N N N  .  N 0 (xN − x¯) and 1 A = X0 X = X0 I − 1 10 X dev dev N N N N

Statistics 784 NC STATE UNIVERSITY 2 / 13 Multivariate Analysis Wishart Distribution

I The Helmert matrix is the orthogonal matrix

 √1 √−1 0 ... 0  2 2  √1 √1 √−2 ... 0   6 6 6   . . . . .   ......  HN =  . . . .   1 1 1 −(N−1)   √ √ √ ... √   N(N−1) N(N−1) N(N−1) N(N−1)  √1 √1 √1 ... √1 N N N N

Statistics 784 NC STATE UNIVERSITY 3 / 13 Multivariate Analysis Wishart Distribution

I If we partition HN as ! H(1) H = N N √1 √1 √1 ... √1 N N N N then 0 1 I = H0 H = H(1) H(1) + 1 10 N N N N N N N N

I So 1 0 I − 1 10 = H(1) H(1) N N N N N N and

1 0 0 A = X0 I − 1 10 X = X0H(1) H(1)X = H(1)X H(1)X N N N N N N N N

Statistics 784 NC STATE UNIVERSITY 4 / 13 Multivariate Analysis Wishart Distribution

I If Z = HN X then the rows of Z are uncorrelated, and hence independent, with the ﬁrst n = N − 1 distributed as Np(0, Σ), and the last distributed as √ Np Nµ, Σ So we can write I n X 0 A = zαzα α=1

where z1, z2,..., zn is a random sample from Np(0, Σ).

I A is said to have the Wishart distribution Wp(Σ, n).

Statistics 784 NC STATE UNIVERSITY 5 / 13 Multivariate Analysis Wishart Distribution Distribution of A

I First assume that Σ = Ip. Z(1) (1)X I Then the elements of = HN are iid N(0, 1). I We can use Gram-Schmidt orthogonalization to write

Z(1) = WT0

where:

I T is (p × p) lower-triangular; I the columns of W are mutually orthogonal and normalized. 0 and A = Z(1) Z(1) = TT0.

Statistics 784 NC STATE UNIVERSITY 6 / 13 Multivariate Analysis Wishart Distribution

I Anderson shows that, when n ≥ p:

I The non-zero elements of T are independent; I The sub-diagonal elements are all distributed as N(0, 1); 2 2 I The diagonal element ti,i satisﬁes ti,i ∼ χn−i+1. 1 I That is, the joint density of these 2 n(n + 1) variables is

p !  p i  Y 1 X X tn−i exp − t2 i,i  2 i,j  i=1 i=1 j=1 p 1 p(n−2) 1 p(p−1) Y 1 2 2 π 4 Γ (n + 1 − i) 2 i=1

Statistics 784 NC STATE UNIVERSITY 7 / 13 Multivariate Analysis Wishart Distribution

I The result for general Σ is that the joint density is

p ! Y 1 tn−i exp − traceΣ−1TT0 i,i 2 i=1 p 1 p(n−2) 1 p(p−1) 1 n Y 1 2 2 π 4 det(Σ) 2 Γ (n + 1 − i) 2 i=1

I A is symmetric, and hence varies in a space of the same 1 dimensionality, 2 p(p + 1), as T. I The density of A is found by transformation: 1 (n−p−1) 1 −1 det(A) 2 exp − trace Σ A 2 p 1 pn 1 p(p−1) 1 n Y 1 2 2 π 4 det(Σ) 2 Γ (n + 1 − i) 2 i=1

Statistics 784 NC STATE UNIVERSITY 8 / 13 Multivariate Analysis Wishart Distribution Characteristic Function

I Let Θ be a real, symmetric (p × p) matrix. I Then p p−1 p X X X trace(AΘ) = Ai,i θi,i + 2 Ai,j θi,j i=1 i=1 j=i+1

I So the joint characteristic function of A1,1, A2,2,..., Ap,p, 2A1,2, 2A1,3,..., 2Ap−1,p is E{exp[itrace(AΘ)]}

I Since A can be written as n X 0 A = zαzα, α=1 n ! n X 0 X 0 trace(AΘ) = trace zαzαΘ = zαΘzα α=1 α=1

Statistics 784 NC STATE UNIVERSITY 9 / 13 Multivariate Analysis Wishart Distribution

I It follows that 1 E{exp[itrace(AΘ)]} = 1 n det(Ip − 2iΘΣ) 2

I Note that if matrices Ai are independent, with Ai ∼ Wp(Σ, ni ), then ! X X Ai ∼ Wp Σ, ni i i

I This follows either from the representation of Ai or from multiplying the individual characteristic functions.

Statistics 784 NC STATE UNIVERSITY 10 / 13 Multivariate Analysis Wishart Distribution Linear Transformation

0 I If A ∼ Wp(Σ, n) and B = CAC for some nonsingular C, then

0 B ∼ Wp(CΣC , n)

I This also follows either from the representation of A or from transforming the characteristic function.

Statistics 784 NC STATE UNIVERSITY 11 / 13 Multivariate Analysis Wishart Distribution Marginal Distribution

I If A ∼ Wp(Σ, n) and A and Σ are partitioned as

A A Σ Σ A = 1,1 1,2 , Σ = 1,1 1,2 , A2,1 A2,2 Σ2,1 Σ2,2

then marginally, Ai,i ∼ Wpi (Σi,i , n), i = 1, 2. I If Σ1,2 = Σ2,1 = 0, then furthermore A1,1 and A2,2 are independent.

I This also follows either from the representation of A or from the structure of the characteristic function.

Statistics 784 NC STATE UNIVERSITY 12 / 13 Multivariate Analysis Wishart Distribution Conditional Distribution

I If A ∼ Wp(Σ, n) and A and Σ are partitioned as above, then

−1 A1,1·2 = A1,1 − A1,2A2,2A2,1 ∼ Wp1 (Σ1,1·2, n − p2)

−1 and is independent of A2,2 and A1,2A2,2. I This is proved using the representation of A using other arguments presented earlier.

Statistics 784 NC STATE UNIVERSITY 13 / 13 Multivariate Analysis