The Locally Gaussian Partial Correlation

Håkon Otneim^∗ Dag Tjøstheim^†

Abstract

It is well known that the dependence structure for jointly Gaussian variables can be fully captured

using correlations, and that the conditional dependence structure in the same way can be described using

partial correlations. The partial correlation does not, however, characterize conditional dependence in

many non-Gaussian populations. This paper introduces the local Gaussian partial correlation (LGPC),

a new measure of conditional dependence. It is a local version of the partial correlation coefficient

that characterizes conditional dependence in a large class of populations. It has some useful and novel

properties besides: The LGPC reduces to the ordinary partial correlation for jointly normal variables, and

it distinguishes between positive and negative conditional dependence. Furthermore, the LGPC can be

used to study departures from conditional independence in specific parts of the distribution. We provide

several examples of this, both simulated and real, and derive estimation theory under a local likelihood

framework. Finally, we indicate how the LGPC can be used to construct a powerful test for conditional

independence, which, again, can be used to detect Granger causality in time series.

1 Introduction

Estimation of conditional dependence and testing for conditional independence are extremely important

topics in classical as well as modern statistics. In the last two decades, for instance, there has been a very

intense development using conditional dependence in probabilistic network theory. This comes in addition to

conditional multivariate time series analysis and copula analysis.

For jointly Gaussian variables, conditional dependence is measured by the partial correlation coefficient.

Given three jointly Gaussian stochastic variables

X

,

X

and

X

and

,

X X

2

and

X X

are conditionally independent

is equal to zero. There is a

• 1
• 2
• 3
• 2

given

X

if and only if the partial correlation between

X

¹ given

rich literature on applications of the partial correlation coefficient to Gaussia³n networks, path analysis and

causality. The Gaussian assumption is strict, however, and it is easy to find non-Gaussian examples where

the partial correlation function completely fails in describing conditional dependence. We will give some

explicit examples of that in Section 4.3.

• 3
• 1

The purpose of this paper is to introduce a new concept for measuring conditional dependence. This concept

retains all the properties of the ordinary partial correlation in the Gaussian case, but seeks to avoid the

weaknesses of this measure in the non-Gaussian case. We do this by fitting a family of Gaussian distributions

to a given continuous multivariate distribution and exploiting the simple conditioning rules of the Gaussian

distribution loca l ly. This approach produces a new measure of conditional dependence, the local Gaussian

partial correlation (LGPC) which, being directly related to the ordinary partial correlation, is easy to interpret,

and reduces to the very same ordinary partial correlation in the Gaussian case. Moreover, it distinguishes

between positive and negative conditional dependence whereas competing non-linear measures report only

the strength of the conditional dependence on some non-negative scale.

The local view gives much more flexibility. It allows the conditional dependence to be stronger or weaker in

certain regions of a multivariate distribution than in others. This is of particular interest in finance, where

description of tail behavior is important. The local aspect also makes it possible to focus tests of conditional

independence to selected parts of the distribution, which may potentially increase the power of the test.

The local approach has been shown to be advantageous in other areas of statistics, such as the measurement of

nonlinear dependence, density estimation and spectral analysis, see for instance Tjøstheim and Hufthammer

^∗Corresponding author. Department of Business and Management Science, NHH Norwegian School of Economics. Helleveien

30, 5045 Bergen, Norway. [email protected]
^†University of Bergen

1

(2013), Otneim and Tjøstheim (2017) and Jordanger and Tjøstheim (2017). The present paper, then, represents the first attempt to model conditional dependence locally. There are several other approaches to modelling conditional dependence and testing for conditional independence in the non-Gaussian case in the literature. For the most part, they revolve around test statistics in conditional independence tests. Such statistics are usually based on measures of distance between some property of the sample and the corresponding property of the population under the null hypothesis of conditional independence. As a

consequence, the conditional dependence measures are not always easy to interpret, and can, as mentioned

above, not distinguish between negative and positive conditional dependence. We present a comprehensive

set of references to this literature in Section 5.

A quick summary of the paper is as follows: Partial correlation in the global case can be defined in several

ways, which all coincide in the Gaussian case. We will examine this in the following section and make a

choice of a measure that is most convenient for localization. The LGPC is defined in Section 3 by using a

distributional local Gaussian approach, where it will be seen that the LGPC can be introduced at several

levels of computational and theoretical complexity. Visualization is important when studying local conditional

dependence, which involves at least three scalar variables. Several simulated and a real data example are

given, in particular cases where the ordinary global partial correlation fails completely. Estimation theory

comes next in Section 4, whereas testing is treated in Section 5, including a fairly comprehensive comparison

with existing tests.

2 Conditional and partial correlation

Let

X

= (X ,   . . . , X_p) be a random vector. We will in this paper denote by (X⁽¹⁾, X⁽²⁾) a partition of

X

, and

1

in our treatment below, X⁽¹⁾ will consist of the first and second component in

X

so that X⁽¹⁾ = (X ,   X₂),

1

and X⁽²⁾ will contain the remaining components: X⁽²⁾ = (X ,   . . . , X_p). In any case, we will assume that the

mean vector µ and covariance matrix Σ of X exist, and we ³will partition them correspondingly, writing

• ꢀ
• ꢁ
• ꢀ
• ꢁ

µµ

Σ₁₁ Σ₁₂ Σ₂₁ Σ₂₂

12

µ =

,

and Σ =

,

(1)

where Σ₁₁ and Σ₂₂ are the covariance matrices of X⁽¹⁾ and X⁽²⁾ respectively. There are two main concepts

of correlation when X⁽²⁾ is given, the partial and the conditional correlation. They coincide in several joint

distributions, among them the Gaussian. We will in this section provide some details about this distinction

and explain our preference for using the partial correlation as a starting point when defining the LGPC. The partial variance-covariance matrix of X⁽¹⁾ given X⁽²⁾ is

Σ_11.2 = Σ₁₁ − Σ₁₂Σ⁻₂₂¹Σ₂₁.

(2)

Note that in the Gaussian case Σ_11.2 is the covariance matrix in the conditional (Gaussian) distribution of

X⁽¹⁾ given
X

⁽²⁾. Similarly, if X⁽¹⁾ = (X ,   X₂), the partial correlation between

X

and

X

2

is then defined as

(3)

• 1
• 1

(1,2)

σ

11.2

q

ρ

=

,

11.2
(1,1) (2,2)

• σ
• σ

11.2 11.2
(i,j)

where

σ

• refers to the element in position (i, j) in the partial covariance matrix
• Σ_11.2. This, in turn, in

11.2

the Gaussian case, can be identified with the correlation matrix in the conditional distribution of X⁽¹⁾ given

X⁽²⁾, and this fact, in particular equations (2) and (3) will serve as the starting point for our definition of

local partial correlation.

The partial variance or covariance of X⁽¹⁾ given X⁽²⁾ can also be considered as the variance or covariance

• between residuals of projections of X and X on the linear space spanned by X⁽²⁾
• ,

• 1
• 2

2

• ꢂ
• ꢃ

(2)

• ˜
• ˜

σ

• = Cov X_i − X_i(X ), X_j − X_j(X⁽²⁾) ,
• i, j  = 1, 2,

11.2

⁽²⁾) =

E

(

X_i)+Σ

Σ⁻¹

₍₂₎ (X⁽²⁾ −E(
X

⁽²⁾)) is the projection of X_i on

X

⁽²⁾. The expression

˜

where X_i

(

X

X_iX⁽²⁾

(2)

in terms of residuals is often taken a^Xs th^Xe starting point when defining the the global partial correlation, see

e.g. Lawrance (1976). Moreover, using eq. (3), it can be shown, that all of the partial correlations between

two variables X_i and X_j in a set of variables

V

having correlation matrix Ω = {ω_{i j}}

=

{ρ_{X X}

}

and given

• i
• j

the other variables in V (i.e. given V \{X_i, X_j}) is obtained as

p_ij ρ_{X X .V  \X X}  = −

,

√

• i
• j
• i
• j

p_iip_jj

where p_ij is element (i, j) in the precision matrix Σ⁻¹ The conditional covariance of X and X given X⁽²⁾ is defined by
.

• 1
• 2

• ꢂ
• ꢃ

Cov(X ,   X |X⁽²⁾) = E (X − E(X |X⁽²⁾))(X − E(X |X⁽²⁾))|X⁽²⁾
,

• 1
• 2
• 1
• 1
• 2
• 2

and this is the covariance between

X

and

X

in the conditional distribution of (X ,   X₂) given X⁽²⁾. The

• 2
• 1

conditional covariance matrix can be ¹written

• ꢀ
• ꢁ

σ_11|X(2) σ_12|X(2) σ_21|X(2) σ_22|X(2)

Σ_11|2

=

resulting in the conditional correlation

σ_12|X(2) σ_11|X(2) σ_22|X(2) ρ_12|X(2)

and

=

,

√

which is the correlation between

coinciding with ρ

• X
• X

in the conditional distribution of (X ,   X₂) given X⁽²⁾, thus

• 1
• 2
• 1

of eq. (3) in the Gaussian case.

11.2

Baba, Shibata, and Sibuya (2004) give the following result for the relationship between conditional and

partial quantities in a more general situation.

Corollary 1. For any random vectors X⁽¹⁾ = (X ,   X₂) and X⁽²⁾ = (X ,   . . . , X_p) the following two conditions

• 1
• 3

are equivalent:

(i) E(X⁽¹⁾|X⁽²⁾) = α + BX⁽²⁾ for a vector α and a matrix B, and Σ_11|2 independent of X⁽²⁾

.
(ii) Σ_11.2 = Σ_11|2 almost surely.

Either of the conditions in Corollary 1 is, according to Baba, Shibata, and Sibuya (2004), valid not only

for the multivariate normal, but also for elliptical, multivariate hypergeometric, multinomial and Dirichlet

distributions. Moreover, for the multivariate Gaussian, zero partial correlation (or conditional correlation)

is equivalent to conditional independence between

X

and

X

given X⁽²⁾. We will now use such results to

construct a generalized partial correlation function, w¹hich cha²racterizes conditional dependence in a broader

class of distributions.

3 The local Gaussian partial correlation

3.1 Definition

In all of the following it is assumed that the

X

-variables are continuous and have a density function. In order

to localize the partial correlation we need to look at the concept of local Gaussian approximations. Let

f

3

be a multivariate density function. Given a point x, we can approximate

f

in a neighborhood of

x

by a

(4) multivariate Gaussian density

• ꢄ
• ꢅ

• 1
• 1

2

T

• ψ(x, v) =
• exp − (v − µ(x)) Σ⁻¹(x)(v − µ(x))

,

• p/2
• 1/2

(2π) |Σ(x)|

where

x

= (x .   . . , x_p),

µ

(

x) = {µ_i and

(

x

)

}

Σ

(

x) = {σ_ij

(

x

)

}

for i, j  = 1, . . . p. Moving to another point

by a

)}, and

y

,

1

there is another (generally different) Gaussian approximation ψ(y, v). In this way we approximate

family of multivariate Gaussian densities defined by a set of smooth parameter functions {µ( , Σ

if is itself a Gaussian density, then the parameter functions collapse to constants corresponding to the true

parameter values, and (x). Hjort and Jones (1996) provide the general framework for estimating

f

x

• )
• (

x

f ψ

(

x

)

≡ f

such parameter functions non-parametrically from a given data set using a local likelihood procedure, and

the basic idea in the following treatment is to replace the components in the partial covariance matrix (2) by

their locally estimated counterparts in order to obtain a local measure of conditional dependence.