Causal Inference from Experimental Data

Decision problem Causal inference Physical and metaphysical array Null hypotheses Symmetry models Mixed model Conclusion

Philip Dawid

30th Fisher Memorial Lecture

10 November 2011

Philip Dawid Causal Inference from Experimental Data Decision problem Causal inference Physical and metaphysical array hypothetical approach Null hypotheses counterfactual approach Symmetry models data Mixed model Conclusion Decision problem

I have a headache. Should I take aspirin?

I Two possible treatments:

I t: take 2 aspirin I c: do nothing

I Outcome measure Y : time it takes my headache to go away

I Loss: L(y) if Y = y.

I Whichever treatment applied, Y is uncertain

I model as random

Which treatment should I choose?

ν y t L(y) t Y~Pt ν 0

c Y~Pc y L(y) νc

Figure: Decision tree

Choose t if EPt {L(Y )} ≤ EPc {L(Y )}

I Yt if t applied

I Yc if c applied

I Both pre-existing: joint distribution for (Yt , Yc )

I Application of t [c] uncovers value of Yt [Yc ]

Choose t if E{L(Yt )} ≤ E{L(Yc )}

In either approach, relevant data might be gathered to:

I reduce uncertainty

I estimate “objective” distributions

I OBSERVATIONAL STUDIES

I EXPERIMENTAL STUDIES

Causal inference: Use such data to help predict my outcome (under any hypothesised treatment) and so guide my decision

Philip Dawid Causal Inference from Experimental Data Decision problem Causal inference Physical and metaphysical array prediction Null hypotheses experiment Symmetry models causal eﬀect Mixed model Conclusion Predictive causal inference

Suppose I have done a study to measure the responses Y for two groups of individuals:

Treatment group: Given aspirin (t), responses Yt1,..., Ytn

Control group: No aspirin (c), responses Yc1,..., Ycm

For my own decision problem, I need to predict my own response Y under two hypothesised scenarios:

I I will take aspirin, t: Y ∼ Pt (or, potential response Yt )

I I will not take aspirin, c: Y ∼ Pc (or, potential response Yc )

IF I can consider myself exchangeable with the members of the treatment group (on all relevant pre-treatment variables), then I can

I estimate my Pt I predict Y , given t

I predict Yt from their responses. Likewise,IF I can consider myself exchangeable with the members of the control group, then I can estimate my Pc from their responses.

For both to be valid, it is necessary (though not suﬃcient) for the two groups to be exchangeable with each other.

This is the reason why we must take care that in our study we are comparing like with like:

I randomisation

I blocking

I covariates

I ...

Treatments t ∈ T , units u ∈ U

Each set may be structured, e.g.:

I factorial/nested/. . . treatments

I row-by-column, split-plot . . . layout

I blocking, covariates,. . .

Can apply any t to any u and measure outcome variable Y

I Select a set U0 of experimental units

I Assign treatments to them

I map τ : U0 → T

— each possibly with random element

Want to assess/compare the eﬀects of the treatments

I ideally for new unit u0 (= me?)

Hypothetical approach (Fisher)

Consider distribution Pt of response Yu of unit u if assigned treatment t

e.g. Yu ∼ N(αt , φY )

Causal contrast is e.g. Et (Yu) − Et0 (Yu)

I How much longer/shorter I expect my headache to last, if I were to take aspirin rather than nothing

I αt − αt0

I Average Causal Eﬀect, ACEu

Counterfactual approach (Neyman, Rubin)

Consider values (ηtu : t ∈ T ) of the responses of each unit u to each treatment t

I POTENTIAL RESPONSES

Causal contrast is e.g. ηtu − ηt0u

I How much longer/shorter my headache would last, if I were to take aspirin rather than nothing

I Individual Causal Eﬀect, ICEu

Counterfactual approach needs to posit the simultaneous existence (and joint distribution) of ηtu and ηt0u.

I We observe ηtu for u ∈ U0, t = τ(u)

I We can never observe both ηtu and ηt0u

I So we can never observe ICEu

Fundamental Problem of Causal Inference (FPCI) (Holland)

Philip Dawid Causal Inference from Experimental Data Decision problem Causal inference Physical and metaphysical array physical array Null hypotheses metaphysical array Symmetry models comparisons Mixed model Conclusion The physical array

t1 Y11 Y12 ······ t2 ········· Y2n t3 Y31 ······ Y3n

u1 u2 u3 ······ uN−1 uN t1 η11 η12 η13 ······ η1,N−1 η1N t2 η21 η22 η23 ······ η2,N−1 η2N t3 η31 η32 η33 ······ η3,N−1 η3N

Deﬁne

I uti := ith unit to which treatment t is applied

I Yti := ηt,uti

t1 η11 η13 ······ t1 Y11 Y12 ······ t2 ········· η2N = t2 ········· Y2n t3 η32 ········· t3 Y31 ······ Y3n

Any model for the (unobservable) metaphysical array induces a model for any observable physical array

But this map is many-to-one

I because any within-unit dependence of the (ηtu) is not reﬂected in physical array

Metaphysical modelling appears more ﬂexible

I Is this a good thing?

Diﬀerent metaphysical models may be observationally indistinguishable

I What if we use that ﬂexibility to make diﬀerent inferences in such cases?

ηtu = αt + βu + tu

βu ∼ N(0, φβ), tu ∼ N(0, φ), indep.

Then Yti ∼ N(αt , φY ), indep., with φY = φβ + φ

TUA: φ = 0 I No observable (physical) consequences!

I Hypothesis of TUA untestable So we should make identical inferences, whether or not TUA assumed (??)

Philip Dawid Causal Inference from Experimental Data Decision problem Causal inference Physical and metaphysical array Neyman Null hypotheses Fisher Symmetry models Mixed model Conclusion Neyman’s null hypothesis Statistical problems in agricultural experimentation (1935)

H0: ηt does not depend on t, where −1 X ηt := n0 ηtu u∈U0 – average outcome over all experimental units, if they were all to receive treatment t Remarks: I refers to metaphysical array, not to any population parameter I depends on scale of measurement I depends on units U0 used in experiment I we can never observe both ηt and ηt0 (FPCI) Philip Dawid Causal Inference from Experimental Data Decision problem Causal inference Physical and metaphysical array Neyman Null hypotheses Fisher Symmetry models Mixed model Conclusion

TUA: ηtu = αt + βu

With TUA, Neyman’s null hypothesis becomes

αt [ηtu] does not depend on t

Remarks:

I No longer depends on U0 i.i.d. I If βu ∼ N(0, φ) (say), we have observations

Yti ∼ N(αt , φ) indep.

I Standard (normal theory or randomisation) analysis

Philip Dawid Causal Inference from Experimental Data Decision problem Causal inference Physical and metaphysical array Neyman Null hypotheses Fisher Symmetry models Mixed model Conclusion No TUA

Neyman considers the validity of standard ANOVA tests based on expected mean squares under randomisation Under his own “average null” hypothesis, usual mean squares for treatments and for error need not have the same expectation if we can not assume TUA

I OK for randomised blocks

I but not for Latin square

So: Is the Latin square design biased?

W & K extend (and correct) Neyman’s analysis

I more complex layouts

I null hypotheses relating to “metaphysical averages” over larger — but still ﬁnite — set U1 of plots

I also possibly over larger set T1 of treatments, from which experimental treatments T chosen at random

I analysis depends on U1, T1, and varies according as TUA assumed or not

As we let U1 become inﬁnite in all directions, we recover standard results, and TUA becomes irrelevant

“. . . apparent inability to grasp the very simple argument by which the unbiased character of the test of signiﬁcance might be demonstrated”

“the sole source of his statement . . . appears to lie in the peculiarity of his own notation”

COPSS R. A. Fisher Lecture 2004

“Fisher never used the potential outcomes framework, originally proposed by Neyman in the context of randomized experiments, and as a result he provided generally ﬂawed advice concerning the use of analysis of covariance to adjust for posttreatment concomitants in randomized trials”

“the treatments are wholly without eﬀect”

Metaphysical: Values of (ηtu) do not depend on t

Physical: Joint distribution of (Yti ) takes no account of treatment asignments Cox (1958): Variation between observed treatment means is consistent with sampling from a homogeneous group of observations

All lead back to standard (ANOVA, permutation. . . ) tests

Philip Dawid Causal Inference from Experimental Data Decision problem invariance Causal inference physical array Physical and metaphysical array covariances Null hypotheses PLM Symmetry models data decomposition Mixed model prediction Conclusion extensions Symmetry Modelling (APD 1988) Develop implications of invariance of attitudes to the data under suitable permutations

I randomisation

I exchangeability Independent of scale of measurement, TUA, . . . Group invariance ⇒

I Structure for covariances between observations

I Synthetic “population linear model” (PLM)

I Decomposition of data into uncorrelated strata Null hypothesis= larger symmetry group

Physical array (Yti )

t1 Y11 Y12 ··· Y1n t2 Y21 Y22 ··· Y2n t3 Y31 Y32 ··· Y3n

Full model: can permute t, and i within each level of t Te /eI Null model: can permute all values arbitrarily Tg· I

 0  γU (t 6= t )  0 0 Full model: cov (Yti , Yt0i0 ) = γT (t = t , i 6= i )  0 0  γ0 (t = t , i = i )

Null model: γT = γU

Full model:

Yti = µ + αt + ti (m, φµ)(0, φα)(0, φ)

(all uncorrelated)

Null model: φα = 0

Full model: Yti = Y.. + (Yt. − Y..) + (Yti − Yt.)

Null model: | {z }

Yti = Y.. + (Yti − Y..)

equality of expected mean-squares

⇒ standard F -test

Use PLM for prediction/ causal inference/ decision-making:

Machine^ ∗ Worker^ / Rung

Ymwr = µ + αm + βw + γmw + mwr (m, φµ)(0, φα)(0, φβ)(0, φγ)(0, φ)

old machine m, old worker w, new run r

Machine^ ∗ Worker^ / Rung

Ymwr = µ + αm + βw + γmw + mwr (m, φµ)(0, φα)(0, φβ)(0, φγ)(0,φ )

old machine m, new worker w

Machine^ ∗ Worker^ / Rung

Ymwr = µ + αm + βw + γmw + mwr (m, φµ)(0, φα)(0,φ β)(0,φ γ)(0,φ )

new machine m, new worker w

Machine^ ∗ Worker^ / Rung

Ymwr = µ + αm + βw + γmw + mwr (m, φµ)(0,φ α)(0,φ β)(0,φ γ)(0,φ )

Straightforward extensions to balanced poset/distributive block structure designs with full symmetry

Can apply general approach to problems with restricted symmetry:

I more complex group theory

I invariant subspaces need not be irreducible

I null hypothesis (larger symmetry group) may lead to splitting of a reducible subspace

Philip Dawid Causal Inference from Experimental Data Decision problem Causal inference Physical and metaphysical array PLM Null hypotheses null hypotheses Symmetry models Mixed model Conclusion Mixed model

Machine ∗ Worker^ / Rung

PLM: Ymwr = (µ)m + (αw )m + mwr

I µ unstructured: ﬁxed eﬀects

I αw ∼ (0, Φ), uncorrelated vectors across w

I mwr ∼ (0, φm), all uncorrelated

Machine / Shiftg / Rung

No diﬀerences between workers:

Machine / Rung

“Fixed” eﬀects are actually “random”:

Machine^ ∗ Worker^ / Rung Relationships between data decompositions determine tests Philip Dawid Causal Inference from Experimental Data Decision problem Causal inference Physical and metaphysical array Null hypotheses Symmetry models Mixed model Conclusion Conclusion

For causal inference and experimentation:

I Think about decision and prediction

I Think hypothetical (Fisher) rather than counterfactual (Neyman)

For building models and hypotheses:

I Think about structure of attitudes to the data

Philip Dawid Causal Inference from Experimental Data Decision problem Causal inference Physical and metaphysical array Null hypotheses Symmetry models Mixed model Conclusion

THANK YOU!

Philip Dawid Causal Inference from Experimental Data