Causal Issues in Personality Psychology: Some Challenges from and for the Rubin and Campbell Perspectives

Stephen G. West Arizona State University Randomized Experiment

Sir Ronald Fisher

Randomization, Manipulation Open Systems

E(XTreatment) = E(XControl) for any covariate X

Unbiased Estimate of Treatment Effect

Strongest Design for Causal Inference With Greatest Statistical Power BUT

Fisher’s Work’s was in agriculture

This context does not fully characterize research with human participants Different Context--Corn plants do NOT

Raise ethical or practical concerns about randomization Fail to comply with Treatment Find a better Treatment in Another Field Move away—lost to measurement Refuse to answer questionnaires Rubin’s Potential Outcomes Approach

• PhD Harvard Statistics • Medicine, Public Health, Economics • Applied • Manipulated Treatments • Precise Assumptions—Deductive, Mathematical • Highly Quantitative • Emphasis on Potential Outcomes Logic • Strong Basis for Causal Inference When – Broken Randomized Experiments – Cannot Randomly Assign Treatments

Rubin (1974, 1978; 2005); Imbens & Rubin (in press) Rubin’s Potential Outcomes Approach

Starts With Platonic Ideal: Compare response of unit [person] to T with response of unit to C under identical conditions and at identical time. Rubin’s Potential Outcomes Approach

Starts With Platonic Ideal: Compare response of unit [person] to T with response of unit to C under identical conditions and at identical time. Potential Outcomes Model

TREATMENT

 i  YT(u) - YC(u)

Unit –level Causal Effect CONTROL

8 Fundamental Problem of Causal Inference

“It is impossible to observe the value of YT(u) and YC(u) on the same unit [person] and, therefore, it is impossible to observe the effect of T on u.” Holland (1986, p. 947) For each unit, there is an observed outcome and nonobserved potential outcome Need to define approximations to Platonic ideal and the necessary assumptions for the approximation to be correct. Extension: Average Causal Effect

TREATMENT

1 n 1 n   YiT  YiC n i1 n i1

CONTROL Average Causal Effect

10 One approximation to Ideal

RANDOMIZATION Given additional assumptions— Clear definition of T and C conditions full treatment compliance no attrition Stable Unit Treatment Value Assumption Groups balanced on all covariates Unbiased Estimate of Average Treatment Effect Rubin Causal Model

CONTROL TREATMENT CONTROL TREATMENT

E(Yi1) = E(Yi1 | zi = 1) E(Yi1) ≠ E(Yi1 | zi = 1) E(Yi0) = E(Yi0 | zi = 0) E(Yi0) ≠ E(Yi0 | zi = 0) 12 Randomization: Fails or Not Possible

• Broken Randomized Experiments -Treatment noncompliance -Missing Data -Interference

Regression Discontinuity Design -quantitative assignment rule

• Observational Studies -Unknown assignment rule -Propensity scores match on all measured covariates (strong ignorability—no unobserved covariates) -Permit balance checks on measured covariates Challenges for Personality from Rubin

• How do we define the potential outcomes? • Treatment Implied—Begin, (End) • E. G. Gender and Employment – Chromosome transplant – Raised opposite gender – Job applicant description Comparisons could be much better specified Conditioning (e.g., Propensity Scores) stronger Define Alternative Treatment, Potential Outcome?? Some Challenges for Rubin 1

• Can Manipulate Some Personality Variables – Self Esteem, Depressed Mood

Do manipulations represent intended construct? (Rubin’s focus is operational—on manipulation) Can we generalize these results to actual behavior of depressed individuals? Some Challenges for Rubin 2

How do we study processes over time? (e.g., normal personality development; developmental psychopathology)?

How do we study how people select situations, alter environments, etc.—What is selection bias for Rubin is potentially some of the richest data in personality research. Campbell’s Working Scientist Approach

• PhD UC Berkeley Psychology • Psychology, Education • Applied, Basic • Manipulated, Non-Manipulated Variables Logic Qualititative, Inductive, Directional Effects • Threats to Validity • Emphasis on Design Elements, Pattern Matching Concern with construct validity, Concern with generalization Campbell-Some Key Concepts for Causal Inference Plausible Threats to Internal Validity -Specific reasons why we can be partly or completely wrong in our causal inference -Depend on Design Obtained pattern of results Prior results and theory -Extensive (exhaustive?) list representing “an accumulation of our field’s criticisms of each others research” Threats—extend to other forms of validity Campbell on Plausible Threats

“We took the position that there could be lots of threats to validity that were logically uncontrolled but that one should not worry about unless they were plausible. The general spirit was that any interpretation of a body of data or research should be regarded as innocent until judged guilty for plausible reasons, as determined through the scientific method of mutual criticism.” Campbell (1988, p. 317). Designs with T & C Conditions; Pretest (baseline) and Posttest Baseline Treatment Outcome X T Y X C Y

• Selection x Maturation • Selection x History • Selection x Instrumentation • Selection x Statistical Regression • Differential Attrition Design Elements and Pattern Matching Identify specific confounding factors that may produce observed results—threats to validity Add design elements to address threats Assess Match of Pattern of Results to Scientific Hypothesis, Potential Threats Working Scientist Approach—Logically rule out threat Campbell: Targeted Design Features Ruling out Threats to Internal Validity

“[Each] basic design can be improved substantially by adding thoughtfully chosen design features to address threats to validity that are plausible in the context of the experiment.”

Shadish, Cook, & Campbell (2002, p.145) Design Element 1: Nonequivalent Dependent Measures Design Element 2: Multiple Pretest, Posttest Measures Beyond Causal Inference 1

But, what about the theoretical meaning of constructs? Campbell Construct Validity of Conceptual Independent, Dependent Variables Rubin Largely mute— but raises challenges for Campbell in terms of precise specification, equivalence Campbell’s Construct Validity

Preference for Manipulated IVs, but willing to consider non-manipulated variables Specific Principles for Determining whether Construct represents manipulation, measure Does Pattern of Results Match That predicted by theory? Qualitative, Logical, not Quantitative Construct Validity 2

Useful Within Personality Research Convergent, Discriminant Validity (across measures, over time) Multitrait-Multimethod matrices Problems: Qualititative Answers (yes/no), Theory Embeddedness, Decreased Certainty about proper comprisons, controls, Beyond Causal Inference 2

But what about generalization of results beyond the original study? Units, Treatments, Observations, Settings Campbell—clear extratheoretical priniciples— Cook’s grounded theory Rubin—mechanisms for generalization are limited. In quest for clear causal inference, it is easy to overlook these other research priorities, particularly important in personality research. Challenges from Personality Research

Personality Processes—Within Person Individual Differences—Between Persons These involve potentially different questions, answers, different designs, analyses Limited Ability to Manipulate Stable Individual Differences Strong Interest in Processes, Development Humans are active in choosing, constructing environments—not selection bias, but some primary data of personality. Choice in Economics Some Possible Lessons 1 Rubin—Careful attention to possible outcomes, ruling out selection issues, assumptions, quantification Campbell—specification of threats, prevention of problems, addition of design features, pattern matching. Fallibility, cultivate critics Both are incomplete—consider epidemiology (biological mechanisms, Susser, Robbins), Pearl’s approach from computer science (process), economics, control systems engineering. But, personality involves own issues not well addressed by standard theories—within vs. between, environmental construction/choice, non-manipulatiion. These need to be understood and respected. Towards an Integration of Causal Inference

Exciting new developments in many fields. “What is encouraging about all these developments is the possible emergence of a truly interdisciplinary theory of causation. … “Because of the sheer number of disciplines involved, and the many terminological differences, [and the different contexts] …the going is slow and the progress is incremental, measured in decades. But, progress it is and a shared theory of causal inference does now indeed seem feasible” Shadish (in press). Thank You

Palm Walk, Arizona State University