Genoeconomics: a Primer and Progress Report

Genoeconomics: A Primer and Progress Report

Daniel J. Benjamin CESR and Economics Department, USC

Berkeley Institute for Transparency in the Social Sciences • 9 June 2017 Main Collaborators Jonathan Beauchamp (University of Toronto) David Cesarini (New York University) Christopher Chabris (Geisinger Health Systems) Tõnu Esko (University of Tartu) Magnus Johanesson (Stockholm School of Economics) Philipp Koellinger (VU Amsterdam) David Laibson (Harvard University) James Lee (University of Minnesota) Aysu Okbay (VU Amsterdam) Patrick Turley (Harvard University) Peter Visscher (University of Queensland)

We gratefully acknowledge NIH’s NIA and OBSSR, NSF, the Ragnar Söderberg Foundation, and the Swedish Research Council for financial support! Some Promises for Social Science

1. Biological mechanisms for behavior § Complementary with neuroeconomics. 2. Non-genetic empirical work A. Empirical proxies for structural parameters (preferences, abilities, internal prod’n fns.) B. Instrumental variables C. Control variables 3. Studying genetic heterogeneity and targeting interventions § E.g., older individuals with at-risk cognitive health. Outline

1. Super Quick Genetics Primer 2. Traditional Candidate-Genes Studies 3. The Power Problem 4. Genome-Wide Association Studies 5. Example: Educational Attainment Genetics Primer

• Human DNA is a sequence of ~3 billion pairs of nucleotide molecules (spread across 23 chromosomes). • This human genome has 20,000-25,000 subsequences called genes. • Genes provide instructions for building proteins that in turn affect body function. • At the vast majority of locations, there is no variation in nucleotides across individuals. • Single-nucleotide polymorphisms (SNPs): Nucleotides where individuals differ (a small % of all nucleotides). (There are also other types of variation.) • At vast majority of SNP locations, there are only 2 possible nucleotides: – major allele (more common) – minor allele (less common). • From each parent, may inherit either allele; SNP unaffected by which received from whom. • Genotype for each SNP: #minor alleles (0,1,2). Note. Genotyping costs from multiple sources. Sequencing costs from NIH (genome.gov/sequencingcosts). Outline

1. Super Quick Genetics Primer 2. Traditional Candidate-Genes Studies 3. The Power Problem 4. Genome-Wide Association Studies 5. Example: Educational Attainment Traditional Candidate-Gene Study • Specify ex ante hypotheses about small set of � SNPs (often � = 1) based on believed biological function. • Estimate � = �� + �� + �.

• Set significance threshold a = .05 / K. • Virtually all existing work in social-science genetics. (Reviews: Ebstein, Israel, Chew, Zhong, and Knafo, 2010; Beauchamp et al., 2011; Benjamin et al., 2012) • Eminently reasonable, and has worked when hypotheses are direct. (e.g., APOE and Alzheimer’s disease) • But in social-science genetics, replication record has been inconsistent. – My own Icelandic saga. – Example: candidate genes for cognitive function.

Pooled estimates (11 SNPs + APOE)

• Eminently reasonable, and has worked when hypotheses are direct. (e.g., APOE/Alzheimer’s) • But in social-science genetics, replication record has been inconsistent. – My own Icelandic saga. – Example: candidate genes for cognitive function. • Problem with premise: assumed a few genes have large effects (detectable with N ≈ 100 to 3,000). – But we now know that effects of any particular genetic variant likely to be very small. – Having plausible hypotheses lent spurious credibility to statistically suspicious findings. The Three Big Problems 1. Multiple hypothesis testing – Typically many measured loci in the dataset. – Typically several phenotypes. – Many possible specifications/controls. 2. Population stratification – Genotype is correlated with ancestry, which is correlated with environmental factors. 3. Low power

– Is N large enough, given true effect size bj and significance threshold (and prior)? Outline

1. Super Quick Genetics Primer 2. Traditional Candidate-Genes Studies 3. The Power Problem 4. Genome-Wide Association Studies 5. Example: Educational Attainment The Power Problem

• Existing studies reporting significant gene-behavior associations usually had samples in the range 50 to 3000. (Ebstein et al., 2010; Beauchamp et al., 2011; Benjamin et al., 2012) • Implicit assumption: SNPs have explanatory power R2 ≥ 2%. – But would be found in N ≈ 60,000 GWASs of cognitive performance (Davies et al., 2015) and personality (De Moor et al., 2015). • Credible evidence from physical and medical literature: – Smoking and CHRNA3: R2 ≈ 0.5%. – BMI and FTO: R2 ≈ 0.3%. – Largest identified SNP for height: R2 ≈ 0.4%. • Credible evidence from GWAS for behavioral phenotypes: – Educational attainment: R2 ≈ 0.02%. – Depressive symptoms / neuroticism: R2 ≈ 0.04%. – Subjective well-being: R2 ≈ 0.02%. Calibration: Power Analysis

Suppose: • � = 0.05 • R2 ≈ 0.02% • N = 3,000

What is power? 12%.

Sample size for 80% power? 39,150. Bayesian Credibility (based on Wacholder et al., 2004; Ioannidis, 2005; Benjamin et al., 2012; Bayarri et al., 2016)

Given significant at α = .05, assuming effect size R2 = 0.02%.

Sample size N = 100 N = 10,000 N = 100,000 (power = .05) (power = .29) (power = .99) Prior 0.1% 0.1% 0.6% 2% prob- 1% 1% 6% 17% ability 5% 5.2% 24% 51% 10% 10.4% 39% 69% Outline

1. Super Quick Genetics Primer 2. Traditional Candidate-Genes Studies 3. The Power Problem 4. Genome-Wide Association Studies 5. Example: Educational Attainment Genome-Wide Association Study (GWAS) • Atheoretical testing of all SNPs measured using modern technologies (~0.5-2.5M). – For polygenic traits, biological knowledge unlikely to pinpoint a few genes of large effect. • Set significance threshold a = 5 ´ 10-8 (since ~1M independent SNPs in genome). – Makes clear need for much larger samples. • Therefore, new research organization and culture: “Consortia” meta-analyze results from GWASs in many samples. – Coordinated “analysis plan” easy to preregister. Example: Schizophrenia Stefansson et al. (2009) Example: Schizophrenia Ripke et al. (2011) Example: Schizophrenia Ripke et al. (2014) Addressing the Problems 1. Multiple hypothesis testing (MHT) – All measured SNPs are tested. – Genome-wide significance threshold addresses. – (But with more large-scale GWAS data available, MHT may occur with phenotypes.) 2. Population stratification – Loosely speaking, can use the genome-wide data to estimate ancestry, then control for it. (Price et al., 2006) 3. Low power – Obtain large samples via consortia. Now at Genome-Wide Significance… (based on Wacholder et al., 2004; Ioannidis, 2005; Benjamin et al., 2012; Bayarri et al., 2016)

Given significant at α = 5 ´ 10-8, assuming effect size R2 = 0.02%.

Sample size N = 100 N = 10,000 N = 100,000 (power = .00) (power = .00) (power = .16) Prior 0.1% 0.13% 36% 100% prob- 1% 1.3% 85% 100% ability 5% 7% 97% 100% 10% 13% 98% 100% These calculations make clear that in a world of tiny R2, we need large N.

They also provide a Bayesian perspective for why we need stringent significance thresholds—not only for GWAS, but also for candidate-gene studies.

27 Outline

1. Super Quick Genetics Primer 2. Traditional Candidate-Genes Studies 3. The Power Problem 4. Genome-Wide Association Studies 5. Example: Educational Attainment The SSGAC

0.009

0.008 Rho=1 0.007 Rho=0.8 Rho=0.6 0.006 Rho=0.4

2 0.005

R Rho=0.2

0.004

0.003

0.002

0.001

0 0 10,000 20,000 30,000 40,000 50,000 60,000 70,000 80,000 90,000 100,000 Sample Size The SSGAC

• In 2011, David Cesarini, Philipp Koellinger, and I founded the Social Science Genetic Association Consortium (SSGAC). • What outcomes to study? – In tradeoff between larger sample and higher-quality measure, given plausible effect sizes, larger sample gives more power. (Chabris et al., 2013) • Proof-of-concept: education attainment, available in many medical datasets. EA1 • Discovery phase: 41 datasets with total sample size of N ≈ 100,000. – Each cohort ran GWAS of EduYears (years of schooling) and/or College (a binary indicator) according to preregistered analysis plan. – Uploaded GWAS results to a secure, central server. – Meta-analyzed GWAS results across cohorts to obtain overall GWAS coefficients and SEs. – One genome-wide significant association with EduYears and two with College. • Replication phase: 12 independent datasets with total sample size of N ≈ 25,000. – All three hits replicated at Bonferonni-adjusted 5% significance level.

Effect size R2 ≈ 0.02% an order of magnitude smaller than for complex physical / medical traits. EA2

• 63 datasets with sample size of N = 293,723. • Similar preregistered analysis plan as EA1, except focused exclusively on EduYears (not College). • Found 74 approximately uncorrelated genome-wide significant SNPs. • After submission, first release of UK Biobank became available (N ≈ 110,000); used for replication. – Replication record somewhat better than expectation if all hits were true positive: • 72 out of the 74 lead SNPs have a consistent sign. • 52 are significant at the 5% level. • 7 reach genome-wide significance.

EA3 (in progress)

• Currently at N ≈ 770,000. • Essentially the same analysis plan as EA2.

Transformation of the Field • Many results in social-science genomics had inconsistent replicability. – Much debate, not enough cumulative knowledge. • Rising standards for empirical work (along with many other sciences). – Recognition of realistic effect sizes. – Adjustment for multiple hypothesis testing. – Ex ante power calculations → larger samples. – Assessment of (Bayesian) credibility of findings. – Often, pre-registration of analysis plans. • Large datasets becoming available → may increase MHT problems. Summer School in Social-Science Genomics Sponsored by the Russell Sage Foundation • When/Where: June 11-23, 2017 at the Pepper Tree Inn in Santa Barbara, CA • Eligibility: Ph.D. students, postdoctoral researchers, untenured faculty. • Applications due Feb 12, 2017. • For more information, visit http://www.rsfgenomicsschool.com/