Lessons learnt – data linkage study of children: a to inform policy and practice in health, family services and education

Presented by A/Professor Stefanie Schurer School of Economics | Charles Perkins Centre www.stefanie-schurer.com

The University of Sydney Page 1 Overview

Question How do we know whether a specific policy was effective in helping vulnerable (here: Indigenous) children to thrive?

RCTs? Yes-gold standard (JPAL MIT, UChicago Urban Lab)

What if RCTs are not available?

Solution 1. Policy Evaluation Methods: Quasi-experimental methods, natural experiments. 2. Data: Linked administrative data financed through an NHMRC Partnership Project, NHMRC Targeted Call, NHMRC CRE 3. Examples 3.1 Standard early-life health care (special care nursery admission) 3.2 Income management

The University of Sydney Page 2 Policies may have positive and negative spill-over effects

+

- Policy Lever

The University of Sydney Page 3 WHAT ARE NATURAL EXPERIMENTS?

The University of Sydney Page 4 Major external shocks Epidemics, famines, economic crises or individual-specific shocks to study causes of disease

Institutional decision rules Admission to SCN or An elite school using a score threshold

Gradual roll-out of a program Income management SNAP (US Food Stamps)

The University of Sydney Page 5 Provocative Question 1 Are neonatal intensive care units or special care nurseries cost- effective investments into Indigenous children’s health and wellbeing?

The University of Sydney Page 6 NICU/SCN Admission (AIHW, 2011)

The University of Sydney Page 7 SPN 3,000$-NICU 4,000$ up to 200,000$ until the baby leaves hospital

The University of Sydney Page 8 Provocative Question 2 Did income management under the NTER (2007-2008) lead to better health, developmental, and education outcomes for Indigenous children?

The University of Sydney Page 9 Income management locations today

Source: Department of Social Services, www.dss.gov.au. Accessed 18 October 2017. The University of Sydney Page 10 In the media

The University of Sydney Page 11 Northern Territory Data Linkage Project

NT DoH

AMSANT NT DoE Sven Silburn & others

NT NT DFCS DoAG-J

The University of Sydney Page 12 NT Data Linkage Project (Joint with Menzies)

The University of Sydney Page 13 Disclaimer

– Data: Northern Territory (NT) Early Childhood Data Linkage Project, ``Improving the developmental outcomes of NT Children: A data linkage study to inform policy and practice across the health, education and family services sectors'', funded through a Partnership Project (2014-2017) between the National Health and Medical Research Council (NHMRC) and the NT Government.

– We follow NHMRC Values and Ethics: Guidelines for Ethical Conduct in Aboriginal and Torres Strait Islander Health Research (2003) and the Australian Institute of Aboriginal and Torres Strait Islander Studies (AIATSIS) Guidelines for Ethical Research in Australian Indigenous Studies (2012)

Reciprocity, Respect, Equality, Responsibility, Survival and Protection, Spirit and Integrity

– Ethics agreement HREC Reference Number: 2016-2611 Project Title: Improving the developmental outcomes of Northern Territory children: A data linkage study to inform policy and practice in health, family services and education (Human Research Ethics Committee of the Northern Territory Department of Health and Menzies School of Health Research)

The University of Sydney Page 14 Project 1 Special care nursery care Guthridge, Schnepel and Schurer (2018). Early Life Health Interventions and Childhood Development: Evidence from Special Care Nursery Assignment in 's Northern Territory. In preparation for submission.

The University of Sydney Page 15 Clinical Decision Rules for SCN admission

The University of Sydney Page 16 Link between birth weight and SCN admission

The University of Sydney Page 17 No discontinuities across thresholds

The University of Sydney Page 18 Some results

The University of Sydney Page 19 Back of the envelope cost-calculation

Neonatology is expensive – In Australia, one day in a SCN costs the public health insurance – provider (Medicare) A$3,000 (US$2,157) – In our data, the marginal baby born to an indigenous mother – Stays on average 9 days in a SCN: $27,000 (US$19,416).

The University of Sydney Page 20 Project 2: Income Management

1. Cobb-Clark, Kettlewell, Schurer, and Silburn (2018). The Effect of Quarantining Welfare on School attendance in Indigenous communities. IZA Discussion Paper Nr 11449 (under peer review) 2. Doyle, Schurer, and Silburn (2017). Do welfare restrictions improve child health? Estimating the impact of income management in the Northern Territory. Life Course Centre Working Paper Nr 2017-21. In prep for submission. 3. Schurer, Silburn, Havnen, Doyle, Kettlewell, Cobb-Clark (2018). Descriptive analysis using Longitudinal Study of Indigenous Children (LSIC). In prep for a The Conversation article.

IN THE NEWS: ABC TV News (Sydney, Melbourne, , Brisbane), ABC News Online, ABC Radio (National, NewsRadio, triple j, Perth, Sydney), AAP News Corp Australia’s, Daily Mail Australia and Yahoo!7; The Wire; The Guardian, NITV; NT News, 8HA , 2MCE Orange and 4K1G , CAAMA Radio, 2MCE Orange, Koori Mail. Briefing PM&C 4 Dec 2017

The University of Sydney Page 21 Motivation of IM in 2007: part of the NTER

1) “To stem the flow of cash going towards substance abuse and gambling and ensure that funds meant to be for children's welfare are used for that purpose'' (Brough 2007).

1) “To promote socially responsible behaviour, particularly in relation to the care and education of children” (Social Security and Other Legislation Amendment (Welfare Payment Reform Act 2007 No. 130, 2007 123TB Objects, Section (a)).

3) Reduce humbugging: protect female household members (Howard 2007)

The University of Sydney Page 22 Roll Out of Income Management over Time

The University of Sydney Page 23 Roll Out of Income Management

The University of Sydney Page 24 Methods used to identify a causal impact

The University of Sydney Page 25 High frequency data: methods and assumptions

If data is high frequency – eg school attendance (daily measures for all children) - exploit variation in program timing (e.g. Hoynes & Schanzenbach, 2009).

A1: Income management is orthogonal to trends in school attendance A2: Timing of roll out unrelated to community characteristics

I. Event study II. Difference-in-Difference Compare individual school Use communities receive IM later as attendance trends before and after a control group for those receive IM the introduction of IM (distance earlier (school FE, time FE, grade from t=0) level, day of week, qrt., school- specific time FE)

Data: 1.5 years +/- around implementation date: 9,162 students, 130 schools 3.5 million student-day observations

The University of Sydney Page 26 Event study

The University of Sydney Page 27 Low frequency data: Methods and assumptions If your data is low frequency – birth outcomes in communities (each child has only one birth, low number of birth per community) same as Almond, Hoynes and Schanzenbach (2011)

Treatment group: Control group: Children who were in utero <= 28 Children who were not in utero weeks when IM was introduced when IM was introduced, or were (Alternative definitions are tested for) in utero in week 29+

Data: Perinatal data, 1,153 births between Sep 2007 – Jan 2009 Control variables: Community fixed effects, seasonal effects (in extensions: control for maternal behaviour)

A1: communities where IM was introduced earlier versus later do not differ in relevant characteristics A2: Similar birth outcome trends pre-treatment.

The University of Sydney Page 28

Some results and their robustness

100

0

-100

Treatmenteffect

-200 -300

Raw

Benchmark Month-year Matching: R Healthy BW Quarter-year Matching: NN Later sampleOutlier control Earlier Sample First-half rollout Matching: Kernel Steps btw rollout Matching: Stratified No seasonDummy controls wet season Second half rollout Excl. partial treatment Control access to care Drop small communities Control maternal health beh. Control maternal HB nonmissing

The University of Sydney Page 29 Discussion

– Routine, linked administrative data + detailed knowledge about policies allow to produce scientific evidence on policy effectiveness – Experiments are hiding everywhere, but one needs to know well the institutional framework! – Building linked admin data sets is hard – Need for long-term relationships between Government institutions (the data custodians) and researchers (the potential data users) – WIN-WIN?

The University of Sydney Page 30 The future research model

The University of Sydney Page 31 Final words on current research projects

NMHRC Centre of Research Excellence EMPOWER: Health Systems, Adversity and Child Well Being (2015-2020), an interdisciplinary collaboration of the BetterStart Research Group at the Adelaide School of Public Health and the Robinson Research Institute. CIA John Lynch, CIB Michael Sawyer, CIC Ben Mol, CID Claire Roberts, DIE Gustaaf Decker, CIF Nigel Stocks, CIG Stefanie Schurer, CIH Lyle Gurrin, [CII Naomi Dwyer, CIJ Kerrie Bowering]

NMHRC Targeted Call: Against the Odds (2018-2023), Against the odds: Understanding the factors influencing wellbeing among Indigenous youth in the Northern Territory. CIA Stefanie Schurer, CIB Lisa Cameron, CIC Pat Dudgeon, CID Steve Guthridge, CIE Guyonne Kalb, CIF Olga Havnen, CIG Tanja Hirvonen, CIH Peter Shaw

The University of Sydney Page 32