The Effects of School-Based Decision Making on Educational Outcomes in Low- and Middle-Income Contexts
Total Page:16
File Type:pdf, Size:1020Kb
International Development Coordinating Group The effects of school-based decision making on educational outcomes in low- and middle-income contexts Roy Carr-Hill, Caine Rolleston, Rebecca Schendel A Campbell Systematic Review Published: November 2016 2016:9 Search executed: July 2014 – January 2015 The Campbell Library comprises: • Systematic reviews (titles, protocols and reviews) • Policies and Guidelines • Methods Series Go to the library to download these resources, at: www.campbellcollaboration.org/library/ Better Evidence for a Better World Colophon Title The effects of school-based decision-making on educational outcomes in low- and middle-income contexts: a systematic review Authors Roy Carr-Hill1 Caine Rolleston1 Rebecca Schendel1 1UCL Institute of Education DOI 10.4073/csr.2016.9 No. of pages 169 Citation Carr-Hill R, Rolleston C, Schendel R. The effects of school-based decision making on educational outcomes in low- and middle-income contexts: a systematic review Campbell Systematic Reviews 2016:9 DOI: 10.4073/csr. 2016.9 ISSN 1891-1803 Copyright © Carr-Hill et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited. Roles and The review was designed and conducted by Roy Carr-Hill, Caine Rolleston responsibilities and Rebecca Schendel with support from Tejendra Pherali, Edwina Peart and Emma Jones. The members of the review team will update the review if and when new rigorous evidence (and suitable funding) becomes available. Editors for Editor: Hugh Waddington this review Managing editor: Emma Gallagher Sources of support UK Department for International Development Declarations of None of the team members have any financial interests in the review, nor interest have any team members been involved in any other systematic review focused on this topic or in the development of any of the interventions investigated. Corresponding Caine Rolleston author UCL Institute of Education, London E-mail: [email protected] Full list of author information is available on page 102 Campbell Systematic Reviews Editor-in-Chief Julia Littell, Bryn Mawr College, USA Editors Crime and Justice David B. Wilson, George Mason University, USA Charlotte Gill, George Mason University, USA Education Sandra Jo Wilson, Vanderbilt University, USA International Birte Snilstveit, 3ie, UK Development Hugh Waddington, 3ie, UK Social Welfare Brandy Maynard, St Louis University, USA Knowledge Translation Robyn Mildon, CEI, Australia and Implementation Cindy Cai, AIR, USA Methods Therese Pigott, Loyola University, USA Ryan Williams, AIR, USA Managing Editor Chui Hsia Yong, The Campbell Collaboration Co-Chairs Crime and Justice David B. Wilson, George Mason University, USA Peter Neyroud, Cambridge University, UK Education Sarah Miller, Queen's University, UK Gary W. Ritter, University of Arkansas, USA Social Welfare Mairead Furlong, National University of Ireland Brandy Maynard, St Louis University, USA Knowledge Translation Robyn Mildon, CEI, Australia and Implementation Cindy Cai, AIR, USA International Peter Tugwell, University of Ottawa, Canada Development Hugh Waddington, 3ie, UK Methods Ariel Aloe, University of Iowa, USA The Campbell Collaboration was founded on the principle that systematic reviews on the effects of interventions will inform and help improve policy and services. Campbell offers editorial and methodological support to review authors throughout the process of producing a systematic review. A number of Campbell's editors, librarians, methodologists and external peer reviewers contribute. The Campbell Collaboration P.O. Box 4404 Nydalen 0403 Oslo, Norway www.campbellcollaboration.org Table of contents PLAIN LANGUAGE SUMMARY 5 EXECUTIVE SUMMARY/ABSTRACT 7 Background 7 Objectives 7 Methods 8 Results 9 Conclusions and implications for policy, practice and research 10 BACKGROUND 11 Description of the problem 11 Description of the intervention 11 How the intervention might work 12 Why it is important to do the review 14 OBJECTIVES 16 METHODS 17 Criteria for inclusion and exclusion of studies in the review 17 Search strategy for identification of relevant studies 21 Keyword strategies for databases and websites 22 Screening of studies 23 Data extraction 26 Criteria for determination of independent findings 26 Statistica procedures and conventions 27 Treatment of qualitative studies 30 RESULTS 31 Flow of studies 31 Interventions 32 Descriptive statistics 39 Interpreting the meta-analysis findings 40 Overall intervention effects 41 Examination of heterogeneity: moderator analysis 49 Analysis of bias in the included studies 62 Examination of heterogeneity: study sub-groups 67 Barriers and enablers 76 Integration of findings 81 3 The Campbell Collaboration | www.campbellcollaboration.org IMPLICATIONS 84 Summary of main results 84 Quality of the Evidence 85 Limitations 85 Agreements and Disagreements with Other Reviews 86 Deviations from the published protocol 87 CONCLUSIONS 88 Implications for practice and policy 88 Implications for research 88 REFERENCES 90 References to included studies 90 References to studies excluded in the final stages 93 Existing reviews consulted during initial research 97 Supporting literature 98 INFORMATION ABOUT THIS REVIEW 102 Review Authors 102 Roles and Responsibilities 102 Sources of Support 103 Declarations of Interest 103 Plans for Updating the Review 103 APPENDICES 104 List of search locations 104 Detailed search strategy 105 Contacted authors 113 Code lists 114 SUPPLEMENTS 159 Supplement 1: effect size data computed 159 Supplement 2: Details of included impact studies 165 Supplement 3: Details of included non-causal studies 169 4 The Campbell Collaboration | www.campbellcollaboration.org Plain language summary SCHOOL-BASED DECISION-MAKING HAS POSITIVE EFFECTS ON EDUCATION OUTCOMES – BUT LESS SO IN LOW-INCOME COUNTRIES Decentralising decision-making to schools has small to moderate positive effects in reducing repetition and dropouts, and increasing test scores. These effects are mainly restricted to middle-income countries, with fewer and smaller positive effects found in low-income countries or disadvantaged communities. WHAT DID THE REVIEW STUDY? Many governments have addressed the low quality of education by devolving decision- making authority to schools. It is assumed that locating decision-making authority within schools will increase accountability, efficiency and responsiveness to local needs. However, there is limited evidence of the effectiveness of these reforms, especially from low- income countries. Existing reviews on school-based decision-making have tended to focus on proximal outcomes and offer very little information about why school-based decision-making has positive or negative effects in different circumstances. This review addresses two questions: 1. What is the impact of school-based decision-making on educational outcomes in low- and middle-income countries (L&MICs)? 2. What are the barriers to, and enablers of, effective models of school-based decision- making? What studies are included? Included studies for the analysis of impact evaluated the change in decision-making authority from a higher level of decision-making authority to the level of the school on educational outcomes. Outcomes were either proximal, for example attrition, equality of access, increased enrolment, or final, for example test scores, psychosocial and non-cognitive skills. Included studies had to have a comparison group and data which were collected since 1990. The analysis of impact included 26 studies, covering 17 interventions. The review identified nine studies to assess barriers and enablers of school-based decision-making. 5 The Campbell Collaboration | www.campbellcollaboration.org What is the aim of this review? This Campbell systematic review assesses the effectiveness of school-based decision-making. The review summarises findings from 17 impact studies and nine studies of barriers and enablers. WHAT ARE THE MAIN FINDINGS OF THIS REVIEW? School-based decision-making has small effects in reducing dropouts and repetition. There is a moderate positive effect on average test scores, though the effects are smaller for language and maths. The effects are not large, but comparable to those found in many other effective educational interventions. The positive impact is found in middle-income countries, with no significant effect in low- income countries. School-based decision-making reforms appear to have a stronger impact on wealthier students with more educated parents, and for children in younger grade levels. School-based decision-making reforms appear to be less effective in disadvantaged communities, particularly if parents and community members have low levels of education and low status relative to school personnel. WHAT DO THE FINDINGS OF THIS REVIEW MEAN? Implications for policy and practice 1. School-based decision-making reforms in highly disadvantaged communities are less likely to be successful. Parental participation seems to be the key to the success of such reforms. 2. The involvement of school management committees in personnel decisions appears to play a role in improving proximal outcomes, such as teacher attendance, but success is also likely to be linked to the overall teacher job market and the prospects of long-term employment. 3. The specifics of programme design appear to be crucial. Given the limited