Evaluation of the World Bank’s Assistance to Basic Education in Romania A Country Case Study Sue E. Berryman Amber Gove Dana Sapatoru Anca Tirca Director-General, Independent Evaluation: Vinod Thomas Director, Independent Evaluation Group, World Bank: Ajay Chhibber Manager: Alain Barbu Task Manager: H. Dean Nielsen 2007 The World Bank This paper is available upon request from IEG. Washington, D.C. ENHANCING DEVELOPMENT EFFECTIVENESS THROUGH EXCELLENCE AND INDEPENDENCE IN EVALUATION The Independent Evaluation Group is an independent unit within the World Bank Group; it reports directly to the Bank’s Board of Executive Directors. IEG assesses what works, and what does not; how a borrower plans to run and maintain a project; and the lasting contribution of the Bank to a country’s overall development. The goals of evaluation are to learn from experience, to provide an objective basis for assessing the results of the Bank’s work, and to provide accountability in the achievement of its objectives. It also improves Bank work by identifying and disseminating the lessons learned from experience and by framing recommendations drawn from evaluation findings. IEG Working Papers are an informal series to disseminate the findings of work in progress and to encourage the exchange of ideas about development effectiveness through evaluation. The findings, interpretations, and conclusions expressed here are those of the author(s) and do not necessarily reflect the views of the Board of Executive Directors of the World Bank or the governments they represent. The World Bank cannot guarantee the accuracy of the data included in this work. The boundaries, colors, denominations, and other information shown on any map in this work do not imply on the part of the World Bank any judgment of the legal status of any territory or the endorsement or acceptance of such boundaries. ISBN-13: 978-1-60244-079-1 ISBN-10: 1-60244-079-4 Contact: Knowledge Programs and Evaluation Capacity Development Group (IEGKE) e-mail: [email protected] Telephone: 202-458-4497 Facsimile: 202-522-3125 http:/www.worldbank.org/ieg ii Case Study: Romania ACRONYMS AND TRANSLATIONS Capacitate 8th grade selection examination CEDU2000+ Center Education 2000+ (Open Society Foundation) CEEC Central Eastern European Countries DFI Direct Foreign Investment DfiD (U.K.) Department of EBRD European Bank for Reconstruction and Development EBI European Bank of Investment EC European Commission ECA Europe and Central Asia EDP Editura Didactica si Pedagogica, State publishing house ERP Education Reform Project EU European Union GDP Gross National Product GoR Government of Romania ICR Implementation Completion Report IES Institute for Educational Sciences IEG Independent Evaluation Group (World Bank) Judet County-level government MER Ministry of Education and Research NAEB National Assessment and Examinations Board NAES National Assessment and Examinations Service OECD Organization for Economic Cooperation and Development PAL Programmatic Adjustment Loan PIRLS Progress in Reading Literacy Study PISA Program for International Student Assessment QAG Quality Assurance Group REP Rural Education Project SOE State-owned Enterprise SRP School Rehabilitation Project TAB Textbook Approvals Board TIMSS Trends in International Mathematics and Science Study iii Case Study: Romania TABLE OF CONTENTS Page Preface v Executive Summary vi 1. Introduction and Analytic Framework 1 2. Basic Education in the National Context 2 Transition from Communism to EU accession 2 National goals for access and quality of basic education before and after 1990 9 World Bank support for expanding and improving basic education 11 3. Summary of recent changes in basic education in Romania 12 Governance 12 Financing and financial management 15 Human resource management 18 Teaching and learning 21 4. Basic Education outputs and outcomes and their implications for household 29 demand Basic education outputs 29 Basic education learning outcomes 31 Implications of basic education outputs and outcomes for household demand for 37 education 5. World Bank contribution to sectoral changes (1990-2004) 38 Relevance and efficacy/impact of Bank assistance 38 Efficiency and sustainability of changes supported by the Bank 42 Counterfactual: Did the Bank make a difference? 43 6. Lessons Learned from Bank assistance to basic education 44 7. Conclusions 46 Development effectiveness of Bank support to Romania’s Basic Education 46 System Improving the effectiveness of future Bank support efforts 46 References 48 Annexes 52 A. Timeline of events and development assistance 53 B. Inventory of development assistance: World Bank and other agencies 59 C. Trends in public expenditure for education 61 D. Trends in major basic education development indicators 63 E. Data collection methodology 73 F. School visits and questionnaire and observation instrument for school visits 75 G. Other statistical tables 86 H. Bank documents reviewed 95 I. List of those interviewed 96 J. Interview guidance questionnaire for Romanian interviewees 97 K. Interview guidance questionnaire for Bucharest Resident Mission staff 100 iv Case Study: Romania TABLE OF TABLES Page Table 2.1. Selected Economic Indicators for Romania, 1992-2003 4 Table 3.1. Public Expenditures in Education as % of GDP by Country and Average 16 for OECD Table 3.2. Teacher salaries and vertical compression ratio as of February 1, 2005 21 Table 3.3. Ratio of Teachers’ Salaries to GDP Per Capita by Country and Level of 21 Education (2002) Table 3.4 Recent Changes in Basic Education in Romania 27 Table 4.1. Performance of ECA and EU Countries on PISA (2000) 34 Table 5.1. Relevance of Bank Assistance to Romania’s Basic Education System 38 Table 5.2. Summary of Evaluations of Bank’s Education Lending in Romania 41 TABLE OF FIGURES Page Figure 2.1. Employment by main sectors in the EU accession and candidate 7 countries, 2002 and EU average, 2001 Figure 2.2. Demographic trends: 1990-2035 8 Figure 3.1. Public education expenditure as percent of GDP and total public 16 expenditure Figure 3.2. Trends in local expenditures (1996-2003) 17 Figure 3.3. Players in Romania’s basic education reform (1990-2004) 26 Figure 4.1. Basic education gross enrollment and graduation rates (1990-2003) 30 Figure 4.2. Basic education repetition and dropout rates (1990-2003) 30 Figure 4.3. Percent passing 8th and 12th grade examinations by year 32 Figure 4.4. TIMSS performance for Romania and ECA average by year 33 Figure 4.5. ILO unemployment rate and education, by gender: Romania, third 35 quarter, 2004. Figure 4.6. Average years of schooling and observed changes in the returns to an 36 additional year of school: Romania, 1960 to 2000. v Case Study: Romania Preface From 1990, the year of the World Conference on Education for All (EFA), through mid-2005, the World Bank committed approximately $12.5 billion in support of the expansion and improvement of primary education in developing countries. By the early years of the current century support to primary education was nearly half of the Bank’s lending portfolio in education. Sector studies and Bank strategies have emphasized the critical role of primary education --especially the basic knowledge and skills it provides. Expansion and improvement of primary education are often at the center of a country’s poverty reduction efforts. In 2006, the Independent Evaluation Group issued From Schooling Access to Learning Outcomes: An Unfinished Agenda, which assessed the development effectiveness of World Bank assistance to improve countries’ knowledge and skills base through the provision of quality primary education to all children, especially since 1990. The evaluation drew on many sources of information, including desk reviews of the portfolio of primary education lending and analytic work, in-depth project evaluations and country case studies. The country case studies assessed the overall cumulative support of the Bank (lending and non-lending) to primary education in the context of historical and concurrent factors that impinged upon and shaped them. In particular, they addressed three questions: (a) What changes have taken place in primary education service delivery and outcomes since 1990? (b) To what extent have Bank efforts (though lending and non-lending channels) contributed to those changes? and (c) To what extent would the changes have taken place in the absence of Bank support? The four case study countries – Mali, Pakistan, Peru, and Romania -- were selected based on their performance (strong or weak) in improving learning outcomes and their per capita income, from among those countries that had received at least US$100 million in support from the World Bank for primary education. Each case study was undertaken by a team of 4 members, 2 educator-researchers from outside the country and 2 from within. The studies were reviewed both by the headquarters evaluation team and by World Bank project and sector managers for the country. The Romania case study was based on a two-week mission in March of 2005 as well as a review of dozens of World Bank and Romanian documents. The evaluation team was comprised of Sue Berryman, Amber Gove, Dana Sapatouro and Anca Tirca. The team gratefully acknowledges the generous time and thoughtful responses of interviewees, especially of parents, students, teachers and school principals. Government officials, donors, NGO representatives, and business leaders were also very giving of their time, providing insight into the dramatic changes in Romanian education