IMPROVING PRIMARY EDUCATION OUTCOMES FOR THE MOST VULNERABLE CHILDREN IN RURAL MONGOLIA PROJECT (June 2012 – June 2016) Byambatsogt Jugder, MA (Consultant) SAVE THE CHILDREN 2014 MID-TERM EVALUATION REPORT 2 SAVE THE CHILDREN JAPAN, MONGOLIA PROGRAM MID-TERM EVALUATION REPORT Table of Contents LIST OF ABBREVIATIONS ............................................................................................... 3 EXECUTIVE SUMMARY ................................................................................................. 4 I. MID-TERM EVALUATION OBJECTIVES AND METHODOLOGY ................................... 6 II. PROJECT DESCRIPTION ............................................................................................. 9 III. EVALUATION OF MID-TERM PROGRESS (as of November 2014): ......................... 11 Component 1: Strengthening school preparation programs .......................................... 13 Component 2: Introducing extracurricular after-school programs ................................ 17 Component 3: Pilot compensatory education programs ................................................ 19 Project Management, M&E and Knowledge Dissemination .......................................... 21 Stakeholders’ Assessment of the Project Progress ........................................................ 26 V. RECOMMENDATIONS .............................................................................................. 34 ANNEX 1: PROJECT BENEFICIARY DATA ....................................................................... 39 ANNEX 2: LIST OF PEOPLE PARTICIPATING ................................................................. 42 SAVE THE CHILDREN JAPAN, MONGOLIA PROGRAM 3 MID-TERM EVALUATION REPORT LIST OF ABBREVIATIONS CDC Child Development Center CEC Community Education Council ECE Early Childhood Education ECASP Extracurricular After-School Program FGD Focus Group Discussion JSDF Japan Social Development Fund LPC Local Project Committee M&E Monitoring and Evaluation MECS Ministry of Education, Culture and Science MTE Mid-Term Evaluation MoU Memorandum of Understanding NFDE Non-Formal and Distance Education PSC Project Steering Committee SCJ-M Save the Children Japan in Mongolia TAT Technical Advisory Team ToR Terms of Reference ToT Training of Trainer 4 SAVE THE CHILDREN JAPAN, MONGOLIA PROGRAM MID-TERM EVALUATION REPORT EXECUTIVE SUMMARY The Save the Children Japan, Mongolia achieving its target results1 in the 30 selected Program (SCJ-M) is implementing the project rural soums in 4 target aimags. A number “Improving Primary Education Outcomes of positive results emerged from the project for the Most Vulnerable Children in Rural interventions: Mongolia” under a US$2.46 million grant from • Psycho-emotional preparedness of 5 World Bank (JSDF) since June 2012. The main year old children from herder families objective of this project is aimed at addressing improved through the exposure to the education issues of rural herders’ children educational toys and books, and entering primary school without formal Early interaction with schools and teachers Childhood Education (ECE) or having learning before entering 1st grade. and psycho-emotional difficulties - issues • Cognitive and communication skills which can lead to them dropping out of of target children distinctly improved school in their primary grades. The project is as reported by the teachers, parents, expected to address these issues through (i) respective CEC members and other strengthening school preparation programs for stakeholders. new school entrants (5-6 years of age) from • School readiness of new 1st grade remote rural communities; (ii) introducing entrants, who benefitted from the extracurricular after-school programs in mobile toy and book libraries and home- schools for vulnerable children (6-10 years of based school preparation program in the age) living away from home; and (iii) piloting previous year, has improved and in some compensatory education programs for lower cases exceeded the readiness of other primary out-of-school children (8-10 years of new entrants. age) in rural communities. • Reading, comprehension and As the project reaches its mid-term, the presentation skills of dormitory students Mid-term evaluation aims to analyze the in conveying main ideas to peers through project performance to date, to inspect drawing, crafting and role playing positive and negative factors affecting the improved. project performance; to assess the project • Reading speed increased and vocabulary results in terms of positive effects to target and knowledge of Mongolian language children, parents and local community; to and literature were enriched. These identify deficiencies that could affect the improved skills allowed students to project implementation in negative ways better express themselves to others and provide recommendations for further and increased their self-confidence in actions. The mid-term evaluation conducted making friends with others. from October 17 to November 30, 2014 used • Parental and community involvement the following methodologies: (i) document and support in implementing the project review; (ii) face-to-face interviews; (iii) activities improved. focus group discussions; and (iv) survey • Project management, handled by well- questionnaires. established management structures at Since its launch on November 30, 2012 the project has made steady progress in 1 It is evident that Component 3 is unlikely achieve its target of at least 300 beneficiaries. SAVE THE CHILDREN JAPAN, MONGOLIA PROGRAM 5 MID-TERM EVALUATION REPORT central and local levels operated smoothly, without any major problems to date. There are variations in terms of qualitative achievements made by different soums depending for various reasons. However, due to lack of established Monitoring and Evaluation system in the project, it was difficult to evaluate the project performance in terms of quality - i.e. learning outcomes. Based on the MTE findings, it is recommended to intensify the activities of all project components by (i) further strengthening the achieved level of the project implementation in target soums; (ii) improving qualitative assessment of the project results i.e. improvement in learning outcomes of the target children; (iii) improving project management through capacity building of the project staff and establishing a robust M&E system; (iv) establishing proper incentive mechanism for the CEC members while improving local ownership and initiatives; and (v) providing of learning materials and stationeries to support extracurricular after-school activities for children in the dormitories. 6 SAVE THE CHILDREN JAPAN, MONGOLIA PROGRAM MID-TERM EVALUATION REPORT BACKGROUND In June, 2012, Save the Children I. MID-TERM EVALUATION Japan, Mongolia Program (SCJ-M) OBJECTIVES AND METHODOLOGY received a US$2.46 million grant from the Japanese Social Development Fund Since its official project launch on (JSDF) administered by the World Bank November 30, 2012 the project has made (WB) to implement the project “Improving steady progress in implementing its planned Primary Education Outcomes for the Most activities and achieving the target results in Vulnerable Children in Rural Mongolia” the 30 selected rural soums in Arkhangai, (June 2012 to June 2016). The main Dornod, Uvurkhangai and Sukhbaatar objective of this project is to improve the aimags. As it is the end of the project’s education outcomes of about 7,500 of the second year, SCJ-M has commissioned the most vulnerable children (aged 5 to 10) in Mid-Term Evaluation (MTE) to analyze the four educationally underperforming and project performance to date: to inspect underserved rural provinces of Mongolia. positive and negative factors affecting the The project interventions are aimed at project performance; to assess the project addressing the education issues of rural results in target children, parents and local herders’ children entering primary school community; to identify deficiencies that without formal Early Childhood Education could affect the project implementation in (ECE), or having learning and psycho- negative ways and provide recommendations emotional difficulties which cause them on further actions. Specifically, the MTE will to drop out from school in their primary focus on the following: grades. • Assessment of the results achieved The project is expected to address during the project implementation; these issues through (i) strengthening school preparation programs for new • Evaluation of whether the project school entrants (5-6 years of age) from outcomes are measured accurately remote rural communities; (ii) introducing by the project outcome indicators, extracurricular after-school programs in M&E indicators and provision for schools for vulnerable children (6-10 years advice on this; of age) living away from home; and (iii) • Identification and analysis of piloting compensatory education programs positive and negative factors for lower primary out-of-school children (8- affecting the project performance, 10 years of age) in rural communities. and determination of, and advise The Grant Agreement between SCJ-M on potential risks in the further and the WB was officially signed on June 21, implementation of the project; 2012 in the presence of representatives from • Evaluation of the new ways, new Ministry of Education, Culture and Science ideas and initiatives under the project (MECS). In addition, a Memorandum of were piloted and implemented, and Understanding (MoU) on implementation