INDEPENDENT EVALUATION OF THE MULTILINGUAL EDUCATION NATIONAL
ACTION PLAN IN CAMBODIA
ANNEXES – VOLUME II
July 2018 – February 2019
Cambodia
EVALUATION REPORT
MAY 2019
Photo: Indigenous children mapping their experiences in multilingual education Photo Credit ©UNICEF Cambodia/2018/Jessica Ball
EVALUATION REPORT
Independent Evaluation of the Multilingual Education
National Action Plan in Cambodia
July 2018 – February 2019
Cambodia
Authors: Jessica Ball and Mariam Smith, with support from Srom Bunthy, Srey Mao, Ka Nom, Thuk Bun, Cheun Kham Phin, Chab A and Pim Kheav
Submitted to UNICEF Cambodia Country Office on 18 February 2019.
Edited by Deirdre Smith in April-May 2019. MAY 2019
Independent Evaluation of the Multilingual Education National Action Plan in Cambodia
INDEPENDENT EVALUATION OF THE MULTILINGUAL EDUCATION NATIONAL ACTION PLAN IN CAMBODIA (Volume II)
© United Nations Children’s Fund, Phnom Penh, 2019 United Nations Children’s Fund P.O. Box 176 Phnom Penh, Cambodia 12201 [email protected]
May 2019
The United Nations Children’s Fund (UNICEF) in Cambodia produces and publishes evaluation reports to fulfil a corporate commitment to transparency. These reports are designed to stimulate the free exchange of ideas among those interested in the study topic and to assure those supporting UNICEF’s work that it rigorously examines its strategies, results and overall effectiveness. The Evaluation of the Multilingual Education National Action Plan in Cambodia was prepared by Jessica Ball and Mariam Smith. Deirdre Smith edited the report. The evaluation was commissioned by UNICEF Cambodia on behalf of the Ministry of Education, Youth and Sport. It was managed by a team led by Erica Mattellone (Evaluation Specialist, UNICEF) with support provided by Phaloeuk Kong and Saky Lim (Monitoring and Evaluation Officers, UNICEF), Nhonh Sophea (Education Specialist, UNICEF), Katheryn Bennett (Chief, Education, UNICEF), Tyler Henry (UNICEF Intern) and Elizabeth Fisher (Research and Evaluation Associate, UNICEF). The evaluation was supported by a reference group: H.E. Prak Kosal (Director, ECED, MoEYS), Thong Rithy (Director, SED, MoEYS), Pen Thavy (Deputy Director, SED, MoEYS), Um Sophany (Chief, ECED, MoEYS), Jan Noorlander (Acting Country Director, CARE), Pleuk Phearom (Indigenous community representative), Theang Savoeun (Education Coordinator, ICC), Nhonh Sophea (Education Specialist, UNICEF), Katheryn Bennett (Chief, Education, UNICEF), Ream Rin (Education Officer, UNICEF), Akihiro Fushimi (UNICEF EAPRO), Riccardo Polastro (UNICEF EAPRO), Hiroaki Yagami (UNICEF EAPRO) and Erin Tanner (UNICEF EAPRO). The purpose of this report is to facilitate the exchange of knowledge among UNICEF personnel and partners. The contents do not necessarily reflect the policies or views of UNICEF and UNICEF accepts no responsibility for error. The designations in this publication do not imply an opinion on the legal status of any country or territory, or of its authorities, or the delimitation of frontiers. UNICEF holds the copyright for this report. Written permission is required to reprint, reproduce, photocopy or in any other way cite or quote from this report. UNICEF has a formal permission policy that requires a written request be submitted. For non-commercial uses, permission will normally be granted free of charge. Please write to UNICEF Cambodia to initiate a permission request.
i Independent Evaluation of the Multilingual Education National Action Plan in Cambodia
Title: INDEPENDENT EVALUATION OF THE MULTILINGUAL EDUCATION NATIONAL ACTION PLAN IN CAMBODIA
Geographic region of the evaluation: North-eastern provinces of Cambodia: Mondulkiri, Ratanakiri, Stung Treng and Kratie
Timeline of the evaluation: July 2018–February 2019
Date of the report: 18 February 2019
Country: Cambodia
Evaluators: Jessica Ball Mariam Smith
Organization commissioning the evaluation: UNICEF Cambodia
ii Independent Evaluation of the Multilingual Education National Action Plan in Cambodia
Acronyms ASEAN Association of Southeast Asian Nations CDD Curriculum Development Department CFS Child-Friendly School CPS Community Preschool DP Development Partner DOE District Office of Education DTMT District Training and Monitoring Team ECED Early Childhood Education Department EMIS Education Management Information System EMT Evaluation Management Team ESP Education Strategic Plan ESWG Education Sector Working Group EMU Evangelical Mission to the Unreached FGD Focus Group Discussion HCEP Highland Children’s Education Evaluation ICC International Cooperation Cambodia IPA Indigenous Peoples’ Association IPO Indigenous Peoples’ Organization KII Key Informant Interview LTF Local Translator and Facilitator M&E Monitoring and Evaluation MEF Ministry of Economy and Finance MENAP Multilingual Education National Action Plan MENAP 2 Multilingual Education National Action Plan 2019–2023 (under development) MLE Multilingual Education MoEYS Ministry of Education, Youth and Sport MoI Ministry of Interior MSC Most Significant Change MT Mother Tongue MTB Mother Tongue-Based NER Net Enrolment Rate NFED Non-Formal Education Department NGO Non-Government Organization NIS National Institute of Statistics NSDP National Strategic Development Plan OECD/DAC Organisation for Economic Cooperation and Development’s Development Assistance Committee PED Primary Education Department POE Provincial Office of Education PRD Pedagogical Research Department PTTC Provincial Teacher Training College SED Special Education Department RBM Results-Based Management RG Reference Group RUPP Royal University of Phnom Penh SDGs Sustainable Development Goals SitAn Situational Analysis SSC School Support Committee ToA Theory of Action ToC Theory of Change ToR Terms of Reference
iii Independent Evaluation of the Multilingual Education National Action Plan in Cambodia
TTD Teacher Training Department UNDRIP United Nations Declaration on the Rights of Indigenous Peoples UNICEF United Nations Children’s Fund UNICEF EAPRO UNICEF East Asia and Pacific Regional Office
iv Independent Evaluation of the Multilingual Education National Action Plan in Cambodia
Contents
Annex 1. Terms of reference for evaluation of Cambodia Multilingual Education National Action Plan, UNICEF Cambodia ...... 1 Annex 2. Technical brief: Terms of reference for evaluation of Cambodia Multilingual Education National Action Plan, UNICEF Cambodia ...... 11 Annex 3. Evaluation team members ...... 14 Annex 4. Questions for key stakeholder groups ...... 16 Annex 5. Documents reviewed ...... 26 Annex 6. Participant lists ...... 33 Annex 7. Location, type and size of schools visited ...... 41 Annex 8. Extended presentation of costing for the MENAP ...... 44 Annex 9. Informed consent protocol ...... 51 Annex 10. Evaluation design matrix ...... 53 Annex 11. Extended description of evaluation methodology ...... 60 Annex 12. Extended description of qualitative data analysis ...... 72 Annex 13. Quality assurance...... 77 Annex 14. Effectiveness of the MENAP: Extended findings from quantitative analysis of education data ...... 79 Annex 15. Summary of findings, conclusions and recommendations ...... 90 Annex 16. International evidence on MLE ...... 95 Annex 17. Resources for teaching and learning in Indigenous languages created by International Collaboration Cambodia ...... 104 Annex 18. Actor-focused theory of change and theory of action ...... 105
Tables Table 1. Proposed evaluation timeline ...... 9 Table 2. Evaluation questions ...... 16 Table 3. Key informant interviews ...... 33 Table 4. List of participants in FGDs and group interviews ...... 35 Table 5. Summary of all schools visited for data collection ...... 41 Table 6. Location, Indigenous language and size of primary schools visited for data collection ...... 42 Table 7. Location, size and Indigenous language of preschools visited for data collection ...... 43 Table 8. CARE expenditure on MENAP implementation activities ...... 44 Table 9. UNICEF spending on MENAP activities 2016-2018 ...... 45 Table 10. UNICEF expenditure on teacher training ...... 46 Table 11. UNICEF expenditures on MENAP monitoring and evaluation ...... 47 Table 12. UNICEF expenditure on MLE capacity development ...... 48 Table 13. Breakdown of UNICEF spending on scholarships for ethnic minority students ...... 49 Table 14. Breakdown of UNICEF scholarships by province and community ...... 50 Table 15. Evaluation design matrix of the Cambodia Multilingual Education National Action Plan .. 53 Table 16. Qualitative data analytic categories and frequencies by province ...... 74 Table 17. Qualitative data analytic categories and frequencies by stakeholder ...... 75 Table 18. Overview of MENAP primary school statistics for 2017/2018 school year, Grades 1–3 .. 79 Table 19. MLE teachers Grades 1–3, gender equity (SED data) ...... 80 Table 20. MLE preschool teachers, gender equity (SED data) ...... 80 Table 21. MLE student enrolment, gender equity (CARE data) ...... 80 Table 22. MLE student enrolment – gender equity in primary school, preschool (SED data) ...... 81 Table 23. MLE preschool and primary school classes, schools, districts and students by province: SED Data ...... 81 Table 24. Breakdown of MLE primary schools by Indigenous language of instruction ...... 82
v Independent Evaluation of the Multilingual Education National Action Plan in Cambodia
Table 25. Demographics: MLE enrolment (2018) compared to total primary enrolment by province ...... 83 Table 26. Student/teacher ratios by province, data from CARE ...... 84 Table 27. Student/teacher ratios by province, data from SED ...... 84 Table 28. Targeted vs. actual enrolment by province in 2018 ...... 85 Table 29. MLE and non-MLE provincial enrolments for Grades 1–3 during the MENAP period ...... 86 Table 30. Indigenous resources for learning and teaching produced by ICC ...... 104 Table 31. Indigenous language resources produced in Ratanakiri ...... 105 Table 32. Indigenous children in MLE primary schools ...... 107 Table 33. Indigenous children in MLE preschools ...... 107 Table 34. Local Indigenous communities/guardians ...... 108 Table 35. Indigenous associations/organizations/networks ...... 108 Table 36. Khmer children in MLE schools ...... 109 Table 37. Khmer community members/guardians...... 109 Table 38. Communes ...... 109 Table 39. MLE primary school teachers ...... 110 Table 40. MLE preschool teachers ...... 110 Table 41. School directors ...... 111 Table 42. District Offices of Education ...... 111 Table 43. Provincial Offices of Education ...... 112 Table 44. Provincial Teacher Training Colleges ...... 113 Table 45. MoEYS outcomes (SED, Primary Education Department, Early Childhood Education Department, Curriculum Development Department, Teacher Training Department, Finance) ...... 113 Table 46. Cambodian universities/academic institutions ...... 114 Table 47. Development partners ...... 115
Figures Figure 1. Breakdown of CARE expenditure on MENAP implementation activities ...... 44 Figure 2. Overall breakdown of UNICEF MLE spending ...... 46 Figure 3. UNICEF expenditures on MLE teacher training ...... 47 Figure 4. UNICEF support of MENAP monitoring and evaluation ...... 48 Figure 5. UNICEF MLE capacity development spending ...... 49 Figure 6. Prototypical community field visit ...... 61 Figure 7. Evaluation methodology for field visits in each province ...... 62 Figure 8. Discovery workshop in Phnom Penh ...... 63 Figure 9. Outcome harvesting format ...... 65 Figure 10. Qualitative data analytic categories – overview ...... 73 Figure 11. Changes in the number of MLE primary schools from 2015 to 2018 ...... 82 Figure 12. Teacher staffing targets vs. actual teacher staffing at MLE primary schools ...... 84 Figure 13. Targeted vs. actual enrolment in MLE primary schools during the MENAP period ...... 85 Figure 14. Changes in MLE enrolment, Kratie, 2015–2018 ...... 87 Figure 15. Changes in MLE enrolment, Mondulkiri, 2015–2018 ...... 87 Figure 16. Changes in MLE enrolment, Ratanakiri, 2015–2018 ...... 88 Figure 17. Changes in MLE enrolment, Stung Treng, 2015–2018 ...... 88 Figure 18. Changes in MLE enrolment, provincial comparisons, 2015–2018 ...... 89 Figure 19. Multiple stakeholders with potentially different goals for MLE ...... 100 Figure 20. Possible education outcomes for cross-sectional and longitudinal assessment ...... 101 Figure 21. Inputs to the success of MLE ...... 103 Figure 22. Overview of the actor-focused theory of change ...... 106
vi Independent Evaluation of the Multilingual Education National Action Plan in Cambodia
Annex 1. Terms of reference for evaluation of Cambodia Multilingual Education National Action Plan, UNICEF Cambodia
Introduction In 2014, the Cambodian Ministry of Education, Youth and Sport (MoEYS) developed, in collaboration with UNICEF and in consultation with other stakeholders, the first Multilingual Education National Action Plan (MENAP) for 2014–2018. The MENAP was endorsed and launched in October 2015, with broad participation from provincial and district offices of education (POEs/DOEs) in five north-eastern provinces, and from the MoEYS Primary Education Department, Early Childhood Education Department, and other key technical department staff. The objectives of this plan were to: (i) ensure ethnic minority boys and girls have inclusive access to quality and relevant education; (ii) build the capacity of national and sub-national education officials to manage and monitor multilingual education (MLE) implementation; (iii) scale up MLE provision in relevant provinces; and (iv) promote demand for quality MLE among school support committees, parents and local authorities. These terms of reference (ToR) set out the purpose, objectives, methodology and operational modalities for an individual consultancy to evaluate the MENAP at the end of its implementation, and inform its new strategic direction. This independent evaluation is expected to begin in July 2018 and be completed by February 2019.
Background and description of object of the evaluation At the national level, Cambodia has been successful in getting children to enrol in and attend school. Further efforts are necessary to ensure that children in the most remote and disadvantaged areas of the country successfully complete basic education and achieve equal participation. In Cambodia, the Khmer ethnolinguistic community makes up 96 per cent of the country’s population of 13.4 million.1 There are 38,327 Indigenous people (around 3 per cent of the total population) from over 10 ethnic groups.2 The majority of the Indigenous population lives in five provinces of north-eastern Cambodia. These provinces struggle with low preschool enrolment, high early student dropout and high student repetition. The provision of MLE is a strategy to overcome the multiple obstacles faced by ethnic minority children in accessing quality education; it is a unique strategy in the East Asia region. UNICEF supported MoEYS to develop and implement the MENAP based on the following legislation, conventions, declarations and policies: (i) the UN Convention on the Rights of the Child (Articles 2, 20, 29 and 30); (ii) the UN Convention on the Rights of Indigenous Peoples; (iii) the Dakar Education Declaration (2000); (iv) the Cambodian Constitution (Article 66); (v) Cambodia’s Education Law (Article 24); (vi) the National Policy for Ethnic Minority Development (2008); (vii) the Education for All Education Action Plan; (viii) the National Policy on Early Childhood Care and Development (2010); (ix) the Education Strategic Plan 2014–2018; and (x) Prakas Number 48, 2013, Identification of Language for Learners of Khmer Nationality and Ethnic Minority Origin. The implementation of the current MENAP is documented in MoEYS education congress reports,3 UNICEF annual reports, and donor reports.4
1 Census 2008. 2 The largest indigenous groups are the Kreung, Tampoun, Broa, Joray, Phnong, Kouy and Stieng. 3 Ministry of Education, Youth and Sport, ‘The Education Congress: The Education Youth and Sport Performance in the Academic Year 2013–2014 and Goals for Academic Year 2014–2015’, March, 2015; Ministry of Education, Youth and Sport, ‘The Education Congress: The Education Youth and Sport Performance in the Academic Year 2014–2015 and Goals for Academic Year 2015–2016’, March, 2016; Ministry of Education, Youth and Sport, ‘The Education Congress: The Education Youth and Sport Performance in the Academic Year 2015–2016 and Goals for Academic Year 2016–2017’, March, 2017; Ministry of Education,
1 Independent Evaluation of the Multilingual Education National Action Plan in Cambodia
The MoEYS Primary Education Department and Early Childhood Education Department implemented the MENAP from 2014 to 2016. In 2017, the Special Education Department took on the role of implementing the MENAP, together with the five POEs of Ratanakiri, Kratie, Mondulkiri, Stung Treng and Preah Vihear provinces. UNICEF provided technical support to develop the MENAP, in collaboration with CARE International. The Early Childhood Education Department coordinated and provided teacher training on MLE to community preschool teachers in targeted community preschools in the five provinces. In addition to the training package for community preschool teachers, CARE International provided technical support and developed additional training materials to train teachers on how to introduce Khmer language in community preschools where children were from ethnic minority groups. In early childhood MLE, CARE International, with support from UNICEF, trained the Early Childhood Education Department trainers and POE trainers. The national trainers provided training in cascade form on MLE to community preschool teachers. In 2017, while the Special Education Department was gradually assuming its role of overseeing the delivery of MLE, POEs started to provide training to teachers on their own. This was the first time MLE teacher training was decentralized. The effectiveness of this training has not yet been assessed, and therefore will not form part of this evaluation. In primary MLE, UNICEF provided technical and financial support to MoEYS to implement the MENAP. The MENAP set intended results by year for both primary education and preschools to achieve by 2018.4
Purpose, objectives and scope of work The main purpose of this evaluation is to measure the extent to which the MENAP has been implemented, and how well it has been implemented, to strengthen the quality of MLE in the five provinces (summative). The evaluation will also help MoEYS and UNICEF identify lessons, good practices and innovations to explore ways to improve its support to more inclusive education in Cambodia (formative). As the MENAP is ending in December 2018, it was agreed with MoEYS that the evaluation would be carried out before the end of 2018 to inform the strategic direction of the next five-year plan, and the new Education Strategic Plan. In addition, the evaluation will be shared with UNICEF’s Regional Office for East Asia and the Pacific (EAPRO) and other offices to foster learning on MLE across the organization, as proposed in the Dissemination Plan. MoEYS, including the Special Education Department, Primary Education Department, Early Childhood Education Department, Curriculum Development Department, Teacher Training Department, Preschool Teacher Training Centre, POEs and DOEs, will be the primary users of this evaluation, as well as UNICEF Cambodia (primary duty bearers), notably the Education Section. Other stakeholders include commune councils, parents and their children (primary rights holders), NGOs (such as CARE International and others), EAPRO and UNICEF Headquarters (Programme Division) (secondary duty bearers). The evaluation has four primary objectives: 1. Assess the extent to which the MENAP has met its specific objectives (incl. the identification of enabling factors, barriers and bottlenecks).
Youth and Sport, ‘The Education Congress: The Education Youth and Sport Performance in the Academic Year 2017–2018 and Goals for Academic Year 2018–2019’, March, 2018. 4 UNICEF Cambodia, Terms of Reference: Individual Consultancy for Evaluation of Cambodia Multilingual Education National Action Plan, 2018. 4 Please refer to pages 12 and 13 of the MENAP for a full list of intended targets, at the following link: https://unicefmy.sharepoint.com/:b:/g/personal/emattellone_unicef_org/EZ33dvdKblVEvE2zEKbRb5kBZRO16K vmkxzTBI28y ENu2w?e=tWnMOH
2 Independent Evaluation of the Multilingual Education National Action Plan in Cambodia
2. Review and validate the strategies and activities implemented to strengthen MLE as part of MENAP 2014–2018, and re-construct the theory of change. 3. Assess support provided by national and sub-national authorities to the implementation of the MENAP, including the support provided by UNICEF and other implementing partners. 4. Document lessons learned, good practices and innovations that can inform the development of the new five-year MENAP. The evaluation will be retrospective and cover the period from 2014 to 2018 (up to the completion of the data collection), but it will be forward-looking in providing conclusions and recommendations. The timing of the evaluation is such that it will assess the quality and value of the MENAP, as well as its likely sustainability and impact. The evaluation will assess the MENAP in the context of the current Education Strategic Plan; the decentralization and deconcentration process; the National Strategic Development Plan; the National Social Protection Strategy for the Poor and Vulnerable; and other strategies. Geographically, the evaluation will cover four provinces included in the MENAP. Within each of the four provinces, the evaluation team will visit a sample of MLE schools and non-MLE schools for comparison purposes. Schools will be selected purposively using a maximum variation sample. This means that the evaluators will aim to select for maximum diversity across a number of sampling criteria, such as urban versus rural schools, accessibility versus remoteness, and school performance. The sampling criteria and specific schools in the five provinces will be confirmed in the inception report. The evaluation will be conducted in a participatory manner, involving duty bearers5 and rights holders6 in all steps. The primary interviewees will be staff from MoEYS, UNICEF, POEs and DOEs, school principals, school support committees, international and national NGOs, local authorities, parents and children.
Evaluation framework and questions Evaluation evidence will be assessed using the Organisation for Economic Co-operation and Development Assistance Committee’s criteria7 of relevance, effectiveness, efficiency, sustainability and impact. These criteria are prioritized because they capture the evaluation questions presented below. In addition, the evaluation will incorporate equity, gender equality and human rights considerations as cross-cutting issues. Key evaluation questions (and sub-questions) are clustered according to the evaluation criteria provided. This initial list of questions will be further refined and unfolded by the evaluator and included in the inception report following a desk review of key documents. Relevance of the extent to which the MENAP is suited to ensuring MLE to children (right holders) in five provinces, including: • To what extent has MENAP addressed the needs of ethnic minority girls and boys in terms of access to quality and relevant education? • To what extent are the objectives of the MENAP still valid? Were the MENAP objectives set realistically to be achieved in five years? • Are the activities and strategies of the MENAP consistent with its overall objectives and the attainment of the intended impacts and effects? • How relevant is UNICEF’s support for MENAP in building the capacity of national and sub- national education officials to manage and monitor MLE, and scale up MLE provision?
5 Primary duty bearers include MoEYS and UNICEF Cambodia. 6 Primary rights holders include commune councils, parents and their children. 7 http://www.oecd.org/dac/evaluation/daccriteriaforevaluatingdevelopmentassistance.htm.
3 Independent Evaluation of the Multilingual Education National Action Plan in Cambodia
• Has the MENAP been designed and implemented taking into consideration the Education Strategic Plan, the decentralization and deconcentration process, the National Strategic Development Plan, and other relevant strategies? Effectiveness of the support provided by MoEYS, UNICEF and other implementing partners in achieving its outcomes, including: • To what extent have the expected outcomes of MENAP been achieved or are likely to be achieved? • What are the results of MENAP in terms of: o Number of MLE preschools and primary schools o Number of students in MLE preschools and primary schools o Number of teachers in MLE preschools and primary schools
• What were the major factors influencing the achievement or non-achievement of the MENAP (incl. enabling factors, barriers and bottlenecks)? • Are results achieved similar in all five provinces? Which provinces perform better/worse and for what reason? • How satisfied were the parents and their children with MLE services? How do local authorities value the quality of the MENAP? Is this different among the five provinces? Efficiency of the management of the MENAP to ensure timely and efficient use of resources: • How well has the MENAP been managed in terms of the technical and financial resources provided to teachers and school operations? • Has the MENAP been implemented in the most cost-effective way compared to alternative approaches? • In what ways, and to what extent, do the costs incurred to implement the MENAP justify the results achieved on quality and access to education? • Does (will) MENAP implementation reach its target? Within the timeframe set in the plan? Impact resulting from the implementation of the MENAP (positive and negative changes, intended and unintended) in the five provinces: • Is there evidence of the extent to which the MENAP may have contributed to ensuring ethnic minority children have access to equitable, inclusive, quality and relevant education? • In what ways and to what extent has the MENAP changed the capacities of national and sub- national education officials to manage and monitor MLE implementation? • What difference has the MENAP made in terms of promoting demand for quality MLE among commune councils, school support committees, parents and children? • Are there any unintended results either positive or negative associated with the implementation of the MENAP? Sustainability of the benefits of the MENAP: • What are the key barriers and bottlenecks towards achieving sustainability of the MENAP? • To what extent can components of the new MENAP be integrated and implemented under the full ownership of the government, both technically and financially? • To what extent has UNICEF support contributed to enhancing sustainability of the MENAP? • Are there any lessons that can be learned to promote government ownership on a wider range of issues (including consideration of political will, social norms and perceptions)? Cross-cutting considerations: • To what extent are age- and gender-disaggregated data collected and monitored? • In what ways and to what extent has the MENAP integrated an equity-based approach into the design and implementation of its services? • Does the MENAP actively contribute to the promotion of children’s and women’s rights, especially the most vulnerable?
4 Independent Evaluation of the Multilingual Education National Action Plan in Cambodia
• To what extent and how does the MENAP ensure an equity focus?
Evaluation approach and methodology Based on the objectives of the evaluation, this section indicates a possible approach, methods and processes for the evaluation.8 Methodological rigour will be given significant consideration in the assessment of the proposals. Hence, consultants are invited to interrogate the approach and methodology proffered in the ToR and improve on it, or propose an approach they deem more appropriate. This should be guided by the UNICEF revised Evaluation Policy (2018),9 the United Nations Evaluation Group Norms and Standards for Evaluation (2016),10 the UN SWAP Evaluation Performance Indicator, the UNICEF Procedure for Ethical Standards and Research, Evaluation and Data Collection and Analysis (2015)11 and the UNICEF-Adapted UNEG Evaluation Report Standards (2017).12 The evaluation should consider issues of equity, gender equality and human rights. In their proposal, consultants should clearly refer to triangulation, sampling plan, methodological limitations and mitigation measures. They are encouraged to demonstrate methodological expertise in evaluating initiatives related to inclusive education, particularly MLE. It is expected that the evaluation will employ both a theory-based (re-constructing the theory of change) and a mixed-methods approach, drawing on key background documents and the monitoring framework (developed by CARE International for primary education). All key documents, together with a contact list of all MENAP relevant informants will be provided to the evaluator once a contractual agreement has been made. At a minimum, the evaluation will draw on the following methods:
• Desk review of background documents and other relevant data, including strategy documents, prior monitoring reports, evaluation reports and other documents judged relevant • Literature search and review, and analysis of secondary quantitative data (Education Management Information System), review of material on the environment in which MENAP operates, and recent development plans and strategies • Key informant interviews with POE directors, POE staff responsible for MLE in primary, POE staff responsible for MLE community preschools, and CARE International staff in Ratanakiri, Kratie and Mondulkiri provinces • Focus group discussions (FGDs) with MLE teachers of primary schools, MLE teachers of community preschools, school support committees, and parents and their children • Case studies of children who attend/ed MLE community preschools and children who transferred from MLE community preschools to MLE primary schools • Cost-effectiveness analysis of the support provided to MLE • Structured surveys of MLE beneficiaries to gather quantitative data on the quality of education provided. The data collected should be disaggregated by age, gender, province, etc., where relevant, and should focus on MENAP strategies and activities both at the national and sub-national level. Sampling of key informant interviews and FGDs should be done in consultation with UNICEF.
8 The proposed methodology is just indicative, and is based on internal experience of conducting similar evaluations. 9 UNICEF’s revised Evaluation Policy: https://www.unicef.org/about/execboard/files/2018-14-Revised_Eval- ODS-EN.pdf 10 United Nations Evaluation Group Norms and Standards for Evaluation (2016): http://www.unevalutoin.org/document/detail/1914, UNEG Standards: http://www.uneval.org/document/detail/22. 11 https://www.unicef.org/supply/files/ATTACHMENT_IV-UNICEF_Procedure_for_Ethical_Standards.PDF. 12https://www.unicef.org/evaluation/files/UNICEF_adapated_reporting_standards_updated_June_2017_FINAL. pdf.
5 Independent Evaluation of the Multilingual Education National Action Plan in Cambodia
There are several limitations to the evaluation which can hinder the process, notably: (i) disaggregated data may not be available at the local level, or the quality of available data may not be satisfactory; (ii) interviewing government counterparts may depend on their availability; and (iii) the rainy season may hamper the data collection process. The applicants should discuss the above or other potential limitations in their proposal. The evaluation should include the following steps: Step 1: Desk review of relevant background documents and literature search. The evaluator will review key background documents to understand MENAP strategies and activities since its inception in 2014 to date, and search literature for secondary data to understand the context in which the MENAP operates. Step 2: Preparation of inception report that includes evaluation methodology and tools. The methodology should be prepared to cover all the intended objectives of the evaluation. The evaluation methodology design will be finalized in agreement with the reference group (see below). The inception report should be prepared based on the Evaluation Norms and Standards of UNEG and submitted to the evaluation manager for approval. Step 3: Data collection. The application of mixed methods (qualitative and quantitative) is expected, which should be human-rights based, including child-rights based, and equity and gender sensitive, as noted above. Step 4: Data analysis. Collected data should be analysed by using relevant analysis methods that should be clearly described in the report. Step 5: Sharing preliminary findings. The evaluator will share preliminary findings with the reference group. While feedback will be taken into consideration and incorporated into the draft report, the consultant is encouraged to guard against validity threats, such as personal bias. Step 6: Draft report. The consultant prepares a draft report, with conclusions, lessons learned and recommendations drawn from the data. The report structure should follow UNICEF’s evaluation report guidance. Step 7: Finalization of the evaluation report. The consultant will present the final draft evaluation conclusions and recommendations for the reference group and other key stakeholders at a multi- stakeholder workshop, using a PowerPoint presentation and other methodologies for presenting in a participatory manner. Recommendations of the evaluation should be presented and prioritized. Comments and feedback on the findings and recommendations should be incorporated to finalize the report. Good practices not covered therein are also to be followed. Any sensitive issues or concerns should be raised with the evaluation manager as soon as they are identified.
Management and coordination The evaluation will be conducted by an external evaluation consultant who will be mainly responsible for the overall evaluation, including designing the evaluation methodology, developing tools, guiding national researchers/enumerators in data collection (as necessary), analysing data, and drafting the inception report and the final reports with recommendations. The evaluator will operate under the supervision of UNICEF’s evaluation specialist, who will act as evaluation manager and therefore be responsible for the day-to-day oversight and management of the evaluation and for the management of the evaluation budget. The evaluation manager will assure the quality and independence of the evaluation and guarantee its alignment with UNEG Norms and Standards, Ethical Guidelines and other relevant procedures. The manager will provide quality assurance, checking that the evaluation findings and conclusions are relevant, and recommendations are implementable. The manager will contribute to the dissemination of the evaluation findings and follow up on the management response. The evaluation manager will work in collaboration with UNICEF’s Education Section, who will facilitate
6 Independent Evaluation of the Multilingual Education National Action Plan in Cambodia consultation and coordination with MoEYS. The regional education adviser and the regional evaluation adviser will provide additional quality assurance. MoEYS and the UNICEF Cambodia Country Representative will approve the final report. A reference group will be established, bringing together: the director or deputy director of the Special Education Department and officers of this department responsible for MLE; Early Childhood Education Department staff responsible for community preschools; Primary Education Department staff formerly responsible for MLE in primary schools; CARE International and UNICEF Early Childhood Education officers and Education officers. The reference group will have the following role: (i) contribute to the preparation and design of the evaluation, including providing feedback and comments on the inception report and on the technical quality of the work of the evaluator; (ii) provide comments and substantive feedback to ensure the quality, from a technical point of view, of the draft and final evaluation reports; (iii) assist in identifying internal and external stakeholders to be consulted during the evaluation process; (iv) participate in review meetings organized by the evaluation manager and with the evaluator as required; and (v) play a key role in learning and knowledge sharing from the evaluation conclusions and recommendations, contributing to disseminating the evaluation results, and following up on the implementation of the management response.
Evaluation deliverables and timeline Evaluation products expected for this exercise are: 1. An inception report in English of maximum 20 pages, excluding annexes and a summary note in preparation for data collection (in both English and Khmer) 2. A report of the preliminary evaluation findings from primary data collection (in English), including a literature review analysis and a PowerPoint presentation to facilitate a stakeholder consultation exercise 3. A draft and final report (in English), including a complete first draft to be reviewed by the reference group and UNICEF EAPRO, and a penultimate draft of maximum of 40 pages, excluding annexes. The executive summary of the report should be translated into Khmer 4. A PowerPoint presentation (in both English and Khmer) to be used to share findings with the reference group and with government and non-government organization (NGO) stakeholders for use in subsequent dissemination events 5. A four-page executive summary (in both English and Khmer) that is distinct from the executive summary of the evaluation report, which is intended for a broader, non-technical and non- UNICEF audience. Other interim products are:
• Minutes of key meetings with the evaluation manager and reference group • Presentation materials for the meetings with the evaluation manager and the reference group. These may include PowerPoint summaries of work progress and conclusions to that point. • Video and photo materials to be collected during the evaluation to enrich presentations and the report • Bi-weekly reports to the evaluation manager to track progress of the implementation of the evaluation. Outlines and descriptions of each evaluation product are meant to be indicative, and include:
• Inception report: The inception report will be key to confirming a common understanding of what is to be evaluated, including additional insights into executing the evaluation. At this stage, the evaluator will refine and confirm evaluation questions, confirm the scope of the evaluation, further improve on the methodology proposed in the ToR and their own evaluation proposal to improve its rigour, and develop and validate evaluation instruments. The report will include, among other elements: i) evaluation purpose and scope, confirmation of objectives and the main themes of the evaluation; ii) evaluation criteria and questions; iii)
7 Independent Evaluation of the Multilingual Education National Action Plan in Cambodia
evaluation methodology (i.e., sampling criteria), a description of data collection methods (quantitative and qualitative) and data sources (including a rationale for their selection), draft data collection instruments, for example questionnaires, with a data collection toolkit as an annex, an evaluation matrix that identifies descriptive and normative questions and criteria for evaluating evidence, and a discussion on the limitations of the methodology and mitigation measures; iv) ethical protocols; v) quality control procedures; vi) training plan for national researchers/enumerators (if any); vii) field work plan including team composition, logistics, field monitoring, etc.; viii) plans for data analysis (quantitative and qualitative), including a discussion on how to enhance the reliability and validity of evaluation conclusions; ix) proposed structure of the final report; x) evaluation work plan and timeline, including a revised work and travel plan; xi) resources requirements (i.e., detailed budget allocations tied to evaluation activities, work plan deliverables); and xii) annexes (i.e., organizing matrix for evaluation questions, data collection toolkit, data analysis framework, an evaluation summary note for external communication purposes). The inception report will be 20 pages in length (excluding annexes), or approximately 8,000 words, and will be presented at a formal meeting of the reference group. • A report of initial evaluation findings: This report will present the initial evaluation findings from primary data collection, comprising the desk-based document review and analysis of the 2014–2018 MENAP and literature search. The report developed prior to the first drafts of the final report should be 10 pages, or about 4,000 words in length (excluding annexes, if any) and should be accompanied by a PowerPoint presentation that can be used for validation with key stakeholders. • A draft and final evaluation report: The report will not exceed 40 pages, or 16,000 words, including the executive summary and excluding annexes.13 • PowerPoint presentation: Initially prepared and used by the evaluator in presentations to the reference group, a stand-alone PowerPoint will be submitted to the evaluation manager as part of the evaluation deliverables. • A four-page executive summary for external users will be submitted to the evaluation manager as part of the evaluation deliverables. Infographics should be developed as part of the evaluation summary. • Reports will be prepared according to the UNICEF Style Guide and UNICEF Brand Toolkit (to be shared with the winning applicant) and UNICEF-Adapted UNEG Evaluation Report Standards (2017) as per GEROS guidelines (see footnote 13). All deliverables must be in professional level Standard English and must be proofread by a native English speaker. • The first draft of the final report will be received by the evaluation manager who will work with the evaluator on necessary revisions before sending the report to the reference group for comments. The evaluation manager will consolidate all comments on a response matrix, and request that the evaluation team indicate actions taken against each comment in the production of the penultimate and final drafts. Applicants are invited to reflect on each outline and affect the necessary modifications to enhance their coverage and clarity. Products are expected to conform to the stipulated number of pages where that applies. The results of the evaluation will be disseminated per the dissemination plan, and be made available to the wider public on UNICEF Cambodia’s website and unicef.org.
13 UNICEF has instituted the Global Evaluation Report Oversight System (GEROS), a system where final evaluation reports are quality assessed by an external company against UNICEF/UNEG Norms and Standards for evaluation reports. The evaluation team is expected to reflect on and conform to these standards as they write their report. The team may choose to share a self-assessment based on the GEROS with the evaluation manager.
8 Independent Evaluation of the Multilingual Education National Action Plan in Cambodia
An estimated budget has been allocated for this evaluation. As reflected in Table 1, the evaluation has a tentative timeline from July 2018 to February 2019. Adequate effort should be allocated to the evaluation to ensure timely submission of all deliverables, approximately 14 weeks on the part of the evaluator (four months). Table 1. Proposed evaluation timeline14
Activity Deliverable Time estimate Responsible party
Steps 1 & 2: Desk review and 3 weeks, concurrent inception report (Jul, 2018) 1. Inception meeting by Skype with Meeting minutes Week 1 Evaluator, evaluation the evaluation specialist and manager Education section 2. All relevant documents are Draft inception Week 1–3 Evaluator reviewed and inception report, report that complies with UNICEF requirements, is submitted 3. Present draft inception report to PowerPoint Week 3 Evaluator, evaluation the reference group presentation manager, reference group 4. Receive inception report and Evaluation Week 3 Evaluation manager, feedback to evaluator commenting matrix reference group 5. Submit inception report, confirm Final inception Week 3 Evaluator, evaluation planning for field visit report manager, reference group Step 3: Data collection 5 weeks, consecutive (Jul to Aug, 2018) 1. Pilot data collection tools and - Weeks 4–8 Evaluator conduct field-based data collection based on the methodology described in the inception report Steps 4 & 5: Data analysis and 3 weeks, consecutive preliminary findings (Sep, 2018)
1. Relevant analysis methods Initial evaluation Week 9–11 Evaluator, evaluation applied to analyse primary and findings report manager, reference secondary data and prepare (incl. desk review group initial evaluation findings report and literature and presentation search), PowerPoint presentation, meeting minutes Step 6 & 7: Draft and final report 5 weeks, consecutive (Sep, 2018 – Up to Feb, 2019) 1. Prepare and submit first draft of Draft report Week 11–12 Evaluator evaluation report 2. Receive first draft and feedback Evaluation Week 13–14 Evaluation manager, to evaluator commenting matrix reference group
14 Please note that the timing of the data collection may change depending on the possibility of carrying out key informant interviews and FGDs, and other contextual factors.
9 Independent Evaluation of the Multilingual Education National Action Plan in Cambodia
Activity Deliverable Time estimate Responsible party
3. Prepare and submit penultimate Draft report Week 15 Evaluator draft of evaluation report
4. Submit and present final report Final report, Week 16 Evaluator, evaluation to reference group and other executive manager, reference stakeholders in a multi summary, group stakeholder workshop, and PowerPoint prepare presentation and other presentation, materials meeting minutes
Evaluator profile The evaluation will be conducted by an individual consultant who should bring the following competencies:
• An advanced university degree (Masters or higher) in monitoring and evaluation, education, international development, public policy, development economics or similar, including knowledge of inclusive education • Extensive evaluation experience (at least 10 years) with an excellent understanding of evaluation principles and methodologies, including capacity in an array of qualitative and quantitative evaluation methods, and UNEG Norms and Standards • Extensive experience in planning, implementing, managing or monitoring and evaluation, preferably in the education sector • A strong commitment to delivering timely and high quality results, i.e. credible evaluations that are used to improve strategic decisions • In-depth knowledge of UN human rights, gender equality and equity agendas • Good team leadership and management track record, as well as excellent interpersonal and communication skills to help ensure that the evaluation is understood and used • Specific evaluation experience of MLE is strongly desired, but is secondary to a strong mixed- method evaluation background • Previous experience of working in an East Asian context is desirable, together with an understanding of the Cambodian context and cultural dynamics • The consultant must be committed and willing to work independently, with limited regular supervision; s/he must demonstrate adaptability and flexibility, client orientation, proven ethical practice, initiative, concern for accuracy and quality • S/he must have the ability to concisely and clearly express ideas and concepts in written and oral form, as well as the ability to communicate with various stakeholders in English. The consultant must strictly adhere to the UNEG ethical guidelines and code of conduct. The evaluation does not need to go through an ethical review board, however the consultant should clearly identify any potential ethical issues and approaches, as well as processes for ethical review and oversight of the evaluation process in her/his proposal.
10 Independent Evaluation of the Multilingual Education National Action Plan in Cambodia
Annex 2. Technical brief: Terms of reference for evaluation of Cambodia Multilingual Education National Action Plan, UNICEF Cambodia
Overall goal The evaluation assessed the extent to which the MENAP had been implemented and the ways in which implementation had strengthened the quality of MLE in the four north-eastern provinces where it was introduced. The evaluation assessed the quality and value of the MENAP and its likely contribution and sustainability. The evaluation was retrospective, covering the period from 2014 to 2018, and also forward-looking in providing recommendations for a subsequent iteration of the MENAP.
Specific objectives 1. Assess the extent to which the MENAP has met its specific objectives (including the identification of enabling factors, barriers and bottlenecks) 2. Review and validate the strategies and activities implemented to strengthen MLE as part of the MENAP 2014–2018, and re-construct the theory of change 3. Assess support provided by national and sub-national authorities to the implementation of the MENAP, including the support provided by UNICEF and other implementing partners 4. Document lessons learned, good practices and innovations that can inform the development of the new five-year MENAP.
Evaluation questions Relevance - the extent to which the MENAP is suited to ensuring MLE to children (rights holders), including:
• To what extent has the MENAP addressed the needs of ethnic minority girls and boys in terms of access to quality and relevant education? • To what extent are the objectives of the MENAP still valid? Were the MENAP objectives set realistically to be achieved in five years? • Are the activities and strategies of the MENAP consistent with its overall objectives and the attainment of the intended impacts and effects? • How relevant is UNICEF’s support for the MENAP in building the capacity of national and sub- national education officials to manage and monitor MLE, and scale up MLE provision? • Has the MENAP been designed and implemented taking into consideration the Education Strategic Plan, the decentralization and deconcentration process, the National Strategic Development Plan, and other relevant strategies?
Effectiveness of the support provided by MoEYS, UNICEF and other implementing partners in achieving its outcomes, including:
• To what extent have the expected outcomes of the MENAP been achieved or are likely to be achieved? • What are the results of the MENAP in terms of: o Number of MLE preschools and primary schools o Number of students in MLE preschools and primary schools o Number of teachers in MLE preschools and primary schools • What were the major factors influencing the achievement or non-achievement of the MENAP (including enabling factors, barriers and bottlenecks)? • Are results achieved similar in all four provinces? Which provinces perform better/worse and for what reason? • How satisfied were the parents and their children with MLE services? How do local authorities value the quality of the MENAP? Is there a difference between the four provinces?
11 Independent Evaluation of the Multilingual Education National Action Plan in Cambodia
Efficiency of the management of the MENAP to ensure timely and efficient use of resources:
• How well has the MENAP been managed in terms of the technical and financial resources provided to teachers and school operations? • Has the MENAP been implemented in the most cost-effective way compared to alternative approaches? • In what ways, and to what extent, do the costs incurred to implement the MENAP justify the results achieved on quality and access to education? • Does (will) the MENAP implementation reach its target? Within the timeframe set in the plan?
Impact resulting from the implementation of the MENAP (positive and negative changes, intended and unintended) in the four provinces:
• Is there evidence of the extent to which the MENAP may have contributed to ensuring ethnic minority children have access to equitable, inclusive, quality and relevant education? • In what ways and to what extent has the MENAP changed the capacities of national and sub- national education officials to manage and monitor MLE implementation? • What difference has the MENAP made in terms of promoting demand for quality MLE among commune councils, school support committees, parents and children? • Are there any unintended results, either positive or negative, associated with the implementation of the MENAP?
Sustainability of the benefits of the MENAP:
• What are the key barriers and bottlenecks towards achieving sustainability of the MENAP? • To what extent can components of the new MENAP be integrated and implemented under the full ownership of the government, both technically and financially? • To what extent has UNICEF’s support contributed to enhancing the sustainability of the MENAP? • Are there any lessons that can be learned to promote government ownership on a wider range of issues (including consideration of political will, social norms and perceptions)?
Cross-cutting considerations:
• To what extent are age- and gender-disaggregated data collected and monitored? • In what ways and to what extent has the MENAP integrated an equity-based approach into the design and implementation of its services? • Does the MENAP actively contribute to the promotion of children’s and women’s rights, especially the most vulnerable? • To what extent and how does the MENAP ensure an equity focus?
Methodology The evaluation used a participatory, mixed-method approach, including qualitative and quantitative approaches, such as: document review, key informant interviews, FGDs, multi-stakeholder workshops, and secondary analysis of available administrative and costing data. Limited school/classroom observation was included, as schools were closed during the period of the evaluation. Direct assessment of MLE teacher competencies and student learning processes or achievements were outside the scope of this evaluation (concurrent studies under other auspices are examining these impacts). A review of previous study findings that address these outcomes was included. A maximum variation strategy was used to establish a sample of MLE preschool and school communities that are representative of MLE sites across the four provinces. A sample of non-MLE schools was also assessed for comparison. The evaluation approach placed high value on Indigenous participation throughout, including on the reference group (two members), in the composition of the national team assisting the international
12 Independent Evaluation of the Multilingual Education National Action Plan in Cambodia evaluators to collect and organize data (eight members), and in the intensity of effort to elicit the self- reported experiences of Indigenous children, parents, teachers and village leaders with strategies of the MENAP. Key informants included: ministry officials and development agency/NGO staff concerned with the MENAP; staff of provincial and district education offices and district training and monitoring teams; school directors, school support committee and commune council members, and village leaders. Information from documents and participants was cross checked and triangulated to enhance the reliability of the information collected and the credibility of the evaluation’s conclusions and recommendations. Key strategies for assessing contributions of the MENAP and triangulating perspectives were outcome harvesting and most significant change methods. The evaluation had an inception phase, including: meeting with a multi-agency reference group; submission, revision and approval of an inception report; piloting methods and tools, revision and translation; six weeks of fieldwork in four provinces and in Phnom Penh; preliminary data analysis and presentation of initial observations in a feedback session with the reference group and in a multi- agency outcome harvesting workshop; and completion of a final report based on analysis and interpretation of all data.
13 Independent Evaluation of the Multilingual Education National Action Plan in Cambodia
Annex 3. Evaluation team members Dr. Jessica Ball was the primary consultant and co-led the evaluation. Dr. Ball holds a doctoral degree in psychology, a master of public health degree in international health planning and behaviour change, a master of arts in development-clinical psychology, and a bachelor degree in psychology. She has social science research expertise in qualitative, quantitative and mixed methods. She has conducted research and led programme innovations in Southeast Asia over the past 30 years and is familiar with Indigenous language and education issues, and on-going innovations and reforms in the region. She has conducted numerous evaluations of innovative education, social service and training initiatives in the region and elsewhere. She has worked on three continents at all levels of education, including early childhood education, primary and secondary school, and tertiary education. She maintains a dynamic, multidisciplinary programme of research: Early Childhood Development Intercultural Partnerships (www.ecdip.org). Dr. Ball is internationally renowned in the field of MLE and for her work promoting equity and inclusion in education, social services and health care for Indigenous children. Ms. Mariam Smith co-led the evaluation. Ms. Smith holds a Master of Science degree in pedagogy, and she has professional education in MLE. She has worked as a primary school teacher and as a researcher in Cambodia, where she lived with her family for 12 years. She is proficient in Khmer, Bunong, Swedish and English. She is familiar with the MENAP through her engagement with International Cooperation Cambodia and SIL. She has a good understanding of Indigenous issues and community systems in Cambodia. Ms. Smith is part of a number of on-going monitoring evaluation research initiatives for international development organizations in Asia and Sweden. She is a specialist in outcome harvesting and outcome mapping, informed by utilization-focused evaluation. She is skilled in qualitative, quantitative and mixed methodologies involving interviewing, facilitation of FGDs, discovery workshops, and qualitative data analysis and triangulation. Mr. Srom Bunthy served as leader of the in-country team. He organized field visits and interviews in provincial capitals, and led focus groups and workshops. He is Indigenous (Bunong) and speaks Khmer, English and some Bunong. He has worked with local NGOs on participatory action research projects, including a project on building community involvement in bilingual education, and he has worked on MLE advocacy, engaging in education workshops at national level, and facilitating workshops in the northeast. He worked for the government in a vocational training programme, and has been the chairperson of the Education Sector Working Group in Mondulkiri. He completed a bachelor degree in English Education at the Human Resources University in Cambodia and has completed several courses on translation and on participatory research methods. He has visited MLE programmes in the Philippines and Thailand, and co-presented with Mariam Smith at a language and education conference in Bangkok in 2010. Currently he owns and operates a printing business in Mondulkiri and works as a freelance consultant. Mr. Samorn Lamy served as a focus group and workshop facilitator and translator. He is Indigenous (Bunong) and Khmer, and is proficient in Bunong and Khmer. He has been active in developing MLE in north-eastern Cambodia, working from a community engagement perspective. He worked for International Cooperation Cambodia (ICC) for several years on MLE initiatives, where he contributed to training, community involvement and action research projects, and as an illustrator for books produced for mother-tongue based adult literacy programmes in Mondulkiri. In 2008 he worked for Health Unlimited. In 2011 he participated in a study visit on MLE in the Philippines and has facilitated numerous participatory workshops within the context of formal and non-formal education, particularly focused on culture. He currently runs a business in his home village in Mondulkiri and is a leader for the community forest. He is involved in Indigenous Peoples networks in Mondulkiri and the north- eastern region of Cambodia. Mr. Lamy is particularly skilled in engaging with children in the context of research and community engagement, applying his talents as an illustrator and communicator in visual methods of data collection and representation. Ms. Pleuk Phearom served as a translator and facilitator. She is Indigenous (Bunong) and proficient in Bunong and Khmer. She has worked for several years on various teacher training and participatory
14 Independent Evaluation of the Multilingual Education National Action Plan in Cambodia action research projects, including on MLE, and in education technical support, including with International Cooperation Cambodia. She is a member of the Bunong Community Network and active with the Indigenous Peoples Association as a leader and coordinator. Ms. Srey Mao served as a translator and facilitator. She is an Indigenous (Brao) woman who is proficient in Brao and Khmer. She has assisted Non-Timber Forest Products in Ratanakiri to collect data about primary school aged children in local communities and has worked as a language translator and assistant in trainings by CARE and International Cooperation Cambodia. Ms. Ka Nom served as a translator and facilitator. She is an Indigenous (Brao) woman who is proficient in Brao and Khmer. She has been active in her community in Ratanakiri with various initiatives and is frequently a representative at meetings that concern community wellness. Mr. Thuk Bun served as a translator and facilitator. He is Indigenous (Tampuan) and proficient in Tampuan, Krung, Lao, Kavet and some English. He worked for CARE from 2002–2017 as a teacher trainer and member of a language committee for material production in Tampuan and Krung. From 1999–2002 he worked with Non-Timber Forest Products as a community coordinator. Mr. Chuen Kham Phin served as a translator and facilitator. He is Indigenous (Krung) and proficient in Krung and Tampuan. He has worked as a contract teacher of Grades 1 and 2 at L’em Primary School in Ratanakiri for the past two years. He was a member of a literacy class committee in that community for five years. From 2007–2009 he worked as a health educator with Health Unlimited. From 2009–2011 he was a trainee with the Indigenous Youth Development Project, where he collected data and wrote reports on children’s education and health to support the work of Non-Timber Forest Products. Mr. Chab A served as a translator and facilitator. He is Indigenous (Kavet) and proficient in Kavet and Khmer. He works as a contract teacher in a public school in Ratanakiri and as a literacy teacher in non-formal education through International Cooperation Cambodia. He also works for Non-Timber Forest Products. Ms. Pim Kheav served as a translator and facilitator. She is Indigenous (Tampuan) and proficient in Tampuan and Khmer, and has some fluency in Krung. Before completing high school in 2010, she was a student in the Indigenous Youth Development Project. She has experience with community facilitation and has a range of computer skills. She worked with Non-Timber Forest Products on a project on out-of-school children, where she coordinated quarterly meetings of school support committees and local NGO partners. Her roles were to identify and support out-of-school children, procure learning materials for students living in poverty, track students’ performance, enter and update data for the project, collaborate with communities needing schools, and advocate with school authorities (DOE). Dr. Ken Moselle contributed support for analysis of quantitative data and report preparation. He is a statistician and research methodologist. He works as a director of applied clinical research within one of five major health service regions in the province of British Columbia, Canada. He is co-founder and co-director of the Data Science Studio in British Columbia. Dr. Moselle has extensive experience in a variety of child and youth development and education research and monitoring programmes in Canada, the United States, Malaysia and Singapore. He has a doctorate in psychology from the University of California at Berkeley and a degree in psychology and philosophy from Yale University. Mr. Phil Smith supported by preparing the sampling, making maps and diagrams, and general development advice. He works as a capacity development advisor at the Swedish Mission Council, an umbrella organization channelling funds from SIDA for international development work. Mr. Smith holds a Master of Arts in International Development and Management and serves as an advisor for Learning Loop, a consultancy firm working with learning frameworks to engage in complex social change.
15 Independent Evaluation of the Multilingual Education National Action Plan in Cambodia
Annex 4. Questions for key stakeholder groups Questions for which answers were sought for this evaluation are listed here. Additional questions were asked as experiences with aspects of the MENAP, concerns and perceived outcomes – both intended and unintended – were identified throughout data collection. Table 2. Evaluation questions
Stakeholder No. Questions for outcome harvesting and Reference to overlapping with most significant change ToR
1.1 When did you start using MLE in your school?
1.2 What were some important steps in this process?
1.3 What was it like before you started with MLE?
1.4 What are you doing differently now and when did that start? Why has this changed? (Dig for descriptions of methodology, community engagement and use of resources)
1.5 Of all of these events, what was the most significant change and why?
1.6 What are some ways that relationships have changed with the POE and MoEYS?
Service 1.7 Is that significant? How? providers in the community 1.8 How has your relationship changed with people in your community?
1.9 Is that significant? How?
1.10 How have things changed in your classroom?
1.11 Is that significant? How?
1.12 How are students doing things differently/achieving now?
1.13 What are ways that you have been restricted/limited Identification of in how you implement quality MLE? Why? barriers and bottlenecks
1.14 In what ways do you experience support for your Support of work with MLE? UNICEF, CARE
Stakeholder No. Questions for outcome harvesting and Reference to overlapping with most significant change ToR
2.1 When did you first start school?
2.2 How old were you when you first started school? Children in school 2.3 When did you first start learning how to read and write in your mother tongue?
16 Independent Evaluation of the Multilingual Education National Action Plan in Cambodia
2.4 What was it like before you started to use your mother tongue in school?
2.5 Is the teacher using your mother tongue in the classroom now? When did that start? 2.6 What kinds of things are you doing in the classroom now that you couldn’t do in the past? When and how did that happen?
2.7 What do you like most about school? When did that happen?
2.8 What are you able to do now that you are proud of? Children in How did you learn this? (Dig for descriptions of school quality education, participation, significance of the (continued) MLE programme).
2.9 Tell me about your mother tongue language abilities? Please tell about some things you can do in your mother tongue.
2.10 Tell me about your Khmer language abilities? Please tell about some things you can do in Khmer.
2.11 Of all of the things that have happened in your classroom over the last year in terms of the teacher and what you have been learning, what is most significant change and why?
2.12 How does your education help you and your family in your daily life?
Stakeholder No. Questions for outcome harvesting and Reference to overlapping with most significant change ToR
3.1 Do you remember when MLE was starting to be implemented in your village? What was your interest in MLE then?
3.2 What is your interest in MLE now?
3.3 How has your interest in MLE changed in the past one or two years?
Parents/ 3.4 What are your hopes for the future in terms of your guardians and children’s schooling? village leaders 3.5 What do you notice is different about the education that your children have today compared to in the past?
3.6 How are the children able to use their knowledge from school in the community? How are they able to use the knowledge and resources from the community in their school learning? (Probe for cultural and traditional knowledge as appropriate).
17 Independent Evaluation of the Multilingual Education National Action Plan in Cambodia
3.7 How does the MENAP support indigenous identity?
3.8 How does the MENAP affect other aspects of your life or the life of your community?
Stakeholder No. Questions for outcome harvesting and Reference to ToR overlapping with most significant change
4.1 Among the most significant change stories that were collected, which one do you feel was most significant Community for the quality of the children’s education in your synthesis village? Why? session (all of 4.2 What have been the most important barriers to the above positive change? present 4.3 What are your hopes for the future with regard to schooling for children in your community?
Stakeholder No. Questions for outcome harvesting and Reference to ToR overlapping with most significant change
5.1 What do you think is the main purpose of MLE? How MENAP did you learn about MLE? Objective 4
5.2 When did you start using MLE in your province? What
have been some important steps in this process?
What was it like before you started with MLE?
5.3 What are you doing differently now and when did that start? Why has this changed? (Dig for descriptions of practices within the office, methodology, training, community engagement, use of resources, and collection and disaggregation of data).
5.4 Of all of these events, what was the most significant change and why? 5.5 What changes have taken place in how you MENAP document and monitor student learning? Can you tell Objective 1 us about your qualitative data collection? Strategy 3 POE 5.6 How are students doing things differently/achieving now?
5.7 What are ways that you have been able to be
adaptive to the context so that children can best
access quality education? (For example, using
decentralized calendars, supporting teachers who
can competently teach, not testing in Khmer in 3rd
grade).
5.8 What has changed in teachers’ ability to use diverse MENAP Strategy 1 and appropriate methods? Why?
18 Independent Evaluation of the Multilingual Education National Action Plan in Cambodia
5.9 What do you think is the main purpose of MLE? How MENAP did you learn about MLE? Objective 4
5.10 When did you start using MLE in your province? What were some important steps in this process? What was it like before you started with MLE? 5.11 What are you doing differently now and when did that start? Why has this changed? (Dig for descriptions of practices within the office, methodology, training, community engagement, use of resources, and collection and disaggregation of data).
5.12 Of all of these events, what was the most significant change and why?
5.13 What changes have taken place in how you MENAP
document and monitor student learning? Can you tell Objective 1
us about your qualitative data collection? Strategy 3
5.14 How are students doing things differently/achieving now?
5.15 What are ways that you have been able to be adaptive to the context so that children can best access quality education? (For example, using POE decentralized calendars, supporting teachers who (continued) can competently teach, not testing in Khmer in 3rd grade)
5.16 What has changed in teachers’ ability to use diverse MENAP Strategy 1 and appropriate methods? Why?
5.17 What are some ways that relationships have changed with the communities, the DOE or MoEYS?
5.18 Are there other relationships that have changed?
5.19 How have things changed in your office? Why is that significant?
5.20 What are the changes taking place in responsibility MENAP for teacher training? Objective 2
5.21 How are you supporting the inclusion of children from MENAP other non-dominant languages (such as Lao and Objectives 1, 3 Cham)?
5.22 What is the progress on the six-year pilot mother MENAP tongue-based model? Who is taking action? Objective 1
5.23 What are the major factors that are enabling change? Identification of enabling factors
5.24 What have been the major obstacles to change? Identification of barriers and bottlenecks
19 Independent Evaluation of the Multilingual Education National Action Plan in Cambodia
5.25 What are ways that you have been restricted/limited Identification of in how you implement quality MLE? Why? barriers and bottlenecks
5.26 What challenges do you face in education as Khmer Identification of speakers move up to live in indigenous communities? barriers and How can these be addressed? bottlenecks
5.27 In what ways are you supported in your work with Support of MLE? UNICEF, Care
5.28 What system is in place for tracking the costs of ToR efficiency and delivering MLE, including teacher training, teacher sustainability salaries, materials development and provision, school expansion, operating costs?
5.29 What are the costs of delivering MLE, compared to ToR efficiency, the costs of delivering non-MLE? sustainability, impact
Stakeholder No. Questions for outcome harvesting and Reference to ToR overlapping with most significant change
Discovery 6.1 Which outcomes do you agree with? Do you have workshop (all of anything you would like to edit/add? the above present and 6.2 Which outcomes do you not agree with? Why? questioned, 6.3 Do you have anything to add or edit regarding the except for outcomes and their contribution? children, DOE and other 6.4 In order for the children to be successfully engaged potential in quality education what do you feel is important to stakeholders) work on for another MENAP phase?
6.5 What have been some of the obstacles in achieving quality MLE in your area?
6.6 How do you think that linguistic issues related to education can best be coordinated nationally?
Stakeholder No. Questions for outcome harvesting and Reference to ToR overlapping with most significant change
7.1 What was your role in supporting the MENAP?
7.2 What behavioural changes do you see taking place as a result of your work or the work (within the POE, among indigenous communities, in MLE Other potential classrooms)? stakeholders at 7.3 What behavioural changes do you see taking place provincial level as a result of other supporting actors (government, NGOs, etc.)? What are the changes within, for example, the POE, among indigenous communities, in MLE classrooms?
7.4 What were the major obstacles to change?
20 Independent Evaluation of the Multilingual Education National Action Plan in Cambodia
Stakeholder No. Questions for outcome harvesting and Reference to ToR overlapping with most significant change
8.1 What do you feel is the main purpose of MLE? How MENAP did you learn about MLE? Objective 4
8.2 Who has the main responsibility for MLE in MoEYS now? How is that different from before?
8.3 What was your role in supporting the MENAP?
8.4 What behavioural changes do you see taking place as a result of your work or the work (within the POE, among indigenous communities, in MLE classrooms)?
8.5 What behavioural changes do you see taking place as a result of other supporting actors (government/NGOs, etc.)? What are the changes MoEYS within, for example, the POE, among indigenous communities, in MLE classrooms?
8.6 What are the changes taking place in responsibility MENAP
for teacher training? Objective 2
8.7 How are you enabling POEs to adapt administrative and educational practices to the context so that children can best access quality education? (For example, using decentralized calendars)
8.8 What changes have taken place in providing MLE to MENAP ‘new’ indigenous languages (Kuy, Jarai)? What was Objective 3 your role and why? Strategy 4
8.9 What changes have taken place in providing MLE to MENAP
other non-dominant languages to ensure inclusion? Objectives 1, 3
(Lao, Cham, etc.) What was your role and why?
8.10 What changes have taken place in how you MENAP document and monitor student learning? Can you tell Objective 1 us about your qualitative data collection? Strategy 3
8.11 What were the major obstacles to change? MENAP Objective 1 MoEYS Strategy 3 (continued) 8.12 In what ways do you feel support for your work with Support of MLE? UNICEF, Care
8.13 What is the progress on the six-year pilot mother MENAP tongue-based model? Who is taking action? Objective 1
8.14 How does the MENAP support indigenous identity?
8.15 Who leads curriculum/literature development? Relevance is part of MENAP Objective 1
21 Independent Evaluation of the Multilingual Education National Action Plan in Cambodia
8.16 How are linguistic issues coordinated now? How do you feel that linguistic issues related to education can best be coordinated nationally?
8.17 Of the most significant change stories that were selected in the provincial discovery workshops, which one do you feel was most significant for the effective implementation of the MENAP? Why?
Stakeholder No. Questions for outcome harvesting and Reference to ToR overlapping with most significant change
9.1 LEAVE BLANK
9.2 Who has the main responsibility for MLE in MoEYS
now? How is that different from before?
9.3 What was your role in supporting the MENAP?
9.4 What behavioural changes do you see taking place as a result of your work or the work? (Within the POE, among indigenous communities, in MLE classrooms). 9.5 What behavioural changes do you see taking place CARE as a result of other supporting actors (government/NGOs, etc.)? What are the changes within, for example, the POE, among indigenous communities, in MLE classrooms)?
9.6 What are the changes taking place in responsibility MENAP for teacher training? Objective 2
9.7 How are you enabling POEs to adapt administrative and educational practices to the context so that children can best access quality education? (For example, using decentralized calendars). 9.8 What changes have taken place in providing MLE to MENAP ‘new’ indigenous languages (Kuy, Jarai)? What was Objective 3 your role and why? Strategy 4
9.9 What changes have taken place in providing MLE to MENAP other non-dominant languages to ensure inclusion Objectives 1, 3 (Lao, Cham, etc.)? What was your role and why?
9.10 What changes have taken place in how you MENAP
document and monitor student learning? Can you Objective 1
tell us about your qualitative data collection? Strategy 3
9.11 What were the major obstacles to change? MENAP Objective 1 Strategy 3
22 Independent Evaluation of the Multilingual Education National Action Plan in Cambodia
9.12 In what ways do you feel support for your work with Support of UNICEF MLE?
CARE 9.13 What is the progress on the six-year pilot mother MENAP (continued) tongue-based model? Who is taking action? Objective 1 9.14 How does the MENAP support indigenous identity?
9.15 Who leads curriculum development? Relevance is part of MENAP Objective 1
9.16 How are linguistic issues coordinated now? How do you feel that linguistic issues related to education can best be coordinated nationally?
9.17 Of the most significant change stories that were selected in the provincial discovery workshops, which one do you feel was most significant for the effective implementation of the MENAP? Why?
Stakeholder No. Questions for outcome harvesting and Reference to ToR overlapping with most significant change
10.1 LEAVE BLANK
10.2 Who has the main responsibility for MLE in
MoEYS now? How is that different from before?
10.3 What was your role in supporting the MENAP?
10.4 What behavioural changes do you see taking place as a result of your work or the work? (Within the POE, among indigenous communities, in MLE classrooms). UNICEF 10.5 What behavioural changes do you see taking place as a result of other supporting actors (government/NGOs, etc.)? What are the changes within, for example, the POE, among indigenous communities, in MLE classrooms?
10.6 What are the changes taking place in MENAP Objective 2 responsibility for teacher training?
10.7 How are you enabling POEs to adapt
administrative and educational practices to the
context so that children can best access quality
education? (For example, using decentralized
calendars).
10.8 What changes have taken place in providing MLE MENAP to ‘new’ indigenous languages (Kuy, Jarai)? What Objective 3 was your role and why? Strategy 4
23 Independent Evaluation of the Multilingual Education National Action Plan in Cambodia
10.9 What changes have taken place in providing MLE MENAP to other non-dominant languages to ensure Objectives 1, 3 inclusion? (Lao, Cham, etc.) What was your role and why?
10.10 What changes have taken place in how you MENAP document and monitor student learning? Can you Objective 1 tell us about your qualitative data collection? Strategy 3 10.11 What were the major obstacles to change? MENAP Objective 1 Strategy 3
UNICEF 10.12 In what ways do you feel support for your work (continued) with MLE?
10.13 What is the progress on the six-year pilot mother MENAP tongue-based model? Who is taking action? Objective 1
10.14 How does the MENAP support indigenous identity?
10.15 Who leads curriculum/literature development? Relevance is part of MENAP Objective 1
10.16 How are linguistic issues coordinated now? How do you feel that linguistic issues related to education can best be coordinated nationally?
10.17 Among the most significant change stories that were selected in the provincial discovery workshops, which one do you feel was most significant for the effect of implementing the MENAP? Why?
Stakeholder No. Questions for outcome harvesting and Reference to ToR overlapping with most significant change
Which outcomes do you agree with? Do you have anything you would like to edit/add?
Which outcomes do you not agree with? Why?
Do you have anything to add or edit regarding the outcomes and their contribution? PNH discovery workshop incl. In order for children to be successfully engaged in Indigenous quality education, and to improve the quality of people education and children’s success, what do you think is important to work on for another MENAP phase?
What were some of the obstacles to achieving quality MLE in your area?
What were some of the factors that enabled success in achieving quality MLE in your area?
24 Independent Evaluation of the Multilingual Education National Action Plan in Cambodia
How do you think that linguistic issues related to education can best be coordinated nationally?
25 Independent Evaluation of the Multilingual Education National Action Plan in Cambodia
Annex 5. Documents reviewed Alidou, Hassana, et al., ‘Optimizing Learning and Education in Africa: The language factor,’ Association for the Development of Education in Africa, Biennial Meeting, ADEA, Windhoek, 27-31 May 2006. Atchoarena, David and Lavinia Gasperini (eds), Education for Rural Development: Towards new policy responses, UNESCO International Institute for Educational Planning, 2003. Ball, Jessica, ‘Educational Equity for Children from Diverse Language Backgrounds: Mother tongue- based bilingual or multilingual education in the early years–Summary’, UNESCO, Paris, 2011. Ball, Jessica, ‘Nothing About Us Without Us: Restorative research partnerships involving Indigenous children and communities’, Ethical Research with Children, Open University Press/McGraw Hill Education, Berkshire, UK. 2015. Beller, Simone, ‘Fostering Language Acquisition in Daycare Settings: What does the research tell us?’ Working Papers in Early Childhood Development, No. 49. Bernard van Leer Foundation, P.O. Box 82334, 2508 EH, The Hague, the Netherlands, 2008. Benson, Carolyn J., ‘Real and Potential Benefits of Bilingual Programmes in Developing Countries’, International Journal of Bilingual Education and Bilingualism, vol. 5, no. 6, 2002, pp. 303-317. Benson, Carolyn J., ‘The Importance of Mother Tongue-Based Schooling for Educational Quality’, Background paper for the Education for All Global Monitoring Report: The Quality Imperative, UNESCO, Paris, 2004. Benson, Carolyn J., ‘Educating Learners in their Home Languages: Establishing and maintaining successful programmes’, Centre for Applied Linguistics (CAL), Washington, D.C., 2009. Benson, Carolyn J., ‘Consultancy Report: Research and collaboration on MLE implementation in Cambodia with special attention to L1 and Khmer writing’, CARE, Phnom Penh, 2018. Benson, Carolyn, Sarah French and Dany Khieu, ‘Report of Teachers College Team Field Visit: Continuing longitudinal research, making research contacts and awareness-raising about MLE in Cambodia’, CARE Cambodia, Phnom Penh, 22 June 2018. Benson, Carol, Kevin Wong and Yinying Li, ‘Evaluation of the State of Multilingual Education in Cambodia’, Teachers College, Columbia University, New York, CARE Cambodia, 2015. Brock-Utne, Birgit, ‘Education for All: In whose language?’ Oxford Review of Education, vol. 27, no. 1 Oxford, UK, 2001, pp.115-134. Brock-Utne, Birgit, ‘Language-in-Education Policies and Practices in Africa with a Special Focus on Tanzania and South Africa—Insights from research in progress’, International Handbook on Globalisation, Education and Policy Research, Springer, Dordrecht, Netherlands, 2005, pp. 549-565. Burchill, Marlene et al., ‘Reflections on 'Aboriginalising' the Research Process: ‘Hunting and gathering’ as a focus group methodology’, International Journal of Critical Indigenous Studies, vol. 4, no. 2, 2011, pp. 29-39. Burton, Lisa Ann, ‘Mother Tongue-Based Multilingual Education in the Philippines: Studying a top- down policy from the bottom up’, Unpublished Ph.D. thesis, University of Minnesota, Minneapolis, Minn., 2013. CARE Cambodia, ‘CARE Cambodia: Empowering women through education’, 2015. CARE Cambodia, ‘Multilingual Education in Cambodia: Strengths and challenges’, PowerPoint presentation at education retreat, 2018. CARE Cambodia, ‘Multilingual Education for Ethnic Minority Children in Cambodia’, Phnom Penh, 2017.
26 Independent Evaluation of the Multilingual Education National Action Plan in Cambodia
CARE Cambodia and CARE USA, ‘Bending Bamboo: Quality education for ethnic minority children in Cambodia’, CARE, Phnom Penh, 2016. Caxaj, Susana C., ‘Indigenous Storytelling and Participatory Action Research: Allies toward decolonization? Reflections from the peoples’ international health tribunal’, Global Qualitative Nursing Research vol. 2, 2015. Chilisa, Bagele, Indigenous Research Methodologies, Sage Publications, London, 2011, pp. 35-44. Coleman, Hywel (ed.), Language and Social Cohesion in the Developing World, British Council and Deutsche Gesellschaft fur Internationale Zusammenarbeit (GIZ) GmbHPinnock et al., 2015. Collier, Virginia P., ‘How Long? A synthesis of research on academic achievement in a second language’, TESOL Quarterly 23, no. 3 (1989): 509-531. Cummins, Jim, ‘The Acquisition of English as a Second Language’, Reading Instruction for ESL Students, International Reading Association, Delaware, 1994. Cummins, Jim, ‘Bilingual Education and Special Education: Issues in assessment and pedagogy’, College Hill, San Diego, 1984. Cummins, Jim, ‘Fundamental Psycholinguistic and Sociological Principles Underlying Educational Success for Linguistic Minority Students’, Social Justice Through Multilingual Education, Clevedon, UK, Multilingual Matters, 2008, pp. 19-35. Cummins, Jim, Language, Power and Pedagogy: Bilingual children in the crossfire, Multilingual Matters, vol. 23, 2000. Cummins, Jim, ‘BICS and CALP: Empirical and theoretical status of the distinction’, Encyclopedia of Language and Education (2nd Ed.), vol. 2, Springer Science and Business Media LLC, New York, 2008, pp. 71-83. Datta, Ranjan, ‘Traditional Storytelling: An effective Indigenous research methodology and its implications for environmental research’, AlterNative: An International Journal of Indigenous Peoples, vol. 14, no. 1, 2018, pp. 35-44. De Grauw, Anton, ‘JTWG-Education Retreat 2018’, Meeting summary, 20-22 June 2018. Drawson, Alexandra S., Elaine Toombs and Christopher J. Mushquash, ‘Indigenous Research Methods: A systematic review’, The International Indigenous Policy Journal, vol. 8, no. 2, 2017, p. 5. Dutcher, Nadine, ‘Promise and Perils of Mother Tongue Education’, 2003. Edwards, Viv, ‘Education and the Development of Early Childhood Bilingualism’, Voces Diversae: Lesser-Used Language Education in Europe, Belfast, 2006, pp. 16-24. Edwards, Viv and Jacob Marriote Ngwaru, ‘African Language Publishing for Children: Where next?’ National Centre for Language and Literacy, Reading, UK, 2010. Edwards, Viv and Jacob Marriote Ngwaru, ‘African Language Publishing for Children in South Africa: Challenges for translators’, International Journal of Bilingual Education and Bilingualism, vol. 14, no. 5, 2011, pp. 589-602. Edwards, Viv and Jacob Marriote Ngwaru, ‘African Language Books for Children: Issues for Authors’, Language, Culture and Curriculum vol. 25 no. 2012, pp.123-138. Edwards, Viv and Jacob Marriote Ngwaru, ‘Language Capital and Development: The case of African language publishing for children in South Africa’, International Journal of the Sociology of Language, 2014, no. 225, 2014, pp. 29-50. Frawley, Jack, ‘Child-Friendly Schools (CFS) and Multilingual Education (MLE): The CFS-MLE framework’, CARE Cambodia, Phnom Penh, 2017.
27 Independent Evaluation of the Multilingual Education National Action Plan in Cambodia
Frawley, Jack, ‘Multilingual Education and the Child-Friendly Schools Policy: Adding MLE indicators to the CFS indicator framework’, CARE Cambodia. Phnom Penh: CARE Cambodia, 2017. Funnell, Sue C., and Patricia J. Rogers, Purposeful Program Theory: Effective use of theories of change and logic models, Vol. 31, John Wiley & Sons, 2011. Graham, Ann, Mary Ann Powell, Nicola Taylor, Donnah Anderson and Robyn Fitzgerald, ‘Ethical Research Involving Children’, UNICEF Office of Research – Innocenti, Florence, 2013. H.E. Naron, Hang Chuon, Closing remarks delivered at the Asia-Pacific Regional ECD conference, Siem Reap, Cambodia, 3 March 2017. Heikkila, Stina, ‘Multilingual Education in Cambodia’ PowerPoint presentation for CARE Cambodia, 11 June 2017. Heugh, Kathleen, ‘Cost Implications of the Provision of Mother-Tongue and Strong Bilingual Models of Education in Africa’, Association for the Development of Education in Africa (ADEA), 2012, pp. 138- 156. Heugh, Kathleen, ‘The Case Against Bilingual and Multilingual Education in South Africa: Laying bare the myths: Many languages in education: Issues of implementation’, Perspectives in Education, vol. 20, no. 1, 2002, pp. 171-196. Kelsall, Tim, Sothy Khieng, Chuong Chantha and Tieng Muy, ‘The Political Economy of Primary Education Reform in Cambodia’, 2016. Kioko, Angelina N., Ruth W. Ndung’u, Martin C. Njoroge and Jayne Mutiga, ‘Mother Tongue and Education in Africa: Publicising the reality’, Multilingual Education, vol. 4, no. 1, 2014, p. 18. Kosonen, Kimmo, ‘The Use of Non-Dominant Languages in Education in Cambodia, Thailand and Vietnam’, In Language Issues in Comparative Education, Sense Publishers, Rotterdam, 2013, pp. 39- 58. Krause, Brooke and Ali Joglekar, ‘Preliminary Quantitative Findings on MLE in Cambodia’, In annual conference of the Comparative and International Education Society, Vancouver BC, March, vol. 9. 2016. Kraft, Richard J., ‘Primary Education in Ghana: A report to USAID’, USAID/Ghana Consultancy Report, 2003. Kruger, Haidee, ‘Language-in-Education Policy, Publishing and the Translation of Children’s Books in South Africa’, Perspectives: Studies in Translatology, vol. 17, no. 1, 2009, pp. 33-61. Kurtz, Cynthia F. and David J. Snowden, ‘The New Dynamics of Strategy: Sense-making in a complex and complicated world’, IBM Systems Journal, vol. 42, no. 3, 2003, pp. 462-483. Lall, Marie, ‘Diversity in Education in Myanmar’, Pyoe Pin Programme/UK AID, Yangon, 2016. Lall, Marie and Ashley South, ‘Comparing Models of Non-State Ethnic Education in Myanmar: The Mon and Karen national education regimes’, Journal of Contemporary Asia, vol. 44, no. 2, 2014, pp. 298-321. Lee, Scott, Ron Watt and Jack Frawley, ‘Effectiveness of Bilingual Education in Cambodia: A longitudinal comparative case study of ethnic minority children in bilingual and monolingual schools’, Compare: A Journal of Comparative and International Education, vol. 45, no. 4, 2015, pp. 526-544. Lightbown, Patsy, ‘Easy as Pie? Children learning languages’, Concordia Working Papers in Applied Linguistics, vol. 1, 2008, pp. 5-29. Lo Bianco, J., ‘Language and Social Cohesion, Malaysia, Myanmar, Thailand’, Final desk review, Bangkok: East Asia and Pacific Regional Office, UNICEF, 2013.
28 Independent Evaluation of the Multilingual Education National Action Plan in Cambodia
Lo Bianco, J., ‘Conflict, Language Rights and Education: Building peace by solving language problems in Southeast Asia’, Center for Applied Linguistics, 2016. Lo Bianco, Jo, ‘Scaling Up: Expanding bilingual education’, UNICEF, Bangkok, 2016. Mahboob, Ahmar and Priscilla Cruz, ‘English and Mother-Tongue-Based Multilingual Education: Language attitudes in the Philippines’, AJELS, vol. 1, 2013, pp. 1-19. Malone, Dennis L., ‘Developing Curriculum Materials for Endangered Language Education: Lessons from the field’, International Journal of Bilingual Education and Bilingualism, vol. 6, no. 5, 2003, pp. 332-348. Malone, Susan, ‘Mother Tongue-Based Multilingual Education: Implications for education policy’, In Seminar on Education Policy and the Right to Education: Towards More Equitable Outcomes for South Asia's Children (2007 Kathmandu), 2007, pp. 17-20. Malone, Susan, E., ‘Planning Mother Tongue-Based Education Programs in Minority Language Communities’, SIL International, 2010. Malone, Susan and Patricia Paraide, ‘Mother Tongue-Based Bilingual Education in Papua New Guinea’, International Review of Education, vol. 57, no. 5-6, 2011, pp. 705-720. Metila, Romylyn A., Lea Pradilla and Alan B. Williams, ‘The Challenge of Implementing Mother Tongue Education in Linguistically Diverse Contexts: The case of the Philippines’, The Asia-Pacific Education Research, vol. 25, no. 5-6, pp. 781-789. Middelborg, Jorn, ‘Highland Children’s Education Project: A pilot project on bilingual education in Cambodia’, UNESCO, Bangkok, 2005. Ministry of Education, Youth and Sport, ‘Early Childhood Care and Development Sub-Sector Congress Report’ MoEYS, Phnom Penh, 2018. Ministry of Education, Youth and Sport, ‘The Education Congress: The Education Youth and Sport Performance in the Academic Year 2013-2014 and Goals for Academic Year 2014-2015’, March 2015. Ministry of Education, Youth and Sport, ‘The Education Congress: The Education Youth and Sport Performance in the Academic Year 2014-2015 and Goals for Academic Year 2015-2016’, March 2016. Ministry of Education, Youth and Sport, ‘The Education Congress: The Education Youth and Sport Performance in the Academic Year 2015-2016 and Goals for Academic Year 2016-2017’, March 2017. Ministry of Education, Youth and Sport, ‘The Education Congress: The Education Youth and Sport Performance in the Academic Year 2017-2018 and Goals for Academic Year 2018-2019’, March 2018. Ministry of Education, Youth and Sport, ‘Education Strategic Plan 2009-2013’, MoEYS, Phnom Penh, 2009. Ministry of Education, Youth and Sport, ‘Education Strategic Plan 2014-2018’, official document, MoEYS, Phnom Penh, 2014. Ministry of Education, Youth and Sport, ‘Guidelines on the Implementation of the Accelerated Learning Programme at Primary Education’, MoEYS, Phnom Penh, 2013. Ministry of Education, Youth and Sport, ‘Guidelines on the Implementation of Education for Indigenous Children in Highland Provinces’, official document, MoEYS, Phnom Penh. Ministry of Education, Youth and Sport, ‘Guidelines on Secondary School Support Committees’, MoEYS, Phnom Penh, 12 July 2012. Ministry of Education, Youth and Sport, ‘Master Plan for Capacity Development in the Education Sector 2014-2018’, MoEYS, Phnom Penh, January 2015. Ministry of Education, Youth and Sport, ‘Multilingual Education National Action Plan (MENAP) 2015- 2018’, MoEYS, Phnom Penh, 2015.
29 Independent Evaluation of the Multilingual Education National Action Plan in Cambodia
Ministry of Education, Youth and Sport, ‘National Action Plan on Early Childhood Development 2016- 2018’, MoEYS, Phnom Penh, 13 August 2014. Ministry of Education, Youth and Sport, ‘National Policy on Early Childhood Care and Development’, MoEYS, Phnom Penh, 19 February 2010. Ministry of Education, Youth and Sport, ‘Prakas on Identification of Languages for Khmer National Learners Who are Indigenous People’, Official document, MoEYS, Phnom Penh, 2013. Ministry of Education, Youth and Sport, ‘Prakas Number 48, Identification of Language for Learners of Khmer Nationality and Ethnic Minority Origin’, Official document, MoEYS, Phnom Penh, 2013. Ministry of Education, Youth and Sport, ‘Presentation on Education Retreat Results’ (Powerpoint presentation), 1 October 2018. Ministry of Education, Youth and Sport, ‘Policy on Child-Friendly Schools for Basic Education’, MoEYS, Phnom Penh, 2012. Ministry of Education, Youth and Sport, ‘Primary Education Sub-Sector Congress Report’, MoEYS, Phnom Penh, 2018. Ministry of Education, Youth and Sport, ‘Road Map for New MENAP 2019-2023’, MoEYS, Phnom Penh, 2018. Ministry of Education, Youth and Sport, ‘Teacher Policy’, MoEYS, Phnom Penh, May 2013. Ministry of Education, Youth and Sport, ‘Training of Trainers Module: Implementing bilingual education in pre-schools’, MoEYS, Phnom Penh, 29 November 2012. Moll, Luis C., Cathy Amanti, Deborah Neff and Norma Gonzalez, ‘Funds of Knowledge for Teaching: Using a qualitative approach to connect homes and classrooms’, Theory into Practice, vol.31, no. 2, 1992, pp. 132-141. Nowaczyk, Monika, ‘Advocating for Multilingual Education in Cambodia: Experiences and strategies’, CARE Cambodia, Phnom Penh, 2015. O'Reilly, Karen, Key Concepts in Ethnography, SAGE, London, 2009. Person, K., ‘The Impact of Early Childhood Mother Tongue Interventions on Grade 1 Literacy Skills of Ethnic Minority Children in Thailand’, paper presented at the Annual Conference of Asia Pacific Regional Network for Early Childhood, Kathmandu, 5 June 2018. Pinnock, Helen, Pamela Mackenzie, Elizabeth Pearce and Catherine Young, ‘Closer to Home: How to help schools in low- and middle-income countries respond to children’s language needs’, Save the Children and CfBT, Save the Children, London, 2011. Purdon, Alisa, ‘A Feasibility Study for Implementing Bilingual Education for Ethnic Minority Children in Five Provinces of Cambodia: Kratie, Mondulkiri, Stung Treng, Preah Vihear and Ratanakiri’, CARE, Phnom Penh, 2006. Royal Government of Cambodia, Article 24, Education Law, 2007. Royal Government of Cambodia, Article 66, The Constitution of the Kingdom of Cambodia, 1993. Royal Government of Cambodia, The National Policy for Ethnic Minorities Development, 2008. Royal Government of Cambodia, National Policy on Early Childhood Care and Development, 2010. Royal Government of Cambodia, National Policy on Non-Formal Education, n.d. Royal Government of Cambodia, National Strategic Development Plan, 2014. Royal Government of Cambodia, Rectangular Strategy for Growth, Employment, Equity and Efficiency: Building the foundation toward realizing the Cambodia Vision 2050. Phase IV, paper
30 Independent Evaluation of the Multilingual Education National Action Plan in Cambodia presented at the Royal Government of Cambodia of the Sixth Legislature of the National Assembly, Phnom Penh, September 2018. Royal Government of Cambodia, Sub-Decree on the Management of Community Pre-Schools, 25 December 2017. Royal Government of Cambodia, 2015, UNICEF Country Programme Action Plan 2016-2018, p. 8. Shaeffer, Sheldon and Kreng Heng, ‘Joint Formative Evaluation of Child-Friendly School Policy Implementation in Cambodia’, MoEYS and UNICEF Cambodia Country Office, Phnom Penh, 2016. Skutnabb-Kangas, Tove, Linguistic Genocide in Education–Or Worldwide Diversity and Human Rights? Lawrence Erlbaum, Mahwah, NJ, 2012. South, Ashley and Marie Lall, ‘Language, Education and the Peace Process in Myanmar’, Contemporary Southeast Asia: A Journal of International and Strategic Affairs, vol. 38, no. 1, 2016, pp. 128-153. South, Ashley and Marie Lall, ‘Schooling and Conflict: Ethnic education and mother tongue-based teaching in Myanmar’, The Asia Foundation/USAID, 2016, Retrieved from asiafoundation.org/resources/pdfs/SchoolingConflictENG.pdf. Smith, Mariam and Tim Sangvat (contributors), ‘Cambodia: A bilingual education programme for youth and adults from the Phnong community’, Mother Tongue-Based Literacy Programmes. Case studies of good practice in Asia, UNESCO, Bangkok, 2007. Tandon, Prateek and Tsuyoshi Fukao, ‘Educating the Next Generation: Improving teacher quality in Cambodia’, The World Bank, 2015. Tin, Tan Bee, ‘Student-Teacher-Made Language Teaching Materials: A developmental approach to materials development’, Focus on ELT Materials, Selangor Darul Ehsan: Pearson Malaysia, 2006, pp. 207-227. Thomas, Wayne P. and Virginia Collier, ‘School Effectiveness for Language Minority Students’, Resource Collection Series, no. 9, National Clearinghouse for Bilingual Education, Washington, DC, 1997. Thomas, Wayne P. and Virginia P. Collier, ‘A National Study of School Effectiveness for Language Minority Students’ Long-Term Academic Achievement’, UC Santa Cruz Center for Research on Education, Diversity and Excellence, Santa Cruz, Calif, 2002. Thomas, Wayne P. and Virginia P. Collier, ‘Why Dual Language Schooling’, Fuente Press, Albuquerque, NM, 2017. Tupas, Ruanni, ‘Inequalities of Multilingualism: Challenges to mother tongue-based multilingual education’, Language and Education, vol. 29, no. 2, 2015, pp.112-124. UNESCO, ‘The Use of the Vernacular Languages in Education’, Monographs on Foundations of Education, no. 8, UNESCO, Paris 1953. UNESCO Bangkok, ‘Advocacy Brief on Mother Tongue-Based Teaching and Education for Girls’, UNESCO, Bangkok, 2005. UNESCO, ‘Advocacy Kit for Promoting Multilingual Education: Including the excluded’, UNESCO, Bangkok, 2007. UNESCO, ‘Mother Tongue Instruction in Early Childhood Education: A selected bibliography’, UNESCO, Paris, 2008. UNESCO ‘MTB MLE Resource Kit: Including the excluded–Promoting multilingual education’, UNESCO, Bangkok, 2016.
31 Independent Evaluation of the Multilingual Education National Action Plan in Cambodia
UNESCO, ‘Situational Analysis of Out-of-School Children in Nine Southeast Asian Countries’, UNESCO Bangkok Office, Paris, 2017. UNICEF Office of Research, ‘Procedure for Ethical Standards in Research, Evaluation and Data Collection and Analysis’, 2015. UNICEF Cambodia, Terms of Reference: Individual consultancy for evaluation of Cambodia Multilingual Education National Action Plan, 2018. UNICEF Cambodia, ‘Updated Situation Analysis of Children and Women in Cambodia’, UNICEF Cambodia, 2017. United Nations, United Nations Declaration on the Rights of Indigenous Peoples, 2008. United Nations Evaluation Group, ‘UNEG Ethical Guidelines for Evaluation’, UNEG, 2008. United Nations Evaluation Group, ‘Norms and Standards for Evaluation’, UNEG, New York, 2016. United Nations, Department of Economic and Social Affairs, Population Division, ‘World Population Prospects: The 2017 revision, key findings and advance table’, Working Paper 248, 2017. Van Gerwen, Frans, et al., ‘Outcome Evaluation of the Education Capacity Development Partnership Fund (CDPF): Phases 1 and 2: Final report – Volume 1’, MoEYS Directorate General of Policy and Planning and UNICEF Cambodia Country Office, Phnom Penh, 2018. Walter, Steve and Carol Benson, ‘Language Policy and Medium of Instruction in Formal Education’, The Cambridge Handbook of Language Policy, Cambridge University Press, Cambridge, 2012, pp. 278-300. The World Bank, ‘Equity and Development’, World Development Report 2006, The World Bank and Oxford University Press, Washington, DC, 2006. World Education Forum, ‘The Dakar Framework for Action’, 2000. Zanolini, Arianna, ‘ECE Longitudinal Study Evaluation’, UNICEF Cambodia, Phnom Penh, 2011.
32 Independent Evaluation of the Multilingual Education National Action Plan in Cambodia
Annex 6. Participant lists The evaluation team had the privilege of meeting approximately 696 participants (317 / 45 per cent female) 15 who shared their perspectives, experience and knowledge. Table 3 is a list of all the individuals who took part in key informant interviews, and two survey participants. Table 4 is a log of all participants in FGDs and group interviews. Table 3. Key informant interviews Date Type Gender Position/role 8 Aug, 2018 School M MLE Teacher 8 Aug, 2018 School M Principal of primary school 9 Aug, 2018 INGO M UNICEF: Programme Officer 12 Aug, 2018 School M Teacher; NGO Volunteer 12 Aug, 2018 School F MLE Preschool teacher 12 Aug, 2018 NGO M NGO Project Manager 13 Aug, 2018 Child M Out-of-School Child 13 Aug, 2018 Community F Parent 13 Aug, 2018 Community M School support committee leader, deputy village chief 14 Aug, 2018 School F School director 14 Aug, 2018 NGO M NGO Project Manager 9, 10, 17 Aug, 2018 INGO M CARE: Management 10, 15, 17 Aug, 2018 INGO F CARE: Project Manager 10, 17 Aug, 2018 INGO F CARE Programme Officer 15 Aug, 2018 POE M POE Official 15 Aug, 2018 POE M POE Official 16 Aug, 2018 INGO M UNICEF Programme Officer 17 Aug, 2018 INGO F UNICEF Programme Officer 17 Aug, 2018 MoEYS M MoEYS Department Director 30 Aug, 2018 MoEYS F MoEYS Department Officer 4 Sep, 2018 POE F POE Officer 7 Sep, 2018 Community F Community member 7 Sep, 2018 MoEYS F MoEYS Department Officer 10 Sep, 2018 NGO M NGO Advisor 7 Sep, 2018 INGO M INGO Officer 13 Sep, 2018 School M School director of MLE primary school 13 Sep, 2018 School M School director of MLE primary school 13 Sep, 2018 Donor Agency U USAID 13 Sep, 2018 Donor Agency U DVV (Deutscher Volkshochschule-Verband) 14 Sep, 2018 Provincial Teacher M PTTC Training College (PTTC) 14 Sep, 2019 School F MLE Preschool teacher 17 Sep, 2018 POE M POE Official 17 Sep, 2018 POE M POE Official 17 Sep, 2018 POE F POE Official 22 Sep, 2018 Community F Parent 24 Sep, 2018 NGO M NTFP Director 24 Sep, 2018 NGO M ICC Education officer 24 Sep, 2018 INGO F Care Officer 1 Oct, 2018 MoEYS M Deputy Director Dept of Curriculum Development
15 This total number of participants in focus groups discussions and key informant interviews adds up to 805 participants (356 / 44 per cent female), a higher number due to participation in more than one group for triangulation and substantiation purposes.
33 Independent Evaluation of the Multilingual Education National Action Plan in Cambodia
Date Type Gender Position/role 4 Oct, 2018 Royal University of M Royal University of Phnom Penh Phnom Penh 4 Oct, 2018 Indigenous peoples’ M CIPO organizations TOTAL 41* (F: 14) *The two unknown were survey responses (marked as gender unknown)
34 Independent Evaluation of the Multilingual Education National Action Plan in Cambodia
Table 4. List of participants in FGDs and group interviews Date Group Female Male Province Language Data collection methods 8 Aug, 2018 Children 10 8 Mondulkiri Bunong Pictorial mapping, FGD, most significant change story selection, synthesis discussion 8 Aug, 2018 Parents including 4 3 Mondulkiri Bunong, Khmer Pictorial mapping, FGD, most significant change school support story selection, synthesis discussion committee 8 Aug, 2018 School leadership 2 2 Mondulkiri Bunong, Khmer Pictorial mapping, FGD, most significant change story selection, synthesis discussion 9 Aug, 2918 Multi-stakeholder 7 14 Phnom Penh Mixed Workshop with World Café style FGD mapping workshop outcomes and contributions 10 Aug, 2018 School leadership 2 1 Mondulkiri Bunong FGD and community members 11 Aug, 2018 Village and school 3 2 Mondulkiri Bunong FGD leaders 12 Aug, 2018 Children 5 8 Mondulkiri Bunong Pictorial mapping, FGD, most significant change story selection, synthesis discussion 12 Aug, 2018 Parents including 3 3 Mondulkiri Bunong, Khmer Pictorial mapping, FGD, most significant change school support story selection, synthesis discussion committee 12 Aug, 2018 School leadership 2 1 Mondulkiri Bunong Pictorial mapping, FGD, most significant change story selection, synthesis discussion 12 Aug, 2018 School teachers in 0 4 Mondulkiri Bunong, Khmer FGD non-MLE school and parents 13 Aug, 2018 Children 10 2 Mondulkiri Bunong Pictorial mapping, FGD, most significant change story selection, synthesis discussion 13 Aug, 2018 Parents including 10 10 Mondulkiri Bunong Pictorial mapping, FGD, most significant change school support story selection, synthesis discussion committee 13 Aug, 2018 School leadership 0 3 Mondulkiri Bunong Group interview
14 Aug, 2018 Children 6 9 Mondulkiri Bunong Pictorial mapping, FGD, most significant change story selection, synthesis discussion
35 Independent Evaluation of the Multilingual Education National Action Plan in Cambodia
Date Group Female Male Province Language Data collection methods 14 Aug, 2018 Parents including 7 4 Mondulkiri Bunong, Khmer Pictorial mapping, FGD, most significant change school support story selection, synthesis discussion committee 14 Aug, 2018 School leadership 2 0 Mondulkiri Bunong Group interview, most significant change story selection, synthesis discussion 14 Aug, 2018 School leadership 1 3 Mondulkiri Bunong Group interview 15 Aug, 2018 POE and DOE 2 13 Mondulkiri Mixed FGD 15 Aug, 2018 Mondulkiri 5 18 Mondulkiri Mixed Most significant change story selection, Provincial discussion around outcomes, barriers, hopes, Discovery rating workshop 5 Aug, 2018 DOE 1 1 Mondulkiri Khmer Group interview 5 Sep, 2018 DOE 0 2 Stung Treng Khmer Group interview 5 Sep, 2018 School support 0 3 Stung Treng Lao Group interview committee, teachers, school director 6 Sep, 2018 School leadership 1 1 Stung Treng Bunong Pictorial mapping, FGD, most significant change story selection, synthesis discussion 6 Sep, 2018 Parents including 11 1 Stung Treng Bunong Pictorial mapping, FGD, most significant change school support story selection, synthesis discussion committee 6 Sep, 2018 Children 20 20 Stung Treng Bunong Pictorial mapping, FGD
7 Sep, 2018 Children 4 16 Ratanakiri Krung Pictorial mapping, FGD, most significant change story selection, synthesis discussion 7 Sep, 2018 Parents including 3 4 Ratanakiri Krung Pictorial mapping, FGD, most significant change school support story selection, synthesis discussion committee 7 Sep, 2018 Teacher 2 0 Ratanakiri Krung Pictorial mapping, FGD, most significant change story selection, synthesis discussion 7 Sep, 2018 Parents including 1 3 Ratanakiri Krung FGD school support committee 7 Sep, 2018 Teacher 3 1 Ratanakiri Krung FGD 8 Sep, 2018 School leadership 3 0 Ratanakiri Krung FGD
36 Independent Evaluation of the Multilingual Education National Action Plan in Cambodia
Date Group Female Male Province Language Data collection methods 8 Sep, 2018 Parents and 2 2 Ratanakiri Krung FGD school support committee 8 Sep, 2018 Children 12 12 Ratanakiri Krung FGD 8 Sep, 2018 Parents including 1 4 Ratanakiri Krung FGD school support committee 8 Sep, 2018 Teachers and 3 2 Ratanakiri Krung FGD librarian 12 Sep, 2018 School leadership 2 1 Kratie Bunong Pictorial mapping, FGD, most significant change story selection, synthesis discussion 12 Sep, 2018 Parents and 1 2 Kratie Bunong, Khmer Pictorial mapping, FGD, most significant change school support story selection, synthesis discussion committee 13 Sep, 2018 Parents and 13 7 Kratie Bunong Pictorial mapping, FGD, most significant change school support story selection, synthesis discussion committee 13 Sep, 2018 Children 15 15 Kratie Bunong Pictorial mapping, FGD, most significant change story selection, synthesis discussion 13 Sep, 2018 Parents and 1 1 Kratie Khmer FGD school support committee 14 Sep, 2018 Parents and 9 2 Kratie Bunong FGD school support committee 14 Sep, 2018 School leadership 1 1 Kratie Khmer, Bunong FGD
14 Sep, 2018 Parents and 9 5 Kratie Bunong FGD school support committee 15 Sep, 2018 School leadership 1 2 Kratie Khmer FGD 15 Sep, 2018 Parents and 0 2 Kratie Khmer, Bunong FGD school support committee
37 Independent Evaluation of the Multilingual Education National Action Plan in Cambodia
Date Group Female Male Province Language Data collection methods 15 Sep, 2018 School leadership 0 3 Kratie Mil/Khmer, FGD Khmer 15 Sep, 2018 Parents and 0 2 Kratie Mil, Khmer FGD school support committee 18 Sep, 2018 Discovery 6 10 Kratie Khmer Bunong Discovery workshop: World Café style workshop discussions around outcomes, discussion on barriers 18 Sep, 2018 MLE teachers 16 17 Kratie Bunong FGD, survey 19 Sep, 2018 DOE leadership 0 2 Stung Treng Khmer FGD 19 Sep, 2018 Parents including 0 3 Stung Treng Kavet FGD school support committee 19 Sep, 2018 Teachers 2 0 Stung Treng Kavet FGD 19 Sep, 2018 Students 2 0 Stung Treng Kavet FGD
20 Sep, 2018 POE officers 1 1 Stung Treng Khmer FGD 20 Sep, 2018 PTTC leadership 0 3 Stung Treng Khmer FGD 20 Sep, 2018 School leadership 3 1 Stung Treng Brao FGD 20 Sep, 2018 Parents including 6 3 Stung Treng Brao FGD school support committee 20 Sep, 2018 Children 6 3 Stung Treng Brao FGD 21 Sep, 2018 DOE leadership 0 2 Stung Treng Khmer FGD 21 Sep, 2018 Discovery 5 10 Stung Treng Khmer, Kavet, Discovery workshop: World Café style workshop Brao, discussions around outcomes, discussion on Bunong barriers 22 Sep, 2018 Children 1 1 Ratanakiri Tampuan Group interview 22 Sep, 2018 Preschool teacher 1 1 Ratanakiri Tampuan Group interview and DOE Officer 22 Sep, 2018 Children 12 17 Ratanakiri Tampuan Pictorial mapping, FGD, most significant change story selection, synthesis discussion 22 Sep, 2018 Parents including 3 1 Ratanakiri Tampuan Pictorial mapping, FGD, most significant change school support story selection, synthesis discussion committee
38 Independent Evaluation of the Multilingual Education National Action Plan in Cambodia
Date Group Female Male Province Language Data collection methods 22 Sep, 2018 Teacher 0 2 Ratanakiri Tampuan Pictorial mapping, FGD, most significant change story selection, synthesis discussion 22 Sep, 2018 Parents including 0 6 Ratanakiri Tampuan Group interview school support committee
22 Sep, 2018 Teacher 0 3 Ratanakiri Tampuan Group interview 23 Sep, 2018 School leadership 1 2 Ratanakiri Tampuan Pictorial mapping, FGD, most significant change story selection, synthesis discussion 23 Sep, 2018 Parents and 5 5 Ratanakiri Tampuan Pictorial mapping, FGD, most significant change school support story selection, synthesis discussion committee 23 Sep, 2018 Children 24 18 Ratanakiri Tampuan Pictorial mapping, FGD, most significant change story selection, synthesis discussion 23 Sep, 2018 School leadership 2 1 Ratanakiri Tampuan FGD 23 Sep, 2018 Parents and 3 3 Ratanakiri Tampuan FGD school support committee 23 Sep, 2018 Children 8 4 Ratanakiri Tampuan FGD 23 Sep, 2018 Teachers and 1 3 Ratanakiri Tampuan FGD school support committee 23 Sep, 2018 School leadership 0 3 Ratanakiri Krung FGD 24 Sep, 2018 School leadership 3 0 Ratanakiri Tampuan, FGD Khmer, Krung 24 Sep, 2018 Parents and 0 7 Ratanakiri Tampuan, FGD school support Krung committee 24 Sep, 2018 Children 6 1 Ratanakiri Khmer and FGD indigenous 25 Sep, 2018 POE leadership 2 4 Ratanakiri Khmer only FGD 25 Sep, 2018 DOE leadership 0 13 Ratanakiri Khmer only FGD 25 Sep, 2018 Discovery 6 37 Ratanakiri Khmer Krung Discovery workshop: World Café style workshop Tampuan discussions around outcomes, discussion on barriers
39 Independent Evaluation of the Multilingual Education National Action Plan in Cambodia
Date Group Female Male Province Language Data collection methods 28 Sep, 2018 Final discovery 14 8 Phnom Penh Mixed Discovery workshop, World Café style workshop discussions around outcomes and barriers 2 Oct, 2018 Special Education 1 1 Phnom Penh Khmer Group interview Department (SED) 2 Oct, 2018 ECE 1 2 Phnom Penh Khmer Group interview 4 Oct, 2018 MoEYS directors 1 1 Phnom Penh Khmer Group interview
TOTAL F/M 342 422 TOTAL 764
40 Independent Evaluation of the Multilingual Education National Action Plan in Cambodia
Annex 7. Location, type and size of schools visited The list of schools visited is shown in Table 5, indicating the type of school, its location and the language of instruction. The total number of primary schools visited was 28, of which 24 were MLE schools. In addition, 11 preschools were visited. Table 6 lists each primary school, indicating the number of grades and school size, and Table 7 lists all the preschools visited. Table 5. Summary of all schools visited for data collection
Number of Number of Language of MLE Communities Province District Commune MLE instruction primary without MLE preschools schools
O’Reang Dak Dam Bunong 1 1 O’Reang Sen Monorom Bunong 2 1 Pec Crieda Bu Sra Bunong 1 Mondulkiri Sen Romenea Bunong 1 2 Monorom Koh Nhek Sok San Bunong 1 Sesan Kbal Romeas Bunong 1
Stung Sesan Komphun Brao 1 Treng Sesan Samkhuoy Only Khmer 1 Siempang Santhepheap Kavet 2 2 Koun Mom Ta Ang Krueng 2 Koun Mom Teun Tampuan 1 O’Chum L’Ak Krueng 2 O’Chum Poy Krueng 1 1 Ratanakiri O’Chum O’Chum Only Khmer 1 Banlung Yeak Laom Tampuan 1 Borkeo Lung Khung Tampuan 1 Borkeo Seung Tampuan 2 1 Lumphat Pa Tang Tampuan 1 Snuol Ksem Bunong 1 1 Sambour Kbal Domrei Bunong 1 Kratie Sambour Kbal Domrei Only Khmer 2 Sambour Sandan Bunong 1 Chetr Borei Kantuot Bunong 1 1 TOTAL 14 22 6 24 11 4 districts communes languages
41 Independent Evaluation of the Multilingual Education National Action Plan in Cambodia
Table 6. Location, Indigenous language and size of primary schools visited for data collection Language of Number of Size of Province District Commune instruction grades school16 O’Reang Dak Dam Bunong 3 Large O’Reang Sen Monorom Bunong 3 Small O’Reang Sen Monorom Bunong 6 Small Mondulkiri Pec Crieda Bu Sra Bunong 6 Medium Sen Monorom Romenea Bunong 6 Medium Koh Nhek Sok San Bunong 6 Medium Sesan Kbal Romeas Bunong 3 Medium Sesan Komphun Brao 6 Small Stung Treng Sesan Samkhuoy Only Khmer Unknown Unknown Siempang Santhepheap Kavet 6 Medium Siempang Santhepheap Kavet 6 Medium Koun Mom Ta Ang Krueng 3 Medium Koun Mom Ta Ang Krueng 6 Medium Koun Mom Teun Tampuan 5 Small O’Chum L’Ak Krueng 6 Large O’Chum L’Ak Krueng 5 Small Ratanakiri O’Chum Poy Krueng 6 Large O’Chum O’Chum Only Khmer Unknown Unknown Borkeo Lung Khung Tampuan 6 Large Borkeo Seung Tampuan 6 Medium Borkeo Seung Tampuan 6 Medium Lumphat Pa Tang Tampuan 5 Medium
Snuol Ksem Bunong 4 Medium Sambour Kbal Domrei Bunong 4 Small Sambour Kbal Domrei Only Khmer Unknown Unknown Kratie Sambour Kbal Domrei Only Khmer Unknown Unknown Sambour Sandan Bunong 5 Large Chetr Borei Kantuot Bunong 5 Medium
TOTAL 14 districts 20 communes 6 languages 28 (24 MLE)
16 School size was categorized according to total student population: Small = 1-70 students; Medium = 70-130 students; Large = 130-300 students.
42 Independent Evaluation of the Multilingual Education National Action Plan in Cambodia
Table 7. Location, size and Indigenous language of preschools visited for data collection
Language of Number of District Commune Province instruction preschools
O’Reang Dak Dam Bunong 1 Mondulkiri O’Reang Sen Monorom Bunong 1 Sen Monorom Romenea Bunong 2 Stung Treng Siempang Santhepheap Kavet 2 O’Chum Poy Krueng 1 Ratanakiri Banlung Yeak Laom Tampuan 1 Borkeo Seung Tampuan 1 Kratie Snuol Ksem Bunong 1 Chetr Borei Kantout Bunong 1 TOTAL 8 districts 9 communes 4 languages 11 preschools
43 Independent Evaluation of the Multilingual Education National Action Plan in Cambodia
Annex 8. Extended presentation of costing for the MENAP Special Education Department budget for Mondulkiri, Ratanakiri, Stung Treng and Kratie CARE expenditures for implementation of the MENAP Table 8 shows the project expenditures by CARE for the purpose of MLE during the MENAP period, while Figure 1 shows the breakdown of the expenditures as percentages of the total amount spent.
Table 8. CARE expenditure on MENAP implementation activities Project name Amount Per cent total expensed (US$) EEM-ANCP 282,007 41.92 CCOSC 95,940 14.26 MLE teacher training 94,637 14.07 PCTFI fund 58,932 8.76 Technical MLE assistance 50,422 7.49 Bending Bamboo 33,205 4.94 MoEYS technical assistance on CFS for EM 24,265 3.61 ECE Department technical assistance 17,477 2.60 Consultancy on MLE 15,892 2.35 Total 672,777 100
Figure 1. Breakdown of CARE expenditure on MENAP implementation activities
2.60% 2.35% 3.61% 4.94%
7.49%
41.92%
8.76%
14.07%
14.26% EEM-ANCP CCOSC MLE Teacher Training PCTFI Fund Technical MLE Assistance Bending Bamboo MoEYS Technical Assistance on CFS for EM ECED Technical Assistance
44 Independent Evaluation of the Multilingual Education National Action Plan in Cambodia
UNICEF expenditure for implementation of the MENAP Overall UNICEF spending on MENAP activities The following charts describe UNICEF’s spending on MLE-related programming and capacity development during the 2016-2018 UNICEF Cambodia Country Programme.
• The largest portion of UNICEF funds (US$ 265,600 / 35.5 per cent) was spent on teacher training in the four north-eastern provinces • Second to investment in teacher training, funds were spent on education scholarships for Indigenous students (US$ 258,907 / 34.6 per cent) • Monitoring (US$ 80,984 / 10.8 per cent) and capacity development (US$ 77,472 / 10.3 per cent) for MENAP implementation were the third and fourth largest areas of UNICEF investment • Incentives paid to MLE teachers (US$ 34.451 / 4.6 per cent), partnerships with national and international NGOs (US$ 23,171 / 3.1 per cent) and on-going development of the new MENAP (US$ 8,011 / 1.1 per cent) represented minor spending categories for UNICEF’s MLE programme.
These expenditures are represented graphically in Figure 2. More detailed breakouts within the various categories of support in Table 9 are presented in the remaining tables and figures in this section. Table 9. UNICEF spending on MENAP activities 2016-2018 Category of support Funds spent Per cent total Teacher training 265,600 35.5 Education scholarships for Indigenous students 258,908 34.6 Monitoring of MLE 80,984 10.8 Capacity development to officials at sub-national level 77,472 10.3 Incentives for MLE teachers 34,451 4.6 Partnering with NGOs 23,171 3.1 Revise MENAP 8,022 1.1 Total 748,608 100
45 Independent Evaluation of the Multilingual Education National Action Plan in Cambodia
Figure 2. Overall breakdown of UNICEF MLE spending
3.10% 1.07% 4.60%
10.35%
35.48%
10.82%
34.59% Teacher Trainings Scholarships for Indigenous Students Monitoring Capacity Development to Officials at Sub-National Incentives for MLE Teachers Partnering with NGOs Revise MENAP
Disaggregated costs
UNICEF spending on teacher training As Table 10 and Figure 3 show, most UNICEF line item teacher training funds were directed towards teacher training programmes. More specifically, these funds either went to the full MLE teacher training course to new teachers (US$ 95,392 / 35.9 per cent) or the fast-track short training course (US$ 89,496 / 33.7 per cent). The total amount spent on teacher training (US$ 265,600.41), divided by the number of MLE teachers trained (n=247, including preschool teachers), yields an estimated US$ 1,038.87 for each MLE teacher. Table 10. UNICEF expenditure on teacher training
Teacher training category Cash expended (US$) Per cent total MLE teacher training on MLE methodologies 95,392 35.92 MLE fast training for state teachers 89,496 33.70 Support MLE teacher training and expansion 58,511 22.02 Support to MoEYS to enhance teacher training 6,943 2.61 Refresher training in MLE teaching 6,036 2.27 Children’s enrolment in MLE 5,170 1.95 Support teacher training and expansion of MLE ECE 4,054 1.53 Total 265,602 100
46 Independent Evaluation of the Multilingual Education National Action Plan in Cambodia
Figure 3. UNICEF expenditures on MLE teacher training
1.95% 2.27% 1.53% 2.61%
35.92% 22.02%
33.70%
MLE teacher training on MLE methodologies MLE fast training for state teachers Support MLE teacher training and expansion Support to MoEYS to enhance teacher training Refresher training in MLE teaching Children enrolment in MLE Support teacher training & expansion of MLE ECE
UNICEF spending on MLE monitoring UNICEF’s financial support to MLE monitoring during the past three years largely centred around providing sub-national support to provincial level monitoring (see Table 11 and Figure 4). UNICEF also provided national level monitoring support to MoEYS. Table 11. UNICEF expenditures on MENAP monitoring and evaluation
Monitoring category Funds spent Per cent total Sub-national monitoring 34,494 42.6 NE sub-national monitoring 21,288 26.3 Intensive monitoring for timely tech 7,462 9.2 Primary/ECE MLE monitoring (national level) 7,017 8.6 National level monitoring 5,723 7.1 Primary MLE monitoring (national level) 5,000 6.2 Total 80,984 100
47 Independent Evaluation of the Multilingual Education National Action Plan in Cambodia
Figure 4. UNICEF support of MENAP monitoring and evaluation
6.17% 7.07%
Sub-national monitoring NE sub-national monitoring 8.65% Intensive monitoring for timely tech 42.59% Primary/ECE MLE monitoring (national-level) National level monitoring Primary MLE monitoring (national-level) 9.21%
26.29%
UNICEF spending on capacity development UNICEF spending on capacity development is summarized in Table 12. These data are also depicted graphically in Figure 5.
Table 12. UNICEF expenditure on MLE capacity development
Capacity development category Funds spent Per cent total MoEYS officials implementation capacity 52,948 68.3 development MoEYS training in MLE monitoring and 11,419 14.7 evaluation MLE capacity development at sub-national 10,000 12.9 level (CDPF 2017) MoEYS MLE capacity development 1,652 2.2 Provincial capacity development 1,453 1.9 Total 77,472 100
48 Independent Evaluation of the Multilingual Education National Action Plan in Cambodia
Figure 5. UNICEF MLE capacity development spending
2.2% 1.88%
12.92%
14.74%
68.34%
MoEYS officials implementation CD MoEYS training in MLE M&E MLE CD at sub-national levels (CDPF 2017) MoEYS MLE CD Provincial Capacity Development
UNICEF spending on scholarships for ethnic minority students Beginning in the 2016/2017 school year, UNICEF began providing additional scholarships to Indigenous children who were already receiving the government educational scholarship. Table 13 shows the total amount for the top-up scholarship programme for Indigenous children, disaggregated by lower and upper secondary school. Table 14 shows the number of scholarships by province and school year disbursed. While these are not scholarships to attend MLE schools, the scholarships do represent an extension of UNICEF’s support to Indigenous education in northeast Cambodia. Table 13. Breakdown of UNICEF spending on scholarships for ethnic minority students
Scholarship classification Funds spent Per cent total Lower secondary scholarships 43,496 16.8 Upper secondary scholarships 215,412 83.2 Total 258,908 100
49 Independent Evaluation of the Multilingual Education National Action Plan in Cambodia
Table 14. Breakdown of UNICEF scholarships by province and community
Province by No. of Indigenous communities receiving school year scholarships scholarships Kratie 41 Kuy, Bunong, Khorl, Stieng Mondulkiri 105 Bunong Ratanakiri 220 Kavet, Jarai, Tampuan, Kreung, Brao, Kachok Stung Treng 63 Kuy, Kavet, Tampuan, Jarai 2016 / 2017 total 429 Kratie 67 Kuy, Bunong, Kraol, Stieng Mondulkiri 210 Bunong Ratanakiri 401 Kavet, Jarai, Tampuan, Kreung, Brao, Kachok Stung Treng 109 Kuy, Kavet, Tampuan, Jarai 2017 / 2018 total 787 Total 1,216
50 Independent Evaluation of the Multilingual Education National Action Plan in Cambodia
Annex 9. Informed consent protocol
Multilingual Education National Action Plan evaluation
Consent form A team has been commissioned by UNICEF to evaluate the government’s MLE initiative. The division of the government is the Ministry of Education, Youth and Sport. The initiative is called the Multilingual Education National Action Plan. It is a five-year plan from 2015 to 2018. To find out how the initiative is working, our team would like to meet and talk with people like you.
The team doing the evaluation is led by Jessica Ball and Mariam Smith. We are independent of the government and have been asked by UNICEF to do this evaluation. We have a number of Cambodian team members working with us.
We seek to learn about various perspectives from various contexts related to MLE. Your participation in interviews, in FGDs, and by telling us and others your stories about the changes you have experienced in relation to education is very important to us. We are asking if you would share your experiences and allow us to take notes and use your experiences, along with experiences of many other people we will be talking with – to assess the government’s MLE initiative.
We may be using what you share with us in workshops with other stakeholders and some information and stories may be included in reports for the purpose of the evaluation.
If you are happy to share your experiences with us, and for us to use your stories, please complete and sign this form.
I agree to participate, and I agree to the use of my information, written stories, and pictures or films of me for any purpose, and in any way, that assists the evaluation of the MENAP and supports inclusive education for children in Cambodia.
I know that I have a choice whether to participate or not to participate. I am choosing freely to participate. I know I can refuse or withdraw my participation at any time.
I agree that my name can be used in connection with the information I give.
I do NOT agree that my name can be used in connection with the information I give.
NOTES, CONDITIONS, REQUESTS For example: name to be changed and a suggested substitute name
I confirm that the above agreements were made with my knowledge and consent.
My name Age (if under 18)
Address or contact details
Signature Date
If the person is under 18 years of age:
51 Independent Evaluation of the Multilingual Education National Action Plan in Cambodia
I confirm that I am the legal guardian of the child named above and therefore may grant permission for my child to participate and this subject release on behalf of the child:
My name ______
Address or contact details ______
Signature ______
Date______
………………………………………………………………………………………………………
Facilitator/interviewer: ______
Date: ______
Location: ______
52 Independent Evaluation of the Multilingual Education National Action Plan in Cambodia
Annex 10. Evaluation design matrix Table 15. Evaluation design matrix of the Cambodia Multilingual Education National Action Plan Evaluation question Evaluation sub-questions Information required Source of information themes
Relevance of the To what extent has MENAP addressed Analysis of MLE access and quality • MENAP documents (all official extent to which the the needs of ethnic minority girls and assessment data. Qualitative data policies and implementation MENAP is suited to boys in terms of access to quality and from MLE students, teachers and directives) ensure MLE to children relevant education? administrators. • Baseline, monitoring and end-line (rights holders) in the data from relevant government four north-eastern departments and ministries (MoEYS) provinces • Needs assessments and or situational analyses on the educational status of ethnic minorities in north-eastern provinces • Key informant interviews and focus groups with MLE students (current and past) and school faculty
Relevance To what extent are the objectives of the Primary data attained through • Key informant interviews and focus MENAP still valid? Were the MENAP outcome harvesting groups with all relevant stakeholders objectives set realistically to be • Desk review of quantitative MLE achieved in five years? data
Relevance Are the activities and strategies of the Analysis of MENAP actual • Desk review of MENAP MENAP consistent with its overall implementation compared to its implementation guidelines objectives and the attainment of the action plan • Qualitative data from key informant intended impacts and effects? interviews and FGDs
Relevance How relevant is UNICEF’s support for Quantitative data comparing mean • MoEYS Primary, ECE and SED data MENAP in building the capacity of enrolment and achievement rates of on MLE student enrolment and national and sub-national officials to achievement and teacher training,
53 Independent Evaluation of the Multilingual Education National Action Plan in Cambodia
Evaluation question Evaluation sub-questions Information required Source of information themes
manage and monitor MLE, and scale MLE students/schools with non-MLE and CARE checklist for assessment up MLE provision? students/schools of learning at MLE schools (if Qualitative data gauging results- available and out of the prototyping based management capacity of phase) stakeholders • Key informant interviews with implementing stakeholders
Relevance Has the MENAP been designed and Desk review of relevant MoEYS and MoEYS MENAP implementation implemented taking into consideration other social policies related to MLE policies and guidelines the Education Strategic Plan, the decentralization and deconcentration Outcome harvesting from field visits process, the National Strategic in NE provinces Development Plan, and other relevant strategies?
Effectiveness of the To what extent have the expected Comparison of stated enrolment • Baseline data and needs support provided by outcomes of MENAP been achieved or goals to current total enrolment assessments from provinces MoEYS, UNICEF and are likely to be achieved? figures • Workshop materials, presentation other implementing materials, national MLE policies, partners in achieving Outcome harvesting from field visits practices and systems its outcomes in NE provinces
Effectiveness What are the results of MENAP in Enrolment information, teacher Quantitative enrolment data from MLE terms of: staffing information schools from 2014-2018 a. Number of MLE preschools and primary schools b. Number of students in MLE preschools and primary schools c. Number of teachers in MLE preschools and primary schools?
54 Independent Evaluation of the Multilingual Education National Action Plan in Cambodia
Evaluation question Evaluation sub-questions Information required Source of information themes
Effectiveness What were the major factors influencing Outcome harvesting from field visits FGDs and key informant interviews with the achievement or non-achievement of all relevant stakeholders MENAP (incl. enabling factors, barriers Barrier analysis for any bottlenecks and bottlenecks)?
Effectiveness Are results achieved similar in all four Desk review of MLE quantitative Quantitative enrolment data on MLE provinces? Which provinces perform data schools from 2014-2018 better/worse and for what reason? Outcome harvesting from field visits FGDs and key informant interviews with in NE provinces all relevant stakeholders
Effectiveness How satisfied have the parents and Outcome harvesting from field visits FGDs and key informant interviews with their children been with MLE services? in NE provinces relevant parents and children (most How do local authorities value the significant change) quality of the MENAP? Is this different between the four provinces?
Efficiency of the How well has MENAP been managed Analysis of MENAP budgets from Budgets from all national partners and management of in terms of the technical and financial CARE, UNICEF and MoEYS NGO implementing partners pertaining MENAP to ensure resources provided to teachers and to MENAP-specific activities timely and efficient use school operations? Analysis of the evolution of MENAP of resources policies, practices and systems, Breakdown of UNICEF and CARE including UNICEF’s role in financial and technical support to each leveraging resources and MoEYS counterpart partnerships to do so MENAP budgets, implementation plans Comparison of stated enrolment and any relevant documents from goals to historical total enrolment MoEYS figures
55 Independent Evaluation of the Multilingual Education National Action Plan in Cambodia
Evaluation question Evaluation sub-questions Information required Source of information themes
Capacity analysis of MoEYS MENAP implementation functions, including analysis of UNICEF and implementing partners’ support
Efficiency Has MENAP been implemented in the Cost-benefit analysis based on MLE Budgets from all national partners and most cost-effective way compared to academic achievement and NGO implementing partners pertaining alternative approaches? enrolment metrics to MENAP-specific activities
Quantitative data on MLE schools from 2014-2018
Efficiency In what ways, and to what extent, do Comparison of stated enrolment Quantitative enrolment and the costs incurred to implement goals to current total enrolment achievement data on MLE schools from MENAP justify the results achieved on figures 2014-2018 quality and access to education? Cost-benefit analysis based on MLE Budgets from all national partners and enrolment metrics NGO implementing partners pertaining to MENAP-specific activities
Efficiency Did (will) the MENAP implementation Desk review of disaggregated Quantitative enrolment and reach its target? Within the timeframe enrolment and academic achievement data on MLE schools from set in the plan? achievement data, and MENAP 2014-2018 guidelines Key informant interviews, FGDs, Outcomes harvested from field visits outcome harvesting and most significant change in communities
Impact resulting from Is there evidence of the extent to which Quantitative data analysis of MLE Quantitative enrolment and the implementation of the MENAP may have contributed to enrolments achievement data on MLE schools from MENAP (positive and ensuring Indigenous children have 2014-2018 negative changes, Desk review of MENAP policies
56 Independent Evaluation of the Multilingual Education National Action Plan in Cambodia
Evaluation question Evaluation sub-questions Information required Source of information themes intended and access to equitable, inclusive, quality FGDs, key informant interviews, unintended) in the four and relevant education? Qualitative analysis of most outcome harvesting and most significant provinces significant change, key informant change with all relevant stakeholders interviews and FGDs
Outcomes harvested from field visits
Impact In what ways and to what extent has Capacity analysis of MoEYS FGDs, key informant interviews, the MENAP changed the capacities of MENAP implementation functions, outcome harvesting, and most national and sub-national education including analysis of UNICEF and significant change with all relevant officials to manage and monitor MLE implementing partners’ support stakeholders implementation? Outcomes harvested from field visits
Impact What difference has the MENAP made Qualitative analysis of most FGDs, outcome harvesting and most in terms of promoting demand for significant change key informant significant change with all relevant quality MLE among commune councils, interviews and FGDs stakeholders school support committees, parents and children? Outcomes harvested from field visits
Impact Are there any unintended results either Qualitative analysis of most FGDs, key informant interviews, positive or negative associated with the significant change, key informant outcome harvesting and most significant implementation of the MENAP? interviews and FGDs change with all relevant stakeholders
Outcomes harvested from field visits
Sustainability of the What are the key barriers and Situational analysis of MLE Evaluation team observations benefits of MENAP bottlenecks towards achieving implementation sustainability of the MENAP? FGDs, key informant interviews, outcome harvesting and most significant change with all relevant stakeholders
57 Independent Evaluation of the Multilingual Education National Action Plan in Cambodia
Evaluation question Evaluation sub-questions Information required Source of information themes
Sustainability To what extent can components of the Budgets from all national partners Budgets from all national partners and new MENAP be integrated and and NGO implementing partners NGO implementing partners pertaining implemented under the full ownership pertaining to MENAP-specific to MENAP-specific activities of the government, both technically and activities financially? Key informant interviews with national Capacity analysis of national and and sub-national leaders implicated in sub-national implementation transitioning the MENAP to full capabilities government ownership
Sustainability To what extent has UNICEF’s support Capacity analysis of MLE FGDs, key informant interviews and contributed to enhancing sustainability implementers (MoEYS, POE/DOE, outcome harvesting with national and of the MENAP? schools) sub-national implementers
Equity issues To what extent are age and gender Desk review of disaggregated FGDs, key informant interviews with disaggregated data collected and enrolment and academic MENAP implementation, and monitoring (Cross-cutting monitored? achievement data and evaluation partners considerations)
In what ways and to what extent has Desk review of MENAP guidelines Government policies and guidelines Equity issues the MENAP integrated an equity-based approach into the design and Qualitative analysis of most All gender-disaggregated quantitative implementation of its services? significant change, key informant MLE data interviews, FGDs and outcomes harvested FGD, outcome harvesting, most significant change with Indigenous community members (rights-holders)
Equity issues Does the MENAP actively contribute to Qualitative analysis of most Most significant change stories from the promotion of children’s and significant change key informant indigenous community members (rights- women’s rights, especially the most interviews and FGDs holders vulnerable? Outcomes harvested from field visits
58 Independent Evaluation of the Multilingual Education National Action Plan in Cambodia
Evaluation question Evaluation sub-questions Information required Source of information themes
Equity issues To what extent and how does the Desk review of MENAP policies All relevant primary and secondary data MENAP ensure an equity focus? collected from the field and during the Qualitative analysis of most desk review significant change key informant interviews and FGDs
Outcomes harvested from field visits
59 Independent Evaluation of the Multilingual Education National Action Plan in Cambodia
Annex 11. Extended description of evaluation methodology This annex details activities that will be carried out in the provinces, including outcome harvesting, most significant change, FGDs, DOE and POE meetings and individual interviews. The case studies will occur in four of the five provinces that are using MLE supported by the MENAP. They will involve visits to two preschools and two schools serving predominantly Indigenous children, and one preschool and one school serving predominantly Indigenous children, that are not using MLE. Key informants for the case studies will include local level partners, stakeholders and beneficiaries, including children, parents and teachers. The consultant will meet with these informants in various configurations. A mixed methodology will be used, involving mainly FGDs (also called discovery workshops) and individual interviews. Some survey type questionnaires will also be used. Mixed methods will be used to investigate to what extent local authorities, teachers, school support committees, parents and children are experiencing MLE in teaching and learning at the preschool and lower primary levels, and the outcomes of MLE efforts to date. Where there is no MLE teaching, mixed methods will investigate perceived outcomes of teaching and learning for Indigenous children who are 3 to 8 years old, and local views on whether MLE would enhance teaching and learning outcomes. An outcome harvesting methodology will be used initially (simplified to match available time for the field visits). Subsequently, structured interviews with open-ended questions and questions answered with rating scales will be held with selected key informants in order to gather information that can be triangulated with outcomes identified during the outcome harvesting process. Interviews will include (but not be limited to) teachers, leaders of school support committees, district education officers and provincial education officers. Case studies will primarily be based on the informants’ inputs, and on a review of documents from which the consultant will cull statements about outcomes of the MENAP to date. In this way, the case studies will produce findings that will be used to respond to the driving questions for this evaluation, including: a) The extent to which outcomes identified in reports of the MENAP to date are confirmed by local level key informants b) The value or significance of these outcomes as assessed by local key informants c) The extent to which outcomes can be attributed directly and/or indirectly to the implementation of specific MENAP-supported strategies and activities d) Identification and assessment of possible other contributions to confirm outcomes from other sources e) Identification of barriers, bottlenecks and enabling factors that have affected the strength and impact (positive/negative) of identified outcomes f) Identification of possible non-planned outcomes resulting from MENAP-related strategies and activities, including positive and negative g) Identification of the specific actions and outputs of the MENAP in that location that have contributed to the outcomes identified.
Specific case study characteristics The foci of the case studies are outcomes/actions culled from previous reports of the MENAP, as well as those identified by reference group members. The 16 case studies will each consist of approximately one day of investigation (desk study, interviews, and focus-group meetings) in four provincial capitals, four districts in the same four provinces, and approximately two to three villages in each province. The prototypical field visit in this evaluation will proceed as follows: a) Introductory meeting with directors of the POE and DOE and evaluator(s) to explain programme, planning and expectations (0.5 hour)
60 Independent Evaluation of the Multilingual Education National Action Plan in Cambodia
b) Individual key informant interviews (4-6 interviews), with key staff, key implementers and key stakeholders (maximum of 3 hours, interviews maximum of 0.5 hour). When possible, the list of key informants per province/district will be prepared prior to the visit. c) Identification and collection of core documents at the level of institution (POE, DOE, school). Because documents will be translated by the team, it will be important that they are selected strategically as ones that the stakeholders actually use as guidance for and/or records of their work (total 1 hour). d) Focus group meeting with POE/DOE (1.5 hours) e) Focus group meeting with school directors (district level) (1 hour). This will require support from the DOE to bring the school directors together. f) Focus-group meeting with teachers (if available) (1 hour) g) Focus-group meeting with school support committee members (1 hour) h) Surveys with POEs, DOEs, school directors (done at the end of the respective focus group meetings) i) Invitations to discovery workshop for summary presentation and exploratory discussion at POE/DOE/institution to clarify, confirm, correct, elaborate and explain outcomes harvested and stories (2 hours). Data collection activities for each case study will consist of approximately 10 hours of contact time (excluding travel). The following diagrams, in Figures 6, 7 and 8, show the sequence of linked activities for the cumulative and iterative data collection at village, district and provincial levels.
Figure 6. Prototypical community field visit
61 Independent Evaluation of the Multilingual Education National Action Plan in Cambodia
Figure 7. Evaluation methodology for field visits in each province
62 Independent Evaluation of the Multilingual Education National Action Plan in Cambodia
Figure 8. Discovery workshop in Phnom Penh
Discovery workshop in Phnom Penh
Detailed description of outcome harvesting activities Further details on the method of outcome harvesting are provided here. The method starts with a desk review of documents relating to the MENAP, followed by outcome harvesting interactions and the generation of stories about change using the technique of most significant change.
Desk review Reviewing findings and conclusions from secondary sources is one of the early steps in the outcome harvesting method described subsequently. The consultants will be mapping out the positive and negative outcomes defined as behavioural changes found in the documentation. These will be shared with the primary users and other informants of the evaluation and continually edited to best encompass the various perspectives that will be present in those outcomes. It is common however that documentation for monitoring and evaluation purposes includes very little mention of outcomes, especially not in concrete terms, as qualitative descriptions of behavioural change. These concrete, qualitative descriptions focused on behavioural change are the target of the outcome harvesting methodology. The desk review will enable the consultant to identify stakeholders and others involved in the implementation of the MENAP. This will enable the consultant to adjust activities and questions for subsequent interviews, FGDs and workshops.
Outcome harvesting Outcome harvesting is a methodology designed to support learning and discovery in contexts of complex social change. The tool grew out of and is informed by outcome mapping, while also being inspired by
63 Independent Evaluation of the Multilingual Education National Action Plan in Cambodia utilization focused evaluation (Michael Q. Patton). In outcome harvesting, through the participation of various change agents, the consultant will harvest (or hunt and gather) outcomes, as changes in behaviour (actions, relationships, policies and practices) among social actors.17 This particular definition of outcomes is a strong defining boundary of the tool. Interest in other types of outcomes will involve other methods, hence the use of a multi-method approach in this evaluation. Another defining feature of outcome harvesting is the choice to determine contribution rather than attribution. The focus on contribution stems from the philosophy that in social change processes, evaluators are not able to establish a direct correlation to their contribution. Instead, it is likely there may be many actors contributing to change, and behavioural changes are not possible to prove. The outcome harvesting tool therefore allows the evaluators/harvesters to focus on the contributions of the change agents that are deemed plausible. The change agents for this evaluation include UNICEF, MoEYS, CARE International, and possibly others. Each will be invited to express outcomes of which they are aware.
Most significant change Most significant change is similarly a tool designed to support learning in contexts of complex social change. Documenting most significant change entails not only the collection of stories of change, but seeks to understand the reasons for the change, and includes a selection process that make values and perspectives explicit.18 For the purposes of the MENAP, the collection of stories will primarily focus on the community level, where each group of stakeholders will be invited to share their stories. The communities will, in a combined synthesis session, be able to choose the most significant change story from their community to be shared at the provincial discovery workshops. The selected stories from each of the visited communities will then be shared at the provincial discovery workshop, giving four types of stakeholders (community member, teacher, POE staff and other stakeholders) present in the workshop the chance to vote on the most significant change story regarding the MENAP. They will be able to tag that choice with the reason they chose it. A few most significant change stories may be collected at the provincial level, but this will not be the main focus of the most significant change process. At the national level, during FGDs and in interviews with stakeholders, these groups will be asked to choose and discuss the most significant change stories generated in communities and selected at the discovery workshops.
Sequence of primary data collection activities For outcome harvesting activities, the consultant will be joined by local translators and facilitators, who will not only serve as linguistic and cultural translators, but will also lead FGDs in such a way that potential outcomes can be harvested and substantiated. The six steps described below are iterative processes. They may not follow the sequence shown here, and they may overlap. The drafting of outcomes is likely to continue even while substantiating outcomes in consultation with the various informants who participate in interviews and FGDs. Each step is essential for the harvest and requires the participation of change agents who are typically those people closest to the change. Their participation enables meaning making, increases reliability and validity, and enables
17 A set of 10 principles for outcome harvesting are also part of the tool. They are: ensure usefulness throughout the evaluation; harvest social change outcomes; formulate an outcome as an observable change; establish plausible influence/contribution; facilitate the identification and formulation of outcomes; nurture appropriate participation (engage, interact, assist); rigorously aim for credible-enough outcomes; strive for less because it will be more; reveal patterns of social change (not depth); learn experientially. 18 For more information on the most significant change tool, see: https://europa.eu/capacity4dev/iesf/document/‘most-significant-change’-technique-davies-dart-2005.
64 Independent Evaluation of the Multilingual Education National Action Plan in Cambodia the usefulness of evaluation results for the primary users. Each of the outcome harvesting steps fit well within the processes that have already been outlined in the ToR for the evaluation.19 Step 1: Design the harvest UNICEF has informed the consultants of the evaluation purpose and who the primary users are. Finalizing the work plan and methodology and choosing the communities and other stakeholders/key informants for data collection are also part of this step. Step 2: Review documentation and draft outcomes During the review of secondary data sources, the consultant will draft outcome statements culled from previous documentation provided by UNICEF. Step 3: Engaging with informants The draft outcomes will be given to relevant stakeholders who will be encouraged to add additional outcomes not necessarily expressed or captured in previous reports. A main opportunity to do this will be in individual and group meetings held in Phnom Penh on August 9 and 10, before going to communities. Step 4: Substantiation During the substantiation stage, the consultant will collect data confirming and refining outcomes that have been identified in Steps 2 and 3. For this evaluation, the consultant will use FGDs as the main forum for gathering and substantiating outcomes. These discussions will be facilitated in an informal manner so as to provide a context that will enable relevant, authentic and credible interaction in the context of Indigenous peoples. Note that while the consultant has formulated questions for various stakeholder groups, these questions might not be asked directly in a semi-structured or linear way. Rather, it is hoped that many of these questions will be answered through other kinds of interaction, including formal and informal gatherings and individual conversations. After introductions to the purpose, the facilitators will ask participants to tell them about the situation and will find appropriate ways to dig further into understanding their perspectives, using the questions as a mental guide. Conversations, observations and interviews will be used as appropriate in each context. The following form will be used to collect the data:
Figure 9. Outcome harvesting format
Outcome Outcome description (Who did Contribution Significance to Significance title what, how, when, and with whom from change individuals/ to reference DIFFERENTLY?) These can be agents (MENAP groups reporting group changes in behaviour, practices, duty bearers) the change policies and relationships among groups, organizations, individuals, and institutions.
The essential parts of the outcome harvesting tool are outcomes and contribution. The outcome title can help to organize the data. The significance column is generally recognized by outcome harvesting practitioners as a useful added element in outcome harvesting, as it supports the exploration of values and assumptions. The consultant will likely be able to gain the stated significance, especially as the tool will be used in combination with most significant change.
19 A one-page summary of outcome harvesting can be accessed at: http://outcomeharvesting.net/outcome- harvesting-one-%20pager/
65 Independent Evaluation of the Multilingual Education National Action Plan in Cambodia
Step 5: Analysis and interpretation The consultant will analyse and interpret primary data in an iterative process, engaging stakeholders along the way. There may be various categorizations by stakeholders that will provide insight into enabling conditions, implementation processes, barriers and bottlenecks, as well as unintended outcomes. The consultant will work with several lenses in analysing the outcomes. Some lenses which have been identified by the consultants as important and which are identified in the ToR are:
• Gender, women’s rights • Child rights • Equity • Inclusive education • Quality education • Cultural relevance • Increased demand for MLE by communities • Acceptance of MENAP among authorities and duty bearers • Cost • Capacity for implementation • Negative outcomes • Support by national authorities • Support by sub-national authorities • Support by UNICEF, and by other implementing partners • Barriers • Extent of ownership by the government (technically and financially) • Ideas/hope for the future • How it matches/does not match the theory of change.
Step 6: Supporting the use of findings Findings will be interpreted in terms of lessons learned, promising practices, innovations and recommendations. These will be communicated through a final report. UNICEF will develop and execute a dissemination plan separate from the evaluation.
Community engagement Substantiation of the outcomes that have been drafted will take place in FGDs, through observations and interviews, as appropriate, for each context in the selected communities of each province. During this time, it is expected that the process will harvest positive and negative outcomes that have not yet been mentioned by the change agents/informants during the drafting process. Facilitators will use drawings and encourage participants to write, draw or use physical objects to represent significant points in their discussions. After a brief introduction to one another and the purpose of the evaluation, the three main groups of FGDs will meet in parallel, allowing for each group of stakeholders to bring forward their perspectives in smaller forums initially. Two local translators and facilitators will be crucial in facilitating discussions effectively with the groups of children and guardians. They will not be as crucial in discussions with service providers, who are often competent in Khmer. Service providers in the community FGDs among service providers in the schools will include all available teachers, the principal and preschool teachers. Outcomes, their contribution and their significance will be explored around the topics of community engagement in education, teaching methodology, support from supervisors in the education system, student achievements and similar themes. The following questions will be asked in connection with drawing up events on a timeline:
66 Independent Evaluation of the Multilingual Education National Action Plan in Cambodia
1. When did you start using MLE in your school? What were some important steps in this process? What was it like before you started with MLE? What are you doing differently now and when did that start? (Why has this changed? Dig for descriptions of methodology, community engagement and use of resources). 2. Of all of these events, what was the most significant change and why? 3. What are some ways that relationships have changed with the POE and MoEYS? How has your relationship changed with people in your community? How have things changed in your classroom? Why is that significant? 4. How are students doing things differently/achieving now? The facilitator will lead a discussion with the group, identifying if there are any most significant change stories that could be insightful coming from this discussion. These may be told naturally during the FGD but may also require time with the storyteller after the FGD or the synthesis session. See Section 3 for more information about how we plan to use most significant change. After the FGDs with service providers, they will be invited to participate in the joint synthesis, sharing the timelines and commenting on each other’s conclusions. Children in school The FGDs with children attending schools will include eight people (four girls, four boys). They will represent all three grades and will include preschool children if one exists. If the primary grades do not yet have MLE, they will still include Grades 1 to 3, as children may still have been in MLE preschools or have been affected by MLE in some way. The facilitators will make the participation meaningful for all children involved. Outcomes, their contribution and their significance will be explored around the topics of the quality of education (methodology), perceptions around classroom experiences and learning, relevance and student achievements. The following questions will be asked in connection to drawing up events on a timeline: 1. When did you first start school? When did you first start learning how to read and write in your mother tongue? What was it like before you started to use your mother tongue in school? Is the teacher using your mother tongue in the classroom now? When did that start? 2. What kinds of things are you doing in the classroom now that you could not do in the past? (When and how did that happen)? 3. What do you like most about school? When did that happen? (Dig for descriptions of quality education, participation, significance of the MLE programme). 4. What are you able to do now that you are proud of? How did you learn this? 5. Tell me about your Khmer language abilities? Can you share some things you can do in Khmer? 6. Of all of these events, what was the most significant change and why? 7. How does your education help you and your family in your daily life?
The engagement of children around written text will also be explored. The facilitator will then lead a discussion with the group, identifying if there are any most significant change stories that could be insightful coming out of this discussion. These may be told naturally during the FGD but may also require time with the storyteller after the FGD or the synthesis session. See Section 3 for more information about how we plan to use most significant change. After the FGDs with school children, they will be invited to participate in the joint synthesis, sharing the timelines and commenting on each other’s conclusions. Facilitators of the synthesis session will ensure that the voices and perceptions of children are heard. Guardians/village leaders The FGDs will include parents and other guardians (four men, four women) whose children are currently in school, as well as one representative from the community leadership. Outcomes, their contribution and their significance will be explored around the themes of community engagement (men and women),
67 Independent Evaluation of the Multilingual Education National Action Plan in Cambodia relevant education, perceptions of student achievement, and interest in MLE. The following questions will be asked in connection with drawing up events on a timeline: 1. Do you remember when MLE was starting to be implemented in your village? What was your interest in MLE then? What is your interest in MLE now? How has the interest in MLE changed in your community? What are your hopes for the future? 2. What do you notice is different about the education that your children have today compared to in the past? How are the children able to use their knowledge from school in the community? How are they able to use the knowledge and resources from the community in their school learning? (Probe for cultural and traditional knowledge as appropriate). The facilitator will then lead a discussion with the group, identifying if there are any most significant change stories that could be insightful coming out of this discussion. These may be told naturally during the FGD but may also require time with the storyteller after the FGD or the synthesis session. See Section 3 for more information about how we plan to use most significant change. After the FGDs with school children, they will be invited to participate in the joint synthesis, sharing the timelines and commenting on each other’s conclusions. Facilitators of the synthesis session will ensure that the voices and perceptions of children are heard. The synthesis discussions will start with each FGD briefly sharing their timeline/visual representation of their discussion. The main questions after this will be: 1. Among the most significant change stories that were collected, which one do you feel was most significant for the quality of the children’s education in your village? Why? 2. What have been the most important barriers to positive change? 3. What are your hopes for the future? The data collected in the selected communities will later be compared with perspectives heard from the other communities during the discovery workshops in the province. These will be described further under the next heading. If a DOE is in the vicinity, interviews will be held with it to hear perspectives, using the same questions as the questions prepared for the service provider focus group.
Provincial level Substantiation of the outcomes that have been drafted will take place in three different settings:
• FGDs specifically with the POE • Discovery workshops with a wide variety of stakeholders. The discovery workshops are essentially FGDs, but the term discovery workshop implies an inductive process of revealing experiences, unpacking global statements about what ‘works’ or is ‘good’ to get at a more differentiated accounting of experiences and changes, and uncovering underlying processes that are driving change/no change. • Key informant interviews with other stakeholders as appropriate. During these occasions, we expect to harvest positive and negative outcomes that have not yet been mentioned by the change agents/informants during the drafting process.
FGD with the POE The FGDs will include the director, Primary Education Office, monitoring people, MLE teacher trainer, and those responsible for preschools. After a brief introduction to one another and the purpose of the evaluation, the facilitator will lead the discussions, exploring outcomes, their contribution and their significance on the themes of student achievements, community engagement in education, teaching methodology, and support from the POE and other stakeholders. The following questions will be asked in connection with drawing up events on a timeline:
68 Independent Evaluation of the Multilingual Education National Action Plan in Cambodia
1. When did you start using MLE in your province? What have been some important steps in this process? What was it like before you started with MLE? What are you doing differently now and when did that start? (Why has this changed? Dig for descriptions of practices within the office, methodology, training, community engagement, use of resources, and collection and disaggregation of data). Of all of these events, what was the most significant change and why? 2. How are students doing things differently/achieving now? 3. What are some ways that relationships have changed with the communities, the DOE or MoEYS? Are there other relations that have changed? How have things changed in your office? Why is that significant? 4. What have been the major obstacles to change?
The facilitator will then lead a discussion with the group, identifying if there are any most significant change stories that could be insightful coming out of this discussion. These may be told naturally during the FGD but may also require time with the storyteller after the FGD or the synthesis session. See Section 3 for more information about how we plan to use most significant change.
Discovery workshops After an introduction of all attending the workshop, and an introduction to the purpose of the discovery workshops, the outcomes already drafted will be presented to the following groups of stakeholders:
• One community representative and one teacher/preschool teacher from the communities where the evaluation team has conducted FGDs • One community representative and one teacher/preschool teacher from the community representatives from an equal number of other communities which we have prepared in advance to reflect on the effects of the MENAP • One DOE representative from each DOE with MLE • One Indigenous Peoples’ Association representative/Cambodia Indigenous Youth Association • representative • Three to four representatives from the POE responsible for decisions and implementation of MLE in the province • Other NGOs closely related to the implementation of the MENAP. Appropriate mixed groups with a maximum of six people will discuss the following questions: 1. Which outcomes do you agree with? Do you have anything you would like to edit/add? 2. Which outcomes do you not agree with? Why? 3. Do you have anything to add or edit regarding the outcomes and their contribution? 4. In order for the children to be successfully engaged in quality education, what do you feel is important to work on for another MENAP phase? 5. What have been some of the obstacles to achieving quality MLE in your area?
Each participant will be able to use ‘dot-mocracy’ to vote for the outcome they feel is most significant and why. The facilitators will support the three most significant change storytellers chosen as the most significant from the community level to share their stories at the discovery workshop. The local translator and facilitator will support the story with graphic illustrations to serve as a reminder, in support of those who may be illiterate. Groups of community members, teachers, POE staff and other stakeholders will have 15 minutes to discuss and choose one of the three stories, deciding on a reason why they chose this story as the most significant change regarding the implementation of the MENAP. The most significant change process will then be followed by comments and discussions as a large group, and concluding remarks by the evaluation team reflecting on common and significant themes raised by participants.
69 Independent Evaluation of the Multilingual Education National Action Plan in Cambodia
Following the FGDs, there will be time for interviewers to collect one or two most significant change stories that have been identified as possible stories for the discovery workshop. There will also be time for interviews with other important stakeholders (NGOs and others) as relevant in each location. Interviews with other key informants will explore outcomes, and their contribution and significance around the topics of implementation of MLE plans, methodology and student achievements, depending on the role of those stakeholders. Some questions will be: 1. What has been your role in supporting the MENAP? 2. What behavioural changes do you see taking place as a result of your work or the work (within the POE, among indigenous communities, in MLE classrooms)? 3. What behavioural changes do you see taking place as a result of other supporting actors (government, NGOs, etc.)? What are the changes within, for example, the POE, among indigenous communities, in MLE classrooms? 4. What have been the major obstacles to change?
National level Substantiation of outcomes will take place at the national level, involving stakeholders engaged in policy, administrative, technical and financial support of the MENAP. Outcomes, their contribution and significance will be explored around the themes of implementation of MLE plans, training and support of MLE duty bearers, student achievements, and relevance of education, depending on the role of stakeholders. Some questions will be: 1. What has been your role in supporting the MENAP? 2. What behavioural changes do you see taking place as a result of your work or the work (within the POE, among indigenous communities, in MLE classrooms)? 3. What behavioural changes do you see taking place as a result of other supporting actors (government/NGOs, etc.)? What are the changes within, for example, the POE, among indigenous communities, in MLE classrooms? 4. What are the changes taking place in responsibility for teacher training? 5. What have been the major obstacles to change? 6. How does the MENAP support indigenous identity? Among the most significant change stories that were selected in the provincial discovery workshops, which one do you feel was most significant for the effect of implementing the MENAP? Why?
Stakeholder survey questions Stakeholder survey questions (see Annex 7) will be asked after interactions with the various stakeholders, as consultants will aim to avoid bias or leading the respondents before the outcome harvesting or most significant change processes. Some questions may be answered during FGDs or may be asked during FGDs as opportunities arise. Some questions may be asked in individual interviews in person, via email or over Skype. The questions aim to answer questions raised in the ToR regarding relevance, effectiveness, efficiency, sustainability and cross-cutting considerations. This provides an additional approach to maximize the likelihood of gathering data that achieves the evaluation goals. In addition to these questions, participants at all levels will be asked to provide a rating from 1 to 5 in response to three final questions: 1. Overall, when you think about the implementation of the MENAP how well do you think it is going? 2. How hopeful are you that children from Indigenous communities in your country will become proficient in the national language, Khmer? 3. How hopeful are you that Indigenous minority children in your country will become proficient in their mother tongue/Indigenous language?
70 Independent Evaluation of the Multilingual Education National Action Plan in Cambodia
At the community level, participants will each be given a set of cards to show their ratings using facial expressions to show positive or less positive responses. Each person will be asked to give an individual response anonymously. The consultancy team will record responses.
71 Independent Evaluation of the Multilingual Education National Action Plan in Cambodia
Annex 12. Extended description of qualitative data analysis Methodological notes The following tables refer exclusively to qualitative data gathered through key informant interviews, FGDs and village, provincial and national discovery workshops, and the initial multi-stakeholder workshop. Quantitative data including school, student and teacher data, costing and survey data were analysed statistically as described in the main body of the evaluation report.
• There were 805 excerpts in total. Each excerpt included text ranging from one sentence to whole paragraphs. • The last 10 categories, except for the tag ‘Negative’, were tagged after data from Mondulkiri and initial interviews in Phnom Penh were processed and analysed. The Mondulkiri excerpts were analysed separately to establish the data tagging system. The excerpts from Mondulkiri under the nine new categories were therefore used in the evaluation, but are not available in this summary table. • The tag ‘negative’ was meaningful only when combined through filters of other tags. For example, ‘negative’ might refer to non-MLE schools, confirming the effectiveness of MLE. Filtering ‘negative’ together with ‘attendance’ would then provide a small set of excerpts to analyse for the evaluation. In total, there were 2,167 tags for excerpts. Figure 10 displays the frequency with which a given tag appeared in any of the qualitative data analysis records. In Table 16 these excerpts are broken down by province. In Table 17 the same set of excerpts are broken down by stakeholder.
72 Independent Evaluation of the Multilingual Education National Action Plan in Cambodia
Figure 10. Qualitative data analytic categories – overview Qualitative Data Analytic Categories Number of Data Points
Literacy 34 Attendance 72 Relevance 144 Interest of students 39 Understanding of students 27 Khmer issues 55 Teaching practices 105 Engagement of community 102 Student achievement and confidence 67 Integration of Indigenous Culture 83 Awareness of program 23 Management practices 163 Resources for MLE 60 Teacher training 153 Ownership of MLE 87 Curriculum 49 Tags Classroom materials 32 Expertise in government 76 Data collection 31 Preschool 70 Pilot 13 Gender 7 Support 147 Effectiveness 113 Efficiency 26 Impact 28 Sustainability 80 Rights 35 Assimilation 10 Expansion 13 Relationship 46 Assessment 1 Negative 176 0 45 90 135 180 225
73 Independent Evaluation of the Multilingual Education National Action Plan in Cambodia
Table 16. Qualitative data analytic categories and frequencies by province
points
Penh
ata
Treng
d
Tag
Total Mondulkiri Kratie Stung Ratanakiri Phnom Literacy 34 8 10 4 10 2 Attendance 72 13 11 18 26 4 Relevance 144 26 32 28 38 20 Interest of students 39 10 7 8 13 1 Understanding of students 27 4 6 2 15 0 Khmer issues 55 8 9 13 14 11 Teaching practices 105 30 9 22 34 10 Engagement of community 102 23 17 18 37 7 Student achievement and confidence 67 14 15 12 22 4 Integration of Indigenous culture 83 15 16 13 32 7 Awareness of programme 23 9 4 3 4 3 Management practices 163 32 17 29 38 47 Resources for MLE 60 20 8 7 13 12 Teacher training 153 30 14 39 45 25 Ownership of MLE 87 15 11 10 19 32 Curriculum 49 7 7 7 14 14 Classroom materials 32 4 6 8 9 5 Expertise in government 76 6 4 12 14 40 Data collection 31 2 2 4 13 10 Preschool 70 7 16 6 24 17 Pilot 13 1 5 3 3 1 Gender 7 2 0 3 2 0 Support 147 6 22 40 39 40 Effectiveness 113 0 30 33 36 14 Efficiency 26 4 8 9 3 2 Impact 28 0 9 9 7 3 Sustainability 80 4 9 18 30 19 Rights 35 0 3 11 13 8 Assimilation 10 1 2 2 2 3 Expansion 13 1 1 3 5 3 Relationship 46 0 2 19 14 11 Assessment 1 0 0 0 0 1 Negative 176 46 13 28 47 42 TOTAL 2,167 348 325 441 635 418
Red: No qualitative data excerpt Green: Three to five qualitative data excerpts Orange: One qualitative data excerpt Blue: Six or more excerpts Yellow: Two qualitative data excerpts
74 Independent Evaluation of the Multilingual Education National Action Plan in Cambodia
Table 17. Qualitative data analytic categories and frequencies by stakeholder
team
points
ata
PP Ratanakiri Kratie Stung
Mondulkiri Stung
Kratie
Mondulkiri PP Ratanakiri
d
Tag
Total Children Community Teachers DoE/PoE Evaluation Mixed Mixed MoE NGO NGO UNICEF NGO Mixed Mixed PTTC PTTC Treng University Mixed Treng Literacy 34 7 12 6 1 3 1 1 1 0 0 0 0 2 0 0 0 0 0 Attendance 72 8 12 21 12 0 3 2 1 0 0 1 0 8 1 0 0 0 3 14 Relevance 8 29 28 18 2 6 5 # 2 0 0 5 11 7 1 1 0 6 4 Interest of 39 11 6 6 5 0 1 0 0 0 1 0 0 5 2 0 0 0 2 students Understanding of 27 3 6 13 2 0 0 0 0 0 0 0 0 3 0 0 0 0 0 students Khmer issues 55 2 4 18 10 1 0 0 9 2 0 2 0 4 2 0 0 0 1 Teaching 10 8 13 25 18 0 9 5 5 0 0 0 0 11 2 1 0 0 8 practices 5 Engagement of 10 8 25 29 7 0 3 4 2 1 0 1 0 18 2 0 0 0 2 community 2 Student achievement and 67 5 9 14 12 0 8 3 1 0 0 0 0 9 2 0 0 0 4 confidence Integration of 83 7 17 26 15 1 1 5 2 1 0 0 1 6 1 0 0 0 0 Indigenous culture Awareness of 23 0 3 8 2 0 1 1 2 1 0 0 0 3 2 0 0 0 0 programme Management 16 1 6 32 43 2 6 12 30 1 1 3 3 12 6 0 0 0 5 practices 3 Resources for 60 1 3 12 14 0 6 2 8 0 1 1 2 5 4 0 0 0 1 MLE 15 Teacher training 9 14 39 32 0 9 10 8 0 5 1 5 11 3 0 1 0 6 3 Ownership of MLE 87 0 4 12 19 0 1 10 18 2 4 0 3 7 3 0 0 1 3 Curriculum 49 0 0 12 10 0 3 2 12 0 0 0 2 6 2 0 0 0 0 Classroom 32 2 1 11 6 0 1 2 3 0 0 0 1 4 1 0 0 0 0 materials Expertise in 76 2 2 7 15 0 1 6 26 1 4 2 2 4 2 0 1 1 0 government Data collection 31 0 1 9 8 0 0 4 5 0 1 0 1 1 0 0 0 0 1 Preschool 70 3 3 16 20 0 2 5 10 0 0 2 0 7 1 0 0 0 1 Pilot 13 1 5 3 1 0 0 0 1 0 0 0 0 1 1 0 0 0 0 Gender 7 0 0 2 3 0 1 0 0 0 0 0 0 1 0 0 0 0 0 14 Support 1 4 33 40 0 2 14 22 1 1 2 6 9 3 1 1 1 6 7 (Continued next page) Red: No qualitative data excerpt Green: Three to ten qualitative data excerpts Orange: One qualitative data excerpt Blue: Eleven or more excerpts Yellow: Two qualitative data excerpts
75 Independent Evaluation of the Multilingual Education National Action Plan in Cambodia
Treng
Treng
team
points
ata
PP Ratanakiri Kratie Stung
Mondulkiri Stung
Kratie
Mondulkiri PP Ratanakiri
d
Tag
Total Children Community Teachers DoE/PoE Evaluation Mixed Mixed MoE NGO NGO UNICEF NGO Mixed Mixed PTTC PTTC University Mixed Effectiveness 113 6 22 19 22 0 0 0 14 0 0 0 3 12 4 1 1 0 9
Efficiency 26 1 1 2 10 0 0 1 1 0 0 0 1 0 5 0 1 0 3
Impact 28 2 6 7 2 0 0 3 0 0 0 0 0 4 3 0 0 0 1
Sustainability 80 2 2 13 21 1 0 1 17 0 0 0 6 8 1 0 1 1 6
Rights 35 1 4 6 9 1 0 3 5 0 0 0 0 6 0 0 0 0 0
Assimilation 10 1 1 0 4 0 0 0 3 0 0 0 0 1 0 0 0 0 0
Expansion 13 0 2 0 4 0 0 1 2 0 0 0 2 2 0 0 0 0 0
Relationship 46 2 7 13 4 0 0 3 8 0 0 0 2 2 1 0 0 0 4
Assessment 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0
Negative 176 8 15 33 35 5 8 8 27 4 3 3 7 10 3 1 0 0 6
TOTAL
167
110 239 475 424 16 73 113 259 16 21 18 52 193 64 5 7 4 78
, 2
Red: No qualitative data excerpt Green: Three to ten qualitative data excerpts Orange: One qualitative data excerpt Blue: Eleven or more excerpts Yellow: Two qualitative data excerpts
76 Independent Evaluation of the Multilingual Education National Action Plan in Cambodia
Annex 13. Quality assurance The consultants met UNICEF’s expectations regarding the quality of evaluation processes and deliverables as articulated in the UNICEF-Adapted United Nations Evaluation Group Norms and Standards for Evaluation (v.6/2017). This included: • Ensuring the quality of data collected and the integrity of analysis reflected in the evaluation deliverables • Ensuring that the data collection processes adhere to UNICEF-Adapted UNEG Norms and Standards • Ensuring that the qualitative and quantitative evidence gathered is comprehensive and robust enough to make an informed assessment in line with the evaluation’s objectives, and in support of the conclusion and recommendations put forward by the evaluation • Managing all data collection (e.g., interview processes, FGDs, survey, workshops and desk review of documentary sources relating to the subject of the evaluation), analysis, reporting, and communication • Finalizing travel arrangements, accommodation and equipment to be used during the evaluation (subject to discussion with the evaluation management team) • The evaluation team retains the final authorship of the deliverables submitted to UNICEF.
The consultants recognized that effective quality control aims to ensure impartial and rigorous evaluation work and compliance with the highest professional standards. Quality control was on-going throughout the evaluation and covered various dimensions, including:
• Process organization within the evaluation team and efficient, collegial relations with UNICEF Cambodia staff and all stakeholders to ensure a positive reflection on the MENAP and its partners • The use of applied research methods according to best professional standards • On-going assessment of risks, limitations, quality defects and delays, and timely implementation of mitigation measures in coordination with UNICEF • Quality of all deliverables, especially the final evaluation report • Utility of the evaluation for its users, including the way the evaluation strategies are conceptualized, consideration of specific decision support needs, readability of the reports and PowerPoints • Timeliness and reliability of service delivery.
Quality assurance by evaluation stakeholders Evaluation management team The evaluation management team oversaw the conduct of the evaluation, ensuring impartiality and transparency throughout the process, with responsibilities including:
• Provided institutional support to the external evaluation team, including an orientation on the subject of the evaluation • Facilitated the external evaluation team’s access to key informants, as well as to specific information needed to carry out the evaluation • Ensured that a reference group was formed and provided technical support through planned meetings and participation • Monitored and assessed the quality of the evaluation deliverables to ensure they met UNEG and UNICEF quality standards • Ensured that all stakeholders, particularly the primary duty bearers internal to MoEYS and UNICEF, were kept informed throughout the evaluation
77 Independent Evaluation of the Multilingual Education National Action Plan in Cambodia
• Recommended approval of the evaluation deliverables • Helped organize and facilitate virtual and in-person briefings and meetings for the purpose of gathering input and feedback from members of the reference group (e.g., draft evaluation deliverables) and primary internal stakeholders • Solicited (through MoEYS and UNICEF) the inputs needed from key stakeholders to compile a management response to the evaluation and disseminate the evaluation products and the final report.
The evaluation management team provided logistical support to the consultant to the extent possible. It assisted in gathering background information, setting up relevant appointments, particularly during the inception phase, and assisted with coordinating field-based visits.
The reference group A reference group was established by UNICEF Cambodia with members identified based on their institutional role with regard to the MENAP. They represented their respective offices or organizations. The evaluation management specialist within UNICEF Cambodia agreed with the consultant’s recommendation to include a minimum of two representatives of Indigenous peoples’ associations and/or community networks in Cambodia in the reference group. The reference group:
• Contributed to conceptualizing the evaluation by reviewing the inception report, speaking with the consultant, and providing feedback and comments on the inception report and on the technical quality of the work of the consultant • Provided comments and substantive feedback to ensure the quality, from a technical point of view, of the draft and final evaluation reports • Assisted in identifying and introducing stakeholders during the evaluation process • Participated in review meetings organized by the evaluation management team and with the evaluation team as required • Will play a key role in learning and knowledge sharing from the evaluation results, contributing to the dissemination plan and disseminating the evaluation findings and recommendations.
78 Independent Evaluation of the Multilingual Education National Action Plan in Cambodia
Annex 14. Effectiveness of the MENAP: Extended findings from quantitative analysis of education data Data sources CARE: MENAP_Enrolment_Data.xlsx; PrimaryMLEData2017.xls; SED translated statistic of MS translation_xlsx; Planning Department
All material appearing in the following tables and diagrams is based on data supplied by CARE, or by the SED. Unless SED is referenced, the data in the tables were supplied by CARE. Almost all of the data pertain to the four provinces that were the focus of the evaluation (Kratie, Mondulkiri, Ratanakiri and Stung Treng). A limited amount of data were supplied for the province of Preah Vihear. Though not the focus of the evaluation, where those data were supplied they have been included in the tables below. This is because of expressed government interest in including the six preschools in Preah Vihear in the overall summary findings of MLE expansion achievements to date. Note on consistency of data across sources Aggregate data supplied by CARE and comparable data supplied by the SED are often but not always in agreement. As there is substantial consistency across sources, this suggests that there is no structural or systematic problem in the way data are being handled across reporters of the data. In order to provide a definitive explanation of the source or sources of the variance, data lineage would need to be documented and traced back to the point where the source data were first collected on a per- student, per-classroom or per-school level, warehoused within some central data repository, and then reported out. Tracing the movement of the data from its source was well outside the scope of the evaluation and could not be carried out by parties who were not on location within Cambodia to trace the data lineage. Where data were made available from CARE and from SED, both are reported. Neither are afforded special source-of-truth status in the evaluation, and with few exceptions (noted in the text) the differences are not materially significant. MLE implementation by province, 2018 The following tables and charts describe changes in enrolment at MLE schools arising from the implementation of the Royal Government of Cambodia’s Multilingual Education National Action Plan (MENAP). Table 18 indicates the raw totals for key MENAP target indicators, including: number of primary schools and preschools implementing MLE; gross student enrolment for these schools; and number of teachers at MLE schools.
Table 18. Overview of MENAP primary school statistics for 2017/2018 school year, Grades 1–3 No. female Non-MLE Total primary No. MLE No. MLE MLE Province MLE teachers primary enrolment schools teachers enrolment (% female) enrolment (female) 21 747 Kratie 14 38 36,179 36,926 (55.3%) 720* 18 1,025 Mondulkiri 23 54 7,448 8,473 (33.3%) 1,025 29 2,503 Ratanakiri 34 88 22,619 25,122 (33.0%) 2,725 Stung 9 620 9 25 15,228 15,848 Treng (33.3%) 620 77 4,895 Total 80 205 81,474 86,369 (37.6%) 5,090 *Figures obtained from SED in October 2018 are in italics. Figures obtained from CARE are in regular type.
79 Independent Evaluation of the Multilingual Education National Action Plan in Cambodia
CARE source data included figures on numbers of female students and teachers. In Table 18, the number of female teachers is compared to teacher totals. The percentages of female teachers in Mondulkiri, Ratanakiri and Stung Treng were quite close: 33.3 per cent for Mondulkiri, 33.0 per cent for Ratanakiri and 33.3 per cent for Stung Treng. By contrast, 55.3 per cent of the MLE Primary Grade 1–3 teachers in Kratie were female. Table 19 shows SED data on MLE Grades 1–3 teachers within provinces, broken down by gender. The proportions follow fairly closely those obtained with the CARE data. Table 20, again working with SED data, shows gender breakdowns for MLE preschool teachers, including data on Preah Vihear province. For Stung Treng, four of the eight teachers are female. For the other four provinces, the proportion of female teachers ranges from 72.7 per cent (Mondulkiri) to 100 per cent (Kratie, Preah Vihear).
Table 19. MLE teachers Grades 1–3, gender equity (SED data)
Total Female Total Female Total % Total Total Provinces contract contract state state female female teachers teachers teachers teachers teachers teachers teachers Kratie 38 21 0 0 38 21 55.3 Mondulkiri 7 3 42 14 49 17 34.7 Ratanakiri 42 8 37 14 79 22 27.8 Stung Treng 25 10 1 0 26 10 38.5 Total 112 42 80 28 192 70 36.5
Table 20. MLE preschool teachers, gender equity (SED data)
Total Female Preschool % Total Provinces Other preschool preschool teachers female teachers teachers Kratie 8 8 8 100.0 Mondulkiri 8 3 state teachers 11 8 72.7 Ratanakiri 59 59 46 78.0 Stung Treng 8 8 4 50.0 Preah Vihear 6 6 6 100.0 Total 89 92 72 78.3
Table 21 breaks down the student enrolment data by gender. With regard to the proportion of female students in the MLE Primary Grade 1–3 classes, Kratie was an outlier: 43.9 per cent of the students were female. For both Mondulkiri and Ratanakiri, 48.1 per cent of the students were female, and 51.5 per cent of the students in Stung Treng were female. For these three provinces, the percentage of female students in primary classrooms was very close to the national average of 48 per cent reported by the Ministry of Education, Youth and Sport.
Table 21. MLE student enrolment, gender equity (CARE data) Range % Non-MLE MLE MLE enrolment Non-MLE female enrolment Province enrolment female enrolment students in female (Grades 1–3) (Grades 1–3) (Grades 1–3) schools (Grade 1–3) Kratie 747 328 (43.9%) 30%-69% 36,179 17,032 (47.1%) Mondulkiri 1,025 493 (48.1%) 41%-65% 7,448 3,602 (48.4%) Ratanakiri 2,503 1,205 (48.1%) 24%-53% 22,619 10,850 (48.0%) Stung Treng 620 319 (51.5%) 30%-53% 15,228 7,358 (48.3%)
80 Independent Evaluation of the Multilingual Education National Action Plan in Cambodia
Total 4,895 2,345 (47.9%) 81,474 38,842 (47.6%)
Table 22 (SED data) shows the proportion of female students in MLE primary school Grades 1–3 compared to female students in preschools. For Kratie, 59.1 per cent of the 159 preschool students were female, but only 42.8 per cent (43.8 per cent using the CARE data) of primary Grade 1–3 students were female. The difference is a reduction of 16.3 per cent. For the other provinces, the shift was not nearly so great, with a 2.2 per cent drop in the proportion of female students for Mondulkiri, a 1.3 per cent drop for Ratanakiri, and a 2.1 per cent drop for Stung Treng.
Table 22. MLE student enrolment – gender equity in primary school, preschool (SED data) Total no. Primary/ Total no. % female Province female preschool students students students Primary 720 308 42.8 Kratie Preschool 159 94 59.1 Primary 1,025 493 48.1 Mondulkiri Preschool 177 89 50.3 Primary 2,725 1,302 47.8 Ratanakiri Preschool 1061 521 49.1 Primary 620 319 51.5 Stung Treng Preschool 151 81 53.6 Preah Vihear Preschool 133 70 52.6 Primary 5,090 2,422 47.6 Total Preschool 1,681 855 50.9
Languages Table 23 presents information on the languages of instruction within the MLE preschool and primary school classes in the four provinces that were the focus of the evaluation, together with data on Preah Vihear province. The data are supplied by SED.
Table 23. MLE preschool and primary school classes, schools, districts and students by province: SED Data Primary/ Province preschool Classes Schools Districts Languages Primary 38 14 3 Bunong Kratie Preschool 8 8 3 Primary 57 23 5 Kavet, Brao Mondulkiri Preschool 8 8 3 Primary 92 34 5 Bunong Ratanakiri Preschool 59 59 8 Primary 28 9 2 Tampuan, Stung Treng Preschool 8 8 2 Krung Preah Vihear Preschool 6 6 1 Kuy Primary 215 80 15 Total Preschool 89 89 17
81 Independent Evaluation of the Multilingual Education National Action Plan in Cambodia
Table 24 shows disaggregation according to the Indigenous language of instruction among these MLE primary schools. As shown, during the four years of the MENAP there was an increase in the number of schools implementing MLE in Tampuan, Kreung, Kavet and Bunong, especially during the 2014/2015 and 2015/2016 school years.
Table 24. Breakdown of MLE primary schools by Indigenous language of instruction Year 2015 2016 2017 2018 Brao 1 1 1 1 Bunong 25 35 36 38 Kavet 8 8 7 7 Kreung 10 15 17 17 Tampuan 10 21 17 17
Changes in the number of MLE schools between 2015 and 2018 A comparison of the number of MLE schools over the course of the MENAP (see Figure 11) reveals that there was insignificant growth in the number of schools implementing MLE, except in Ratanakiri province. However, as shown later in this section, total enrolment in MLE schools throughout the north- eastern provinces rose between 2015 and 2018. Figure 11. Changes in the number of MLE primary schools from 2015 to 2018
40 36 35 34 34
30
25 23 21 20 20 20 15 14 14 15 13 12
10 9 9 9 9
Number ofMLE Primary Schools 5
0 2015 2016 2017 2018 2015 2016 2017 2018 2015 2016 2017 2018 2015 2016 2017 2018 Kratie Mondulkiri Ratanakiri Stung Treng Province and Year
Demographic positioning of MLE programmes within local communities Table 25 provides a demographic profile of MLE enrolment relative to total primary enrolment within provinces. Data show pronounced differences between provinces with regard to the proportion of MLE enrolment compared to total primary enrolment. For example, looking at all four provinces together, Kratie has the largest primary enrolment (42.8 per cent of the total primary enrolment across the provinces), however it has the lowest proportion of MLE enrolments compared to non-MLE enrolments (2.1 per cent). By contrast, Ratanakiri has 29.1 per cent of total primary enrolments (compared to 42.8 per cent in Kratie) but 9.7 per cent of those enrolments are in MLE programmes. Mondulkiri has 10.4 per
82 Independent Evaluation of the Multilingual Education National Action Plan in Cambodia cent of the total primary enrolments, while 12.1 per cent of the primary students in Kratie are enrolled in MLE primary schools in Grades 1–3. These differences in the demographic positioning of MLE across the four provinces impact the visibility of MLE within communities and may impact on local perceptions of the importance or value of MLE.
Table 25. Demographics: MLE enrolment (2018) compared to total primary enrolment by province Total enrolment – Proportion: MLE MLE Total primary proportion of grand to non-MLE Province enrolment enrolment total primary enrolment (Grades 1-3) (Grades 1-3) enrolment (Grades 1-3) 747 2.1% Kratie 36,926 42.8% (720)* (2.1%) 1,025 12.1% Mondulkiri 8,473 10.4% (1,025) (13.8%) 2,503 11.1% Ratanakiri 25,122 29.1% (2,725) (12.2%) 620 4.1% Stung Treng 15,848 18.3% (620) (4.1%) Total 4,895 86,369 100.0% *SED figures are italicized
MLE implementation compared to 2018 targets: Staffing Figure 12 compares teacher staffing targets to actual teacher staffing in MLE primary schools in 2018.20 Because the targets were set for Grades 1–6 within provinces for the schools offering MLE in Grades 1– 3, the teaching hiring actuals also aggregate across Grades 1–6 within provinces. The staffing target number for teachers working at MLE schools set by the MENAP in 2015 has been met or exceeded in all provinces, with the exception of Ratanakiri. The Ratanakiri staffing actual is 75.4 per cent of the targeted level. This hiring shortfall (relative to targets) in Ratanakiri could be a reflection of supply of candidates, or it could reflect capacity on the hiring/recruitment side (or both). The data do not cast any light on the cause.
20 Final target figures for the MENAP were set for the 2018/2019 school year. Figure 12 reflects the targets for 2017/2018 teacher staffing, given the latest available figures; MoEYS, Multilingual Education National Action Plan, 2015, pp. 13.
83 Independent Evaluation of the Multilingual Education National Action Plan in Cambodia
Figure 12. Teacher staffing targets vs. actual teacher staffing at MLE primary schools
140 130
120 98 100
80 68 59 60 51 46 41 37 No. No. Teachersof 40
20
0 Ratanakiri Mondulkiri Stung Treng Kratie
2015-2018 Target No. of Teachers, Grades 1-6 Actual No. of Teachers 2018, Grades 1-6
Table 26 and Table 27 present pupil-teacher ratios by province. Table 26 is based on CARE data, and Table 27 is based on SED data. For Kratie, Ratanakiri and Stung Treng the results in both tables are quite close, with a consistent pattern. For Kratie, the ratio fell below the mean, while for Stung Treng, the ratio fell almost exactly on the mean. For Ratanakiri, the ratio was well above the mean, and for Mondulkiri, the ratio was well below the mean if the CARE enrolment data are used (Table 26). If the SED data are used, the ratio is closer to the mean and comparable to the ratios for Kratie.
Table 26. Student/teacher ratios by province, data from CARE
No. of MLE No. of MLE MLE enrolment Pupil/teacher Pupil/class Province teachers classes (Grades 1-3) ratio for MLE ratio for MLE Kratie 38 38 747 19.7 19.7 Mondulkiri 54 57 1,025 19.0 18.0 Ratanakiri 88 92 2,503 31.7 27.2 Stung Treng 25 28 620 24.8 22.1 Total 205 215 4,895 23.9 22.8
Table 27. Student/teacher ratios by province, data from SED
No. of MLE MLE enrolment Student/teacher Province teachers (Grades 1-3) ratio for MLE Kratie 38 720 18.9 Mondulkiri 49 1,025 20.9 Ratanakiri 79 2,725 34.5 Stung Treng 26 620 23.8 Total 192 5,090 24.0
MLE implementation compared to targets, 2018: Student enrolment Table 28 provides information that relates actual student enrolment in 2018 to targets set in 2015. Note that the enrolment numbers reflect totals for Grades 1–6 for schools with MLE programmes, because
84 Independent Evaluation of the Multilingual Education National Action Plan in Cambodia the enrolment targets are set for Grades 1–6. These same data are depicted graphically in Figure 13. As the numbers indicate, Kratie and Stung Treng came very close to achieving the established targets. Ratanakiri actual enrolment fell short of the target by 22 per cent based on the CARE data, or 16.4 per cent based on the SED figures. Mondulkiri enrolment exceeded the target by 11.4 per cent based on the CARE data, or 6.6 per cent based on the SED data. Kratie also exceeded targets, by 14.7 per cent based on CARE data or 112.1 per cent based on SED data. Stung Treng enrolment was very close to target. Overall, actual enrolment fell short of the targets by 8.7 per cent based on the CARE data or 6.6 per cent based on the SED data, although the shortfall was essentially attributed entirely to the programme in Ratanakiri.
Table 28. Targeted vs. actual enrolment by province in 2018
Enrolment Province Actual % of target target Kratie 1,178 114.7 1,027 Kratie with SED correction 1,151 112.1 Mondulkiri 1,252 111.4 1,124 Mondulkiri with SED correction 1,198 106.6 Ratanakiri 3,056 78.0 3,919 Ratanakiri with SED correction 3,278 83.6 Stung Treng 897 98.1 914 Stung Treng with SED correction 897 98.1 Total 6,383 91.3 6,984 Total with SED correction 6,524 93.4
Figure 13. Targeted vs. actual enrolment in MLE primary schools during the MENAP period
4500 3919 4000 3500 3278 3056 3000 2500 2000
1500 112412521198 11781151 914 897 897 1027 1000 500 0 Ratanakiri Mondulkiri Stung Treng Kratie
2015-2018 Target Enrolment Actual Enrolment 2018 - CARE Data Actual Enrolment 2018 - SED Data
Longitudinal trends in enrolment Table 29 indicates the primary enrolment figures for each province for Grades 1–3 over the course of the four MENAP years. For each province for each year, there are two data elements. The first is the MLE enrolment for Grades 1–3 for that province for that year. Directly below, in italics, is the primary
85 Independent Evaluation of the Multilingual Education National Action Plan in Cambodia
Grade 1–3 enrolment, minus the MLE enrolment, to enable comparison of MLE to non-MLE enrolment within the province. Data from Table 29 for MLE enrolment are displayed graphically in Figures 14 to 17 to illustrate changes in enrolment within the province. For these figures, the enrolment rate for 2015 is set as a baseline, and enrolments for 2016, 2017 and 2018 are expressed in terms of percentages of those 2015 figures. Figure 18 provides a comparison of the four provinces. As a coarse correction for changes in population demographics and associated numbers of children enrolled in Grades 1–3, these MLE data are plotted against the non-MLE Grade 1–3 enrolment figures. From the information in Figures 14 to 18, it is evident that in all provinces except Stung Treng enrolment increased from the 2015 baseline, although by 2017 enrolment levelled off or dropped slightly. In Stung Treng, there was a small drop in 2017, but enrolment levels recovered by 2018 to a level that followed the overall increase in student enrolment in that province from 2015 to 2018. Note that the figures in Table 29 are based on UNICEF data. If the figure 1,025 is used as the real enrolment figure of Mondulkiri for 2018, that would represent an enrolment increase of approximately 27 per cent over 2017, and an increase of 270 per cent from 2015. By contrast, the largest increase for any of the other provinces from 2015 was found in Ratanakiri, with an increase of 189 per cent.
Table 29. MLE and non-MLE provincial enrolments for Grades 1–3 during the MENAP period
2015 2016 2017 2018 Province MLE (% fem) MLE (% fem) MLE (% fem) MLE (% fem) Province (% fem) Province (% fem) Province (% fem) Province (% fem)
594 (45.8%) 782 (47.3%) 852 (46.6%) 747 (43.9%) Kratie 34,907 (47.4%) 36,180 (47.6%) 36,400 (47.8%) 36,179 (47.1%)
374 (47.6%) 699 (45.8%) 781 (45.3%) 1,025 (48.1%) Mondulkiri 7,100 (49.5%) 7,406 (48.3%) 7,824 (48.9%) 7,726 (48.4%)
1,320 (49.5%) 1,940 (48.1%) 2,389 (48.7%) 2,503 (48.1%) Ratanakiri 22,126 (46.8%) 22,493 (47.8%) 22,222 (47.4%) 22,619 (48.0%)
550 (50.4%) 548 (49.3%) 491 (52.1%) 620 (51.5%) Stung Treng 12,738 (48.7%) 14,152 (48.0%) 14,830 (48.3%) 15,228 (48.3%)
Total MLE 2,838 (48.6%) 3,969 (47.7%) 4,513 (48.1%) 4,895 (47.9%) enrolment
Total non-MLE 76,871 (47.7%) 80,231 (47.8%) 81,276 (47.9%) 81,752 (47.7%) enrolment
86 Independent Evaluation of the Multilingual Education National Action Plan in Cambodia
Figure 14. Changes in MLE enrolment, Kratie, 2015–2018
Kratie - Changes in Enrolment Relative to 2015 160.0% 140.0% 120.0% 100.0% 80.0% 60.0%
40.0% Per Per centChange 20.0% 0.0% 2015 2016 2017 2018 Year
Kratie: MLE Enrolment Kratie: non-MLE Provincial Enrolment
Figure 15. Changes in MLE enrolment, Mondulkiri, 2015–2018
Mondulkiri - Changes in Enrolment Relative to 2015 300.0%
250.0%
200.0%
150.0%
100.0%
Per Per centChange 50.0%
0.0% 2015 2016 2017 2018 Year
Mondulkiri: MLE Enrolment Mondulkiri: non-MLE Provincial Enrolment
87 Independent Evaluation of the Multilingual Education National Action Plan in Cambodia
Figure 16. Changes in MLE enrolment, Ratanakiri, 2015–2018
Ratanakiri - Changes in Enrolment Relative to 2015 200.0% 180.0% 160.0% 140.0% 120.0% 100.0% 80.0% 60.0%
Per Per centChange 40.0% 20.0% 0.0% 2015 2016 2017 2018 Year
Ranatakiri: MLE Enrolment Ranatakiri: non-MLE Provincial Enrolment
Figure 17. Changes in MLE enrolment, Stung Treng, 2015–2018
Stung Treng - Changes in Enrolment Relative to 2015 140.0% 120.0% 100.0% 80.0% 60.0% 40.0% Per Per centChange 20.0% 0.0% 2015 2016 2017 2018 Year
Stung Treng: MLE Enrolment Stung Treng: non-MLE Provincial Enrolment
88 Independent Evaluation of the Multilingual Education National Action Plan in Cambodia
Figure 18. Changes in MLE enrolment, provincial comparison, 2015–2018
Changes in Enrolment - Provincial Comparison 250.0%
200.0%
150.0%
100.0%
Per Per centChange 50.0%
0.0% 2015 2016 2017 2018 Year
Mondulkiri: MLE Enrolment Ratanakiri: MLE Enrolment Grand Total: MLE Enrolment across Provinces Kratie: MLE Enrolment Stung Treng: MLE Enrolment
89 Independent Evaluation of the Multilingual Education National Action Plan in Cambodia
Annex 15. Summary of findings, conclusions and recommendations
Evaluation question Findings Conclusions Recommendations Relevance 1. To what extent is the Multilingual Primary rights holders and duty bearers The MENAP has provided a relevant and A renewal of the MENAP for a five-year Education National Action Plan (MENAP) uniformly confirmed the relevance of the well-suited mechanism for transitioning term is warranted on the basis of primary suited to ensuring multilingual education provision of education in a language that MLE from the path-finding work of CARE rights holders’ need for education that (MLE) is accessible to children (rights children can understand, using a to government ownership. includes Indigenous languages of holders) in the four provinces where it has culturally enriched MLE curriculum and instruction and consideration of the been substantially implemented? having Indigenous language speakers as Indigenous children and parents value successes of the inaugural MENAP. The teachers. and support MLE as a way to preserve new 2019-2023 MENAP (MENAP 2) culture and language and to provide should define quality, add new elements, Indigenous children, parents and opportunities. provide more explicit intermediate teachers emphasized the need for outcomes, and include costed, actor- curriculum renewal to amplify cultural Co-location of MLE preschools and focused, time-bound, results-oriented components and include pedagogical primary schools provides continuity in activities and accountability indicators. A direction for teaching Indigenous literacy. children’s opportunities to learn in their greater commitment from the government Indigenous language and Khmer. for the Programme Budget to support The plan has contributed to the goals of implementation of the next MENAP is the Education Strategic Plan and the The next MENAP should have an explicit required to ensure government National Strategic Development Plan, theory of change that is discussed and ownership and fulfilment of the vision. and the processes of decentralization and agreed upon by primary rights holders as deconcentration. well as primary duty bearers, and that is costed and includes a medium-term, results-based formative evaluation plan as well as a summative evaluation planned for the end of a five-year term. Effectiveness 2. Effectiveness of the support provided The plan has met its targets in terms of The MENAP has achieved its main The emergent capacity of MoEYS to by MoEYS, UNICEF and other increased teachers, classrooms and targets of increased opportunities for assume responsibility for the MENAP implementing partners in achieving its students in MLE in Grades 1 to 3. ethnic minority children to access MLE in should be incorporated into the design of outcomes. the first three years of primary school, the 2019-2023 MENAP. Indigenous language of instruction, building on the foundational infrastructure culturally relevant curriculum and and successes achieved by CARE. The emerging potential of MLE Indigenous teachers were identified by preschools can be capacitated through primary rights holders as key efforts to improve their quality to meet the determinants of increased enrolment of criteria for annexation to MLE primary
90 Independent Evaluation of the Multilingual Education National Action Plan in Cambodia
Indigenous children, while improved There are persistent gaps in the quality of schools, according to the Royal school and economic conditions were education for all children, while many Government of Cambodia’s Prakas No. identified by duty bearers as families face poverty. 245 on the management of community determinants of increased enrolment of preschools. Linking MLE preschools to all children in the north-eastern Data gaps and inconsistencies MLE primary schools will provide provinces. underscore the immediate need to continuity in children’s language-of- strengthen education data systems to instruction opportunities. Seasonal demands of family farming, track progress. poverty and lack of safety are persistent A robust education management barriers to enrolment and regular information system will enable mapping attendance in some communities. An and tracking of ethnic minority language alternative school calendar innovated in a teaching, learning needs and progress, Kratie community was identified in many enabling evidence-based decision- communities as a possible solution. making about expansion and targeted quality improvements in each of the All stakeholder groups gave high ratings provinces covered by MENAP 2. for children’s likelihood of developing proficiency in Khmer and much lower ratings for their likelihood of becoming proficient in their Indigenous language. Efficiency 3. Efficiency of the management of the The government committed roughly 3 per Primary rights holders and duty bearers Increased quantity, quality and retention MENAP to ensure timely and efficient use cent of gross domestic product to uniformly agree on successful of MLE primary school teachers and core of resources. education from 2014 to 2018. implementation of the MENAP, referring trainers with Indigenous language in particular to increased MLE proficiency should be the convergent goal The MENAP was not a costed plan and classrooms, teachers and enrolled of a number of strategies and activities its implementation did not have a specific students. Facilities for MLE primary undertaken by MoEYS. MoEYS should Programme Budget. The government classrooms and schools have been increase the quantity of MLE teachers to contributed through its nationwide improved and expanded through financial keep pace with expansion to additional initiative to convert community schools to contributions by local and international communities and language populations. state schools, resulting in increased donors and in-kind contributions by Retention of existing talent and salaries for teachers and the inclusion of school support committees. recruitment of more core trainers and formerly community schools with MLE to frontline teachers who are literate in an the Programme Budgets of POEs. The Programme Budget for Indigenous language should be priorities. implementation of the MENAP in the four Development partners should support the The SED, which was mandated to north-eastern provinces is insufficient for government in on-going efforts to provide implement the plan starting in 2016, the full range of activities that have been and enhance pre-service and in-service reported a total budget of US$ 20,988, carried out by development partners. training.
91 Independent Evaluation of the Multilingual Education National Action Plan in Cambodia
dispersed in 2018 specifically for teacher training in Ratanakiri and Mondulkiri. All MLE primary school teachers were other responsibilities of this department generally perceived as strongly reportedly depended on UNICEF funding. committed and doing their best with limited resources. Yet, education officers UNICEF contributed US$ 748,609 from described the difficulty of recruiting 2016 to 2018. CARE contributed preschool and primary school teachers US$ 672,776 from 2015 to 2018. who are well-educated, proficient in an Management of the MENAP has occurred Indigenous language, and willing to work primarily at the sub-national level. in often remote and difficult circumstances for low wages. POEs have mobilized core trainers, district officers, school cluster leaders Among preschool teachers, high turnover and local authorities to support MLE is prevalent, calling for a continuous teachers. infusion of capacity (re)building. A new sub-decree (no. 245) provides a After significant initial investments to framework for preschool teachers who create curriculum and infrastructure, qualify to be placed on government POEs report that the operational costs of payroll, to convert preschools to state MLE preschools and schools are only schools, and to co-locate preschools with slightly greater than for non-MLE schools. primary schools, depending on preschools meeting criteria, such as a There are shortfalls reported across all suitable facility and other requirements. provinces in dedicated funding for language-specific teacher training and A generally inefficient education monitoring, and Indigenous language management information system makes resources. The capacity of the PTTC with it difficult to determine, in quantitative significant numbers of Indigenous terms, the relative contributions of the trainees has not yet been harnessed to MENAP compared to alternatives. amplify the cadre of well-qualified, Indigenous teachers for MLE.
Indigenous peoples’ organizations and community networks have not yet been mobilized as advocates and supporters of the MENAP.
Impact
92 Independent Evaluation of the Multilingual Education National Action Plan in Cambodia
4. Impact resulting from the Children are more engaged in learning in Indigenous children in MLE are benefiting A review of the MLE primary school implementation of the MENAP (positive and beyond the classroom, with by enrolling, attending more regularly, curriculum would respond to calls by MLE and negative changes, intended and aspirations to continue in higher actively participating and aspiring to teachers for Indigenous literacy unintended) in the four provinces. education. further their education to an extent not components and by all primary rights seen before MLE. holders for a larger amount and more up- MLE has created an avenue for retention to-date culturally specific content. of language and culture and reduced Duty bearers at the sub-national level discrimination against Indigenous people have consolidated basic infrastructure for in school and social settings. MLE that was built primarily by CARE prior to and during the first two years of Demand for MLE, including extension the MENAP. through Grade 6, has increased. The POEs in all provinces expressed MLE has increased Indigenous parents’ their willingness to collaborate with SED, engagement in children’s education and recognizing that SED is in the early their own aspirations to become literate in stages of building functional their mother tongue. relationships.
Some senior education officials at the sub-national and national level are reluctant to fully support the intent of MLE, favouring a quick transition to Khmer language even before children reach school age, thereby negating the need to sustain MLE over time. Sustainability 5. Sustainability of the benefits of Sub-national duty bearers report more The MENAP has institutionalized a To sustain commitment and respond to MENAP. demand than supply of resources to mechanism for authorized use of critics, advocacy and capacity building respond to increased demands for MLE. Indigenous languages in preschools and should promote system-wide lower primary school. Commitment, understanding of the purpose and Some stakeholders express a view of capacity and relationships are being built potential of MLE not only as a tool to MLE only as a temporary measure while among primary rights holders and duty increase ethnic minority children’s Indigenous languages persist, using the bearers at sub-national levels. participation in education but also to shortest possible bridging model, such as embrace diversity, protect Cambodia’s is currently in place. The demand created by the MENAP cultural resources and promote social among primary rights holders, including harmony. The government has the capacity to Indigenous children, parents, school assume full ownership of a new MENAP support committees and local authorities,
93 Independent Evaluation of the Multilingual Education National Action Plan in Cambodia
if the capacity of the SED is strengthened can be expected to grow to encompass and a process is institutionalized to more Indigenous languages and other ensure collaboration with other relevant ethnolinguistic minority groups, and to departments. include extension to Grade 6 in order to support children’s literacy in their mother Primary rights holders view the use of tongue. ethnic minority languages in education as a means to secure children’s ethnic The government is in the early stages of identity, cultural knowledge and capacity taking ownership of the MENAP. MoEYS to transmit Indigenous language has the authority and means to build on proficiency to future generations. the administrative systems, teaching cadre, MLE curricula and demand within Indigenous communities that has been created.
The concentration of responsibility for the MENAP in a new department (SED) poses some risk. Insufficient human resources within SED were noted at the provincial level and by SED officers, who identified a need for added capacity in Indigenous languages and cultures, and additional staff to enable adequate monitoring in often hard-to-reach communities.
94 Independent Evaluation of the Multilingual Education National Action Plan in Cambodia
Annex 16. International evidence on MLE
The challenge Children21 whose first language is not used in education have limited academic success, and as a result are less likely to be able to contribute to a country’s economic and intellectual development.22
Children and their primary caregivers arrive at the threshold of formal schooling (when they do) with a precious resource: their mother tongue or first language (also widely known as L1). Some children’s mother tongue is privileged in formal education as the language of instruction, which typically is presented to children and families as the best language for learning, the highest status language, and the only language allowed. Other children’s mother tongue is dismissed, denied, or given only token support. For children whose first language is not the language of education, this is subtractive education. 23 Some have charged that this is language discrimination 24 and therefore a violation of several international agreements (enumerated subsequently). Children in this submersion approach have to ‘sink or swim’ in an unfamiliar language medium. Many children sink. For those who swim, most never develop proficiency in their mother tongue, and may forget it altogether. This is a loss to them, their families, their ethnolinguistic communities, their nations and the world. Some children who persevere through the transition to schooling in a new language succeed in school and some do not. Changes in their educational achievement can be increased by providing preschool and primary school in their mother tongue (mother tongue-based education) or a mix of their mother tongue and the dominant language (MLE).25 These are both socially inclusive approaches that are increasingly used around the world to reach children who are among the most at risk of persistent marginalization and low quality of life.
How MLE works MLE is the education practice of relying primarily on learners’ mother tongue, and the culturally based knowledge that the mother tongue expresses as a foundation for learning, with some introduction of a second language during part of the school day or week26 (see Annex 3). Ideally, children are supported to become fully literate in their mother tongue while learning elements of one or more additional languages. Research shows that after children have achieved literacy in one language, it is relatively easy for them to become literate in additional languages.27 Children who become literate in their mother tongue can then build on this cognitive skill to become literate in a national or international language.28 Though many parents and educators do not grasp this, MLE does not require a choice of the mother tongue or a national language. It involves both, but in a carefully planned sequence over several years that starts with the language children know when they enter school.29
21 This and other statements referring to ‘children’ point to population-level trends. There are always individual differences in any population, and some children thrive in various education systems. The current statement refers to the general trend. 22 World Bank, 2006, p. 4. A collection of resources on MLE is available here: http://www.unescobkk.org/education/multilingual-education/resources/publications 23 Ball, 2011. 24 Benson, 2004; Brock-Utne, 2001; Skutnabb-Kangas, 2009. 25 Dutcher, 2003. 26 Benson, 2009; Dutcher, 2003. 27 Lightbown, 2008. 28 Cummins, 2009. 29 Pinnock et al., 2011.
95 Independent Evaluation of the Multilingual Education National Action Plan in Cambodia
A global issue Around the world, having to start school in an unfamiliar language is a known contributor to persistent high rates of non-enrolment, low attendance and engagement in learning, and early school leaving. Of the estimated 56 million children still out of school, and those who drop out before completing Grade 1, many belong to Indigenous and other ethnolinguistic minority groups.30 Many governments are working to put into action the ideal that no one will be discriminated against based on their membership in a particular population group. They see MLE as a matter of educational equity, child rights, and a moral imperative. Recent research, including studies in ASEAN member countries, shows that language-in- education policies that embrace MLE can promote social cohesion and peace.31 Equitable educational opportunities for all children are part of the Sustainable Development Goals, in particular Goal 4 – Quality education: Ensure inclusive and equitable quality education and promote lifelong learning opportunities for all. Equity in education means being able to understand the language of instruction and to learn one’s own language through formal education systems, which will make the other 16 Sustainable Development Goals more achievable. MLE promotes universal enrolment in early childhood and primary education, gender equality and women’s empowerment, healthy living and access to health care, workforce participation and sustainable development.32 International declarations, conventions and charters support the use of children’s mother tongue in education, including: United Nations Convention on the Rights of the Child (UNCRC) (1989) Article 30: • Stipulates the right of Indigenous Peoples to use their own language in schooling
UNCRC General Comment 7: • Young children are holders of all rights enshrined in the Convention • Early childhood is a critical period for realization of these rights • Early childhood: birth through transition to school (8 years) • Programmes and policies are required to realize rights in early childhood • Recognize and incorporate diversities in culture, language and child rearing
UNCRC Article 29 • Education of the child shall be directed to development of respect for the child’s parents, and the child’s own cultural identity, language and values, as well as for the national values of the country in which the child is living
UN Convention and Recommendation against Discrimination in Education:
30 UNESCO, 2017. 31 Lall and South, 2013, 2016; South and Lall, 2016a, 2016b; Lo Bianco, 2013, 2016; Person, 2018. 32 UNESCO, 2017.
96 Independent Evaluation of the Multilingual Education National Action Plan in Cambodia
• Specifically recognizes the right of the members of national minorities to carry on their own educational activities, including the use or the teaching of their own language
UN Declaration on the Rights of Persons Belonging to National or Ethnic, Religious and Linguistic Minorities (1992, Article 4): • Affirms the rights of minorities, including Indigenous Peoples, to learn and/or have instruction in their mother tongue or heritage language • UNCRC Articles 5, 29, 30, General Comment 7 • UN Convention and Recommendation against Discrimination in Education
Evidence supporting MLE High quality MLE has been shown to improve children’s readiness for success in secondary school and in life (see Annex 5). MLE preschool can help children and their parents to feel welcome and safe. MLE preschool can support children’s development of oral language skills, which are the foundation of literacy, and their general learning and socio-emotional development. MLE in lower primary school can also introduce children to the sounds and written form of one or more additional languages, including the national language or another language.
Investigators of MLE33 agree on a similar conclusion articulated by Kraft (2003,19):34 “When children master the basics of literacy in the Mother Tongue, with trained teachers, quality, age-appropriate books and instructional materials, and sufficient time in which to master the basics, they can and do transfer easily into a second (international) language. They can more easily become the fluent bilingual, biliterate citizens every advanced country of the world so desperately seeks and needs.”
Briefly, a review of evidence concludes that MLE:
✓ Offers a welcoming communication environment for young children and their families who speak a non-dominant language ✓ Shows children and parents that their language is a legitimate language of learning ✓ Reinforces children’s confidence that their communication skills and culturally based knowledge are normative and worthy, building their self-efficacy as capable learners ✓ Enhances oral language skills in the mother tongue, which are the foundation of literacy in the mother tongue (see Annex 4) ✓ Ensures that minority language children receive at least some instruction in their mother tongue before they start school, which may be in another language ✓ Provides exposure to and supports learning an additional language–which is typically the language of instruction used in secondary schools, thereby ensuring children’s successful transition to upper levels of schooling.
Outcomes of MLE beyond education: Social cohesion The languages used in schooling can be a strategy for citizen enfranchisement or marginalization. Research shows that MLE promotes rather than detracts from national unity and security. As has been dramatically demonstrated in recent years in countries including Myanmar and Thailand, intra-national fragmentation and armed conflict may be related to inter-ethnic tension represented and embodied in language-in-education policies. When language issues are negotiated so that ethnolinguistically distinct
33 Alidou et al., 2005; Cummins, 2000. 34 Kraft, 2003.
97 Independent Evaluation of the Multilingual Education National Action Plan in Cambodia population groups have at least some of their goals met through education, intra-national tension may be reduced. A study from 2012 to 2016 in Myanmar, Malaysia and Thailand uncovered ways in which language and conflict are linked and the kinds of policy and educational interventions that could mitigate, resolve and prevent conflict and promote social cohesion.35 Another recent study examined MLE in Myanmar and illustrated the implications of language-in-education policy for conflict or peace.36
Decentralized approaches to MLE Some countries have recently explored MLE through decentralization, with or without financial and technical support. A decentralized approach often means that local (township, district, state, province) governments have some flexibility in language-in-education policy, but also have fiduciary responsibility for implementing self-defined policy. This can mean relative autonomy, but can also cut off access to centralized systems, such as accreditation and licensing that often come with financial support, centralized systems of training and materials development, and so on. Decentralization may also result in inequities whereby only some children in a country have opportunities to access MLE in their language, while the rest of the nation’s children have no access to those language learning opportunities through formal education. Countries such as Aotearoa/New Zealand have realized the benefits of centralized support for bilingual learning, increasing social cohesion between Indigenous and non-Indigenous populations. The Philippines is the only country in Southeast Asia to have instituted a national policy requiring MLE (in 11 selected languages) in all lower primary schools (Grades 1–3). The Philippines has recently opted for a national policy requiring MLE in lower primary school, with decentralized decision making about how to implement the policy. Districts can decide on their approach. In some districts, schools and classrooms are on different language tracks, where children learning in one language as the primary language of instruction are grouped together, separate from other language speakers. Implementation challenges and ideological resistance to the MLE initiative in the Philippines appear to be due, in part, to the hands-off approach of the national government, which provides little guidance, teacher training, or materials for teaching in the many mother tongues of the country’s diverse population.37 Notwithstanding, there are benefits to supporting a diverse system of education, including meeting the needs of a culturally and linguistically diverse population, improving trust by ethnolinguistic minorities in government systems, and building social cohesion, federalism and democracy. Other countries in the region have become engaged in deliberations about MLE, but the vast majority of MLE programmes in the region are supported by community members, non-government organizations and international donors. Cambodia is outstanding in the region in its commitment to MLE at both preschool and lower primary school levels for children in the five north-eastern provinces where most Indigenous children live. It combines a decentralized approach emphasizing administrative oversight by the provincial offices of education with centralized planning and support.
35 Lo Bianco, 2013. 36 South and Lall, 2016a, 2016b. 37 Burton, 2013; Mahboob and Cruz, 2013.
98 Independent Evaluation of the Multilingual Education National Action Plan in Cambodia
Common challenges in MLE implementation38 Quality. Many countries and communities struggle to provide quality education at any level, regardless of the language(s) of instruction. Even with MLE, low quality programmes will not have desired outcomes. MLE is one component of a comprehensive strategy to improve education quality. Length of time in MLE. Theories and research supporting MLE emphasize the length of time needed to realize its potential benefits. Ideally, MLE is offered at the beginning of a child’s journey through primary school, delivered primarily in their mother tongue, with study of additional language(s) until they are literate (proficient; ready to learn from what they read).39 Often, children are offered no choice except primary school in a new language. Even so, research suggests there is value in offering preschool in their mother tongue, in order to encourage participation in the school system, and then introducing the language of instruction before children have to transition to education in a new language.40 Ideological and structural barriers. Many teachers and education systems around the world embody and transmit a ‘melting pot’ national ideology that does not embrace minority languages and cultures as legitimate and valued in education. With this attitude, teachers, school administrators and others may overtly or covertly resist implementation of MLE, even when policies and curriculum allow and encourage it. This appears to account significantly for challenges in the nationwide implementation of MLE in the Philippines.41 These attitudes are often transmitted to parents. Teacher training gaps. Teacher training programmes typically do not prepare teachers to engage in multilingual teaching and learning. Teachers and community-based teacher assistants must be trained to support oral language acquisition, to use children’s mother tongue proficiently.
Teaching and learning materials.42 Creating sufficient quality and quantity of teaching and learning materials in the mother tongue requires sufficient: funds; time; input from early learning experts; input from proficient speakers of minority languages who also have some understanding of language acquisition; and input from cultural knowledge holders who can bring local funds of knowledge and relevant skills into teacher training to develop age-appropriate, locally meaningful teaching and learning materials.43 These challenges appear to account significantly for challenges in sustaining MLE gains in Papua New Guinea.44
38 Lack of written versions (orthographies) of minority languages is a frequently identified challenge for MLE preschool. It is not highlighted here because it is not absolutely essential for programmes in the early years to introduce written language to young children, although many programmes do. The main goal of early learning programmes in terms of literacy is to support children’s oral language development. Story telling using books can be ‘read’ dialogically by teachers in the mother tongue, regardless of the language in which they are written. This promotes oral language development as well as print readiness. However, it is ideal if orthographies are available, and if written material can be produced for preschool children to see their language in written form, and to engage in phoneme recognition and other emergent literacy skills. 39 Cummins, 2000; Thomas and Collier, 1999; 2001. 40 Ball, 2011. 41 Burton, 2012; Tupas, 2015. 42 On the creation of teaching and learning materials, the works of Edwards and colleagues are particularly recommended. Edwards, 2006; Edwards and Ngwaru, 2010, 2011, 2012, 2014; Tin, 2006. 43 Malone, 2003; Moll, Amanti and Gonzalez, 2005. 44 Malone and Paradie, 2011.
99 Independent Evaluation of the Multilingual Education National Action Plan in Cambodia
Figure 19. Multiple stakeholders with potentially different goals for MLE
Government and development partner support. Significant and sustained financial investment is needed to develop human resource capacity to deliver MLE and to produce locally meaningful, quality curriculum for children whose cultures are distinct from the mainstream and whose languages embody the philosophy and social orientation of their cultures.
Parent awareness. Parents have often absorbed derogatory attitudes in mainstream society against their mother tongue, as well as misconceptions about how children become literate. They may have doubts about whether their children will be successful in life if they are not switching to a national language as early as possible. A vigorous social media campaign involving teachers, school support committees, government and national thought leaders needs to encourage parents to see their mother tongue as a legitimate and effective language for formal education, and encourage parents to use their mother tongue as the primary language at home.
Measuring effectiveness. Whether MLE is ‘effective’ depends on the desired outcomes of education. There are always many desired outcomes, valued differently by different stakeholders. In this sense, measuring effectiveness is a political and social decision as well as part of evaluating education delivery. What MLE can contribute to education and society depends on what, when and how outcomes are measured. For example, a valued outcome may be participation: e.g., enrolment of children who would otherwise be out of school, increased attendance, reduced drop out, and continuation at progressively higher levels of education. There are many tools to measure learning indicators, such as ability to follow instructions given in a particular language, vocabulary, story-telling, problem solving, numeracy, general fund of knowledge, and subject content mastery. Some measurement strategies are biased to favour language proficiency or content knowledge in the dominant language rather than the mother tongue. This will greatly influence assessments of the effectiveness and contribution of MLE, with a cascade of impacts on parent support, teacher confidence, political will and government investment.
100 Independent Evaluation of the Multilingual Education National Action Plan in Cambodia
Figure 20. Possible education outcomes for cross-sectional and longitudinal assessment
The timing of assessment matters greatly in MLE. Many studies of MLE measure outcomes at the end of Primary 2 or 3. However, researchers have found that what happens after Primary 3 is critical for academic success: MLE must be continued until children are fully literate, typically by the end of primary school. Some research shows that ‘early exit’ from MLE in Primary 3, in the midst of literacy acquisition, may not produce improved outcomes. Children who continue in MLE (‘late exit’) until the end of primary school (typically six years) have been found to show superior outcomes compared to non-MLE peers, if academic success including multilingual ability is measured at the end of lower secondary school.45 The ‘gold standard’ for evaluating effectiveness is to measure a variety of outcomes for children, including some who have been in one kind of programme and some who have been in another kind of programme (e.g., only their mother tongue, compared to only the national language, compared to a mix of mother tongue and the national language). Costs. “It’s too expensive” is one of the arguments that make it seem impossible to introduce or expand MLE. The costs of not delivering MLE may also be significant. Studies point to the immense costs to society of low literacy rates, low achievement, drop out, and the social, economic and psychological deprivations for children in ethnolinguistic communities without access to MLE.46 Many scholars have pointed out that offering education in only one language may represent a false economy.47
Essential ingredients for successful MLE: Getting it right from the start 1. Supportive language-in-education national policy and strategic plans at national, provincial and community levels. 2. Agreement from national or local accrediting bodies that education using MLE that meets quality criteria will not be included in accreditation or licensing mechanisms, equitable teacher salaries and promotions, student scholarships, and other formal financial, status and achievement recognitions.
45 Cummins, 2000; Thomas and Collier, 1997, 2002. 46 Heugh, 2006. 47 Brock-Utne, 2005; Heugh, 2006.
101 Independent Evaluation of the Multilingual Education National Action Plan in Cambodia
3. Significant investment of appropriate expertise, time, money and community consultation for development, production, distribution and on-going updating of culturally relevant and linguistically accurate teaching and learning materials. 4. A module for teacher training programmes and professional development that prepares teachers and education officers at all levels to understand MLE and how to communicate to deliver MLE. 5. A cadre of accomplished MLE teachers, proficient in mother tongue(s), to become core trainers, advocates and leaders for expansion and improvement of MLE. 6. Parent involvement, including as assistants in MLE classrooms and as advocates with local and national education authorities of MLE. 7. Social awareness campaigns that directly address common misconceptions about children’s capacity to learn more than one language and the way MLE can work to produce multilingual children who can succeed at all levels of education, in the workforce and in society. 8. A monitoring and evaluation system that includes direct observation of teaching behaviours and learning activities to ensure fidelity to sound principles and practices of MLE. 9. Tools for assessing learner achievement that include learning of the mother tongue and in the mother tongue, as well as achievements in the second language. 10. Documentation of the implementation of MLE and research that objectively assesses the contributions of MLE to education and broader national and local development goals (e.g., workforce development, social harmony, national heritage preservation, national unity). 11. Knowledge mobilization of lessons learned and promising practices to inform expansion and improvement of MLE and to promote social inclusion in quality education locally and globally.
Key lessons learned to date for MLE in the early years 1. MLE can be a catalyst for positive change in education outcomes and society. 2. Attention must be paid to the realities that children will face when they transition from preschool to primary school, and from primary school to secondary school, and make language of instruction programme design decisions accordingly. One approach does not fit all community or country contexts. It is important to develop a responsive approach that considers the goals and needs of local communities, and the specific geography, socio-economic conditions, ethnic and linguistic composition of children to be served, cultural diversity, strengths and resource capacities of specific contexts. 3. To achieve quality, significant and highly visible support from government and development partners is needed for the first several years, while teachers and local language speakers are trained, teaching and learning materials in local languages are produced, and parents and local policy makers are socialized to understand and support MLE as a process that can serve children, communities and the country well. 4. Compromises are likely to be needed to establish local consensus on one variety of a particular language to be used for the programme, and in linguistically diverse communities, to select one or two predominant languages to start with, adding more as education capacity strengthens. 5. A gradual approach, with specific targets at different levels, will build capacity at a pace that resources and social processes can support. 6. A strong assessment process established before implementation will provide on-going feedback to inform course-corrections and accountability to communities, authorities governing early childhood programmes, and donors. 7. Collaboration and harmonization. Many stakeholders can contribute resources, expertise, encouragement and feedback during planning and implementation of MLE. A multi-stakeholder planning process that develops a pathway that combines strengths, avoids redundancies, and optimizes strategies at various levels can create a strong platform for multiplying buy-in and actualizing the potential of MLE to support inclusive education. 8. On-going system-level work is required to change attitudes and introduce MLE in teacher training colleges, and to convince policy makers about the importance of supporting MLE to
102 Independent Evaluation of the Multilingual Education National Action Plan in Cambodia
close equity gaps for Indigenous and other ethnolinguistic minority children and families, reduce conflict, build social cohesion, honour international agreements, and develop the economy.
Figure 21. Inputs to the success of MLE
103 Independent Evaluation of the Multilingual Education National Action Plan in Cambodia
Annex 17. Resources for teaching and learning in Indigenous languages created by International Cooperation Cambodia (ICC) Table 30 lists resources for learning and teaching available in six Indigenous languages spoken in north- eastern Cambodia. Table 31 lists the resources produced in Ratanakiri.
Table 30. Indigenous resources for learning and teaching produced by ICC
104 Independent Evaluation of the Multilingual Education National Action Plan in Cambodia
Table 31. Indigenous language resources produced in Ratanakiri
ICC book production title list for Ratanakiri No. Amount and title of Tampuan books produced Study Social Folk Easy Culture Life skills / Agriculture Dictionary Total book studies stories readers health 45 7 59 21 8 7 3 3 153
No. Amount and title of Kreung books produced Study Social Folk Easy Culture Life skills / Agriculture Dictionary Total book studies stories readers health 46 19 66 29 4 15 7 2 188
No. Amount and title of Brao books produced Study Social Folk Easy Culture Life skills / Agriculture Dictionary Total book studies stories readers health 40 18 186 37 14 12 6 2 315
No. Amount and title of Kavet books produced Study Social Folk Easy Culture Life skills / Agriculture Dictionary Total book studies stories readers health 9 7 20 9 2 0 2 2 51
No. Amount and title of Jarai books produced Study Social Folk Easy Culture Life skills / Agriculture Dictionary Total book studies stories readers health 2 1 88 10 1 0 0 0 102 809 Note: The Study book titles include Learn to Read and Big Books
105 Independent Evaluation of the Multilingual Education National Action Plan in Cambodia
Annex 18. Actor-focused theory of change and theory of action Evidence-based actor-focused theory of change after MENAP 2014–2018 and for drafting MENAP 2 The MENAP was not explicitly organized around a theory of change. An actor-focused theory of change is strongly recommended for MENAP 2. This following section brings together a summary of observations gathered in evaluating the MENAP, with recommendations for designing MENAP 2. It uses an explicit theory of change that specifies the various actors essential for a system of education that fulfils the rights of Indigenous and other ethnic minority girls and boys, so they can learn their language and be included in education that is relevant, meaningful and effective. This diagram depicts the social network for the described theory of change, expanding on the introduction and summary description in Volume 1. It includes supporting evidence for each component from the evaluation. Figure 22 shows current and potential relationships among actors in this system using dotted lines.
Figure 22. Overview of the actor-focused theory of change
For each actor in the theory of change, the evaluation gathered evidence of: • Emerging behaviour (yellow) • Observed behaviours (green) • Behaviours understood, not yet seen, but necessary for effective implementation of the MENAP (orange) • Behaviours that contradict behaviours necessary for effective implementation of the MENAP (red).
106 Independent Evaluation of the Multilingual Education National Action Plan in Cambodia
The sub-sections which follow describe the current status of each actor and group, and their expected role in a new MENAP, starting with children and other actors at the local level, then identifying the roles of local duty bearers and state duty bearers, and finally pointing to supportive functions of development partners and other actors in the education system.
Rights holders and actors at the local level Indigenous children in MLE primary schools Indigenous girls and boys are the primary beneficiaries of the MENAP and also key actors in the education system. In the evaluation, they explicitly asked to represent their rights, the benefits that had accrued to them to date through the MENAP, and to act as ambassadors of MLE beyond their local school and village. Indigenous children demonstrated observed and emerging changes in behaviour leading to the fulfilment of their right to quality education and social inclusion. Table 32. Indigenous children in MLE primary schools
Expected change (intermediate outcome) Current status
Reading and writing in the mother tongue proficiently (BICS Not seen. and CALPS). Children are learning to read and write in the mother tongue (BICS) but not yet reading and writing to learn in the mother tongue (CALPS)
Developing close relationships with teachers Observed
Expressing strong interest in mother tongue language and Observed culture
Competently engaging with wider society Observed
Accessing higher-level education, community leadership Emerging roles, and jobs
Confidently communicating and actively engaged in the Observed classroom (talking, group work, answering questions, etc.)
Attending school regularly Observed
Indigenous children in MLE preschools Indigenous children in preschool demonstrated none of the expected behaviours in the current evaluation. Preschool-aged children are important actors in the system, needing to be recognized as such for the sake of relevance and fulfilled rights to education for all, including Indigenous girls and boys.
Table 33. Indigenous children in MLE preschools
Expected change (intermediate outcome) Current status
Developing mother tongue oral language and communication skills Not seen Confidently attending Grade 1 after completing MLE preschool Not seen Developing close relationships with MLE teachers Not seen Actively engaged and communicating in the classroom Not seen Attending school regularly Not seen
107 Independent Evaluation of the Multilingual Education National Action Plan in Cambodia
Indigenous communities Indigenous communities demonstrated emerging and observed behaviours in support of MLE. Indigenous communities need to be encouraged and empowered to continue to amplify their crucial role in MENAP implementation. They need to advocate for their rights to ensure that programmes are relevant to their changing Indigenous cultures and their engagement in wider society. Table 34. Local Indigenous communities/guardians
Expected change (intermediate outcome) Current status
Supporting children to go to school Emerging
Recognizing the value of MLE to support Indigenous Observed. cultural identity Advocating for more cultural content and a longer and stronger MLE bridge (stronger foundation in mother tongue).
Recognizing the value of MLE to equip their children’s Observed engagement in wider society
Monitoring student enrolment and attendance through Observed. Expressing strong interest in school support committees mother tongue language and culture.
Monitoring teacher attendance Observed
Contributing labour and funds for school facilities Observed
Indigenous associations/organizations/networks Indigenous associations, organizations, and networks demonstrated a growing interest in formal education. They have been extensively involved in non-formal education. They have not yet been recognized as having a role in formal education, and are not yet familiar with the MENAP and MLE in primary schools. Future roles that would benefit MENAP relevance, effectiveness, efficiency, impact and sustainability include mobilization and accountability activities at local, provincial and national level, contributions to curriculum review and revision, and motivating MLE teachers.
Table 35. Indigenous associations/organizations/networks
Expected change (intermediate outcome) Current status
Engaging in decisions regarding Indigenous children’s education, including Not seen curriculum development at national level
Engaging in decisions regarding Indigenous children’s education at provincial and Not seen local levels
Mobilizing Indigenous communities in understanding and engaging in MLE Not seen
Representing the voices and insights of Indigenous groups regarding MLE Not seen effectiveness and relevance
Mentoring and reassuring teachers and school support committees that MLE is Not seen authorized by national government
Advocating for supportive policies and guidelines for implementation of MLE Not seen
Khmer children in MLE primary schools and preschools
108 Independent Evaluation of the Multilingual Education National Action Plan in Cambodia
Although not a focus of the current evaluation, testimonials in villages where Khmer children attend MLE preschools and primary schools suggest that Khmer children are achieving in school. They are important actors as primary rights holders and potential beneficiaries of opportunities to become bilingual through formal education. Involvement of Khmer children and families in MLE can have a significant effect on social harmony in villages and national social cohesion. Table 36. Khmer children in MLE schools
Expected change (intermediate outcome) Current status Khmer children
Learning Khmer and an Indigenous language Emerging, but not when there are two language streams in separate classrooms
Achieving in school Observed
Khmer community members/guardians Khmer community members demonstrated an emerging recognition of the value of MLE. Khmer community members have participated on school support committees. Khmer families also have a role to play in recognizing equitable, relevant and effective education for all children in the village.
Table 37. Khmer community members/guardians
Expected change (intermediate outcome) Current status
Recognizing the value of MLE to support Indigenous cultural Emerging identity Recognizing the value of MLE to help their children engage in Not seen wider society
Local duty bearers Communes Communes demonstrated support for MLE in both primary school and preschool, and support for education in general. They have a continued role to play in encouraging school enrolment, and advocating for, monitoring and ensuring effective MLE. Commune leaders can also contribute to mapping, including Indigenous language speakers, out-of-school children, birth rates affecting demand for preschool and primary school, and in-migration.
Table 38. Communes
Expected change (intermediate outcome) Current status
Taking financial responsibility for preschools Observed
Providing administrative support for preschools Observed
Engaging community members Emerging. Involved with facilities but not curriculum content.
Collaborating with POEs/DOEs to raise awareness of education Observed
MLE primary school teachers
109 Independent Evaluation of the Multilingual Education National Action Plan in Cambodia
MLE preschool teachers expressed confidence in providing instruction in the mother tongue. They uniformly demanded updating and expanding the cultural content of curricula and extending the number of years that the mother tongue is used as a language of instruction. Primary school teachers have a key role in the MENAP, as they are closest to ethnic minority children as rights holders. They can respond to children’s needs, ensure their voices are heard at other levels of the education system, and communicate challenges and opportunities to community members. They can share effective practices with other MLE teachers, extending the reach of innovative practices to other communities.
Table 39. MLE primary school teachers
Expected change (intermediate outcome) Current status
Confidently using the mother tongue as the language of Observed instruction Employing teaching techniques that use local cultural Observed resources and expertise Creating appropriate teaching and learning materials in the Emerging. Requesting more training in mother tongue, collaborating with others mother tongue literacy skills. Using the MLE curriculum Emerging. Using CARE curriculum but in some schools adding MoEYS standard curriculum as well. Communicating with parents regularly Emerging. Communicating with parents only about attendance. Developing close relationships with children Observed
MLE preschool teachers Little evidence was accrued that MLE preschool teachers are providing culturally relevant content in education or using Indigenous language in oral communication, reading or writing. A wide variety of practices are used across different preschools, as teachers express a lack of direction as to what MLE is and what it entails. MLE preschool teachers lack teaching and learning resources for MLE and lack knowledge, confidence, time, or the means to create their own resources. Preschool teachers play a crucial role in attracting Indigenous children to education and providing an understandable first experience of schooling in a language they understand. As a cadre, they could create a meaningful MLE curriculum, and teaching and learning resources, to achieve the target of quality in education for ethnic minority children.
Table 40. MLE preschool teachers
Expected change (intermediate outcome) Current status
Confidently using the mother tongue as the Emerging (in a minority of sites). Primarily using primary language of instruction Khmer to deliver preschool teaching and learning curriculum, and materials produced for Khmer contexts, inhibiting use of the mother tongue. Employing teaching techniques that use cultural Not seen resources and expertise Creating appropriate teaching and learning Emerging (in a minority of sites). Lacking mother materials in the mother tongue, collaborating with tongue skills and MLE knowledge to do this. others Using an appropriate MLE curriculum Contradictory. Using Khmer curriculum. Preschool teachers struggling.
110 Independent Evaluation of the Multilingual Education National Action Plan in Cambodia
Expected change (intermediate outcome) Current status Communicating with parents regularly Emerging Developing close relationships with the children Emerging
School directors School directors demonstrated that they might support or obstruct MLE depending on their understanding of it. They are in a pivotal role as primarily liaisons with parents, teachers and local and provincial authorities. With parents, communication can be strengthened to provide relevant education and increase impact. With government, school directors can provide accurate data and share findings of enabling factors, barriers, bottlenecks and innovative practices to improve the relevance, effectiveness, efficiency and sustainability of MLE.
Table 41. School directors
Expected change (intermediate outcome) Current status
Coordinating with local authorities and communities Observed regarding the value of education and MLE Building social cohesion through integration of Indigenous Contradictory. Providing separate and other children streams/classrooms for Khmer and Indigenous children Providing valuable and culturally relevant support to teachers Not seen. Requesting more training in in MLE teaching methodology MLE. Engaging with communities to provide practical, culturally Not seen relevant education Communicating with parents regularly Observed Exploring how out-of-school children can be reached Emerging. Data are not readily available. Collecting useful quantitative and qualitative data for MLE Emerging reports (identifying Indigenous identity) and communicating these
District Offices of Education DOEs demonstrated administrative support to MLE schools. They do not, however, provide significant support for students’ cultural and linguistic needs by being competent in Indigenous language and literacy skills, ensuring that core trainers are Indigenous (almost none are), or providing the means and authority to test learners in MLE in their Indigenous language. DOEs play a pivotal role in MENAP implementation, with the authority to request, manage and close MLE schools, to monitor enrolments and promotions, and gather data reflecting achievements.
Table 42. District Offices of Education
Expected change (intermediate outcome) Current status
Investigating expansion of MLE schools and Observed preschools Coordinating and liaising for MLE support and Not seen expertise with IP organizations and networks
111 Independent Evaluation of the Multilingual Education National Action Plan in Cambodia
Expected change (intermediate outcome) Current status Administratively supporting MLE schools Observed Engaging communities to get involved in supporting Observed MLE programmes Providing valuable and culturally relevant support to Requesting more training in MLE. Core teachers in MLE teaching methodology trainers requesting mother-tongue literacy skills. Some only communicate the value of Khmer literacy. Inspecting and testing children using tools appropriate Not seen for MLE Collecting data for MLE reports useful for learning Emerging about progress (identifying indigeneity)
Provincial Offices of Education POEs demonstrated a mixed picture of observed and emerging outcomes, as well as some outcomes not previously seen. Crucial steps have been taken to include Indigenous community teachers on government payrolls. An important role for POEs in the next MENAP is to engage with local Indigenous organizations and language networks to more effectively mobilize their support for MLE with advocacy, community liaison, teacher support, and expanded curriculum materials. POEs also need to engage Indigenous core trainers with Indigenous language proficiency.
Table 43. Provincial Offices of Education
Expected change (intermediate outcome) Current status
Coordinating and liaising for MLE support and expertise outside the education Emerging system (with other government institutions and NGOs)
Coordinating and liaising for MLE support and expertise with Indigenous Not seen organizations and networks
Including MLE-trained preschool teachers on government payroll Emerging (very few)
Including MLE-trained primary school teachers on government payroll by Observed allowing them to become contract teachers
Allowing contract teachers to become state teachers Observed
Taking administrative responsibility for MLE schools, giving them state school Observed status Designating core trainers for MLE Observed
Taking responsibility for training and monitoring of MLE schools Emerging
Inspecting and testing children using tools appropriate for MLE Not seen
Collecting data for MLE reports useful for learning about progress (identifying Emerging Indigenous identity)
Revising the system of deploying government teachers so that Indigenous Not seen teachers can serve their own communities or at least their own language communities
112 Independent Evaluation of the Multilingual Education National Action Plan in Cambodia
Expected change (intermediate outcome) Current status
Supporting a longer MLE bridge that ensures academic proficiency in both Not seen Indigenous and Khmer languages
Creating a more visible confirmation that MLE is allowed by MoEYS Not seen
Creating and advocating for supportive policies and guidelines for Not seen implementation of MLE
Provincial Teacher Training Colleges PTTCs were not included in the MENAP. In the current evaluation, PTTCs reported not having been trained in MLE, not doing MLE teacher training, and not mentoring MLE teachers. The PTTC in Stung Treng48 expressed strong interest and pointed to the large number of Indigenous trainees annually as a source of teachers for MLE. In the next MENAP, PTTCs could be lead actors in MLE teacher training by having Indigenous core trainers (perhaps those very experienced Indigenous trainers who worked for CARE but were decommissioned when SED took over), creating efficiencies and sustainability.
Table 44. Provincial Teacher Training Colleges
Provincial Teacher Training Colleges Current status
Providing relevant, quality MLE teacher training for all Indigenous teachers Not seen
Providing introductions to MLE to all teachers in north-eastern Cambodia Not seen
Duty bearers at the national level Ministry of Education Youth and Sport MoEYS demonstrated a mixed picture of observed, emerging, not seen and contradictory behaviour, noted in the table below. Administrative, reporting and legislative functions have been the focus of the MoEYS contribution to date. Some of the main roles needed in future are those of advocating for sufficient Programme Budget to expand and improve the quality of MLE, as well as coordinating efforts for curriculum improvements, monitoring and resource development with the Early Childhood Education Department and Primary Education Department, and leading efforts to harness the resources and capacities of local NGOs that have provided non-formal education for over two decades. SED needs to ensure ethnic minority children’s inclusion in relevant, quality education with demonstrated internal capacity for leadership in curriculum review, training, monitoring and support for meaningful Indigenous community engagement.
Table 45. MoEYS outcomes (SED, Primary Education Department, Early Childhood Education Department, Curriculum Development Department, Teacher Training Department, Finance)
Expected change (intermediate outcome) Current status
Coordinating and liaising for MLE support and expertise within Not seen government Coordinating and liaising for MLE support and expertise outside Emerging government systems Coordinating and liaising for MLE support and expertise with Not seen Indigenous organizations and language networks
48 The PTTC in Kratie also expressed an interest, but it currently has few Indigenous trainees.
113 Independent Evaluation of the Multilingual Education National Action Plan in Cambodia
Expected change (intermediate outcome) Current status Creating supportive policies and guidelines for implementing MLE Observed Allocating financial support for teacher salaries and Programme Observed Budgets Providing financial support for training Emerging Managing data collection and learning processes so that MLE Emerging programmes remain relevant (identifying ethnic identity and the mother tongue) Advocating for the need and purpose of MLE in Indigenous Contradictory. communities Communicating the purpose of MLE in terms of Khmer literacy/acculturation Leading a curriculum review Not seen Inspecting and testing children using tools appropriate for MLE Not seen Providing support for teaching communities of practice Not seen Coordinating quality teacher training Not seen Providing training of trainers/managers for MLE Emerging, however mother tongue/MLE expertise is limited Establishing a functional working group/department that can lead the Not seen MLE programming with innovative and efficient practices to support quality education Support a longer MLE bridge that ensures academic proficiency in both Not seen Indigenous and Khmer languages Coordinating processes for approving orthographies for languages that Not seen could be used for MLE Establishing a passionate and effective leadership team across Not seen MoEYS departments Supporting all levels of the education system to understand MLE Not seen Expand and incentivise scholarships for Indigenous students’ Not seen transition to lower secondary school, and for teacher training
MENAP supporting actors Cambodian universities and academic institutions Cambodian universities reported that they have not yet been part of the MENAP or the provision of MLE. For MENAP 2, these institutions could be engaged in research to support MLE, for example by helping to develop sound learning assessment tools in Indigenous languages, collaborating with international scholars on the design and conduct of methodologically rigorous studies of innovative practices and longitudinal student outcomes, contributing to PTTC teacher training pedagogy, and learning from the achievements of MLE/MENAP. This will increase the relevance, efficiency and sustainability of the MENAP.
Table 46. Cambodian universities/academic institutions
114 Independent Evaluation of the Multilingual Education National Action Plan in Cambodia
Cambodian universities/academic institutions Current status
Leading/conducting research on Indigenous language and orthography Not seen development Leading/conducting research on MLE teaching/curriculum Not seen Engaging with regional and international expertise and research on MLE Not seen
Development partners Development partners, notably CARE International and UNICEF, with support from several key donors, demonstrated their defining roles in laying the groundwork for the MENAP and implementing MLE in preschools and primary schools. Abundant evidence attests to their crucial roles providing leadership, funds and technical expertise, drawing on in-country and international expertise. Development partners have created delivery systems and built capacity for a gradual transition to government responsibility for MLE and other targets of the MENAP. Development partners will continue to play a crucial role in building up leadership roles and supporting the different levels of the system, as well as actors in the theory of change to fulfil their roles.
Table 47. Development partners
Expected change (intermediate outcome) Current status
Advocating for MLE Observed
Providing funds to various parts of the education system Observed Providing training opportunities for government education staff at Observed national level and provincial level (for primary and preschool) as UNICEF needed
Providing resources Observed
Practically supporting a functional working group/department that Observed can lead MLE with innovative and efficient practices to support quality education
Providing MLE teacher training Observed
Monitoring MLE teachers Observed Providing school support committee training Observed
CARE Leading and supporting research and evaluation initiatives Observed Liaising with government offices, development partners and Observed donors
Supporting monitoring and training done by POEs Observed
Supporting a review of MLE curriculum to update relevance and Not seen cultural content
Providing technical support for a pilot late exit model Not seen
115 Independent Evaluation of the Multilingual Education National Action Plan in Cambodia
Expected change (intermediate outcome) Current status
Global Building schools, paying for assistant teachers/teachers Observed Partnership for Education, United World Schools, Plan International
Kruesar Yoeng, Teaching MLE preschool teachers Observed Bondoh Koma
Production of teaching and learning resources in the mother Observed tongue ICC/NTFP/ other Engaging communities in non-formal education and adult literacy Observed development initiatives partners locally Support to the Royal Academy in orthography approval processes Observed connected to Indigenous people
Providing essential links to engage Indigenous networks in Not seen education
Next steps for MENAP 2 MENAP 2 needs to be designed with an explicit theory of change and associated theory of action, which clearly identifies actors responsible for implementing specific strategies and outcomes, and how they will be accountable for these at agreed-upon intervals distributed over the five-year period. The process of designing MENAP 2 needs to respond to the cultural and linguistic needs and goals of primary rights holders. Indigenous children have demonstrated their capacity and eagerness to contribute to goal setting and curriculum review. Indigenous peoples’ organizations and Indigenous core trainers need to be centrally involved, ideally as staff of SED or through technical support to SED. The theory of change needs to be revisited regularly and updated, as evidence from research and practice becomes available.
116