Burkina Faso

FINAL REPORT

Evaluation of the Education: Quality, Inclusive, Participative (EQuIP) Programme

PREPARED BY: INTERNATIONAL SOLUTIONS GROUP Helen O’Reilly Karishma Budhdev-Lama Brian O’Callaghan

PREPARED FOR: Plan Ireland

LIST OF ACRONYMS

AFDC Association for the Facilitation of Community Development (Burkina Faso) AGRICE Guinean Association for the Rehabilitation & Integration of Blind People AJDB Association Jeunes pour le Développement de Batié AME Mothers Association (Burkina Faso) AMI International Medical Assistance (Guinea Bissau) APAES Parents Association (Guinea Bissau) APE Parents Association (Burkina Faso) APRODEL Association for the Promotion of Local Integrated Development (Guinea Bissau) AVEC Association Villageoise d’Epargne et de Crédit BAC Baccalauréat BEPC Brevet d’Etudes du Premier Cycle du Second Degré (Burkina Faso) CCD Crianças Com Deficiência/Children With Disabilities CCEB Chef de Circonscription d´Éducation de Base CE1 Cours élémentaire 1ère année CE2 Cours élémentaire 2ème année CEB Circonscriptions d’Education de Base CECI Centre d'Etudes et de la coopération internationale canadienne CEP Certificat d'études primaires / Primary leaving examination CFA Central African Franc/Communauté Financière Africaine CM1 Cours Moyen 1ère année CM2 Cours Moyen 2ème année COGES Comités de gestion d'actifs CP1 Cours Préparatoire 1ère année CP2 Cours Préparatoire 2ème année CWD Children with disabilities DAC Development Assistance Committee DPENA Provincial Directorate for National Education & Literacy (Burkina Faso) DRE Direcção Regional da Educação (Guinea Bissau) DRENA Regional Directorate for National Education & Literacy (Burkina Faso) DR-MESS Direction Régionale Ministère des Enseignements Secondaire et Supérieur (Burkina Faso) EBE Educação Básico Elementar EQuIP Education: Quality, Inclusive, Participative FEC Fundação Fé e Cooperação - Portuguese NGO (Guinea Bissau) FIDA Fonds International de Développement Agricole (IFAD) GAP Groupe d’Animation Pédagogique (Burkina Faso) INDE National Institute for Education Development (Guinea Bissau) ISG International Solutions Group KII Key Informant Interview M&E Monitoring & Evaluation MENA Ministry for National Education & Literacy (Burkina Faso) MER Monitoring, Evaluation & Reporting MICS5 Quinto Inquérito dos Indicadores Múltiplos (Guinea Bissau) NGO Non-governmental organisation

International Solutions Group www.theisg.com 2 OCDE/CAD Comité d’aide au développement de l’Organisation pour la Coopération et le Développement PDSEB Programme de développement stratégique de l’éducation de base (Burkina Faso) PU Programme Unit PUM Programme Unit Manager RAJS African Youth Network for Health & Development (Burkina Faso) REACH Reading Assessment for Children REP Revised Education Policy UNESCO United Nations Educations, Science and Culture Organisation UNFPA United Nations Population Fund UNICEF United Nations Childrens Fund USAID United States Agency for International Development USD United States Dollar VSL Village Savings and Loans WARO (Plan) West Africa Regional Office

International Solutions Group www.theisg.com 3 TABLE OF CONTENTS

A. EXECUTIVE SUMMARY ...... 5

MAIN FINDINGS ...... 6 B. INTRODUCTION ...... 15

PURPOSE AND OBJECTIVES OF EVALUATION ...... 15 C. METHODOLOGY ...... 16

GENERAL METHODOLOGICAL APPROACH ...... 16 QUANTITATIVE METHODS ...... 16 Sample Calculation ...... 17 QUALITATIVE METHODS ...... 18 DATA COLLECTION PROCESS ...... 18 DATA COLLECTION SCHEDULE ...... 19 RISKS AND LIMITATIONS ...... 20 D. FINDINGS PER EQUIP OBJECTIVES ...... 21

OBJECTIVE 1: TO IMPROVE LEARNING OUTCOMES FOR CHILDREN IN TARGET SCHOOLS...... 23 OBJECTIVE 2: TO INCREASE ACCESS TO EDUCATION OF VULNERABLE CHILDREN, ESPECIALLY GIRLS AND CHILDREN WITH DISABILITIES. ... 29 Sub-Objective 2.1: To ensure girls enrol and complete quality primary and secondary education ...... 31 Sub-Objective 2.2: To create a conducive environment for inclusive education with a focus on children with disabilities...... 38 OBJECTIVE 3: TO IMPROVE CITIZENS’ ACTIVE PARTICIPATION IN EDUCATIONAL DECISION-MAKING AT ALL LEVELS ...... 43 OBJECTIVE 4: TO EFFECT CHANGE IN POLICY, FUNDING AND APPROACHES TO QUALITY INCLUSIVE EDUCATION AT A NATIONAL LEVEL. ... 45 E. EQUIP STRUCTURE AND PROGRAMME MANAGEMENT ...... 47 Plan Country Education Programmes...... 49 F. PROGRAMME MONITORING AND EVALUATION ...... 51 G. PROGRAMME PERFORMANCE AGAINST THE DAC CRITERIA ...... 53

RELEVANCE ...... 53 EFFICIENCY ...... 54 EFFECTIVENESS ...... 55 IMPACT ...... 57 SUSTAINABILITY ...... 57 H. CONCLUSION AND KEY RECOMMENDATIONS ...... 59 ANNEX 1: EQUIP RESULTS FRAMEWORK – BURKINA FASO ...... 62 ANNEX 2: EQUIP RESULTS FRAMEWORK – GUINEA BISSAU ...... 67 ANNEX 3: ASSIGNMENT TERMS OF REFERENCE ...... 72 ANNEX 4: EVALUATION ITINERARY ...... 78 ANNEX 5: LIST OF KEY PERSONS INTERVIEWED AND GROUP MEETINGS/FGDS ...... 80 ANNEX 6: DOCUMENTS CONSULTED ...... 82 ANNEX 7: DATA COLLECTION TOOLS ...... 83

International Solutions Group www.theisg.com 4 A. Executive Summary

This report presents the findings and recommendations of the final evaluation of the four-year “EQuIP” (Education: Quality, Inclusive, Participative) education programme implemented by Plan in Burkina Faso and Guinea Bissau. The report has been produced in conjunction with two separate country evaluation reports, in French for Burkina Faso and in Portuguese for Guinea Bissau. The EQuIP programme is focused on the three elements of the Right to Education: quality, access, and governance, which correspond to the following objectives:

Objective 1: To improve learning outcomes for children in target schools. Objective 2: To increase access to education of vulnerable children, especially girls and children with disabilities. Sub-Objective 2.1: To ensure girls enrol and complete quality primary and secondary education Sub-Objective 2.2: To create a conducive environment for inclusive education with a focus on children with disabilities. Objective 3: To improve citizens’ active participation in educational decision-making at all levels Objective 4: To effect change in policy, funding and approaches to quality inclusive education at a national level.

While four countries participate in the EQuIP Programme, due to logistical constraints and the ongoing challenges around the 2014/2015 outbreak of Ebola in West Africa, this evaluation was only conducted in Burkina Faso and Guinea Bissau. The evaluation’s overall purpose was to assess EQuIP programme performance against the results framework, taking into consideration its relevance, effectiveness, efficiency, sustainability, and impact. The findings will contribute to shared learning and accountability of achievements to partners, beneficiaries, and donors and inform the future development of EQuIP and similar initiatives. The evaluation included a desk review of relevant documentation, primary field research carried out in Burkina Faso in May 2015 and Guinea Bissau in April and May 2015, as well as secondary research of available data. In the interests of consistency and comparability, and in discussion with Plan it was agreed that the same qualitative and quantitative data collection tools would be used for this final evaluation as were used for the baseline survey in both countries (2012) and for the mid-term evaluation in Burkina Faso (2014). ISG prepared additional questions which were used to interview members of school governments, VSL groups, and children with disabilities in both countries. In Burkina Faso the evaluation covered 14 primary schools, 5 junior secondary schools and 1 secondary school, a total of 20 schools. All 99 schools in Noumbiel Province are included in the project, 92 at primary level and 7 at secondary level. No junior secondary or secondary schools had been included in the baseline survey or mid-term evaluation. In Guinea Bissau the evaluation covered 15 primary schools in the regions of Bafatá and Gabú, including two pilot schools providing inclusive education, one in each region. EQuIP covers 41 primary schools in Bafatá and Gabú, 35 of which only offer the first four grades at primary level. EQuIP does not cover any of the secondary schools in Guinea Bissau.

International Solutions Group www.theisg.com 5 Main findings EQuIP is an ambitious programme, with a theory of change broad enough in scope to leave no major gaps in what needs to be achieved to improve the quality of education. The success of such a programme depends on the effective implementation of a wide range of activities, which must be well managed by Country Coordinators and delivered by the Ministries of Education and other key partners, with the necessary follow-up and support. The programme is implemented in large part through the Ministries of Education in Guinea Bissau and Burkina Faso. Whilst the Ministries of Education in both countries have been active in designing and facilitating EQuIP training, and providing follow-up and support, there is a marked contrast in their capacity and therefore the outcomes achieved and level of ownership in each country. In Guinea Bissau, whilst Regional Inspectors at primary level and personnel from INDE, have been actively involved in designing and facilitating training, the Ministry of Education does not have the capacity to provide the necessary level of follow-up and support required. Due to the very high level of staff turnover, many of the head teachers and teachers who had benefitted from training are no longer working in EQuIP schools. In Burkina Faso the Ministry of Education staff at primary level have a strong sense of ownership of the programme and see EQuIP as providing them with inputs and resources to tackle key issues, such as teaching reading, using Ministry structures and systems. The Ministry of Education in Noumbiel has the capacity to provide follow-up and support to schools after training. Other strengths are that all the teachers are proficient in the training language, French, and there are signed contracts with EQuIP in place, laying out roles and responsibilities and project activities to be covered. Key personnel within the Ministry of Education, including the Primary Inspectors, generally have the capacity to implement activities and follow up as necessary. EQuIP works in partnership with national organisations in each country which bring their specific areas of expertise to implement key components of the programme. In Guinea Bissau, APRODEL supports the Village Savings and Loans (VSL) component of the project, and VSL groups are active in 38 out of the 41 EQuIP communities. The VSL programme benefits from having the Plan country VSL specialist based in the Programme Unit (PU) in Bafatá. AGRICE supports inclusive education, designing and facilitating training and providing support to students with disabilities and their teachers in the pilot schools for inclusive education in Bafatá and Gabú. In Burkina Faso, AFDC supports the VSL (AVEC in French) component of the project, and VSL groups are active in three out of the five communes in Noumbiel province. The African Youth Network for Health and Development (RAJS) provides sexual and reproductive health education inputs at post-primary level. Political upheavals in both countries have affected project implementation, particularly in Guinea Bissau. Between the coup d’état in 2012 and June 2014, when a new president took office, the education sector was affected by a succession of teachers strikes, delayed salaries and high staff turnover at both regional and school level, which all affected programme implementation and effectiveness. The popular uprising in Burkina Faso in November 2014 also resulted in disruption to the programme, including an even higher

International Solutions Group www.theisg.com 6 turnover of staff than usual in the province of Noumbiel and major delays with the provision of foodstuffs to school canteens. In terms of EQuIP programme management by the Country Coordinators, the evaluation noted differing levels of effectiveness between the two countries, as follows: In Guinea Bissau, the Country Coordinator is based in Bissau, even though there are two PUs in the project area, in Bafatá and Gabú. It was found that all aspects of programme management are less effective because the Country Coordinator is based too far from the programme area, leading to unnecessary delays in communications and missed opportunities for facilitating networking, coordination and cooperation. All the project partners consulted stated that the project would be more effective if the Coordinator were based within the EQuIP programme area. The Country Coordinator needs to be in regular contact with all the stakeholders involved, including staff from the two PUs and the two Regional Education Offices. There are other international organisations providing training and support for education in Bafatá and Gabú, and coordination and cooperation with them is essential in order to ensure EQuIP’s inputs are as focussed and effective as possible. In Burkina Faso the Country Coordinator is based near the project area, receives a good level of support from the PU in Gaoua, and is managing the programme effectively. Progress against programme objectives Objective 1: To improve learning outcomes for children in target schools The evaluation found that learning outcomes at primary level, as reflected in the pass rates for exams at the end of the primary cycle and the REACH reading scores, have improved in both countries. However, the REACH results revealed disparities in standards across EQuIP schools in both countries which need to be addressed. Guinea Bissau Due to the high level of staff turnover many of the head teachers, deputy head teachers and teachers who have benefitted from training under EQuIP are no longer in post. Of the 109 teachers in the schools surveyed only 10 participated in EQuIP related training. Even though steps have been taken by Plan to improve data collection and management, there was insufficient data to effectively monitor EQuIP learning outcomes. Basic in-school data was unavailable to the evaluation team in six of the 15 schools covered by the evaluation, which head teachers said was due to the fact that they had recently been transferred and no handover of records for them to refer to. It is not possible to determine exactly how much of the improvement in learning outcomes that was noted via exam pass rates can be attributed to EQuIP and how much to inputs from Fundação Fé e Cooperação (FEC), a Portuguese NGO which is also working to raise the quality of education in Gabú and Bafatá. No overall improvement was observed in the use of child-friendly and participatory teaching methods. This is likely due to the overall focus of the EQuIP-delivered training for teachers being on curriculum content rather than on pedagogy/teaching methodology, due to the very low baseline skills of teachers,

International Solutions Group www.theisg.com 7 and also due to the high level of teacher turnover, which dissipates programme gains and keeps skill levels at a low average baseline. Burkina Faso The early reading scheme introduced by EQuIP is considered by teachers, head teachers, inspectors and regional officials to be very successful, and they are advocating for the programme to be scaled up. The CEP exam success rate results have shown strong improvements in Noumbiel and are above regional and national success rates. However, the BEPC exam success rate at junior secondary level in the region and at national level has fallen, and is particularly poor for girls. The BAC success rate is also very low, at only 16.6%. These reflect the limited inputs related to improving learning outcomes at post primary and secondary level. The number of teachers in target schools using child-friendly and participatory teaching methods has risen substantially in Noumbiel, as much of the training for them was focussed on improving their pedagogy/teaching methodology. Thus, this element of the programme has been successful. Objective 2: To increase access to education of vulnerable children, especially girls and children with disabilities. Qualitative evaluation data indicates that EQuIP has indeed been successful in sensitising communities and increasing access for all, including the two main target groups: girls and children with disabilities. Although household surveys conducted by the evaluation team regarding indicated decreases (in both countries) from the baseline figures regarding the proportion of people indicating awareness of the rights of children with disabilities, and the proportion of parents and guardians who declared enrolling their children in school regardless of their gender, differences in analysis between the baseline, interim and endline surveys may account for these discrepancies. Overall, while it does not appear that the final target was achieved, a positive trend was established. More girls than boys now enrol in primary schools in both countries, and the gender parity targets set for the programme have been achieved, at 0.84 for primary and junior secondary and 0.39 for secondary in Noumbiel and 1.06:1 in both regions in Guinea Bissau. There are now 146 children with disabilities (64 girls and 82 boys) enrolled in the 20 schools covered by the evaluation in Noumbiel and 71 (18 girls and 53 boys) enrolled in the 15 schools in Guinea Bissau, including the two pilot schools for inclusive education which have been established under EQuIP. Assuming that the level of disability is equally distributed among the sexes, more work therefore is needed to be done to bring girls with disabilities in to schools. The evaluation team saw no disaggregated data on learning achievements and progress of children with disabilities, but, anecdotally from the pilot school in Gabú, Burkina Faso, all the children with disabilities repeated the prior year, indicating poorer-than-average academic performance amongst this group. In both countries externalities, such a lack of infrastructure to cope with the demands of the growing school population, are a major constraint. In Guinea Bissau this problem has been exacerbated by the Ministry of Education reducing the maximum number of shifts in primary schools from three to two for the academic year 2014/2015.

International Solutions Group www.theisg.com 8 There are still an undetermined number of children out of school in both countries, including girls, children with disabilities, child workers, orphans and Koranic school students (who study the Koran and receive very little additional education). Some of these children have never entered the education system, others have dropped out. Sub-Objective 2.1: To ensure girls enrol and complete quality primary and secondary education More girls than boys now enrol in primary schools in both countries. However, dropout rates for girls remain high, the reasons for which include distance to school, early marriages/pregnancies, lack of a school canteen, and child labour due to poverty. However, dropout rates have fallen dramatically for girls in Guinea Bissau; as have the failure rates, particularly in the 6th grade. In Guinea Bissau 35 out of the 41 EQuIP schools only offer the first four grades, so many girls are unable to continue into the fifth and sixth grades. For 2013/14 the transition rate for girls from 4th to 5th grade was 51%. All EQuIP primary schools in Burkina Faso offer the 6 primary grades. In both countries access to post-primary education is constrained by long distances between many homes and the school, necessitating long walks for children that inhibit many from attending. There has been a rise in the proportion of students who think that school offers a gender-equal environment, Results in both countries are very positive in terms of gender equality and respect for differences. Due to over-age enrolment, many girls are of child-bearing age before completing their primary education. In both countries there is need for sexual and reproductive health education to be provided from the last two years of the primary cycle onwards. Pregnancy rates are high in Noumbiel, despite the input on sexual and reproductive health at post-primary level, but the majority of young mothers are returning to school after they have given birth, to continue their studies, a positive trend. Membership of VSL groups, some of which directly support EQuIP schools, has helped to bring more children in to school and to reduce the number of dropouts among both sexes. In addition to generating funds to help cover education costs, VSL members also play a valuable role in awareness raising and sensitisation, supporting school management committees, parents’ associations and Plan community development facilitators in mobilising community members to participate in a range of activities. VSL groups are active in all but three of the EQuIP communities in Guinea Bissau, but have only been introduced in three out of the five communes covered by EQuIP in Burkina Faso. Sub-Objective 2.2: To create a conducive environment for inclusive education with a focus on children with disabilities. Training on inclusive education has been provided in EQuIP-supported schools in all programme countries. This training has been delivered to a broad range of stakeholders, including over 400 teachers, inspectors, members of school management committees and parents’ associations, regional staff and some head teachers. Teachers in the two pilot schools have received training, and in May 2015 2 days training was provided for 53 teachers. Observation of classroom practice for the evaluation showed strong positive performance among teachers with respect to inclusion, with 90% of primary teachers in Burkina Faso taking inclusion and participation

International Solutions Group www.theisg.com 9 into consideration when preparing their lesson plans and activities, though at 52%, it was less in Guinea Bissau, reflecting the high level of teacher turnover which dissipates training-related gains. In both countries head teachers, teachers and students appeared to be very supportive of children with physical disabilities in school. However, stakeholders surveyed expressed a general consensus that the necessary conditions were not in place for students with sensory or intellectual disabilities to receive a quality education, due to the lack of specialised training for teachers and resources. Even in the two pilot schools in Guinea Bissau, where specialised learning resources and support are in place, teachers reported they did not feel they had the necessary skills or sufficient resources. Students with disabilities pointed out problems they had accessing and using bathrooms and lack of specialists in sign language. Problems with transport to and from school is a major cause of dropout for children with disabilities. More consideration needs to be given to the health and other requirements of children with disabilities, including access to aids such as glasses or hearing aids. Collaboration needs to be heightened with the Ministry of Health in both countries, and with the Ministry of Social Action in Burkina Faso, to provide the support needed by children with disabilities and their families. Objective 3: To improve citizens’ active participation in educational decision-making at all levels The evaluated team recorded increases in the number of functioning parent’s associations (87% Guinea Bissau; 80% Burkina Faso), school management committees (100% Guinea Bissau; 90% Burkina Faso) and school governments in both field visit countries. Some of these groups are very proactive in identifying issues for support, but the level of activity and participation in school management and decision-making varies greatly across schools. Evidence from the evaluation also indicated that resolution of problems is still a challenge to these bodies, for example resolving poor teacher attendance or the quality of education being delivered in a school. The existence of these bodies within a school community does not necessarily equate to students or parents being sufficiently empowered to resolve issues, indicating the need for further support to these groups. The evaluation noted strong improvements in the learning environment across both field visit countries, with a rise in the number of students who say they feel safe and secure at school. Objective 4: To effect change in policy, funding and approaches to quality inclusive education at a national level. EQuIP is in line with the issues of inclusive education and improving the quality of education as highlighted in the current education sector plan for Burkina Faso. Plan Guinea Bissau reports participation in the development of the national sector plan for 2016-2025. However, the evaluation team saw few documented outcomes of evidence-based advocacy from EQuIP, suggesting that further focus on this objective is warranted. It should be noted that the political environment in both field visit countries is very challenging, with both countries having undergone military coups in the recent past, both suffering from chronic underinvestment in the education sector (including divestment of support from major donors). In Burkina Faso, the Ministry of Education at Provincial and Regional level is proactive in trying to scale up the early reading initiative introduced by EQuIP, not only within the region but also with a view to

International Solutions Group www.theisg.com 10 extending initiatives to a national level. This is one example of positive progress and should be followed- up on by Plan. Overall, limited progress has been made against this objective, and inputs need to be strengthened for evidence to be documented and used to inform national policy development and implementation.

Monitoring and evaluation The evaluation team noted ongoing challenges with the availability of data needed for project monitoring, evaluation and reporting (MER). Despite the inputs already made to address the lack of data in Guinea Bissau the problem is still acute. As noted above, even though EQuIP has trained head teachers and their deputies in data collection and management, high turnover/poor handover has led to basic data being unavailable in six of the 15 schools covered by the evaluation. The Ministries of Education do not have the resources they need to be able to collect, analyse and disseminate data within the necessary timeframes for EQuIP, nor in enough detail. Available data does not cover all areas of programme intervention, and is not sufficiently disaggregated for EQuIP’s purposes. For example, data on learning achievements is not readily available disaggregated by type of disability. More emphasis needs to be given to monitoring the progress and achievements of students with disabilities, both producing disaggregated data and spending time with children individually to document and follow up on the individual difficulties they face in accessing a quality education. Monitoring, evaluation and learning within EQuIP, the MER departments in the PUs and the MER departments at country level require additional resources. At PU level, as monitoring and evaluation coordinators are responsible for all programmes, monitoring and evaluation is mainly limited to documenting outputs/activities, rather than EQuIP outcomes. The majority of the students and parents interviewed in both field visit countries were unfamiliar with EQuIP as a specific programme (although they were participating in/benefiting from the programme activities), suggesting a limited level of participatory monitoring of programme progress (i.e. involving the wider community). Community participation at all stages of programme planning, implementation, monitoring and evaluation should be strengthened. This is in line with Plan’s child-centred community development approach and will also raise the currently low profile and understanding of EQuIP at community level The results-based management framework has been revised and updated, on an annual basis and in line with other information received such as the mid-term evaluation of EQuIP in Burkina Faso. However, the results of this final evaluation indicate more limited progress against many indicators than had been recorded to date, and even regression from the baseline position in some cases. This final evaluation indicates that a number of Year 4 targets have not been met. In order to minimise differences in measurement/calculation of indicators, the final evaluation team utilised the same evaluation and analysis tools as had been applied for the baseline in both countries and for the mid-term evaluation in Burkina Faso. Despite this, differences in sampling (selection biases), application of tools (response biases) and

International Solutions Group www.theisg.com 11 analysis may have influenced results and margins of error. The lack of availability of the original baseline survey team to respond to analysis questions contributed to this limitation. Further resources need to be made available to facilitate effective participatory programme monitoring against all the objectives indicators in the results-based framework, to inform programme planning and development.

Main recommendations For immediate action: Guinea Bissau and Burkina Faso

 community participation in all stages of programme planning, implementation, monitoring and evaluation should be strengthened. This is in line with Plan’s child-centred community development approach and will help to ensure EQuIP evolves to meet beneficiaries’ needs in the most effective way. It will also raise the profile and understanding of EQuIP at community level.

 Objective 1: Given the enormous disparities in the REACH reading results across the schools covered, REACH tests should be conducted across the remaining EQuIP schools, and the lowest performing schools should benefit from an intensive reading programme, to help the teachers improve their teaching of reading and to raise the reading levels of the children. Primary inspectors should be supported to monitor reading results across their schools on a regular basis, and to take further action as necessary.

 Objective 1 & sub-objective 2.2: More emphasis should be placed on developing individual learning plans for students with disabilities and on documenting their learning outcomes, so disaggregated data is available to measure progress and inform planning.

 Sub-objective 2.2: In-depth interviews should be conducted with a range of children with disabilities, including both those in school and a sample of those who have dropped out, and their families, to determine how disability-friendly schools are for those children.

 Objective 2: Additional sensitisation needs to take place to bring more girls with disabilities in to schools, Guinea Bissau

 General: The Country Coordinator should be located within the project area.  Objective 1: In-service training for teachers should continue to be provided on a regular basis throughout the academic year, through the fortnightly cluster meetings, in collaboration with other organisations providing teacher training , follow-up and support, including INDE, UNESCO, GPE and FEC.

 Objective 1: Specialist input on teaching and learning in a bilingual or multilingual environment should be included in training programmes for teachers, to improve learning outcomes.

International Solutions Group www.theisg.com 12  Objective 1: EQuIP should collaborate with the EU-funded data management project to support the Regional Education Directorate in Gabu (which is being run in partnership with FEC). Objective 1

 Objective 2: EQuIP should approach the Ministry of Education, at regional and central levels, to see if the third shift can be reinstated in schools where this is needed to accommodate the number of children wishing to attend.

 Objective 2: Greater coordination and collaboration is required with the Ministry of Health, to provide the necessary support for children with disabilities.

 Objective 2.2: The recently built resource centre in Gabu should become the focus for resources and activities to support inclusive education, with a focus on children with disabilities  Objective 4: EQuIP should increase focus on evidence-based advocacy to feed in to the current development of the national sector plan for 2016-2025, and to inform national policy development and implementation. Burkina Faso

 General: EQuIP should provide additional support to facilitate communication (by phone and email) between Batié and the PU in Gaoua.  Objective 1: Discussions should be held with the Ministry of Education, at provincial and regional level, regarding the feasibility of re-instating the monthly training and planning sessions (Groupes d’Animation Pedagogique - GAP) which used to be held for primary teachers, or supporting another format if not.

 Objective 1: At post-primary level in Burkina Faso EQuIP should provide support as necessary, to improve the timely supply of adequate supplies of textbooks to schools in Noumbiel from central stocks, and the most equitable distribution of textbooks to students in all secondary schools.

 Objective 2: VSLs should be extended to the two communes where they are not currently operating.

 Sub-objective 2.1: Education Assistants working in “Service Vie Scolaire” in secondary schools should be included in training and follow-up activities related to pregnancies, children with disabilities and other vulnerable children.

 Sub-objectives 2.1 & 2.2: There should be greater coordination and collaboration with the Ministry of Health, and also with the Ministry of Social Action, to provide the necessary support for children with disabilities, pregnant schoolgirls and other vulnerable children. For future development of the EQuIP programme: Guinea Bissau and Burkina Faso

 General: Key staff from the Ministries of Education and other partner organisations should be trained in participatory monitoring and evaluation

International Solutions Group www.theisg.com 13  Objective 3: EQuIP should support students and parents dealing with difficult issues such as unacceptable behaviour by teachers and head teachers. This should include facilitating the participatory development of codes of practice for each school with guidelines on actions to be taken if the school code of practice is broken.

 Objective 2: More research needs to be done to establish how many children are still out of the system, the barriers preventing them from accessing education, and strategies developed to bring them in to schools and keep them there. Guinea Bissau  Objective 2: EQuIP should advocate for the provision of the first six grades at all primary schools. Currently 35 out of the 41 EQuIP schools only offer the first four grades.  Objective 2: EQuIP should support any collaboration and coordination with Koranic schools/madrassas (dependent on policy decisions made in future by the Ministry of Education)

 Objective 2.1: EQuIP should develop a partnership with an organisation which can facilitate the introduction of sexual and reproductive health materials in the last two years of the primary cycle, subject to agreement from the Ministry of Education, religious leaders and other community leaders. Burkina Faso

 Objectives 1, 2 & 4: EQuIP needs to consider how it can best support the development of “the Continuum”, integrated complexes for the provision of pre-school, primary and secondary education.

 Objective 1: Input to improve learning outcomes at post-primary and secondary level should be expanded, to reduce the high repetition rate and improve the exam success rates at post-primary and secondary level, particularly for girls.

 Sub-objective 2.1: sexual and reproductive health education should be provided for the last 2 years of the primary cycle onwards, as well as at post-primary level, in conjunction with RAJS. Subject to agreement from the Ministry of Education, religious leaders and other community leaders.

International Solutions Group www.theisg.com 14 B. Introduction Purpose and objectives of evaluation Four countries benefit from Plan Ireland’s Education: Quality, Inclusive, Participative (EQuIP) Programme in West Africa: Guinea Bissau, Burkina Faso, Sierra Leone and Guinea. The EQuIP programme is focused on the three elements of the Right to Education: quality, access and governance, which correspond to the following objectives:

The overall purpose of this final evaluation is to assess the performance of the programme against the results framework, taking into consideration its relevance, effectiveness, efficiency, sustainability and impact. The specific objective of this final evaluation is to provide the decision-makers in the Government of Burkina Faso, Guinea Bissau, Irish Aid, Plan Ireland and the wider public with sufficient information to: a. Make an overall independent assessment about the past performance of the programme, paying particular attention to the impact of the programme against its objectives; b. Identify key lessons and propose practical recommendations for follow-up actions. The evaluation findings will contribute to shared learning and accountability of programme achievements to partners, beneficiaries and donors. Good practices, lessons learned and challenges experienced/ strengths and weaknesses during implementation are analysed with a view to extracting lessons and best practices for the future and recommendations will help to guide and inform future similar projects and programmes. A reconciliation of the EQuIP results frameworks for both Burkina Faso and Guinea Bissau, comprising a comparison of the baseline data, the endline data (from this evaluation) and programme targets, is presented in Annex 1 and Annex 2. The terms of reference (ToR) for the evaluation are included in Annex 3.

International Solutions Group www.theisg.com 15 C. Methodology General Methodological Approach The evaluation consisted of a desk review of relevant documentation, primary field research in mid-2015 in Burkina Faso and Guinea Bissau and secondary research of data available for both countries. While the programme was implemented across four countries, the final evaluation field research was only conducted in Burkina Faso and Guinea Bissau due to the 2014/2015 West Africa outbreak of Ebola virus in the other countries. A baseline survey was conducted in 2012 in both Burkina Faso and Guinea Bissau at the start of the programme to determine baseline values of key indicators related to programme objectives. A mid-term evaluation was also conducted in Burkina Faso, with a report produced in April 2014. With a view to achieving strong levels consistency and comparability, and facilitate comparisons between commencement and completion of the programme, ISG used the same quantitative and qualitative research tools that were applied for the baseline survey, and also for the mid-term evaluation in Burkina Faso. However, incomplete documentation of the sampling and research process by the baseline survey team (and lack of responses to the final evaluation team’s efforts to contact the baseline survey experts) served as a limitation to exact replication of the tool implementation and analysis. Risks and limitations to the research (and mitigation strategies) are discussed further below. ISG’s lead consultant served as team leader for the evaluation in both countries, with a national consultant in charge of logistics and management in both field locations. In addition, the national consultant for Guinea Bissau also joined the team in Burkina Faso, in order to ensure consistency in data collection and treatment across the two countries. The national consultant in Burkina Faso was contracted to help support the evaluation, and was involved in the initial part of the data collection process. Four national enumerators were contracted in Guinea Bissau for data collection and six in Burkina Faso. In total, a sample of 35 schools was selected. In Burkina Faso this consisted of 20 out of the 99 schools covered by EQuIP, 14 primary, five junior secondary and one secondary. In Guinea Bissau this consisted of 15 of the 41 EQuIP schools, all at primary level. Discussions were also held with Plan and EQuIP staff at the Country Offices in Bissau and Ouagadougou and with staff in the PUs in Gaoua (Burkina Faso), Bafatá and Gabú (Guinea Bissau). In conducting this final evaluation, ISG therefore utilised the following methods: Quantitative Methods . REACH reading assessments . School equality scorecards . Classroom observations . Household surveys . Analysis of educational performance data (programmatic and Government) ISG also added some additional questions in some of the tools to provide greater clarity or depth of responses.

International Solutions Group www.theisg.com 16 Sample Calculation For the purposes of the quantitative portion of the evaluation, the schools participating in EQuIP across both countries (99 for Burkina Faso; 41 for Guinea Bissau) were considered to be the two distinct populations to be sampled from. Given that a control group of schools was not established during the programme baseline, a comparison group of non-participating schools was not selected. Counterfactual data (i.e. data on what would have happened in the absence of the program) was instead gathered via, and triangulated between, the qualitative element of the evaluation and review of quantitative information already available in the programme countries – specifically national education statistics from the Ministries of Education. The following formula1 was used to calculate the sample size (n) for the survey:

c 2 x = Z( /100) r(100-r) N x 2 n = /((N-1)E + x) (N - n)x E = Sqrt[ /n(N-1)]

where N is the size of the population under scrutiny (in this case, schools in the two programme countries), r is fraction of relevant responses, and Z(c/100) is the critical value for confidence level c, which represents how often the true percentage of the population who would pick an answer lies within the confidence interval. The following samples were selected using the above formula with the following parameters:

Burkina Faso Guinea Bissau Population to be sampled 99 schools 41 schools Margin of Error2 13% 14% Confidence Level3 90% 90% Response Distribution4 80% 80% Sample Size 20 15

In total, a sample of 35 schools was selected. This consisted of 20 schools in Burkina Faso, and 15 schools in Guinea Bissau (representing 20% and 30% of total participating EQuIP schools, respectively). Note that in the baseline survey, sample selection of schools was undertaken without reference to the overall population of schools in the programme areas, thus limiting the validity of the sample. This evaluation, while utilising margins of error of 13-14% and confidence levels of 90%, is expected to achieve levels of statistical validity at least equal to, and likely greater than, the baseline survey.

1 From Rea, Louis, Designing and Conducting Survey Research, Jossey-Bass; 2nd edition, 1997 2 the amount of error that is allowed, e.g. 70% of respondents indicate ‘yes’ with a margin of error of +/- 16% 3 How sure we can be that the response we receive to a question is representative of the overall population 4 A measure of how variable the likely responses are to be. Given a largely positive mid-term review of EQuIP, it is anticipated that the variation will be skewed towards positive findings, hence the higher value of 80%. A predicted random distribution of findings (or if the variability was unknown) would dictate a response distribution of 50%)

International Solutions Group www.theisg.com 17 Qualitative Methods . Focus group discussions: groups of key stakeholders, including students, teachers and parents . Key Informant Interviews: of a range of stakeholders, including EQuIP and Plan staff at CO and PU levels, regional and provincial education officials, head teachers, inspectors, staff in partner organisations, children with disabilities, members of VSL groups and school governments . Community meeting in Coiada, Guinea Bissau . Desk Review: including reports on the EQuIP programme, national policy documents for Burkina Faso and Guinea Bissau, relevant research commissioned by Plan and Plan policy documents, strategic and planning documents and analysis of data gathered as part of the original baseline. The evaluation team prepared specific questions for use in focus group discussions or individual interviews with members of school governments or other student bodies, members of VSL groups and children with disabilities. These were additional questions prepared by the team, to be used if time allowed. The specific tools used for the evaluation are provided in Annex 7. Data Collection Process Initial data collection (in Guinea Bissau) entailed focus group discussions held with groups of ten girls or boys, and school equality scorecards administered to the same groups. After feedback and discussion on that first day, the evaluation team decided to reduce the number of students participating in the focus group discussions and responding to the school equality scorecard from ten to five per group, which was found to be more effective. The team in Burkina Faso also decided to limit the student groups to five. To promote frank responses, discussions with the students (girls and boys) were held in separate groups, and the students were chosen at random. In addition, discussions were also held with members of VSL groups, children with disabilities, and members of school governments/other student bodies. In every school visited the team interviewed students with disabilities. In Guinea Bissau, we ensured that one of the enumerators could use sign language, and also selected teams of enumerators in both countries with a range of local languages, to facilitate communication. Some children were able to communicate with us directly, with others we communicated through other students or members of staff. We were not able to elicit a response from some students. A total of 34 head teachers were interviewed. In one school the head teacher was absent at the time of the evaluation visit. In Burkina Faso two additional activities were added to the schedule. The team leader met with a group of 17 young teachers who were attending training on multi-grade teaching in Bonkossera , and she was also invited to address a group of teachers from Batié II attending the 3-day education conference for primary teachers in Noumbiel, which gave her the opportunity to talk briefly about the evaluation and some of the findings. In Guinea Bissau, in addition to the planned activities, the team was able to facilitate a community meeting in Coiada which was attended by over 60 community members.

International Solutions Group www.theisg.com 18 Meetings were held with EQuIP and Plan staff in the two Country Offices, at the beginning and part-way through the evaluation in Guinea Bissau, and at the beginning of the evaluation in Burkina Faso. Meetings were also held in the PUs in Gabú and Bafatá (Guinea Bissau) during the evaluation, and in the PU in Gaoua (Burkina Faso) there were a series of meetings at the beginning and end of the evaluation. A full breakdown of the field visit itineraries including persons consulted in both countries is provided in Annexes 4 and 5. Please see Annex 7 for copies of the data collection tools. Data Collection Schedule The data collection schedules were prepared and shared with Plan staff in both countries prior to the fieldwork taking place. The schedule for Guinea Bissau was followed as planned, but the schedule for Burkina Faso had to be amended. On Monday 4 May 2015, it was announced that primary schools in Noumbiel would be closed for three days, from Monday 11 to Wednesday 13 May, to allow teachers to attend a three-day conference. This took place at the time of ISG’s field visit. Therefore, the schedule was adjusted in consultation with Plan staff, with as many school-based activities as possible being carried out in primary schools prior to their closure, and ISG’s enumerators returning to some communities later to complete data collection which could be done outside school. This included the implementation of household surveys. One of the original schools included in the sample, a primary school, was replaced by a secondary school. As planned, a total of 20 schools were covered by the evaluation in Burkina Faso. Because of security issues in Noumbiel, the evaluation team was accompanied by an armed police escort. This led to some delays and unanticipated time constraints, which affected the number of household surveys which could be carried out in particular. Despite these challenges, most of the expected total sample was achieved. In addition, the team collected other data which was not included in the original terms of reference, from members of school governments, VSL groups, and children with disabilities.

International Solutions Group www.theisg.com 19 Risks and Limitations The evaluation team notes the following risks and limitations to the evaluation, with corresponding mitigation strategies:

Risk/Limitation Mitigation

Research was conducted in diverse linguistic and The evaluation team utilised national cultural settings, offering challenges to enumerators and consultants. In addition, understanding/aggregating data and generalising programme staff with understanding of the conclusions across the programme. stakeholders and context of Plan’s programming accompanied much of the research. Differences in sampling (selection biases), The final evaluation team utilised the same application of tools (response biases) and analysis evaluation and analysis tools as had been applied between the baseline, midline and endline may for the baseline in both countries and for the have influenced results and margins of error. mid-term evaluation in Burkina Faso. The lack of availability of the original baseline In addition, triangulation of the findings from the survey team to respond to analysis questions field research across multiple sources contributed to this limitation. (stakeholder categories), methods (individual and group interviews), and investigators, was used to cross-check increase the reliability and validity of findings and conclusions. Only two countries (of four EQuIP countries) were No mitigating strategy for this limitation/risk was selected for field visits, leading to a risk of bias or available, and evaluation results should be insufficient data to extrapolate across the reviewed with this limitation in mind. programme. The Results Framework (RF) Indicators may not Analysis of supporting programme always represent the complexity of the processes documentation and flexibility in approach to the being assessed such as the integration of sectors research at individual country level and on targeted for a variety of stakeholders within the combining quantitative and qualitative individual EQuIP programme country data. Limited resources among the evaluation team The evaluation team focused on analysing the meant that not all areas of research of relevance main areas of research as outlined in the scope to the evaluation could be explored in depth. of work. Unavailability of previous staff may hamper Triangulation of findings from multiple sources to institutional memory beyond the written reports ensure cross-validity and capture robust data.

International Solutions Group www.theisg.com 20 D. Findings per EQuIP Objectives

The EQuIP programme is focused on the three elements of the Right to Education: quality, access and governance, which correspond to four objectives and two sub-objectives. This section explores the findings of the evaluation against each of the objectives and sub-objectives and provides recommendations under each. We acknowledge that some findings span across objectives, however, for the purposes of this report, we have placed each finding under the most appropriate objective. Some issues affecting programme implementation and outcomes Political events and implications for EQuIP Achievements against objectives must be considered in the context of political events in both countries since EQuIP’s design and inception. Overall, the socio/political/economic context of both countries has created (and continues to create) substantial challenges to socio-economic development in general, and the education sector in particular, which suffers from chronic underinvestment in both countries. This is reflected in poor overall performance among macro-level educational indicators, a key consideration when reviewing the performance of the EQuIP programme. Guinea Bissau underwent a coup d’état in 2012, and between then and June 2014, when a new president took office, the education sector (already the least-resourced in West Africa) has been affected by a succession of teachers strikes, delayed salaries and high staff turnover, involving teachers, head teachers, inspectors, and regional education directors. Due to the very high level of staff turnover many of those who had benefitted from training under EQuIP, are no longer working in the programme area, so the effectiveness of EQuIP activities, including collection and processing of data and in-service teacher training programmes, has been affected. There was a popular uprising in Burkina Faso in November 2014, and a transitional government was put in place. As a result of the political changes there has been a high turnover of key government staff, including at provincial and regional level, in addition to the usual high turnover of staff in Noumbiel, Regular high staff turnover in Noumbiel Many young and inexperienced teachers are allocated to positions in the province of Noumbiel at the start of each academic year, and a high proportion of these teachers work in the province for the minimum period (up to three years) they have to serve before applying for a transfer within the region or to another region. This means that many of the more experienced teachers who have been trained under EQuIP leave the province. Given this situation training should be provided throughout the academic year, not in longer blocks once per year during the holidays, so overlap between incoming and outgoing teachers can increase peer-to-peer learning and promote retention of skills in a given school. Inspectors and Provincial level staff are also allocated to Noumbiel, and are often there without their families, so may not stay for an extended period. High staff turnover (beyond normal rotation periods) is not currently included as a risk in the final risk table for the project.

International Solutions Group www.theisg.com 21 Language Guinea Bissau Formal education is in Portuguese. Most of the teachers interviewed said that they are not confident speaking or writing in Portuguese. Students are being taught in a language they do not understand and sometimes teachers and their students do not have a common language. In Coiada (Gabú) for example members of the community complained that even after three or four years in school their children are not able to read or write. They pointed out that their children are confused by having lessons in a mixture of Fula and Portuguese. The children see Portuguese written in their textbooks and on the blackboard, but are generally taught by their teachers in Fula. One of the first year teachers, who does not speak Fula, shares no common language with his students. Burkina Faso Formal education is in French, and all the teachers are proficient in French. After the first year of school, when students are generally taught in a mixture of French and their mother tongue, teachers find the students are generally prepared to deal with the demands of learning in French.

International Solutions Group www.theisg.com 22 Objective 1: To improve learning outcomes for children in target schools. General Findings EQuIP has delivered quite different training programmes for teachers in Guinea Bissau and Burkina Faso. In both countries EQuIP provides training for inspectors on topics which are new to them. The inspectors then facilitate training for teachers and afterwards should provide follow-up and support, to ensure teachers apply what they have learned effectively in the classroom. In general, the level of follow-up and support is much stronger in Burkina Faso than in Guinea Bissau, but inconsistencies exist across EQuIP- supported schools in both countries. One reason for this is that the level of support given to staff will vary according to the capacity of different inspectors within the teams in both countries. The evaluation found that learning outcomes, as reflected in the pass rates for exams at the end of the primary cycle, have improved in both countries, as have the REACH reading test scores. However, the targets set for Year 4 for these indicators in both countries have not been met. The REACH results revealed tremendous disparities in reading levels across EQuIP schools in both countries, which need to be addressed. The BEPC exam success rates at junior secondary level in Burkina Faso have fallen, and are particularly poor for girls. The BAC success rate is also very poor, at only 16.6% Guinea Bissau The completion rates have improved in both regions, as have the gross enrolment rates, especially in Bafatá. The CEP exam success rates have improved from the baseline of 29.8% to 35.4% in Bafatá, but have not met the Year 4 target of 60%. In Gabú the success rates have risen from the baseline of 15.3% to 68.4%, a major improvement, but again have not met the Year 4 target of 84%. Using improved reading levels as a proxy indicator for learning outcomes, the REACH reading scores showed an overall improvement, but with major discrepancies – both across the two regions and across schools within each region, indicating serious inconsistencies in the quality of teaching. The percentage of non-readers has dropped from the baseline of 77.6% to 54.4% in the final evaluation. There is a big difference in overall reading levels between the two regions, with 40.28 % non-readers in Bafatá and 68.56% in Gabú. The Year 4 target of 25% was not met in either region. The highest score for the REACH section on reading and comprehension was 90%, and the lowest 0%, which was obtained in two schools, Sintcha-Lali and Coiada. This means that children from those schools cannot read a very short and simple text or answer any questions on it. REACH tests should be conducted across the remaining EQuIP schools, and the lowest performing schools from these tests and those already conducted should benefit from an intensive reading programme, to help the teachers improve their teaching of reading and to raise the reading levels of the children. Primary inspectors should then be supported to monitor reading results across their schools on a regular basis, and to take further action as necessary. In Coiada members of the community complained to the evaluation team that even after three to four years in school their children are not able to read or write. This came as a big shock and disappointment to illiterate parents who had not realised how low the level was until a child in the fourth year was asked

International Solutions Group www.theisg.com 23 to write something as simple as a name and failed. One of the girls who went on to the fifth year in a school in Gabú was so embarrassed at her low level, compared to other children, that she returned and re-entered the fourth year in Coiada. The community members believe their children’s right to a good education is being violated on a daily basis. In Gabú the CEP pass rate rose from the baseline of 15.3% for both girls and boys to 68.4% (66.9% for girls and 69.8% for boys) in 2014, but still falling short of the Year 4 target of 84%. In Bafatá the pass rate rose from 14.6% for girls and 15.2% for boys in 2011 to 35.4 % (29.5% for girls and 41.2% for boys) in 2014, again short of the Year 4 target of 60%. These pass rates are below the national average of 88.32% (87.1% for girls and 89.55% for boys). The majority of primary teachers are untrained. Some have not even completed the full primary cycle and very few have completed the full cycle of secondary education. 67% of the teachers in the schools visited had not studied past the ninth year of schooling, and one head teacher had only completed the 6th year. The very low academic level of the majority of the teachers means that many of them struggle to understand the curriculum content themselves. Most are not proficient in Portuguese. The focus of EQuIP’s in-service training for primary teachers has therefore been on Portuguese, Mathematics and Integrated Science, delivered as follows:

 15 days training in October 2012 for 100 teachers (10 women);  30 days through cluster training September – December 2013 for 80 teachers (10 women) ;

 12 days training for 123 teachers October – December 2014 (21 women). These trainings were all delivered by Regional Inspectors. Even though steps have been taken by EQuIP to improve data collection and management, there is still insufficient data to monitor learning outcomes at all levels. Even basic data was not available in 6 out of the 15 schools covered by the evaluation, which head teachers said was due to the fact that they had recently been transferred and there were no records for them to refer to. Due to staff turnover, many of those who received the training outlined below are no longer working in EQuIP schools:

 Training on school management and administration, data collection and leadership was given to 50 head teachers and their deputies (all male) for 5 days in 2012;

 50 head teachers and their deputies (all male) for 2 days in 2013;

 50 head teachers and their deputies (2 women) for 4 days in 2014. These trainings were all delivered by Regional Inspectors. Many of the teachers who have received training under EQuIP are no longer working in EQuIP schools either. In ten of the 15 schools visited no more than one teacher had received any training under EQuIP, in four of them no teacher reported receiving any EQuIP training. The 15 schools surveyed currently employ a total of 109 teachers. In addition to the training carried out under EQuIP, the Portuguese NGO FEC has also carried out training activities in Gabú and Bafatá, for teachers and inspectors, which has targeted staff working in and with

International Solutions Group www.theisg.com 24 EQuIP schools. It is not possible to determine exactly how much of the improvement in learning outcomes is due to EQuIP and how much is due to inputs from FEC, a Portuguese NGO which is also working to raise the quality of education in Gabú and Bafatá. The inspectors mentioned that sometimes there are clashes between activities organised by EQuIP, the Ministry of Education and FEC, indicating a need for improved coordination of activities. A team of primary inspectors consulted in Bafatá during the evaluation said that their team is too small, and they lack the resources they need, to be able to provide the necessary level of inputs, follow-up and support for teachers. They now need to cover a wider range of issues during school inspection visits than before, including monitoring of school governments, management committees and parent’s associations as well as classroom observation and support activities. However, during visits to schools in the Bambadinca and Contubel sectors of Bafatá, head teachers and teachers highlighted the work of committed inspectors who cover those areas. These key informants believed that the consistently high scores for reading and comprehension, gender equality, and classroom management were due to the technical capacity and high level of commitment of these inspectors, who regularly visit schools and provide necessary follow-up and support, leading to better learning outcomes in their schools. These successful inspectors should be invited to share their good practice with other inspectors in Bafatá and Gabú, EQuIP is able to support teacher development through cluster-group planning and training meetings organised by INDE, which are held on alternate Saturdays. FEC has also been involved in training coordinators for these groups. Two major advantages of supporting cluster training are that it is already part of the established system, and given the high level of staff turnover staff would benefit from the training as soon as they start teaching in an EQuIP school. EQuIP should set systems in place to coordinate effectively with other training initiatives to strengthen teaching capacity in Gabú and Bafatá, including INDE, FEC, those funded by the Global Partnership for Education and a UNESCO teacher training project which includes setting up a system for the management of learning outcomes. This would enable EQuIP’s resources to be used more effectively, ensuring that training is followed up with an adequate level of monitoring and support. FEC for example is already providing methodology training for teachers from 1st to 12th grade and has a proven track record in providing effective training and follow-up, according to information received from primary inspectors, head teachers and teachers. FEC have worked on various capacity building interventions for Plan Guinea Bissau, including one with teachers from 30 schools in Bafatá between September 2011 and August 2014. Plan and FEC are already working in partnership to increase provision of quality education at pre-school level Teachers said that if certification could be provided for training, then it would give them with an incentive to attend and implement training. Discussions should be held with the Ministry of Education and other actors to see whether it would be feasible to provide any certification, and if so how that process should be managed. There are also other initiatives, such as an EU-funded FEC project to improve data collection in Gabú, which need to be considered in determining EQuIP’s inputs in the future.

International Solutions Group www.theisg.com 25 Burkina Faso Using improved reading levels as a proxy indicator for learning outcomes, the REACH reading scores in Noumbiel showed an overall improvement. The percentage of children in grade 3 who cannot read has fallen from the baseline figure of 49% to 29%. This is a significant improvement, although has failed to meet the year 4 target of 15%. The evaluation team noted discrepancies in results across schools. The highest score for the REACH section on reading and comprehension was 96%, and the lowest score 10% (Gombar), indicating inconsistencies in the quality of teaching. EQuIP has focused on improving the teaching of reading in Noumbiel, with an early reading scheme developed by EQuIP introduced in 12 schools in April 2014, and then scaled up to all schools in the province in October 2014. All the teachers, head teachers, inspectors and regional officials interviewed as part of the evaluation reported the scheme to be very successful, and provincial and regional officials are advocating for the programme to be scaled up to national level. The evaluation team witnessed strong enthusiasm among students in primary classes who were keen to prove their reading skills. In Noumbiel the CEP pass rate rose from 65.9% in 2011 (62.3% for girls and 69.3% for boys) to 92.8% in 2014 (90.9% for girls and 94.7% for boys). These figures are higher than both the regional pass rates (girls 79.3%, boys 86.8%, total 83%) and national pass rates (girls 80.2%, boys 84.5%, total 82.2%). As all the schools in Noumbiel are covered by EQuIP we can attribute the higher pass rates in Noumbiel to the impact of EQuIP support. The low BEPC and BAC exam success rates at post primary and secondary level reflect the very limited inputs related to improving learning outcomes, comprising:

 5 days of training for 58 teachers on active teaching techniques and assessment in 2012;  5 days for 64 teachers on participative teaching methodology and related themes in 2013. Both sets of training were designed and facilitated by secondary inspectors. The BEPC exam success rate at junior secondary level in Noumbiel has fallen, and is particularly poor for girls. Average exam pass rates at post-primary level (BEPC) in 3 of the EQuIP secondary schools visited for the evaluation were only 31.5% (16% for girls and 46% for boys) in 2014. This is higher than the national average of only 23.7%, but is nonetheless low, especially for girls. In the secondary school in Kpuere, for example, of 45 students sitting the exam, only one girl of 16 passed (6%), and 13 of 29 boys (49%). The BAC exam is taken in only one EQuIP secondary school, in which the 2014 pass rate was 16.6% (5 girls and 8 boys passed the exam), as compared to the national pass rate of 37.04%. Repetition rates are high through post-primary and secondary level. For example, in the secondary school in Midebdo 52 of 168 students (31%) are repeating their sixth year, with 36 of those girls. In the sixth year in Legmoin secondary school, of 136 students, 65 (48%) are repeating the year, 31 of them girls. These high repetition rates were repeated across the six secondary schools visited. These poor performance metrics may encourage further dropouts, especially for girls. At primary level the main focus of EQuIP’s training for teachers has been on teaching reading and maths, delivered as follows:

International Solutions Group www.theisg.com 26  2 days on reading and 2 days on Maths for teachers in 2012;  2 days training in reading and Maths for new teachers in 2013 These trainings were all designed and delivered by primary inspectors.

 In 2014 13 primary inspectors and the focal point person from the Provincial Education Department received 2 days training on research-learning and strategies for teaching and learning how to read in the early years of primary school.

 Following this 5 days training was then delivered by inspectors to 30 teachers in 2014;  A further 5 days training for 265 head teachers and teachers in 2015. A system of monthly training and support sessions for primary teachers called “Groupes d’Animation Pedagogique” (GAP) used to be organised in groupings of 5 to 10 schools. These sessions were discontinued by the Ministry for financial reasons, but many teachers and education officials reported they would like to see them reinstated, in order to give more consistent coverage across all EQuIP primary schools. EQuIP should discuss the feasibility of involvement in reinstating the GAP, or a similar system, with the primary inspectors and Provincial Education Director, to see if financially and logistically it could be a viable and effective way of delivering training. A regular system of training and support should also be established for post-secondary and secondary teachers, to tackle the issues of high repetition and dropout rates, and the low level of exam success, particularly for girls. As with Guinea Bissau high levels of staff turnover dilutes the gains from investment in training for teachers who may only remain in the province for a few years. The 20 schools surveyed currently employ a total of 155 teachers. Child-friendly and participative teaching methods Burkina Faso Classroom observations showed that more than 96% of teachers are now using child-friendly and participative teaching methods, 90% taking inclusion and participation into consideration when preparing their lesson plans and activities, and 70% obtaining the highest score for implementing this in the classroom. This is a significant improvement from the baseline figure of 57.1% of primary school teachers adopting child friendly and inclusive teaching methods. At all the primary level schools visited in Noumbiel, students were sitting in groups, and group work seemed to be well established, with group leaders providing support to others. Guinea Bissau In the baseline 42% of teachers were considered good at classroom management and 15% good at preparing lesson plans and activities. In the evaluation 70% were deemed to be using child-friendly and participative teaching methods, with 52% taking inclusion and participation into consideration when preparing their lesson plans and activities, thus achieving programme targets. However, only 20% obtained the highest score for implementing this in the classroom.

International Solutions Group www.theisg.com 27 These results reflect the somewhat limited input aimed at helping teachers to improve their classroom practices:  In 2012 40 male head teachers received 5 days training in positive pedagogy and gender issues;

 In 2013, 80 teachers (only 10 of whom were women) received 2 days training in positive pedagogy and gender issues Both sets of training were designed by the Ministry of Education/Plan and facilitated by regional inspectors

 INDE delivered 4 days of training for 30 male regional inspectors in 2012 to help them with teacher supervision and support, with input on participatory teaching techniques. When teachers were asked by members of the evaluation team whether they used child-centred teaching methods in the classroom, teachers in six groups responded that they did not know anything about child- centred learning. Even those who had benefitted from some training, head teachers and teachers, found that it had been too limited in terms of time and content, and had lacked the necessary follow-up and support, resulting in a lack of effective application in the classroom. In the majority of the 15 schools visited in Guinea Bissau, only one teacher still in post had received EQuIP training. In four of the schools none of the teachers had received training. Textbooks At primary level the evaluation team noted an improvement in the provision of books in both countries, with most (though not all) children having access to the required textbooks. Guinea Bissau Plan and other partners have provided books for all students, but the evaluation team still noted a few children without books in each class observed – on average, 90% of students had at least one each of reading and maths textbooks (the indicator for this outcome). Burkina Faso While 64% of the primary schools covered by the evaluation reported that all their pupils have full sets of books, head teachers at the six schools visited at post-primary and secondary level reported a serious shortage of textbooks, a major barrier to learning. The first students who have money available to ‘rent’ books for the year can do so, until all the available textbooks are taken, resulting in the majority of students not having access to text books. Some schools have purchased photocopiers to enable copying of books or relevant excerpts, an expensive and temporary solution. In order to build on the success of the early reading programme in Burkina Faso, and to develop reading levels in Guinea Bissau, a selection of reading books should be provided for each primary school where library books have not already been provided. At post-primary level in Burkina Faso EQuIP should provide support as necessary, to improve the timely supply of adequate supplies of textbooks to schools in Noumbiel from central stocks, and the most equitable distribution of textbooks to students in all secondary schools.

International Solutions Group www.theisg.com 28 Objective 2: To increase access to education of vulnerable children, especially girls and children with disabilities. General Findings The household surveys conducted as part of the final evaluation indicated decreases on the baseline figures in both countries in the proportion of people who indicated awareness of the rights of children with disabilities, and the proportion of parents and guardians who declared enrolling their children in school regardless of their gender. The apparent drop between the total positive responses assessed by the evaluation team (41% GNB/47% BFA) and those reported by Plan in the year 3 Results Framework (67.9% GNB/90% BFA) is difficult to interpret. The evaluation team notes, however, that knowledge of CCD rights among families has proven erratic in measurement. For example, positive responses recorded over the life of EQuIP for Guinea Bissau were: • Baseline survey: 49% • Year 1: Data not collected • Year 2 (Plan measurements): 24% • Year 3 (Plan measurements): 67.9% • Endline survey: 47% Methodological differences and response biases may account for these discrepancies, notably between the baseline and year 2/endline, as the baseline survey recorded positive responses only, without verification by provision of example. Overall, however, while it does not appear that the final targets for both countries were achieved, a positive trend was established, indicating some success related to this activity. On analysis of the qualitative data collected, the evaluation team concludes that EQuIP has indeed sensitised communities and increased access for all, including the two main target groups: girls and children with disabilities. Part of this success is due to the work of Plan community development facilitators and VSL facilitators and members, who have played an active role in sensitising parents to the rights of children and encouraging them to enrol their girls and children with disabilities in school. Activities carried out in both countries include training for inspectors, teachers, parents and members of school management committees on inclusive education, gender, children’s rights, and the rights of children with disabilities. In Burkina Faso community development facilitators and VSL members and facilitators have also been included. In Guinea Bissau training has been provided for a broad range of stakeholders including regional staff, members of school governments and parent’s associations, and mothers as well as Plan and partner staff. In May 2015, multidisciplinary groups which had been formed but were not yet active, received training on disability and inclusive education. The Plan Disability Advisor from INO and AGRICE were involved in designing and facilitating this training in Guinea Bissau. The Plan International Disability Advisor inputted into the design. In Burkina Faso school inspectors delivered much of the training. The gender parity targets set for the programme have been achieved, at 0.84 for primary and junior secondary and 0.39 for secondary in Burkina Faso and 1.06:1 in both regions in Guinea Bissau.

International Solutions Group www.theisg.com 29 More girls than boys are enrolling in primary schools, although the number of girls with disabilities in school is much lower than the number of boys. Many students, teachers, head teachers, parents, inspectors and regional staff who were interviewed spoke about the rights of girls and children with disabilities. People acknowledged that changing attitudes towards persons with disabilities is not easy within their societies, so were clear that though progress has been made there is still much work to be done within communities and many children are still out of school. Guinea Bissau There are now 71 children with disabilities (64 girls and 82 boys) enrolled in the 15 schools covered by the evaluation, including the two pilot schools for inclusive education which have been established under EQuIP, in Bafatá and Gabú. In 11 out of the 15 schools the children with disabilities had physical disabilities, for example problems in their lower or upper limbs, which did not pose any serious barriers for them in terms of learning in the classroom. In Dara there was a boy with hearing difficulties, 3 boys with visual problems and a girl who could not walk. In Sintcha Lali there was a deaf girl, who really did not seem to participate in the lesson at all. The other students with disabilities were all in the pilot schools. Only 3 out of the 21 students with disabilities in the pilot school in Bafatá are girls, with 8 boys and only 1 girl in the first year. There is a similar situation in the pilot school in Gabú, as only 4 out of the 22 students with disabilities are girls, with 10 boys and only 2 girls in the first year. Much more work therefore needs to be done to bring girls with disabilities in to schools There is currently no disaggregated data available to assess the learning achievements and progress of children with different disabilities, but as an example all the children with disabilities in the pilot school in Gabú had to repeat the year. EQuIP works in partnership with AGRICE, which carries out a range of activities including the design and facilitation of training, supporting students with disabilities and their teachers in the pilot schools and identifying children with disabilities in the communities. There is a women’s group in Gabú, which sensitises families of children with disabilities to send their children to school. Members of this group include parents and a sibling of students with disabilities attending the pilot school in Gabú. The group receives some support from EQuIP for its activities. Burkina Faso There are now 146 children with disabilities (64 girls and 82 boys) enrolled in the 20 schools covered by the evaluation. In Burkina Faso the EQuIP coordinator participates in 3-monthly meetings with representatives from a range of organisations, including the Ministries of Education and Social Action, AFDC, the Centre for Community-Based Rehabilitation, parent’s associations and associations of people with disabilities. This network is an excellent initiative and should be further developed.

International Solutions Group www.theisg.com 30 Infrastructure In both countries the infrastructure is inadequate to cope with the demands of the growing school population, and some classes are studying in makeshift classrooms made out of locally available materials. The lack of classrooms is particularly critical in some communities in Guinea Bissau, with EQuIP schools having insufficient places to accommodate all the children wishing to enter or continue their studies. In communities such as Dara, the Ministry of Education reduced the number of shifts from three to two for 2014/15. In Dara, Caur, and Coiada respondents were clear that many students were out of school in their communities due to the lack of places, both students who had attended the school the previous year as well as new students wishing to enrol. EQuIP should approach the Ministry of Education, at central level, to see if the third shift can be reinstated in schools where this is needed to accommodate the children wishing to attend. Collaboration with Madrassas in Guinea Bissau Ministry of Education representatives are recognising the need for collaboration between the growing numbers of madrassas and the formal system, as there are a significant number of children attending both schools. In Dara primary school, for example, the inspector for Pitche estimated that as many as 70% of students also attend the nearby madrassa. There is currently no link between the two systems, but the EQuIP focal point inspector in Gabú said that work is underway to formalise links between the two systems, and the Regional Director also spoke of the importance of greater coordination and looking into formal recognition of the teaching provided by madrassas. Given the lack of spaces for children to enrol within the formal system in some communities such as Dara and Coiada, madrassas may be the children’s only opportunity at present to receive an education. Out of school children In FGDs and KIIs in both countries informants spoke of the many marginalised children who are still out of school in the EQuIP project areas, including girls, children with disabilities, child workers, orphans and talibés.5 No data was available, but we were told that some of these children had never entered the education system while others had dropped out. More research is required to establish how many children are still out of the system and the barriers preventing them from accessing education. Strategies then need to be drawn up to bring them in to the education system and keep them there. The Ministry of Education requested the inspectors in Noumbiel to collect data on talibés in May 2015. EQuIP should draw on Plan’s experience supporting children in Daaras in Senegal6 to help develop a response to this issue. Sub-Objective 2.1: To ensure girls enrol and complete quality primary and secondary education Specific EQuIP training activities related to gender equality were:

5 A talibé (from Arabic – seeker/student), is a (usually male) student of the Quran at an Islamic school (Madrassa) or Mosque 6 Specifically, the USAID-funded Basic Education project in Senegal.

International Solutions Group www.theisg.com 31 Guinea Bissau: Training on gender issues has been delivered to 40 head teachers, members of school governments, 80 teachers, and members of parent’s associations. Burkina Faso: Primary inspectors and primary teachers have received training on gender-sensitive teaching methods. In Burkina Faso 61.7% of those questioned responded it was equally important to enrol girls and boys, with 23.3% of respondents saying they thought it was more important to enrol girls. In Guinea Bissau only 52% of parents said school was equally important for girls and boys (55% in Bafatá and 48.6% in Gabú). Despite the fact that the above percentages represent a reduction on the baseline figures7, there has been a substantial rise in the number of girls enrolling in school, and the gender parity targets set for the programme have been achieved, at 0.84 for primary and junior secondary and 0.39 for secondary in Burkina Faso and 1.06:1 in both regions in Guinea Bissau. However, dropout rates remain high, and retention, particularly amongst girls, is a major issue. Further, results in both countries from surveys of students and teachers, and qualitative research among students, teachers and parents were positive in terms of gender equality and respect for differences, reflecting a positive change in attitudes and behaviour not reflected in the surveys of households. According to information gathered from key informants, teacher training and intensive sensitisation work carried out in schools and communities, including by members of VSL groups and parent’s associations, have contributed to this outcome. Many female and male students talked about how their parents now insist that they come to school, and check on their attendance. Guinea Bissau There are more girls than boys in the first three years of the education system, and the ratio of girls has risen from the baseline figure of 46.5% to 51%. But in the last three years (4th to 6th grade) there are more boys than girls, and the ratio of girls in the sixth year was lower in 2013/14 than in the baseline. However, the number of girls dropping out during the primary cycle has reduced considerably under EQuIP, and success rates have risen, especially in the 6th grade. Burkina Faso Although there are now more girls than boys enrolling in schools, there has been an increase in dropouts of 1.86% at primary level and 2.2% at post-primary level since 2011. The main reasons repeatedly cited by respondents in both countries in FGDs and KIIs, both children and adults, for girls dropping out of school are distance between home and school, early marriage and pregnancy, household poverty, child labour, lack of school canteens, the low quality of education and a lack of female role models. These are discussed in detail as follows: 1. Distance from home to school

7 Although not within the scope of the final evaluators to accurately determine, response bias to these questions (the interviewee telling the interviewer what they feel they ‘wanted to hear’) may have played a significant role in the discrepancies between the baseline/endline figures.

International Solutions Group www.theisg.com 32 Guinea Bissau: Only 6 of the 41 EQuIP schools in Gabú and Bafatá go up to the 6th grade, the remaining 35 schools only offer the first four grades. The majority of students wishing to complete their primary education therefore need to attend school some distance from their communities. This causes many children to abandon schooling, particularly girls and children with disabilities, due to the dangers and difficulties of travelling further distances to school. EQuIP should therefore continue to advocate for the provision of 5th and 6th year classes in all primary schools. This constraint is even more significant with respect to transition to secondary level, unless students live nearby or have relatives they can stay with near where the secondary school is located. Burkina Faso: All the primary schools teach six grades, but many students do not continue their studies at post-primary level. For those who continue their studies, it is not uncommon for parents to find rented accommodation for girls and boys at post-primary and secondary level, which they share with other girls and boys from the same village. This can lead to serious problems, as the young people are often left in this situation without any adult supervision or any financial or other support. Girls can find themselves in very difficult situations staying away from home, at risk of physical or sexual exploitation. Interviewees noted that some of these girls become pregnant by other students, or as a result of engaging in transactional sex. Girls and their families need to be sensitised to the dangers of this situation. In one focus group discussion four of the five girl participants reported they were living away from their family home, with one indicating she was experiencing problems. Students also spoke about the risk for orphans, who were more likely to be abused.

“The continuum” was introduced in 2013, which provides pre-school, primary and post-primary education (up to the third year) in one integrated complex. Once “the continuum” is established it will mean more students will be able to continue their studies at post-primary level without having to travel further from their homes. By end of their third year at secondary level, students are more mature to deal with being away from home to access secondary education. EQuIP should therefore support the development of” the continuum”. 2. Early marriage and pregnancy Currently many girls enter primary school when they are already 10 or 11 years old, so they are 15 or 16 years old by the time they complete their primary education, and many girls will drop out before then. Sensitisation needs to take place to encourage families to ensure girls are enrolled at the appropriate age, as well as providing input on sexual and reproductive health from primary level onwards and sensitising communities on the dangers of early marriage. In both countries a range of stakeholders, including teachers, head teachers and parents in FGDs and KIIs, expressed the need for sexual and reproductive health education to start in the 5th and 6th years of the primary cycle Guinea Bissau: Many girls drop out of primary school because of early marriage, often against their wishes. Part of the reason for this is that parents are scared their daughters may get pregnant before marriage, a source of great shame. In Dara the evaluation team was approached for help by a 15-year-old

International Solutions Group www.theisg.com 33 student, who had participated in a FGD earlier that afternoon. The girl explained that her family were arranging a marriage for her, within the coming weeks, against her wishes. Teachers from the school explained that they would not intervene in a situation regarding an arranged marriage. EQuIP, in liaison with Plan’s child protection staff, should consider how best to provide contact details and information for students in such situations who require help. In Coiada members of the community told the evaluation team that they would discontinue early marriage for their girls and encourage them to continue their studies, if the quality of education was improved. Burkina Faso: In Noumbiel the situation is different, as student pregnancies are common, with 39 pregnancies in the 6 secondary schools covered by the evaluation, far in excess of the target of 10 pregnancies for Year 4 for all EQuIP secondary schools. Head teachers and the Coordinator of RAJS said that generally the majority of pregnancies are at junior secondary level, including in 6eme, when they feel girls are more vulnerable. This year the pregnancies were as follows: 6eme - 3, 5eme -15, 4eme – 9, 3eme – 11, 2eme – 0, 1ere – 1. Unlike in some other areas of Burkina Faso, this is not a source of stigma in Noumbiel, where it is culturally acceptable for a young girl to have a child, married or not, and even to have children by different fathers. It is a matrilineal society where inheritance passes through girls from their maternal uncles. Families are proud when their daughter has a child, which then ‘belongs’ to the maternal grandfather. On the basis of data from head teachers, the majority of the young mothers return to school a short time after the birth of their babies if there is someone available to look after the baby while the mother is at school. A 15-year old pregnant student was interviewed in one of the schools in Noumbiel. She said that the father of the baby was also a student at the school, and that neither of them had had any problems with their families when they announced the pregnancy. Both families had been happy at the news. The baby was due in one month, and she was anticipating returning to school after the birth – as some of her school friends had already done. Her mother had said she would look after the baby while she was at school. When asked about contraception, she said that she had been taught about methods of contraception and knew what was available – but gave to understand that she had not been interested in using any. She said that her grades had fallen, because she had felt too tired sometimes to study after school, but that she had been healthy and intended to return and complete her studies after the birth. There are increasing numbers of artisanal gold mines in the EQuIP programme area, and many girls work in the mining communities full or part time. There are also students who go to work in the sites during the school holidays to raise money to cover their education costs for the coming academic year. Some of the girls become pregnant by gold miners. In Noumbiel RAJS is active with a range of sexual and reproductive health activities for EQuIP at post- primary and secondary level, but the numbers of pregnancies are not falling. Part of the reason staff at RAJS feel this is happening is that the sexual and reproductive health education is starting too late, and there is need for sexual and reproductive health education to start in the last two years of the primary cycle. RAJS has indicated they would like to work at primary level too. Due to over-age enrolment, many girls are of child-bearing age before completing their primary education.

International Solutions Group www.theisg.com 34 Pregnancy rates are high in Noumbiel, but the majority of young mothers are returning to school after they have given birth, to continue their studies. Staff at RAJS say that girls should be targeted for additional sexual and reproductive health education in the months before they leave home, during the first two years at junior secondary school when they are particularly vulnerable and the number of pregnancies is highest, before and during the holidays as a high percentage of pregnancies begin then, and when a new gold site opens in the area. Sensitisation activities for parents should also be aimed at girls’ maternal uncles, who are key family leaders in Noumbiel Province. Such inputs could be coordinated with other actors such as CECI (Canadian Centre d’Etudes et de Cooperation Internationale) and AJDB (Association de Jeunes pour le Developpement de Batié ), currently working in gold sites on HIV/AIDS prevention. Each of the EQuIP secondary schools in Noumbiel has one or two Education Assistants on the staff, working in ‘Service Vie Scolaire’. They are responsible for discipline, but also keep data on the number of pregnancies and children with disabilities, and are sometimes involved in organising sexual and reproductive health education activities with RAJS. These members of staff are in key positions to support and inform EQuIP’s work. One particular area of potential involvement would be in facilitating closer collaboration with the Ministry for Social Action, which provides support for pregnant girls and other vulnerable children, as well as children with disabilities. One of the Education Assistants, who was interviewed during the evaluation, said that he had not been invited to attend EQuIP training, and therefore felt excluded. 3. Poverty and child labour Poverty is still one of the main reasons for girls dropping out of school, due to the costs of attending school and the need for girls to engage in paid labour. Some girls miss substantial amounts of schooling because they have, to help with domestic tasks. This can lead to failure, repetition and dropout. Guinea Bissau: Children miss school mainly for the cashew nut harvest, (in April and May each year. Sensitisation has been carried out to try and prevent children from missing school for the cashew harvest, but the reality is that dominance of cashew as a cash crop in the economy of Guinea Bissau means their labour is needed in many families to provide money to buy rice to feed the family during the rainy season and to keep the family going for the rest of the year. Burkina Faso: There are a growing number of artisanal gold mining sites in the area where EQuIP is operating, and many children, both girls and boys, work on sites on a part-time or full-time basis. The programme area is also very near the frontiers with Ghana and Ivory Coast, and many of the students and their families migrate for work. VSL: The VSL groups set up under EQuIP have improved the financial situation for many families, and allowed many girls and boys to continue their studies. Members of VSL groups reported that money from VSL activity helped to provide for their children, including education costs. This has helped to bring more children in to school and to reduce the number of dropouts, girls and boys. The evaluation team noted some confusion in both countries regarding membership, however, with some members of credit groups not being sure whether the group they belonged to was linked to EQuIP. Despite

International Solutions Group www.theisg.com 35 this, it was clear that the presence of EQuIP and other credit groups is playing a positive role in generating funds to help cover education costs. Some groups are directly linked to schools, for example the EQuIP group in Coiada in Guinea Bissau, which has a social fund to help meet education costs for children, and many of the VSL groups provide items to support school canteens. In addition to generating funds to help cover education costs, VSL members are also playing a valuable role in awareness raising and sensitisation, supporting COGES, parents’ associations and Plan community development facilitators in mobilising community members to participate in a range of activities. Guinea Bissau: 34% of respondents in the household survey said they were members of a VSL or another credit group, and stated that most of the money was used for small-scale business and not directly for meeting education expenses. However, a survey recently carried out by the Association for the Promotion of Local Integrated Development (APRODEL), found that 40.6% of the credit was used for education as the main use, and 35.2% as a secondary use, indicating that VSL schemes are having a positive impact on increasing access to schooling for children. VSL groups are active in all but three of the 41 EQuIP communities in Guinea Bissau (Cambadju, Braima Sori and Wacaba), Burkina Faso: 32% of respondents in the household survey reported being members of a VSL or other credit group, with an average of 31% of VSL funds allocated to education. VSL groups are active in only three of the five communes, Batié , Midebdo and Legmoin. They have not yet been introduced by EQuIP in Boussoukoula or Kpuere, due to a lack of resources. 4. Lack of school canteens The lack of functioning school canteens at primary level impacts attendance and often then leads to drop out. If children attend morning school and do not receive a meal, they are often too tired and hungry to walk back to school for the afternoon session. EQuIP supports the development of school gardens and canteens at both primary and secondary level, through the involvement of VSL groups, school management committees, school governments and parent’s associations. Some of these groups in both countries manage school gardens and canteens, and support the canteens through the purchase of food items and soap. Guinea Bissau: Many schools had functioning canteens, usually run by mothers of students. However, some students reported that the canteens did not always or usually provide food, but had provided food on the day of the evaluation team’s visit. Therefore, it may be that not all canteens function as well as they appeared to the evaluation team. Burkina Faso: During the 2014/2015 school year, most school canteens in Noumbiel did not operate until the last few months of the year, as they received foodstuffs only towards the end of that period. We were told this was because storage areas where foodstuffs for school canteens were being kept in Ouagadougou had been raided during the popular uprising in 2014. Parents had contributed what they could, and produce from school gardens was also used. In KIIs head teachers said that the cost of meals at secondary level (CFA75, €0.12) prevents many students from buying meals. 5. Education Quality

International Solutions Group www.theisg.com 36 Guinea Bissau: In a community meeting in Coiada (Gabú), men and women complained that even after three to four years in school their children are not able to read or write. People expressed disappointment regarding how little their children had learned at school. As most of the adults are illiterate, they were not in a position to monitor what was being taught at school. They gave the example of a child in the fourth year, who was asked to write something as simple as family names and failed. They also talked about one of their girls who went on to the fifth year in Gabú, but was so embarrassed at her low level, compared to other children, that she returned and re-entered the fourth year in Coiada. A group of fathers said that they had well understood the message about the importance of enrolling their girls in school, and they had responded by enrolling their girls – the only constraint being the limited number of places available at the school. However, they are questioning the value of their girls and boys attending school, given that the quality of education they are being offered is so low. The fathers said that if their girls were receiving a better education then they would allow them to continue their schooling for as long as the girls wished, and would delay their marriages until they had finished studying, but given the situation they will continue the tradition of marrying their girls early. 6. Female role models There is only one female head teacher in an EQuIP school in each country, and interviewees reported few female teachers in the rural areas covered by the project, due to the difficult working conditions and problems with transport. which means that female students in rural areas have few female role models within the school environment, or women to provide informal guidance and support. There are very few female teachers in the rural areas in both countries, The evaluation team visited Bonkossera Primary School, the EQuIP school with a female head teacher in Burkina Faso, and discovered that the head teacher had experienced a very hostile reaction from the community initially. With support from the inspector responsible for her school (who confirmed how difficult it had been for her) she persevered and now in her second year she is more accepted by the community. While efforts to increase the number of female teachers (and head teachers) may be beyond the scope of EQuIP, the evaluation team does recommend that Plan prioritise support and training for schools with such teachers, in order to promote gender equality among the wider community, and provide positive role models for girls.

International Solutions Group www.theisg.com 37 Sub-Objective 2.2: To create a conducive environment for inclusive education with a focus on children with disabilities. In both countries discussions with various respondents including head teachers, teachers, students and parents resulted in frequent discrepancies between reported numbers of children with disabilities attending school. This can be partly explained by people having different perceptions of whether someone has a disability or not. For consistency, the figures used in this report are those provided by the head teachers. The number of number of children with disabilities enrolled in schools under EQuIP has increased greatly, with 71 children currently enrolled in the 15 schools covered by the evaluation in Guinea Bissau, including 43 students in the 2 pilot schools for inclusive education in Bafatá and Gabú, and 146 in the 20 schools in Burkina Faso (64 girls and 82 boys). In both countries head teachers, teachers and students appeared to be very supportive of children with physical disabilities in school. However, there was a general consensus that the necessary conditions were not in place for students with sensory or intellectual disabilities to receive a quality education, due to the lack of specialised training for teachers and resources. Problems with transport to and from school is a major cause of dropout for children with disabilities. More consideration needs to be given to the health and other requirements of children with disabilities, including access to aids such as glasses or hearing aids. Collaboration needs to be developed with the Ministry of Health in both countries, and with the Ministry of Social Action in Burkina Faso, to provide the support needed by children with disabilities and their families. Guinea Bissau Disability has been a major focus of the EQuIP programme, with training on inclusive education for teachers at the pilot schools, inspectors, head teachers, Plan staff, implementing partners, the regional education directorate, parent’s associations and school management committees. However, teachers outside the pilot schools have only recently (May 2015) received training:

 2 days of training for 53 teachers on disability and inclusive education, designed by the Plan INO Disability Advisor with input from Plan International Disability Advisor. This was facilitated by INO’s disability Advisor and by the AGRICE Pilot Inclusive Education Project Coordinator. The lack of input is reflected in the evaluation figure of only 52% of teachers observed (which includes teachers in the pilot schools) taking inclusion and participation into consideration when preparing their lesson plans and activities. One example of the lack of inclusion was in Sintcha Lali, where the evaluation team met a deaf girl who was in class, but had no books, and did not appear to be actively involved in the lesson. She would have preferred to be in a class with other deaf children and with a teacher specialised in sign language. Within this context it is not surprising that teachers in two schools in Guinea Bissau reportedly stated openly that students with disabilities who they teach should attend a special school, and that according to parents in another school, students with disabilities were asked to leave.

International Solutions Group www.theisg.com 38 Pilot schools for inclusive education in Bafatá and Gabú In the pilot schools the students and teachers have access to a level of support, from two members of staff from the Guinean Association for the Rehabilitation and Integration of Blind People (AGRICE), and Braille resources, which are not available in other schools.

 In 2013 11 teachers from the pilot schools (only one female participant) received 4 days training on inclusive teaching methods, human rights, Braille and sign language.

 In 2014 there were 10 more days of training for 17 teachers from the pilot schools which covered inclusive teaching methods, human rights and community awareness on disability issues. During a focus group discussion in Gabú teachers said that the training they had received was too limited to enable them to teach their students effectively. In order to be confident in Braille and sign language they would need to receive much more extensive training. They have access to resources such as Braille, and support is available from AGRICE staff, but they do not feel they have the necessary skills or sufficient resources to teach children with sensory disabilities. Some of the teachers felt that the students would be taught more effectively in a class together with a specialised teacher. All the students with disabilities in the pilot school in Gabú had repeated the year. One of the deaf students in Bafatá pilot school indicated that, while he was happy to be studying within the mainstream formal system, he felt frustrated at the lack of resources and teachers who could offer sign language. He added that, although he enjoyed being in class and was not subject to any kind of discrimination from teachers or students, he was sure that he would make better academic progress if he was with other deaf children, with teachers specifically trained to teach deaf children. All students (even the 19 and 20-year olds) had been put in the first year when they entered the pilot schools. The AGRICE coordinator said the plan is for students to be put in the most age-appropriate class for the next academic year, where the teacher will need to support them to do the work in line with their academic level, not their age – which socially will be much better for the students, but will bring a different set of challenges for them and their teachers. The students will be the only ones in class studying at that level and will need a lot of individual support from their teachers. When the evaluation team visited the pilot school in Gabú, only 8 of the 22 disabled students enrolled were present that day. Teachers said that they believed one of the first year students and five of the second year students had dropped out, but were not sure about the others. They said they did not know the reasons why, but assumed that problems with transport would have led to some of those dropouts. The AGRICE Coordinator felt some of these absences were due to the cashew nut harvest,8 and was hopeful that some of the missing children would return to school at the end of the harvest. Even though the pilot school in Gabú has had work done to make it accessible for students with disabilities, a 14-year-old student who uses a wheelchair explained that he could not use the bathrooms at the school – because the corridor with the ramp to access the bathrooms was not wide enough for his wheelchair, and the actual design of the toilet meant he could not use it.

8 Cashew nuts comprise Guinea-Bissau’s major cash crop and export commodity

International Solutions Group www.theisg.com 39 The school in Gabú is in an urban area, and used to have a piped water supply, but there is currently no running water in this school, so it is difficult for the students to keep the bathrooms clean. The dirty bathrooms are unsuitable for all, but two blind female students pointed out that whereas other students with sight can see to navigate their way through the filth on the ground they cannot, and so they feel they cannot enter that area at all. The two girls said they were also affected by the lack of water, as they cannot go out of school to find water to drink elsewhere and cannot afford to buy bottled water. All of the students interviewed in the pilot schools said they knew nothing about EQuIP, and had had no opportunity to make any inputs to project development. Collaboration with other stakeholders The new resource centre which has recently been constructed in Gabú, which is just around the corner from one of the pilot schools, should be used as a hub for resources and expertise to meet the needs of children with disabilities and other vulnerable children. In Gabú the Team Leader met with 5 representatives of a Women’s Group, who carry out sensitisation activities to encourage parents of children with disabilities to send them to school. Due to the lack of transport they are unable to do any sensitisation in any of the areas outside Gabú. They emphasised how hard it is for the families to meet the health costs for children with disabilities. They explained it is not uncommon for fathers to abandon their families when a child with a disability is born, so many of the children are being looked after by single mothers. This group receives some financial support from EQuIP. The evaluation team understands that some work has already been done in establishing multi-disciplinary support groups in Bafatá and Gabú. These groups are not yet functional so need to be activated as soon as possible. Burkina Faso: Despite the limited amount of EQuIP training focussing on inclusive education, classroom observations showed that 90% of teachers take inclusion and participation into consideration when preparing their lesson plans and activities.

 12 primary inspectors received 3 days training on inclusive education in 2013, which they then passed on to 338 teachers (268 men and 70 women) in a further 3 days of training.  In 2014 VSL facilitators and members also received some input on inclusive education, as part of a 3-day training which also covered other key issues. Examples of education provision for children with disabilities in Burkina Faso Some head teachers have been very proactive in persuading parents to let their children come to school, and providing them with the support they need.  In Zinka a young boy, Marc, was at home, and his parents were not interested in sending him to school. The head teacher persuaded Marc’s parents to enrol him, and then he managed to get a tricycle from the Ministry of Social Action so Marc could get to and from school. Marc is now 8 years old and in CP2. He gets extra homework for Maths and writing – which he loves doing. Teachers help him get to school on his tricycle, and members of the school government built a

International Solutions Group www.theisg.com 40 little ramp so he can get into the classroom more easily, as he walks with a stick. After school had ended, Marc stayed in the grounds, playing with friends, until some of them gave him a helping hand on his way home. Marc is one of 4 students with disabilities at the school.  In Batié C school in Burkina Faso the team met a 14-year old girl who had been brought in to school as a result of EQuIP. This girl has physical, intellectual and sensory disabilities, and does not want to go into a classroom, but remains outside sitting on the ground in the school yard all day. The headmaster says she appears to be happy, and he believes she is benefitting from contact with other children. Other students check up on her during the day, and make sure she has water to drink. In this case there is no expectation that this girl will make any academic progress, but her parents and the head teacher believe this is the best place for her to be at the moment.  In Bousoukoula secondary school the head teacher was able to provide a list of 16 children with disabilities studying at the school, with their ages, type of handicap and their end of term averages. Seven of the students have auditory problems, 3 visual, 5 have stutters and 1 a physical disability. The marks ranged between 6.6 and 18.74 out of 20. Four of the students (3 with auditory problems and one who stutters) had over 17 out of 20. Seven out of the 16 are girls, including the one with the highest mark of 18.74. Plans to establish a special school for children with disabilities in Noumbiel Teachers, head teachers, and inspectors who were consulted were all clear that all children who can benefit from studying in a mainstream school, where any additional support which is needed can be provided, without any additional specialised training such as Braille or sign language, should be there. However, they felt that some children, whose needs cannot be met effectively within a mainstream school, should attend a specialised school on a short-term or long-term basis. The Provincial Director for National Education & Literacy (DPENA), reported they wish to establish a specialised school in the province for children with sensory and intellectual disabilities where they would be able to access specialist staff and resources. They felt that for some children attendance at a “classe transitoire d’inclusion”, a transitory inclusion class, where students could learn Braille and sign language before entering a mainstream school, may be the best option. For Noumbiel the constant high level of staff turnover is a disincentive for investing in specialised training for teachers who may only remain in the province for a few years. Collaboration with the Ministries of Health and Social Action and other stakeholders Parents of children with disabilities and head teachers explained how hard it can be for students to access health care and the aids they need, such as prescription glasses, hearing aids, wheelchairs and walking supports or the psychological support they may need. Coordination and collaboration with the Ministries of Health and Social Action and Welfare need to be strengthened, to ensure students have the necessary support for them and their families at home and at school. If the programme is scaled up to other geographical areas, then these Ministries should be closely involved from the beginning Guinea Bissau: Multi-disciplinary support groups have been established, in Bafatá and Gabú, but are not yet operational. These networks need to be activated as soon as possible.

International Solutions Group www.theisg.com 41 Burkina Faso: The Provincial Education Department has passed information on children with disabilities in schools to the Ministry for Social Action, which is now visiting those children at home and school to establish the health and social support they need. EQuIP is now starting to work with the Ministry of Social Action, so this collaboration needs to be further developed. The EQuIP coordinator is active in a network of governmental and civil society organisations, which holds quarterly meetings to focus on health, education and other issues for people with disabilities. KIIs with children with disabilities The evaluation team took on the additional task of attempting to interview all available children with disabilities in every school visited. However, due to the absence of some children when the team visited their school (for example only 8 out of the 22 students with disabilities were present when the team visited the pilot school in Gabú) and the constraints in interviewing children with more severe auditory, speech or intellectual disabilities, the team was unable to reliably quantitatively determine the number of boys and girls with disabilities who report that their school is disability-friendly. A sign language interpreter had been organised by the team in Guinea Bissau, but unfortunately he could only help with one interview, as there was only one deaf student at one of the pilot schools in Gabú who could use sign language. This student told the team he had learned sign language out of school. More time is needed than was available during the evaluation for this activity, especially as many of the students are young and understandably very shy meeting strangers, even with people who spoke a range of local languages on our team and interpreters to hand (teachers and other students). The respondents to specific questions related to their disabilities were generally those children with physical disabilities which pose no barriers to them learning once they are in school and who are able to make good progress. Some of the students reported that they needed extra help, and most of them found their teachers very willing to do this. No one reported any serious problems with other students. Assessment of learning achievements of children with disabilities There is currently no disaggregated data available in either country to assess the learning achievements and progress of children with different disabilities. In Burkina Faso more information on children with disabilities is now being routinely collected, due to, and with the assistance of, EQuIP, but there is a lack of resources to input and analyse that data at regional level. Transport Students with disabilities, their parents, head teachers and teachers all spoke about the problem of transport so children can travel between home and school. Lack of transport often leads to absence and dropout. Some students need bicycles or tricycles so they can come or be brought to school. In one case in Burkina Faso a 14-year-old girl with one leg cycled five kilometres to school, and then when she could no longer use the bicycle she hopped to school. Eventually she found it too exhausting, and dropped out.

International Solutions Group www.theisg.com 42 Objective 3: To improve citizens’ active participation in educational decision-making at all levels General Findings There has been an increase in the number of functioning parent’s associations, school management committees and school governments in both countries. Some of these groups are very proactive, but the level of activity and participation in school management and decision-making varies greatly across schools.  There are active parent’s associations in 13 of the 15 schools covered in Guinea Bissau. and in 18 out of the all 20 schools in Burkina Faso, where there are also active Mothers Associations in all but one of the primary schools.  There are active school management committees (COGES) in all 15 schools in Guinea Bissau, and 19 out of the 20 schools in Burkina Faso.  There are school governments in 12 out of the 15 schools in Guinea Bissau, and in all but one of the 20 schools in Burkina Faso (Bobera) Some are the school governments are very proactive, for example organising sensitisations on the importance of education for girls and children with disabilities, visiting parents when girls or children with disabilities are not attending school, and organising study groups. Caetano Semedo (the pilot school in Gabú) and Batié C (in Noumbiel) are two schools with proactive school governments. In these particular schools the level of activity seemed to reflect the dynamism of those students involved. Others exhibited such a low level activity that respondents gave conflicting answers to the evaluation team as to whether the school actually had a school government or not. Many such groups are focussed on keeping their schools clean, and the school gardens watered. In some schools parents and students had been involved in preparing a code of conduct, but in general codes had not been prepared in a participatory way, and were not on public display. Teachers, head teachers, parents, students and inspectors should all participate in drawing up codes of conduct in EQuIP schools, ensuring that the document is publicised and available for all to see, in languages they understand, with a clear understanding by all of actions taken if the code is violated, and outlining the role of the head teachers, inspectors, student government, COGES, and parent’s associations in that process. An area for EQuIP to develop further is how to support students and parents where there are difficult issues which need to be resolved. For example, in one secondary school in Burkina Faso both students and parents talked about two male teachers who were verbally and physically abusive to students, and It was alleged that their behaviour had already caused at least two girls to drop out from school and was causing significant distress to other students. In a school in Guinea Bissau students reported being subject to physical violence from the head teacher and another teacher, and that one of the teachers was often intoxicated in school. In another school in Guinea Bissau one community reported teachers’ poor attendance and punctuality at their school, and their dissatisfaction with the quality of education their children are receiving. This school has an active COGES, parent’s association, a VSL group which supports a social fund for education costs, and a school garden. But despite the level of community organisation

International Solutions Group www.theisg.com 43 and participation the parents are frustrated because their expectations have been raised and are not being met, and they feel powerless to address the issues which are affecting their children. There is need for guidance on addressing complaints and action to be taken by stakeholders in these situations, as the existence of these bodies within a school community is not sufficient to give the students or their parents the power to resolve the issues. In all schools visited by the evaluation team, students noted they felt happy at school and were not subjected to violence (97% GNB/74% BFA). In many instances the team saw that students stayed in the school even when classes were over, playing and chatting, and were clearly very at ease in their surroundings. In both countries there have been improvements in the learning environment, with a rise in the number of students who say they feel safe at school. Knowledge of and participation in EQuIP EQuIP is not known to most of the students and parents interviewed in either country, indicating that as beneficiaries they have not been involved in giving any feedback on the project or contributing to its development. Student and parent bodies should be more involved in EQuIP, which means they will need access to more information and opportunities to participate in the programme planning, monitoring and development. They should be invited to share experiences and learning from EQuIP. Moving forward, member of school governments, parent’s associations, and school management committees should be more involved in the planning, implementation, and monitoring and evaluation of the programme, and feed into its future development.

International Solutions Group www.theisg.com 44 Objective 4: To effect change in policy, funding and approaches to quality inclusive education at a national level. National Education Sector Plans Guinea Bissau EQuIP is in line with Guinea Bissau’s national education policy, as outlined in the Government’s Triennial Action Plan for the Development of Education 2010-2013. Priorities established for the education sector in 2010 are increasing intake and reducing repetition and dropout rates at primary level. The Ministry of Education is currently developing the Education Sector Plan for 2016-2025, with input from Plan and other major actors at central level, including UNICEF and UNESCO. The Plan Education Specialist and other education staff are working to ensure that inclusive education is covered in this plan, and is feeding in to the process with evidence from EQuIP. Based on experience with EQuIP, Plan should be able to submit evidence-based advocacy to support proposed inputs to the development of the national education plan for 2016-2025. However, there are ongoing problems with the availability of data at all levels, and there needs to be more data collection and analysis, and documentation of good practices, for this process to be strengthened Burkina Faso EQuIP is fully in line with the issues of inclusive education and improving the quality of education as highlighted in the current education sector plan, the Programme Sectoriel de l’Éducation et de la Formation (PSEF) 2012-2021. This states that inclusive education has not only gender considerations, but for all vulnerable or marginalised children, including children with disabilities. It points out the need to map children with special needs, and develop the accessibility of schools and support health, financial and psychosocial resources. It states the objectives of achieving complete gender equality throughout the education system by 2025. It also includes fighting against violence, development of gender-sensitive teaching methodology and improvement of community participation in management and governance. The document also refers to ongoing teacher training, classroom observations and regular Groupes d’Animation Pedagogique (GAP). Evidence-based advocacy in Burkina Faso In Burkina Faso, the Ministry of Education at Provincial and Regional level is proactive in trying to scale up the early reading initiative introduced by EQuIP, not only within the region but also with a view to extending the initiative to a national level. They say that now teachers are able to cover all the material in good time, which they were not able to do before. Inspectors have submitted reports to the Provincial Director of Education (DPENA), detailing the successes, who then forwarded information from these reports to the Regional Director of Education (DRENA), who passed on relevant information to central Ministry. The Provincial Director has written to Central Ministry directly to ask for a team to come and see the success in teaching reading for themselves. The Regional Director says he also shares information with other Regional Directors, and some have already expressed an interest in the scheme. So DPENA and DRENA are being proactive in trying to scale up an EQuIP interventions they find successful. They did not indicate that they required any support in this process, as they feel ownership of the scheme and say it is up to the Ministry to allocate resources if the decision is made to scale it up.

International Solutions Group www.theisg.com 45 Staff turnover In both countries there have been major political upheavals, which have contributed to an even higher level of staff turnover than usual, at all levels, including key persons at provincial and regional level. Least progress has been made against this objective, and inputs need to be strengthened for evidence to be documented and used to inform national policy development and implementation in both countries. Specialist advocacy input and support may be needed in this area, in line with the original regional staffing plans for the programme. .

International Solutions Group www.theisg.com 46 E. EQuIP Structure and Programme Management

Project management at regional level The EQuIP Programme Manager, who is based in the Country Office in Burkina Faso, is responsible for the EQuIP programmes in Sierra Leone and Guinea as well as in Burkina Faso and Guinea Bissau. The Programme Manager maintains regular contact with EQuIP staff in Burkina Faso and Guinea Bissau and provides support and guidance through regular phone, email and Skype communication and monitoring missions. The Programme Manager and staff in-country all agreed that the communication they have is of critical importance, but the support available to each country is limited because one member of staff is covering all four country EQuIP programmes. The level of support needed for each country programme obviously varies, in line with the context and the capacity of the Country Coordinator. There appears to be limited interaction and networking between the Coordinators in Burkina Faso and Guinea Bissau with their colleagues in the other two countries covered by EQuIP. This may be partly due to language barriers. Initially two advisors were included in the EQuIP structure at regional level, one for quality and the other for advocacy. There was a Quality Advisor in post, but she was not replaced after she left, and short term consultancy inputs are now being made. There is currently no input on advocacy. The Plan INO Disability Coordinator has been involved in designing and facilitating training in Guinea Bissau. Project management at country level The Country Coordinators need to collaborate with a significant number of partners, including Ministries of Education, organisations sub-contracted in to provide specific areas of expertise for the programme, Plan’s extensive VSL, community mobilisation and sensitisation networks, and education specialists and other programme managers. They also need to manage a broad range of inputs. In terms of EQuIP programme management by the Country Coordinators, there is a marked contrast in effectiveness between the two countries. Guinea Bissau The EQuIP Coordinator is based in the Country Office, in Bissau, even though there are two PUs in the project area (Bafatá and Gabú). It was found that all aspects of programme management, implementation, monitoring and evaluation are less effective because the EQuIP Coordinator is based too far from the programme area to make frequent visits, leading to unnecessary delays in communications and missed opportunities for facilitating networking, coordination and cooperation. All the project partners consulted stated that the project would be more effective if the Coordinator were based within the programme area. The Country Coordinator needs to be in regular contact with all the stakeholders involved, including staff from the two PUs and the two Regional Education Offices. There are other international organisations providing training and support for education in Bafatá and Gabú, and coordination and cooperation with them is essential in order to ensure EQuIP’s inputs are as focussed and effective as possible.

International Solutions Group www.theisg.com 47 The planned appointment of an appropriately skilled and experienced EQuIP Coordinator, to be based in the programme area, will improve all aspects of project management and development. Ideally this person should be located within the newly-constructed resource centre in Gabú, which is in close proximity to the pilot school Caetano Semedo. The resource centre could then be a hub for project activities and resources. Also the Plan PU opened fairly recently, so Plan is far less established than in Bafatá, Also the focal point inspector who had been involved in the project since its inception had to withdraw due to health problems, so Gabú needs more support. Burkina Faso The Country Coordinator is based in the nearest PU to the programme area, in Gaoua. The PU is near enough to the programme area for the Coordinator to make frequent visits, and he receives a good level of support from the PU. All project partners said they were satisfied with the level of support received from the Coordinator. The only problem highlighted was that of Ministry of Education and other project partners needing access to improved internet and phone communications with the PU in Batié, as they sometimes have problems submitting documentation to the Country Coordinator in Gaoua within the tight timeframes required. Working with Ministries of Education The main partners in both countries are the Ministries of Education. Whilst Ministry of Education staff in both countries have been active in designing and facilitating EQuIP training and providing follow-up and support, there is a marked contrast in the level of activity within each country. In Guinea Bissau EQuIP is operating in two regions, working with two different regional directorates. The Ministry of Education lacks the capacity to provide consistent follow-up and support after training, and to monitor outcomes. The context is further complicated by the fact that the majority of teachers have a low academic level, are untrained, and are not proficient in Portuguese. According to the primary inspection team in Bafatá, there is sometimes poor coordination between EQuIP and other education activities organised by the Ministry of Education and FEC, a Portuguese NGO which is very active in education in the same two regions as EQuIP. Therefore, in Guinea Bissau there is the added challenge of coordinating with other overlapping activities, to ensure synergy. In Burkina Faso the EQuIP programme brings together Plan and the Regional and Provincial education authorities (DRENA, DPENA and DR-MESS), in an effective partnership to work towards objectives which are in line with the stated priorities of the Ministry of Education (PSDEB 2012 – 2021). EQuIP covers all the schools in Noumbiel Province, at primary, post-primary and secondary level. There are written agreements in place between EQuIP and the Ministry of Education, signed at provincial level, therefore the different parties are clear about their roles and obligations. The Ministry of Education has the structures and staff with the necessary skills and capacity to take ownership of the programme, to design and facilitate much of the training provided under EQuIP, and give follow-up and support. Primary inspectors and head teachers in particular are very articulate about improvements which have been made under EQuIP. There is a good relationship between the Project Co-ordinator and Ministry of Education staff at all levels. There are signed contracts with EQuIP in place, laying out roles and responsibilities and project activities to be covered. The feedback to the evaluation team was that Ministry of Education staff see EQuIP as a project which is enabling them to tackle issues which are key for them, such as teaching reading more effectively.

International Solutions Group www.theisg.com 48 Working with local partners and organisations In both countries EQuIP partners with local organisations, with specific areas of expertise, to implement certain components of the programme. In order to deliver and develop the Village Savings and Loans (VSL) component of the project, EQuIP works with local partners, AFDC in Burkina Faso and APRODEL in Guinea Bissau. The Plan VSL Specialist for Guinea Bissau is based in Bafatá, and works closely with APRODEL. The EQuIP Project Co-ordinator in Burkina Faso works closely with AFDC, holding monthly meetings with the AFDC coordinator and some of the community facilitators. VSL groups are active in three out of the five communes in Noumbiel province in Burkina Faso, and in 38 out of 41 communities in Guinea Bissau. Persons interviewed in both countries spoke of how the funds generated through VSL help to cover education costs, and the value of the active roles members and community facilitators play in raising awareness on key issues, and mobilising enhanced community participation in a range of activities. In Guinea Bissau AGRICE, which has expertise on supporting children with disabilities, supports EQuIP’s work on inclusive education. AGRICE’s activities include the provision of training inputs on inclusive education, human rights and the rights of children with disabilities and supports teaching for children with disabilities in the two pilot schools. The African Youth Network for Health & Development (RAJS) in Burkina Faso provides sexual and reproductive health education inputs for EQuIP at post-primary and secondary level, and is wanting to extend its activities to working with students in the last two years of the primary cycle if possible. Given the enormity of the challenges in improving the quality of education in Guinea Bissau, EQuIP needs to work in closer collaboration with other organisations such as INDE and FEC, who are already involved in providing different types of training for teachers and other education personnel in Gabú and Bafatá. There are also other initiatives, such as the EU-funded FEC project to improve data collection in Gabú, which need to be considered in determining EQuIP’s inputs in the future. In Burkina Faso EQuIP should continue to strengthen its partnership with the Ministry for Social Action, which can provide support for children with disabilities, pregnant girls and other vulnerable children. In both countries EQuIP needs to strengthen coordination with the Ministries of Health and other governmental agencies which can provide support to programme beneficiaries. Plan Country Education Programmes Both Plan Country Education Programmes complement and support EQuIP as they focus on improving education access and quality, inclusion, and targeting vulnerable children including children with disabilities. They also include school governance and improving participation of citizens in education decision-making at all levels. Burkina Faso Under the Country Strategic Plan for 2015-2020 the education programme focuses on the right to education, improving education access and quality for children who are outside the education system, and targeting vulnerable children including children with disabilities or special educational needs. It also

International Solutions Group www.theisg.com 49 includes school governance and improving participation of citizens in education decision-making at all levels and advocates for improvements in learning conditions in a safe school environment. Following on from the BRIGHT projects I and II there is a BRIDGE project which targets post-primary children in two provinces, to improve access to and the quality of post-primary education. The Education Coordinator who is based in the PU in Gaoua with the EQuIP Coordinator manages a 5-year initiative (2012-2017) “Développer les capacités des jeunes pour faire croître l'économie” (DCAJ), to build young people's capacity for economic growth. The Education Coordinator also works on sponsorship initiatives, and supports social mobilisation in the region. In Gauoa PU there are regular meetings, to bring staff together to ensure coordination and mutual support. Guinea Bissau The education programme is focussed on inclusion, school governance, early childhood care and development, and increasing access and quality of learning in primary schools in the regions of Gabú and Bafatá. The programme includes construction and repair of pre-school and primary schools, including one new pre-school centre next to the EQuIP primary school in Dara. Pedagogical kits have been provided for 15 primary schools in Bafatá and Gabú. Training has been provided for inspectors, teachers, head teachers and parent’s associations in collaboration with FEC. School management committees and school governments were created and trained. VSL groups have been developed, with input from APRODEL. Awareness raising activities have been carried out, including radio programmes on the importance of girls’ education and the inclusion of children with disabilities. The International Day of Persons with Disabilities was celebrated (with UNICEF, UNDP and other NGOs), and two local NGOs were given support to develop advocacy action plans.

International Solutions Group www.theisg.com 50 F. Programme Monitoring and Evaluation

The evaluation team noted ongoing challenges with the availability of data needed for project monitoring and evaluation. This problem is particularly acute in Guinea Bissau, where for example in 6 out of the 15 schools head teachers did not even have basic data to share. The Ministries of Education do not have the resources they need to be able to collect, analyse and disseminate data within the necessary timeframes for EQuIP, nor in enough detail. For example, in Guinea Bissau the office where the Regional Education Director for Gabú and his team are based has no electricity. In Burkina Faso the Regional Education Directorate had collected data on inclusive education, but at the time of the evaluation this was still in raw forma, due to the lack of resources to process and analyse the data. Data, if available, does not cover all areas of programme intervention, and is not sufficiently disaggregated for EQuIP’s purposes. For example, data would not be readily available on learning achievements by level or type of disability. There is also a lack of resources available for monitoring, evaluation and learning within EQuIP, the MER departments in the PUs and the MER departments at country level. At programme and PU level monitoring and evaluation is mainly limited to documenting activities conducted. At the PU level, for example, the monitoring and evaluation coordinators are responsible for all programmes, and therefore do not have the time to monitor and evaluate EQuIP outcomes. EQuIP is not known to most of the students and parents interviewed in either country, indicating that there has not been participatory monitoring of programme progress involving the wider community. This lack of widespread participation by beneficiaries in programme monitoring and development means their input has not been used to establish what is working well and what needs to be adapted, and they have not been given the opportunity to be as active as they should have been in their own development under Plan’s Child-Centred Community Development Approach (CCCD). The results-based management framework has been revised and updated, on an annual basis and in line with other information received such as the mid-term evaluation of EQuIP in Burkina Faso. However, the results of this evaluation show more limited progress against many indicators than had been recorded to date, and even regression from the baseline position in some cases. This final evaluation shows that a number of Year 4 targets have not been met. In order to provide comparable measures of programme progress, the same evaluation tools were used for the final evaluation as had been applied for the baseline in both countries and for the mid-term evaluation in Burkina Faso. Thus, the discrepancy between successive year’s results is of concern. This report has noted specific instances where measured results are not in line with previously reported results, along with possible reasons for the discrepancy. In summary, these reasons are: - Potential selection biases between the baseline, interim and endline surveys, including selection of higher-performing students for tests by teachers, noted (and prevented) by the endline evaluators; - Cognitive biases amongst respondents who, for example, believed they understood topics such as children’s rights without a genuine grasp of them;

International Solutions Group www.theisg.com 51 - Response biases from survey respondents who may have been reluctant to display their lack of awareness on topics, or to demonstrate what they would perceive as ‘unwelcome’ attitudes to enumerators (particularly if such enumerators were not objective); - Sampling errors due to the relatively small number of schools and different samples sizes selected for surveys; - Over-reporting by partner/programme staff who may have been keen to demonstrate positive programme performance; - Differences in analysis of key indicators between survey iterations – the final evaluation team attempted to contact the baseline survey team to discuss analysis procedures but were unsuccessful. Further resources need to be made available to facilitate effective participatory programme monitoring against all the indicators in the results-based framework. This will enable EQuIP staff, Plan staff, partners and beneficiaries to be able to use the framework as an effective tool to inform programme planning and management in a meaningful way. In Gabú, EQuIP should work in collaboration with an EU-funded initiative on education data management, involving the Regional Government and Regional Education Department in partnership with FEC. In Guinea Bissau it is not possible to attribute how much of the improvement in learning outcomes is as a result of EQUIP’s interventions and how much due to FEC’s work, as they are both active in the same areas. More emphasis needs to be given to monitoring the progress and achievements of students with disabilities, both producing disaggregated data and spending time with children individually to document and follow up on the individual difficulties they face in accessing a quality education.

International Solutions Group www.theisg.com 52 G. Programme performance against the DAC criteria Relevance EQuIP is very broad in scope, with many components and activities, and its success depends on the successful coordination of a range of outcomes, ensuring that as more children come in to the system, schools are managed in a participative way and are ready to provide a quality education for all, in line with their needs. This is very ambitious – especially given the context of two countries that have gone through significant political upheavals and begin from a very low point in terms of educational outcomes. However, despite the complexity of needs, relevance of the programme is unquestionable. In Burkina Faso, EQuIP addresses issues identified within the national education sector plan, the PDSEB for 2012—2021, within the structures and mechanisms of the Ministries of Education responsible for primary, post-primary and secondary education. The Ministries of Education at regional and provincial level have very much assumed ownership and management of the programme, using it to address problems such as the low reading level of students, and advocating for the scale-up of successful activities. In Guinea Bissau, EQuIP is also in line with the sector policies currently being implemented, and experience from the programme is feeding in to the development of the Education Sector Plan for 2016-2025. Unfortunately, the Ministry of Education is not in a position to support the programme as actively as needed, with a serious lack of follow-up and support after training is given, impacting the achievement of intended outcomes. EQuIP is also in line with Plan’s national education programmes in Guinea Bissau and Burkina Faso, and with WARO’s regional education strategy. The log frame is logical, and the project design is appropriate for achieving the overall programme goal, but is really over ambitious in scope in view of the resources available. In Guinea Bissau the scope of the project should have been reduced, due to the lack of capacity within the Ministry of Education and the turmoil caused by the coup – unless additional partnerships with organisations like FEC could have been forged to take over implementation of some of the activities and provide a level of follow-up not otherwise available. As outlined within the report, valuable progress has been made against each of the first three objectives, but has been very limited on Objective 4. The final evaluation shows despite the log frame being revised on a regular basis some of the targets for Year 4 have not been achieved. Despite programme effectiveness (see below) in improving educational outcomes and providing education services, the evaluation team noted anecdotal reports of many out-of-school children, some of whom have never entered the system, and others who have dropped out. There is a paucity of quantitative data regarding this, but it is clear that there is a significant need for EQuIP-style interventions – the excess of demand for school places over availability is testament to this. Given high educational staff turnover in both countries and poor support to teachers in Guinea Bissau, EQuIP training is extremely relevant. Indeed, in many cases the quality/quantity of teaching staff is the weak link in the education system – many initiatives that contribute to positive educational outcomes (school governance structures, VSL groups, parent involvement) are present, but the overall quality is impeded by teaching capacity.

International Solutions Group www.theisg.com 53 While the progress of children with disabilities is difficult to assess due to limited data, the evaluation findings indicate that the many children with disabilities in programme communities can benefit from access to even a bare minimum of facilities tailored to their needs, and that peers, parents and teachers are generally willing to accommodate them. This applies generally to those with physical, rather than intellectual disabilities. The more specialised support and resources required for children with more profound disabilities make this cohort of extremely vulnerable children difficult to reach with EQuIP. With respect to children’s’ rights, there is clearly a need for enhanced awareness and avenues of protection for children, especially girls, who are particularly vulnerable to exploitation and abuse. Individual instances brought to the attention of the evaluation researchers (and the varied cultural dynamics in play in programme communities) make it clear that this area of intervention is both relevant and highly important. What further work may be necessary is on establishing mechanisms or avenues for enforcement of children’s’ rights in a safe and effective manner. Finally, the presence and success of EQuIP-facilitated VSL groups in three of five communities covered by EQuIP in Burkina Faso and 38 out of 41 in Guinea Bissau indicate they are key to assisting parents in covering the costs of education, particularly in communities where children are at risk of being taken from school to participate in income-raising activities (such as gold-mining or cashew-nut harvesting). Efficiency The staffing structure is very light in-country, for a very broad and ambitious programme, and relies on having very competent staff as Country Coordinators. The programme in Guinea Bissau required more support from Plan INO, and earlier action should have been taken to ensure the Coordinator was located within the project area. Basing the Coordinator in Bissau may have been an inefficient use of resources. More systematic participatory monitoring and evaluation, involving partners and beneficiaries, would have benefited the regular revision of the log frame. However, thanks to Plan’s experience and well established relationships with the Ministries of Education and the communities, and drawing on Plan’s expertise in key areas such as community sensitisation and VSL, much has been achieved with the resources used. In Burkina Faso, actors at the local level reported that they were satisfied with the management and administration of EQuIP by the Coordinator in Gaoua, and although some reported they would like to have a focal person for EQuIP based in Batié, the issue mainly concerned the difficulties of communication and sending documentation to the PU in Gaoua within tight timeframes. Due to the strength of the primary-level inspectors in Noumbiel, who are involved in facilitating much of the EQuIP training themselves, and other development activities, there is generally a good level of follow- up to training delivered, to help teachers apply what they have learned in training in their classrooms. This has obviously affected outcome quality and efficiently builds on available assets. In Guinea Bissau, the evaluation team agreed with actors at the local level expressed their concerns about the way the programme is managed, as the Coordinator is based in Bissau and does not spend as much time in the field as is necessary to manage the programme effectively. All aspects of programme management, implementation, monitoring and evaluation would be carried out more efficiently if the

International Solutions Group www.theisg.com 54 EQuIP Coordinator was based in the programme area, avoiding unnecessary delays in communications and facilitating networking, coordination and cooperation. Due to inherent capacity limitations within the Ministry of Education, in Guinea Bissau there has been a lack of follow-up on training activities, for example to support teachers in applying what they have learned in the classroom. This has affected the quality of outcomes. Teachers did not feel in general that prolonged teacher training interventions during the holidays were as useful as regular and ongoing training during term time, for example during the regular fortnightly cluster meetings. There has been considerable turnover in staff at all the key levels, impacting on efficiency of the programme. In Guinea Bissau this has included the Regional Director level, as well as head teachers and teachers. In Burkina Faso this has included the Provincial Director, head teachers and teachers. Teachers in Noumbiel province are generally young and inexperienced, as many teachers request transfers to leave the province as soon as they have completed the minimum time stated in their contract (3 years to transfer to another province within the region and 6 years to move to another region). This means that many members of staff who had been trained and involved in the project have moved to new positions, outside the EQuIP programme area. Depending on when training has been delivered and when new staff have come in to post, some new recruits have not necessarily received any EQuIP training. For example, some training has been provided in blocks during the school summer holidays, so anyone joining at the start of the new academic year has not benefitted. In order to overcome this problem, staff should receive regular training inputs during the academic year, with monitoring and support to ensure this training is applied. EQuIP has successfully capitalised on the expertise of local organisations through partnerships to support work with VSL groups, for sexual and reproductive health education (in Burkina Faso) and for support working with children with disabilities (Guinea Bissau). In both countries EQuIP needs to work more closely with other Ministries, such as Health and Social Action, who can support more vulnerable children. Effectiveness Some performance indicators have improved greatly under EQuIP, including for example the CEP exam pass rates and the scoring on the School Equality Scorecards. Other indicators, such as the continuing high rates of dropout, are a cause for concern. The risks associated with high staff turnover, which existed even before the political turmoil erupted in both countries, were not included on the original list of risks. When the problem became even more acute in both countries, but particularly in Guinea Bissau, it was not identified and dealt with at an early stage. Whilst some data is available to gauge the effectiveness of education interventions, for example enrolment figures and exam success rates, there is a general lack of information to follow progress against some of the programme indicators in the results framework. This is partly due to a lack of necessary disaggregated data, for example to assess the learning achievements and progress of children with disabilities. It is also due to the shortage of human and financial resources allocated for monitoring and evaluation, which should be carried out by both within EQuIP. Whilst Plan staff and partners showed an appreciation of the importance of the results framework,

International Solutions Group www.theisg.com 55 they also shared a sense of frustration that the necessary resources are not available to be able to follow all the indicators in a meaningful way. The Ministries of Education in both countries are showing commitment to the programme, with a greater sense of ownership in Burkina Faso due to the skills and capacities within the Ministries and also the hard work and energy of key personnel, including some of the primary inspectors. The quality of programme outcomes would be strengthened with the placement of the Coordinator in the project area in Guinea Bissau. The evaluation results show that EQuIP has been successful in increasing access for the two main target groups, girls and children with disabilities. However, there are still many children out of school, some of whom have never entered the system, and others who have dropped out. The programme has made many inputs to improve the quality of education, but due to the high level of staff turnover in both countries and the lack of follow-up and support in Guinea Bissau, not all of the training given has been implemented effectively within EQuIP schools. There has been a significant drop in the number of non-readers, and although the targets for year four have not been met, numbers of non- readers are likely to drop quickly in Burkina Faso next year as the benefits of the early reading approach started in 2014 are fully felt. Results are likely to improve in Guinea Bissau too, with more children having access to books, and more focus placed on reading. In both countries respondents reported an improvement in teaching methodology, and in Burkina Faso this was very evident in many schools, with the widespread use of group work. There is a lack of disaggregated data to confirm the progress of children with disabilities, but the evaluation findings indicate that some children with disabilities are doing well within the system - generally those with physical disabilities who may find it more difficult to get to school, but who have no barriers to learning once at school; whereas learning outcomes for children with disabilities who require special resources and more specialised support are more limited, given the amount of training teachers have received and the lack of specialist resources. The evaluation team found that teachers, head teachers, inspectors and Regional and Provincial staff were familiar with EQuIP, and its activities and achievements. However, children and parents in general were unfamiliar with the programme itself, though they were participating in/benefiting from its activities. The level of participation by parents and students is varied, but developing. There are active parent’s associations in most schools in Guinea Bissau, and in all 20 schools in Burkina Faso albeit with varied levels of activity. Mothers Associations are also active in all but one of the primary schools visited in Burkina Faso. There are school management committees in all 15 schools in Guinea Bissau, and in 19 schools in Burkina Faso, one of which was reported as not being active. School governments are also in the majority of EQuIP schools in Guinea Bissau, and in all but one of the 20 schools in Burkina Faso (Bobera). Some proactive initiatives being taken by school governments and students associations were noted in some schools, addressing a range of issues they have identified. In other schools, activities are much more limited, for example focusing mainly on tasks such as cleaning and watering. Students need to be supported to develop their role in management and decision-making.

International Solutions Group www.theisg.com 56 EQuIP has raised awareness of the rights of all children, especially girls and children with disabilities, in all programme schools and communities, and more children from these main target groups are in school. VSL groups are active in three out of the five communities covered by EQuIP in Burkina Faso and in 38 out of the 41 communities in Guinea Bissau. Members of VSL groups and other stakeholders explained how these groups are helping to cover education costs, both through funds run by VSL groups specifically for schools and through the increased financial capacity of individual members. VSL facilitators and members are also playing a key role in sensitisation and increasing awareness among communities. Impact From both primary and secondary data collection, the evaluation has identified areas where EQuIP is having a very positive impact. For example, there is an ongoing increase in the number of children enrolling in school, including girls and children with disabilities, due to the very effective sensitization and mobilization which has been carried out. It is evident that there is now more openness open in talking about children with disabilities both within the family setting and communities, and improved recognition of the rights of all children to an education. However, access in itself is not enough for either children with disabilities or other learners. The learning achievements of all students must be monitored, and children with disabilities also require teachers who are equipped with the resources and skills to deal with their needs, and a network of support, including from health and social services. The issue of transport, access to school meals and financial support for working children is a major barrier to regular access to school for many children. At primary level there have been major changes in the way children are taught, especially in Burkina Faso. Many of the inspectors, head teachers and teachers are enthused by what they see happening in their schools, in particular the excellent CEP results and the progress children are making in reading. Other indicators, including completion, repetition and dropout still give cause for concern in both countries. More emphasis needs to be placed on monitoring learning outcomes and assessment for all children, at post-primary and secondary as well as primary level. There is need for specialist input on quality, including teaching and learning in a bilingual or multilingual context. Despite the successes in increasing access and improving quality there are still many children out of school, and factors such as the economic status of families, child labour, early marriages and pregnancies, the distance to school lack and lack of school canteens mean there is a continued high dropout rate. Both countries have experienced political upheavals since the start of the programme, which have affected its implementation. For example, there has been a very high turnover of staff in key positions. Despite these problems implementation has progressed, but the level of achievement on outcomes has been affected. Sustainability Traditional views of girls and children with disabilities have started to change, communities are responding by enrolling their children in school, and the challenge is to ensure they stay in the system and receive a quality education.

International Solutions Group www.theisg.com 57 The objectives of the EQuIP programme are priorities for the Government of Burkina Faso within its education sector plan, and will hopefully be reflected in the new education sector plan being developed for Guinea Bissau for 2016-2015. Plan is active in this process, working closely with all the other major actors at central level, to try to ensure that inclusive education is given as high a priority as possible in this document. In Burkina Faso, the Ministry of Education staff at all levels feel a strong sense of ownership of the programme and are very proud of the achievements to date. People talk with pride about the changes they personally have been able to implement and the positive outcomes for the students. EQuIP supports the national education sector plan, and project interventions and Ministry follow-up are within the normal structures. Ministry staff are responsible for designing and delivering training, unless it is in an area where people do not have any prior experience, when external expertise is brought in to start the process, and then the Ministry takes charge and follows up. The Ministry of Education is very proactive in terms of advocacy, and its members are determined to share their successes and scale–up activities with the Ministry’s support. Therefore, the sustainability of the programme gains is likely. The high level of staff turnover does, however, present an impediment to sustainability at all levels. In Burkina Faso this is because many teachers request for a transfer as soon as they have spent the requisite minimum time in the province. This means that trained and experienced teachers leave, and new untrained and inexperienced teachers generally take their place. Some of the key Ministry of Education actors for EQuIP, including primary inspectors, have been sent to the province, but their families are not with them – so they may not stay in those positions for the long-term, and their successors may not sustain the gains in the same way. In Guinea Bissau the sense of ownership is not generally as strong, but stakeholders and beneficiaries are aware of the gains made, and will want to protect those achievements. Participation by beneficiaries needs to be further strengthened to ensure sustainability. As in Burkina Faso, staff turnover is also a risk to sustainability.

International Solutions Group www.theisg.com 58 H. Conclusion and Key Recommendations

The Theory of Change for this programme is logical in that EQuIP is broad enough in scope to leave no major gaps in what needs to be achieved to improve the quality of education, but the reality is that there are not adequate resources to achieve all the anticipated outcomes or to facilitate monitoring and evaluation, including regular participatory monitoring, to inform regular revisions of the results framework. Progress has been made against all the objectives, to varying degrees, but often interventions have been made without the necessary level of follow-up to be effective, or adequate monitoring to establish to what extent outcomes are being achieved. EQuIP should certainly be continued, to build on the gains already made, but the scope of the project should be in line with the resources available. For immediate action: Guinea Bissau and Burkina Faso  community participation in all stages of programme planning, implementation, monitoring and evaluation should be strengthened. This is in line with Plan’s child-centred community development approach and will help to ensure EQuIP evolves to meet beneficiaries’ needs in the most effective way. It will also raise the profile and understanding of EQuIP at community level

 given the enormous disparities in the REACH reading results across the schools covered, REACH tests should be conducted across the remaining EQuIP schools, and the lowest performing schools should benefit from an intensive reading programme, to help the teachers improve their teaching of reading and to raise the reading levels of the children. Primary inspectors should be supported to monitor reading results across their schools on a regular basis, and to take further action as necessary. Objective 1

 more emphasis should be placed on developing individual learning plans for students with disabilities and on documenting their learning outcomes, so disaggregated data is available to measure progress and inform planning. Objective 1 & sub-objective 2.2

 In-depth interviews should be conducted with a range of children with disabilities, including both those in school and a sample of those who have dropped out, and their families, to determine how disability-friendly schools are for those children. Sub-objective 2.2

 additional sensitisation needs to take place to bring more girls with disabilities in to schools, Objective 2 Guinea Bissau

 the Country Coordinator should be located within the project area. Programme management  in-service training for teachers should continue to be provided on a regular basis throughout the academic year, through the fortnightly cluster meetings, in collaboration with other organisations

International Solutions Group www.theisg.com 59 providing teacher training, follow-up and support, including INDE, UNESCO, GPE and FEC. Objective 1  specialist input on teaching and learning in a bilingual or multilingual environment should be included in training programmes for teachers, to improve learning outcomes. Objective 1

 EQuIP should collaborate with the EU-funded data management project to support the Regional Education Directorate in Gabu (which is being run in partnership with FEC). Objective 1

 EQuIP should approach the Ministry of Education, at regional and central levels, to see if the third shift can be reinstated in schools where this is needed to accommodate the number of children wishing to attend. Objective 2

 there should be greater coordination and collaboration with the Ministry of Health, to provide the necessary support for children with disabilities. Objective 2

 the recently built resource centre in Gabu should become the focus for resources and activities to support inclusive education, with a focus on children with disabilities Objective 2.2  more focus should be placed on preparing evidence-based advocacy from EQuIP to feed in to the current development of the national sector plan for 2016-2025, and to inform national policy development and implementation. Objective 4 Burkina Faso

 there should be more support provided by EQuIP to facilitate communication (by phone and email) between Batié and the PU in Gaoua. Programme management

 discussions should be held with the Ministry of Education, at provincial and regional level, regarding the feasibility of re-instating the monthly training and planning sessions (Groupes d’Animation Pedagogique - GAP) which used to be held for primary teachers, or supporting another format if not. Objective 1

 at post-primary level in Burkina Faso EQuIP should provide support as necessary, to improve the timely supply of adequate supplies of textbooks to schools in Noumbiel from central stocks, and the most equitable distribution of textbooks to students in all secondary schools. Objective 1

 VSL should be extended to the two communes where it is not currently operating. Objective 2  Education Assistants working in “Service Vie Scolaire” in secondary schools should be included in training and follow-up activities related to pregnancies, children with disabilities and other vulnerable children. Sub-objective 2.1

 there should be greater coordination and collaboration with the Ministry of Health, and also with the Ministry of Social Action, to provide the necessary support for children with disabilities, pregnant schoolgirls and other vulnerable children. Sub-objectives 2.1 & 2.2

International Solutions Group www.theisg.com 60 For future development of the EQuIP programme: Guinea Bissau and Burkina Faso

 key staff from the Ministries of Education and other partner organisations should be trained in participatory monitoring and evaluation  support should be given to students and parents dealing with difficult issues such as unacceptable behaviour by teachers and head teachers. This should include facilitating the participatory development of codes of practice for each school with guidelines on actions to be taken if the school code of practice is broken. Objective 3

 more research needs to be done to establish how many children are still out of the system, the barriers preventing them from accessing education, and strategies developed to bring them in to schools and keep them there. Objective 2 Guinea Bissau

 EQuIP should advocate for the provision of the first six grades at all primary schools. Currently 35 out of the 41 EQuIP schools only offer the first four grades. Objective 2

 EQuIP should support any collaboration and coordination with madrassas (dependent on policy decisions made in future by the Ministry of Education) Objective 2

 EQuIP should develop a partnership with an organisation which can facilitate the introduction of sexual and reproductive health materials in the last two years of the primary cycle, subject to agreement from the Ministry of Education, religious leaders and other community leaders. Sub- Objective 2.1 Burkina Faso  EQuIP needs to consider how it can best support the development of “the Continuum”, integrated complexes for the provision of pre-school, primary and secondary education. Objectives 1, 2 & 4

 input to improve learning outcomes at post-primary and secondary level should be expanded, to reduce the high repetition rate and improve the exam success rates at post-primary and secondary level, particularly for girls. Objective 1

 sexual and reproductive health education should be provided for the last 2 years of the primary cycle onwards, as well as at post-primary level, in conjunction with RAJS. Subject to agreement from the Ministry of Education, religious leaders and other community leaders. Sub-objective 2.1

International Solutions Group www.theisg.com 61 Annex 1: EQuIP Results Framework – Burkina Faso

Goal/Outcome Indicator Evaluation findings/vs. baseline target & Year 4 target Interim Goal: Completion rate disaggregated by gender in primary, Baseline: To increase access to and quality of primary and junior secondary (JS) and secondary schools (at national Primary Total: 52.1% Girls: 49.1% Boys: 55.1% secondary education for all girls and boys level – note that EQuIP is implemented in a small area and Junior Secondary Total: 17.5% Girls: 14.3% Boys: 21.2% has little impact on this national indicator) Secondary Total: 8.1% Girls: 6.0% Boys: 10.2%

Evaluation: Primary Total:46.7% Girls: 47.8% Boys: 45.7% Junior Secondary Total: 14.2% Girls: 9.5% Boys: 18.3% Secondary Total: 4.9% Girls: 2.8% Boys: 6.7%

Year 4 target: REVISED PLAN targets: Primary: 65% (50/50 M/F) JS: 28% (50/50 M/F) Secondary: 11% Gross enrolment rate in primary, JS and secondary Baseline: schools disaggregated by gender Primary: Total: 77.6% Girls: 75% Boys: 80.2% Junior Secondary Total: 32.3% Girls: 28.6% Boys: 36.6% Secondary: Total: 10.7% Girls: 7.7% Boys: 13.9%

Evaluation: Primary Total:72.6% Girls: 75.4% Boys:70.2 % Junior Secondary Total: 48% Girls: 44.3% Boys: 53% Secondary Total: 7% Girls: 3.8% Boys: 9.7%

Year 4 target: REVISED Primary: 85%; Junior secondary: 42% ;Secondary : 15% Outcome 1: Exam success rates for boys and girls in target primary Baseline: Increased learning outcomes for children in (CEP), junior secondary (BEPC) and secondary schools CEP (2011): Total: 61.4% Girls: 54.6%; Boys:67.4% target primary and secondary schools (Baccalaureate) BEPC: Total: 38.82% Evaluation:

International Solutions Group www.theisg.com 62 CEP (2014): Total: 92.8% Boys: 94.7% Girls: 90.9 % BEPC (2014): Total: 31.5% Girls: 16% Boys: 46% (average of 3 schools) BAC (2014): 16.6% (1 school, 5 girls & 8 boys passed)

Year 4 target: CEP: Girls: 95% Boys: 96% BEPC: Girls: 25% Boys: 44% Proportion of Grade 3 school children [boys and girls] Baseline: who cannot read in target primary schools Total: 49% Girls: 56% Boys: 41%

Evaluation: Total: 29% Girls: 29% Boys: 29%

Year 4 target: Total: 15% Girls: 15% Boys: 15% 1.1: Teachers adopt child friendly and inclusive Proportion of teachers in target schools who use child- Baseline: teaching methods friendly and participatory teaching methods Primary school: 57.1% Good JSS: data not collected

Evaluation: Classroom Management: 72.22% good; Lesson plans and activities: 70.25% good; Average: 71.24% Year 4 Target: Primary school: 95% Junior Secondary school: 90% 1.2 Students avail of increased opportunities for Pupil-textbooks ratio in target primary, JS and secondary Baseline: Reading 1,69:1; Maths: 1:1 learning schools Evaluation (Natl statistics): Reading 0,9: 1 ; Maths: 1.06 Region Sud-Ouest: Reading 1,21: 1 ; Maths: 1.52:1 At primary level 9 schools said every student had all the books they needed, but 6 schools said students were missing some books. Post-primary level has a shortage of textbooks, varying between schools and subjects. 5 Maths books for 50 students for example in one class. Year 4 Target: number of reading book per pupils=1.5 : 1 number of mathematics books per pupils= 1.5:1

International Solutions Group www.theisg.com 63 1.3 Teachers receive regular & relevant guidance Number of teacher supervisions documented in relation Baseline: Data not collected from education authorities to improve their with the total number of teachers in EQuIP schools Evaluation: Quantitative data not available. Number of visits varies teaching methods in target schools greatly across primary, post-primary & secondary schools. Head teachers reported that visits were more frequent under EQuIP.

Year 4 Target: Each teacher receive at least 2 supervision visits per year 1.4 Provincial education authorities promote Methods of work used by education authorities are The early reading scheme is being promoted beyond provincial level, to best practices from the EQuIP programme influenced by EQuIP regional level; & steps are being taken to promote it at national level – upwards in their region dependent on support from MENA. Outcome 2.1: Gross completion rate in target primary, JS and secondary Baseline: Increased numbers of girls completing basic schools disaggregated by gender Primary: 58.7% (62.2% total) education [primary and junior secondary] Junior Secondary: 5.45% (10.9% total) Secondary: 1.0% (2% total)

Evaluation: Primary: 47.8% (46.7% total ) Junior Secondary: 9.5% (14.2% total) Secondary: 2.8% (4.9% total)

Year 4 Target: National targets by 2015 (PDSEB) Primary: 75.1% (50/50 M/F) JS: 12% Secondary: 4% Gross enrolment rate in primary, JS and secondary Baseline: schools disaggregated by gender. Primary: B=78.7% G=80.4% T=79.5% Junior Secondary B=33.6% G=20.6% T=27.4% Secondary: B=5.3% G=1.9% T=3.7%

Evaluation: Primary: B=70.2% G=75.4% T=72.6% Junior Secondary B=53% G=44.3% T=48% Secondary: B=9.7% G=3.8% T=7%

Year 4 Target: Primary: G=80%; JS: G=45%; Secondary: G=5%

International Solutions Group www.theisg.com 64 2.1.1 Parents in targeted communities have Proportion of loans used for education purposes Evaluation: 31% of loans used for education increased commitment and financial capacity to Year 4 Target: 25% send their daughters to school Proportion of households with a poverty index 50% Baseline: 75% of households in rural areas in Burkina live with less than below the national poverty line in EQuIP communities $1.25/day (extreme poverty line) Source IFAD, 2011 Evaluation: 44.6 % of the population live below the poverty level, on less than USD 1.25 per day (Human Development Report 2013) Year 4 Target: Target to be determined on the basis of the baseline data that will be collected in year 3 Proportion of parents and guardians (women and men) Baseline: Total: 86.7% Men: 81.3% Women: 92.9% in EQuIP communities who declare enrolling their Evaluation: Total: 61.7% children in school regardless of their gender (15% prefer to enrol boys, 23,3% prefer to enrol girls) Year 4 Target: Total: 90% Men: 95% Women: 85 % 2.1.2 More child friendly school environment Proportion of boys and girls in target schools who think Baseline: Total: 86.7% Men: 81.3% Women: 92.9% that the school offers a gender equal environment Evaluation: Total: 92% Men: 95% Women: 89% Year 4 Target: Total: 90% Men: 95% Women: 85 % 2.1.3 Increased opportunities for girls to Gender parity index in GER in target JS and secondary Baseline: Junior secondary: 0.61: 1 / Secondary: 0.35: 1 transition to secondary school schools Evaluation: Junior secondary: 0.84: 1 / Secondary: 0.39: 1 Year 4 Target: Junior secondary: 0.85: 1 / Secondary: 0.40: 1 2.1.4 Secondary school pupils have increased Number of pregnancies in target JS and secondary Baseline: 43 pregnancies from a total number of 775 girls knowledge on practices related to sexual and schools Evaluation: 39 pregnancies in 6 secondary schools covered reproductive health Year 4 Target: 10 pregnancies Outcome 2.2: Number of children (0-18), boys and girls, with Baseline: No complete data set available for the region. 17 children with A more conducive environment for inclusive disabilities, regardless of type of disability identified in physical disabilities reported in Noumbiel. education for children with disabilities exists in EQuIP communities 9 CWDs declared by parents during the baseline, conducted on a sample Noumbiel of communities in Noumbiel Evaluation: Total 146; Girls 64; Boys 82 (in 20 EQuIP schools) Year 4 Target: N/A Proportion of girls and boys with disabilities enrolled in Baseline: 6 CWDs identified during the baseline study target primary, JS and secondary schools Evaluation: Total 20.6% [in 20 EQuIP schools; Girls 44%; Boys 56%] Year 4 Target: 65%

International Solutions Group www.theisg.com 65 2.2.1 Reduction in stigma related to disabilities Proportion of parents in target communities who declare Baseline: 20% of parents declare knowing rights of children with knowing the rights of children with disabilities and can disabilities (28.1% men/10.7% women) give at least 1 example Evaluation: Total: 45% Men: 42.5% Women: 50% Year 4 Target: Total: 90% Men:80% Women: 95 % 2.2.2 Target schools become more disability Proportion of boys and girls with a disability who report Baseline: Data not collected friendly that their school is disability-friendly Evaluation: Data not available. Not all CWDs were present in schools covered, & many were not able to communicate. Therefore, insufficient data could be collected within the constraints of the evaluation. Qualitative data indicates additional efforts are required to meet targets. Year 4 Target: 85% of school CWD's indicate schools are disability- friendly 2.2.3 Plan and partners are using learning from Number of interventions of Plan and partners that have No information on this EQuIP to inform other initiatives aimed at specific elements that assist children with disabilities to supporting children with disabilities achieve their rights Outcome 3: Citizens in target communities share Proportion of functioning COGES, APE & AME in EQuIP Baseline: 0 ownership for better education with school schools Evaluation: 80 % of APEs are functioning well, 85% of AMEs, & 90% of authorities by actively participating in school have active COGES educational decision-making Year 4 Target: 75% of PTAs are functioning 3.1 Improved learning environment for children Proportion of boys and girls who declare feeling safe in Baseline: Boys: 64% Girls: 67% EQuIP schools Evaluation: Boys: 82% Girls: 89% Year 4 Target: Boys: 75 % Girls: 90% 3.2 Children are influencing school management Proportion of EQuIP schools with a functioning School Baseline: Data not collected in relation to their educational needs Government Evaluation: 90% Year 4 Target: 100% Outcome 4: MoE strategies for rolling out its The implementation of the National Education Policy MoE is documenting & presenting best practices with a view to rolling education policy take into account best practices [PDSB 2012-2021] proceeds according to plan out – including early reading programme in relation to quality and inclusion 4.1 Plan’s advocacy approach is SMART [Specific, Impact of EQuIP on school, provincial, regional & national Plan is active at central level, as well as PU level. measurable, achievable, relevant, time bound policy development & implementation MoE is being proactive, within MoE structures & mechanisms. 4.2 Best practices in relation to participatory Best practices in participatory school governance are Best practices & lessons learned are being shared by MoE at provisional, governance are used in informing collected across schools and used to support the regional & national level. Information in inspectors’ reports is passed on implementation of the Education Sector Plan development of standards of good governance in schools through the DPENA & DRENA to central Ministry.

International Solutions Group www.theisg.com 66 Annex 2: EQuIP Results Framework – Guinea Bissau

Goal/Outcome Indicator Evaluation findings/vs. baseline target & Year 4 target Interim Goal: Completion rate disaggregated by gender in Baseline: To increase access to and quality of primary and primary schools National: Total: 85.5%; Girls: 73.2%; Boys: 98.9% secondary education for all girls and boys Bafatá: Total: 40.2% / Gabú: Total: 54.4% Evaluation9: National: Total: 75.1%; Girls: 73.2%: Boys: 76.5% Bafatá: Total: 57.9%; Girls: 51%; Boys: 61.5% Gabú: Total: 81.7%; Girls: 82.6%; Boys: 81.2% Year 4 target: 100% by 2020 Gross enrolment rate in primary schools Baseline: disaggregated by gender National (2010) Total: 123% Girls: 119% Boys: 127% Bafatá: (G6) Total: 24.4% Girls: 22.3% Boys: 26,5% Gabú: (G6) Total: 50% Girls: 47% Boys: 53% Evaluation: National: Total: 86.1%; Girls: 82.1%: Boys: 90.1% Bafatá: Total: 89.4%; Girls: 88.3%; Boys: 91% Gabú: Total: 66.6%; Girls: 65.6%; Boys: 67.7% Year 4 target: 100% by 2020 Outcome 1: Exam success rates (CEP) for boys and girls in target Baseline -Database for EQuIP schools Increased learning outcomes for children in target primary schools Bafatá Total: 14.910% Girls: 14.6% Boys: 15.2% primary schools Gabú: Total: 15.3% Girls: 15.3% Boys: 15.3% Evaluation - Database for EQuIP schools Bafatá Total: 35.4% Girls: 29.5% Boys: 41.2% Gabú: Total: 68.4% Girls: 66.9% Boys: 69.8% Year 4 target: Bafatá Total: 60% Girls: 64% Boys: 54% Gabú: Total: 84% Girls: 84% Boys: 84%

9 Year 4 data was not available at national level, so Year 3 (2013/2014) data has been used instead 10 This is an approximation as the figure reported in the Results Framework was incorrectly calculated

International Solutions Group www.theisg.com 67 Proportion of Grade 3 school children [boys and Baseline: 77.6% girls] who cannot read in target primary schools Evaluation: 54.4% (40.3% in Bafatá & 68.56% in Gabú) Year 4 target: 25% 1.1: Teachers adopt child friendly and inclusive Proportion of teachers in target schools who obtain Baseline: teaching methods an overall mark of ‘good’ or ‘need improvement’ Classroom Management: 42% good; 45% needs improvement during academic inspections Lesson plans and activities: 15% good; 63% needs improvement Evaluation: Classroom Management: 38.33% good; 30.56% needs improvement Lesson Plans and activities: 19.72% good; 32.78% needs improvement Year 4 target: Classroom Management: 70% good; Lesson plans and activities: 50% good; 1.2 Teachers have improved competencies in Average mark obtained by a cohort of trainee Baseline: Data not collected Portuguese and maths teachers in mathematics and Portuguese in EQuIP Evaluation (Yr 3 Results): schools Bafatá: Port. Pre test-54.62%; Port. Post test-58.99%; (Difference: +4.37%) Math pre test - 52%; Math Post test - 72.08%; (Difference: +20.08%) Gabú: not available Annual targets: 20-30% increase between pre and post-test results in both subjects 1.3 Students have access to basic resources needed Percentage of children (boys and girls) who have at Baseline: N/A for learning least 1 reading book and 1 mathematics book Evaluation: 90% of students have at least 1 of each textbook Year 4 Target: 95% have Portuguese & Maths books 1.4 Teachers receive regular & relevant guidance Number of teacher supervisions documented in Baseline: N/A from education authorities to improve their teaching relation with the total number of teachers in EQuIP Evaluation: 0.72 (2014) methods in target schools schools Supervisory visits to each school vary greatly, no data available for visits per teacher, but clearly far from reaching target. Year 4 Target: 3 supervisory visits/teacher 1.5 Provincial education authorities promote best Methods of work used by education authorities are Inspectors said they are sharing best practices with non-EQuIP schools, practices from the EQuIP programme upwards in influenced by EQuIP when feasible, including through fortnightly cluster meetings for their region teachers, but the team saw no documented evidence of this.

International Solutions Group www.theisg.com 68 Outcome 2.1: Transition rates of girls from grades 4 to 5 and 5 to Baseline: Increased numbers of girls completing basic 6 in target regions Total: 55% Girls: 56% Boys:53% education Evaluation: Bafatá: Gabú Total: 35.4% Total: 68.4% Total:51.9% Girls: 29.5% Girls: 69.8% Girls: 49.65% Boys: 41.2% Boys: 66.9% Boys: 54.05% Year 4 Target: 40% Gender parity index in enrolment rate in target Baseline: Age (7-12)*EQuIP Baseline programme study overall report primary schools Girls: 47.2%; Boys: 52.8% Evaluation: Data not available Year 4 target: 1:1 in both regions 2.1.1 Parents in targeted communities have Proportion of loans used for education purposes Baseline: N/A increased commitment and financial capacity to Evaluation: 2014 EQuIP VSL survey: main use of credit 40.6% & secondary send their daughters to school use of credit 35.2% Year 4 target: 25% of savings used for educational purpose Proportion of parents and guardians (women and Baseline: 82% men) in EQuIP communities who declare enrolling Evaluation: Bafatá: 55 % Gabú:48.6% their children in school regardless of their gender Year 4 target: Bafatá: 60% Gabú:30% 2.1.2 More gender-equal school environment Proportion of boys and girls in target schools who Baseline: Girls:66% Boys: 85% think that the school offers a gender equal Evaluation: 84% environment Year 4 target: 90% Outcome 2.2: Number of children (0-18), boys and girls, with Baseline: Girls: 15 Boys:2311 A more conducive environment for inclusive disabilities, regardless of type of disability Evaluation: Data not available education for children with disabilities exists identified in EQuIP communities Year 4 target: 2% of primary school going population

11 The baseline measure was number, rather than a proportion, precluding meaningful comparison against endline data

International Solutions Group www.theisg.com 69 Proportion of girls and boys with disabilities Baseline: Data not available enrolled in target primary schools Evaluation: Data not available [but Bafatá 36 (10 girls, 26 boys); Gabú 35 (8 girls, 27 boys) Total 71 (In 15 schools including 2 pilot schools), (In 15 schools including 2 pilot schools), corresponds to 1.8% of total Class 1-6 population in schools. 25.35% girls / 74.65% boys Target: 40% girls Year 4 target: 40% girls 2.2.1 Reduction in stigma related to disabilities Proportion of parents in target communities who Baseline: Women: 33.33% Men:58.33% declare knowing the rights of children with Evaluation: Women: 3.4% Men: 44.07% disabilities and can give at least 1 example Year 4 target: Women: 75% Men: 80% 2.2.2 Target schools become more disability friendly Number of boys and girls with disability who report Baseline: N/A that their school is disability-friendly Evaluation: Data not available. Not all CWDs were present in schools covered, & many were not able to communicate. Therefore, insufficient data could be collected within the constraints of the evaluation. Qualitative data indicates additional efforts are required to meet targets. Year 4 target: 90% 2.2.3 Plan and partners are using learning from Number of interventions of Plan and partners that The evaluation team was not made aware of any such initiatives EQuIP to inform other initiatives aimed at supporting have specific elements that assist children with children with disabilities disabilities to achieve their rights Outcome 3: Citizens in target communities share Proportion of target schools with functioning Baseline: 23 SMCs ; 16 PTAs ownership for better education with school School Management Committees and parent’s Evaluation: All SMCs are functioning, & 87% of parent’s associations (13 authorities by actively participating in educational associations out of 15 schools) decision-making Year 4 target: 90% 3.1 Improved learning environment for children Proportion of boys and girls who declare feeling Baseline: Girls: 66% Boys: 88% safe in EQuIP schools Evaluation: Girls: 93% Boys: 100% Year 4 target: Girls: 80% Boys: 80% 3.2 Children are influencing school management in Proportion of schools in the region with a Baseline: 3 relation to their educational needs functioning School Government (with a plan of Evaluation: 80% (in 12 out of the 15 schools covered) action implemented) Year 4 target: 70% Outcome 4: The priorities identified in the next Education The evaluation team was informed that Plan is active in the development MoE strategies for rolling out its education policy Development Plan (2016-2025) reflect best of the education sector Plan for 2016-2025, and experience from EQuIP take into account best practices in relation to quality practices derived from Plan's education is feeding in to that process and inclusion programme, including EQuIP

International Solutions Group www.theisg.com 70 4.1 Plan’s advocacy approach is SMART (Specific, Evidence from the EQuIP programme and Plan’s The evaluation team was informed that Plan is active in development of measurable, achievable, relevant, time bound) advocacy strategy are used by national education the education sector plan for 2016-2025; EQuIP experience is feeding in coalitions in policy influencing and strategy to that process development 4.2 Best practices in relation to participatory Implementation of the current Education The evaluation team was informed that best practices are being fed in to governance are used in informing implementation of Development Plan 2011-2013) reflects child rights the development of the Education Sector Plan 2016-2025 at central level the Education Sector Plan in the school curriculum and an in-service training by the Education Specialist. approach for teachers.

International Solutions Group www.theisg.com 71 Annex 3: Assignment Terms of Reference

SUMMARY

Plan Ireland is requesting applications from research organisations, universities, academic institutes or think tanks for the final evaluation of Plan Ireland’s flagship Education programme ‘EQuIP’ (Education: Quality, Inclusive, Participative). The study should be carried out between April and October 2015.

The programmes overall objective is to increase access to and quality of primary and secondary education for all girls and boys in 4 target countries (Burkina Faso, Guinea, Guinea Bissau and Sierra Leone) in West Africa.

Plan Ireland secured funding for the EQuIP programme through Irish Aid’s Programme Funding for 4 years from January 1st, 2012. A fifth ‘bridging’ year has been accorded in order to appropriately evaluate the programme cycle and prepare for the subsequent programme cycle.

Due to the Ebola crisis in West Africa, the final evaluation will only target Burkina Faso and Guinea – Bissau.

The deadline for submitting the application is 23rd of February 2015.

INTRODUCTION TO PLAN IRELAND AND EQUIP

Plan is an international child cantered development organisation working in 50 developing countries. With a focus on long term sustainable development, Plan assists children and their communities across a range of sectors including education, health, livelihoods and water/sanitation. Established in 2003, Plan Ireland supports Plan’s work worldwide with a particular focus on West Africa. Funding is received from a variety of sources including through the public via child sponsorship, Irish Aid, EU, trusts, foundations, trade unions and corporate firms.

Plan Ireland and Irish Aid are working in partnership to achieve the common vision of a world in which all children realise their full potential. Achieving this vision depends on both boys and girls enjoying their right to good education. It is largely recognised, that education is one of the most important investments a country can make and is critical to reducing poverty and inequality.

The EQuIP programme concentrates on grass root initiatives that contribute to increasing the quality of learning (such as teacher training and training of school directors). That makes schools more girl friendly and increases awareness amongst key duty bearers of the importance of education for all children, especially girls and disabled children.

These grass roots initiatives are supported by capacity building of local education authorities and strategic engagement on policy issues at national level that impact on access to and the quality of education. Key regional elements of the EQuIP strategy include:

 Quality Teacher Training  Regional Research on access of children with disabilities to education and protection services  Policy Engagement  Social Accountability  Participatory Monitoring of programme progress.

EQuIP overall description

The five results of the action are:

 Increased learning outcomes for children  Increased numbers of girls completing the education cycle in targeted primary (and secondary) schools

International Solutions Group www.theisg.com 72  A more conducive environment for inclusive education for children with disabilities exists  Citizens in target communities share ownership for better education with school authorities by actively participating in educational decision-making  MoE strategies for rolling out its education policy take into account best practices in relation to quality and inclusion

IMPLEMENTING PARTNERS

The Plan country offices in Burkina Faso and Guinea Bissau implements the projects in partnership with relevant local ministries and local implementing partners manage activities on the ground.

PURPOSE AND OBJECTIVES OF THE EVALUATION

The overall purpose of the evaluation is to assess the level of attainment of targets, results and objectives. The evaluation will contribute to shared learning and provide accountability to partners, beneficiaries and donors. It will identify lessons learned and make recommendations to guide and inform future similar projects and programs.

The evaluation will assess the performance of the project against key parameters including the project’s relevance, effectiveness, efficiency, sustainability, timelines of activity implementation, and its strengths and weaknesses. In addition, the strengths, weaknesses and challenges during implementation will be analysed with a view to extract lessons and best practices for the future.

OBJECTIVES

The final evaluation will provide the decision-makers in the Government, Irish Aid, Plan and the wider public with sufficient information to:

 make an overall independent assessment about the past performance of the project paying particularly attention to the impact of the project actions against its objectives;

 Identify key lessons and to propose practical recommendations for follow-up actions.

The evaluation will be based on the five evaluation criteria endorsed by the OECD-DAC (relevance, efficiency, effectiveness, sustainability and impact. The evaluation will also assess the integration and impact of cross cutting issues in the project).

The key parameters to be assessed by the evaluation are:

 Relevance: describes how well a project addresses a real problem of the beneficiaries and how well it matches the development policies and strategic objectives of the country/region.

 Efficiency: stands for how well the inputs are transformed into output and outcomes.

 Effectiveness: measures the degree to which the project’s outputs have provided benefits and contributed to the project purpose.

 Impact: describes how and to which degree the project has contributed to the solution of the problem and to the achievement of the overall objective. While actual impact can only be measured ex-post, the evaluation should nevertheless scrutinise the impact prospects, i.e. the project’s likely contribution to the project’s overall objective and results.

 Sustainability: measures the likelihood of a continuation in the stream of benefits produced by the project after the period of external support has ended.

International Solutions Group www.theisg.com 73  Mutual reinforcement (coherence): assesses the extent to which activities undertaken complement partner country's policies and other donor interventions. The consultants are required to use their professional judgment and experience to review all relevant factors and to bring these to the attention of the project partners.

SCOPE OF THE EVALUATION

The evaluation will cover the accomplishment of all the expected results as outlined in the project document and detailed in the annual work plans during the period of 1st January 2012 – December 2015. All the project result areas, outcomes and activities stated in the logical framework matrix of the project will be assessed (these documents are available on request to interested parties).

PROJECT REGIONS AND TARGET BENEFICIARIES

Guinea Bissau Burkina Faso Primary schools 41 92 junior schools senior school 7 girls 3,284 6,285 boys 3,332 7,313 total pupils 6,616 13,598 Girls with handicap in EQUIP schools 13 153 Boys with handicap in EQUIP schools 26 92 Total children with handicap in EQUIP schools 39 245 directors 41 99 Teachers 131 404 inspectors 20 35 VSLA groups total 68 228 Participants estimated including 2014 2,220 4,941 The evaluation team will select sample districts, schools, and communities from the project region in consultation with relevant stakeholders. They will use statistically acceptable parameters, and will ensure that the sample selected is representative of the geographical coverage of the project as well as the target beneficiaries. SPECIFIC TASKS

 The specific tasks of the evaluator will be to:

 Develop an evaluation framework and methodology and refine these with the project team.

 Develop a sampling frame that will be discussed and adopted by the evaluation team

 Develop an evaluation plan

 Undertake extensive document review of the project

 Conduct field visits, focus group discussions and interviews with relevant beneficiaries, partners and project team members to collect information on various parameters of the evaluation which are directly related to identified outcomes and outcome indicators in the results framework for the EQuIP programme. Both quantitative and qualitative information are required to adequately evaluate the outcomes and include (but not exhaustive) :

International Solutions Group www.theisg.com 74 Quantitative information will be related specifically to key indicators related to:

 Access to education for girls and children with disabilities (enrolment rates, dropout rates, repetition rates, primary completion rates, transition rates to secondary school)

 Teachers receive regular & relevant guidance from education authorities to improve their teaching methods

 Early grade reading assessments to determine levels of reading at Grade 3.

 Economic situation of families, in particular village savings and loans scheme members, and their ability to support the financial cost of education for all their children (both boys and girls)

Qualitative information will be collected in relation to:

 Community attitudes towards children with disabilities and attitudes relating to education for girls.

 Children’s (boys and girls) perception of the learning environment including specifically the level of gender inclusion in schools

 Teachers’ capacities to ensure a quality learning environment.

 Education Authorities capacity to provide guidance and supervision

 Level of awareness and knowledge of HIV AIDS

 School Management Committees, Parent Teacher Associations, Children’s Parliaments and the general communities’ level of engagement in school governance.

 The influence the project has had on local, regional and national educational policies

All information shall be disaggregated according to sex, area of residency (village, town or city) and disability.

The above information should be collected in a sample of communities participating in the EQuIP programme as well as in communities not participating in the programme as a counter factual to allow for impact evaluation. Ethical considerations in doing this should be considered and it may not be appropriate to collect all of the above information in these control areas.

Prepare the evaluation draft report and present the findings to Plan Ireland and the respective Country Offices

Prepare and submit the final report to Plan Ireland and the respective Country Offices

Present the final findings of the evaluation to Plan Ireland and the respective Country Offices and conduct an in country workshop in both Burkina Faso and Guinea Bissau to present the final findings of the evaluation.

EVALUATION APPROACH AND METHODOLOGY

The evaluation approach should be developed and implemented in three main phases: An inception phase, a field investigation phase and a synthesis and feedback phase.

Inception Phase

This phase will involve the thorough and systematic review of the relevant programming documents related to the project, as well as documents shaping the wider strategy/policy framework. The consultant will analyse the logical framework in order to determine relevant issues and evaluation questions. Specific tasks to be undertaken during the inception phase will include:

Develop and present an indicative methodology.

Interview the project management team

International Solutions Group www.theisg.com 75 Present each identified evaluation question stating the information already gathered and provide a first partial answer to the question, identify the issues still to be covered and the assumptions still to be tested, and describe a full method to answer the question.

Identify and present the list of tools to be used and applied in the field/data collection phase;

Prepare and submit a detailed work plan with an indicative list of people to be interviewed, surveys to be undertaken, dates of visit, itinerary, and name of team members involved.

Define issues and gaps requiring further analysis;

Prepare and share inception report of not more than 6 pages

Field Investigation Phase

The Field Phase should start upon approval of the inception Report by Plan’s evaluation contact point. The consultant will:

 Undertake the field mission applying the plan developed during the Inception Phase. This plan has to be applied in a way that is flexible enough to accommodate any last-minute challenges in the field. If any significant deviation from the agreed work plan or schedule is perceived as creating a risk to the quality of the evaluation, these should be immediately discussed with the evaluation manager.

 Hold a briefing meeting with the project team in the first days of the field phase.

 Undertake school visits and stakeholder interviews including relevant education authorities.

 Ensure adequate contact and consultation with the key project staff

 Ensure adequate contact and consultation with, and involvement of, the different stakeholders; working closely with the relevant government authorities and agencies during their entire assignment. Use the most reliable and appropriate sources of information and harmonise data from different sources to allow ready interpretation.

 Summarise fieldwork at the end of the field phase, discuss the reliability and coverage of data collection, and present preliminary findings in a meeting with the project management team and relevant MoE administrations.

All data sets and original completed questionnaires and data collection sheets will become the property of Plan and will be shared with the draft Evaluation Report

Synthesis and Feedback Phase

This phase is mainly devoted to the preparation of the draft final report in English and relevant official language (French / Portuguese). The consultants will make sure that:

Their assessments are objective and balanced, affirmations accurate and verifiable, and recommendations realistic.

When drafting the report, they will acknowledge clearly where changes in the desired direction are known to be already taking place, in order to avoid misleading readers and causing unnecessary irritation or offence.

The consultant will submit the draft evaluation report to the Plan evaluation contact point for review. On the basis of comments expressed by Plan & the relevant education authorities, the consultant will amend and revise the draft report. On the basis of comments made by participants, the consultant will prepare the final version of the report and will present this to Plan Ireland and to the Country Offices through an in country workshops.

International Solutions Group www.theisg.com 76 EXPERTISE REQUIRED

The following are the qualification requirements for the evaluation consultant:

- Advanced University Degree in public education or related field. - A minimum ten (10) years of relevant professional experience in the areas of basic/ primary education, teacher development, capacity building of education administrations, education economics and in the definition and implementation of sectoral policies in the education sector; - Extensive experience in the evaluation of technical assistance projects, preferably, those under the education portfolio; - Fully conversant with the principles and working methods of project cycle management. Knowledge of the activities of multilateral development donors will be an added advantage; - Solid knowledge of, and practical experience with gender analysis and planning; - Full working knowledge of English and excellent report writing skills - French and Portuguese fluency within the evaluation team - Competence and adequate experience in the use of qualitative and quantitative methods of data collection and analysis including: sampling, desegregation of data, structured and semi-structured interviewing, focus group discussions, and observation and triangulation research methods. - Ability to interpret and analyse complex qualitative and quantitative data, and to present findings and recommendations in a clear and concise way. - Excellent inter-personal communication skills including experience of facilitation and presentation. - Ability to work equally well with communities and international organisations. - Knowledge and sensitivity to political and social contexts of West African Communities. - Experience in West Africa would be an advantage.

TIME FRAME

The evaluation will be carried out from March 2015

The consultant will be required to submit a technical proposal indicating the number of days and rate for the consultancy work with a realistic action plan.

Phase Envisioned Timeline Inception Phase 16.03.2015-24.04.2015 Field investigation Phase 27.04.2015-22.05.2015 ***NOTE data collection must be completed by the end May so as to not interfere with State exams Draft report 31.08.2015 Final report 30.09.2015 SUPERVISION AND MANAGEMENT The overall supervision of the evaluators will rest with Plan Ireland. The consultant will report to a designated manager. The consultant will be expected to work closely with, and interact with the implementing agency, the MOEs and relevant in country staff and implementing partners.

International Solutions Group www.theisg.com 77 Annex 4: Evaluation Itinerary

Plan Ireland’s Education: Quality, Inclusive, Participative (EQuIP) programme From April 26th to May 7th 2015 Sunday, April 26th o Departure from Bissau to Gabú 14h00 - 17h00 Bissau o Check in Hotel 17h00 – 17h30 Gabú o Team meeting 17h30-19h30 Gabú o Briefing with enumerators o Present team

o Discuss conditions/logistics

o Discuss tools

o Dinner in Gabú 20h00-21h00 Gabú o Overnight in Gabú 21h00-XXXX Gabú Monday, April 27th o Departure from Gabú to Pitche schools two teams 06h45-08h00 Gabú o Team 1 - EBU de Dara– State School in Pitche Gabú o Team 2 - EBE de Sintcha Lali– State School in Pitche Gabú o Return to Gabú, both team 16h00-17h45 Gabú o Personal time 17h45-18h30 Gabú o Team meeting debriefing /Bottlenecks 18h30-19h30 Gabú o Preparation next day

o Overnight in Gabú 19h30-xxxxx Gabú Tuesday, April 28th o Departure from Gabú to Gabú schools two teams 07h00-07h30 Gabú o Team 1 - Escola Caetano Semedo– State School in Gabú Gabú o Team 2 - EBE de Cancisse– State School in Gabú Gabú o Helen O Reilly - KII with inspectors in DRE (Bafatá Regional Directorate for Education) Bafatá o Return to Hotel all teams

o Personal time 16h00-16h30 Gabú o Team meeting debriefing /Bottlenecks 16h30-18h00 Gabú o Preparation next day 18h00-19h00 Gabú o Overnight in Gabú 19h00-xxxxx Gabú Wednesday April 29th o Departure from Gabú to Gabú schools two teams 07h00-07h30 Gabú o Team 1 - EBE de Cataba Alfa- State School in Gabú o Team 2 - EBE de Tabadjan– State School in Gabú o Return to Gabú both teams 16h00-17h15 Gabú o Personal time 17h15-18h30 Gabú o Team meeting debriefing / Bottlenecks 18h30-19h30 Gabú o Preparation next day

o Overnight in Gabú 19h30-xxxxx Gabú Thursday April 30th o Departure from Gabú to Gabú schools two teams 07h00-07h30 Gabú o Team 1 & 2 - EBE de Coiada- State School in Gabú o Helen O Reilly - KII with inspectors in DRE (Gabú Regional Directorate for Education) 12h00-15h00 Gabú o Departures from Gabú to Bissau 15h00-xxxxx Bissau WEEKEND Monday, May 4th o Departure from Bissau to Bambadinca schools two teams 06h00-08h00 Bissau o Team 1 - Escola Comunitaria de Amedalai – Community School in Bambadinca (Bafatá) o Team 2 - Escola Comunitaria de Fa Mandinga – Community School in Bambadinca (Bafatá) o Departure from Bambadinca to Bafatá

16h00-17h00 o Hotel check in /Personal time Bambadinca o Team meeting debriefing 16h30-18h00 o Bottlenecks

International Solutions Group www.theisg.com 78 18h00-19h30 Bafatá o Preparation next day Bafatá o Overnight in Bafatá 19h30-xxxxx Bafatá Tuesday, May 5th o Departure from Hotel to Bafatá schools two teams 07h00-07h30 Bafatá o Team 1 - EBE de Caur – Community School in Bafatá o Team 2 - EBU de Cumuda – State school in Bafatá

o Return to Hotel both teams 16h00-17h00 Bafatá o Personal time 16h30-18h00 Bafatá

o Team meeting debriefing 18h00-19h30 Bafatá o Bottlenecks

o Preparation next day

o Overnight in Bafatá 19h30-xxxxx Bafatá Wednesday, May 6th o Departure from Bafatá to Bambadinca schools two teams 07h00-08h00 Bafatá o Team 1 - Comunitaria de Canquenhe– Community School in Contubel (Bafatá) o Team 2 - Xime –in Bambadinca (Bafatá)

o Departure from Contubel to Bafatá

16h00-17h00 Contubel o Personal time

17h30-18h30 Bafatá o Team meeting debriefing

o Bottlenecks 18h30-19h30 Bafatá o Preparation next day

Overnight in Bafatá 19h30-xxxxx Bafatá Thursday, May 7th o Departure from Bafatá to Bambadinca schools two teams 07h00-08h00 Bafatá o Team 1 - Escola Comunitaria de Bambadinca Cossara – Community School in Bafatá o Team 2 - Escola comuniatria de Queroane – Community School in Bafatá

16h00-18h00 Bafatá o Departure from Bafatá to Bissau

18h00 Bafatá o Overnight in Bissau

International Solutions Group www.theisg.com 79 Annex 5: List of key persons interviewed and group meetings/FGDs

Burkina Faso Ministry of Education Regional Education Director (DRENA) Dah Thomas Provincial Education Director (DPENA) Ernest Kini Inspector Batié I Ouattara Isa Gaoua & Noumbiel Inspector Batié II Dah Emmanuel Inspector Midebdo Belembaogo Omer Pedagogical advisor Batié I Bationan O Jacques PLAN Burkina Faso EQuIP Programme Manager Aubert Ouango Education Specialist Clement Traore Plan CO Ouagadougou Research and Evaluation Coordinator Ida Tamini Nikyema Grants Coordinator Gerard Dah Education Advisor Hema Siriki PUM Meda Clement EQuIP Coordinator BFA Simane Ouaba Plan Gaoua Education Coordinator Ouedrago Aime M&E Coordinator Ouedrago Hassamy Programme Partners Coordinator Koubou Dah RAJS African Network Manager Edith Hien Ministry of Social Action and National Solidarity Provincial Director Sanou Justin AFDC VSL Coordinator Charlotte Kamboulet Centre for Community Based Rehabilitation Director Abbe Paul Dah Data collection tools Number of schools Planned sample Actual sample Focus groups students - girls 18 20 18 Focus groups students - boys 18 20 18 Focus groups teachers 20 20 20 Focus groups CWD 15 0 20 Focus groups VSL/Credit 7 0 7 Focus group APE 20 20 15 Interviews with directors 20 20 20 Interviews with inspectors - 2 3 Interviews with students’ assocs/school governments 13 0 19 Equality scorecard - girls 19 20 19 Equality scorecard - boys 19 20 19 Equality scorecard - teachers 20 20 20 Reading test – girls 14 70 70 Reading test - boys 14 70 70 Classroom observation 18 20 18 Household surveys 12 100 60 Key informant interviews Total Regional Education Director 1 Provincial Education Director 1 Provincial Director Action Social 1 Primary Education inspectors 3 Director RBC 1 VSL Coordinator, AFDC 1 Total 8 Guinea Bissau

International Solutions Group www.theisg.com 80 Ministry of Education Regional Education Director Anssumane Sissé Gabú Focal Point for EQuIP / Inspector Bernardo Mendes Regional Inspector (Pitche) Lameny Sonco Coordinating inspector João Lopes Regional Inspector (Ganadu) Chico Antonio Dingo Bafatá Regional Inspector Marcelo Mendes Focal Point for EQuIP / Inspector Abulai Biai Teacher EBE Coiada Nelito José Andrade PLAN Guinea Bissau Country Director Allassane Drabo EQuIP Programme Coordinator Eugénia Djompe Plan CO Bissau Education Specialist Lucy Monteiro M&E Specialist Bacar Mané Protection Specialist PUM Emanuel Fernandes Plan PU Bafatá VSL Specialist Joaquim Sanhá Education Coordinator João Paulo da Costa Plan PU Gabú M&E Coordinator Mahoro Bwock Institution AGRICE Focal Point for Eastern Region Carlos Nivaldo Junior Caetano Semedo Parents Association Umaro Sanhá & Amadu Baldé EBE Coiada President of School Management Committee Banuno Baldé Gabú Women’s Group for Inclusive Education – 5 members

Group Total Men Women Heads of households 75 56 19 School head teachers 15 14 1 Teachers 53 35 18 Members of parent’s associations 59 34 25 Students 155 78 77 REACH tests 132 69 63 Total 489 286 203 Number of meetings Group Planned Held Teachers 15 15 Male students 15 15 Female students 15 15 Children with disabilities 0 6 VSL group members 0 2 Students association/school government 0 3 members Gabú Women’s Group for Inclusive Education 0 1 Total 45 56

International Solutions Group www.theisg.com 81 Annex 6: Documents Consulted12

Plan International Global Education Strategy 2013 Plan International Global Education Strategy 2015-2020 WARO Education Thematic Framework EQuIP Baseline Survey Report & training manual 2012 Progress out of Poverty Index Technical Overview Burkina Faso Annual Statistics 2011/2012/2013 Burkina Faso Programme De Développement Strategique De L’éducation De Base 2012 - 2021 Plan Burkina Faso, South West Program Unit Long Term Plan 2010-2014 Plan Burkina Faso Country Strategic Plan 2012-2015 Burkina Faso EQuIP Mid-Term Evaluation 2014 Guinea Bissau Ministry of Education Three-year Plan for the Development of Education: 2011 - 2013 GNB Country Programme Progress Report FY14 GNB CSP 2013 – 17 Plano De Longo Termo Do Pu 2013/2017, Bafatá Plano De Longo Termo Do Pu 2013/2017, Gabú Plan Guinea Bissau Annual Report 2014 GNB Out-of-school children report UNICEF 2014 GNB Advocacy Education Plan 2014 Mid Term Review Country Report – Guinea Bissau REACH Scoring and Data Analysis Procedure Scoring and Analysis Procedures for Classroom Observations Irish Aid PCM Guidelines 2012/2013/2014 Irish Aid/Plan Joint Appraisal Jan 2012 MoU Irish Aid/Plan Ireland PG14-2012 Plan Ireland Irish Aid Programme Funding Application Irish Aid EQuIP Years 1, 2, 3 Narrative & Technical Reports Good Practices Document - Plan Disability Research Plan Disability Research: Guinea, Niger, Sierra Leone, Togo LITERATURE REVIEW - Plan Disability Research WARO- Outside The Circle 2013 Rapport de l’évaluation du projet EQUIP / Projet de PLAN International, Evaluation menée par l’équipe de Handicap International Review on Plan West Africa’s approaches to teacher training and quality, WARO Early grade reading ability, a pilot project on in service teacher training and reading skills, Report for Plan 2014 Update on EQuIP Educational Indicators for Plan Guinea Bissau, 2014

12 These are the primary official documents consulted. A range of other programme data, reports, and other informal documentation was also consulted as part of the research

International Solutions Group www.theisg.com 82 Annex 7: Data Collection Tools

Please click the links to access listed documents.

FGD COGE V1 Reach palavras conhecidas

Microsoft Office Word 97 - 2003 Document

FGD Meninos e Meninas V1 Reach Palavras nao conhecidas

FGD Professores V1 School equality Score Card Meninas V1

KII Directores V2 School equality Score Card Meninos V1

KII Inspectores V2 School equality Score Card Professores V1

Observaçao de professores em sala de aula V1 Tools for EQUIP Evaluation BFA V1

Questionário a Agregados Familiares V1 Reach Questionário de Leitura

International Solutions Group www.theisg.com 83