Evaluation Design Report (Revised) – 9 February 2016 EVALUATION DESIGN REPORT Ex-Post Performance Evaluation in the Education Sector; Phase I – Morocco PREPARED BY: International Solutions Group PREPARED FOR: Millennium Challenge Corporation Contact Us Brian O’Callaghan, Director of Programs @theisg.com · www.theisg.com Evaluation Design Report (Revised) – 9 February 2016 EVALUATION DESIGN REPORT Ex-Post Performance Evaluation in the Education Sector; Phase I – Morocco Disclaimer: The views and opinions expressed herein are those of the author(s) and do not necessarily represent those of MCC or any other U.S. Government entity International Solutions Group www.theisg.com 1 Evaluation Design Report (Revised) – 9 February 2016 Table of Contents ACRONYMS AND ABBREVIATIONS ......................................................................................................... 4 1. INTRODUCTION ...................................................................................................................................... 5 1.1. BACKGROUND.............................................................................................................................................. 5 1.2. COUNTRY CONTEXT ..................................................................................................................................... 5 1.2.1. Artisan and the Fez Medina (AFM) Project, Vocational Training Activity ......................................... 5 1.2.2. Functional Literacy and Vocational Training (FLVT) Activity............................................................ 6 1.3. OBJECTIVES OF THE EVALUATION DESIGN REPORT ...................................................................................... 7 1.4. DEFINITIONS ................................................................................................................................................ 8 2. FLVT PROJECT OVERVIEW AND THE INTERVENTIONS TO BE EVALUATED ......................... 10 2.1. FLVT PROJECT OVERVIEW: FUNCTIONAL LITERACY (FL) ......................................................................... 10 2.1.1. FL sub-Activity Logic ........................................................................................................................ 10 2.1.2. FL Participants/Beneficiaries ........................................................................................................... 12 2.1.3. FL Locations ..................................................................................................................................... 13 2.1.4. FL Indicators, Targets, and Reported Achievements ......................................................................... 13 2.1.5. MCA-Morocco Suggested FL Evaluation Methodology .................................................................... 14 2.2. FLVT PROJECT OVERVIEW: VOCATIONAL TRAINING (VT) ........................................................................ 15 2.2.1. VT sub-Activity Logic ........................................................................................................................ 15 2.2.2. VT Participants/Beneficiaries ........................................................................................................... 16 2.2.3. VT Locations ..................................................................................................................................... 17 2.2.4. VT Indicators, Targets, and Reported Achievements ......................................................................... 17 2.2.5. MCA-Morocco Suggested VT Evaluation Methodology .................................................................... 19 2.3. FLVT PROJECT OVERVIEW: QUALIFICATIONS AND COMPETENCIES (PEAQC) .......................................... 19 2.3.1. PEAQC sub-Activity Logic ................................................................................................................ 19 2.3.2. PEAQC Participants/Beneficiaries ................................................................................................... 20 2.3.3. PEAQC Locations ............................................................................................................................. 20 2.3.4. PEAQC Indicators, Targets, and Reported Achievements ................................................................. 20 2.3.5. MCA-Morocco Suggested PEAQC Evaluation Methodology ........................................................... 22 2.4. FLVT PROJECT OVERVIEW: ECONOMIC RATES O F RETURN (ERR) ............................................................ 22 2.4.1. FLVT Economic Rates of Return by Components ............................................................................. 22 3. LITERATURE/DESK AND FIELD REVIEW ........................................................................................ 23 3.1. DESK AND FIELD REVIEW .......................................................................................................................... 23 3.2. SUMMARY OF DESK REVIEW ...................................................................................................................... 23 3.2.1. Baseline Data to Support Claims of the Problem .............................................................................. 25 3.2.2. Indicators for Monitoring Project Results......................................................................................... 26 3.2.3. FLVT Project Inputs, Outputs and Outcomes .................................................................................... 26 3.3. GAPS IN DESK REVIEW .............................................................................................................................. 27 3.3.1. Reporting and Evaluation of Income and Employment Indicators ................................................... 27 3.3.2. Gender Strategy ................................................................................................................................ 28 3.4 BEST PRACTICES AND LESSONS LEARNED .................................................................................................. 29 4. EVALUATION DESIGN .......................................................................................................................... 30 4.1. EX-POST PERFORMANCE EVALUATION: TERMINOLOGY ............................................................................. 30 4.1.1. Evaluation Challenges ...................................................................................................................... 31 4.2. EVALUATION SUGGESTIONS FROM MCC .................................................................................................... 31 4.2.1. Functional Literacy Evaluation ........................................................................................................ 33 4.2.2. Vocational Training Evaluation ........................................................................................................ 35 4.2.3. PEAQC Evaluation ........................................................................................................................... 40 4.2.4. ERR Assessment ................................................................................................................................ 41 4.2.5. Summary of Evaluations.................................................................................................................... 42 4.2.6 Evaluation Instruments ...................................................................................................................... 42 4.2.7. Selection of Beneficiary Participants ................................................................................................ 44 International Solutions Group www.theisg.com 2 Evaluation Design Report (Revised) – 9 February 2016 4.2.8. Summary of Project and Program Indicators ................................................................................... 44 4.2.9 Data Arrays ........................................................................................................................................ 46 5. EVALUATION WORK PLAN ................................................................................................................. 48 5.1. EVALUATION TEAM .................................................................................................................................... 54 5.5 Staffing Plan .......................................................................................................................................... 55 6. BUDGET .................................................................................................................................................. 56 ANNEX 1: MOROCCO IN-COUNTRY VISIT SCHEDULE AND INFORMANT MEETINGS .............. 58 ANNEX 2: REFERENCES .......................................................................................................................... 59 ANNEX 3: CONTACT AND APPOINTMENT SCHEDULE FORM ......................................................... 61 ANNEX 4: SUMMARY OF AVAILABLE DATABASES ............................................................................ 62 International Solutions Group www.theisg.com 3 Evaluation Design Report (Revised) – 9 February 2016 ACRONYMS AND ABBREVIATIONS AFM Artisan and Fez Medina Project (Artisanat Fès Médina Projet) APP Agency for Partnership & Progress (Agence du Partenariat pour le Progrès) CBA Competency-Based Approach CFA Center of Apprentice Training (Centre