Lot No. 4 : Project Final Evaluation : « Financial services », Agency for Partnership for Progress – MCA ‐ Morocco Contract No. APP/2012/PP10/QCBS/ME‐16‐lot 4 Deliverable 1: Methodology Report Submitted by : North South Consultants Exchange JUNE 19TH 2013 TABLE OF CONTENTS 1.INTRODUCTION ............................................................................................................................ 1 1.1.CONTEXT ................................................................................................................................................... 1 1.2.OVERVIEW OF THE FINANCIAL SERVICES PROJECT ..................................................................................... 2 1.3.PURPOSE OF THE FSP FINAL EVALUATION ............................................................................................. 4 2.METHODOLOGY ......................................................................................................................................... 5 2.1. COMPREHENSIVE APPROACH .......................................................................................................... 5 2.2. STAKEHOLDERS .......................................................................................................................... 6 2.2.1. APP ................................................................................................................................................... 6 2.2.2. Supervisory Institution ...................................................................................................... 7 The Ministry of Economy and Finance (MEF) will be consulted as a supervisory authority for AMCs. The consultations aim to: ................................................................................................................ 7 2.2.3. AMC Management ........................................................................................................... 7 2.2.4. AMC clients ..................................................................................................................... 9 2.2.5. Jaida Fund ...................................................................................................................... 9 2.2.6. FNAM 9 2.3. GENERAL QUESTIONS..................................................................................................................10 2.4. RELEVANCE .............................................................................................................................11 2.5. EFFECTIVENESS .........................................................................................................................12 2.6. EFFICIENCY 15 2.7. COHERENCE ............................................................................................................................16 2.8. SUSTAINABILITY ........................................................................................................................................ 17 2.9. IMPACT ..................................................................................................................................................... 17 2.10. DEVELOPMENT OF RESEARCH TOOLS ................................................................................................18 2.10.1. Collection of documents .................................................................................................18 2.10.2. One‐to‐one Interviews ....................................................................................................18 2.10.3. Quantitative surveys ......................................................................................................20 2.10.4. Sampling Frame ............................................................................................................21 2.10.5. Sampling .....................................................................................................................22 2.11. PREPARATION FOR THE DATA COLLECTION .........................................................................................26 2.11.1. Setting up of a data room ...............................................................................................26 2.11.2. Development of kits for the one‐to‐one interviews ...............................................................27 2.11.3. Development of quantitative survey kits ............................................................................27 2.11.4. Development of the researchers' training material ..............................................................27 2.11.5. Organization of the survey teams .....................................................................................28 2.11.6. Training of supervisors and researchers .............................................................................28 2.11.7. Survey Organisational strategy ........................................................................................28 2.11.8. Pilot survey and refining of survey questionnaires. ...............................................................29 2.12. DATA COLLECTION ...................................................................................................................29 2.12.1. Documents ...................................................................................................................29 2.12.2. One‐to‐one interviews ....................................................................................................30 2.12.3. Quantitative Surveys ......................................................................................................30 2.13. DATA PROCESSING ...................................................................................................................30 2.13.1. One‐to‐one interviews ....................................................................................................30 2.13.2. Quantitative Surveys ......................................................................................................30 2.14. DATA ANALYSIS 32 3. IMPLEMENTATION TIME TABLE .....................................................................................................33 4. CHANGES TO THE TECHNICAL OFFER ..............................................................................................34 ANNEXE A : BIBLIOGRAPHY AND DOCUMENTS CONSULTED ..................................................................35 ANNEX B: QUESTIONNAIRES AND GUIDELINES‐ ONE‐TO‐ONE INTERVIEWS ..............................................39 B.1. APP ....................................................................................................................................39 B.1.1. General Management‐ One‐to‐one Interview .......................................................................39 B.1.2. APP – M&E Management .................................................................................................42 I B.1.3. APP – Financial Services Project Management ......................................................................45 B.2. BANQUE AL‐MAGHRIB ................................................................................................................49 B.3. MINISTRY OF ECONOMY AND FINANCE ..............................................................................................52 B.4. JAIDA FUND ............................................................................................................................56 B.5. AMC / RMS ............................................................................................................................................. 61 B.6. FNAM ..................................................................................................................................................... 67 B.7. MOBILE AGENCY MANAGERS .........................................................................................................71 ANNEX C : QUESTIONNAIRES – QUANTITATIVE SURVEYS ......................................................................73 C.1. CREDIT AGENTS/ AGENCY MANAGERS TRAINED IN SOCIAL AND ENVIRONMENTAL NORMS ........................................73 C.2. CLIENTS OF MOBILE AGENCIES .......................................................................................................76 ANNEXE D : KITS COMBINATION – ONE TO ONE INTERVIEWS AMC .........................................................79 ANNEXE E: CURRICULUM VITATE OF THE QUANTITATIVE SURVEY SUPERVISORS…………………..…………….………83 ANNEXE F: LIST OF MICROCREDIT ASSOCIATION AND MOBILE AGENCIES ................................................97 ANNEXE G: DETAILLED WORKPLAN ................................................................................................. 101 II List of Tables TABLE 1: DEFINITION OF KEY TERMS .......................................................................................................................... 5 TABLE 2: AMCS IN MOROCCO ................................................................................................................................ 8 TABLE 3: EVALUATION QUESTIONS‐ IMPLEMENTATION OF MID‐TERM EVALUATION RECOMMENDATIONS .................................................. 10 TABLE 4: EVALUATION QUESTIONS – PARTICIPATION OF STAKEHOLDERS DURING THE PROJECT DESIGN AND IMPLEMENTATION PHASE ................... 11 TABLE 5: EVALUATION QUESTIONS – RELEVANCE ............................................................................................................ 11 TABLE 6: EVALUATION QUESTIONS‐ EFFECTIVENESS