Assessing the Impact of Microfinance Programmes: Learning Through a Collaborative Project in Pakistan and Zimbabwe
Total Page:16
File Type:pdf, Size:1020Kb
Assessing the Impact of Microfinance Programmes: Learning through a Collaborative Project in Pakistan and Zimbabwe Joana da Silva Afonso The thesis is submitted in partial fulfilment of the requirements for the award of the degree of Doctor of Philosophy of the University of Portsmouth. July 2018 Declaration Whilst registered as a candidate for the above degree, I have not been registered for any other research award. The results and conclusions embodied in this thesis are the work of the named candidate and have not been submitted for any other academic award. Word Count: 84,000 2 Abstract The demand for accountability of publicly funded social programmes has increased substantially during the past two decades and, thus, so did the need to evaluate the outcomes of these programmes. Evaluating impact is, however, a complex process and different perspectives have emerged regarding evaluation goals and methodologies. The term ‘impact’ has, therefore, been associated with different concepts, which translate into different approaches in terms of rigour and usefulness, going from formal impact assessments, which privilege methodological rigour, to internal evaluations, which are associated with a learning/improvement perspective. The Lendwithcare (LWC) impact project positioned in between these two approaches and it can be seen as a collaborative project between the project leader (LWC), the academic consultant (University of Portsmouth) and the field partners (Akhuwat Islamic Microfinance and THRIVE Microfinance). The research conducted within the PhD resulted from my participation in the project as academic consultant. It is based on a case study methodology implemented in two settings (Pakistan and Zimbabwe), using qualitative methodologies, including participation- observation. The analysis of the case studies was based on the utilisation-focused evaluation model developed by Michael Patton, with the choice of the model being rooted in its pragmatic approach and its fit with the conditions of the LWC impact project. The PhD thesis explores two main research questions. The first deals with the lessons learnt from the implementation of the individual evaluations. The main field challenges and respective solutions were identified, giving attention to the resulting compromises in terms of rigour and the advantages associated with involving the field partners. The second explores the common project approach adopted by LWC in the two settings, identifying its main elements and how these have influenced the perception of the partners regarding their participation in the evaluations. The intention of all partners to continue the project as well as the interest of other microfinance institutions to implement similar evaluations seems to indicate that LWC and its partners have perceived relevant advantages in this approach. 3 Table of Contents Table of Contents 4 List of Tables 8 List of Figures 8 List of Abbreviations 9 Acknowledgements: 10 1. Introduction 11 1.1 Research aims and context 15 1.2 Microfinance: a brief introduction 19 1.3 Microfinance today: the accountability drive and evaluation 21 1.4 Research design 26 1.5 Research main contributions and originality 27 1.6 Organization of the thesis 30 2 Outcomes and Impact Evaluation 32 2.1 Programme evaluation: clarifying concepts and perspectives 32 2.1.1 Outputs and Outreach 33 2.1.2 Outcomes and Impact 34 2.2 Overview of evaluation theories 38 2.2.1 An evaluation theory tree 38 2.2.2 Evaluation theories: the methods branch 39 2.2.3 Evaluation theories: the value branch 41 2.2.4 Evaluation theories: the use branch 43 2.3 Evaluation models and approaches 45 2.3.1 Context, Input, Process, Product model (CIPP) 45 2.3.2 Utilisation-focused model (UFE) 47 2.3.3 Constructivist model 49 2.3.4 Other evaluation models 49 3 Evaluation in Microfinance 51 3.1 Outreach, outcomes and impact of microcredit: a theoretical approach 51 3.1.1 Microcredit expected outcomes 52 3.2 Outreach, outcomes and impact of microcredit: the empirical research 54 3.2.1 Systematic reviews and reports 56 3.2.2 Methodological choices 59 3.2.3 Mixed results 61 3.3 Challenges in the implementation of evaluations in microfinance 66 4 4 Poverty Scorecards in Microfinance Evaluation 72 4.1 Poverty measurement in microfinance 72 4.2 Poverty scorecards in microfinance 75 4.3 Poverty Probability Index (PPI) 77 4.3.1 Methodological issues and implementation challenges 80 4.3.2 Empirical evidence 83 5 Methodological Issues 86 5.1 Case Study 86 5.2 Data Collection: the central role of participant observation 89 5.3 Data Analysis: a utilization-focused framework 102 5.3.1 The choice of evaluation model 102 5.3.2 The UFE-based framework of analysis 106 5.4 The LWC assessment project: an overview 108 5.4.1 Lendwithcare: the crowdfunding platform 108 5.4.2 Lendwithcare Impact Assessment Project 110 6 Case Study: Akhuwat Islamic Microfinance, Pakistan 117 6.1 Macro-Context 118 6.2 Micro-context 121 6.2.1 AIM and Islamic Microfinance 121 6.2.2 Microcredit programme: Family Enterprise Loan 122 6.3 LWC Evaluation Project 124 6.3.1 Introduction 124 6.3.2 Design of the Evaluation 125 6.3.2.1 Primary Intended Users: motivations 126 6.3.2.2 Role of Evaluator 128 6.3.2.3 Household Survey: preparation 130 6.3.3 Implementation Process 132 6.3.3.1 Primary Intended Users: behaviours 132 6.3.3.2 Role of the Evaluator 133 6.3.3.3 Household Survey: implementation and analysis 134 6.3.4 Interpretation and Use of Findings 139 6.3.4.1 Primary Intended Users: perceptions and intentions 139 6.3.4.2 Role of Evaluator 141 6.3.4.3 Household Survey: results and reports 142 6.4 Discussion 144 5 7 Case Study: THRIVE Microfinance, Zimbabwe 154 7.1 Macro-context 154 7.2 Micro-context 157 7.2.1 THRIVE Microfinance 157 7.2.2 Microcredit Programme 158 7.3 LWC Evaluation Project 161 7.3.1 Introduction 161 7.3.2 Design of the Evaluation 162 7.3.2.1 Primary Intended Users: motivations 162 7.3.2.2 Role of the evaluator 164 7.3.2.3 Household Survey: preparation 165 7.3.3 Implementation Process 168 7.3.3.1 Primary Intended Users: behaviours 168 7.3.3.2 Role of the evaluator 168 7.3.3.3 Household Survey: implementation and analysis 169 7.3.4 Interpretation and Use of Findings 173 7.3.4.1 Primary Intended Users: perceptions and intentions 173 7.3.4.2 Role of the evaluator 174 7.3.4.3 Household Survey: results and reports 175 7.4 Discussion 178 8 Cross-Case Analysis: Evaluation Challenges and a Common Project Approach 186 8.1 Cross-case synthesis: similarities and differences 187 8.2 Insights from a UFE-based comparative analysis 192 8.2.1 Relevant elements in the evaluation design, implementation and use of findings 192 8.2.1.1 Design 192 8.2.1.2 Implementation 194 8.2.1.3 Use of Findings 196 8.2.2 The personal factor and the role of the evaluator 200 8.3 A common project approach to evaluation in microfinance 205 8.3.1 Feasibility 206 8.3.2 Advantages and limitations 208 8.3.3 Implications for the MFIs and the sector 210 9 Conclusions 213 9.1 Setting up the research 213 6 9.2 The parallel with utilisation-focused evaluations 215 9.3 Contributions towards transparency and credibility 216 9.4 Strengths and weaknesses of the research 219 9.5 Final considerations 222 References 223 Annexes 247 List of Annexes 247 7 List of Tables 5.1 Interviews: Perceptions on the Evaluation Project 95 5.2 UFE-Based Framework of Analysis 106 5.3 Lendwithcare Portfolio 109 6.1 Pakistan Financial Inclusion Data 119 6.2 Pakistan Microfinance Sector (2015-2017) 120 6.3 AIM Main Indicators 122 6.4 LWC/AIM Survey Respondents by Geographical Area 134 6.5 Client Average Monthly Income and Expenses (AIM Survey) 142 6.6 Quantile Regression Results – AIM 143 6.7 Mann Whitney Tests for Gender and Educational Level 144 7.1 Zimbabwe Financial Inclusion Data 155 7.2 LWC/THRIVE Survey Respondents by Geographical Area 172 7.3 Average Monthly Income and Expenses (THRIVE Survey) 175 7.4 Quantile Regression Results – THRIVE 176 List of Figures 2.1 Programme Evaluation Chain 33 2.2 Evaluation Theory Tree 39 3.1 The Virtuous Cycle of Credit 52 4.1 PPI Form for Pakistan (English version) 78 5.1 Lendwithcare Crowdfunding Model 110 6.1 AIM Lending Process (2015-2017) 123 6.2 LWC/AIM Project Milestones 125 6.3 LWC/AIM Evaluation Stakeholders Map 126 7.1 THRIVE Lending Process 160 7.2 LWC/THRIVE Project Milestones 162 7.3 LWC/THRIVE Evaluation Stakeholders Map 163 8 List of Abbreviations AIM Akhuwat Islamic Microfinance AIMS Assessing Impact of Microenterprise Programmes ANDC Associação Nacional de Direito ao Crédito BRAC Bangladesh Rural Advancement Committee BRDB Bangladesh Rural Development Board CFI Centre for Financial Inclusion CIPP Context, Input, Process, Product DFID Department for International Development e-MFP European Microfinance Platform CGAP Consultative Group to Assist the Poor GDP Gross Domestic Product GRO Group Relations Officer IFAD International Fund for Agricultural Development IFPRI International Food Policy Research Institute ILO International Labour Organisation IPA Innovations for Poverty Action IRDB Inter-American Development Bank JCSEE Joint Committee on Standards for Educational Evaluation J-PAL Abdul Latif Jameel Poverty Action Lab LIC Low Income Countries LMIC Lower Middle-Income Countries LWC Lendwithcare MFI Microfinance Institution MIS Management and Information System NBFI Non-Banking Financial Institution NGO Non-Governmental Organisation ODA Overseas Development Administrator OECD Organisation for Economic Co-operation and Development PAK Pakistan PAT Poverty Assessment Tool p.c.