MCC Indonesia Green Prosperity Community Based Off Grid Rural
Total Page：16
File Type：pdf, Size：1020Kb
Evaluation Design Report MCC Indonesia Green Prosperity Project Grant Facility Community-Based Off Grid Renewable Energy Grant Portfolio v.1 January 2019 This report was prepared independently by Social Impact, Inc. at the request of MCC. EVALUATION DESIGN REPORT MCC Indonesia Green Prosperity Project Grant Facility Community-Based Off Grid Renewable Energy Grant Portfolio v.2 Submitted: January 2019 Submitted to: Millennium Challenge Corporation 875 Fifteenth Street, NW Washington, DC 20005-2221 Contracting Officer’s Representative: Vivian Agbegha Contract MCC-13-BPA-0017, Task Order MCC-17-CL-0005 Submitted by: Social Impact, Inc. 2300 Clarendon Blvd., Suite 1000 Arlington, VA 22201 703.465.1884 www.socialimpact.com DISCLAIMER The authors’ views expressed in this publication do not necessarily reflect the views of the Millennium Challenge Corporation (MCC) or the United States Government. EDR: MCC Indonesia GP Grant Facility Window 3A – Renewable Energy, v.1 CONTENTS 1 Introduction & Background .............................................................................................. 5 1.1 Country context ................................................................................................................................... 5 1.2 Objectives of this report ...................................................................................................................... 5 2 Overview of Compact & Interventions ............................................................................ 6 2.1 Overview of the Project and Implementation Plan ............................................................................. 6 2.1.1 Original Project Description ......................................................................................................... 7 2.1.2 Project Participants, Geographic Coverage, and Outputs .......................................................... 9 2.2 Theories of Change .......................................................................................................................... 19 2.2.1 Window 3A Grant Theories of Change ..................................................................................... 19 2.2.2 Other Grant Theories of Change ............................................................................................... 21 2.3 Cost Benefit Analysis & Beneficiary Analysis ................................................................................... 25 2.4 Literature Review .............................................................................................................................. 26 2.4.1 Summary of Existing Evidence ................................................................................................. 26 2.4.2 Gaps in Literature ...................................................................................................................... 28 2.4.3 Policy Relevance of the Evaluation ........................................................................................... 29 3 Evaluation design ........................................................................................................... 32 3.1 Evaluation Questions ........................................................................................................................ 32 3.1.1 Window 3A Evaluation Questions ............................................................................................. 32 3.1.2 Case Study Evaluation Questions............................................................................................. 33 3.2 Evaluation Design Overview ............................................................................................................. 33 3.2.1 Justification for Selected Grants ............................................................................................... 34 3.2.2 Window 3A Evaluation Design .................................................................................................. 34 3.2.3 Case Study Evaluation Design .................................................................................................. 36 3.3 Window 3A Quantitative Approach ................................................................................................... 38 3.3.1 Methodology .............................................................................................................................. 38 3.3.2 Timeframe of Exposure ............................................................................................................. 40 EDR: MCC Indonesia GP Grant Facility Window 3A – Renewable Energy, v.1 3.3.3 Study Sample ............................................................................................................................ 41 3.3.4 Primary Data Collection ............................................................................................................. 45 3.3.5 Secondary Data ......................................................................................................................... 50 3.3.6 Analysis Plan ............................................................................................................................. 50 3.4 Window 3A Qualitative Approach ..................................................................................................... 53 3.4.1 Methodology .............................................................................................................................. 53 3.4.2 Timeframe of Exposure ............................................................................................................. 54 3.4.3 Study Sample ............................................................................................................................ 55 3.4.4 Primary Data Collection ............................................................................................................. 56 3.4.5 Analysis Plan ............................................................................................................................. 59 3.5 Case Study Qualitative Approach ..................................................................................................... 60 3.5.1 Methodology .............................................................................................................................. 60 3.5.2 Timeframe of Exposure ............................................................................................................. 60 3.5.3 Study Sample ............................................................................................................................ 61 3.5.4 Primary Data Collection ............................................................................................................. 63 3.5.5 Secondary Data ......................................................................................................................... 64 3.5.6 Analysis Plan ............................................................................................................................. 65 3.6 Portfolio Evaluation Challenges & Limitations .................................................................................. 66 3.6.1 Risks to the W3A Anekatek Solar, East Sumba Impact Evaluation Design ............................. 66 3.6.2 Risks to the Non-Experimental Designs in the Portfolio Evaluation ......................................... 69 3.6.3 Limitations of Interpretations of the Results .............................................................................. 70 4 Administrative ................................................................................................................. 71 4.1 Summary of IRB Requirements and Clearances ............................................................................. 71 4.2 Data Protection ................................................................................................................................. 71 4.3 Preparing Data for Public Use .......................................................................................................... 71 4.4 Dissemination Plan ........................................................................................................................... 71 4.5 Evaluation Team Roles and Responsibilities ................................................................................... 72 4.6 Evaluation Timeline and Reporting Schedule ................................................................................... 73 SOCIALIMPACT.COM ii EDR: MCC Indonesia GP Grant Facility Window 3A – Renewable Energy, v.1 5 References ....................................................................................................................... 75 6 Annexes ........................................................................................................................... 82 6.1 Annex 1: Stakeholder Comments and Evaluator Responses .......................................................... 82 6.1.1 Local Stakeholder Comments and Evaluator Responses ........................................................ 82 6.1.2 MCC Evaluation Management Committee Comments and Evaluator Responses .................. 82 6.2 Annex 2: Evaluation Budget ............................................................................................................ 89 6.3 Annex 3: Instruments ........................................................................................................................ 90 6.3.1 Household Questionnaire