Anc hor Final Report Benin Policy Reform and Institutional Strengthening Project: Evaluation Design December 23, 2019 Sarah Hughes, Patricia Costa, Cullen Seaton, Arif Mamun, Dara Bernstein, Nils Junge and Catalina Torrente Submitted to: Millennium Challenge Corporation 1099 Fourteenth St, NW Suite 700 Washington, DC 2005 Project Officers: Julian Glucroft and Shreena Patel Contract Number: 95332419C0025 Submitted by: Mathematica 1100 1st Street, NE, 12th Floor Washington, DC 20002-4221 Phone: (202) 484-9220 Fax: (202) 863-1763 This page has been left blank for double-sided copying. Evaluation Design Report Mathematica LIST OF ACRONYMS ABERME Agence Béninoise d’Electrification Rurale et de Maîtrise d’Énergie AFD Agence Française de Développement ANM Agence Nationale de Normalisation, de Métrologie et du Contrôle de la Qualité ARE Autorité de Regulation de l’Electricité BAI Bureau d’analyse et d’investigation CA Constraints analysis CBA Cost-benefit analysis CEA Cost-effectiveness analysis CEB Communauté Electrique du Bénin CSC Comité de Suivi et du Controle DAEM Développement d’Accès à l’Energie Modern DAF Direction Administrative et Financière DB Doing Business DESE MCA-Benin II Economics and Monitoring and Evaluation Department DFIF Department for International Development DGDDI Direction Générale des Douanes et Droits Indirects DGRE Direction Générale des Ressources Energetiques DOC Drivers of change EE Energy efficiency EIF Entry into force ERR Economic rate of return ES Enterprise Survey EU European Union FGD Focus group discussion FTP File transfer protocol GoB Government of Benin GoT Government of Togo INSAE Institut National de la Statistique et de l’Analyse Économique IPP Independent power producer IRB Institutional Review Board iii Evaluation Design Report Mathematica ITT Indicator tracking table KII Key informant interview KPI Key performance indicator LBNL Lawrence Berkeley National Laboratory M&E Monitoring and evaluation MCA Millennium Challenge Account MC Management contractor MCC Millennium Challenge Corporation MW Megawatts PIEA Public Information and Education Activity PNEE Plateforme Nationale Dédiée aux Normes et de l’Etiquetage Energétique PPA Power purchase agreement PRIS Policy, Regulation and Institutional Support PSA Power Sector Assessment PTA Plan de travail annuel PV Photovoltaic SBEE Société Béninoise d'Energie Electrique TOR Terms of references USAID U.S. Agency for International Development iv Evaluation Design Report Mathematica Contents LIST OF ACRONYMS .................................................................................................................................. III I. INTRODUCTION AND BACKGROUND ............................................................................................... 1 A. Country context ............................................................................................................................. 1 B. Objectives of the report ................................................................................................................. 2 II. OVERVIEW OF THE REFORM PROJECT AND IMPLEMENTATION PLAN ...................................... 3 A. Overview of the Reform Project .................................................................................................... 3 B. Evolution of the Reform Project ..................................................................................................... 5 C. Policy, Regulation and Institutional Support Activity (Policy Activity) ............................................ 5 1. Regulation and Tariff Sub-Activity ......................................................................................... 5 2. Energy Efficiency Sub-Activity ............................................................................................. 10 3. Independent Power Producer (IPP) Sub-Activity ................................................................ 12 D. Utility Strengthening Activity ........................................................................................................ 14 E. Public Information and Education Activity ................................................................................... 19 F. Implementation plan .................................................................................................................... 20 G. Project participants ...................................................................................................................... 20 H. Summary of cost and benefit analysis and beneficiary analysis ................................................. 22 III. LITERATURE REVIEW ....................................................................................................................... 23 A. The `Textbook Model’ of regulatory reform ................................................................................. 23 B. IPPs ............................................................................................................................................. 24 C. Energy efficiency ......................................................................................................................... 25 1. Appliance standards ............................................................................................................ 26 2. Consumer behavior ............................................................................................................. 26 D. Institutional support ..................................................................................................................... 27 1. Management contracts ........................................................................................................ 27 2. Independent regulation ........................................................................................................ 28 E. Literature gaps ............................................................................................................................. 28 IV. SUMMARY OF EVALUABILITY ASSESSMENT ................................................................................ 31 V. EVALUATION DESIGN ....................................................................................................................... 33 A. Evaluation design overview ......................................................................................................... 33 B. Implementation analysis of the Reform Project ........................................................................... 35 v Evaluation Design Report Mathematica 1. Overarching research questions and approach .................................................................. 35 2. Data sources ........................................................................................................................ 37 3. Analytic approach ................................................................................................................ 39 C. Evaluation of the PRIS Activity .................................................................................................... 41 1. Energy Efficiency Sub-Activity ............................................................................................. 42 2. Independent Power Producer Sub-Activity .......................................................................... 44 3. Regulation and Tariff Policy Sub-Activity............................................................................. 45 D. Evaluation of the Utility Strengthening Activity ............................................................................ 49 E. Evaluation of Public Information and Education Activity ............................................................. 54 F. Cost-benefit analysis of the PRIS Project ................................................................................... 55 1. Appropriateness of conducting the cost-benefit analysis separately for the PRIS Project .................................................................................................................................. 56 2. Developing an accounting framework that captures several perspectives ......................... 57 3. Potential costs and benefits of the PRIS Project by activity ................................................ 58 4. Analytic issues ..................................................................................................................... 62 G. Approach to data collection ......................................................................................................... 63 1. Primary data collection (quantitative and qualitative) .......................................................... 63 2. Secondary data .................................................................................................................... 68 H. Anticipated challenges................................................................................................................. 70 VI. ADMINISTRATIVE .............................................................................................................................. 73 A. Summary of IRB requirements and clearances .......................................................................... 73 B. Data protection ............................................................................................................................ 73 C. Preparing data files for access, privacy and documentation ....................................................... 74 D. Dissemination plan