USAID/UKRAINE MID-TERM PERFORMANCE EVALUATION OF THE FINANCIAL SECTOR TRANSFORMATION (FST) ACTIVITY EVALUATION REPORT (UPDATED10.22.2019) 1 September 2019 This report is made possible by the support of the American People through the United States Agency for International Development (USAID). The contents of this report were prepared by IBI under IDQ No. 7200AA18D00011, Task Order No. 72012119F00002. The views expressed herein are the sole responsibility of IBI and do not necessarily reflect the views of USAID or the United States Government 2 CONTENTS ACRONYMS ............................................................................................................................................ 5 EXECUTIVE SUMMARY ....................................................................................................................... 6 FINDINGS/CONCLUSIONS................................................................................................................ 6 RECOMMENDATIONS .......................................................................................................................... 8 ANALYSIS ............................................................................................................................................. 10 INTRODUCTION AND PROGRAMMING CONTEXT ...................................................... 10 EVALUATION PURPOSE AND EVALUATION QUESTIONS ...................................... 11 KEY QUESTIONS ................................................................................................................................... 11 EVALUATION APPROACH AND LIMITATIONS ............................................................... 11 EVALUATION APPROACH ..................................................................................................................................................... 12 EVALUATION LIMITATIONS ................................................................................................................................................ 13 FINDINGS/CONCLUSIONS ............................................................................................................ 14 EVALUATION QUESTION 1: IN WHAT WAYS HAS FST CONTRIBUTED TO BUILDING AND ADVANCING CONFIDENCE IN THE FINANCIAL SECTOR? HOW SUCCESSFUL WERE THESE EFFORTS? ....................................................................................................................................................... 14 Sub-Component A: Increased Public Confidence In, Understanding, and Use of Financial System ......................................................................................................................................................................................................... 14 Sub-Component B: Transformed Non-Banking Financial Sector Regulatory Environment 17 Sub-Component C: Increased Financial Inclusion and Expanded Digital Finance Solutions18 Sub-Component D: Balanced and Sustainable Pension System............................................................ 18 EVALUATION QUESTION 2: TO WHAT EXTENT DID FST AFFECT SMES’ AND MARGINALIZED GROUPS ACCESS TO FINANCE? ........................................................................ 19 EVALUATION QUESTION 3: TO WHAT EXTENT DID FST CHANGE THE WAYS IN WHICH FINANCIAL INSTITUTIONS AND FINANCIAL CONSUMERS COMMUNICATE? ................ 20 EVALUATION QUESTION 4: WHAT WERE THE UNINTENDED EFFECTS OF THE FST ACTIVITY? ...................................................................................................................................................... 21 EVALUATION QUESTION 5: IN WHAT AREAS RELATED TO FST’S SCOPE IS USAID ASSISTANCE NO LONGER NEEDED? ................................................................................................. 22 RECOMMENDATIONS ..................................................................................................................... 22 CURRENT ACTIVITIES: .............................................................................................................................. 23 FUTURE ACTIVITIES: .................................................................................................................................. 24 ANNEX A: SCOPE OF WORK ............................................................................................................... 25 ANNEX B: EXECUTIVE SUMMARY ..................................................................................................... 26 ANNEX C: EVALUATION INSTRUMENTS .......................................................................................... 33 ANNEX D: KEY INFORMANTS ............................................................................................................. 40 ANNEX E: DOCUMENTS REVIEWED ................................................................................................. 43 ANNEX F: EVALUATION TEAM ........................................................................................................... 45 3 ANNEX G: CONFLICT OF INTEREST FORMS ..................................................................................... 46 ANNEX H: WORK PLAN ................................................................................................................ 50 TABLE OF CONTENT .......................................................................................................................... I ACRONYMS ............................................................................................................................................ II INTRODUCTION ................................................................................................................................. 1 EVALUATION OBJECTIVES AND KEY QUESTIONS ................................................................. 1 Key Questions ................................................................................................................................................. 1 TECHNICAL APPROACH ................................................................................................................... 2 Data Collection and Analysis Methods ...................................................................................................... 2 Data Management and Security ................................................................................................................... 3 Data Analysis Methods................................................................................................................................... 3 Potential Evaluation Limitations ................................................................................................................... 5 TEAM COMPOSITION AND RESPONSIBILITIES ......................................................................... 6 Timeline ............................................................................................................................................................. 6 Draft Deliverables Schedule and Work Plan ...................................................................................................................... 6 ANNEX A: STATEMENT OF WORK .............................................................................................. 1 ANNEX B: DRAFT INSTRUMENTS.................................................................................................. 3 ANNEX C: PRELIMINARY KEY INFORMANT LIST .................................................................... 7 4 ACRONYMS CDCS Country Development Cooperation Strategy CSO Civil Society Organization DGF Deposit Guarantee Fund EQ Evaluation Question ET Evaluation Team EU European Union FGD Focus Group Discussion FSR National Commission for State Regulation of Financial Services Market (commonly referred to as the Financial Services Regulator) FST Financial Sector Transformation FY Fiscal Year GOU Government of Ukraine IBI International Business Initiatives KII Key Informant Interview MEL Monitoring, Evaluation, and Learning MSI Management Systems International NABU Independent Association of Ukrainian Banks NBU National Bank of Ukraine NSSMC National Securities and Stock Market Commission PFU Pension Fund of Ukraine PMP Performance Management Plan SMEs Small and Medium Enterprises TA Technical Assistance UNDP United Nations Development Program USAID United States Agency for International Development WB World Bank 5 EXECUTIVE SUMMARY Economic growth programming is a complex and integral part of USAID’s technical assistance to Ukraine, in addition to issues of corruption, democratic governance, and conflict mitigation. Post- Euromaidan reform initiatives resulted in a multi-stakeholder plan, Comprehensive Program for Financial Sector Development of Ukraine Until 2020, which laid out key restructuring goals for the NBU and other elements of the financial sector. USAID’s FST activity, implemented by DAI in partnership with key Government of Ukraine (GOU) agencies, industry associations, civil society organizations, and the private sector, works to transform the financial sector into a transparent, fair, and diversified system that supports firms and consumers. The original design of FST was built on assumptions about legislative progress coming out of the Comprehensive Program, but much of the financial sector reform agenda stalled in Verkhovna Rada. FST was intended to be a flexible project in order to respond to changing context, and so became over time a large and wide-ranging project