Formative Evaluation of the Community Infrastructure Improvement Project (CIIP) A-034874-001 Final Evaluation Report January 2O15 Prepared for the Pakistan Program, Asia Branch The Department of Foreign Affairs, Trade and Development Submitted by: Peter Hoffman Rashida Dohad SALASAN Consulting Inc. Map of Pakistan from the Perry-Castañeda Library Map Collection at University of Texas at Austin http://www.lib.utexas.edu/maps TABLE OF CONTENTS LIST OF ABBREVIATIONS ........................................................................................ 5 Executive Summary .................................................................................................. 6 1. Rationale and Purpose of the Evaluation ....................................................... 10 2. The Specific Objectives of the Evaluation ..................................................... 11 3. Evaluation Approaches, Data Collection and Sampling Methods ................ 12 3.1 Evaluation Approaches ............................................................................................................ 12 3.2 Data Collection Methods ......................................................................................................... 12 3.3 Sampling Methods ................................................................................................................... 13 3.4 Evaluation Design and Data Analysis ....................................................................................... 14 3.5 Limitations and Challenges ...................................................................................................... 15 4. The Context of the Development Intervention ............................................... 17 4.1 DFATD, Pakistan Government and Partner Policies, Objectives and Strategies ..................... 17 4.2 The Socio-economic Context of the Intervention ................................................................... 18 5. Evaluation Objective and Scope ..................................................................... 22 5.1 Timeframe, Budget and Geographic Areas ............................................................................. 22 5.2 Expected Outcomes................................................................................................................. 22 5.3 Components of the Intervention ............................................................................................. 23 5.4 Project Stakeholders ............................................................................................................... 24 5.5 Organizational Structure ......................................................................................................... 25 6. Findings 27 6.1 Presentation of Findings on Project Achievements ................................................................ 27 6.2 Presentation of Findings on Project Performance .................................................................. 51 7. Conclusions ...................................................................................................... 59 8. Recommendations ........................................................................................... 61 9. Lessons ........................................................................................................... 62 3 Annexes Annex A: Logic Model ...................................................................................... 63 Annex B: Performance Management Framework ........................................... 64 Annex C: Document Sources ........................................................................... 69 Annex D: Persons Contacted ........................................................................... 71 Annex E: Activities and Itinerary ..................................................................... 76 Annex F: Terms of Reference .......................................................................... 78 Annex G: Field Visit Site Selection Matrix ............................................................ 91 Annex H: Evaluation Matrix .............................................................................. 92 Annex H Proposed Data Collection Protocols ............................................ 105 Annex I: A History of Budget Revisions ...................................................... 108 4 LIST OF ABBREVIATIONS Awaz CDS AWAZ-Center for Development Services AWP Annual Work Plan BBCM Broad Based Community Meetings BDS Business Development Support BHU Basic Health Unit CBA Cost Benefit Analysis CBO Community Based Organization CIP CARE International in Pakistan CIIP Community Infrastructure Improvement Project CNIC Computerized National Identity Card DAC District Advisory Committee DFATD Department of Foreign Affairs, Trade and Development DCO District Coordinating Officer EIA Environmental Impact Assessment GOP Government of Pakistan IP Implementing Partner LGAP Local Government Association of Punjab LM Logic Model M&E Monitoring and Evaluation MFI Microfinance Institution MoU Memorandum of Understanding NGO Non Governmental Organization PAC Project Advisory Committee PAD Project Approval Document PIP Project Implementation Plan PSC Project Steering Committee PMF Performance Measurement Framework RMP Rural Maintenance Program (Bangladesh) RMT Road Maintenance Team SHG Self Help Group SMEDA Small and Medium Enterprise Development Agency TF Takhleeq Foundation TMA Tehsil Municipal Association TORs Terms of Reference TOT Training of Trainers TVET Technical and Vocational Education and Training UC Union Council VDC Village Development Council VO Village Organization WBS Work Breakdown Structure WEE Women’s Economic Empowerment 5 Executive Summary Title: Community Infrastructure Improvement Project Project Number: 7055088-A-034874 Evaluation Title: Formative Evaluation of the The findings, conclusions, Community Infrastructure Improvement recommendations and lessons listed are Project, Pakistan those of the consultant and do not Evaluation Type: formative necessarily reflect the views of DFATD or Commissioned by: DFATD Program Branch the Government of Canada. DFATD does Consultant: Salasan Consulting Inc. not guarantee the accuracy of the Date: December 14, 2013 information provided in this report. Rationale and Purpose of the Evaluation The rationale for this formative evaluation of the Community Infrastructure Improvement Project (CIIP) in Pakistan is to provide: a rigorous and independent assessment and analysis of the CIIP; an analysis of the CIIP that will inform learning opportunities in the delivery of services to the intended beneficiaries, poor rural women and their families; findings on the project’s attention to the cross-cutting themes of gender equality, governance and environmental stewardship; findings that will assist CARE Pakistan and its Implementing Partners (IPs), the Local Government Association of the Punjab (LGAP) at the District, Tehsil and Union Council levels of Punjab and Sindh in their efforts to improve the socio-economic status of rural disadvantaged women in Punjab and Sindh. Specific Objectives of the Evaluation The specific objective of the evaluation are to: a) assess the effectiveness, efficiency, relevance and sustainability of the results of the CIIP project taking into consideration the cross-cutting themes and the evaluation issues: o Cross-cutting themes: Gender Equality, Environmental sustainability and Governance o Evaluation issues: Ownership and Results-Based Management, Design Considerations; Risk management, and Additional Issues. b) measure the immediate results and progress toward intermediate results in each of the project’s three distinct components - Infrastructure Maintenance; Life Skills and Income Diversification, and Capacity Building of Local Government Institutions; and c) provide findings, conclusions and recommendations to facilitate decision-making about the remaining implementation phases. Development Context The CIIP is a project that aims to improve the access of rural women and men to services by improving the year round use of earthen road networks in conjunction with local governments by using paid maintenance teams comprised of poor women and by so doing increasing their income generating capacities. The context within which the CIIP operates is characterized by the following factors: (i) the weakness of local government systems; (ii) the poor access to services, including roads; (iii) the difficult economic situation facing the country; (iv) the high levels of poverty across the country; (v) the country’s challenging economic situation; (vi) the low status of women, particularly in rural Pakistan; and (vii) the extreme environmental challenges facing the country. According to the United Nations Human Development Report, Pakistan’s human development indicators, especially those for women, fall significantly below those of countries with 6 comparable levels of per-capita income. On the World Economic Forum’s 2013 Gender Equality Index, Pakistan ranked 141 out of the 142 countries in the index. Maternal mortality rates are high (260 per 100,000 live births) and labour force participation rates for women are 23 percent (compared with 83 percent for men). Seventy-five percent of Pakistani families are landless and this sets the stage for the perpetuation of bonds of patronage between landlords and local government bodies. On Transparency International’s 2013 Corruption Index, Pakistan ranked 127th out of 175 countries. Deeply