Assessing Impacts of Forest Governance Interventions: Learning from World Bank Experience
Total Page:16
File Type:pdf, Size:1020Kb
MAY 2014 A PROFOR WORKING PAPER PROFOR WORKING PAPER ASSESSING IMPACTS OF FOREST GOVERNANCE INTERVENTIONS: LEARNING FROM WORLD BANK EXPERIENCE Nalin Kishor Maria Ana de Rijk ACKNOWLEDGMENTS This report has been prepared by Nalin Kishor and Maria Ana de Rijk. Helpful comments and suggestions were received from Aparajita Goyal, Arianna Legovini, Avjeet Singh, Dan Miller, Dan Stein, Diji Chandrasekharan Behr, Ijeoma Emenanjo, Peter Jipp, Selene Castillo and Tuukka Castrén and Valerie Hickey. Veronica Jarrin provided operational support since the inception of this activity. Sujatha Venkat Ganeshan provided editorial, layout and publication support. This work was funded by the Program on Forests (PROFOR), a multi-donor partnership managed by a Secretariat at the World Bank. PROFOR finances forest-related analysis and processes that support the following goals: improving people’s livelihoods through better management of forests and trees; enhancing forest governance and slaw enforcement; financing sustainable forest management; and coordinating forest policy across sectors. Learn more at www.profor.info. DISCLAIMER All omissions and inaccuracies in this document are the responsibility of the authors. The views expressed do not necessarily represent those of the institutions involved, nor do they necessarily represent official policies of PROFOR or the World Bank. SUGGESTED CITATION Kishor, Nalin and Maria Ana de Rijk, 2014. Assessing impacts of forest governance interventions: Learning from World Bank experience. Washington DC: Program on Forests (PROFOR). Material in this paper can be copied and quoted freely provided acknowledgment is given. For a full list of publications please contact: Program on Forests (PROFOR) 1818 H Street, NW Washington, DC 20433, USA [email protected] www.profor.info/knowledge PROFOR is a multi-donor partnership supported by the European Commission, Finland, Germany, Italy, Japan, the Netherlands, Switzerland, the United Kingdom and the World Bank. ACRONYMS AAA Analytic and Advisory Activities AES Agriculture and Environmental Services AfDB African Development Bank AFR Africa ALSS Arun Valley Living Standards Survey APL Adaptable Program Loan BNDES National Bank for Economic and Social Development BTOR Back-to-Office Report CCT Conditional Cash Transfer CIDA Canadian bilateral assistance CIF Climate Investment Funds CoP Community of Practice CWIQ Core Welfare Indicators Questionnaires DFS District Forest Service DGFP General Directorate of Forests and Pastures DIME Development Impact Evaluation Initiative DoF Department of Forestry DoFI Department of Forestry Inspection DP Development Partners DPL Development Policy Loan DPO Development Policy Operation DRC Democratic Republic of Congo EAP East Asia Pacific EC European Commission ECA Europe and Central Asia EIA Environmental Impact Assessment EITI Extractive Industries Transparency Initiative EMGRG Economic Management and Governance Reform Grant Program ENA-FLEG Europe and North Asia Forest law Enforcement and Governance in Europe and North Asia ENPI European Neighborhood Policy Instrument ENV TAL Technical Assistance Loan for the Environmental Sustainability Agenda EPA Environmental Protection Agency EU European Union FBiH Federation of Bosnia and Herzegovina ESW Economic and Sector Work FC Forestry Commission (of Ghana) FNCP Forest Nature and Conservation Project FFA Federal Forest Agency 3 FLEGT Forest Law Enforcement Governance and Trade FM Financial Management FMIS Forest Management Information System FMU Forest Management Unit FPUA Forest and Pasture Users Associations FSC Forest Stewardship Council GAC Governance and Anticorruption GEF Global Environment Facility GEO Global Environment Objective GFW Global Forest Watch GIS Global Information Systems GIZ German Society for International Cooperation GoB Government of Brazil GoG Government of Ghana GoL Government of Lao PDR IBAMA Brazilian Institute for the Environment and Natural Resources IBRD International Bank for Reconstruction and Development IBTV Igman-Bjelasnica-Treskavica-Visocica ICCN Congolese Nature Conservation Institute ICMBio Institute Chico Mendes Biodiversity ICR Implementation Completion Report ICT Information and Communication Technologies IDA International Development Association IE Impact Evaluation IEG Independent Evaluation Group IOI Intermediate Outcome Indicators INPE Brazilian Space Research Agency ISR Implementation Status Report IV Instrumental Variable LAC Latin America and the Caribbean LED Law Enforcement Division LSMS Living Standards Measurement Surveys LUCC Land Use and Cover Change MDA Ministries, Departments, and Agencies M&E Monitoring and Evaluation MECNT Ministry of Environment, Nature Conservation and Tourism (of the Democratic Republic of Congo) METT Management Effective Tracking Tool MINEF Ministry of Environment and Forestry MINFOF Ministry of Forestry and Wildlife (of Cameroon) MIT Massachusetts Institute of Technology MMA Ministry of Environment MNRE Ministry of Natural Resources and Environment MoFEP Ministry of Finance and Economic Planning MTCC Malaysian Timber Certification Council MTR Mid-Term Review NBCA National Biodiversity Conservation Areas NFPP Natural Forest Protection Programme NGO Non-Governmental Organization NP National Park NPMO National Project Management Office NREG Natural Resources and Environmental Governance Program 4 NRM Natural Resources Management NTFP Non-Timber Forest Products NUKCFP Nepal-United Kingdom Community Forestry Project OLS Ordinary Least Squares OM Outcome Mapping ONP Ordubad National Park PA Protected Areas PAD Project Appraisal Document PAF Program Assessment Framework PAME Protected Areas Management Effectiveness PAS National Sustainable Amazon Program PD Program Document PDO Project Development Objective PEFC Programme for the Endorsement of Forest Certification (PEFC) PES Payment for Environmental Services PFA Production Forest Areas PIU Project Implementing Unit PMC Project Management Center PMT Project Management Team PP Project Paper PRL Programmatic Reform Loan PROCEDE Programa de Certificacion de Derechos Ejidales PROFOR Program on Forests PRSC Poverty Reduction Support Credit PRSP Poverty Reduction Strategy Paper PSFM Participatory Sustainable Forest Management PSM Propensity Score Matching RCT Randomized Control Trials RDD Regression Discontinuity Design SAC Structural Adjustment Credit SAR South Asia Region SDNP Shah Dag National Park SEM Sustainable Environmental Management SFA State Forestry Administration SFB Brazilian Forest Service SFI State Forest Inventory SFM Sustainable Forest Management SIL Specific Investment Loan SUFORD Sustainable Forestry for Rural Development TA Technical Assistance TAL Technical Assistance Loan TTL Task Team Leader VD Village Development VPA Voluntary Partnership Agreement WB World Bank WRI World Resources Institute WWF World Wildlife Fund 5 Contents Executive Summary ....................................................................................................................................................... 9 Chapter 1: Background and Motivation ........................................................................................................................ 11 Chapter 2: Monitoring, Evaluation, and Impact Evaluation: Definitions and Scope ...................................................... 14 2.1 Monitoring .......................................................................................................................................................... 14 2.2 Evaluation .......................................................................................................................................................... 14 2.3 Impact Evaluation (IE) ....................................................................................................................................... 15 2.4 Monitoring and Evaluation (M&E) ...................................................................................................................... 16 Chapter 3: The Challenge of Measuring Impacts for Interventions to Improve Forest Governance............................. 19 3.1 General Challenges when Measuring Impacts .................................................................................................. 19 3.2 Challenges to Estimating Impacts in Forest Governance Interventions ............................................................. 20 Chapter 4: M&E Approaches Used Routinely in World Bank’s Forestry Programs and Projects ................................. 23 4.1 Results Frameworks .......................................................................................................................................... 23 4.2 Policy Matrices................................................................................................................................................... 23 Chapter 5: Looking Back and Learning from the Evidence .......................................................................................... 25 5.1 Selecting the Sample of Forest Programs, Projects, ESWs, and TAs with Governance Components ....... 25 5.1.1 Project Identification........................................................................................................................... 25 5.1.2 Collecting Project Information ............................................................................................................ 29 5.1.3 Selection of ESWs and TAs ..............................................................................................................