Knowledge Reservoir for Impact Evaluation Rob
Total Page:16
File Type:pdf, Size:1020Kb
GODAN ACTION LEARNING PAPER Knowledge Reservoir for Impact Evaluation A collection of resources on Impact Evaluation of Open Data for Development Rob Lokers Wageningen Environmental Research Laura Miguel Ayala Wageningen Environmental Research 22 February 2019 Dr Evangelia Berdou Independent consultant GODAN ACTION LEARNING PAPER •••• KNOWLEDGE RESERVOIR FOR IMPACT EVALUATION Executive summary GODAN Action supports data users, producers and intermediaries to effectively engage with open data and maximise its potential for impact in the agriculture and nutrition sectors. In particular, we work to strengthen capacity, to promote common standards and best practice, and to improve how we measure impact. GODAN Action developed an evaluation framework aimed to evaluate open data initiatives in development countries. This framework was also applied and tested in the thematic topics of weather data, land data and nutrition data. The developed knowledge, as well as the material that was used as an input to that process, are useful resources for any organisation that wants to evaluate impact of open data in general and specifically in the context of development. This GODAN Action deliverable provides a compilation of these resources, the GODAN Action knowledge reservoir. The knowledge reservoir provides short descriptions and links to the impact evaluation resources that were developed by the GODAN Action project, specifically aimed at the development of a framework targeting open data in developing countries. Besides, it refers to additional sources and background material that was used in the project. 2 GODAN ACTION LEARNING PAPER •••• KNOWLEDGE RESERVOIR FOR IMPACT EVALUATION Contents 1 Introduction 04 2 GODAN Action resources 04 Introduction 04 GODAN Action resources 04 3 Impact evaluation methods 06 Introduction 06 Resources on impact evaluation methodologies 06 4 Secondary resources 06 4.1 Case studies 06 4.2 Impact evaluation methodologies 07 4.3 Open data indices 08 5 References 09 3 GODAN ACTION LEARNING PAPER •••• KNOWLEDGE RESERVOIR FOR IMPACT EVALUATION 1 Introduction This document provides the knowledge reservoir for context, and include literature reviews, a framework for impact evaluation of open data initiatives for development impact evaluation and a range of impact evaluation case that has been developed by the GODAN Action project. It studies in the thematic topic that GODAN Action has provides references to the key deliverables of the GODAN worked on: open weather data, open land data and open Action work on impact evaluation, and additionally nutrition data. also a comprehensive compilation of other relevant resources. As such it is a useful reference base for any The subsequent sections of this document describe initiative considering impact evaluation of initiatives that (1) the most relevant of already existing impact and engage with open data, specifically where it concerns a outcome evaluation methods, that have inspired the development context. work of GODAN Action and the development of its impact evaluation framework; (2) an inventory of case The first section of the document provides short studies, methodologies and open data indices that could descriptions and references for the deliverables support the understanding of impact evaluation and (3) developed by the GODAN Action initiative to support an extensive list of references that are relevant for the impact evaluation. These deliverables are specifically area of impact evaluation and that have been referenced aimed at impact evaluation of open data in a developing by the GODAN Action activities in this field. 2 GODAN Action resources GODAN action has through its activities in the area of broadly applicable. The framework particularly aims at impact evaluation developed a stack of resources that supporting two types of users: professional evaluators is specifically aimed at impact evaluation of open data who wish to know more about evaluation in the nexus initiatives in a development context. Up till now, material of open data and development, and practitioners who that focuses specifically on that domain is scarce. GODAN wish to learn more about how to maximize the potential Action has examined the current state of open data impact of open data initiatives and improving their approach to evaluation, in a broader context as well as specifically maximize impact. Through its design, the framework can aimed at the development context. Moreover, GODAN be used in different phases of open data initiatives, from Action is developing a dedicated framework for impact effective project design, improving potential impacts, to evaluation, and tests and improves this framework by intermediate process evaluations, and finally post-project applying it to use cases in various thematic topics. These outcome evaluation. thematic topics cover the areas of open weather data, open land data and open nutrition data. Moreover, GODAN Action will encourage uptake and support the uptake of the framework by external initiatives and organisations. This is expected to result in more case studies on the longer term. Guidelines for analysing pathways to impact (GODAN Action, 2019) https://doi.org/10.7490/f1000research.1117778.1 The report describes an impact evaluation framework for open data initiatives that is also applied and tested it in various areas in the agriculture and nutrition domain. The framework uses some elements from existing methods, but adds components that make it more 4 GODAN ACTION LEARNING PAPER •••• KNOWLEDGE RESERVOIR FOR IMPACT EVALUATION Open Data Impact Narratives – Stories A review of relevant methods and of impact of open data in agriculture frameworks for impact evaluation of and nutrition (GODAN Action, 2019) open data (GODAN Action, 2018) https://doi.org/10.7490/f1000research.1117566.1 https://doi.org/10.7490/f1000research.1115589.1 This report describes impact narratives for a set of case This report outlines the main lessons from a review of studies on weather, land and nutrition data, examined approaches developed to evaluate the impact of open by GODAN Action. It contains descriptions of the impact data with a specific focus on open data engagement in chains and theory of change and the analysis of these developing countries. The review took into account a chains to evaluate impact. It provides examples of range of academic and policy oriented empirical studies how different use cases exploit different mechanisms that examine different aspects of initiatives incorporating to achieve impact and which drivers and barriers exist. open data, as well as methodological and conceptual Specific Theories of Change have been developed for frameworks designed to support open data evaluation. each case study which feed into advocacy and capacity building material, serving as practical examples of impact GODAN Action Learning Paper - New and enforcing the evidence base of open data. insights in impact evaluation of open data initiatives (GODAN Action, 2018) Impact narratives for weather data: pathways to encourage and support https://www.godan.info/sites/default/files/documents/ co-development of farm management Learning%20note_impact%20evaluation.pdf services (GODAN Action, 2017) This short briefing summarises GODAN Action's approach https://doi.org/10.7490/f1000research.1117548.1 to the focal area of impact evaluation of open data initiatives. It describes the state-of-play and learning This document presents a framework for understanding after the development of the first version of the impact and examining the potential of open weather data for evaluation framework and testing it on case studies in agriculture, which is illustrated through a set of use the area of weather data. It focuses on lessons learned cases. For that purpose a weather data specific Theory from the approach, and more specifically on open data of Change and impact chains have been derived. The programme leaders and impact evaluators. document also describes a framework of pre-conditions for achieving impact and examples of potential impact indicators. Three case studies in the area of open weather data are examined and documented. Land Data Impact Evaluations(GODAN Action, 2019) https://doi.org/10.7490/f1000research.1117549.1 This document contributes to our Guidelines for analysing pathways to impact. It describes the data needs and activities in the land data domain, as well as possible conditions for users to make use of certain land data (and ultimately, generate ‘impact’), based on an extensive analysis of the various perspectives (local, regional, global) through interviews with relevant stakeholders. Theories of Change for each level were proposed, with the purpose of informing the Guidelines, which aim ultimately to serve as a tool to analyse the impact of Open Data in different contexts. 5 GODAN ACTION LEARNING PAPER •••• KNOWLEDGE RESERVOIR FOR IMPACT EVALUATION 3 Impact evaluation methods For the development of its evaluation framework, GODAN The Common Assessment Framework, output of a Action has thankfully studeid, used and integrated useful workshop hosted by the Web Foundation and GovLab concepts and components that are already part of existing NYU in May 2014, puts forward a broader, and more approaches and methodologies. This section provides a general view of impact of open data than the one short summary of the methods we found to be specifically outlined in Sunlight’s Foundation report. The Common useful and that either cover the area of open data, the Assessment