GODAN ACTION LEARNING PAPER

Knowledge Reservoir for Impact Evaluation A collection of resources on Impact Evaluation of for Development

Rob Lokers

Wageningen Environmental Research Laura Miguel Ayala Wageningen Environmental Research 22 February 2019 Dr Evangelia Berdou

Independent consultant GODAN ACTION LEARNING PAPER •••• KNOWLEDGE RESERVOIR FOR IMPACT EVALUATION

Executive summary

GODAN Action supports data users, producers and intermediaries to effectively engage with open data and maximise its potential for impact in the agriculture and nutrition sectors.

In particular, we work to strengthen capacity, to promote common standards and best practice, and to improve how we measure impact.

GODAN Action developed an evaluation framework aimed to evaluate open data initiatives in development countries. This framework was also applied and tested in the thematic topics of weather data, land data and nutrition data.

The developed knowledge, as well as the material that was used as an input to that process, are useful resources for any organisation that wants to evaluate impact of open data in general and specifically in the context of development. This GODAN Action deliverable provides a compilation of these resources, the GODAN Action knowledge reservoir.

The knowledge reservoir provides short descriptions and links to the impact evaluation resources that were developed by the GODAN Action project, specifically aimed at the development of a framework targeting open data in developing countries. Besides, it refers to additional sources and background material that was used in the project.

2 GODAN ACTION LEARNING PAPER •••• KNOWLEDGE RESERVOIR FOR IMPACT EVALUATION

Contents

​1 Introduction 04

2​ GODAN Action resources 04

Introduction 04

GODAN Action resources 04

3​ Impact evaluation methods 06

Introduction 06

Resources on impact evaluation methodologies 06

4 Secondary resources 06

4.1 Case studies 06

4.2 Impact evaluation methodologies 07

4.3 Open data indices 08

5 References 09

3 GODAN ACTION LEARNING PAPER •••• KNOWLEDGE RESERVOIR FOR IMPACT EVALUATION

1 Introduction

This document provides the knowledge reservoir for context, and include literature reviews, a framework for impact evaluation of open data initiatives for development impact evaluation and a range of impact evaluation case that has been developed by the GODAN Action project. It studies in the thematic topic that GODAN Action has provides references to the key deliverables of the GODAN worked on: open weather data, open land data and open Action work on impact evaluation, and additionally nutrition data. also a comprehensive compilation of other relevant resources. As such it is a useful reference base for any The subsequent sections of this document describe initiative considering impact evaluation of initiatives that (1) the most relevant of already existing impact and engage with open data, specifically where it concerns a outcome evaluation methods, that have inspired the development context. work of GODAN Action and the development of its impact evaluation framework; (2) an inventory of case The first section of the document provides short studies, methodologies and open data indices that could descriptions and references for the deliverables support the understanding of impact evaluation and (3) developed by the GODAN Action initiative to support an extensive list of references that are relevant for the impact evaluation. These deliverables are specifically area of impact evaluation and that have been referenced aimed at impact evaluation of open data in a developing by the GODAN Action activities in this field. 2 GODAN Action resources

GODAN action has through its activities in the area of broadly applicable. The framework particularly aims at impact evaluation developed a stack of resources that supporting two types of users: professional evaluators is specifically aimed at impact evaluation of open data who wish to know more about evaluation in the nexus initiatives in a development context. Up till now, material of open data and development, and practitioners who that focuses specifically on that domain is scarce. GODAN wish to learn more about how to maximize the potential Action has examined the current state of open data impact of open data initiatives and improving their approach to evaluation, in a broader context as well as specifically maximize impact. Through its design, the framework can aimed at the development context. Moreover, GODAN be used in different phases of open data initiatives, from Action is developing a dedicated framework for impact effective project design, improving potential impacts, to evaluation, and tests and improves this framework by intermediate process evaluations, and finally post-project applying it to use cases in various thematic topics. These outcome evaluation. thematic topics cover the areas of open weather data, open land data and open nutrition data. Moreover, GODAN Action will encourage uptake and support the uptake of the framework by external initiatives and organisations. This is expected to result in more case studies on the longer term. Guidelines for analysing pathways to impact (GODAN Action, 2019)

https://doi.org/10.7490/f1000research.1117778.1

The report describes an impact evaluation framework for open data initiatives that is also applied and tested it in various areas in the agriculture and nutrition domain. The framework uses some elements from existing methods, but adds components that make it more

4 GODAN ACTION LEARNING PAPER •••• KNOWLEDGE RESERVOIR FOR IMPACT EVALUATION

Open Data Impact Narratives – Stories A review of relevant methods and of impact of open data in agriculture frameworks for impact evaluation of and nutrition (GODAN Action, 2019) open data (GODAN Action, 2018)

https://doi.org/10.7490/f1000research.1117566.1 https://doi.org/10.7490/f1000research.1115589.1

This report describes impact narratives for a set of case This report outlines the main lessons from a review of studies on weather, land and nutrition data, examined approaches developed to evaluate the impact of open by GODAN Action. It contains descriptions of the impact data with a specific focus on open data engagement in chains and theory of change and the analysis of these developing countries. The review took into account a chains to evaluate impact. It provides examples of range of academic and policy oriented empirical studies how different use cases exploit different mechanisms that examine different aspects of initiatives incorporating to achieve impact and which drivers and barriers exist. open data, as well as methodological and conceptual Specific Theories of Change have been developed for frameworks designed to support open data evaluation. each case study which feed into advocacy and capacity building material, serving as practical examples of impact GODAN Action Learning Paper - New and enforcing the evidence base of open data. insights in impact evaluation of open data initiatives (GODAN Action, 2018) Impact narratives for weather data: pathways to encourage and support https://www.godan.info/sites/default/files/documents/ co-development of farm management Learning%20note_impact%20evaluation.pdf services (GODAN Action, 2017) This short briefing summarises GODAN Action's approach https://doi.org/10.7490/f1000research.1117548.1 to the focal area of impact evaluation of open data initiatives. It describes the state-of-play and learning This document presents a framework for understanding after the development of the first version of the impact and examining the potential of open weather data for evaluation framework and testing it on case studies in agriculture, which is illustrated through a set of use the area of weather data. It focuses on lessons learned cases. For that purpose a weather data specific Theory from the approach, and more specifically on open data of Change and impact chains have been derived. The programme leaders and impact evaluators. document also describes a framework of pre-conditions for achieving impact and examples of potential impact indicators. Three case studies in the area of open weather data are examined and documented. Land Data Impact Evaluations(GODAN Action, 2019)

https://doi.org/10.7490/f1000research.1117549.1

This document contributes to our Guidelines for analysing pathways to impact. It describes the data needs and activities in the land data domain, as well as possible conditions for users to make use of certain land data (and ultimately, generate ‘impact’), based on an extensive analysis of the various perspectives (local, regional, global) through interviews with relevant stakeholders. Theories of Change for each level were proposed, with the purpose of informing the Guidelines, which aim ultimately to serve as a tool to analyse the impact of Open Data in different contexts.

5 GODAN ACTION LEARNING PAPER •••• KNOWLEDGE RESERVOIR FOR IMPACT EVALUATION

3 Impact evaluation methods

For the development of its evaluation framework, GODAN The Common Assessment Framework, output of a Action has thankfully studeid, used and integrated useful workshop hosted by the Web Foundation and GovLab concepts and components that are already part of existing NYU in May 2014, puts forward a broader, and more approaches and methodologies. This section provides a general view of impact of open data than the one short summary of the methods we found to be specifically outlined in Sunlight’s Foundation report. The Common useful and that either cover the area of open data, the Assessment Framework suggests that the success of area of development or both. open data initiatives depend on four dimensions: Context/ Environment, Data, Use and Impact. The presented draft The Social Impact of Open Data (Keseru framework provides suggested questions and indicators and Chan, Sunlight Foundation, 2015) for each of these components, and looks at the existing projects that have piloted relevant methods. http://www.opendataresearch.org/dl/symposium2015/ odrs2015-paper20.pdf Researching the emerging impacts of open data: revisiting the ODDC The study describes a methodological framework to conceptual framework (Davies and support developing more robust indicators of social and political change for the ecosystem of open data Perini, ODDC, 2016) initiatives. An important concept is applying and revising the approach of outcome Mapping to the field of open http://ci-journal.net/index.php/ciej/article/ data. The study also delivered a searchable repository viewFile/1281/1217 or https://pdfs.semanticscholar.org/ of more than 100 examples on the outputs, outcomes fbf3/b67a2f4e1593eab547d29f364ef196322bfb.pdf and impacts of open data and digital technology projects This framework revisits the conceptual framework and three distinctive theories of change for open data developed for the “Exploring the Emerging Impacts of and digital transparency initiatives from the Global South. Open Data in Developing Countries” (ODDC) research The Common Assessment Framework project. It addresses the need for the framework to adapt on open data to the development challenges where open data is to be applied, whilst identifying the value of the framework overall to support cross-case learning and theory building. https://webfoundation.org/2014/06/ It incorporates lessons from a large number of case towards-common-methods-for-assessing-open-data/ studies.

4 Secondary resources

Besides the main resources for impact evaluation as 4.1Case studies developed by GODAN Action and the most relevant impact evaluation methodologies that inspired that Many case studies have been developed in various work, there is a substantial amount of resources that is initiatives that examine open data initiatives and their related to impact of open data and impact evaluation. This impact. Such studies tend to, in addition to ‘what’ and section describes the most interesting material that was ‘how’ questions, also raise ‘why’ questions – as in why identified as part of the GODAN Action work up till now. certain initiatives are successful or unsuccessful. Up It describes (1) open data case studies that also look at till now, a majority of these case studies have adopted impact, (2) theories, methodologies and framework for a qualitative approach, with only a few examples of impact evaluations or impact assessment and (3) open quantitative impacts. Qualitative case studies are the data indices. most widely adopted type of research design adopted in the field.

6 GODAN ACTION LEARNING PAPER •••• KNOWLEDGE RESERVOIR FOR IMPACT EVALUATION

List of open data case studies Impact evaluation of Open Data in a development context • GODAN Action: Impact narratives of open weather data initiatives - https://f1000research.com/ • Davies, T. (2014). Open Data in Developing documents/8-1759 Countries: Emerging Insights from Phase 1. World Web Foundation. • Exploring the Emerging Impacts of Open Data in Developing Countries (ODDC) - 17 qualitative case Impact evaluation in a development studies context The Social Impact of Open Data - Mapping social • Stern, E., Mayne, J., Stame, N., Forss, K., & Davies, change in 3 use case from the Global South • R. (2012). Broadening the Range of Designs and • Cañares, M., de Guia, J., Narca, M., & Arawiran, Methods for Impact Evaluations, Final report, J. (2014). Opening up the gates: Will open data London: Department for International Development initiatives make local governments in the Philippines, (DFID). more transparent? ODDC. • Laing, K., & Todd, L. (2015). Theory-Based • Chattapadhyay, S. (2014). Opening Government Methodology: Using Theories of Change for Data through Mediation: Exploring the Roles, Development, Research and Evaluation. Newcastle: Practices and Strategies of Data Intermediary Newcastle University. Organisations in India. Impact evaluation of Open Data • Mutuku, L., Mahihu, C. (2014). Open Data in Developing Countries, Understanding the Impacts • Carrara, W., San Chan, W., Fischer, M., & van of Kenya Open Data Applications and Services. Steenbergen, E. (2015). Creating Value through ODDC and *iHub Research. Open Data. Brussels: European Commission: Directorate General for Communication Networks, • Hogge, B. (2015). Open data: six stories about Content and Technology. impact in the UK. London: Omidyar Network. • Davies, T., & Perini, F. (2016). Researching the • Scrolini. (2014). Open Cities: the case of Montevideo. Emerging Impacts of Open Data: Revisiting ODDC. the ODDC Conceptual Framework. Journal of Community Informatics, 12(2), 148-178 • Van Shalkwyk, F., Willmers, M., & Czerniewicz. (2014). Open Data in the Governance of South • Koski, H. (2015). The Impact of Open Data: A African Higher Education. Cape Town: ODDC and Preliminary Study. Helsinki: Finnish Ministry of OpenUCT. Finance. • Mejabi, V. O., Azeez, L. A., Adedouin, A., & • Ubaldi, B. (2013). " data: Oluwaseyi, M. O. (2014). Case Study Report on Towards empirical analysis of open government Investigation of the Use of Online National Budget data initiatives" OECD Working Papers on Public of Nigeria. ODDC. Governance(22): 0_1 4.2 Impact evaluation • Zuiderwijk, A. and M. Janssen (2014). "Open data policies, their implementation and impact: A methodologies framework for comparison." Government Information Quarterly 31(1): 17-29.s A lot of work has been done on general impact evaluation. However, material on impact evaluation of open data General impact evaluation is still relatively scarce, specifically work that combines the perspectives of open data and development. This • Befani, B. (2012). Models of Causality and Causal section provides linkages to resources that provide Inference. A review prepared as part of the DFID valuable knowledge on impact evaluation, in the context study: ‘Broadening the Designs and Methods for of development, open data or the combination. Impact Evaluation’.

7 GODAN ACTION LEARNING PAPER •••• KNOWLEDGE RESERVOIR FOR IMPACT EVALUATION

• Bamberger, M. Introduction to mixed methods in impact evaluation. Impact Evaluation Notes No. 3. 2012. Interaction 4.3 Open data indices

Indices that describe the state of open government data are developed to be used primarily as a baseline tool to chart progress towards the increased availability of open data and as a tool for advocacy.

They address mainly ‘what’/’how’ types of questions (as in, ‘what is the state of open data worldwide?’ and, ‘how are different governments doing?’) and provide a high-level view of the progress that is being made towards . Indices and their associated scores tend to focus more on the conditions for impact, such as the technical characteristics of datasets that enable re-use, rather than impact. • Open Data Census • Global Open Data Index (GODI) • Open Data Barometer

8 GODAN ACTION LEARNING PAPER •••• KNOWLEDGE RESERVOIR FOR IMPACT EVALUATION

5 References

This section provides a list of references that have been used in the various GODAN Action activities performed in the context of impact evaluation. Some of them are directly related to open data for development and evaluation or evaluation case studies, and might have already been referred to in the previous sections. Others are related to the applied work that has been performed in the areas of open weather data, open land data and open nutrition data. General references related to open data for development and evaluation

Anderson, J.E. (1974). Public Policy-Making, Boston, Houghton Mifflin.

Antle, J. M., Basso, B., Conant, R. T., Godfray, H. C. J., Jones, J. W., Herrero, M. et al. (2017). Towards a new generation of agricultural system data, models and knowledge products: Design and improvement. Agricultural Systems, 155, 255-268.

Bamberger, M. Introduction to mixed methods in impact evaluation. Impact Evaluation Notes No. 3. 2012. Interaction http:// bit.ly/TiAvrQ

Berdou E and Miguel Ayala L. GODAN Action – a review of relevant methods and frameworks for impact evaluation of open data [version 1; not peer reviewed]. F1000Research 2018, 7:809 (document) (https://doi.org/10.7490/ f1000research.1115589.1)

Berdou E, Miguel Ayala L, Lokers R et al. Impact narratives for weather data: pathways to encourage and support co-development of farm management services [version 1; not peer reviewed]. F1000Research 2019, 8:1759 (document) (https://doi.org/10.7490/f1000research.1117548.1)

Berdou E and Lokers R. Nutrition data impact evaluations: guidelines for analysing pathways to impact [version 1; not peer reviewed]. F1000Research 2019, 8:1129 (document) (https://doi.org/10.7490/f1000research.1117077.1)

Baviskar. (2010). Winning the Right to Information in India: Is Knowledge Power? In J. Gaventa & R. McGee (Eds.), Citizen Action and National Policy Reform. London: Zed Books.

Befani, B. (2012). Models of Causality and Causal Inference. A review prepared as part of the DFID study: ‘Broadening the Designs and Methods for Impact Evaluation’.

Cañares, M., de Guia, J., Narca, M., & Arawiran, J. (2014). Opening up the gates: Will open data initiatives make local governments in the Philippines, more transparent? ODDC.

Cañares, M., & Shekhar, S. (2016). Open Data and Sub-National Governments: Lessons from Development Countries. Journal of Community Informatics, 12(2), 99-119.

Carrara, W., San Chan, W., Fischer, M., & van Steenbergen, E. (2015). Creating Value through Open Data. Brussels: European Commission: Directorate General for Communication Networks, Content and Technology.

Chambers, R. (2007). Who Counts? The Quiet Revolution of Participation and Numbers. IDS Working Paper 296, available for download at: http://www.ids.ac.uk/files/Wp296.pdf, last accessed 01.11.2011.

Chambers, R. (2008). Chapter 1: The Provocation: Dinosaurs. In Revolutions in Development Inquiry (pp. 1-24). London: Earthscan.

Chattapadhyay, S. (2014). Opening Government Data through Mediation: Exploring the Roles, Practices and Strategies of Data Intermediary Organisations in India.

Cole, R. J. (2012). Some observations on the practice of open data as opposed to its promise. The Journal of Community Informatics, 8(2).

9 GODAN ACTION LEARNING PAPER •••• KNOWLEDGE RESERVOIR FOR IMPACT EVALUATION

Collinson, S. (2003). Introduction. In S. Collinson (Ed.), Power, Livelihoods and Conflict: Case Studies in Political Economy Analysis for Humanitarian Action (pp. 7-20). London: Humanitarian Policy Group-Overseas Development Institute.

Conradie, P. and S. Choenni (2014). "On the barriers for local government releasing open data." Government Information Quarterly 31(Supplement 1): S10-S1

Cui, B., Liao, Q., Lam, W. W. T., Liu, Z. P., & Fielding, R. (2017). Avian influenza A/H7N9 risk perception, information trust and adoption of protective behaviours among poultry farmers in Jiangsu Province, China. BMC Public Health, 17(1), 463.

Davies, S. (1994). Introduction: Information, Knowledge and Power. IDS Bulletin, 25.2, 5-15.

Davies, T. (2014). Open Data in Developing Countries: Emerging Insights from Phase 1. World Web Foundation.

Davies, T., & Perini, F. (2016). Researching the Emerging Impacts of Open Data: Revisiting the ODDC Conceptual Framework. Journal of Community Informatics, 12(2), 148-178.

Deloitte. (2013). Market Assessment of Public Sector Information. London: UK Department for Business Innovation and Skills.

Ferris, L., & Rahman, Z. (2006). Responsible Data in Agriculture. Wageningen: GODAN.

Fox, J. (2007). The Uncertain Relationship between Transparency and Accountability. Development in Practice, 17(4), 663-671.

Frank, M., & Walker, J. (2016). User Centred Methods for Measuring the Quality of Open Data. Journal of Community Informatics, 12(2), 47-68.

Gaventa, J., & Barrett, G. (2010). So What Difference Does it Make? Mapping the Outcomes of Citizen Engagement. Citizenship, Participation and Accountability Research Centre, IDS.

Gaventa, J., & McGee, R. (2013). The impact of transparency and accountability initiatives. Development Policy Review, 31(s1), s3-s28.

GODAN Action briefing paper: “The necessity for Open data for land and property rights”, April 2018.

Goldstein, M. (2012). The Tao of Impact Evaluation. World Bank Blog: http://blogs.worldbank.org/impactevaluations/ the-tao-of-impact-evaluation.

Gurstein, M. (2011). Open Data: Empowering the Empowered or Effective Data Use for Everyone? First Monday, available at: http://ow.ly/JXvw100gzk4 , last accessed 07.08.16., 16(2).

Gurstein, M. (2012). Two Worlds of Open Government Data: Getting the Lowdown on Public Toilets in Chennai and Other Matters. The Journal of Community Informatics, 8(2).

Heft-Neal, S., Lobell, D. B., & Burke, M. (2017). Using remotely sensed temperature to estimate climate response functions. Environmental Research Letters, 12(1).

Herring, R. J. (2004). Data as Social Product. In R. Kanbur (Ed.), Q-squared: Quantitative and Quantitative Methods of Poverty Appraisal (pp. 141-151). Delhi: Permanent Black.

Hogge, B. (2015). Open data: six stories about impact in the UK. London: Omidyar Network.

Hossain, M. A., Dwivedi, Yogesh, K., & Rana, N. P. (2016). State-of-the-art in open data research: Insights from existing literature and a research agenda. Journal of Organizational Computing and Electronic Commerce, 26(1-2), 14-40.

Huijboom, N. and T. Van den Broek (2011). Open data: an international comparison of strategies. European journal of ePractice 12(1): 4-16.

10 GODAN ACTION LEARNING PAPER •••• KNOWLEDGE RESERVOIR FOR IMPACT EVALUATION

Janssen, M., Charalabidis, Y., & Zuiderwijk, A. (2012). Benefits, Adoption Barriers and Myths of Open Data and Open Government. Information Systems Management, 29(4), 258-268.

Jerven, M. (2013). Poor numbers: how we are misled by African development statistics and what to do about it. London and Ithaka: Cornell University Press.

Jerven, M. (2014). The Political Economy of Agricultural Statistics and Input Subsidies: Evidence from India, Nigeria and Malawi. Journal of Agrarian Change, 14(1), 129-145.

Keseru, J., Chan, J. (2015). The Social Impact of Open Data. Sunlight Foundation.

Keiron, O. (2011). Transparent Governments, not Transparent Citizens: A report on privacy and transparency for the Cabinet Office. London: Cabinet Office.

Koski, H. (2015). The Impact of Open Data: A Preliminary Study. Helsinki: Finnish Ministry of Finance.

Krätke, F., & Byiers, B. (2014). The Political Economy of Official Statistics: Implications for the Data Revolution in Sub-Saharan Africa. Discussion Paper no 5-Partnership in Statistics in the 21st Century-PARIS21, Available for download at: http://ow.ly/3yowgF, last accessed 04.11.15.

Labonne, J., & Chase, R. S. (2009). The Power of Information: The Impact of Mobile Phones on Farmers; Welfare in the Philippines. Policy Research Working Paper 4996, Sustainable Development Network, the World Bank.

Laing, K., & Todd, L. (2015). Theory-Based Methodology: Using Theories of Change for Development, Research and Evaluation. Newcastle: Newcastle University.

Lämmerhirt, D., Rubinstein, M., & Oscar, M. (2017). The State of Open Government Data in 2007: Creating meaningful open data through multi-stakeholder dialogue. London: Open Knowledge International.

Lokers R, Berdou E, Miguel Ayala L and Mey L. Guidelines for analysing pathways to impact: Evaluation of open data for development [version 1; not peer reviewed]. F1000Research 2019, 8:2115 (document) (https://doi.org/10.7490/ f1000research.1117778.1) Open Data Impact Narratives: stories of impact of open data in agriculture and nutrition

Malakooti, B. (2013). Operations and Production Systems with Multiple Objectives. John Wiley & Sons.

Manyika, J., Chui, M., Groves, P., Farell, D., Van Kuiken, S., & Almashi Dosi, E. (2013). Open Data: Unlocking information and performance with Liquid Information: Mckinzey.

Margoro, T. A. S., Shigeo. (2011). The Barriers of the Indonesian Extension Workers to Disseminate Agricultural Information to Farmers. International Journal of Basic and Applied Sciences, 11(2), 80-86.

Mejabi, V. O., Azeez, L. A., Adedouin, A., & Oluwaseyi, M. O. (2014). Case Study Report on Investigation of the Use of Online National Budget of Nigeria. ODDC.

Mey L, Berdou E, Miguel Ayala L and Lokers R. Open Data Impact Narratives – Stories of Impact of Open Data in Agriculture and Nutrition [version 1; not peer reviewed]. F1000Research 2019, 8:1838 (document) (https://doi.org/10.7490/ f1000research.1117566.1)

Mey L, Miguel Ayala L and Lokers R. Land data impact evaluations: Guidelines for analysing pathways to impact [version 1; not peer reviewed]. F1000Research 2019, 8:1758 (document) (https://doi.org/10.7490/f1000research.1117549.1)

Michael P. Canares, Andrew Young, Stefaan Verhulst: “Open Development Cambodia. Opening Information on Development Efforts”, Open Data for Developing Economies, July 2017.

Mutuku, L., & Mahihu, C. (2014). Understanding the impact of open data applications and services. Kenya: ODDC and *iHub Research.

Open Data Institute. (2015). Open Data means business: UK innovation across sectors and regions. 2015: ODI.

11 GODAN ACTION LEARNING PAPER •••• KNOWLEDGE RESERVOIR FOR IMPACT EVALUATION

Open Knowledge International, ‘Open Data Index’, 2015.

Pichel, “How Open Data in Land Information Impacts Government Efficiency and Citizen Access to Data”, Technical University of Munchen Thesis, February 2017.

Raman, N. V. (2012). Collecting Data in Chennai City and the Limits of Openness. Journal of Community Informatics, 8(2).

Sanderfur, J., & Glassman, A. (2014). The Political Economy of Bad Data: Evidence from African Survey & Administrative Statistics. Global Centre for Development Background Paper, Available for download at: http://ow.ly/3yownA, last accessed 04.11.15.

Scrolini. (2014). Open Cities: the case of Montevideo. ODDC.

Stern, E., Mayne, J., Stame, N., Forss, K., & Davies, R. (2012). Broadening the Range of Designs and Methods for Impact Evaluations, Final report, London: Department for International Development (DFID).

Thomas, P. (2009). Bhoomi, Gyan Ganga, e-governance and the right to information: ICTs and development in India. Telematics and Informatics, 26(1), 20-31.

Ubaldi, B. (2013). "Open government data: Towards empirical analysis of open government data initiatives." OECD Working Papers on Public Governance(22): 0_1.

Van Shalkwyk, F., Willmers, M., & Czerniewicz. (2014). Open Data in the Governance of South African Higher Education. Cape Town: ODDC and OpenUCT.

World Wide Web Foundation. (2017). Open Data Barometer, 4th Edition: Research Handbook Final. London: World Wide Web Foundation.

Winseck, D. (2011). The political economies of media: The transformation of the global media industries: Introductory Essay. In D. Winseck & D. Y. Jin (Eds.), The political economies of media: The transformation of the global media industries (pp. 3-48). London: Bloomsbury Publishing.

World Web Foundation, Open Data Barometer Global Report. Third Edition, April 2016, p. 20.

Zuiderwijk, A. and M. Janssen (2014). "Open data policies, their implementation and impact: A framework for comparison." Government Information Quarterly 31(1): 17-29.s References related to open weather, land and nutrition data

Cadasta Foundation, “Overview of Property Rights Data”: https://cadasta.org/open-data/overview-of-property-rights-data/, consulted d.d. November 21st 2018.

Dodds, L., Longden, A., McLellan, S., & Smith, A. (2017). The state of weather data infrastructure. London: Open Data Institute.

FIG, “FIG Statement on the Cadastre” FIG publication No 11, 2014. http://www.fig.net/resources/publications/figpub/pub11/ figpub11.asp

Food and Agriculture Organization: “Land Tenure and Rural Development”, 2002.

Global Donor Working Group on Land, “About Land Governance”: https://www.donorplatform.org/about-land-governance. html, consulted d.d. November 21st 2018.

Global Donor Working Group on Land: “Renewed commitment on Indicator 1.4.2. For achieving global land security in the SDGs”, Policy Brief No. 14, June 2017.

Global Land tool Network, “The Continuum of Land Rights”, UN-Habitat: http://mirror.gltn.net/index.php/land-tools/gltn-land- tools/continuum-of-land-rights, consulted d.d. October 22nd 2018.

12 GODAN ACTION LEARNING PAPER •••• KNOWLEDGE RESERVOIR FOR IMPACT EVALUATION

Global Land Tool Network, “Handling land : Innovative Tools for Land Governance and Secure Tenure”. UN-Habitat, 2012.

National Ocean Service, “What’s the difference between land cover and land use”, consulted d.d. October 22nd 2018: https:// oceanservice.noaa.gov/facts/lclu.html

Land Tenure and Rural Development, FAO Land Tenure Studies, No 3, Rome, 2002. http://www.fao.org/DOCREP/005/ Y4307E/y4307e05.htm.

Mendelsohn, R., Kurukulasuriya, P., Basist, A., Kogan, F., & Willams, C. (2007). Climate Analysis with Satellite versus Weather Station Data. Climatic Change, 81(1), 71-83.

Tennison, J., Dodds, L., Pesce, V., Zervas, P., Recommendations for filling identified gaps: weather data. GODAN Action, 2017. https://doi.org/10.7490/f1000research.1115857.1

Pesce, V., Mey, L., L’Hénaff, P., Tejo-Alonso. C., Land and nutrition data in the Land Portal and the Global Nutrition Report: a gap exploration report, 2018. https://doi.org/10.7490/f1000research.1116235.1

Rights and Resources Initiative: “Who Owns the World’s Land”, September 2015.

Rights and Resources Initiative, Vasundhara, and Natural Resources Management Consultants: “Potential for Recognition of Community Forest Resource Rights under India’s Forest Rights Act”, 2015.

Rights and Resources Initiative: “What Future for Reform? Progress and slowdown in forest tenure reform since 2002”, 2014. Safarov, I., Grimmelikhuijsen, S. G., & Meijer, A. J. (2017). Utilization of Open Government Data: A Systematic Review of Types, Conditions, Effects and Users. Information Polity, 22(1), 1-24.

Stig Enemark et al, “Fit-For-Purpose Land Administration”, March 2014.

United Nations Human Settlements Programme (UN-Habitat): “Handling Land: Innovative tools for land governance and secure tenure”, 2012.

Wiley, Liz Alden: “Whose Land Is It? Commons and Conflict States”, 2008.

WMO. (2008). Capacity Assessment of National Meteorological and Hydrological Services in Support of Disaster Reduction. Geneva: World Meteorological Organization.

WMO. (2012). Results of the Survey on the Role and Operation of National Meteorological and Hydrological Services, Conducted in June-August 2011. Geneva: World Meteorological Organisation.

WMO. (2015). Role and Operation of National Meteorological and Hydrological Services: A statement by the World Meteorological Organisation for Directors of NMHSs. Geneva: World Meteorological Organisation.

13 GODAN Action is supported by the UK Department for International Development (DFID), led by Wageningen Environmental Research with international partners AgroKnow, AidData, CTA, FAO, GFAR, IDS, Land Portal, and the ODI.

For more information visit the GODAN website www.godan.info/godan-action

Follow GODAN on Twitter: @godanSec

This GODAN Action publication is licensed under a Attribution 3.0 Unported License.