Data, metrics and monitoring in CGIAR – a strategic study December 2014 Cover image: courtesy Neil Palmer/CIAT The ISPC encourages appropriate use of its published information, including quotation, but asks that the source be acknowledged. The recommended reference for this publication is: ISPC. 2014. Data, metrics and monitoring in CGIAR – a strategic study. Rome, Italy, CGIAR Independent Science and Partnership Council (ISPC). 88 pp. Data, metrics and monitoring in CGIAR – a strategic study i Contents Acronyms and abbreviations .............................................................................................................................................................iii I. ISPC Synthesis and Commentary ...........................................................................................................................1 Background and rationale of the study .............................................................................................................................. 3 ISPC commentary on key findings and recommendations of the study .................................................................. 4 II. Panel Report: Data, metrics and monitoring in CGIAR – A strategic study ................................................... 11 Preface ....................................................................................................................................................................................................12 Executive summary .............................................................................................................................................................................13 1. Introduction .......................................................................................................................................................................................16 1.1 What do metrics mean in the context of this report? ...........................................................................................17 1.2 Why are metrics needed? ............................................................................................................................................17 2. An assessment of the current issues for CGIAR in developing a coherent framework ...........................................18 2.1 A summary of responses to a survey of CRPs on data, metrics and indicators ..........................................18 2.2 How the CRPs have approached program-level indicators .............................................................................19 2.3 Gaps and challenges .....................................................................................................................................................20 3. Criteria for selection of indicators...............................................................................................................................................23 3.1 Simple metrics .................................................................................................................................................................23 3.2 Composite indicators ....................................................................................................................................................23 3.3 Composite indicators: The example of total factor productivity (TFP) ...........................................................24 3.4 Selecting indicators from an understanding of the theory of change ............................................................27 3.5 The feasibility of using national or global statistics ..............................................................................................29 3.6 Linking metrics and indicators across scales and levels ...................................................................................29 4. CGIAR metrics .................................................................................................................................................................................31 4.1 Indicators of IDOs and SLOs, and their relationships ..........................................................................................31 4.2 Candidate metrics and indicators for the CGIAR SLOs.......................................................................................32 4.3 Intermediate development outcomes .....................................................................................................................34 4.4 Indicators of ‘difficult’ criteria ......................................................................................................................................36 4.5 Alignment with the Sustainable Development Goals .........................................................................................36 4.6 Foresight ...........................................................................................................................................................................39 4.7 Summary ............................................................................................................................................................................39 5. Data management .........................................................................................................................................................................41 5.1 Emerging trends in agricultural research and data ..............................................................................................41 5.2 Big data ..............................................................................................................................................................................41 Data, metrics and monitoring in CGIAR – a strategic study ii 5.3 Meta analysis, data annotation and potentials for interaction between data sets ....................................43 5.4 What has been done: Success stories.....................................................................................................................44 5.5 Opportunities for providing greater access to research data ..........................................................................44 5.6 CGIAR developments in open access and data management ........................................................................45 5.7 Creating systematic data sets ....................................................................................................................................47 5.8 Implications for data collection, collation and storage .......................................................................................48 5.9 The ethics of data use ...................................................................................................................................................48 6. Monitoring – Roles and responsibilities ..................................................................................................................................49 6.1 Data needs and responsibilities ................................................................................................................................49 6.2 Baseline studies ..............................................................................................................................................................50 6.3 Other initiatives on monitoring agriculture, land use, health and poverty ....................................................51 6.4 Use of metrics and indicators for accountability ...................................................................................................51 7. CGIAR metrics and indicator systems as an integrating framework ..............................................................................53 8. Conclusions and recommendations ........................................................................................................................................55 References .............................................................................................................................................................................................59 Annexes Annex 1. Glossary of terms and definitions ....................................................................................................................64 Annex 2. Summary of findings of a questionnaire survey conducted in October 2013 on CRP metrics .........................................................................................................................................................................66 Annex 3. Specific metrics/indicators used/planned for monitoring progress (CRP metrics survey questionnaire October 2013) .............................................................................................................................................73 Annex 4. Estimates of the size and scale of impact in IDO indicators from the three most advanced research areas (Extracted from A4NH CRP Concept Note) ..................................................................76 Annex 5. List of selected monitoring and metrics initiatives relevant to CGIAR mission .................................77 Annex 6. Further examples of long-term data collection projects ..........................................................................81 Annex 7. Common IDOs among the CRPs – Links with SLOs taken from the common IDO table in the April 2014 guidelines .................................................................................................................................................83