Science Advice for Policy by European Academies MAKING SENSE OF SCIENCE FOR POLICY UNDER CONDITIONS OF COMPLEXITY AND UNCERTAINTY Evidence Review Report No. 6 This Evidence Review Report has been produced under the auspices of the SAPEA Consortium, as a deliverable under Grant Agreement 737432 “Science Advice for Policy by European Academies” (SAPEA) that was signed by the Consortium and the European Commission on 22 November 2016. The text of this work is licensed under the terms of the Creative Commons Attribution licence which permits unrestricted use, provided the original author and source are credited. The licence is available at: http://creativecommons.org/licenses/by/4.0. Images are not covered by this licence. A list of the experts who contributed to the report is available in Annex 7. In accordance with Article 35, paragraph 1 of the aforementioned Grant Agreement, the Consortium identified all relevant interests of the experts requested to contribute to the Evidence Review Report, assessed whether an interest constituted a conflict of interests, and took — where relevant — measures to exclude that an interest could compromise or be reasonably perceived as compromising the impartiality or objectivity of the report. Further information about SAPEA’s working processes are available at www.sapea.info/guidelines. The information, facts and opinions set out in this report are those of the authors and do not necessarily reflect the opinion of the European Commission. The SAPEA Consortium is not responsible for the use which may be made of the information contained in this report by anyone, including the European Union institutions and bodies or any person acting on their behalf. This report can be viewed online at: www.sapea.info/making-sense-of-science ISBN - 978-3-9820301-3-5 DOI 10.26356/MASOS SAPEA, Science Advice for Policy by European Academies. (2019). Making sense of science for policy under conditions of complexity and uncertainty. Berlin: SAPEA. https://doi.org/10.26356/MASOS PUBLISHER CONTACT SAPEA SAPEA Communications Office c/o acatech 13 Rue d’Egmont Pariser Platz 4a Brussels 1000 | Belgium 10117 Berlin | Germany [email protected] Making sense of science for policy under conditions of complexity and uncertainty Informs the European Commission Group of Chief Scientific Advisors’ Scientific Opinion 7 (forthcoming) 1 Version history Version Date Summary of changes 2019.1.0 09 July 2019 First published version 2019.1.1 01 August 2019 First printed version. Corrected minor typos 2 About SAPEA SAPEA (Science Advice for Policy by European Academies) brings together outstanding expertise in engineering, humanities, medicine, natural and social sciences from over 100 academies, young academies and learned societies across Europe. SAPEA is part of the European Commission’s Scientific Advice Mechanism. Together with the Group of Chief Scientific Advisors, we provide independent scientific advice to European Commissioners to support their decision-making. We also work to strengthen connections between Europe’s academies and Academy Networks, and to stimulate debate in Europe about the role of evidence in policymaking. The Evidence Review Report informs the GCSA’s Scientific Opinion on the topic (forthcoming), which will be available here: http://ec.europa.eu/research/sam/index.cfm?pg=science Funded through the EU’s Horizon 2020 programme, the SAPEA consortium comprises Academia Europaea (AE), All European Academies (ALLEA), the European Academies’ Science Advisory Council (EASAC), the European Council of Academies of Applied Sciences, Technologies and Engineering (Euro-CASE), and the Federation of European Academies of Medicine (FEAM). For further information about SAPEA, visit www.sapea.info 3 Table of Contents 3 ABOUT SAPEA 7 FOREWORD 10 PREFACE 12 EXECUTIVE SUMMARY 18 CHAPTER 1: INTRODUCTION 21 CHAPTER 2: SCIENCE AS A SOURCE OF ADVICE FOR POLICYMAKING 21 2.1 The purpose of science for policymaking 25 2.2 The relevance of scientific expertise for policymaking 27 2.3 Complexity, uncertainty and ambiguity: three conditions of scientific knowledge 27 2.3.1 Complexity 29 2.3.2 Scientific uncertainty 34 2.3.3 Socio-political ambiguity 36 CHAPTER 3: WHAT CAN SCIENCE OFFER TO POLICYMAKING? THE POTENTIAL CONTRIBUTION OF SCIENCE, EVIDENCE AND KNOWLEDGE 36 3.1 Science advice in contemporary societies 37 3.2 Conditions: science as a form of knowledge 41 3.3 Scientific evidence: what constitutes ‘good’ evidence 45 3.4 Plurality of knowledge beyond scientific information and evidence 46 3.5 Dealing with normative issues and assumptions: knowledge, interests and values 48 3.6 Process: science as advice to policy 50 3.7 Challenges: dissent, ignorance and uncertainty 54 CHAPTER 4: USING SCIENTIFIC EVIDENCE IN POLICYMAKING CONTEXTS 54 4.1 The function of scientific evidence in policymaking 57 4.2 The needs of policymakers: a classification of functions of science for policymaking 58 4.2.1 Enlightenment 59 4.2.2 Orientation 59 4.2.3 Designing and evaluating strategies 60 4.2.4 Integration 60 4.2.5 Co-creation 61 4.3 Heuristics and frames in dealing with complex and uncertain information 61 4.3.1 Cognitive heuristics and biases 66 4.3.2 Technical and issue biases 69 4.3.3 The effects of framing 71 4.4 Communicating science and uncertainty 79 CHAPTER 5: THE POTENTIAL FOR IMPROVING THE USE OF SCIENTIFIC ADVICE FOR POLICYMAKING 79 5.1 Translating scientific evidence into policy-relevant science advice 83 5.2 Wicked problems, post-normal science and knowledge quality assessment 4 Table of Contents 86 5.3 Guidelines for integrating uncertainty into policy advice 87 5.3.1 The Guidance approach of the Netherlands Environmental Assessment Agency 88 5.3.2 Sensitivity auditing 89 5.3.3 NUSAP 90 5.4 Guidelines for taking potential social impact into account 93 5.5 Guidelines for integrating different types of knowledge into the policy process 96 5.6 Guidelines for integrating values in evidence-informed decision-making 99 5.7 Building a transdisciplinary approach for designing a working interface between expertise and policies 99 5.7.1 Emerging design principles for the ‘working interface’ 100 5.7.2 Trustworthiness 101 5.7.3 Independence 102 5.7.4 Transparency 103 5.7.5 Inclusiveness, accessibility and responsiveness 104 5.7.6 Deliberation 104 5.7.7 Rigour 105 5.7.8 Precaution 106 5.7.9 Responsibility 107 5.7.10 Democracy 108 5.7.11 Implications 108 5.8 Possibilities: the European landscape for science advice 111 5.9 The landscape of science advisory mechanisms 118 CHAPTER 6: LESSONS LEARNED 118 6.1 Basic insights from the report 124 6.2 Lessons for the organisation of scientific advice to policymaking on the European level 126 6.3 Science-policy-society interface within a European framework 130 CHAPTER 7: REFERENCES 158 ANNEX 1: ADDRESSING THE SCOPING QUESTIONS 167 ANNEX 2: BACKGROUND TO THE REPORT 169 ANNEX 3: LITERATURE SEARCH STRATEGY STATEMENT 171 ANNEX 4: GLOSSARY OF KEY DEFINITIONS AND TERMS 178 ANNEX 5: LIST OF ABBREVIATIONS 180 ANNEX 6: LIST OF FIGURES AND TABLES 181 ANNEX 7: WORKING GROUP MEMBERS 182 ANNEX 8: ACKNOWLEDGEMENTS 5 6 Foreword In the twenty-first century, we face extreme global challenges like climate change, population growth and widespread pollution, as well as contending with major change brought about by technological advances. As a result, science advice has consistently come to be regarded as a vital element in the overall evidence considered by policymakers. At the same time, policymakers are striving to make sense of scientific evidence that is most certainly complex, multifaceted and, quite often, at the very limits of what is known scientifically. It is against this backdrop that the European Group of Chief Scientific Advisers (GCSA) decided to address the question of how to provide good science advice to European Commission policymakers. SAPEA was delighted to be asked to undertake a comprehensive evidence review, which informs the Scientific Opinion of the GCSA. Both reports are published simultaneously. Making sense of science for policy under conditions of complexity and uncertainty is the sixth Evidence Review Report to be published by the SAPEA consortium, an integral part of the European Scientific Advice Mechanism. In drafting this report, SAPEA formed an outstanding working group of European experts from a variety of disciplines, backgrounds and countries. All were determined to bring together the many schools of thought on the topic, as well as their extensive practical experience in science advice, and to reach a position of consensus. Their success in this endeavour is reflected in this Evidence Review Report, which highlights the unique role played by science in effective policymaking. We are confident that the report and its summaries will be useful to policymakers in Europe, and beyond. We would like to thank everyone involved, and express our sincere gratitude to those who have contributed directly to the report, above all, the members of the Working Group and its excellent Chair, Professor Ortwin Renn. Professor Sierd Cloetingh Chair of the SAPEA Board, 2018-2019 President of Academia Europaea, 2014-2020 (Lead Academy for Making sense of science for policy under conditions of complexity and uncertainty) 7 8 Key to the report All terms underlined are defined in Annex 4. All abbreviations used are listed in Annex 5. 9 Preface Making sense of science for policy is an unusual title for an evidence review report. The term ‘sense-making’ is clearly related to interpretation and cannot be covered without reference to individual or social judgements. In short, what makes sense to one person may not make any sense at all to another. While there are, in each society, shared understandings of what certain phenomena mean, there is no universal arbiter who would be able to distinguish between ‘correct’ or ‘incorrect’ sense-making. What is more, the nature of what science can offer to policymakers depends on the basic understanding and shared concepts of mandate, validity, reliability and relevance of scientific statements in the respective policy arena.