Research Assessment in the Transition to Open Science 2019 EUA Open Science and Access Survey Results Bregt Saenen, Rita Morais, Vinciane Gaillard and Lidia Borrell-Damián October 2019 This publication is licensed under the Creative Commons Attribution-NonCommercial CC BY-NC This information may be freely used and copied for non-commercial purposes, provided that the source is acknowledged ( European University Association). European University Association asbl Avenue de l’Yser 24 Rue du Rhône 114 1040 Brussels Case postale 3174 Belgium 1211 Geneva 3, Switzerland +32 (0) 2 230 55 44 +41 22 552 02 96 www.eua.eu · [email protected] Table of Contents Foreword by Professor Paul Boyle 5 Foreword by Professor Jean-Pierre Finance and Professor Bernard Rentier 6 Acknowledgements 7 1. Introduction 8 1.1. 2019 survey and report 8 1.2. EUA activities on research assessment in the transition to Open Science 8 2. Survey methodology and participation 10 2.1. Survey methodology 10 2.2. Survey participation and profile of respondents 10 3. The current state of university approaches to research assessment 15 3.1. Autonomy to develop and implement research assessment 15 3.2. Organisation of research assessment approaches 17 3.3. Research assessment transparency 19 3.4. Academic activities and their evaluation for the purpose of research careers 20 3.4.1. Importance of academic activities for research careers 20 3.4.2. Evaluation of academic activities for research careers 22 3.4.3. Practices used to evaluate academic activities for research careers 25 4. Reviewing university approaches to research assessment 29 4.1. Influence of existing principles and guidelines 29 4.2. The future direction of university approaches to research assessment 30 4.3. Main reasons and objectives 31 4.4. Main barriers and difficulties 32 5. Concluding remarks 34 Annexes 36 Annex 1 – 2019 EUA Open Science and Access Survey on Research Assessment 36 Annex 2 – EUA Workshop Report: Research assessment for researcher recruitment and career progression 46 3 Research Assessment in the Transition to Open Science List of tables and figures Table 1 – Participation in EUA Open (Science and) Access surveys, 2014-19 10 Figure 1 – Number of respondents per country 11 Table 2 – Number of respondents per country 12 Figure 2 – EUA member respondents expressed as a percentage of EUA members in each country 13 Figure 3 – Respondents’ profile 14 Figure 4 – Respondents’ Full Time Equivalent (FTE) researchers 14 Figure 5 – Respondents with research assessment procedures in place or in development 15 Table 3 – Autonomy to develop and implement research assessment approaches 16 Figure 6 – The institutional level at which research assessment is primarily organised 17 Figure 7 – Personnel involved in developing research assessment approaches 18 Figure 8 – Transparency about research assessment for research careers 19 Figure 9 – Importance of academic activities for research careers 21 Figure 10 – Evaluation of academic activities for research careers 23 Figure 11 – Publication metrics used for research careers 25 Figure 12 – Metrics measuring academic attention and uptake used for research careers 26 Figure 13 – Altmetrics measuring social outreach used for research careers 27 Figure 14 – Influence of existing principles and guidelines on approaches to research assessment 29 Figure 15 – Main barriers and difficulties for reviewing approaches to research assessment 32 4 Foreword by Professor Paul Boyle The European University Association (EUA) is one of the leading actors in the transition to Open Science. It represents the independent voice of European universities, making sure the interests of the vibrant European research and innovation community are heard and considered. The Association has made a unique contribution to the Open Science debate. By carrying out regular university surveys and commissioning studies it has built a shared knowledge base on: Open Access policies for research publications and data, the financial cost of access to scholarly publications (Big Deals), research assessment practices, innovative publishing practices (such as Read and Publish agreements) and other key Open Science issues. This report presents the results of the 2019 EUA Open Science and Access Survey on Research Assessment. For the first time, it gathers and shares a comprehensive overview of research assessment approaches by European universities. Research assessment is a powerful tool for making the transition to Open Science a reality. Making evaluation practices more accurate, transparent and responsible will allow universities to establish best practice and work together for our academic community. A concerted approach uniting the main actors will be necessary to move forward. EUA will continue to engage its membership in this process and maintain a close relationship with its partners. The latter notably include the newly proposed European Commission, with Commissioner-designate Mariya Gabriel taking on the integrated education, research and innovation portfolio, and the newly elected European Parliament. As EUA Vice-President and Chair of the EUA Research Policy Working Group I look forward to working together with my colleagues on these issues. Building on the work that has already been done, including the results presented in this report, I am confident that EUA will remain at the forefront of the transition to Open Science. Professor Paul Boyle EUA Vice-President and Chair of the EUA Research Policy Working Group 5 Research Assessment in the Transition to Open Science Foreword by Professor Jean-Pierre Finance and Professor Bernard Rentier Open Science is a paradigm shift. Open publication, open access, open citations, open data, open source software, citizen science - in the same cooperative spirit, all these innovations revolutionize research by rejecting competition, even though many researchers still consider this inevitable. This new science approach seems likely to develop further and, in the long run, to become the norm. However, no matter how hard advocates strive, Open Science will never be achieved unless accompanied by a change in the way researchers are evaluated. Without this, no researcher, (and especially no early- stage researcher,) will take the proven risk of departing from the old principles that continue to paralyse scientific communications: publish as often as possible, in journals with the best possible reputation. Given these considerations, it was interesting to verify current European university practices. In particular, and among many other questions, we wanted to know whether evaluators still favour quantified approaches (such as the journal impact factor and its derivatives) or if they are developing a more qualitative approach in which the amount of scholarly production and publisher are no longer the only criteria used to determine the quality, or even of excellence, of a researcher’s work. To improve understanding of the current situation of research assessment practices at European universities, both in terms of researcher careers and research project evaluations, the EUA Expert Group Science 2.0/Open Science decided to investigate further. In May 2019, it took the initial step of organising a workshop on research assessment for researcher recruitment and career progression. Then it focused the annual EUA Open Access Survey on research assessment at universities. This survey reveals the beginning of a change, but it also indicates that there is still a long way to go before the principles that have become dogmas make room for at least partial consideration of the values of exchange, sharing and cooperation advocated by Open Science. We hope that it will be useful in helping institutions review evaluation criteria and in supporting researchers when it comes to demonstrating the need to reconsider their publication practices. We would like to warmly thank the EUA Secretariat staff involved in this investigation, Dr Rita Morais, Dr Bregt Saenen and Dr Vinciane Gaillard, and to extend a special mention to Dr Lidia Borell-Damián, former EUA Director of Research & Innovation, who has done a remarkable job of making EUA’s positions and actions clear and visible in order to achieve more fair, transparent and open research activity at European universities. Professor Jean-Pierre Finance Professor Bernard Rentier Chair of the EUA Expert Group Science 2.0/ Chair of the EUA Expert Subgroup on Research Open Science Assessment 6 Acknowledgements The authors would like to thank the members of the EUA Expert Subgroup on Research Assessment and the EUA Expert Group Science 2.0/Open Science for their valuable comments. We would particularly like to express our thanks to Professor Bernard Rentier, Chair of the Expert Subgroup, and Professor Jean- Pierre Finance, Chair of the Expert Group, for their support. A draft of this report was also reviewed by Dr Elizabeth (Lizzie) Gadd (Loughborough University, United Kingdom). Her much-appreciated feedback greatly contributed to improving the final version. Finally, the authors would like to thank our colleagues in the EUA Secretariat who provided input and helped design and edit this report. Special thanks go to Enora Bennetot Pruvot, Jessica Carter, Thomas Estermann, Sergiu-Matei Lucaci, Inès Mezher, Lennart Stoy and Christel Vacelet. Dr Bregt Saenen Research & Innovation Policy and Project Officer Dr Rita Morais Research & Innovation Adviser Dr Vinciane Gaillard Deputy Director, Research & Innovation Unit Dr Lidia Borrell-Damián Director,