Bibliometric analysis of the research performance of the JRC under the Euratom Research and Training Programme (2007-2015) Penultimate draft Authors: Koen Jonkers, Juan Carlos Del Rio 2016 0 Table of Contents Foreword ...................................................................................................................... 2 Acknowledgments ......................................................................................................... 3 Executive summary ....................................................................................................... 4 1. Introduction ........................................................................................................... 6 2. Methodological reflections........................................................................................ 8 2.1 Field definition .......................................................................................... 8 2.2 Normalising name variants ......................................................................... 9 2.3 Time frame .............................................................................................. 9 2.4 Comparator organisations .......................................................................... 9 2.5 Reported metrics and research performance indicators ................................ 11 3. Production of (high-impact) publications in Nuclear Science and Technology research 12 3.1 Task description ...................................................................................... 12 3.2 Methodology........................................................................................... 12 3.3 Results .................................................................................................. 13 3.3.1 Analysis of the percentage of highly cited JRC publications per year ...................... 13 3.3.2 Comparison of the JRC NST performance with Group 1 comparator organisations .. 17 3.3.3 Comparison of the JRC NST performance with Group 2 comparator organisations .. 21 4. Discussion and conclusions .................................................................................... 25 Literature References .................................................................................................. 27 List of Figures ............................................................................................................. 28 List of Tables .............................................................................................................. 29 Annex 1 – Journal based Subfield categories ................................................................. 30 Annex 2 - Annual Evolution of publication output for the comparator organisations ........... 32 Annex 3 – Top 10 most highly cited JRC NST publications .............................................. 33 Annex 4 – Description of comparator organisations ........................................................ 40 Group 1 comparators ................................................................................................. 41 Group 2 comparators ................................................................................................. 55 Annex 5 – JRC and comparators for all nuclear research fields ........................................ 65 1 Foreword This report was requested in support of an interim evaluation of the JRC nuclear activities under the Euratom Research and Training Programme to be carried out with the assistance of independent experts by 31 May 2017. The current version of the report is a pen-ultimate draft which still requires registration in PUBSY. It was decided to submit it in this form to allow for potential further fine-tuning in the light of comments, if any, from the independent experts 2 Acknowledgments Support in the data collection was provided by Juan Carlos del Rio and Peter Fako. Juan Carlos del Rio also drafted the annexes. Peter Fako conducted the analysis for table 4. Said Abousahl provided input / validation for the selection of comparator organisations in the field of Nuclear Science and Technology. The report has benefited from internal reviews and comments by Pieter van Nes, Athina Karvounaraki and Xabier Goenaga. In order to check for appropriateness, quality and consistency of the analysis an outside bibliometric expert was contracted: Professor Robert Tijssen of the Centre for Science and Technology Studies of Leiden University. He validated the methodology, reviewed the document and provided additional input during a meeting in Brussels. 3 Executive summary This report provides a bibliometric assessment of the JRC's Nuclear Science and Technology (NST) research activities. It complements a report drafted by Thomson Reuters used as an input for the expert evaluation of JRC research performance under FP7. The current report aims to inform the review of the JRC's activities in this field by a panel of independent experts in the framework of an interim evaluation of the JRC nuclear activities under the Euratom Research and Training Programme. The report deploys widely accepted publication and citation based impact metrics to analyse the JRC's output and scientific impact in this field. These metrics are derived from Thomson Reuter's InCites platform. The JRC produces 1,023 publications in the broad nuclear research field. This analysis focuses on the 774 publications that are made in the subfield of Nuclear Science and Technology. This subfield is considered representative for JRC nuclear research activities. In terms of scientific NST publication output, the JRC is a medium sized actor. The JRC produces roughly 50 % more highly cited publications than the world average. In terms of normalised impact (SNCI) it is at the world average, which implies that a considerable share of JRC publications is cited below the world average. Further analysis of the JRC NST publication output reveals that around 70 % of the high- impact publications are the result of international co-publications. In almost 50 % of the high-impact JRC NST publications, a JRC researcher is the corresponding author. This indicates that high-impact JRC NST publications are to a significant extent the outcome of substantial JRC research activities. The report also compares the JRC publication output and impact with organisations active in nuclear research. The reader will find information for all 20 organisations in the report. The discussion focuses on organisations which share important elements of the nature of the JRC and have both a comparable size and a comparable share of NST activities in their total publication portfolio. The JRC’s share of highly cited publications compares well to the world average and the comparator organisations. The Paul Scherrer Institute in Switzerland and the VTT in Finland have a similar share of top 10 % most highly cited publications at a rate significantly (around 50 %) above the world average. Their Normalised Citation Impact is around 10-15 percentage points above the JRC score. In the United States Argonne National Lab and the Los Alamos National Laboratory produce a similar number of NST publications with a high scientific impact as the JRC. 4 5 1. Introduction The Seventh Framework programme for Research and Technological Development 2007- 2013 (FP7) represented a substantial investment and commitment by the European Commission (EC) to meet one of its top policy aim of enhancing and improving research in Europe. As the European Commission's science and knowledge service, the Joint Research Centre's mission is part of the Framework "to support EU policies with independent evidence throughout the whole policy cycle. Its work has a direct impact on the lives of citizens by contributing with its research outcomes to a healthy and safe environment, secure energy supplies, sustainable mobility and consumer health and safety." In 2014, the JRC proceeded with an evaluation of its research performance as part of a broader effort by the EC to evaluate and monitor the outcome of the FP7. For this purpose Thomson Reuters, an international media conglomerate and data provider prepared a data analytical report1 answering a list of open questions that were designed to measure the quantity and quality of JRC research outputs. Traditional ‘bibliometric’ data (i.e. publication counts, scientific references to publications (‘citations’), and author affiliations) were extracted and complemented with measures of scientific impact to provide a clearer picture of the international comparative state of research at the JRC. Designed to address the full spectrum of JRC activities, Thomson Reuters' assessment did not provide detailed information on the output and impact of JRC research in the field of Nuclear Science and Technology (NST). The ex post FP7 evaluation report noticed this lack of detail. Therefore the current account is required in support of the interim evaluation of the JRC nuclear activities under the Euratom Research and Training Programme to be carried out with the assistance of independent experts by 31 May 2017. In the nuclear field the JRCs mission is to carry out the Commission's research and training tasks as foreseen under the Treaty on the European Community for Atomic Energy (Euratom) signed in 1957. Article 4 of this Treaty made the Commission responsible “for promoting and facilitating nuclear research in the Member States and complementing it by carrying out a Community research and training programme”, whilst Article 8 foresaw the establishment