Infozine Special Issue 1
Total Page:16
File Type:pdf, Size:1020Kb
.. InfozineInfozine S S 1 1 Special Issue 1 | December 12, 2016 The Magazine for Users of Scientific Information Metrics in Research For better or worse? Editors: Jožica Dolenc Philippe Hünenberger Oliver Renn Infozine is published by the Chemistry | Biology | Pharmacy Information Center, ETH Zurich Imprint Infozine Special Issue S1 Metrics in Research – For better or worse? Editors: Prof. Dr. Philippe Hünenberger (Guest Editor) ETH Zürich, Laboratory of Physical Chemistry Vladimir-Prelog Weg 2 8093 Zürich, Switzerland Phone +41 44 632 55 03, [email protected] Dr. Jožica Dolenc ETH Zürich, Chemistry | Biology | Pharmacy Information Center Vladimir-Prelog Weg 10 8093 Zürich, Switzerland Phone +41 44 632 29 48, [email protected] Dr. Oliver Renn ETH Zürich, Chemistry | Biology | Pharmacy Information Center Vladimir-Prelog Weg 10 8093 Zürich, Switzerland Phone +41 44 632 29 64, [email protected] Copy-Editors: Oliver Renn, Philippe Hünenberger Layout: Oliver Renn Cover illustration: Oliver Renn, iPad Pro Drawing with the App Procreate and Apple Pencil Download and use of full text All contributions have been archived in ETH Zurich’s e-collection and can be retrieved at http://e-collection.library.ethz.ch. A PDF of the Special Issue can be retrieved from http://www.infozentrum.ethz.ch/en/downloads-icbp/brochures/. DOI numbers have been assigned by the ETH Library, DOI-Desk. Individual copyrights have been assigned to the contributions. Please use the hashtag #infozineS1metrics @infozentrum (Instagram, Facebook)) @icbpeth (twitter) when mentioning articles in social media. Infozine and its Special Issues are published by the Chemistry | Biology | Pharmacy Information Center, a function within the Department of Chemistry and Applied Biosciences and the Department of Biology at the ETH Zurich. Infozine is published as an English and German edition, Special Issues in English only. ISSN (English Edition) 2504-1851 ISSN (German Edition) 2504-1843 www.infozentrum.ethz.ch Metrics in Research Jožica Dolenc, Philippe Hünenberger, Oliver Renn Editorial Metrics in Research – For better or worse? If you are an academic researcher but decision-makers is precious, speed and weigh the above strengths and flaws of did not earn (yet) your Nobel prize or objectiveness are clearly two major this system, and whether one should your retirement, it is unlikely you assets of metrics-assisted decision- strive to reinforce it, to improve/refine never heard about research metrics. making. And let us not forget a third it, or to abolish it. These metrics aim at quantifying vari- psychological factor: the human brain Clearly, the debate is im- ous aspects of the research process, at (especially that of scientists and portant (maybe vital!) for the future of the level of individual researchers (e.g. managers!) is by construction fasci- academic research. For this reason, in h-index, altmetrics), scientific journals nated by numbers, and their strong this special issue of Infozine, we have (e.g. impact factors) or entire universi- power for classification and ration- collected 18 opinion statements con- ties/countries (e.g. rankings). Although alization. cerning the topic of “research metrics”. such “measurements” have existed in a There are, however, two ma- The potential contributors have been simple form for a long time, their jor downsides to metrics-based deci- invited with the goal of providing a widespread calculation was enabled by sion making. First, the reduction of a wide spectrum of opinions (supportive, the advent of the digital era (large complex entity A (university, scientist, moderate, or critical) and covering a amount of data available worldwide in project, journal, publication) into a sin- wide spectrum of perspectives (includ- a computer-compatible format). And in gle number NA representing quality is a ing those of professors, students, pub- this new era, what becomes technically projection from a high-dimensional lishers, editors, and metrics providers). possible will be done, and what is done space to a single number. Thus, it will This special issue is meant to provide a and appears to simplify our lives will always be reductionistic (incomplete, broad and unbiased spectrum of possi- be used. As a result, a rapidly growing simplistic, distortive, dehumanized), ble viewpoints and arguments on the number of statistics-based numerical and may even in some cases be entirely topic, with the idea to feed into the indices are nowadays fed into decision- off-topic. In fact, most current research thinking of the readers, and help them making processes. This is true in nearly metrics do not measure a scientific define lucidly their own position re- all aspects of society (politics, quality, but rather a scientific output or garding the issue. economy, education and private life), impact (i.e. only – and arguably – one The “metrics system” is de and in particular in research, where component of quality). Second, the facto already in place, and it is spon- metrics play an increasingly important systematic coupling of a reductionistic taneously self-reinforcing. You may role in determining positions, funding, index NA to decisions strongly influ- decide to actively support it, or to ac- awards, research programs, career encing A induces a feedback loop, in cept and do the best out of it, or to fight choices, reputations, etc… which the entity A will start to against it … but, as always, it is ex- In somewhat simplistic terms, optimize itself against NA rather than tremely unwise to let others decide for numerical indicators allow to simplify against quality in a broader sense. At you. the choice between two complex the extreme, this may result in a options A and B, associated with research community striving very quality indices NA and NB, in two ways. competitively for output and impact, First, the choice is immediate, as it and considering collaboration, diver- boils down to solving an inequality (if sity, creativity, curiosity, risk-taking, NA > NB, pick A, otherwise pick B, no education and ethics (definitely other need to dive into the painful components of scientific quality!) as complexity of options A and B). dispensable virtues. Second, the choice is objective (as long The “metrics system” is not a Citation: Dolenc J, Hünenberger P, Renn O. as the procedures to derive NA from A perspective for the future – it is already Editorial: Editorial and NB from B are deterministic and well-installed and gaining strength, be- Metrics in Research – For better or worse? identical, the comparison itself is cause it is fundamentally compatible Infozine 2016, Special Issue 1, 1, DOI: unbiased). In a society where public with the usual mechanisms and main- 10.3929/ethz -a-010748857 resources are tight, so that their stream values of a modern society in Copyright: J. Dolenc, P. Hünenberger, efficient use and fair distribution must the digital era. Yet, individual opinions O. Renn, CC BY NC ND 4.0 be justified, and where the time of diverge widely concerning how to Published: December 12, 2016 Infozine Special Issue No. 1 – 2016 1 Metrics in Research Contents 1 Editorial: Metrics in Research – For better or worse? Jožica Dolenc, Philippe Hünenberger Oliver Renn 3 A brief visual history of research metrics Oliver Renn, Jožica Dolenc, Joachim Schnabl 9 Bibliometry: The wizard of O’s Philippe Hünenberger, ETH Zürich 11 The grip of bibliometrics – A student perspective Matthias Tinzl, ETH Zürich 13 Honesty and transparency to taxpayers is the long-term fundament for stable university funding Wendelin J. Stark, ETH Zürich 15 Beyond metrics: Managing the performance of your work Charlie Rapple, Kudos 17 Scientific profiling instead of bibliometrics: Key performance indicators of the future Rafael Ball, ETH-Bibliothek 20 More knowledge, less numbers Carl Philipp Rosenau, ETH Zürich 21 Do we really need BIBLIO-metrics to evaluate individual researchers? Rüdiger Mutz, ETH Zürich 23 Using research metrics responsibly and effectively as a researcher Peter I. Darroch, Lisa H. Colledge, Elsevier B.V. 25 Metrics in research: More (valuable) questions than answers Urs Hugentobler, ETH Zürich 27 Publication of research results: Use and abuse Wilfred F. van Gunsteren, ETH Zürich 29 Wanted: Transparent algorithms, interpretation skills, common sense Eva E. Wille, Wiley-VCH 31 Impact factors, the h-index, and citation hype – Metrics in research from the point of view of a journal editor Renato Zenobi, ETH Zürich and Associate Editor, Analytical Chemistry 33 Rashomon or metrics in a publisher’s world Gabriella Karger, Karger Publishers 35 The impact factor and I: A love-hate relationship Jean-Christophe Leroux, ETH Zürich and Associate Editor, Journal of Controlled Release 37 Personal experiences bringing altmetrics to the academic market Ben McLeish, Altmetric 39 Fatally attracted by numbers? Oliver Renn, ETH Zürich 41 On computable numbers Gerd Folkers, Laura Folkers, ETH Zürich, Lunds Universitet 43 ScienceMatters – Single observation science publishing and linking observations to create an internet of science Lawrence Rajendran, University Zurich, and ScienceMatters Infozine Special Issue No. 1 – 2016 2 Metrics in Research Oliver Renn, Jožica Dolenc, Joachim Schnabl ETH Zürich A brief visual history of research metrics Research metrics started in chemistry Research metrics are relatively recent considering that scientific journals, his- torically the main basis for these met- rics, have been already established more than 350 years ago. The first re- search metrics introduced were biblio- metrics, i.e. they relied on the statisti- cal analysis of publications, books or journals (Figure 1). Among the meth- ods used for bibliometrics, citation analysis is the most important. The first citation analysis was executed by Gross and Gross in 1927 [1]. They counted citations in articles (manu- ally!) so as to produce a ranking of chemistry journals, aimed as a guide to help chemistry librarians decide which journals to subscribe to. It was a chemist, Eugene Gar- field, who suggested in 1955 [2] to sys- tematically count and analyze citations in the scientific literature.