Publish Peer Reviews Jessica K
Total Page：16
File Type：pdf, Size：1020Kb
COMMENT HISTORY Seventy-five years TECHNOLOGY Why are PUBLISHING Preprints — what is OBITUARY Burton Richter, since physicist Schrödinger governments in favour of good for science is good for charm-quark Nobel laureate, made waves in biology p.548 digital insecurity? p.550 the public p.553 remembered p.554 ILLUSTRATION BY DAVID PARKINS DAVID BY ILLUSTRATION Publish peer reviews Jessica K. Polka and colleagues call on journals to sign a pledge to make reviewers’ anonymous comments part of the official scientific record. ong shrouded in secrecy, the contents (the latter is owned by Springer Nature) have in Chevy Chase, Maryland — and ASAPbio, of peer review are coming into the been doing so for even longer (see ‘Revealing a non-profit organization that encourages open. In the past decade, outlets peer review’). Last year, the organizers of innovation in life-sciences publishing. We Lsuch as eLife, F1000Research, Royal Society Peer Review Week embraced the topic in a are convinced that publishing referee reports Open Science, Annals of Anatomy, Nature broader discussion of transparency. would better inform authors and readers, Communications, PeerJ and EMBO Press We are representatives of two biomedical improve review practices and boost trust in have begun to publish referee reports. Pub- funders — the UK Wellcome Trust and the science. Right now, less than 3% of scientific lishers including Copernicus, BMJ and BMC Howard Hughes Medical Institute (HHMI) journals allow peer reviews to be ©2018 Spri nger Nature Li mited. All ri ghts reserved. 30 AUGUST 2018 | VOL 560 | NATURE | 545 COMMENT published (see go.nature.com/2weh6vn). example. And the public would not be asked CAPITALSREVEALING OF To increase these numbers, our organiza- to place its faith in hidden assessments. tions held a meeting in February this year Studies of published peer reviews are PEERCULTIVATION REVIEW of around 90 invitees from the life sciences, small and often also involve open identi- predominantly from North America and ties or other innovations, making effects Journal editors have long consulted Europe. Scientific authors, reviewers and hard to ascertain. Nonetheless, evidence so referees to select and improve papers. readers participated, along with journal far suggests that the scientific community e focus has shifted to sharing them. editors and leaders of granting agencies. We finds published reports valuable. At The took care to include conservative voices, but EMBO Journal, peer-review files receive the nature of the meeting attracted people about 10% of the hits the papers themselves 1700 ready for change. The ideas in this article do4. A pilot by the publisher Elsevier found were honed at that event, with later assistance that one-third of its website visitors accessed 1750s: The UK Royal Society establishes a committee to vote on from HHMI president Erin O’Shea; molecu- peer-review reports, and several editors 1800 what is published in its journal, lar biologist Needhi Bhalla at the University said they used published reports as instruc- Philosophical Transactions. of California, Santa Cruz; Kenneth Gibbs, tive examples for inexperienced reviewers director of postgraduate training at the (see go.nature.com/2oujfgv). Editors at the US National Institute of General Medical European Journal of Neuroscience, which 1890s: UK scientic societies debate 1900 and abandon the adoption of a Sciences; and researcher Tony Ross-Hellauer launched transparent review at the end of standardized referee system to curb at Know-Center in Graz, Austria. 2016, report that referees are writing better “veritable sewage thrown into the Attendees agreed that the current lack of reviews and returning them more promptly pure stream of science”. 1940 transparency around peer review does not (see go.nature.com/2oxgtyf). serve science, and several journals com- mitted to publishing reviews (although not BARRIERS, PERCEIVED AND REAL necessarily reviewers’ identities) and author So why is the practice still rare? There are rebuttals. Here, we invite more journals to several reasons — some inertial, some 1940s–1960s: Formal peer take up the cause. It is time for transparency conceptual. 1950 review comes to be considered the linchpin of science. Science, to become the norm. Some disciplines are more keen than Nature and the Journal of the others. Nature Communications found that, American Medical Association take DEFINING REVIEW up the practice. given a choice, authors (and reviewers) of The term ‘open review’ has many more than 70% of its evolution and ecology interpretations. ‘Open identities’ means submissions opted for published reports. 1960 disclosing reviewers’ names; ‘open reports’ The figure was less than 50% for submissions (also called transparent reviews or published in atomic, particle and theoretical physics5. 1970s: The term peer review peer review) means publishing the content One concern is that, even if public reviews becomes widely used. of reviews. Journals might offer one or the are anonymous, they might make reviewers other, neither or both1. reluctant to accept assignments or to criticize In a 2016 survey2, 59% of 3,062 respond- freely, because authors could resent criticism 1970 ents were in favour of open reports. Only and retaliate against their presumed review- 31% favoured open identities, which they ers. The BMJ found that publishing peer- feared could cause reviewers to weaken their review reports with reviewers’ names did not criticisms or could lead to retaliation from change the quality of the peer reviews, sug- authors. Here, we advocate for open reports gesting that reviewers were not intimidated6. 1980 as the default and for open identities to be What is more, authors read unsigned review- optional, not mandatory. ers’ reports during standard review anyway. WWW.PEERREVIEWCONGRESS.ORG The vast majority of scientists think that A bigger concern is that published reviews peer review is essential for vetting research might be used unfairly in subsequent evalu- papers3. The process gives authors con- ation of the authors for grants, jobs, awards structive feedback, offers editors insight or promotions. There are few data about 1990 and assures readers of the trustworthiness whether and how authors’ ethnicity, gender, 1989: Inaugural Peer Review of research. Generally, however, only edi- country of origin or institution affect the Congress organized to evaluate the process. It is held every four tors, authors and (sometimes) reviewers see evaluations of papers. Yet there is evidence years. referee reports. That enables several forms of of bias in scientific publishing. Women, abuse: referees might be superficial, rude or for example, are less likely to be first or last biased; authors might respond inadequately authors in high-profile journals, and are 2000 1999–2003: The BMJ decides to 7,8 disclose reviewers’ names after to reasonable criticism; editors might not less likely to be asked for peer reviews assessing eects in a randomized hold authors or reviewers to account; and (see also go.nature.com/2pzyvcw). And trial. The publisher BMC begins predatory publishers will charge fees without workplace evaluations of female profes- publishing signed reviewer reports. Atmospheric Chemistry and Physics providing quality review. sionals also show gender bias (see go.nature. promotes open discussion of Many benefits would accrue from pub- com/2ppat2k). So the concern is that indi- 2010 submissions. lishing peer reviews (see ‘Potential benefits viduals from under-represented minorities 2006–16: Several journals and of published review’). The scientific commu- could receive biased reviews. Assessors for platforms start publishing reviewer nity would learn from reviewers’ and editors’ funding, hiring and promotions could pay comments. They include Biology insights. Social scientists could collect data more attention to negative comments when Direct (2006), The EMBO Journal (2009), eLife (2011), F1000 Research (for example, on biases among reviewers authors are from under-represented groups (2012), PeerJ (2013) and Nature or the efficiency of error identification by or less-prestigious institutions. Some fields 2020 Communications (2016). reviewers) that might improve the process. are also more critical or competitive, which Early-career researchers could learn by might skew reviews. 546 | NATURE | VOL 560 | 30 AUGUST 2018 |© CORRECTED2018 Spri nger Nat u13re L iSEPTEMBERmited. All ri ghts r e2018served . ©2018 Spri nger Nature Li mited. All ri ghts reserved. COMMENT Making referee reports open could allow peer-review materials can take approximately manuscript-management platforms should more-effective research into how competi- 25 minutes per manuscript. This is obviously develop workflows that optimize and auto- tion and bias affect the process. Meanwhile, much less than the time spent coordinating, mate the process of publishing peer reviews, anyone participating in open peer review — conducting and assessing reviews, but is still reducing burdens on journal staff, authors or evaluating it after the fact — should be significant. Most publishing platforms are and reviewers. Indexing services, such as aware of this potential for unfairness. not set up to display, organize or assign digi- PubMed, should find ways to prominently Another risk is the ‘weaponization’ of tal object identifiers link peer reviews to the original paper. reviewer reports. Opponents of certain “It is time for (DOIs) to reviewer Appropriate infrastructure is already being types