The Core Inefficiency of Peer Review and a Potential Solution

article The Core Inefficiency of Peer Review and a Potential Solution

Timothy H. Vines

Time is one of the most important commodi- criteria, such that (at any given journal) few able reviewers. Either of these tasks requires ties in scientific research, and yet so much of it papers are rejected because of issues with the a considerable span of undivided attention. If is spent on peripheral and sometimes unneces- science itself. Another phenomenon exacer- the paper does go out for review, it is evalu- sary activities. One of the most time consum- bates the problem of rejections for novelty or ated by two, three, or even four reviewers over ing aspects of science is making it public. Even scope: there is relentless pressure on academ- several weeks, with each reviewer spending once a paper is written, the process of finding ics to publish their work in the most pres- a median of five hours on their evaluation a journal that is willing to publish it can take tigious journal possible. This pressure stems (Ware 2008). Once the reviews are received, months or years. from increased competition for faculty posi- the editor writes their decision. All of these tions and funding, and it encourages specula- activities impose a time cost on the scientific The dual role of journals tive submissions to high profile journals on community, typically on the most productive the chance that the paper will be accepted. If or community minded researchers. Much of the pain in publishing science stems the paper gets rejected, the next logical step A round of peer review also carries a finan- from the dual purpose of scientific journals. is submission to the second most prestigious cial cost. Every submission must be checked First, journals exist to conduct prepublica- journal, and so on down the cascade. by the editorial office staff to ensure that it tion peer review. Peer review aims to iden- The net result is that journals receive 3–5 meets journal guidelines. The office normally tify and reject papers with fundamental flaws times more submissions than they are able to runs the review process as well, which entails while offering suggestions on how to improve publish. For a typical mid-range subscription checking reviewer identities and emails, send- the remainder. Papers that are improved to journal, 30% of papers are rejected prior to ing review requests, and reminding over- meet the journal’s standards are accepted for review, and 30% are rejected after assessment due reviewers. Many larger journals employ publication, made presentable through the by external reviewers. A significant propor- managing editors to oversee the process and typesetting process, and disseminated to the tion of papers (almost 25% even among ten- ensure consistency in editorial decisions. community. The second purpose of jour- ured faculty; Cassey and Blackburn 2003; Editorial office staffs are paid professionals, nals is filtration, whereby each journal has 2004), thus end up being submitted to two and in the course of a typical 6–10 week review its own scope (i.e., subject area) and qual- or more journals before being accepted for process they spend about eight hours on each ity threshold; the latter is the answer to the publication. paper (T. Vines unpubl.). Some papers come question “what is the weakest or most bor- back in as resubmissions or as revisions, and ing paper we would accept for publication?” The costs of peer review these require additional staff time. These edi- The filtration function of journals enables torial office costs are at the heart of why sci- scientists to identify the most relevant and There is a high cost to reviewing and re- ence publishing seems so expensive, as they most important papers that pertain to their reviewing papers until they find somewhere are incurred for every paper that gets submit- research. they fit. First, every submission must be eval- ted but only recouped for those articles that The unfortunate side effect of filtration uated by an editor. The editor must either get accepted (either as Open Access fees or is that most editorial decisions are based write a decision letter giving the paper an subscription fees). The costs scale with the on how well a paper matches the journal’s immediate rejection, or choose six to ten suit- inverse of the journal’s acceptance rate. Very

LOB-15-0010(10022).indd 36 5/11/15 6:59 PM selective journals assess 10 or more papers table 1. The current Axios editorial board for the field of ecology and a partial list of journals that have before accepting one for publication, and in agreed to look at Axios referrals. the process they spend thousands of dollars Ecology Editorial Board Ecology Target Journals on salaries and related expenses. Stefano Allesina American Journal of Botany Independent peer review: the process Julia Baum American Naturalist Michael Bonsall Animal Biotelemetry Given the high costs in both researcher time C. Titus Brown Annals of Botany ( > 16 hours) and money (~ $300) for just Lauren Buckley AoB PLANTS one round of peer review, we must look for ways in which we can fulfill the dual roles Yvonne Buckley Applications in Plant Sciences of journals while minimizing the number of Jarrett Byrnes Avian Conservation and Ecology rounds of review. One potential solution is Will Cornwell Biological Invasions independent peer review. In the Axios version Melania Cristescu Biological Journal of the Linnean Society of this process (see http://axiosreview.org), Greg Crutsinger BMC Biology the paper undergoes a standard round of Brant Faircloth BMC Ecology review, but the editor is part of an independ- Jeremy Fox BMC Plant Biology ent editorial board and not a journal. The process is as follows. Once the paper Kevin Gaston Botany is written and ready for peer review, authors Nicholas Gotelli Canadian Journal of Fisheries and Aquatic Sciences submit their manuscript to Axios. The W. Stan Harpole Canadian Journal of Microbiology authors also provide a ranked list of four Carlos Herrera Canadian Journal of Zoology target journals where they think it could be Randall Hughes Climate Change Responses published. The paper is assigned to an edi- Ari Jumpponen Ecological Monographs tor, and the editor can either decide that it Bart Kempenaers Ecology and Evolution is not ready for peer review (and return it to the authors), or select suitable reviewers. The Jeremy Kerr Ecology Letters editor can also ask the authors to revise their Katia Koelle eLife target journal list. Nathan Kraft Evolutionary Ecology The remaining papers are then sent out Brian Langerhans Frontiers in Ecology and the Environment for review. The reviewers are asked to com- Margie Mayfield Frontiers in Zoology ment on (a) the novelty of the paper relative Luc de Meester International Journal of Plant Sciences to recent articles in the same area, (b) its suitability for each of the four target jour- Ryan Norris Journal of Biological Research – Thessaloniki nals, and (c) the overall technical merits and Camille Parmesan Journal of Ecology weaknesses of the work. Once two or three Rod Peakall Journal of Fish Biology reviews are returned, the paper is returned to Eric Seabloom Molecular Ecology the editor for their decision. Depending on Diane Srivastava Molecular Ecology Resources the comments from the reviewers, the editor Jonathan Shurin PeerJ may, for example, decide that it is not suitable Matthew Symonds PLoS Biology for either of the top two target journals but that the paper seems like a good fit for target Leho Tedersoo PLoS ONE journals #3 and #4. Mark Urban Systematic Botany With the authors’ permission, Axios Jana Vamosi Theoretical Biology and Medical Modelling then sends a referral to journal #3. A refer- Koen Verhoeven Theoretical Ecology ral is essentially a very detailed presubmission Hillary Young Theoretical Population Biology enquiry. It contains the manuscript itself, reviews from two or three expert reviewers, and the identity of those reviewers. The jour- submit it to journal #3 by their regular sub- reviewers, and send the new version back to nal can then decide whether or not it would mission route. Axios. The editor then checks over the paper like the paper to be submitted. If not, a refer- Of course, the Axios reviewers may have and the response, and makes a decision on ral is sent to target journal #4. If journal #3 is identified a number of substantial problems where the paper can be referred. The referral keen on the manuscript, the authors are asked with the paper, to the extent that none of the process then proceeds as described above. to revise their paper in response to the inde- target journals are likely to request submis- For independent peer review to be suc- pendent round of review, put into journal #3 sion. In this situation the authors are asked cessful, the journal’s Editor in Chief must format, prepare a response to reviewers, and to revise their paper, prepare a response to feel confident in the opinions provided by

may 2015 37

LOB-15-0010(10022).indd 37 5/11/15 6:59 PM the reviewers and editor at the independent weeks (much like a typical journal), and the want them, journals are (by the same token) review organization. It is therefore critical referral stage an extra 2–10 days. Excluding receiving papers they actually want. Unsuit- that the editors at the review organization be the time the authors spend revising the paper, able submissions are steered elsewhere, either respected researchers in their field, preferably we average three months between submission by the Axios editor or at the referral stage. with editorial experience at one or more of the to Axios and publication at the journal. The net effect is that the journal’s acceptance target journals. Table 1 lists the Axios Review As described above, administering a round rate rises. Many rounds of review that would editorial board for the field of ecology, along of peer review involves salary costs for the end in rejection are eliminated, particularly with a subset of the journals that have agreed editorial office staff, and Axios is no different rejections on grounds of fit. The reduction to look at referrals from Axios. in this regard. To cover these costs we charge in the number of rounds of review before a When the review process at the independ- authors $250 USD for the use of our service; paper is accepted also makes life much easier ent review organization is as rigorous as a regu- this amount is sufficient to cover expenses for the reviewer community. lar journal round of review, the referral provides and leave enough spare to deal with unex- Ultimately, independent peer review roughly the same information as a journal pected events. The service is free to journals, might become the “normal” way that sci- regularly uses when making editorial decisions. although some Open Access titles (e.g., those ence is evaluated prior to publication. Papers If the journal does want a referred paper, the from the BMC series) deduct our fee from would typically receive one or two rounds of paper is then something like a resubmission: their Article Processing Charge. peer review (one independent, potentially a the journal has seen comments from external second at the journal), and the current bur- reviewers and it is still keen to see a new ver- Independent peer review: den on the reviewer and editor communities sion. The version submitted to the journal is dramatically reduced. Most importantly, the the benefits thus very unlikely to be rejected on the grounds journals’ dual roles of improvement and filtra- of novelty or scope, and instead the decision The Axios approach to independent peer tion are maintained, while the time currently mostly rests on the quality of the authors’ revi- review clearly benefits individual authors, as spent on unnecessary rejections for scope and sions. The rejection rates for resubmissions are they are able to aim their manuscript at both novelty can be redirected towards more pro- correspondingly much lower (20–35%), even high profile journals and safer choices without ductive activities. at higher profile journals. the hassle of submitting to each in succes- The effectiveness of the Axios approach sion. While all journal submissions run the References is borne out in the acceptance rate of their risk of being rejected, some (generally senior) Cassey, P., and T. M. Blackburn. 2003. Publica- referred papers. Once submitted to a jour- researchers are very proficient at targeting their tion rejection among ecologists. Trends Ecol. nal, fully 80% of manuscripts are accepted papers to the right outlets. However, early Evol. 18: 375–376. doi:10.1016/S0169- for publication. Moreover, because the review career scientists are responsible for writing 5347(03)00160-5 process at the independent board is so simi- and submitting many papers, and the intense Cassey, P., and T. M. Blackburn. 2004. Publica- lar to the standard journal review process competition for funding and jobs means they tion and rejection among successful ecologists. BioScience 54: 234–239. doi:10.1641/0006- that journals are often willing to accept suit- need to “try their luck” at higher profile jour- 3568(2004)054 able papers without sending them back out nals, even over the objections of their more Ware, M. 2008. Peer review: Benefits, perceptions for external review. Currently, about 50% of senior colleagues. It is these researchers that and alternatives. Publishing Research Consor- papers referred by Axios are accepted with- have the most to gain by using Axios. tium. Available from http://www.publishing out further review. The process is also rela- Journals also benefit. Because authors research.net/documents/PRCsummary4Ware tively quick, in that the review stage takes 4–7 are submitting their papers to journals that final.pdf

38 may 2015

LOB-15-0010(10022).indd 38 5/11/15 6:59 PM