• Abstract and Figures
• Public Full-text

Oikos 000: 001–007, 2015 doi: 10.1111/oik.02956 © 2015 Th e Author. Oikos © 2015 Nordic Society Oikos Subject Editor: Dries Bonte. Editor-in-Chief: Dustin Marshall. Accepted 16 August 2015 Will technology trample peer review in ecology? Ongoing issues and potential solutions Pedro R. Peres-Neto P. R. Peres-Neto ([email protected]), Canada Research Chair in Spatial Modelling and Biodiversity, D é pt des Sciences Biologiques, Univ. du Qu é bec à Montr é al, CP 8888, Succ. Centre Ville, Montreal, QC, H3C 3P8, Canada. Th e classical view of peer review is that it is our primary process for assessing and judging whether research results should be published in a scholarly journal. However, the increased pressure to publish and technological developments are transform- ing peer review such that it is becoming a system that judges where work is published rather than whether the research is publishable (a ‘ where rather than if ’ process). Ecology is a fi eld in which publication numbers puts a particular pressure on the review system. In this forum piece, I summarize the issues with the current publication system and discuss how tech- nology is changing it, while suggesting solutions for important prior and ongoing issues with the peer review system. Th e view explored here is that technological developments (e.g. ease of creating journals, internet sites, storage, data generation, sharing of data and analytical code) will not eliminate peer review per se but will allow for a new set of parameters in which ethics and the optimal use of public funding will play a vital role in the evolution of the review process. “ Every scientifi cally minded person agrees with those who 1997, Lee 2012). A study in which a manuscript with eight hold the opinion that in the appraisement or estimate of deliberate errors was sent to 420 experts showed that the the value of an intellectual performance, merit, as acknowl- average number of detected errors was approximately two edged by one or more class or classes of critics, should be the per reviewer and that 16% of the reviewers did not fi nd any deciding factor. ” (Emch 1937) errors (Rothwell and Martyn 2000). Using fake authorship and contact information, Peters and Ceci (1982) submitted Does Emch ’ s view still hold today? papers that had already been published by prestigious research groups to the original journals in which they had been pub- Th e traditional peer review system has certainly played lished. Th e result was that eight papers were rejected for poor a major role in improving the quality of research papers quality (only three were detected as having already been prior to publication. In many fi elds, researchers are com- published, and one was accepted). Th ere is evidence, how- pelled to collect new data, run new experiments, reanalyze ever, showing that researchers understand that peer review their data using diff erent methods, and review their ratio- improves their papers (Mulligan et al. 2012). One conclu- nales, logic, views and even philosophy to comply with the sion here is that although articles seem to be improved by review process and have their research published. In essence, peer review, invalid articles are not necessarily being rejected classical peer review should “ ensure that the valid article and that there is a strong “ lottery ” (chance) component in is accepted, the messy article is improved, and the invalid the publication system (Neff and Olden 2006, Th urner and article rejected ” (Bornmann and Daniel 2010). Most, if Hanel 2011). In fact, diff erent peer review strategies are pos- not all, of us have obviously benefi tted from having our sible and can lead to diff erent results and tradeoff s among research be improved by the peer review system and classical quality, speed of publication, reviewers ’ eff ort and authors ’ peer review as brought and still brings genuinely important impact (Allesina 2012). benefi ts (Aarssen 2012). However, classical peer review has Th e notion I wish to explore here is that current publi- certainly been criticized on the basis that, although it is per- cation behavior, the economics of the publishing industry ceived as a central component of science, there is no evidence and the increasing numbers of articles (more scientists every for whether it works (Smith 2010). Perceived issues with the year) together with technological developments (e.g. ease of peer review system include, among many others, high costs creating journals, sites, storage) are transforming peer review (administration and academic time), delayed research (time such that it is becoming a system that judges where research taken to fi nd appropriate and willing reviewers and review is published rather than one that judges whether research time), nepotism, abuse (stolen ideas), bias (nationality, gen- is publishable. Currently, more than 1.3 million papers are der, language, reputation) and chance (Wenner å s and Wold published in peer-reviewed journals every year (Bj ö rk et al. EV-1 2009), and it is unlikely that our current peer review sys- suff er from blind spots and chance (the lottery syndrome). In tem will ever improve on current issues. Th e ‘ where ’ factor my experience, diff erent philosophies are applied by diff erent (i.e. impact factor and perceived importance of the journal) journals regarding rejection strategies. Some journals desk- will likely continue to play an important role (e.g. judg- reject (i.e. without sending for peer review) high proportion ing the merit of the research and the researchers involved; of papers (20% to 50%) whereas others rarely desk-reject. Mingers and Lipitakis 2013). In my view, however, under- Two or three reviewers generally evaluate a manuscript and standing and accepting the fact that the current peer review the tendency is for journals to almost automatically reject norm judges ‘ where rather than if ’ will stimulate a debate on it if one review is negative (Bornmann and Daniel 2010). how to change the focus from an ‘ elitist model ’ (where the On the one hand, as editors, we need to respect the opinion work is published) to one that improves the overall quality of reviewers, but on the other hand, we need to be criti- of published work and research. Th is shift in focus would cal about what is relevant to the fi eld and how to improve not confl ict with the existing norms that improve the quality manuscripts. Given the current publication space, it is likely of manuscripts but perhaps confl ict with the ways in which that one negative review will continue to serve as a major many academic institutions evaluate merit. rejection criterion. Another issue likely contributing to desk- Th e views regarding the roles of peer review are evolving, rejection is that authors often do not strive to understand but slowly. Some (e.g. Frishauf 2008) have even predicted the culture of the journal (scope, types of papers, research that peer review as we know it will disappear. In my opinion, directions of recently published papers) prior to submis- there are three major issues stalling the operationalization of sion. Conversely, many journals do not provide a clear set new peer review norms that would potentially better serve of descriptors regarding their scope and the nature of papers science. Th e fi rst is awareness of the current peer review and that will be highly appraised by their audience, reviewers and publication system and how technology is aff ecting them. editors. Th e second issue is the continual perception that peer review My understanding of the current publication trend is serves as a gatekeeper to ensure that invalid and inconse- that the ‘ minimum publishable unit ’ has become quite low, quential research is not published. Th e third issue is how at least in ecology and other biological sciences; that is, a future technological developments may help consolidate large percentage of papers that were fi nalized and submitted journals in the same research area. Reducing the number of are eventually published. Th is is the current trend because journals may eventually lower the emphasis on novel and technology allows new journals (especially online ones) to be sexy research as avenues for success and redirect attention relatively easily created, thus serving as venues for the papers to the overall quality and acceptance of the research results that are looking or waiting for a home. One interesting among peers. My primary goal in this piece is to explore trend is that there are journals today with acceptance rates these three notions (see Table 1 for a summary of the issues that would seem quite high in relation to their impact. For tackled here and their potential solutions). Although some of example, PLoS ONE, an open-access journal, accepts 70% the issues summarized here may be known by some research- of submitted papers and yet has a respectable 3.5 impact ers, my discussions with colleagues and students across the factor. Th e view shared by many open-access journals (e.g. biological sciences these past three years have revealed that BMJ open, Frontiers, BMC, PLoS ONE, PeerJ) is that they a large number of them are not critically informed of many do not equate ‘ perceived importance ’ with ‘ perceived nov- of the ongoing issues, and the majority of them have not elty ’ as a criterion for acceptance (see Walker and Rocha da thought about or foreseen many of the potential ramifi ca- Silva 2015 for a survey on emerging trends in peer review). tions of future technological advancements on peer review. As such, they are often defi ned as ‘ neutral impact ’ or ‘ non- In many instances, I certainly play the role of devil ’ s advo- selective ’ review as reviewers are requested to focus ‘ on cate in the sense of providing a provocative view either that I objective criteria of the soundness and validity of the work do not fully share or for which I have not yet foreseen all presented ’ (Frontiers ’ review guidelines).

Load more

  7

Copy Link