Will Technology Trample Peer Review in Ecology? Ongoing Issues and Potential Solutions

Will technology trample peer review in ecology? Ongoing issues and potential solutions

Pedro R. Peres-Neto

P. R. Peres-Neto ([email protected]), Canada Research Chair in Spatial Modelling and Biodiversity, D é pt des Sciences Biologiques, Univ. du Qu é bec à Montr é al, CP 8888, Succ. Centre Ville, Montreal, QC, H3C 3P8, Canada.

Th e classical view of peer review is that it is our primary process for assessing and judging whether research results should be published in a scholarly journal. However, the increased pressure to publish and technological developments are transforming peer review such that it is becoming a system that judges where work is published rather than whether the research is publishable (a ‘ where rather than if ’ process). Ecology is a ﬁ eld in which publication numbers puts a particular pressure on the review system. In this forum piece, I summarize the issues with the current publication system and discuss how technology is changing it, while suggesting solutions for important prior and ongoing issues with the peer review system. Th e view explored here is that technological developments (e.g. ease of creating journals, internet sites, storage, data generation, sharing of data and analytical code) will not eliminate peer review per se but will allow for a new set of parameters in which ethics and the optimal use of public funding will play a vital role in the evolution of the review process.

“ Every scientifi cally minded person agrees with those who 1997, Lee 2012). A study in which a manuscript with eight hold the opinion that in the appraisement or estimate of deliberate errors was sent to 420 experts showed that the the value of an intellectual performance, merit, as acknowl- average number of detected errors was approximately two edged by one or more class or classes of critics, should be the per reviewer and that 16% of the reviewers did not fi nd any deciding factor. ” (Emch 1937) errors (Rothwell and Martyn 2000). Using fake authorship and contact information, Peters and Ceci (1982) submitted Does Emch’ s view still hold today? papers that had already been published by prestigious research groups to the original journals in which they had been pub- Th e traditional peer review system has certainly played lished. Th e result was that eight papers were rejected for poor a major role in improving the quality of research papers quality (only three were detected as having already been prior to publication. In many fi elds, researchers are com- published, and one was accepted). Th ere is evidence, how- pelled to collect new data, run new experiments, reanalyze ever, showing that researchers understand that peer review their data using diff erent methods, and review their ratio- improves their papers (Mulligan et al. 2012). One conclu- nales, logic, views and even philosophy to comply with the sion here is that although articles seem to be improved by review process and have their research published. In essence, peer review, invalid articles are not necessarily being rejected classical peer review should “ ensure that the valid article and that there is a strong “ lottery ” (chance) component in is accepted, the messy article is improved, and the invalid the publication system (Neff and Olden 2006, Th urner and article rejected ” (Bornmann and Daniel 2010). Most, if Hanel 2011). In fact, diff erent peer review strategies are pos- not all, of us have obviously benefi tted from having our sible and can lead to diff erent results and tradeoff s among research be improved by the peer review system and classical quality, speed of publication, reviewers ’ eff ort and authors ’ peer review as brought and still brings genuinely important impact (Allesina 2012). benefi ts (Aarssen 2012). However, classical peer review has Th e notion I wish to explore here is that current publi- certainly been criticized on the basis that, although it is per- cation behavior, the economics of the publishing industry ceived as a central component of science, there is no evidence and the increasing numbers of articles (more scientists every for whether it works (Smith 2010). Perceived issues with the year) together with technological developments (e.g. ease of peer review system include, among many others, high costs creating journals, sites, storage) are transforming peer review (administration and academic time), delayed research (time such that it is becoming a system that judges where research taken to fi nd appropriate and willing reviewers and review is published rather than one that judges whether research time), nepotism, abuse (stolen ideas), bias (nationality, gen- is publishable. Currently, more than 1.3 million papers are der, language, reputation) and chance (Wenner å s and Wold published in peer-reviewed journals every year (Bj ö rk et al.

EV-1 2009), and it is unlikely that our current peer review sys- suff er from blind spots and chance (the lottery syndrome). In tem will ever improve on current issues. Th e ‘ where ’ factor my experience, diff erent philosophies are applied by diff erent (i.e. impact factor and perceived importance of the journal) journals regarding rejection strategies. Some journals desk- will likely continue to play an important role (e.g. judg- reject (i.e. without sending for peer review) high proportion ing the merit of the research and the researchers involved; of papers (20% to 50%) whereas others rarely desk-reject. Mingers and Lipitakis 2013). In my view, however, under- Two or three reviewers generally evaluate a manuscript and standing and accepting the fact that the current peer review the tendency is for journals to almost automatically reject norm judges ‘ where rather than if ’ will stimulate a debate on it if one review is negative (Bornmann and Daniel 2010). how to change the focus from an ‘ elitist model’ (where the On the one hand, as editors, we need to respect the opinion work is published) to one that improves the overall quality of reviewers, but on the other hand, we need to be criti- of published work and research. Th is shift in focus would cal about what is relevant to the fi eld and how to improve not confl ict with the existing norms that improve the quality manuscripts. Given the current publication space, it is likely of manuscripts but perhaps confl ict with the ways in which that one negative review will continue to serve as a major many academic institutions evaluate merit. rejection criterion. Another issue likely contributing to desk- Th e views regarding the roles of peer review are evolving, rejection is that authors often do not strive to understand but slowly. Some (e.g. Frishauf 2008) have even predicted the culture of the journal (scope, types of papers, research that peer review as we know it will disappear. In my opinion, directions of recently published papers) prior to submis- there are three major issues stalling the operationalization of sion. Conversely, many journals do not provide a clear set new peer review norms that would potentially better serve of descriptors regarding their scope and the nature of papers science. Th e fi rst is awareness of the current peer review and that will be highly appraised by their audience, reviewers and publication system and how technology is aff ecting them. editors. Th e second issue is the continual perception that peer review My understanding of the current publication trend is serves as a gatekeeper to ensure that invalid and inconse- that the ‘ minimum publishable unit’ has become quite low, quential research is not published. Th e third issue is how at least in ecology and other biological sciences; that is, a future technological developments may help consolidate large percentage of papers that were fi nalized and submitted journals in the same research area. Reducing the number of are eventually published. Th is is the current trend because journals may eventually lower the emphasis on novel and technology allows new journals (especially online ones) to be sexy research as avenues for success and redirect attention relatively easily created, thus serving as venues for the papers to the overall quality and acceptance of the research results that are looking or waiting for a home. One interesting among peers. My primary goal in this piece is to explore trend is that there are journals today with acceptance rates these three notions (see Table 1 for a summary of the issues that would seem quite high in relation to their impact. For tackled here and their potential solutions). Although some of example, PLoS ONE, an open-access journal, accepts 70% the issues summarized here may be known by some research- of submitted papers and yet has a respectable 3.5 impact ers, my discussions with colleagues and students across the factor. Th e view shared by many open-access journals (e.g. biological sciences these past three years have revealed that BMJ open, Frontiers, BMC, PLoS ONE, PeerJ) is that they a large number of them are not critically informed of many do not equate ‘ perceived importance ’ with ‘ perceived nov- of the ongoing issues, and the majority of them have not elty’ as a criterion for acceptance (see Walker and Rocha da thought about or foreseen many of the potential ramifi ca- Silva 2015 for a survey on emerging trends in peer review). tions of future technological advancements on peer review. As such, they are often defi ned as ‘ neutral impact ’ or ‘ non- In many instances, I certainly play the role of devil ’ s advo- selective’ review as reviewers are requested to focus ‘ on cate in the sense of providing a provocative view either that I objective criteria of the soundness and validity of the work do not fully share or for which I have not yet foreseen all presented’ (Frontiers’ review guidelines). Despite these good possible ramifi cations. intentions, there is strong skepticism regarding open-access publishing (e.g. business model, lack of appropriate copy- Are some papers never published? The role of editing, citation levels, acceptance by peers, scientifi c rigor; technology in the publication business Bohannon 2013, Agrawal 2014). Th e bottom line here is that most papers will eventually be published. Th e important Many of us can certainly share a wealth of experiences regard- question, then, is whether we should weigh the quality of ing rejection by our fi rst (and often second and so on) choice published work based on the reliability of its information or of journal. For the most part, however, the manuscripts were on the success stamp it acquires based on where it has been eventually published in another ( ‘ just as good ’ ) journal. published. One can only hope that the former is the case. Reviewers and associate editors (I speak as a current board member of three journals) see a hefty number of papers that Why do good journals reject good papers in the are rejected by the particular journal for which they edit but age of technology? that are eventually published in another journal. Th e trend in which a large number of papers are rejected from the fi rst As with most businesses, publishing companies use the submitted journal (i.e. many top journals routinely reject rules of supply and demand. Th us, as more research is pro- ∼ 90% of submissions, e.g. Nature, Science, Ecology Letters) duced, the demand for new journals will increase. In the is currently well recognized. Given that editors from top past decade, there has been an explosion in the number of journals are faced with the reality of having to reject papers journals, especially in the online-only open-source segment. at high rates, it is likely that the system will continue to Some high-impact paperback journals (e.g. Ecography) have

EV-2 one system syndrome in the publication system search capability search system independent depositories, several independent depositories may the process down slow is made open access system, reduce quality of papers and academic institutions Administrative costs and control by costs and control by Administrative Citing different version will not reduce the lottery This Perception from peers, reduced Perception None foreseeable Too many papers published many Too are unlikely to adopt such Journals be stolen Ideas may start their own Companies may None foreseeable if the recognition are unlikely to adopt such Journals for users costs involved Potential

, ” elds papers published many Too eld and ” eld eld meat extenders “ in the review ’ nd where a manuscript nd where meat extenders cheating cheating ‘ “ ts ed solutions and their pros cons. eld, fi faster turnout times, improve faster turnout times, improve distribution, reduce review pressure, reduce reduce elitist practices improve visibility of the work improve and journal expectations distribution component, create a sense of community within a fi component, create a sense of community within a fi reviews understanding of the trends within a fi systems; public accountability new ideas are prior to peer-review, disseminated faster system, faster turnout times, reduce elitist practices best fi bid on what to publish bid on what Reduce the costs of publication, Promote well-guided philosophies None foreseeable Improve knowledge about the fi knowledge Improve Faster turnout times, improve turnout times, improve Faster Organization of information, better Reduce selection of reviewers, journals Faster Reduce eld of and researchers; improve improve and researchers; capacity search and research in the fi and research publication business search tools search date journals search tools search number of reviews are recorded and made public of Science (e.g., Peerage SelectedPapers.net) papers Convince journalsConvince academic institutions Convince within subfi Consolidate working Specialized publications, reports Interactive and personalizable Interactive Open-access journals, consoli- journalsConvince review time, reduce lottery Improve Convince journalsConvince review time, reduce lottery Improve Common public systems where Common public systems where s work Make reviews citable quality of reviewers, improve Reward ’ eld and expectations of journal especially online-only as a self-regulatory source system business model fi particular journals journal especially online-only novelty or perceived impact or perceived novelty journal especially online-only review Use a single consolidated open Improve awareness of current awareness Improve Understand the trends in Increase number of papers per Focus on information rather than on information rather Focus Increase number of papers per A credit system associated to Preprinting friendly review review systemsCentralize Public preprint serversRecognize the reviewer Open peer review depositories Publish without peer review quality is improved Public feedback, Liquid journals editingPost-publication of published Update versions Self-regulation of the publication publication and subscription costs about where their research their research about where can be published perceived novelty rather than rather novelty perceived merit peer review system Current business model, and Authors are often not realistic Authors lottery syndromeThe Increase number of papers per Table 1. A summary of the issues discussed in this paper, potential solutions to these issues, operationalization of the propos potential solutions to these issues, operationalization A summary of the issues discussed in this paper, 1. Table IssueElitist publication model: solution Potential Operationalization Pros Cons The tragedy of commons in the tragedy The papers published many Too capability search Improve and personalizable Interactive

EV-3 also recently opted to discontinue their paper-printed ver- often missed by their intended readership. Th ere have been sions to become online-only journals, and some new journals initiatives, such as the Faculty of 1000 (Ͻ http://f1000. (e.g. Methods in Ecology and Evolution) with high impact com/ Ͼ ), in which experts in the fi eld select and review were never published in a paper version. Th is ‘ online only ’ papers that they think are important regardless of the journal approach (not to be equated with open-source) has a num- in which they appeared. Th us, if future technological devel- ber of practical consequences, such as improving distribu- opments regarding information search capability could allow tion, reducing environmental costs related to printing, and one to search eff ectively through ‘ the jungle of publications’ , reducing publication time. One critical consequence of tech- why not consider having only a small number of low-cost nological advancements (e.g. electronic storage and reduced publishing repositories as a possibility? production time), however, is that the ‘ online-only ’ approach theoretically increases the number of papers publishable to an infi nite amount. Why is this not the case? It would seem that If technology allows no publication limitation, the desire of journals to maintain or improve their impact should all valid research be published? The ethics factors and the desire of publishing companies to increase of publicly funded research their odds of having high impact journals at higher costs are “ Th e evolution of new cultural norms can be guided by responsible. ethical imperatives ” Regardless of whether electronic or paperback journals (borrowed from Aarssen 2012, but in a diff erent context) establish a limit to the number of papers to be accepted, there is a current trend in which high-impact journals suggest that One possible view here (not one that I necessarily share in the research not deemed ‘ worth ’ publishing in their main all cases) is that all research that is ethically produced (i.e. journal be invited to be submitted to a sister journal (e.g. the that follows institutional protocols – e.g. animal care, ethical Journal of Evolutionary Biology now suggests submitting norms – and reports results in good faith) should eventually papers that they did not send out for review to Ecology and be published. Th is view certainly has strong and compel- Evolution; Ecology suggests Ecosphere). One ought to ques- ling arguments, ranging from the importance of publishing tion whether this tactic is a way to keep journals ’ impact fac- negative results to the appropriate and maximally benefi cial tors high while ensuring a high volume of publication within use of publicly funded research. Despite the point that sci- their publishing companies (a win-win situation). It is clear entifi c views are often recognized as biased towards success- that many top journals use the ‘ elitist model’ to only accept ful results, the publication of negative results may be seen papers that are likely to increase their impact factor (Aarssen as vital in order to help others potentially avoid previous and Lortie 2009), even though they could accept many more research pitfalls, thereby saving time and public money as that are clearly of appropriate quality for publication. well as improving the research in that particular fi eld. One interesting case is that of sub-atomic physics, in The role of journals in the age of imperfect which particle experiments often involve research teams information search capability composed of hundreds of scientists planning research that may take years and millions of dollars to run a ‘ few ’ experi- As online-only journals with no limitations of space become ments. Should the results of these experiments (positive or more common, together with the fact that a huge percent- negative) always be reported given the amount of resources age of papers go through diff erent levels of peer review that were involved? Should the classical view on peer review until their eventual publication, the obvious conclusion is in this case have a greater role in judging whether the research that peer review is indeed judging where a paper is pub- is somewhat misguided and thus does not merit publication? lished but not whether a paper is publishable. Based on In studies involving hundreds of scientists and large amounts this argument, one would be tempted to propose an alter- of funding, one possible argument is that the research has native approach; that is, why not just have a few publish- already undergone suffi cient ‘ peer review ’ and that the work ing repositories? Th ree arguments (aside from the fi nancial should be published regardless of the opinions of a reduced profi ts for publishing companies) seem possible to be made number of post reviewers. against this view: 1) journals serve as a quality control in If one agrees that ‘ big science’ may be subject to diff erent the sense that high-impact journals help one navigate peer review notions and standards, how should we treat through the jungle of potentially uninteresting and trivial ‘ little science ’ ? Every year, there are thousands of local-scale research (the elitist argument); 2) publications in high or site-based (e.g. a lake that is important to a particular impact journals often help in the recruitment process of region, a small forest plot) observational and experimental scientists by helping to ‘ predict the potential’ of candidates studies that involve relatively similar questions but changing for a particular position. Often, hiring committees make study systems and organisms. Th ese studies are likely to be decisions on such basis as young scientists have not had yet seen through lenses ranging from relevant to ‘ local knowl- the time to accumulate high number of citations (‘ predict- edge without broad scientifi c implication worldwide ’ to ing the future’ , as mentioned by a colleague, by publish- ‘ trivial ’ , depending on the readership. Note, however, that ing in high-impact journals); and 3) specialized journals the knowledge that results from this type of research is con- allow researchers to fi nd material that is relevant to their sidered critical in many circles, such as applied conservation immediate fi elds of expertise in a more effi cient manner. biology and natural history (Dayton 2003). Although there Interestingly, a large number of specialized journals have are views condemning some of the implications involved dramatically increased their impact factor in the past decade in ‘ little science ’ , including irresponsible authorship and or so. Nonetheless, it is well accepted that good papers are wasteful publication (Huth 1986), many fi elds have certainly

EV-4 profi ted from sheer numbers and serendipity. Another argu- eventually improve the quality of their papers and the chances ment, made by an ecologist colleague of mine, is that pub- to have it accepted in the ‘second-best’ journal. Papers can and lishing ‘ little science ’ motivates students and makes them certainly improve through excessive peer review, but the costs to excited about conducting research early in their studies. our ‘ reviewer commons ’ are too high. Another (even worse) trend A somewhat opposite argument (which I don ’ t necessarily is that some “ manuscripts rejected by one journal are submitted to agree) has been made by colleagues in which by aiming at another journal without change” (Blö schl et al. 2014). publishing in high-impact journals, students and scientists Given our current technology, fi ve non-mutually exclu- focus on research that has a broader impact and that help sive potential solutions are worth considering. Th e fi rst is to sharp their minds. Th e reality is that ecology as a fi eld has make use of “ public preprints servers ” to encourage public evolved as a result of both ‘ big ’ and ‘ little science’ . feedback prior to sending the paper to peer review (reviewed Despite the pros and cons of ‘ little science ’ , if the major- by Desjardins-Proulx et al. 2012). Th e second solution is to ity of these types of studies are never published, what are the increase the use of open peer review depositories (e.g. Axios consequences of this hidden knowledge to science and the Review, Peerage of Science, SelectedPapers.net), in which further consequences for the optimal use of public funding? reviewers chose specifi c manuscripts to review and journals Th is may be a key issue, particularly because only 20% of “ bid ” on the reviewed papers. Th ese initiatives are quite good all data produced may be used in a publication (Heidorn for a number of reasons including optimizing publication 2008, Gonzalez and Peres-Neto 2015). Th is issue generates and reviewers’ time (Vines 2015), but one obstacle is the a potential tradeoff between the optimal use of public fund- potential cost to journals or users. Journals must pay ‘ Peerage ing and our views about the ‘ gate keeper ’ view of peer review of Science ’ to acquire papers, Axios charges authors (or in and whether one single view would best serve diff erent fi elds some cases a few open access journals absorb the fees in their and countries, especially given that most journals serve the publication cost) and SelectedPapers.net is free. Moreover, entire international research community. Th e evolution of if these open review systems become popular it is likely that new cultural norms in our peer review system should be per- publishing houses will eventually build their own deposito- haps guided by ethical imperatives involving the optimal use ries and, depending on the rules that are applied (e.g. bids of funding and our views of the ideal minimum publishable only by journals distributed by the same company), the sys- unit (inspired by a phrase from Aarssen 2012). At any rate, tem may not work as independently as was intended. Th e as the number of ‘ little science’ studies increases owing to third solution is for journals to request that authors commit demographics (i.e. more researchers), technological devel- to reviewing papers. PeerJ, for instance, states that “ the duties opment will continue to provide an increasing number and of PeerJ members are pretty small: review at least one paper array of publication opportunities for these papers regardless per year or submit a comment to a published article ” . Th e of the ‘ elitist ’ views that classical peer review should serve fourth solution is to start recognizing and rewarding review- as an academic gatekeeper. Interestingly, it is likely that the ers by making reviews citable (see comment by Rosindell number of highly cited journals will not likely increase in and Pearse < http://blogs.plos.org/biologue/2012/08/03/ proportion to the number of published papers in the future. openly-streamlining-peer-review/ > ). Finally, an attempt that has been growing, not related to technology though, is the Do authors have the time to review? Technological increase of editorial boards that serve themselves as review- solutions to the tragedy of commons in the ers. In this case, board members get recognition through publication system community service, which is well recognized as part of the offi cial academic duties by many institutions. It has certainly been well recognized that our peer review system is becoming clogged and may potentially go bankrupt Can we resolve current issues with our publication (Fox and Petchey 2010). Two major problems contribute to system? the issue. First, there is a great deal of incentive to publish but not to review. Some papers are rejected by multiple jour- Discussions with colleagues over the past three years have nals, and some may use up to 10 or more reviewers along the revealed that they seem to recognize the issues summarized way until they are fi nally accepted. Th is ‘ tragedy of commons ’ here (Table 1), although there is less agreement on the is well summarized by Fox and Petchey (2010): “ Th e result following solutions that would aid in adapting our current is a system increasingly dominated by “ cheats ” (individuals publication system to new norms. who submit papers without doing proportionate reviewing)” . Th e authors suggest the creation of a “ PubCreds ” system 1) Increase the number of papers per journal that maintains a credit system on the basis of the number of It would improve review time (e.g. fewer submissions until reviewers who researchers used to evaluate their work ( “ with- publication success), reduce the pressure on reviewers, and drawals ” ) versus the number of reviews each researcher con- reduce the lottery component in the current publication sys- tributed (“ deposits ” ). In this way, keeping a “ public ” credit tem. Journals could then create sections based on diff erent balance sheet would be a way to reduce “ cheating ” . We cer- criteria, such as novel versus confi rmatory research and local tainly have the technological means to create such a system, versus regional versus global perspectives to help guide their but the challenge here would be convincing publishing com- readership. panies to accept it. Th e second issue is related to the fact that some authors use peer review to replace ‘ friendly ’ or pre-peer- 2) Consolidate journals review. Th ere are certainly researchers who see that by tak- Th e merging of journals with similar target readerships ing their chances fi rst with a high-impact journal, they can would increase our current ability to fi nd related work,

EV-5 reduce ‘ elitist practices ’ and potentially generate a larger, 8) Liquid journals in which peer review is not mandatory dedicated readership with a sense of scientifi c community Fabio Casati, a computer science professor at the University interested in improving the overall quality of the research of Torino founded a ‘ liquid journal’ that publishes papers in their fi elds. For instance, Wiley publishes Global Ecology without peer review (see Smith 2010 for more details). and Biogeography, Ecography and Journal of Biogeography, Casati ’ s point is that scientists spend too much time worry- which have very similar audiences and even share some edi- ing and editing papers rather than thinking or doing science. torial members in common. Note that Ecography belongs to He has conducted research showing that there is little cor- the Nordic Society Oikos and having societal journals may relation between the grades peer reviews give to papers and be seen as important in order to advance their interests and how well they do after publication and his conclusion is that mandates. is better to “ publish and let the world decide ” . Given our current conservative ways (e.g. perceived value of peer review, 3) Centralize review systems academic evaluation, commercial interests), it is unlikely that Centralization would allow reviewers to choose what they such as system will take over our current model. want to review and likely reduce review and publication time. 9) Awareness of the publication business model Th ere is a current believe that open-access journals are 4) Advanced search tools a goldmine. On the contrary, most open-access journals One future (if not present) major current limitation to are free and most (75%) of non-open access (conventional) the continuing increasing of publication numbers will likely journals charge publication fees (ALPSP 2005). By being be the lack of perfect information search capability. In misinformed, some researchers fi nd it controversial to pub- its absence, we will be immersed in a world of too many lish in open-access journals. Th e open access is just an exam- publications to allow for effi cient sorting or suffi cient orga- ple to make the point that awareness is important because nization to develop a logical understanding of the trends researchers can help changing the current publication system and evolution of a particular fi eld. In the Supplementary by making choices according to well-guided philosophies. material Appendix 1, I propose a search system that can be easily implemented with current technology that would seem Closing remarks capable of advancing our current search capability. Has the peer review system changed as a result of technology? 5) Be realistic about where your research can be published I think that there are clear indications that it has. Review- Th is argument can certainly be made without the need of ers do not review enough manuscripts, there are too many technology, though technology can certainly help. Th e search journals, good journals do not publish enough papers, and tool described in the online supplement should improve we have too much to read. Although these issues are most one’ s ability in judging the most appropriate journals for a obviously related to the increase of scientifi c demographics given manuscript. and technological advances (‘ all great things’ ), the solution likely resides not only in technology but also in changing the 6) Post-publication editing optics and ethics of publishing. Technology, especially stor- A critical current issue is what has been referred to as to age capacity, online capability and search tools, has nearly “ salami science” and “meat extenders ” (extracted from Huth provided the necessary capacity to generate new journals to 1986): “ republishing the same material in successive papers meet current and future demographic demands. Moreover, and blending data from one study with additional data to technology allowed ecologists (and other fi elds) to increase extract yet another paper that could not make its way on the data generation, improve data sharing capability via deposi- second set of data alone (“meat extenders”) ” . Current tech- tories (not always equated with willingness to share; e.g. data nology could also certainly help here. Authors could gain papers in Ecology, Ͻ http://datadryad.org/ Ͼ among others) the rights to generate updated versions of the same paper and allow the dissemination of data analysis techniques via rather than producing new ones. Having updated versions code sharing (e.g. R project, Ͻ www.r-project.org Ͼ ). (numbered individually so that they can be referenced sepa- Should all research produced in good faith be published? rately) consolidated into a single place (journals) would cer- Logic and good use of public money, in my mind, suggest tainly help in providing a more comprehensive view about that it should, given a minimum level of review prior to fi nal the research. publication. By no means, I fi nd that the change in technology and demographics are changing peer review into an 7) An idealistic ‘ single consolidated free open source’ as a unnecessary process. Th at said, optics about peer review are self-regulatory system now multiple and is constantly evolving. My prediction is Provided that we have major breakthroughs in search that future technological development will help peer review capability such that information can be found precisely evolve into a system conducted using diff erent parameters of and effi ciently, the adoption of a ‘ single consolidated free logic, ethics, rationale and mindset that proves to be more open source publication system ’ would perhaps seem to be a useful to the research community and to society than the natural way to address the many current issues described here. current system. You may not agree with the solutions dis- Th ere are many arguments for and against a single deposi- cussed here, but if you agree with some of the issues raised, tory system, but such a system is unlikely to become viable, I would like you to think about ways to reform our cur- not because of technological hurdles but rather because of rent system. In this way, we can fi nd more time to dedicate academic, economical and societal views. to one of the most important academic endeavors: reading

EV-6 and learning what has been researched and creating (and Emch, A. 1937. Rejected papers of three famous mathematicians. publishing) better research. – Nat. Math. Mag. 11: 186 – 189. Fox, J. W. and Petchey, O. L. 2010. PubCreds: fi xing the peer review process by ‘ privatising ’ the reviewer commons. – Bull. Acknowledgements – I would like to thank Lonnie Aarssen, Stefano Ecol. Soc. Am. 91: 325 – 333. Allesina and Dries Bonte for comments on the manuscript. James Frishauf, P. 2008. Th e end of peer review and traditional publishing O’ Dwyer and Steven Kembel provided much input in early stages as we know it. – Medscape J. Med. 10: 267. of this forum piece. I also thank more than 100 colleagues who Gonzalez, A. and Peres-Neto, P. R. 2015. Act to staunch loss of helped in sharpening the issues raised here, even though they did research data. – Nature 520: 436. not all necessarily agree with them. Heidorn, P. B. 2008. Shedding light on the dark data in the long tail of science. – Libr. Trends 57: 280 – 289. Huth, E. D. 1986. Irresponsible authorship and wasteful publica- References tion. – Ann. Int. Med. 104: 257 – 259. Lee, C. 2012. Open peer review by a selected-papers network. Aarssen, L. W. 2012. Are peer-review fi lters optimal for the progress – Front. Comput. Neurosci. 6: 1 – 15. of science in ecology and evolution? – Ideas Ecol. Evol. 5: Mingers, J. and Lipitakis, E. A. E. C. 2013. Evaluating a 9 – 12. department ’ s research: testing the leiden methodology in busi- Aarssen, L. W. and Lortie, C. J. 2009. Ending elitism in peer- ness and management. – Inf. Process. Manage. 49: 587 – 595. review publication. – Ideas Ecol. Evol. 2: 18 – 20. Mulligan, A. et al. 2012. Peer review in a changing world: an inter- Agrawal, A. A. 2014. Four more reasons to be skeptical of the national study measuring the attitudes of researchers. – J. Ass. open-access publishing. – Trends Plant Sci. 19: 133. Inf. Sci. Technol. 64: 132 – 161. Allesina, S. 2012. Modeling peer review: an agent-based approach. N e ff , B. D. and Olden, J. D. 2006. Is peer review a game of chance? – Ideas Ecol. Evol. 5: 27 – 35. – BioScience 56: 333 – 340. ALPSP 2005. Th e facts about Open Access: a study of the fi nancial Peters, D. P. and Ceci, S. J. 1982. Peer-review practices of psycho- and non-fi nancial eff ects of alternative business models for logical journals: the fate of published articles, submitted again. scholarly journals. – Kaufman-Wills Group, LLC, Association – Behav. Brain. Sci. 5: 187 – 195. of Learned and Professional Society Publishers. Rothwell, P. M. and Martyn, C. 2000. Reproducibility of Bj ö rk, B.-C. et al. 2009. Scientifi c journal publishing: yearly vol- peer review in clinical neuroscience – is agreement between ume and open access availability. – Inf. Res. 14: paper 391. reviewers any greater than would be expected by chance alone? Bl ö schl, G. et al. 2014. On the future of journal publications in – Brain 123: 1964 – 1969. hydrology. – Water Resour. Res. 50: 2795 – 2797. Smith, R. 2010. Classical peer review: an empty gun. – Breast Bohannon, J. 2013. Who ’ s afraid of peer review? – Science 342: Cancer Res. 12: 513. 60 – 65. Th urner, S. and Hanel, R. 2011. Peer-review in a world with Bornmann, L. and Daniel, H. D. 2010. Th e manuscript reviewing rational scientists: toward selection of the average. – EPJ B 84: process: empirical research on review requests, review sequences 707 – 711. and decision rules in peer review. – Libr. Inf.. Sci. Res. 32: Vines, T. H. 2015. Th e core ineffi ciency of peer review and a poten- 5 – 12. tial solution. – Limnol. Oceanogr. Bull. 24: 36 – 38. Dayton, P. K. 2003. Th e importance of the natural sciences to Walker, R. and Rocha da Silva, P. 2015. Emerging trends in peer conservation. – Am. Nat. 162: 1 – 13. review – a survey. – Front. Neurosci. 9: 169. Desjardins-Proulx P. et al. 2013. Th e case for open preprints in Wenner å s, C. and Wold, A. 1997. Nepotism and sexism in peer biology. – PLoS Biol. 11: e1001563. review. – Nature 387: 341 – 343.

Supplementary material (available online as Appendix oik.02956 at Ͻ www.oikosjournal.org/readers/appendix Ͼ ). Appendix 1.

EV-7