The Four Pillars of Scholarly Publishing: the Future and a Foundation

View metadata, citation and similar papers at core.ac.uk brought to you by CORE

provided by University of Tennessee, Knoxville: Trace

University of Tennessee, Knoxville Trace: Tennessee Research and Creative Exchange

School of Information Sciences -- Faculty Publications and Other Works School of Information Sciences

4-7-2014

The four pillars of scholarly publishing: The future and a foundation.

Jarrett EK Byrnes

Edward B. Baskerville

Bruce Caron

Cameron Neylon

Carol Tenopir University of Tennessee - Knoxville

See next page for additional authors

Follow this and additional works at: https://trace.tennessee.edu/utk_infosciepubs

Part of the Library and Information Science Commons

Recommended Citation Byrnes, Jarrett EK, Edward B. Baskerville, Bruce Caron, Cameron Neylon, Carol Tenopir, Mark Schildhauer, Amber Budden, Lonnie Aarssen, and Christopher Lortie. “The four pillars of scholarly publishing: The future and a foundation.” Ideas in Ecology and Evolution Vol. 7, no. 1 (2014).

This Article is brought to you for free and open access by the School of Information Sciences at Trace: Tennessee Research and Creative Exchange. It has been accepted for inclusion in School of Information Sciences -- Faculty Publications and Other Works by an authorized administrator of Trace: Tennessee Research and Creative Exchange. For more information, please contact [email protected]. Authors Jarrett EK Byrnes, Edward B. Baskerville, Bruce Caron, Cameron Neylon, Carol Tenopir, Mark Schildhauer, A.E. Budden, Lonnie Aarssen, and Christopher Lortie

This article is available at Trace: Tennessee Research and Creative Exchange: https://trace.tennessee.edu/ utk_infosciepubs/154 Ideas in Ecology and Evolution 7: 27–33, 2014 doi:10.4033/iee.2014.7.7.f © 2014 The Author. © Ideas in Ecology and Evolution 2014 iee Received 24 April 2013; Accepted 18 February 2014

Future of Publishing

The four pillars of scholarly publishing: The future and a foundation

Jarrett E.K. Byrnes, Edward B. Baskerville, Bruce Caron, Cameron Neylon, Carol Tenopir, Mark Schildhauer, Amber Budden, Lonnie W. Aarssen, and Christopher J. Lortie

Jarrett E.K. Byrnes ([email protected]), Department of Biology, University of Massachusetts Boston, Boston, MA USA 02139

Edward B. Baskerville, Department of Ecology and Evolutionary Biology, University of Michigan, Ann Arbor, MI USA 48109

Bruce Caron, New Media Research Institute, Santa Barbara, CA USA 93101

Cameron Neylon, PLOS, Carlyle House, Cambridge, UK

Carol Tenopir, School of Information Sciences, University of Tennessee Knoxville, Knoxville, TN USA 37996

Mark Schildhauer, National Center for Ecological Analysis and Synthesis, Santa Barbara, CA USA 93101

Amber Budden, National Center for Ecological Analysis and Synthesis, Santa Barbara, CA USA 93101

Lonnie W. Aarssen, Department of Biology, Queen’s University, Kingston ON, Canada K7L 3N6

Christopher J. Lortie, Department of Biology, York University, 4700 Keele St., Toronto, ON, Canada M3J 1P3

Abstract participating in the process. These four pillars will guide the development of better tools and practices for discov- Scholarly publishing has embraced electronic distribut- ering and sharing scientific knowledge in a modern ion in many respects, but the tools available through the networked world. The current traditional scholarly pub- Internet and other advancing technologies have pro- lishing model arose in the 1600s, and though it has found implications for scholarly communication beyond served its purpose admirably and well, it is time to move dissemination. We argue that to best serve science, the forward by embracing open, rapid transparent publicat- process of scholarly communication must embrace these ion and review. advances and evolve. Here, we consider the current state of the process in Ecology and Evolutionary Biology Many forms of Government have been tried and will be (EEB) and propose directions for this evolution and tried in this world of sin and woe. No one pretends that potential change. We identify four pillars for the future democracy is perfect or all-wise. Indeed, it has been of scientific communication: (1) an ecosystem of said that democracy is the worst form of government scholarly products, (2) immediate and open access, (3) except all those other forms that have been tried from open peer review, and (4) full recognition for time to time. ~Winston Churchill

To see reviewer comments on the initial submitted version of this manuscript, see http://goo.gl/zO1FyW

To make comments on the published manuscript, or initiate new discussions, go to http://goo.gl/zOTqFe

iee 7 (2014) This work is licensed under a Creative Commons Attribution 3.0 License. 27 Introduction the literature they need; this access is crucial for their everyday work (Tenopir 2012). Similarly, the impact of We live in an age of rapid communication open to an high quality science is actually increased when it is ever-growing pool of information and ideas, yet our made open access (Gargouri et al. 2010). Recently the current system of communicating the results of scholar- predominant discussions about open access have moved ly activities dates back to the 1660s and still reflects past whether science needs open access or not and right many of the restrictions of that time. It limits access to on to the details of how it will be funded and how those outside the ivory tower via paywalls, keeps the existing journals and societies will adapt to the change. review process behind the closed doors of anonymity, As we concede that access must be open for the sake of and operates at a speed often far slower than what is good science, we naturally should ask what other possible. Is this a model for 21st-century publishing? Or aspects of the publishing process should also be opened do the general principles of rapid open access that have up. become apparent with the growth of the information age Here, we propose four pillars for a more open future provide a template for a form of scholarly commun- of scholarly publishing: (1) a widening of our definition ication that is both higher in quality and allows for the of scholarly products, (2) immediate open access to more rapid and efficient dissemination and revision of these products at the start of their assessment and refine- ideas? Can we improve our science by changing the ment, (3) open public review for scholarly products, and foundations of scientific discourse? With the rapid rise (4) an improved ability to assess scholarly products and of scholarly discourse online (Fox 2012), the time is the overall contributions of scholars. These pillars right to examine how we can improve the system. Here, (Figure 1) do not stand alone, but meet at a common we present a framework that provides the foundation for foundation: the need to link products together to better these considerations. enable discovery of relevant information. Embracing The debate on open-access publishing reflects the these core principles, we believe, will enhance both the ongoing changes in how scientists interface with content speed and quality of the scientific enterprise. and how their expectations for access have shifted, and moreover points the way towards more significant An Ecosystem of Scholarly Products changes to the system. Many scientists have already recognized that open access enables better science, Scholarly publishing in EEB is largely limited to a because it guarantees that researchers have the access to single species of product—namely the narrative paper in

Figure 1. A brief overview of the four pillars of the future of scholarly publishing built atop the foundation of networked discover of new work. iee 7 (2014) 28 one of only a few forms. This monoculture of scholarly scientific process in EEB. We are cutting off a mode of production is curious, particularly given that an entire scholarly production, and limiting the information avail- subfield of ecology is devoted to the demonstration that able to scientists in the future—and by future, we may diversity can often lead to higher levels of ecosystem mean as little as a year, as blogs can have very limited function (Loreau et al. 2001). Importantly, scientific life-spans (i.e. sometimes only a year). If the purpose of dialog has eclipsed this singular modality in many scholarly communication is to forward EEB by ensuring respects with critical discoveries emerging more freely the dissemination of critical information to future schol- in dynamic discussion via blogs and social media. The ars, this distributed, undervalued set of shared products narrative paper artifact is a product of print publication must change. Beyond just creating new recognized and is increasingly seen as only one vehicle for science venues for this work, a line on a scholars CV listing a knowledge and practice to emerge in a digital world. In data product, software package, non-journal publicat- its recent white paper, the Force11 association has ions, and other scholarly resource needs to be valued in envisioned two aspects of new science artifacts: artifacts the same way as a narrative product for hiring, granting that capture the “relationships between knowledge, funds, and tenure evaluations. These are all building claims, and data,” and artifacts that promote the reprod- blocks for the future of progress in EEB. It is time that ucibility of science workflows (Bourne et al. 2012). It we begin to recognize them as such. is time for EEB to move beyond the devaluation of alternative types of scholarly product and embrace the larger ecosystem of scholarly products lest this infor- Immediate Access mation be lost, neglected, or generally undervalued. Fortunately, we have already begun to recognize that Once a researcher or research team deems the fruits there is more than one scholarly product that is of value of their scientific labor ready for the limelight, it’s time to advancing science. Publication of data has become for them to begin the cycle of peer review. The critique increasingly commonplace, with whole journals devoted of scientific work by one’s peers is the cornerstone of to data papers (e.g., Dataset Papers in Ecology scientific publishing (Goodman et al. 1994), and it is [http://www.hindawi.com/dpis/ecology/], Nature Pub- absolutely essential in order to have scientific work be- lishing Group’s Scientific Data [http://www.nature.com/ come part of the permanent record of human progress. scientificdata/] or see Costas et al. 2013 Appendix 3 for Without a review process, there would be no way to a more comprehensive list) and the establishment of confidently make an initial assessment of the validity of best practices (Chavan and Penev 2011). Open notebook a single piece of work. That said, the review process science—the sharing of immediate results and observ- takes time. Sometimes weeks. Sometimes months. ations from the bench or the trench—has grown enor- Sometimes—with rejection, resubmission, re-review, mously (e.g., http://onsnetwork.org/). Outside of schol- etc. as scientists climb through a series of journals until arly journals, blogs have become fertile ground for the their paper finds a home—years (Ioannidis 1998). Is presentation of short observations (e.g., http://wfsu.org/ science being served by the long delay between when a blog-coastal-health/, http://neurodojo.blogspot.co.uk/ researcher has results ready for the public to scrutinize, 2012/09/Ibacus.html, or http://www.imachordata.com/ and when the scientific community actually gets to view scallopocalypse/) or rich media products that allow them? What are the consequences of this time lag for users to interact with and visualize data in ways not the progress of science? possible in traditional journal articles. Other services The conservative answer is that immediate access to have evolved to allow scientists to collect software and new pieces of scholarly publication before going workflows (e.g., GitHub [http://github.com], RPubs through a peer review process will seriously harm sci- [http://rpubs.com]). ence (Gorman 2001). If this were true, then math, All of these steps in the scientific workflow are also physics, and astronomy should have imploded in the products of the scholarly process and have intrinsic 1990s. By the late 1980s, high-energy physicists were value for not only replicability but also discovery. frequently exchanging manuscripts prior to peer review However, there remains a subset of EEB practioners that via email lists. This process became centralized and for- behave as if these products are second class and not malized at the LANL Preprint Archive, which be-came critical in the annals of scholarly discourse. Given the the arXiv, in the 1990s (Ginsparg 2008). Rather than fragmented landscape wherein these products currently immediate access being an impediment to scientific pro- reside, there is no centralized archival tool for the gress or diluting the field with crackpots, it has advance- observations, rich media displays, or software that can ed the speed of science in the disciplines that use it ensure that they will remain part of the development of (Davis and Fromerth 2007, Gentil-Beccot et al. 2010), science. and provided a valuable forum for new results and ideas Essentially, we are failing to archive and recognize to be discussed widely, beyond just a pool of two to many of the arguably most important elements of the three reviewers. iee 7 (2014) 29 It should be noted that although the arXiv facilitated once this process is complete, the intellectual discourse the widespread adoption of a preprint culture in physics, of review is discarded into the dustbin of the editorial mathematics, and computer science, that preprint culture process. Moreover, we acknowledge that review is a existed in particle physics before the creation of the human process. Inaccuracies, grudges, bias, careless- arXiv (Ginsparg 2008). In EEB, we have very little ness, and more can all creep into the review process, preexisting preprint culture to start with; if we wish to often without intentional malice (see Lee et al 2012 for start one, we must copy the physicists (Desjardins- review). This information is likewise consigned to the Proulx et al. 2013). Although people frequently circulate dustbin. Furthermore, anonymity itself may not be a pre-publication versions of manuscripts to close col- best practice. While anonymity may free a reviewer to leagues, the understanding is that these early versions be critical, it may also free a reviewer to engage in are not for widespread consumption, and certainly not behavior that furthers a personal agenda—conscious or for dissemination on a public email list or website for not—rather than the agenda of science. Thus, open the whole field to see. Thus EEB must undergo a review (Kriegeskorte 2012) provides a way to bring fundamental cultural change in order for the fresh air into the process, reducing bias and improving dissemination of preprints to become standard practice review utility. (Desjardins-Proulx et al. 2013). We cling to the need to closed peer review, often We view immediate access as a fundamental corner- citing the simple and seemingly obvious assumption that stone of the future of scholarly publishing. This extends reviewers are more willing to be openly critical of a beyond preprints of papers before they are submitted to work if they will not suffer any retaliation from the a formal journal, but immediate open quality assured authors. Once a piece of work is accepted, all of those data, as is already done by the Long Term Ecological reviews vanish, only to be seen again inside of offices of Research network (Karasti and Baker 2008) and open the journal’s publisher. Are these two pieces of our and shareable code, as is mandated by some journals current review system beneficial for science? already (ESA Author Instructions 2013). Once a Anonymous review assumes that anonymity will research product is deemed ready for the world by a lab, allow for better commentary and more stringent in order to speed science and improve the quality of the critiques that will ultimately improve the quality of a work itself, it must be immediately accessible to the published article. This is not always the case, however. scientific public for reading, discussion, and judgment. Anonymity allows for a wide variety of abuses within Keeping it behind the closed doors of one or more the peer review process, including suppression of work editorial processes serves little purpose other than to similar to a reviewer's own, nepotism influencing article slow down the dissemination of knowledge. Indeed, as acceptance, and sexism affecting article acceptance, the best way to evaluate a piece of work is to have the among others. None of these help science. Furthermore, most knowledgeable scientists read and then evaluate opening up the identities has been shown to either have that work, we do not serve science by potentially re- no impact on reviewers’ quality of review (van Rooyen stricting their access to only a limited pool of reviewers. et al. 1998) or actually benefit the final finished product Open, immediately accessible manuscripts can be read, (Walsh et al. 2000), particularly when authors and assessed, and critiqued by a larger audience of interested reviewers can interact (Leek et al. 2011). From the scientists, thus improving both the reach and the quality perspective of science, there is little to be lost and much of scientific discourse and avoiding unwitting wasteful to be gained by abandoning anonymity in the peer duplication of effort (and may even lead to new review process. To quote from Smith (2006) "Often I collaborations). found the discourse around a study was a lot more interesting than the study itself." Open Review Furthermore, why is review closed? Why are useful pieces of thoughtful commentary discarded, so that The hallmark of our scholarly publishing system is readers cannot see what their colleagues think of new the acceptance of new work into the corpus of science work. Largely, this is to protect anonymity, which we only after the work has been reviewed and approved, have discussed above. If reviewer identities are open to often after substantial revision, by anonymous peer authors, why should their identities and contributions reviewers in a closed-door process. The current process not be open to the readership? Some journals, such as of reviewing a paper is a triumph of the intellectual Biology Direct (http://www.biologydirect.com/, Koonin endeavor of science. Reviewers attempt to dispassion- et al. 2013), PLOS One (http://www.plosone.org/), ately rake a new piece of scientific information over the F1000Research (http://f1000research.com), and PeerJ coals of rigor. They put a large amount of time, effort, (https://peerj.com/) offer the co-publication of reviews and thought into ensuring the highest quality informat- already. They create a rich starting point for further ion reaches the general scientific audience. And yet, conversation about the import of new work, and are

iee 7 (2014) 30 created by readers who have, by their participation in .com). Once discussion is in the open, the community is the review process, thought long and hard about a wide able to give feedback on its usefulness, generating a rich variety of issues contained in the work. quantitative reputation. A system where not only is the The change in the publication process to open up amount of reviewing done by individuals, but the review is small: final comments are posted alongside a quality, judged both by the author and community—as published piece of work. The benefit to future scientists well will foster a far more rich and meaningful scholarly of seeing these comments is immense, and, as above, ecosystem. Perhaps the greatest benefit is to reviewers the costs are likely small. Moreover, having reviews themselves. It will enable us to see when where and how published alongside scholarly products creates a culture our reviews are the most helpful. We will able to of conversation. This change in culture may well facili- collectively become better reviewers, and improve the tate further comments, responses, and counter quality of the process that births new work in the peer responses. A wide variety of online tools have evolved reviewed world. Furthermore, by tying commentary to to facilitate this type of conversation, and we see them reviewer’s public identities with tools such as the already taking flight on Twitter and in the world of ORCID identifier system, we’re more easily able to blogs (e.g., the #arseniclife example, see Zimmer 2011). create an open community with its own self-enforced If the publications themselves took the lead in changing norms of review that are accessible to all, rather than our culture towards one of more open conversation, a hidden under the cloak of anonymous peer review. better process of scientific dialogue would result. Networked Discovery of New Work via Better Full Reputation and Recognition Review Tools

When we review and edit manuscripts, we are mak- Each of the steps listed above would be a significant ing an intellectual contribution to the development of and positive benefit to the scholarly publishing system. science. We are giving rich thoughts and commentaries They serve a greater purpose, however, when consider- to an author, and helping to shape the development of ed as a whole. They can facilitate the discovery of new the field of science. Because reviews are never seen literature, speeding the development of science. outside of the authors of a paper and editors of a journal, A common problem in the literature is that one can this contribution goes largely unrecognized beyond a miss new work that is highly relevant to them either due brief line in the synergistic activities portion of our to its placement, unknown authors, or being in a curriculum vita. Even in an open system, however, discipline that appears irrelevant to an author. Often, when our colleagues, with a little legwork, could see connections to new work are made by a colleague how much we have reviewed, we still do not have any sending another a reprint. In this way, we already have a sense of whether we are making a meaningful slow informal reputation mediated information distribut- contribution to the scholarly discourse. ion system. If reviews and commentary on papers are Articles have citation counts, download statistics, and open, we can begin to use this information to create other alternative metrics (Priem et al. 2010). Reviews networks of reading habits that can inform what new are transitory pieces of thought that we have no literature we should be discovering. This kind of systematic way of judging. Furthermore, scientists who networked discovery has been a boon for corporations make incredible contributions to the literature by the such as Amazon, Netflix, and others. strength and thoroughness of their reviews go complete- The first step towards better networked discovery is ly unrecognized, lumped with those who would send a open access. Despite living in an information deluge two sentence summary judgment with little detail (Bell et al. 2009), the transformation of this data into beyond, "This looks fine." useful information is often hidden behind paywalls and The reputation and recognition of not just the embargo policies. Thus, the first element to enhance quantity of reviews, but also the quality of reviews is an discovery is the immediate open access to the panopoly essential pillar of any future scholarly review system. of scholarly products that can be available - but with This reputation serves not only to aid editors and sufficient curation so that information can be filtered authors in finding the most useful reviewers, but also and sifted with ease. With the growth of open access serves to help scientists better quantify how well they journals (Laakso and Björk 2012), we are already are contributing to the scholarly discourse around them. moving towards this world. This is a problem that has been solved in many less Furthermore, by incorporating reputation and recog- formal platforms. The past twenty years have witnessed nition systems, scientists can tune the discovery process. a lively development of tools to assess commentary on Readers can tune their discovery process to examine internet discussion boards (e.g., http://reddit.com) and what their most highly regarded peers in their discipline professional and academic question and answer sites are reading. They can find pieces being read or recom- (e.g., http://mathoverflow.com, http://stackoverflow mended by those whose commentaries and thoughts iee 7 (2014) 31 they themselves have highly rated. They can see what questions. To not grapple with them would be a those with whom they strongly disagree find fascinating disservice to science. in the new literature, a progressive way of keeping one's intellectual horizons open to new thoughts and ideas. Acknowledgements There are a huge number of ways this information can be harnessed to facilitate the discovery of new work that This paper is a product from the NSF funded National can change the intellectual development of a reader's Center for Ecological Analysis and Synthesis working science. group 12651 “The future of publishing in ecology, Effective search engines made the modern Web evolutionary biology, and environmental science.” We possible. As the Web has evolved, the use of machine- thank Ross Mounce, Karthik Ram, Karl Cottenie, and learning algorithms to find interesting needles in Bjoern Brembs for comments that greatly improved this gigantic haystacks has been vital for commercial manuscript. Thanks to Advisory Editor Karthik Ram for services such as Amazon and Netflix. Expert human serving as the handling and decision editor for the curation—the current purview of journals—will always manuscript. remain vitally important for evaluating and reflecting on research. But automatic software tools, guided by data, text, human curation, and online social networks, will Handling Editor enable scientists to far more easily stumble upon research in the first place. Search engines have already Karthik Ram – [email protected] had a huge impact on how we find research, but UC Berkeley machine-assisted discovery of new and interesting research from across the academic corpus is just Referees beginning to have an impact (e.g. see current attempts at Research Gate, F1000 [http://blog.f1000.com/2013/04/ Björn Brembs – [email protected] 08/follow-and-filter-your-interests/] Citeulike, Mende- Universität Regensburg ley, and Google Scholar). We look forward to further improvements in how discovery algorithms, human- Ross Mounce – [email protected] computer interaction, and online social networks University of Bath enhance the ability of scientists to make connections.

Conclusions References

The above four pillars—a widening of our definition Bell, G., Hey, T., and A. Szalay. 2009. Computer sci- of scholarly products, immediate open access to some ence. Beyond the data deluge. Science 323:1297– versions of these products, an open public review for 1298. CrossRef scholarly products, a greater ability to recognize both Bourne, P.E., Clark, T., Dale, R., de Waard, A., the quantity and quality of contributions by scholars to Herman, I., Hovy, E.H. and D. Shotton. 2012. the communication process—all build into a system that Improving future research communication and e- leads to better information discovery and faster more scholarship: a summary of findings. Force 11. Mac- intellectually vibrant science. The principles we discuss quarie University ResearchOnline. here are not new or foreign to science, but naturally Costas, R., Meijer, I., Zahedi, Z. and P. Wouters. 2013. extend from current publishing system. Although they The Value of Research Data – Metrics for datasets will require substantial changes to some of our current from a cultural and technical point of view. A Know- scientific publishing practices, we are already seeing the ledge Exchange Report, available from www.know growth of experiments in scholarly communication such ledge-exchange.info/datametrics as Atmospheric Chemistry & Physics (http://www.atmo Chavan, V., and L. Penev. 2011. The data paper: a spheric-chemistry-and-physics.net/), PLOS One, PeerJ, mechanism to incentivize data publishing in biodiv- Push (http://push.cwcon.org/about/), Biology Direct, ersity science. BMC Bioinformatics 12:S2. CrossRef and many many more. The scientific community as a Davis, P.M., and M.J. Fromerth. 2007. Does the arXiv whole appears quite interested in determining how best lead to higher citations and reduced publisher down- to change our practices of scholarly communication to loads for mathematics articles? Scientometrics lead to the highest quality and quickest flow of ideas. 71:203–215. CrossRef We hope that the principles we have laid out above can Desjardins-Proulx, P., White, E.P., Adamson, J.J., Ram, create a robust discussion and further experimentation K., Poisot, T., and D. Gravel. 2013. The case for open by the scientific and publishing community working preprints in biology. PLoS Biology 11:e1001563. together. These discussions will raise many deep CrossRef iee 7 (2014) 32 Fox, J. 2012. Can blogging change how ecologists share Society for Information Science and Technology ideas? In economics, it already has. Ideas in Ecology 64:2–17. CrossRef and Evolution 5:74–77. CrossRef Leek, J.T., Taub, M.A., and F.J. Pineda. 2011. Gargouri, Y., Hajjem, C., Larivière, V., Gingras, Y., Cooperation between referees and authors increases Carr, L., Brody, T., and S. Harnad. 2010. Self-select- peer review accuracy. PLoS One 6:e26895. CrossRef ed or mandated, open access increases citation impact Loreau, M., Naeem, S., Inchausti, P., Bengtsson, J., for higher quality research. PLoS One 5:e13636. Grime, J.P., Hector, A., et al. 2001. Biodiversity and CrossRef ecosystem functioning: current knowledge and future Gentil-Beccot, A., Mele, S., and T.C. Brooks. 2010. challenges. Science 294:804–808. CrossRef Citing and reading behaviours in high-energy Priem, J., Taraborelli, D., Groth, P., and C. Neylon. physics. Scientometrics 84:345–355. CrossRef 2010. October 26. Altmetrics: a manifesto – Ginsparg, P. 2009. The global village pioneers. Learned altmetrics.org. http://altmetrics.org/manifesto/. Publishing 22:95–100. CrossRef Smith, R. 2006. Peer review: a flawed process at the Gorman, J. 2001. February 5. The End of Good heart of science and journals. Journal of the Royal Science? Science News. https://www.sciencenews Society of Medicine 99:178-182. CrossRef .org/article/end-good-science Tenopir, C. 2012. The value of scholarly reading in the Ioannidis, J.A. 1998. Effect of the statistical significance life sciences. Ideas in Ecology and Evolution 5:63– of results on the time to completion and publication 73. CrossRef of randomized efficacy trials. JAMA: The Journal of Van Rooyen, S. 1998. Effect of blinding and unmasking the American Medical Association 279:281–286. on the quality of peer review: A randomized trial. CrossRef JAMA: The Journal of the American Medical Karasti, H., and K.S. Baker. 2008. Digital data practices Association 280:234–237. CrossRef and the long term ecological research program grow- Walsh, E., Rooney, M., Appleby, L., and G. Wilkinson. ing global. International Journal of Digital Curation 2000. Open peer review: a randomised controlled 3:42–58. CrossRef trial. The British Journal of Psychiatry 176:47–51. Koonin, E.V., Landweber, L.F., and D.J. Lipman. 2013. CrossRef Biology direct: celebrating 7 years of open, published Zimmer, C. 2011. May 27. The Discovery of Arsenic- peer review. Biology Direct 8:11. CrossRef Based Twitter. http://www.slate.com/articles/health Kriegeskorte, N. 2012. Open evaluation: a vision for _and_science/science/2011/05/the_discovery_of_arse entirely transparent post-publication peer review and nicbased_twitter.single.html rating for science. Frontiers in Computational Neuro- science 6:79. CrossRef Laakso, M., and B-C. Björk. 2012. Anatomy of open access publishing: a study of longitudinal development and internal structure. BMC Medicine 10:124. CrossRef Lee, C.J., Sugimoto, C.R., Zhang, G., and B. Cronin. 2012. Bias in peer review. Journal of the American

iee 7 (2014) 33