Preprints in the life sciences Jessica Polka Director, ASAPbio Visiting Scholar, Whitehead Institute @jessicapolka | #ASAPbio | @ASAPbio_
@ASAPbio_ | #ASAPbio | @jessicapolka Science is collaborative and competitive
Photo by calafellvalo Photo by George Makris
Speed benefits both science and scientists http://www.biorxiv.org/content/early/2015/07/11/022368 Vale Ron Publishing is taking longer than ever – bioRxiv & PNAS 2015 PNAS & @ASAPbio_ | #ASAPbio | @jessicapolka
Time to first author paper evaluation is to tied Publishing Preprints are manuscripts shared online before the completion of journal-organized peer review.
@ASAPbio_ | #ASAPbio | @jessicapolka Preprints are widespread in other disciplines
• Founded in 1991 • >100,000 preprints posted per year*
*https://arxiv.org/help/stats/2016_by_area/index
@ASAPbio_ | #ASAPbio | @jessicapolka Traditional publishing can hide work for months or years
Peer reviewed paper Months to years
Community feedback, ideas, discussion Public Private
Journal 1 Journal 2 Journal 3 Submit
Manuscript Peer Review Revise Revise
@ASAPbio_ | #ASAPbio | @jessicapolka Emojis by Mozilla (CC BY 4.0) Preprints make work available almost immediately
Preprint server Peer reviewed paper Months to years <48 hrs screening process Community feedback, ideas, discussion Public Private
Journal 1 Journal 2 Journal 3 Submit
Manuscript Peer Review Revise Revise
@ASAPbio_ | #ASAPbio | @jessicapolka Emojis by Mozilla (CC BY 4.0) Preprints are…
• Permanent • Versioned • Citable
@ASAPbio_ | #ASAPbio | @jessicapolka A biologist-driven non-profit working to make life sciences communication faster & more transparent
@ASAPbio_ | #ASAPbio | @jessicapolka Preprinting is growing rapidly in the life sciences
@ASAPbio_ | #ASAPbio | @jessicapolka Jordan Anaya of prepubmed.org How to find preprints
• Prepubmed.org • Osf.io/preprints • scholar.google.com • Just biomedical preprints • Indexes preprints in many fields • Indexes all literature (published (selected servers) papers, meeting abstracts, etc) • RSS feed and monthly statistics • Email alerts available available • Search results are ranked in a not entirely transparent fashion
#ASAPbio 11 Funders encouraging preprints as evidence of productivity in grant applications & reports
Current list and links to policies at asapbio.org/funder-policies
@ASAPbio_ | #ASAPbio | @jessicapolka Why preprint?
@ASAPbio_ | #ASAPbio | @jessicapolka Benefits of preprinting
• Gain visibility (like presenting at a meeting)
• Get more feedback to improve your paper Personal experiences with preprints: • Help interested journal editors find you asapbio.org/preprint- • Find collaborators earlier info/preprint-stories • Create a record of what was done on what date • Demonstrate productivity for jobs & grants • Accelerate discovery
#ASAPbio 14 Feedback
@ASAPbio_ | #ASAPbio | @jessicapolka Feedback: a major benefit of preprinting
• Email • PubPeer • Twitter (& Facebook) • Directly on the preprint site • arXiv, COS, etc have no comments • ~10% of bioRxiv preprints have comments* • Annotation (Hypothesis) • Formal review sites (Academic Karma, Peer Community In…) • Preprint journal clubs
* asapbio.org/biorxiv Feedback can be found all around the web
https://osf.io/sgpe9/
https://www.facebook.com/groups/853552931365745/permalink/1349684805085886/
Preprints themselves as feedback/critique
Christoph Lippert….J. Craig Venter Yaniv Erlich
Identification of individuals by trait prediction Major flaws in "Identification of individuals by trait prediction using whole-genome sequencing data using whole-genome sequencing data"
PNAS 2017 ; published ahead of print bioRxiv 185330; doi: https://doi.org/10.1101/185330 doi:10.1073/pnas.1711125114 September 6, 2017 September 5, 2017 September 11, 2017
Christoph Lippert… J. Craig Venter
No major flaws in "Identification of individuals by trait prediction using whole-genome sequencing data"
bioRxiv 187542; doi: https://doi.org/10.1101/187542 Preprint journal clubs
• Meaningful exercise: send feedback to authors to improve their paper • Teach students how to write a review
Prachee Avasthi at the University of Kansas Medical Center draws material for her “Analysis of Scientific Papers” course exclusively from preprint servers. She’s generously shared https://www.authorea.com/users/8850/articles/198235- welcome-to-prereview her syllabus and introductory slide deck, and the students’ reviews can be found on the Winnower.
asapbio.org/10-ways preprintjc.org #ASAPbio 20 Preprint feedback can inform journal decisions
“In addition, the journal reserves the right--but is not obligated--to consider the comments made to manuscripts posted to preprint servers and factor these comments into final decisions at any stage of the peer review process.”
http://www.fasebj.org/site/misc/edpolicies.xhtml#Preprint_Submissions The dark side of comments
“Internet comment sections are in decline everywhere you look. They are mocked, ridiculed, despised. Many websites have closed them already; others have seen their comments become a racist, sexist bog of eternal stench from which any reasonable person is best advised to stay away.” “Women, in particular, get far too many comments questioning our competence, implying that we might not know the basic literature, that we might not really understand our own results, that said results might turn out to be false or trivial if only someone qualified had a look, or some such. We’re also subject to gendered standards of “professionalism” that do not allow us to respond in kind and give as good as we get. But if you tell me that men, too, can get inane, confused, or malicious comments–why, yes, I agree. More reason to refrain from making the arXiv more like YouTube.”
https://ilaba.wordpress.com/2016/04/10/arxiv-comments-and-quality-control/ Faster match- making
@ASAPbio_ | #ASAPbio | @jessicapolka Preprint servers as a marketplace for editors
@ASAPbio_ | #ASAPbio | @jessicapolka Preprint servers as a marketplace for editors
academickarma.org/unsubmitted
@ASAPbio_ | #ASAPbio | @jessicapolka Preprint servers as a marketplace for editors
“we now have a dedicated team of editors who will focus on identifying [preprints] that are potentially suitable for publication in PLOS Genetics.” *
* Bringing PLOS Genetics Editors to Preprint Servers Gregory S. Barsh, Casey M. Bergman, Christopher D. Brown, Nadia D. Singh, Gregory P. Copenhaver Published: December 1, 2016 https://doi.org/10.1371/journal.pgen.1006448
@ASAPbio_ | #ASAPbio | @jessicapolka But…
@ASAPbio_ | #ASAPbio | @jessicapolka Preprint adoption in perspective
PubMed: ~100,000 articles/month
All life sciences preprints: ~1,500/month
@ASAPbio_ | #ASAPbio | @jessicapolka Seven common concerns about preprints
#ASAPbio 29 Concern 1: I’m going to get scooped ie: preprints are public but not obviously well-respected
Paul Ginsparg, founder of arXiv on scooping:
“It can’t happen, since arXiv postings are accepted as date-stamped priority claims.”
asapbio.org/preprint- info/preprint-faq 39 responses (EMBO Postdoc Fellows meeting, 2016)
#ASAPbio 30 Concern 2: What about journals?
#ASAPbio 31 arXiv and journals coexist
5 years after posting, >70% of arXiv preprints can be matched to a journal article.
FIG. 1. Proportion of arXiv e-prints published in WoS-indexed journals, by arXiv specialty (1995–2011). Inset: Evolution of the proportion arXiv e-prints published in WoS-indexed journals (1995– 2011).
Larivière, V., Sugimoto, C. R., Macaluso, B., Milojević, S., Cronin, B. and Thelwall, M. (2014), arXiv E-prints and the journal of record: An analysis of roles and relationships. J Assn Inf Sci Tec, 65: 1157–1169. doi:10.1002/asi.23044 (preprint: https://arxiv.org/pdf/1306.3261#ASAPbio ) 32 Journals allowing preprints
& many more
#ASAPbio 33 How to check if a journal allows preprints
• Easy to use • Can contain detailed
en.wikipedia.org/wiki/List_of_academic_journals_by_preprint_policy information about policy
• Comprehensive sherpa.ac.uk/romeo
Always double check the policies on the journal website! #ASAPbio 34 As of BioRxiv preprints have been published in… December 2016 journal frequency unpublished 4918 Molecular Ecology 27 Biology Open 14 PLOS ONE 157 Nature Genetics 27 Frontiers in Microbiology 14 Bioinformatics 120 NeuroImage 25 GigaScience 14 Scientific Reports 103 PeerJ 24 ACS Synthetic Biology 13 eLife 97 Evolution 23 Biophysical Journal 13 Genetics 91 Genome Bio and Evolution 23 Journal of Neuroscience 13 PLOS Genetics 73 Nature Methods 21 Mol Biology Of The Cell 13 PNAS 70 Am J of Human Genetics 20 Molecular Ecology Res 13 PLOS Comp Biology 69 Systematic Biology 20 Nature 13 G3 66 F1000Research 18 Nature Neuroscience 13 Nucleic Acids Research 55 Cell Reports 17 Molecular Bio And Evolution 12 Genome Research 46 J of the Royal Soc Interface 16 PLOS Biology 12 BMC Genomics 44 J of Theoretical Biology 16 Proceedings B 12 Mol Bio and Evolution 38 PLOS Neg Trop Diseases 16 Science 12 BMC Bioinformatics 34 Genome Medicine 15 Development 11 …… https://github.com/MWSchmid/crawlBiorxiv/blob/master/journalCounts.csv #ASAPbio 35 ICMJE (International Committee of Medical Journal Editors) recommendations clarified
http://www.icmje.org/recommendations/ Page 8 #ASAPbio 36 Concern 3: We can’t be trusted to share work prior to peer review
#ASAPbio 37 Sharing work before peer review is common practice
• arXiv has been operating for over a quarter of a century • Preprints are used in many other field like social science (SSRN) and economics (RePEC) • Reputation is important
Flickr/NASA Goddard
#ASAPbio 38 Should preprints be cited?
• What is a citation? • Peer review improves, but does not perfect • Policies that disallow citations invite plagiarism
http://fossilsandshit.com/should-we-cite-preprints/
#ASAPbio 39 Concern 4: How should preprints be covered in the media?
Cell phones & cancer Vaccines & autism
#ASAPbio 40 Concern 5: How can we ensure ethical disclosure of data?
Screening standards/expectations are needed
#ASAPbio 41 Concern 6: How should preprints be licensed?
Language removed as of August 2017 “To maximize the impact of an “Creative Commons licenses interim research product, the are not compatible with the NIH strongly encourages [PNAS] License to Publish. awardees to select a Creative Authors are requested to Commons Attribution (CC- select the “no reuse” BY) license or dedicate their distribution/reuse option.” work to the public domain. ”
We have established a task force on licensing
ASAPbio.org/licensing https://www.nature.com/news/biologists-debate-how-to-license-preprints-1.22161 #ASAPbio 42 Concern 7: what is a preprint, & how to find it?
#ASAPbio 43 Current policies vary
• Elsevier: “anywhere at any time” • Wiley: “non-commercial servers” • Nature: “community preprint servers such as arXiv and bioRxiv” • NIH: • Content is findable, accessible, interoperable and reusable. • Interim product metadata, including usage statistics, are open, and easy to access by machines and people (e.g. via application program interfaces). • Content is easy to use by machines and people. This access is both a function of permission (e.g. use of Creative Commons licenses) and technology (e.g. application program interfaces). • Policies about plagiarism, competing interests, misconduct and other hallmarks of reputable scholarly publishing are rigorous and transparent. • Records of changes to the product are maintained, and users have clear ways to cite different versions of the product. • Links to the published version, if available. • A robust archiving strategy that ensures long-term preservation and access.
@ASAPbio_ | #ASAPbio | @jessicapolka Promoting best practices
• Access • Licensing • Pay & registration walls • Assistive technology • Machines • Preservation • Metadata & links to versions/records • Screening & moderation policies • Scope • Pre-screening • Removal
@ASAPbio_ | #ASAPbio | @jessicapolka Tools & resources
@ASAPbio_ | #ASAPbio | @jessicapolka asapbio.org/preprint-info
@ASAPbio_ | #ASAPbio | @jessicapolka Promoting preprint awareness through peers
asapbio.org/asapbio-ambassadors Asapbio.org/stickers
@ASAPbio_ | #ASAPbio | @jessicapolka Thank you!
• ASAPbio board asapbio.org • Ron Vale (Founder/President) @ASAPbio_ • Cynthia Wolberger • Jaime Fraser #ASAPbio • Harold Varmus • Daniel Colon-Ramos [email protected] • Tony Hyman • Harlan Krumholz @jessicapolka • Dick Wilder (non-voting)
@ASAPbio_ | #ASAPbio | @jessicapolka