Beyond the traditional research paper Ed Gerstner 印格致 Scientific Director, Springer , China 26th October 2017 1 Who are we? In May 2015, Nature Publishing Group, , and and Springer Science+Business Media merged to form .

Home of the world’s Springer publishes One of the world’s leading leading international academic journals and open access publishers, science journal, Nature books in all areas of founded in 2000, BMC (founded in 1869), Nature academic research now publishes around 300 Genetics, Nature Cell including fundamental scientific journals in Biology, Nature Materials, science, engineering, biology, medicine, and Nature , Nature mathematics, humanities, beyond, including Genome Communications, Scientific social science, economics Biology, BMC Biology & Reports, Scientific Data and more. BMC Medicine. and many others.

Beyond the traditional research paper | OA Week, Beijing | 26th October 2017 2 Why do we care about OPEN? In part, because OPEN is the future!

2000 2017

Beyond the traditional research paper | OA Week, Beijing | 26th October 2017 3 Why do we care about OPEN? Because it improves the IMPACT research Independent statistical analysis carried out by the Research Information Network of articles published in Nature Communications found that • Open Access articles are viewed three times more often than articles that are only available to subscribers. • Open Access articles are cited somewhat more than subscription articles. — Research Information Network on the effect of Open Access on citations and downloads of Nature Communications articles. http://www.nature.com/press_releases/ncomms-report.html

Beyond the traditional research paper | OA Week, Beijing | 26th October 2017 4 Why do we care about OPEN? Because Premier Li wants science to be more open

“Open access to scientific knowledge and the nurturing of next generation researchers are what are needed nowadays and fit well with our future direction.” “科学知识的开放获取和青年科技人才培养，这契合现 实需求、符合未来方向。”

— Li Keqiang, 29th May 2014

Beyond the traditional research paper | OA Week, Beijing | 26th October 2017 5 Why do we care about OPEN? Because it’s part of the solution to a critical problem facing science!

PloS Medicine 2005 doi: 10.1371/journal.pmed.0020124

David Dobbs, The Atlantic, 18th SEPT 2015

“Fifty-three papers were deemed 'landmark' studies. … Nevertheless, scientific findings were confirmed in only 6 (11%) cases.” — Nature 483, 531–533 (2012) doi:10.1038/483531a

Beyond the traditional research paper | OA Week, Beijing | 26th October 2017 6 The reproducibility crisis We asked researchers about it… • 52% of the researchers we surveyed believe there is a “crisis” of reproducibility. • 70% failed to reproduce another scientist’s experiment and more than 50% failed to reproduce an experiment of THEIR OWN. • Physicists and chemists most confident in published literature that life scientists. • Key culprits — selective reporting, pressure to publish, low statistical power or poor analysis. • One-third respondents had taken active measures to improve reproducibility in their labs in the past 5 years; often independent replication, and better Nature 533, 452–454 (2016); attention to documentation. doi:10.1038/533452a

Beyond the traditional research paper | OA Week, Beijing | 26th October 2017 7 The problem isn’t just fraud Although the lion share of retractions arise from misconduct, poor reproducibility does not

We are not talking about results that were: • Falsified — misconduct is a problem but it’s still in the minority. • Wrong — legitimate observations but subsequent work disproves the hypothesis — that’s how science is meant to work! We are talking about results that are: • Poorly described preventing verification — independent experimenter cannot observe the same results under similar conditions. • Overstated — failure to consider alternative explanations. • Misrepresented — data claimed to be more robust than they actually are, CHERRY PICKING! • Sloppy — failure to account for (or even consider) sources of error, poor use of statistics, poor controls. The problem isn’t restricted to open access journals! Subscription journals suffer just as much.

Beyond the traditional research paper | OA Week, Beijing | 26th October 2017 8 The problem isn’t too much OPEN but too little

An attempt to reproduce key results in eighteen Nature Genetics papers that reported microarray-based gene expression data found that: • Only two could be reproduced fully. • Six were reproduced partially. • Ten could not be reproduced at all.

“The main reason for failure to reproduce was data unavailability, and discrepancies were mostly due to incomplete data annotation or specification of data processing and analysis.” — Nature Genetics 41, 149–155 (2009).

Beyond the traditional research paper | OA Week, Beijing | 26th October 2017 9 The antiseptic power of sunlight

I regularly tell young researchers that one way to decide whether something it ethical or not is to ask themselves, “If this action were to become public knowledge, would I have any reason to feel uncomfortable or ashamed by it?” And if the answer is ‘yes’ or even ‘maybe’, don’t do it!

Poor science thrives in the dark. Good science thrives in the open.

Beyond the traditional research paper | OA Week, Beijing | 26th October 2017 10 More transparent reporting Publishing reproducibility checklists for the world to see

From 1st June 2017, all life science papers that are published in Nature and all other Nature research journals will be accompanied by a reporting summary that contains details of experimental design, reagents, and statistical analysis. (2017). 8 546, Nature

Beyond the traditional research paper | OA Week, Beijing | 26th October 2017 11 More transparent reporting http://www.nature.com/authors/policies/checklist.pdf

Beyond the traditional research paper | OA Week, Beijing | 26th October 2017 12 BUT… … even if all individual studies were well described, there’s still a problem

CC-BY-NC: xkcd.com. Adapted from https://xkcd.com/882/

Beyond the traditional research paper | OA Week, Beijing | 26th October 2017 13 https://xkcd.com/882/ NC: xkcd.com. Adapted from - BY - CC

Beyond the traditional research paper | OA Week, Beijing | 26th October 2017 14 https://xkcd.com/882/ NC: xkcd.com. Adapted from - BY - CC

Beyond the traditional research paper | OA Week, Beijing | 26th October 2017 15 https://xkcd.com/882/ NC: xkcd.com. Adapted from - BY - CC

Beyond the traditional research paper | OA Week, Beijing | 26th October 2017 16 What’s the moral of the story? ANY journal that requires IMPACT will be biased towards FALSE POSITIVES If 20 different groups investigate the question, “Do jelly beans cause acne?”, at least one group should observe a connection with a confidence of greater than 95%. Even if there is no connection! Those groups that find no connection are unlikely to publish their negative results because they are unlikely to get any credit for doing so. But the group that sees a positive connection WILL almost certainly submit their result to a high impact journal. And without any possible access to all the negative results that have been obtained, there is a good chance the journal will publish it.

Which means that ANY journal that requires IMPACT (not just high impact, but more than sound science) will unintentionally encourage false positives.

Beyond the traditional research paper | OA Week, Beijing | 26th October 2017 17 How do we fix this? Assess science on its OWN MERITS and not on WHERE it is published

WE need to stop using journal impact factors as a proxy for assessing research. For many years we at Nature have been arguing against judging research by the impact factors of the journals in which it is published. Which is why Nature Research and BMC signed the San Francisco Declaration on Research Assessment (http://www.ascb.org/dora/). 日 25 月 5 年 2017 报 日 技 科

Beyond the traditional research paper | OA Week, Beijing | 26th October 2017 18 How do we fix this? Recognize that robust, incremental science is as important as high-impact science

Ensure that researchers get adequate credit for publishing reliable, reproducible, robust science in journals that aren’t necessarily restricted to publishing high impact science.

Beyond the traditional research paper | OA Week, Beijing | 26th October 2017 19 How do we fix this? We need to go beyond the traditional research paper BMC Research Notes BMC Research Notes publishes non-traditional but still scientifically useful research outputs that cannot be considered as full research or methodology articles. This includes, but is not limited to: • research proposals, • null and negative results, • additions to established methods, • updates to or confirmations of previous work, That is, for that experiment that didn’t quite work out; that algorithm that didn't provide the results you were expecting; and other hard-learnt lessons in the field.

Beyond the traditional research paper | OA Week, Beijing | 26th October 2017 20 Registered Reports Ensuring that the question and how it is answered is as important as the answer Extraordinary results can make referees LESS critical of the experiments that produced them. Registered Reports are a new class of research paper that puts the scientific question and the design of the experiments that attempt to answer the question, first!

Stage 1 Stage 2 • Is the hypothesis well founded? • Did the experiments follow the protocol? • Are the methods and proposed analyses • Did positive controls succeed? feasible and sufficient? • Are the conclusions justified by the data? • Is the study well-designed?

Beyond the traditional research paper | OA Week, Beijing | 26th October 2017 21 Registered Reports Going beyond the traditional research paper Registered Reports are now being accepted at eight Springer Nature journals: • Nature Human Behaviour • Cognitive Research: Principles & Implications • BMC Biology • Attention, Perception and Psychophysics • BMC Medicine • Journal of Business and Psychology • BMC Ecology • Journal of Cognitive Enhancement

Results-Free Review BMC Psychology is trialing yet another departure from conventional , known as Results-Free Review. The approach is similar to that of Registered Reports but conducted after the experiments are complete. Stage 1 Stage 2

In-principle Results-free review Complete review Publication acceptance

Beyond the traditional research paper | OA Week, Beijing | 26th October 2017 22 But what about the results themselves? The most important currency in science isn’t scientific papers but DATA!

Beyond the traditional research paper | OA Week, Beijing | 26th October 2017 23 Open research is more productive research! Researchers who make their data open, publish more papers A study of 7,000 NSF and NIH research projects in social sciences found that: • Principal investigators who shared their data either formally using an archive or informally by some other means produced twice as many papers (6 papers , median average) compared to those who did not share (3 papers, median average). • Research teams who archived their data generate almost three times as many papers (8 papers, median Pienta, A. M, Alter, G. & Lyle, average) compared o those who didn’t (3 papers, J. The Enduring Value of Social median average). Science Research: The Use and Reuse of Primary • Archived research generated a mean average of Research Data. Presented at 2 additional papers by researchers outside the teams Organisation, Economics and that collected the original data. Non-archived research Policy of Scientific Research generated no additional papers. workshop, Torino, Italy (April, 2010). Open is much more productive than closed.

Beyond the traditional research paper | OA Week, Beijing | 26th October 2017 24 Open research has more impact! Papers published with open data are cited more often Gene expression microarray papers that link to open data receive 9% to 30% more citations than those that don’t. 60% — Piwowar & Vision (2013) https://doi.org/10.7717/peerj.175 28-50% Astronomy papers that link to open data receive 50%

20% more citations than those that don’t. 40% — Henneken & Accomazzi (2011) https://arxiv.org/abs/1111.3618 35% 9-30% Astrophysics papers that link to open data receive 30%

28% to 50% more citations than those that don’t. 20% — Dorch et al. (2015) https://arxiv.org/abs/1511.02512 20%

Paleoceanography papers that link to open data 10% receive 35% more citations than those that don’t. — Sears et al (2011) 0% Gene Astronomy Astrophysics Palaeoceanography https://figshare.com/articles/Data_Sharing_Effect_on_Article_Citat expression ion_Rate_in_Paleoceanography/1222998/1 microarrays

Open has more impact than closed.

Beyond the traditional research paper | OA Week, Beijing | 26th October 2017 25 The first step towards more sharing of data is BETTER MANAGEMENT OF DATA!

More than 40 research funders globally have policies or mandates requiring archiving of data as a condition of grants, including: • National Science Foundation (NSF) • National Institutes of Health (NIH) • Wellcome Trust • Bill and Melinda Gates Foundation Some of these require data to be linked to publications including: • Research Councils UK (as part of open access policy) • Engineering and Physical Sciences Research Council (EPSRC)

Source: Hahnel, M. Global funders who require data archiving as a condition of grants. Figshare (2015). https://dx.doi.org/10.6084/m9.figshare.1281141.v1

Beyond the traditional research paper | OA Week, Beijing | 26th October 2017 26 So we’ve instituted a mandate Nature and BMC require all authors to tell readers how they can get hold of their data

In March 2016 we ran a pilot requiring authors of papers at five Nature journals to declare how readers can obtain the ‘minimal data set’ needed to interpret, replicate and build on the findings they report. This is now a requirement for all papers published in all Nature and BMC journals. As well as being an inherent good, we hope to: • Learn more about how different disciplines share data differently. Nature 536, 138 (20016). To learn more • Learn how we can help them meet the go to http://go.nature.com/2bf4vqn challenges they. • Raise awareness and encourage more research into the open.

Beyond the traditional research paper | OA Week, Beijing | 26th October 2017 27 BUT researchers need help! So we’re looking for better ways to help them open up their data Few researchers have any idea how to share their data effectively • 64% unsure about open licensing of research data 1 • 56% do not use a metadata standard 2 • 54% would like more guidance complying with funder policies 1 • 54% do not have enough time to make data available 1 • 45% unaware of a repository for some of their data 3 • 39% uncertain about meeting costs of making data open 1,2

1. Treadway et al. The state of open data. (2016). https://doi.org/10.6084/m9.figshare.4036398.v1 2. Tenopir et al. PLoS ONE 6: e21101 (2011). “What license is your open http://doi.org/10.1371/journal.pone.0021101 3. Nature Publishing Group survey (2014). data released under?” http://doi.org/10.6084/m9.figshare.1234052

Beyond the traditional research paper | OA Week, Beijing | 26th October 2017 28 Research Data Support Helpdesk Helping authors comply with open data mandates Support for authors: Information on the data policy of their target journal(s) • Identifying and using data repositories • Compliance with funders’ and institutions’ data sharing policies • Data reporting standards Support for editors: Information on best practice for implementing a journal’s data policies, including: • Good practice for data-literature integration • Advice on handling peer review of sensitive/clinical data • Help with identifying appropriate data repositories for a journal’s audience https://www.springernature.com/gp/authors/research-data-policy/helpdesk/

Beyond the traditional research paper | OA Week, Beijing | 26th October 2017 29 Data Support Services pilot Open data is often little more than a data dump Without adequate description, structure or metadata, open data is almost impossible to find. And it’s of limited use to anyone, even when it is found!

Beyond the traditional research paper | OA Week, Beijing | 26th October 2017 30 Data Support Services pilot After curation the data is not just more discoverable it’s more useful Data Support Services help researchers make their open data more valuable, with: • Links to associated, peer-reviewed publications, • Consistent titles and author names, • Clear citation information, • Files preview-able in browser, • Metadata for each file in the archive, • Contextual information, • Clear license/terms of use, • Dataset description/abstract, • Rich usage statistics.

Beyond the traditional research paper | OA Week, Beijing | 26th October 2017 31 Open access books at Springer Nature

ü Open access books are available immediately on publication to view and download

ü CC BY licence allows unrestricted reuse providing the author and original source are properly cited

ü All open access content published adheres to the high standards expected of all Springer Nature titles with the same rigorous peer- review process.

ü OA books receive the same publishing and promotional services as non-OA books.

ü Various book types accepted: monographs, edited collections, proceedings, protocols, short-form books, chapters

ü OA books are also available in print

Beyond the traditional research paper | OA Week, Beijing | 26th October 2017 32 The impact of OA books

Springer Nature OA books are downloaded 7 times more than non-OA books, and receive more citations and online mentions More to come in our white paper ‘The OA effect: How does open access affect the usage of scholarly books?’ • Quantitative and quantitative analysis of the performance and impact of OA books • Available November 2017 Almost 400 open access books published at Springer Nature!

Beyond the traditional research paper | OA Week, Beijing | 26th October 2017 33 Thank you! 谢谢！

Ed Gerstner 印格致 [email protected]

Follow us on WeChat! Interested in a career in publishing? We’re always looking for talented researchers who are interested in pursuing a career in scientific publishing to join us. For more see: www.springernature.com/editorial-and-publishing-jobs

Beyond the traditional research paper | OA Week, Beijing | 26th October 2017