Scientists Are Becoming Increasingly Frustrated by How Long It Seems to Take to Publish Papers

Home , Journal of Cell Science, Physical Review E

Scientists are becoming increasingly frustrated by how long it seems to take to publish papers. But is it really getting worse? THE WAITING GAME BY KENDALL POWELL

hen Danielle Fraser first indexed in the PubMed database that had listed preprint model to accelerate publishing. “We submitted her paper for submission and acceptance dates (see go.nature. need a fundamental rethinking of how we do publication, she had little com/zjdqhn). His study, done for Nature, this,” Vosshall says. idea of the painful saga that found no evidence for lengthening delays2: lay ahead. the median review time — the time between THE PITCH She had spent some submission and acceptance of a paper — has In March 2012, Stephen Royle, a cell biologist 18 months studying thousands hovered at around 100 days for more than 30 at the University of Warwick, UK, started on W of fossil species spread across years (see ‘Paper wait’). But the analysis comes a publication mission of his own. His latest North America from the past 36 million years, with major caveats. Not all journals — includ- work answered a controversial question about and now she had an intriguing result: animal ing some high-profile ones — deposit such how cells sense that chromosomes are lined populations were spread widest across lati- time-stamp data in PubMed, and some jour- up before dividing, so he first sent it to Nature tudes in warm, wet climates. Her work, crucial nals show when a paper was resubmitted, rather Cell Biology (NCB), because it is a top journal to earning her PhD at Carleton University in in his field and an editor there had suggested Ottawa, Canada, might be used to make predic- he send it after hearing Royle give a talk. It tions about the response of mammals to climate was rejected without review. Next, he sent it to change — a key question in ecology today. So, “It takes forever Developmental Cell. Rejected. His next stop, the with her PhD adviser’s encouragement, she sent Journal of Cell Biology, sent the paper out for it to Science in October 2012. to get the work review. It came back with a long list of necessary Ten days later, the paper was rejected with revisions — and a rejection. a form letter. She sent it to another prestig- out, regardless Royle and his lab spent almost six months ious journal, the Proceedings of the National doing the suggested experiments and revis- Academy of Sciences. Rejected. Next, she tried of the journal.” ing the paper. Then he submitted the updated Ecology Letters. Bounced. “At this point, I manuscript to Current Biology. Rejected. EMBO definitely was frustrated. I hadn’t even been than submitted for the first time. “Resetting the Journal. Reviewed and rejected. reviewed and I would’ve loved to know how to clock is an especially pernicious issue,” Him- Finally, in December 2012, he submitted it to improve the paper,” recalls Fraser. “I thought, melstein says, and it means that the analysis the Journal of Cell Science (JCS), where it was ‘Let’s just get it out and go to a journal that will might be underestimating publication delays. reviewed. One reviewer mentioned that they assess the paper’.” Some data suggest that wait times have had already assessed it at another journal and In May 2013, she submitted the paper to increased within certain subsets of journals, thought that it should have been published Proceedings of the Royal Society B, considered such as popular open-access ones and some then. They wrote that the work was “beauti- a high-impact journal in her field. The journal of the most sought-after titles. At Nature, the fully conducted, well controlled, and conserva- sent it out for review — seven months after her median review time has grown from 85 days tively interpreted”. A second reviewer said that initial submission to Science. “Finally!” Fraser to just above 150 days over the past decade, it should not be published. The editor at JCS thought. What she didn’t know was that she had according to Himmelstein’s analysis, and at decided to accept it. The time between first sub- taken only the first steps down the long, bumpy PLoS ONE it has risen from 37 to 125 days over mission to Nature Cell Biology and acceptance road to publication: it would take another roughly the same period. at JCS was 317 days. It appeared online another three submissions, two rejections, two rounds Many scientists find this odd, because they 53 days later3. The work went on to win the JCS of major revisions and numerous drafts before expect advances in digital publishing and the prize as the journal’s top paper for 2013. the paper would finally appear. By that point, proliferation of journals to have sped things up. Despite the accolade, Royle says that the she could hardly bear to look at it. They say that journals are taking too long to multiple rejections were demoralizing for his Fraser’s frustration is widely shared: research- review papers and that reviewers are request- student, who had done the experiments and ers are increasingly questioning the time it takes ing more data, revisions and new experiments needed the paper to graduate. He also thinks to publish their work. Many say that they feel than they used to. “We are demanding more and that the paper deserved the greater exposure trapped in a cycle of submission, rejection, more unreasonable things from each other,” says that comes from publication in a more prestig- review, re-review and re-re-review that seems Vosshall. Journal editors counter that science ious journal. “Unfortunately, the climate at the to eat up months of their lives, interfere with job, itself has become more data-rich, that they work moment is that if papers aren’t in those very top grant and tenure applications and slow down to uphold high editorial and peer-review stand- journals, they get overlooked easily,” he says. the dissemination of results. In 2012, Leslie ards and that some are dealing with increasing And Royle, who has done several publication- Vosshall, a neuroscientist at the Rockefeller numbers of papers. They also say that they are time analyses and blogged about what he found, University in New York City, wrote a commen- taking steps to expedite the process. has shown that this experience is not unusual. tary that lamented the “glacial pace” of scientific Publication practices and waiting times When he looked at the 28 papers that his lab had publishing1. “In the past three years, if anything, also vary widely by discipline — with social published in the previous 12 years, the average it’s gotten substantially worse,” she says now. “It sciences being notoriously slow. In physics, the time to gestate from first submission to publi- takes forever to get the work out, regardless of pressure to publish fast is reduced because of cation was the same as a human baby — about the journal. It just takes far too long.” the common practice of publishing preprints 9 months (see go.nature.com/79h2n3). But is the publication process actually — early versions of a paper before peer review But how much of these delays were his own becoming longer — and, if so, then why? To find — on the arXiv server. Some of the loudest doing? To publish the chromosome paper, out, Nature examined some recent analyses on complaints about publication delays come Royle indulged in the all-too-familiar practice time to publication — many of them performed from those in biological fields, in which com- of journal shopping: submitting first to the most by researchers waiting for their own work to see petition is fierce and publishing in prestigious prestigious journals in his field (often those with the light of day — and spoke to scientists and journals can be required for career advance- the highest impact factor) and then working his editors about their experiences. ment. This month, a group of more than 70 sci- way down the hierarchy. (Nature Cell Biology’s The results contain some surprises. Daniel entists, funders, journal editors and publishers current impact factor is 19; JCS’s is 5.) Journal Himmelstein, a computational-biology are meeting at the Howard Hughes Medical impact factor or reputation are widely used by graduate student at the University of Cali- Institute campus in Chevy Chase, Maryland, scientists and grant-review and hiring commit-

ILLUSTRATION BY MATT MURPHY MATT BY ILLUSTRATION fornia, San Francisco, analysed all the papers to discuss whether biologists should adopt the tees as a proxy for the quality of the paper. On

Some scientists complain that publishing papers takes too long, PAPE WAIT but data show a complex picture. EVIEW TIME P ODUCTION TIME An analysis of all papers in PubMed up to 2015 with listed submission and The same analysis of Pubmed papers suggests that the time between acceptance acceptance dates suggests that the median time from submission to acceptance and publication has dropped, probably because technology has improved. has hovered at around 100 days, although it has gone up at some journals. 75 200 A few journals with quick review times started in 150 50 2000, lowering the median.

100 25 50 1980: 4,353 journals 2015: 9,045 journals in the PubMed database. in the PubMed database. to publication (days) to acceptance (days) 0 0 Median time from acceptance Median time from 1980 1990 2000 2010 2000 2005 2010 2015 Median time from submission Median time from Date accepted Date published online 5 REF. DATA: HIMMELSTEIN; TIME: DANIEL PRODUCTION ROYLE;

WAIT TIMES BY IMPACT FACTO DATA PE PAPE STEPHEN TIMES: WAIT HIMMLSTEIN; REVIEW TIME: DANIEL SOURCES: An analysis of PubMed papers published in 2013 suggests that journals with the An analysis of biology papers published in 3 journals showed that the number of lowest and highest impact factors have the longest review times. panels in experimental gures jumped between 1984 and 2014, a hint that the amount of data per paper is increasing. 160 Print Supplemental

120 40

80 20 40 Number of panels to acceptance (days) 0 0 0–1 1–2 2–3 3–4 4–5 5–10 10–20 20–30 30–50 1984 2014 1984 2014 1984 2014 Median time from submission Median time from Impact factor Cell Nature Journal of Cell Biology the flip side, critics say that editors seek out the organization of Europe’s leading life scientists that say, ‘1-millionth draft’!” She persevered, splashiest papers to boost their publication’s and publisher of the EMBO Journal. “Nobody making more revisions to meet the reviewer’s impact factor, something that encourages jour- has the courage to say, we, as a funding organi- demands, and in June 2014 submitted the nal shopping, increases rejection rates and adds zation, or we, as a tenure committee, are not paper to PLoS ONE for a third time. Success! to the wait time. Journal editors reject this; Ritu going to look at where you publish as opposed The paper4 was published online 23 months Dhand, Nature editorial director in London, to what you publish.” after she had first sent it to Science. The long says that Nature’s policy of selecting original, And the obsession with prestigious journals peer-review and revision process did improve important work “may lead to citation impact is just one source of delay — as Fraser, who was the paper, Fraser says now. “It was really much and media coverage, but Nature editors aren’t battling to publish her paper on ancient animal better.” But did the main conclusion of the paper driven by those considerations”. populations — was about to found out. change? “Not really.” How much time does journal shopping add? Last year, Chris Hartgerink, a behavioural- In the analysis of his group’s research papers, PEER REVIEW sciences graduate student at Tilburg University Royle found that more than half were shopped By October 2013, a full year had passed since in the Netherlands, ran an analysis of the Public around, and that this consumed anywhere from Fraser had first submitted her paper to a jour- Library of Science (PLOS) family of journals a few days to more than eight months. He went nal, and she had pretty much stopped caring since the first one launched in 2003. (He chose on to analyse all the papers published in 2013 about impact factor. By this point, the paper the journals largely because they make the data that are indexed in PubMed, and examined had spent two months in review at Proceedings easily accessible, and because he was waiting for whether higher impact factor correlated with of the Royal Society B, before coming back with a paper to be published in PLoS ONE.) He found longer median publication times. He found an mixed reports — and a rejection. So Fraser that the mean review time had roughly doubled inverted bell-shaped curve — the journals with decided to try PLoS ONE, a journal that says it in the past decade, from 50–130 days to 150–250 the lowest and highest impact factors had longer will publish any rigorous science, regardless of days, depending on the journal (see go.nature. review times than did those in the middle. For its significance, scope or anticipated citations. com/s3voeq). And when Royle looked at eight the vast majority of those in the middle, review It has an impact factor of 3, and a reputation for journals that had published cell-biology papers times stood at around 100 days — matching rapid publication. over the past decade, he found that publication Himmelstein’s analysis. Those with the very PLoS ONE sent the paper out to a single times had lengthened at seven of them, mostly highest impact factors (30–50) had a review reviewer. Two months later, Fraser got a deci- because review times had stretched out. time of 150 days, supporting the idea that sion letter that essentially stated that the paper One contention is that peer reviewers now pitching a paper to a series of top journals could was rejected but might be eligible for re-review ask for more. When Ron Vale, a cell biologist result in significant delays in publication. if the suggested revisions were made. She made at the University of California, San Francisco, Many scientists, editors and publishers the revisions, adding citations and a small analysed biology papers that had been published have long acknowledged that journal name is amount of reanalysis. In March 2014, she resub- in Cell, Nature and the Journal of Cell Biology, a flawed measure of the quality and value of mitted the manuscript, which PLoS ONE sent in the first six months of 1984 and compared a piece of research — but the problem shows out to a different reviewer. Another two months that with the same period in 2014, he found no signs of going away. “Where your paper is passed before she received the new review: that both the average number of authors and published doesn’t say anything about you, your major revisions, please. the number of panels in experimental figures paper’s impact or whether it’s right or wrong,” “I’m just happy they didn’t tell me to go away,” rose by 2–4 fold5. This showed, he argued, that says Maria Leptin, director of EMBO, an recalls Fraser. “I do have e-mails from the time the amount of data required for a publication

150 | NATURE | VOL 530 | 11 FEBRUARY 2016 © 2016 Macmillan Publishers Limited. All rights reserved FEATURE NEWS had gone up, and Vale suspects that much of the in production has halved since the early 2000s, and anyone can read and comment on the added data come from authors trying to meet falling to a stable median of 25 days. paper. “The minute a research story gets into reviewers’ demands. Scientists grumble about Several new journals and online publishing the public domain, it benefits from the collective overzealous critics who always seem to want platforms have promised to speed up the pro- power of different brains looking at a problem,” more, or different, experiments to nail a point. cess even more. PeerJ, a family of journals that says Vale. What’s more, proponents say, preprint “It’s very rare for the revisions to fundamentally launched in 2013, is one of several that now publishing can simply be added onto the con- change a paper — the headline doesn’t change,” encourage open peer review, in which review- ventional publication process. F1000Research, Royle says. His analysis of his group’s publica- ers’ names and comments are posted alongside which launched in 2012, does this by publish- tion times showed that almost 4 months of the articles. The hope is that the transparency will ing papers first, then inviting open peer review average 9-month gestation was spent revising prevent unnecessary delays or burdensome and revision. papers for resubmission. revision requests from reviewers. Some scientists are going a step further, and Many scientists also blame journal editors, The biomedical and life-sciences journal using platforms such as GitHub, Zenodo and who, they say, can be reluctant to provide clear eLife launched in 2012 with a pledge to make figshare to publish each hypothesis, data collec- guidance and decisions to authors when reviews tion or figure as they go along. Each file can be are mixed — unnecessarily stringing out the given a DOI, so that it is citable and trackable. review and revision process. Journal heads disa- Himmelstein, who already publishes his papers gree, and say that their editors are accomplished “It’s very rare for as preprints, has been using the Thinklab plat- at handling mixed reviews. Cell editor-in-chief form to progressively write up and publish the Emilie Marcus in Cambridge, Massachusetts, the revisions to results of a new project since January 2015. “I says that editors at her journal take responsibil- push ‘publish’ and it gets a DOI with no delay,” ity for publication decisions and help authors to fundamentally he says. “Am I really gaining that much by pub- map out a plan for revisions. lishing [in a conventional journal]? Or is it bet- Technological advances mean that research ter to do what is fastest and most efficient to get now involves handling more and more data, change a paper.” your research out there?” editors say, and there is greater emphasis on But preprints and real-time digital publishing making that information available to the com- initial editorial decisions within a few days and platforms are no panacea. Vosshall says that munity. Marcus says that her journal is working to review papers quickly. Reviewers get strict many biologists are “terrified” of preprints to cut review times by, for example, increasing instructions not to suggest the ‘perfect experi- because they fear getting scooped by competi- the number of papers that go through only one ment’, and they can ask for extra analysis only tors or losing credit and intellectual-property round of revision — 14% of their papers did so if it can be completed within 2 months. Other- rights for their ideas. And even after preprint in 2015. In 2009, Cell also restricted the amount wise, the paper is rejected. Randy Schekman, publishing, scientists can still find themselves of supplemental material that could accompany a cell biologist at the University of California, slogging through peer review and chasing high- papers as a way to keep requests for “additional, Berkeley, and editor-in-chief of eLife, says that impact journals for a final publication to adorn unrelated experiments” at bay. these policies mean that more than two-thirds their CV. Vosshall says that the scientific com- PLOS executive editor Veronique Kiermer, of the journal’s accepted papers undergo just munity relies on conventional journals to serve based in San Francisco, declined to discuss the one round of review. as a ‘prestige filter’ so that important papers specifics of Fraser’s paper, but she called its total In a 2015 analysis, Himmelstein created are brought to the attention of the right read- review time of nine months an “outlier” and said a ranking by the median review time for all ers. Without them, “How do we find the good that it was “not ideal to have research being eval- 3,482 journals that had papers with time stamps stuff?” she asks. uated by a single person”. She acknowledges that in the PubMed database from January 2014 to For Fraser, her PLoS ONE publication proved PLoS ONE’s publication time has risen; one fac- June 2015 (see go.nature.com/sscrr6). PeerJ a success. When the paper was finally published tor is that the volume of papers has, too — from had a relatively fast time: 74 days after submis- after its almost-two-year wait, she got positive 200 in 2006 to 30,000 per year now — and it sion. At eLife, it took 108 days, and PLoS ONE responses, she says. It has been viewed nearly takes time to find and assign appropriate editors took 117. By comparison, Cell’s review time was 2,000 times, had 51 shares on Facebook and and reviewers. (PLOS used 76,000 reviewers 127 days; Nature’s was 173 days; PLoS Medicine Twitter and got 280 downloads. The publication in 2015.) Another, says Kiermer, is that the took 177 days; and Developmental Cell was also helped her to secure her current position number of essential checkpoints — including among the slowest of the popular biomedical — as a postdoctoral fellow at the Smithsonian competing-interest disclosures, animal-wel- journals, at 194 days. Marcus notes that com- Institution Museum of Natural History in fare reports and screens for plagiarism — have parison between journals is difficult because Washington DC. “I pretty much got the top increased in the past decade. “We’ll do every- the publications define received, revised and postdoc that I could have gotten.” thing we can both in terms of technology and accepted days differently, and that Developmen- Still, the whole process is not something she looking at workflows to bring these times to tal Cell places a high priority on timely review. wants to endure again — so these days, she tends publication down,” she says. to send her papers to mid-range journals that Dhand says that at Nature, too, editors find it PREPRINTS RECONSIDERED are likely to publish her work right away. “If my harder to find reviewers than in the past, “pre- One way for biologists to accelerate publication ultimate goal is to get a faculty job, I can’t afford sumably because there are so many more papers is by embracing preprints. These allow work to to wait two years on a single paper,” she says. ■ that need reviewing”. Himmelstein found that quickly receive credit and critique, says Bruno the number of papers in PubMed more than Eckhardt, associate editor of Physical Review E Kendall Powell is a writer based in Lafayette, doubled between 2000 and 2015, reaching and a theoretical physicist at the University of Colorado. nearly 1 million articles. Marburg in Germany. “It is almost like going 1. Vosshall, L. B. FASEB J. 26, 3589–3593 (2012). on Facebook — it means you are ready to go 2. Himmelstein, D. S. & Powell, K. Zenodo Repository TECHNOLOGY ADVANCES public,” he says. A preprint submitted to bioRxiv http://doi.org/bb95 (2016). Digital publishing may have had benefits in — a server run by Cold Spring Harbor Labora- 3. Cheeseman, L. P. et al. J. Cell Sci. 126, 2102–2113 shortening ‘production’ time — the time from tory in New York — is published online within (2013). 4. Fraser, D. et al. PLoS ONE 9, e106499 (2014). acceptance to publication — rather than time 24 hours and given a digital object identifier 5. Vale, R. D. Proc. Natl Acad. Sci. USA 112, 13439– in review. In Himmelstein’s analysis, time spent (DOI); subsequent revisions are time-stamped 13446 (2015).