Some Journals Have Admitted to a Problem with Fake Research Papers

Home , Scandal (season 1), Scandal (season 2)

Feature THE BATTLE AGAINST PAPER MILLS Some journals have admitted to a problem with fake research papers. Now editors are trying to combat it. By Holly Else and Richard Van Noorden

hen Laura Fisher noticed sleuths have repeatedly warned that some guest speaker was Elisabeth Bik, a research-in- striking similarities between scientists buy papers from third-party firms tegrity analyst in California known for her skill research papers submitted to to help their careers. Rather, it was extraordi- in spotting duplicated images in papers, and RSC Advances, she grew suspi- nary that a publisher had publicly announced one of the sleuths who posts their concerns cious. None of the papers had something that journals generally keep quiet about paper mills online. authors or institutions in com- about. “We believe that it is a paper mill, so we Bik thinks there are thousands more of these mon, but their charts and titles want to be open and transparent,” Fisher says. papers in the literature. The RSC’s announce- looked alarmingly similar, says The RSC wasn’t alone, its statement added: ment is significant for its openness, she says. WFisher, the executive editor at the journal. “I “We are one of a number of publishers to have “It is pretty embarrassing that so many papers was determined to try to get to the bottom of been affected by such activity.” Since last Janu- are fake. Kudos to them to admit that they have what was going on.” ary, journals have retracted at least 370 papers been fooled.” A year later, in January 2021, Fisher retracted that have been publicly linked to paper mills, an At some journals that have had a spate of 68 papers from the journal, and editors at two analysis by Nature has found, and many more apparent paper-mill submissions, editors have other Royal Society of Chemistry (RSC) titles retractions are expected to follow. now revamped their review processes, aiming retracted one each over similar suspicions; Much of this literature cleaning has come not to be fooled again. Combating industrial- 15 are still under investigation. Fisher had about because, last year, outside sleuths pub- ized cheating requires stricter review: telling found what seemed to be the products of licly flagged papers that they think came from editors to ask for raw data, for instance, and paper mills: companies that churn out fake paper mills owing to their suspiciously simi- hiring people specifically to check images. scientific manuscripts to order. All the papers lar features. Collectively, the lists of flagged Science publishing needs a “concerted, coor- came from authors at Chinese hospitals. papers total more than 1,000 studies, the dinated effort to stamp out falsified research”, The journals’ publisher, the RSC in London, analysis shows. Editors are so concerned by the RSC said. announced in a statement that it had been the the issue that last September, the Committee victim of what it believed to be “the systemic on Publication Ethics (COPE), a publisher-ad- Paper-mill detectives production of falsified research”. visory body in London, held a forum dedicated In January 2020, Bik and other image detec- What was surprising about this was not the to discussing “systematic manipulation of tives who work under pseudonyms — Smut paper-mill activity itself: research-integrity the publishing process via paper mills”. Their Clyde, Morty and Tiger BB8 — posted, on a

mentions a paper mill — despite the RSC’s announcement that it thinks the articles did come from one. But Nature has tallied 370 articles retracted since January 2020, all from authors at Chinese hospitals, that either publishers or independent sleuths have alleged to come from paper mills (see ‘Fraud allegations’). Most were published in the past three years (see ‘Chinese hospital papers on the rise’). Publishers have added expressions of concern to another 45 such articles. Nature has identified a further 197 retractions of papers from authors at Chinese hospitals since the start of last year. These are not ones that have made it onto lists of potential publication-mill products, although some were flagged by sleuths for image concerns, often on the post-publication peer-review website PubPeer.

Industrialized cheating The problem of organized fraud in publishing is not new, and not confined to China, notes Catriona Fennell, who heads publishing services at the world’s largest scientific publisher, Elsevier. “We’ve seen evidence of industrialized cheating from several other countries, including Iran and Russia,” she told Nature last year. Others have also reported on Iranian and Russian paper-mill activities. In a statement this year to Nature, Elsevier said that its journal editors detect and prevent the publication of thousands of probable paper-mill submissions each year, although some do get through. China has long been known to have a problem with firms selling papers to researchers, says Xiaotian Chen, a librarian at Bradley Uni- versity in Peoria, Illinois. As far back as 2010, a blog run by science journalist Leonid Schnei- team led by Shen Yang, a management-studies der, a list of more than 400 published papers researcher then at Wuhan University in China, they said probably came from a paper mill. Bik warned of websites offering to ghostwrite dubbed it the ‘tadpole’ paper mill, because papers on fictional research, or to bypass of the shapes that appeared in the papers’ peer-review systems for payment. In 2013, western blot analyses, a type of test used to WE’RE WONDERING Science reported on a market for authorships ILLUSTRATION BY ANA KOVA ANA BY ILLUSTRATION detect proteins in biological samples. A spate HOW WE PROTECT on research papers in China. In 2017, China’s of media headlines followed. Throughout the Ministry of Science and Technology (MOST) year, the sleuths (not always working together) OURSELVES FROM said it would crack down on misconduct after posted spreadsheets of other suspect papers — a scandal in which 107 papers were retracted at picking up on similar features across multiple PUBLISHING THIS the journal Tumor Biology; their peer reviews studies. By March 2021, they had collectively had been fabricated and a MOST investigation listed more than 1,300 articles, by Nature’s STUFF.” concluded that some had been produced by tally, as possibly coming from paper mills. third-party companies. Journals started to look at the papers. Graf, director of research integrity at Wiley, Physicians in China are a particular target According to Nature’s analysis, around 26% which publishes JCB, said in January that the market because they typically need to publish of the articles that the sleuths alleged came publisher had completed investigations into research articles to gain promotions, but are so from paper mills have so far been retracted 73 papers identified by Bik and others, and had busy at hospitals that they might not have time or labelled with expressions of concern. Many found no reason to act on 11 of them. Seven to do the science, says Chen. Last August, the others are still under investigation. The Jour- others required corrections and 55 have been Beijing municipal health authority published nal of Cellular Biochemistry (JCB), for instance, retracted or will be retracted. a policy stipulating that an attending physi- announced in February1 that, last year, editors Publishers almost never explicitly declare cian wanting to be promoted to deputy chief investigated and retracted 23 of 137 papers on retraction notices that a particular study physician must have at least two first-author alleged to contain image manipulation. is fraudulent or was created by a company to papers published in professional journals; Journals did not identify problems with all order, because it is difficult to prove. None three first-author papers are required to of the papers that had been flagged. Chris of the RSC’s retraction notices, for instance, become a chief physician. These titles affect

Feature a physician’s salary and authority, as well as FRAUD ALLEGATIONS the surgeries they are allowed to perform, Since January 2020, independent sleuths and journals have flagged more than 1,000 published says Changqing Li, a former senior physician papers potentially linked to paper mills — companies that produce fraudulent articles. Hundreds of these papers have been retracted, corrected or labelled with an expression of concern (EOC). ANALYSIS and gastroenterology researcher at a Chinese hospital who now lives in the United States. Retracted EOC Corrected Paper-mill alleged; article so far unchanged “The effect is devastating,” says Li, about the impacts on Chinese science. “The literature environment published in Chinese is already ruined, since hardly anyone believes them or references studies from them.” 0 200 400 600 800 1,000 1,200 1,400 1,600 “Now this plague has eroded into the inter- Number of papers* national medical journals,” he adds. The fact *All papers include authors from Chinese hospitals. Another 197 articles (at least) with authors at Chinese hospitals have been retracted since January 2020. These are not ones that have made it onto lists of potential publication-mill products. Data collated by Nature up to 18 March. that people use paper mills also affects Chi- ORBETTERSCIENCE.COM, SOURCES: NATURE AND SCIENCEINTEGRITYDIGEST.COM na’s reputation globally, says Futao Huang, the publisher FEBS Press in Heidelberg, Ger- don’t seem to be linked to any of the author a Chinese researcher working at Hiroshima many, who screens incoming manuscripts for names. “Individually, these factors may not University in Japan. a number of journals, and helped the RSC with be problematic, but taken together they raise The prevalence of problem papers is leading its investigation. concerns and could be part of a pattern,” she some journal editors to doubt the submissions Potential signs of trouble include papers said. Editors at the forum also noted that a they get from Chinese hospital researchers. from different authors at different institu- manuscript-processing system, ScholarOne, “The increasing volume of this ‘junk science’ tions sharing similar features: western blots can flag up unusual activity when it picks up is wreaking havoc on the credibility of the with identical-looking backgrounds and sus- on submissions from the same computer. A research emanating out of China and increas- piciously smooth outlines, titles that seem ScholarOne alert was also instrumental in the ingly casting doubt upon legitimate science to be variations on a theme, bar charts with RSC’s investigation. from the region,” said a February 2021 edito- identical layouts that supposedly represent In February, Naunyn-Schmiedeberg’s Archives rial2 in the journal Molecular Therapy. different experiments, or identical plots of of Pharmacology said it had been affected by Several other editors echo these concerns flow cytometry analyses, which are used in paper mills. The journal published an editorial3 about the impact of paper mills. “They are studying cells. It seems that these manuscripts listing important features of paper-mill articles. undermining our confidence in the other man- are produced from common templates, with These included non-academic e-mail addresses uscripts received from Chinese groups,” says words and images slightly tweaked to make (which happen to be common with Chinese sci- Frank Redegeld, editor in chief of the European the papers look a little different. entists), authors’ inability to supply raw data Journal of Pharmacology, published by Elsevier. A particular problem is biomedical arti- when asked, and poor English. The journal is China’s science and education ministries cles that claim to investigate understudied retracting 10 studies, and it reports that around have taken steps to curb problematic publi- 5% of all its submissions are from paper mills. cation incentives. They published a notice Publishers and others battling paper mills last February telling research institutions suspect they are only seeing the tip of the ice- — including hospitals — not to promote or berg in the published literature. In part, that’s recruit researchers solely on the basis of the because similarities between images across numbers of papers they publish, and also studies might become obvious only when told them to stop paying cash bonuses for WE HAVE VERY LITTLE many papers are compared. Sleuths also know papers. And in August, China announced the INFORMATION ABOUT that features such as similar western blots introduction of measures to crack down on and flawed nucleotide sequences might be research misconduct, including attempts to THE PEOPLE OR the most obvious signs of paper-mill activity, curb independent contractors who fabricate says Bik. “There may be tonnes of other paper data on others’ behalf. (MOST didn’t respond COMPANIES DOING mills that have done a better job of hiding it,” to Nature’s request for comment on the scale she says. Editors at the COPE forum said they’d of the problem or the impact of its measures.) THIS.” seen paper mills in areas such as computer Some Chinese researchers think these sciences, engineering, humanities and social measures are beginning to work. Li Tang, who genetic regions that might be involved in sciences, for instance. researches science policy at Fudan University cancers. Jennifer Byrne, a molecular oncology The overall size of the paper-mill problem in Shanghai, China, is hopeful that submis- researcher at the University of Sydney, Aus- probably runs to thousands or tens of thou- sions from paper mills in China will fall in the tralia, specializes in exposing flawed papers sands of papers, Bik, Byrne and others think4. future — although she notes that the issue isn’t of this type, by spotting that their experimen- Graf, at Wiley, says it’s hard to estimate. “I don’t confined to Chinese research. tal details sometimes list incorrect nucleotide think it should be understated, I can’t say how Redegeld says he hasn’t yet seen a decrease sequences or reagents, so that the experi- big it is,” he says. “We have very little informa- in the number of suspected paper-mill manu- ments described cannot have taken place. tion about the people or companies doing this. scripts his journal receives, which he estimates Many of these papers are probably doctored I am exasperated by the situation, and that is to be around 15 a month. simply by switching around the type of cancer being polite.” or the genes involved in the study, says Byrne, “It’s detrimental to science as a whole Problem signs although it’s hard to prove they’re from paper because it makes science and scientists look Image-integrity sleuths and journal editors mills. “This problem of incorrect nucleotide unreliable,” says Christopher. Byrne has iden- have identified a range of features in manu- sequences in the literature is rampant,” she says. tified a different concern: she worries that by scripts that could be fingerprints of a paper At last September’s COPE forum, Bik rattled simply appearing in journals, fake studies that mill. “We’re wondering how we protect our- off other red flags for editors to watch out link genes to particular cancers can give the selves from publishing this stuff,” says Jana for, including papers from Chinese hospitals perception of activity in an area where there is Christopher, an image-integrity analyst at and manuscripts with e-mail addresses that none, and might be included in meta-analyses.

“People die from cancer — it is not a game. It have changed their scope to avoid subject publish there, which could be why some jour- is important that the literature describes the areas that seem to be a particular focus of nals seem to be more affected than others. work that takes place,” she adds. paper mills, the publisher says. And several One journal, the European Review for Medical publishers say many of their journals have and Pharmacological Sciences, has retracted Zombie papers updated their policies to require that authors 186 articles since January 2020, most of them Journal editors know that if they reject man- present the raw data behind their western flagged by Bik and Smut Clyde. “We were uscripts they suspect to be fabricated, that blots at the time of submission. Asking for raw shocked by these investigations,” says one of might not kill the paper forever. Fraudulent data is one of the main ways that publishers its editors in chief, Antonio Gasbarrini. manuscripts can be submitted to multiple tell editors to follow up when they think there Many journals are starting to employ ana- journals at the same time: so even if an editor might be something wrong with a manuscript. lysts to try to spot problems in manuscripts rejects it during peer review, they might see it But editors are aware that even raw data can be as they come in. Graf, for instance, says that published elsewhere. faked, especially if paper-mill firms catch on last year Wiley employed and trained 11 peo- This has happened to Christopher, who 3 that such requests are being made. ple to try to spot manipulated images across years ago saw alarming similarities in a clus- “Asking for raw data is not an absolute guar- 24 journals — focusing on the papers most ter of 13 research manuscripts submitted to 2 antee, as you can fake the data. It is a deterrent,” likely to be published. It hopes to expand the journals published by FEBS Press, where she says Sabina Alam, director of publishing integ- programme to more titles. worked. Their western blots seemed to be not rity and ethics at Taylor and Francis. One of its Publishers would like to automate some of only fabricated, but also similar, as if they’d journals, Artificial Cells, Nanomedicine, and this screening process. Many have teamed up been created by tweaking a template. The jour- Biotechnology, is investigating almost 100 pub- with research groups to develop software that nals rejected the manuscripts on her advice. lished papers alleged to be from paper mills. could detect duplicated images across pub- Christopher published a 2018 paper5 warning Alam also says that once they started lished papers, and, last May, an industry group of “systematic fabrication of scientific images” investigations, some authors quickly asked formed to try to set standards for these checks. and urged journals to invest in pre-publica- to withdraw their papers. Some sent raw data Software is improving but isn’t yet capable of tion image screening. She also noted that she’d in unreadable formats or without labels. In all looking through many papers on a massive seen some papers appear in other journals. these cases, journal editors say they’re not sure scale, says IJsbrand Jan Aalbersberg, the head Christopher told Nature that she tried to whether it’s correct to withdraw such articles, of research integrity at Elsevier, who chairs the privately raise the alarm about the papers. or to do something else — and are hoping for group. To do this would also require a giant In 2018, for instance, she and FEBS Letters’ guidance on this from COPE. Bik has pointed shared database of images that publishers managing editor advised the journal Cellular out that some journals have already allowed could check for duplication between papers. Physiology and Biochemistry that a paper it had authors to withdraw papers without stating That will come when software can handle it, published that year was probably fabricated; the reason for retraction. Aalbersberg predicts. it had been simultaneously submitted to FEBS COPE says it will update its existing guid- Suzanne Farley, Springer Nature’s Letters, which had rejected it. But the journal’s ance on how journals should deal with system- research-integrity director, based in London, publisher at the time, Karger in Basel, Switzer- atic manipulation of the publication process, says she thinks that there will be a fall in the pro- land, did not hear of any problem until 2020, and is also creating a task force of editors from portion of paper-mill submissions. “The paper when the paper was flagged up again in Bik and its membership to determine how the organi- mills are aware that publishers are getting bet- others’ ‘tadpole paper mill’ collection, along zation can provide better support on the issue. ter at detecting their submissions, and poten- with other papers in the journal. Karger is now tial paper-mill customers are aware that there investigating all these papers together with Path forwards are now more serious consequences of using the journal’s current publisher, says Christna Publishers say that they are limited in what the services,” she says. (Nature’s news team is Chap, Karger’s head of editorial development. they can do to share information between jour- editorially independent of its publisher.) In This year, Christopher again looked into the nals because even titles within the same stable the meantime, Farley says, there will be more 13 manuscripts that had been submitted to her are editorially independent of one another. retractions and expressions of concern. “We are journals. She found they had all been published They’re wary of sharing information between committed to cleaning house,” she says. in other journals; so far, only three have been titles or publishers about an author that could But Christopher worries that an arms race retracted and one has an expression of concern. be defamatory, and data-protection rules could develop if fraudsters get better at avoid- Many journals have changed their editori- hinder the sharing of authors’ personal data. ing obvious mistakes. One preprint posted to al-review processes to try to combat organized Once fraudsters know they can get a paper bioRxiv last year6, for instance, suggested that fraud. Some Elsevier journals, for instance, into a particular title, they might continue to artificial intelligence techniques could gener- ate fake western blots that were indistinguish- CHINESE HOSPITAL PAPERS ON THE RISE able from real ones. “I’m really worried about The number of English-language journal articles with authors from the sophistication going up,” she says. Chinese hospitals has risen roughly fifty-fold over two decades. 7% of all papers with Only Chinese authors International co-authors authors from China Holly Else is a reporter and Richard Van 40 Noorden is a features editor, both at Nature in London. 30 1. Behl, C. J. Cell. Biochem. https://doi.org/10.1002/ jcb.29906 (2021). 2. Frederickson, R. M. & Herzog, R. W. Mol. Ther. https://doi. 20 org/10.1016/j.ymthe.2021.02.011 (2021). 3. Seifert, R. Naunyn Schmiedebergs Arch. Pharmacol. 394, 431–436 (2021). 10 3% of all papers with 4. Byrne, J. A. & Christopher, J. FEBS Lett. 594, 583–589 authors from China (2020). 5. Christopher, J. FEBS Lett. 592, 3027–3029 (2018). Number of articles (thousands) 0 6. Qi, C., Zhang, J. & Luo, P. Preprint at bioRxiv https://doi.

SOURCE: LENS.ORG SOURCE: 1990 1992 1994 1996 1998 2000 2002 2004 2006 2008 2010 2012 2014 2016 2018 2020 org/10.1101/2020.11.24.395319 (2020).