Recommendations from the Reducing the Inadvertent Spread of Retracted Science: Shaping a Research and Implementation Agenda Project Jodi Schneider*, Nathan D. Woods, Randi Proescholdt, Yuanxi Fu, and the RISRS Team July 2021 *Corresponding author: [email protected] and [email protected] Table of Contents EXECUTIVE SUMMARY 3 Recommendations 4 INTRODUCTION 5 RISRS Project Goals and Assumptions 6 Framing the Problem 8 Scope of this Document 9 THE RISRS PROCESS 10 Scoping Literature Review 10 Interviews 11 Workshop, Dissemination, and Implementation 11 LITERATURE REVIEW AND CURRENT AWARENESS 11 Reasons for Retraction 11 Formats and Types of Retraction 13 Field Variation 15 Continued Citation of Retracted Papers: What Went Wrong? 20 Visibility of Retraction Status 24 Inconsistent Retraction Metadata 25 Quality of Retraction Notices 26 Literature Review Conclusions 28 DEFINING PROBLEMS AND OPPORTUNITIES 29 PROBLEMS AND OPPORTUNITIES DESCRIBED BY THE EMPIRICAL LITERATURE ON RETRACTION 29 Problem Themes Described by the Empirical Literature on Retraction 30 Opportunity Themes Described by the Empirical Literature on Retraction 32 Problem Definition Described by the Empirical Literature on Retraction 35 THEMES FROM STAKEHOLDER INTERVIEWS 36 Problem Frameworks based on Stakeholder Interviews 36 Contentious Themes based on Stakeholder Interviews 38 The Purpose of Retraction According to Stakeholder Interviews 38 Changing the Scholarly Record 39 The Harms of Retraction According to Stakeholder Interviews 39 The Character of Reform According to Stakeholder Interviews 40 SYNTHESIZING THE PROBLEMS AND OPPORTUNITIES 41 Aligning Opportunity Pathways 41 Defining the Scale and Scope of the Problem 42 RECOMMENDATIONS 43 1. Develop a Systematic Cross-industry Approach to Ensure the Public Availability of Consistent, Standardized, Interoperable, and Timely Information about Retractions. 44 Individual Recommendations for Consistent, Standardized, Interoperable, and Timely Information about Retractions 45 2. Recommend a Taxonomy of Retraction Categories/Classifications and Corresponding Retraction Metadata that can be Adopted by All Stakeholders. 46 Individual Recommendations for Taxonomy and Metadata 46 3. Develop Best Practices for Coordinating the Retraction Process to Enable Timely, Fair, and Unbiased Outcomes. 47 Individual Recommendations for Coordinating the Retraction Process 48 4. Educate Stakeholders About Publication Correction Processes including Retraction and about Pre- and Post-Publication Stewardship of the Scholarly Record. 48 Individual Recommendations for Educating Stakeholders 48 Researcher Education 48 Authors 49 Topic Editors 50 Editorial Education 50 IMPLEMENTATION OF THE RECOMMENDATIONS 52 1. Implementation Actions for Recommendation #1: Develop a systematic cross-industry approach to ensure the public availability of consistent, standardized, interoperable, and timely information about retractions. 52 2. Implementation Actions for Recommendation #2: Recommend a taxonomy of retraction categories/classifications and corresponding retraction metadata that can be adopted by all stakeholders. 54 3. Implementation Actions for Recommendation #3: Develop best practices for coordinating the retraction process to enable timely, fair, and unbiased outcomes. 54 4. Implementation Actions for Recommendation #4: Educate stakeholders about publication correction processes including retraction and about pre- and post-publication stewardship of the scholarly record. 55 1 RESEARCH AGENDA 56 1. Research Actions for Recommendation #1: Develop a systematic cross-industry approach to ensure the public availability of consistent, standardized, interoperable, and timely information about retractions. 57 2. Research Actions for Recommendation #2: Recommend a taxonomy of retraction categories/classifications and corresponding retraction metadata that can be adopted by all stakeholders. 58 3. Research Actions for Recommendation #3: Develop best practices for coordinating the retraction process to enable timely, fair, and unbiased outcomes. 58 4. Research Actions for Recommendation #4: Educate stakeholders about publication correction processes including retraction and about pre- and post-publication stewardship of the scholarly record. 60 5. Research Actions Related to RISRS Process Outcomes. 62 6. Research Actions Related to RISRS Citation Research. 63 CONCLUSIONS 64 Implementation Strategy 65 A Call to Implement and Collaborate 65 LIST OF APPENDICES 66 CRediT CONTRIBUTOR STATEMENT FOR THE RECOMMENDATIONS 66 ACKNOWLEDGEMENTS 66 BIBLIOGRAPHY 66 Note: Previous versions of this document appeared in draft as documents at F1000Research, as follows: Draft March 14, 2021 Schneider J, Woods ND, Proescholdt R et al. Reducing the inadvertent spread of retracted science: Shaping a research and implementation agenda [version 1; not peer reviewed]. F1000Research 2021, 10:211 (document) (https://doi.org/10.7490/f1000research.1118522.1) Draft, April 27, 2021 Schneider J, Woods ND, Proescholdt R et al. Reducing the inadvertent spread of retracted science: Shaping a research and implementation agenda [version 1; not peer reviewed]. F1000Research 2021, 10:329 (document) (https://doi.org/10.7490/f1000research.1118546.1) 2 EXECUTIVE SUMMARY Retracted research is published work that is withdrawn, removed, or otherwise invalidated from the scientific and scholarly record. This may occur for many different reasons, which can include error, misconduct, or fraud. Retracting research is intended to stop its continued citation and use, but many retracted papers continue to be used. Retracted research that is integrated into the scientific publication network via citations—either before or after retraction—enables the inadvertent propagation of potentially unsupported or fabricated data, fundamental errors, and unreproducible results, or can lead to misattribution of results or ideas (e.g., in cases of retraction due to dual publication, plagiarism, or ownership). Research over the past decade has identified a number of factors contributing to the unintentional spread of retracted research. Many retracted papers are not marked as retracted on publisher and aggregator sites, and retracted articles may still be found in readers’ PDF libraries, including in reference management systems such as Zotero, EndNote, and Mendeley. Most publishers do not systematically surveil bibliographies of submitted manuscripts, and most editors do not query whether a citation to a retracted paper is justified. When citing retracted papers, authors frequently do not indicate retraction status in bibliographies or in-text citations. The goal of the Reducing the Inadvertent Spread of Retracted Science: Shaping a Research and Implementation Agenda (RISRS) project is to develop an actionable agenda for reducing the inadvertent spread of retracted science. This includes identifying how the gatekeepers of scientific publications can monitor and disseminate retraction status and determining what other actions are feasible and relevant. The RISRS process included an exploratory environment scan, a scoping review of empirical literature, and successive rounds of stakeholder consultation, culminating in a three-part online workshop (October 26, November 9, and November 16, 2020) that brought together a diverse body of 70 stakeholders to engage in collaborative problem solving and dialogue. Workshop discussions were seeded by materials derived from stakeholder interviews and short original discussion pieces contributed by stakeholders. The online workshop resulted in a set of recommendations to address the complexities of retracted research throughout the scholarly communications ecosystem. Recommendations were iteratively updated and developed through a series of surveys and drafts as well as at a followup meeting online February 16, 2021. The RISRS team solicited feedback from presentations to NISOPlus, the Society for Scholarly Publishing, and the European Association of Science Editors. Implementation actions have started through a COPE task force on taxonomy and discussions about a proposed National Information Standards Organization (NISO) Work Item. We welcome your feedback via the project website https://infoqualitylab.org/projects/risrs2020/ or by email to [email protected]. We encourage you to disseminate these recommendations and to envision how you, in your role, and in collaborative partnerships, can make a difference. For instance, you might help form a professional working group to further develop or refine these recommendations; present about retraction and related issues at professional and academic meetings; take on an implementation or policy project; or outline further research to be conducted. 3 Recommendations 1. Develop a systematic cross-industry approach to ensure the public availability of consistent, standardized, interoperable, and timely information about retractions. 2. Recommend a taxonomy of retraction categories/classifications and corresponding retraction metadata that can be adopted by all stakeholders. 3. Develop best practices for coordinating the retraction process to enable timely, fair, unbiased outcomes. 4. Educate stakeholders about publication correction processes including retraction and about pre- and post-publication stewardship of the scholarly record. 4 INTRODUCTION Retracted research is published work that is withdrawn, removed, or otherwise invalidated from the scientific and scholarly record. An article may be retracted due to the unreliability of the