DOCSLIB.ORG

Defining Predatory Journals and Responding to the Threat They Pose: a Modified Delphi Consensus Process

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Defining Predatory Journals and Responding to the Threat They Pose: a Modified Delphi Consensus Process Open access Original research BMJ Open: first published as 10.1136/bmjopen-2019-035561 on 9 February 2020. Downloaded from Defining predatory journals and responding to the threat they pose: a modified Delphi consensus process Samantha Cukier ,1 Manoj Lalu ,1,2 Gregory L Bryson,1,2 Kelly D Cobey,1,3 Agnes Grudniewicz,4 David Moher 1,3 To cite: Cukier S, Lalu M, ABSTRACT Strengths and limitations of this study Bryson GL, et al. Defining Objective To conduct a Delphi survey informing a predatory journals and consensus definition of predatory journals and publishers. ► An international group of stakeholders participated responding to the threat Design This is a modified three- round Delphi survey they pose: a modified Delphi in a consensus process to define predatory journals delivered online for the first two rounds and in- person for consensus process. BMJ Open and publishers. the third round. Questions encompassed three themes: 2020;10:e035561. doi:10.1136/ ► Both empirical evidence and a previous scoping re- (1) predatory journal definition; (2) educational outreach bmjopen-2019-035561 view were used to generate items used in this sur- and policy initiatives on predatory publishing; and (3) vey process. ► Prepublication history and developing technological solutions to stop submissions to ► The last round was held in-person, which allowed additional material for this predatory journals and other low- quality journals. paper are available online. To for discussion and elaboration of items. Participants Through snowball and purposive sampling view these files, please visit ► Although representative of a diverse group of stake- of targeted experts, we identified 45 noted experts in the journal online (http:// dx. doi. holders, participation in the Delphi was limited. predatory journals and journalology. The international org/ 10. 1136/ bmjopen- 2019- ► Inclusion of a larger number of individuals may have group included funders, academics and representatives of 035561). changed the results. academic institutions, librarians and information scientists, Received 06 November 2019 policy makers, journal editors, publishers, researchers Revised 20 December 2019 involved in studying predatory journals and legitimate Accepted 16 January 2020 journals, and patient partners. In addition, 198 authors fail to follow usual publication best practices, of articles discussing predatory journals were invited to including peer review and editorial oversight.2 participate in round 1. These journals have self- interest as a goal, and Results A total of 115 individuals (107 in round 1 and are often motivated to accept as many articles http://bmjopen.bmj.com/ 45 in rounds 2 and 3) completed the survey on predatory as possible to profit from article processing journals and publishers. We reached consensus on 18 items out of a total of 33 to be included in a consensus charges (APCs) which are common in OA definition of predatory journals and publishers. We came journals. It is becoming increasingly difficult to consensus on educational outreach and policy initiatives to distinguish articles published in predatory on which to focus, including the development of a single journals from legitimate journals, as preda- checklist to detect predatory journals and publishers, and tory journals are also finding their way into © Author(s) (or their public funding to support research in this general area. We trusted sources such as PubMed.3 employer(s)) 2020. Re- use identified technological solutions to address the problem: on September 24, 2021 by guest. Protected copyright. permitted under CC BY- NC. No Despite increasing attention to the 4–8 commercial re- use. See rights a ‘one- stop- shop’ website to consolidate information on problem of predatory publishing, there is and permissions. Published by the topic and a ‘predatory journal research observatory’ no agreed- on definition of what constitutes a BMJ. to identify ongoing research and analysis about predatory predatory journal9 (however, since the time 1 journals/publishers. Centre for Journalology, Ottawa of publication, a new definition of preda- Hospital Research Institute, Conclusions In bringing together an international group tory journals and publishers was established Ottawa, Ontario, Canada of diverse stakeholders, we were able to use a modified 10 2Anesthesiology and Pain Delphi process to inform the development of a definition through this Delphi process ). The absence Medicine, The Ottawa Hospital, of predatory journals and publishers. This definition will of a consensus and operationalised definition Ottawa, Ontario, Canada help institutions, funders and other stakeholders generate makes it difficult to accurately identify and 3 School of Epidemiology and practical guidance on avoiding predatory journals and evaluate the problem. Without a definition, Public Health, University of publishers. Ottawa Faculty of Medicine, funders and academic institutions struggle Ottawa, Ontario, Canada to generate practical guidance or policy to 4Telfer School of Management, ensure their members do not publish in pred- University of Ottawa, Ottawa, INTRODUCTION atory journals. Without appropriate attention Ontario, Canada Predatory journals pose a serious threat to to the problem of predatory publishing, the Correspondence to legitimate, open access (OA) journals and to quality of scholarly communication is at risk; 1 Dr David Moher; the broader scientific community. They pose this includes the risk to researchers, academic dmoher@ ohri. ca as authentic OA journals; however, they often institutions and funders, whose credibility Cukier S, et al. BMJ Open 2020;10:e035561. doi:10.1136/bmjopen-2019-035561 1 Open access BMJ Open: first published as 10.1136/bmjopen-2019-035561 on 9 February 2020. Downloaded from may be questioned, and/or patients who will have given Modified Delphi process of their time in hopes of improving interventions or treat- We used online SurveyMonkey software (http://sur vey- ments, when in all likelihood these data would not be monkey. com) to deliver rounds 1 and 2 of our Delphi used.4 survey electronically. Participants were invited via an This paper is part of a programme of scholarship that email which included key information about the study, its aims to establish a consensus definition of predatory purpose, and how it would inform a consensus definition journals and publishers and establish ways in which the of predatory journals and directions for future research. research community should respond to the problem. Rounds 1 and 2 were available online for 3 weeks each. Cobey and colleagues9 reported on the first stage of Two reminders were sent to participants on days 7 and 14. the programme, which was a scoping review to iden- Round 3 was conducted at our Predatory Summit, using tify possible characteristics of predatory journals. The the Poll Everywhere software (http://www. pollevery- authors found that no consensus definition existed and where.com), where participants could respond to survey there was a great deal of heterogeneity in the characteris- questions through live polling, watch results and partici- tics found. In this study, the second stage of the research pate in a face- to- face discussion. programme, we used the characteristics identified from For each of the questions, participants were asked to the scoping review to generate a consensus definition of respond on a 9-point Likert scale (1: strongly disagree; predatory journals and also suggested ways the research 9: strongly agree). We chose 80% agreement as the cut- community should respond to the problem. off for consensus based on findings from a systematic Here we present the details of this modified three- review of Delphi studies.13 We considered consensus to round Delphi consensus study. A related paper describing be reached if 80% of respondents scored the question the consensus statement reached on predatory journals is within the top third (score 7–9 to include) or bottom described elsewhere.10 third (score 1–3 to exclude) of the 9- point scale. Round 1 METHODS Participants ranked the importance of all questions via Prior to commencing this study, a protocol was drafted the online survey. We asked participants for any addi- and was posted on the Open Science Framework (https:// tional comments they wished to provide on each question osf. io/ z6v7f/). using free- text boxes. The Delphi method is a structured method to elicit opinions on given questions from a group of experts Round 2 and stakeholders.11 12 It is especially useful when exact Based on the results and comments from round 1, the knowledge is not available. The participants respond research team removed questions that reached consensus, anonymously to questionnaires that sequentially incor- eliminated or modified ambiguous questions, and http://bmjopen.bmj.com/ porate feedback and are refined. Following each round, included additional questions driven by comments from average group responses are provided to each respon- participants. For example, we received suggestions from dent, allowing them to reconsider their own views on the several participants proposing that we adjust the ques- topic. This is generally performed through electronic tion on collaborator roles and their ranked importance survey; however, for our modified Delphi, the final round in helping to solve the problem of predatory journals. As was held through a face- to- face meeting. a result, we added two additional collaborator roles that could be ranked: researchers and
Load more

Recommended publications
  • Does the Inclusion of Grey Literature Influence Estimates of Intervention Effectiveness Reported in Meta-Analyses?
    ARTICLES Does the inclusion of grey literature influence estimates of intervention effectiveness reported in meta-analyses? Laura McAuley, Ba’ Pham, Peter Tugwell, David Moher Summary Introduction A meta-analysis is multifactorial. Decisions need to be Background The inclusion of only a subset of all available made about how to handle various factors, such as evidence in a meta-analysis may introduce biases and language of publication, quality, and publication status, at threaten its validity; this is particularly likely if the subset of the individual study level. Within the domain of included studies differ from those not included, which may be publication status, a major factor to consider is the the case for published and grey literature (unpublished inclusion of grey literature (ie, studies that are studies, with limited distribution). We set out to examine unpublished, have limited distribution, and/or are not whether exclusion of grey literature, compared with its included in bibliographical retrieval system).1 inclusion in meta-analysis, provides different estimates of The inclusion of grey literature in a meta-analysis may the effectiveness of interventions assessed in randomised help to overcome some of the problems of publication trials. bias, and provide a more complete and objective answer to the question under consideration. However, it has been Methods From a random sample of 135 meta-analyses, we reported that only 31% of published meta-analyses identified and retrieved 33 publications that included both include grey literature.2 This omission may be because the grey and published primary studies. The 33 publications nature of grey literature makes its exclusion more contributed 41 separate meta-analyses from several disease convenient; it is difficult to retrieve, it is frequently areas.
    [Show full text]
  • Peer- Reviewers
    Identifying Patterns and Motivations of ‘Mega’ Peer- Reviewers Danielle Rice ( [email protected] ) McGill University https://orcid.org/0000-0001-5615-7005 Ba’ Pham University of Toronto Justin Presseau Ottawa Health Research Institute: Ottawa Hospital Research Institute Andrea Tricco Unity Health Toronto David Moher Ottawa Hospital Research Institute Research Keywords: Peer review, characteristics of reviewers, Publons Posted Date: May 6th, 2021 DOI: https://doi.org/10.21203/rs.3.rs-480702/v1 License: This work is licensed under a Creative Commons Attribution 4.0 International License. Read Full License Page 1/12 Abstract Background. The demand for peer reviewers is disproportionate to the supply and availability of reviewers. Identifying the factors associated with peer review behaviour can allow for the development of solutions to manage the growing demand for peer reviewers. The objective of this research was to identify factors associated with completing a large number of peer reviews in a given year. Methods. A case-control study design was used to examine factors associated with individuals completing at least 100 peer reviews (‘mega peer reviewers’) from January 2018 to December 2018 as compared to a control group of peer reviewers completing between 1 and 18 peer reviews over the same time period. Data was provided by Publons, which offers a repository of peer reviewer activities in addition to tracking peer reviewer publications and research metrics. A series of independent sample t- tests and chi-square tests were conducted comparing characteristics (e.g., number of publications, number of citations, word count of peer review) of mega peer reviewers to the control group of reviewers.
    [Show full text]
  • Harvard Thesis Template
    Diagnostic Test Accuracy Systematic Reviews: Evaluation of Completeness of Reporting and Elaboration on Optimal Practices Jean-Paul Salameh Thesis submitted to the University of Ottawa in partial Fulfillment of the requirements for the Degree of Master of Sciences in Epidemiology School of Epidemiology and Public Health Faculty of Medicine University of Ottawa © Jean-Paul Salameh, Ottawa, Canada, 2019 1 Preface All of the work presented henceforth was conducted under the supervision of Dr. David Moher and Dr. Mathew McInnes at the University of Ottawa. No Research Ethics Board approval was required for any the presented projects and associated methods. Chapter 2 has been published in Clinical Chemistry [Salameh JP, Moher D, Thombs BD, McGrath TA, Frank R, Dehmoobad Sharifabadi A, Kraaijpoel N, Levis B, Bossuyt PM, McInnes MDF. Completeness of Reporting of Systematic Reviews of Diagnostic Test Accuracy Based on the PRISMA-DTA Reporting Guideline. Clin Chem. 2018 Sep 20. doi: 10.1373/ clinchem.2018.292987]. Following the thesis defence minor revisions were made to the chapter. I was the lead investigator, responsible for all major areas of concept formation, data collection and analysis, as well as manuscript composition. Moher D was involved in the early stages of concept formation and contributed to manuscript edits. All the remaining authors contributed to the data collection process and manuscript edits. McInnes MDF was the supervisory author on this project and was involved throughout the project in concept formation and manuscript composition. Chapter 3 is an original, unpublished work by Salameh JP, Moher D, Bossuyt PM, McGrath TA, Thombs BD, Hyde CJ, Macaskill P, Deeks J, Leeflang M, Korevaar D, Whiting P, Taikwongi Y, Reitsma JB, Cohen JF, Frank RA, Hunt HA, Hooft L, Rutjes AWS, Willis BH, Gatsonis C, Levis B, and McInnes MDF.
    [Show full text]
  • Counting Publications and Citations Is Not Just Irrelevant
    Editorial Br J Sports Med: first published as 10.1136/bjsports-2020-103146 on 24 December 2020. Downloaded from research processes (or research funding) Counting publications and citations is with subsequent positive outcomes, whereas other frameworks emphasise not just irrelevant: it is an incentive the social interactions and networks that develop between scientists and non- that subverts the impact of academic stakeholders during a research project.4 Frameworks that evaluate clinical research research impact have merit. However, many are hampered by their theoret- 1 2,3 4 ical underpinning and lack empirical Fionn Büttner , Clare L Ardern , Paul Blazey , validation. The absence of field-specific Serenna Dastouri,5 Heather A McKay ,6 David Moher,7,8 frameworks (eg, in sport and exercise 4,9 medicine) is notable for fields in which Karim M Khan scientists receive large grants from major federal and international funding agen- NOT EVERYTHING THAT CAN BE lead scientists to neglect other important cies with the expectation of subsequently COUNTED COUNTS areas. As federal and international health demonstrating research impact. More than one million scientists publish research funding agencies increasingly peer- reviewed research each year.1 demand that research should have Health research strives to generate new impact beyond academia, researchers discoveries or consolidate existing knowl- and academic institutions must adapt. NOT EVERYTHING THAT COUNTS CAN edge to benefit the lives of humans. But We aim to draw the sport and exer- BE COUNTED: WHY EVALUATING does published health research impact cise medicine community’s attention to RESEARCH IMPACT IS CHALLENGING patients, policy, the economy, or society? the concept of research impact, high- At least three challenges confront the task Common metrics that are purported light existing ways of assessing research of reliably evaluating research impact.
    [Show full text]
  • Official Title Evaluating the Impact of Assessing During
    Official title Evaluating the impact of assessing during peer review the CONSORT checklist submitted by authors: protocol for a randomised controlled trial NCT Number NCT03751878 Date of the document 15/05/2019 Title (SPIRIT i1*) Evaluating the impact of assessing during peer review the CONSORT checklist submitted by authors: protocol for a randomised controlled trial. *All items corresponding to Standard Protocol Items: Recommendations for Interventional Trials (SPIRIT) and key applicable items of the Guidelines for the Content of Statistical Analysis Plans are referenced in brackets. Authors David Blanco1, 2 (ORCID: 0000-0003-2961-9328), Sara Schroter3 (ORCID: 0000-0002-8791-8564), Adrian Aldcroft3 (ORCID: 0000-0003-0106-720X), David Moher4 (ORCID: 0000-0003-2434-4206), Isabelle Boutron5 (ORCID: 0000-0002-5263-6241), Jamie J Kirkham6**(ORCID: 0000-0003-2579-9325) and Erik Cobo1**(ORCID: 0000-0002-3534-5602). 1Statistics and Operations Research Department, Universitat Politècnica de Catalunya - Barcelona Tech, Barcelona, Spain. 2INSERM, U1153 Epidemiology and Biostatistics Sorbonne Paris Cité Research Center (CRESS), Methods of Therapeutic Evaluation of Chronic Diseases (METHODS) Team, Paris Descartes University, Sorbonne Paris Cité, Paris F-75014, France. 3The BMJ, London, UK 4Centre for Journalology, Ottawa Hospital Research Institute, Ottawa, Canada. 5Centre d’Epidémiologie Clinique, Paris Descartes University, Paris, France. 6Department of Biostatistics, University of Liverpool, Liverpool, UK. Correspondence to David Blanco; [email protected]
    [Show full text]
  • Organizers' Biography
    Organizers’ Biography Greg Bryson, MD, FRCPC, MSc is an Associate Professor and Vice Chair for Research in the Department of Anesthesiology and Pain Medicine at the University of Ottawa. He is an Associate Scientist at the Ottawa Hospital Research Institute and the Chair of the Canadian Anesthesiologists' Society's Research Advisory Committee. He is the Deputy Editor-In-Chief of the Canadian Journal of Anesthesia. Kelly Cobey, PhD is an Investigator in the Centre for Journalology. In her capacity as Publications Officer she provides educational outreach on best practice in academic biomedical publishing. Kelly also consults with researchers one-to-one to provide feedback on research designs and reports. She actively contributes to research on journalology topics, including projects related to predatory journals and research ethics. Kelly is a member of EQUATOR Canada, an Adjunct Professor in the School of Epidemiology, Public Health and Preventative Medicine at the University of Ottawa, and an Honorary Researcher at The University of Stirling. She obtained her PhD in Psychology (University of Groningen, The Netherlands) and has an MRes in Biology (University of Liverpool, England) and BSc in Psychology and Biology (McMaster University, Canada). Prior to her current post Kelly worked as a Lecturer at The University of Stirling (Scotland), and held a Fyssen Research Fellowship (University of Paris North, France). Agnes Grundniewicz, PhD is an Assistant Professor at the Telfer School of Management, University of Ottawa. She received her Ph.D. in Health Services Research from the Institute of Health Policy, Management & Evaluation at the University of Toronto. She holds a Scientist appointment at the Institut du savoir Montfort and is an affiliate investigator at the Bruyère Research Institute.
    [Show full text]
  • An Extension to the PRISMA Statement for Reporting Literature Searches in Systematic Reviews Running Title: PRISMA-S Extension
    PRISMA-S: An Extension to the PRISMA Statement for Reporting Literature Searches in Systematic Reviews Running Title: PRISMA-S Extension Melissa L. Rethlefsen, MSLS, AHIP Ana Patricia Ayala, MISt, AHIP (Corresponding Author) Research Services Librarian Associate Dean, George A. Smathers Libraries Gerstein Science Information Centre Fackler Director, Health Science Center Libraries University of Toronto University of Florida [email protected] [email protected] ORCID: 0000-0002-3613-2270 ORCID: 0000-0001-5322-9368 David Moher, PhD Shona Kirtley, MA, MSc Centre for Journalology, Clinical Epidemiology Senior Research Information Specialist Program, Ottawa Hospital Research Institute UK EQUATOR Centre The Ottawa Hospital, General Campus, Centre Centre for Statistics in Medicine (CSM) for Practice Changing Research Building Nuffield Department of Orthopaedics, 501 Smyth Road, PO BOX 201B, Ottawa, Rheumatology and Musculoskeletal Sciences Ontario, Canada, K1H 8L6 (NDORMS) ORCID 0000-0003-2434-4206 University of Oxford [email protected] Botnar Research Centre Windmill Road Matthew J. Page, PhD Oxford OX3 7LD Research Fellow [email protected] School of Public Health and Preventive ORCID: 0000-0002-7801-5777 Medicine, Monash University, Melbourne, Australia. Dr. Siw Waffenschmidt ORCID: 0000-0002-4242-7526 Head of the information management unit Jonathan B. Koffel, MSI Institute for Quality and Efficiency in Health Emerging Technology and Innovation Strategist Care, Cologne, Germany [email protected] University of Minnesota ORCID: 0000-0001-6860-6699 [email protected] ORCID: 0000-0003-1723-5087 PRISMA-S Group* (see acknowledgements) 1 PRISMA-S Extension Rethlefsen, Kirtley, Waffenschmidt, Ayala, Moher, Page, Koffel, PRISMA-S Group Abstract Background: Literature searches underlie the foundations of systematic reviews and related review types.
    [Show full text]
  • Supporting Information: PRISMA-Ecoevo
    Supporting Information: PRISMA-EcoEvo 1 Supporting Information 2 Preferred Reporting Items for Systematic reviews and 3 Meta-Analyses in ecology and evolutionary biology: A 4 PRISMA extension 5 6 Rose E. O’Dea1, Malgorzata Lagisz1, Michael D. Jennions2, Julia Koricheva3, Daniel W.A. 7 Noble1,2, Timothy H. Parker4, Jessica Gurevitch5, Matthew J. Page6, Gavin Stewart7, David 8 Moher8, Shinichi Nakagawa1 9 1 Evolution & Ecology Research Centre and School of Biological and Environmental 10 Sciences, University of New South Wales, Sydney, NSW, 2052, Australia. 11 2 Research School of Biology, 46 Sulivans Creek Road, Australian National University, 12 Canberra, 2600, Australia. 13 3 School of Biological Sciences, Royal Holloway University of London, Egham, Surrey, 14 TW20 0EX, U.K. 15 4 Department of Biology, Whitman College, Walla Walla, WA 99362, U.S.A. 16 5 Department of Ecology and Evolution, Stony Brook University, Stony Brook, New York 17 11794-5245, U.S.A. 18 6 School of Public Health and Preventative Medicine, Monash University, Melbourne, VIC, 19 3004, Australia 20 7 School of Natural and Environmental Sciences, Newcastle University, Newcastle upon 21 Tyne, NE1 7RU, U.K. 22 8 Centre for Journalology, Clinical Epidemiology Program, Ottawa Hospital Research 23 Institute, General Campus, 501 Smyth Road, Room L1288, Ottawa, ON K1H 8L6, Canada. 24 ORCID IDs 25 Rose E. O’Dea: 0000-0001-8177-5075 26 Malgorzata Lagisz: 0000-0002-3993-6127 27 Michael D. Jennions: 0000-0001-9221-2788 28 Julia Koricheva: 0000-0002-9033-0171 29 Daniel W.A. Noble: 0000-0001-9460-8743 30 Timothy H. Parker: 0000-0003-2995-5284 31 Jessica Gurevitch: 0000-0003-0157-4332 32 Matthew J.
    [Show full text]
  • An Extension to the PRISMA Statement for Reporting Literature Searches in Systematic Reviews* Melissa L
    174 ORIGINAL INVESTIGATION DOI: dx.doi.org/10.5195/jmla.2021.962 PRISMA-S: an extension to the PRISMA statement for * reporting literature searches in systematic reviews Melissa L. Rethlefsen, AHIP; Shona Kirtley; Siw Waffenschmidt; Ana Patricia Ayala, AHIP; David Moher; Matthew J. Page; Jonathan B. Koffel; PRISMA-S Group See end of article for authors’ affiliations. Background: Literature searches underlie the foundations of systematic reviews and related review types. Yet, the literature searching component of systematic reviews and related review types is often poorly reported. Guidance for literature search reporting has been diverse and, in many cases, does not offer enough detail to authors who need more specific information about reporting search methods and information sources in a clear, reproducible way. This document presents the PRISMA-S (Preferred Reporting Items for Systematic reviews and Meta-Analyses literature search extension) checklist, and explanation and elaboration. Methods: The checklist was developed using a three-stage Delphi survey process, followed by a consensus conference and public review process. Results: The final checklist includes sixteen reporting items, each of which is detailed with exemplar reporting and rationale. Conclusions: The intent of PRISMA-S is to complement the PRISMA Statement and its extensions by providing a checklist that could be used by interdisciplinary authors, editors, and peer reviewers to verify that each component of a search is completely reported and, therefore, reproducible. Keywords: systematic reviews; reporting guidelines; search strategies; literature search; information retrieval; reproducibility This article has been approved for the Medical Library Association’s Independent Reading Program <http://www.mlanet.org/page/independent-reading-program>.
    [Show full text]