The Importance of High-Quality Content: Curation and Re-evaluation in Scopus

2021 Scopus Only Day for Russia Editors

Mar 2021 Tracy Chen, Product Manager Scopus Various studies have indicated that there is an escalation in predatory journals, however, it is near impossible to determine the extent as they appear and disappear continually.

The term ‘predatory journals’ was coined by Jeffrey Beall in 2010 who acted as unofficial ‘watchdog’ of predatory publishing since Threat to science: then. Predatory journals Beall works with a binary classification in which a journal is considered either predatory or not. Decisions were not are on the rise systematically explained, and it is not possible to make a more detailed quantification of “predatoriness”.

There was criticism for Beall and the website eventually closed on 17 January 2017

In 2019 a group of researchers reached a consensus definition.* An important part of this statement is: “entities that prioritize self- interest at the expense of scholarship”.

*Source: Grudniewics et al. (2019) Predatory journals: no definition, no defence and Cukier et al (2020) Defining predatory journals and responding to the threat they pose: a modified Delphi consensus process Common warning signs* of predatory journals

Fake impact factors, incorrect addresses, misrepresentations of the editorial False or misleading board, false claims of indexing or membership of associations and misleading information claims about the peer review

An unprofessional-looking web page — with spelling or grammar mistakes or Deviation from best editorial irrelevant text — should also raise red flags. and publication practices

Little or no information about how editorial decisions are made, what fees apply, and peer review organized; absent contact information; no details about article Lack of transparency processing charges; editors and members of their editorial boards are often unverifiable.

Aggressive solicitation such as repeated e-mails, excessively flattering in tone Aggressive, indiscriminate (A clear warning sign is that the invitee’s expertise is outside the journal’s solicitation scope)

(*A warning sign does not in itself constitute evidence of predatory activity)

Source: https://www.nature.com/articles/d41586-019-03759-y Driving forces

• Publish or perish: For many academics, career progression depends on the research papers they publish.

• Technology: Easy to set up a website, spamming thousands of potential authors and receiving electronic payments

• Exploitation of the open access model: Pay-to-publish model misused

• Inexperience / Online environment: Working online without access to expertise to distinguish bogus journals etc. How Scopus enables better decision-making

Framework for Publication & Usage Global Trusted sources of Quality evaluation Metrics View dissemination local journals Scopus provides the Scopus provides Scopus provides wide Scopus provides a Scopus invites information and data arrays of metrics to data coverage representative, journals which comply model as assess various required to have an curated dataset of with selection criteria bibliographic source entities to facilitate overview of actual scholarly sources to apply for indexing for evaluation responsible use of global research status which is continually and provides material required to assess the metrics* and make decisions updated, and titles and capacity building quality of research without bias monitored and to inform journal output and make deselected if they are editors about best those decisions predatory or below practice standards Expert Curated content selection by the independent Content Selection & Advisory Board (CSAB)

• The CSAB is an independent board of subject experts from all over the world. • Comprised of 17 Subject Chairs. • Board members are chosen for their expertise in specific subject areas; many have (journal) Editor experience. Transparent Scopus selection criteria for serial content

1) All titles should meet all technical criteria in order to be considered for Scopus review: Roman script Publication ethics Peer-review English abstracts Regular publication references statement

2) Eligible titles are reviewed by the CSAB according to 14 selection criteria:

Journal Policy Quality of Content Journal Standing Regularity Online Availability

• Convincing editorial • Academic • Citedness of journal • No delay in • Content available concept/policy contribution to the field articles in Scopus publication schedule online • Type of peer-review • Clarity of abstracts • Editor standing • English-language • Diversity geographic • Quality and journal home page distribution of editors conformity with stated • Quality of home page • Diversity geographic aims & scope distribution of authors • Readability of articles Once a title is indexed in Scopus, the journal performance will be monitored on an ongoing basis Scopus policy / course of action

Poor quality journals have lower than average performance but could still be relevant to cover in Scopus, e.g.: • Niche journals - research published in these journals could still be of high quality and these journals do not necessarily need to be removed from Scopus.

Predatory journals are a threat to science and should be avoided to be covered in Scopus. • Usually, journals that are included in Scopus benefit from wider global visibility and resulting increase of impact and quality. However, sometimes this does not happen, and the journal may become predatory. • When making decisions about research, it is essential that these decisions are based on data that you can trust. Therefore, predatory journals are a threat to the integrity of Scopus and science in general. • Because predatory publishing is ill-defined and subject to personal interpretation, independent review of individual journals by academic subject experts in each field is essential. What Scopus does to address the issue

• Scopus, together with the CSAB, own the responsibility of curating content on an on-going basis as a defence against low quality and predatory journals.

• We have been addressing the issue for some years now and have developed a process of continuous monitoring in combination with re-evaluation by the independent CSAB.

• Validating concerns and to take a well-informed decision is complex and time-consuming. Additional information that shows the questionable integrity and quality is needed for validation.

• The decision to potentially exclude a journal from Scopus should not be taken lightly, given the ongoing risk of discontinuing legitimate sources, or excluding genuine articles published in bad journals. Identifying potential poor quality or predatory journals

All +25k journals in Scopus are monitored on ongoing basis and flagged for reevaluation based on:

• Our own observation or direct feedback from users and stakeholder’s publication concerns about the publishing standards or publication ethics of the journal or publisher are investigated.

• Metrics and benchmarks for publication output, citation impact and self-citations are used to identify journals that are underperforming compared to peer journals in their field.

• A machine learning tool analyzes the performance of journals according to aspects like output growth, changes in author affiliation, citation behavior, etc. to track outlier performance (=‘RADAR’)

• During their review, the CSAB can indicate whether any accepted title should be evaluated again in the future. This data is collected and further analyzed to ensure continuous curation. The re-evaluation process

Monitor Curate

Identify titles based on publication concerns, under performance, outlier performance or continuous curation. In-depth re-evaluation by the Content Selection & Advisory Board (CSAB) Flag Discontinued titles broken down by reason of identification (2016-2020) 990 titles re-evaluated

Reason of 44% 33% 12% 11% identification publication under outlier continuous concerns performance performance curation

Titles Discontinued 289 165 65 17

536 titles discontinued since 2016 What happens with journals for which the decision is made to discontinue?

• No new content is added to Scopus. • Content already indexed remains as a matter of scientific record and to ensure stability and consistency of research trend analytics. • In exceptional cases of proven severe unethical publication practice, content already indexed in Scopus may be removed. • CiteScore will no longer be given for discontinued titles.

An overview of all discontinued journals, including the last content indexed in Scopus, is available in the Discontinued Sources List on https://www.elsevier.com/solutions/scopus/how-scopus-works/content Collaborations with the community in 2020

❖ Collaboration with Russia Academy of Sciences ✓ Russia community and RAS provided a continuously added list of suspicious titles; ✓ Scopus to validate and send relevant titles for re-evaluation timely (within 1 month time); ✓ Till date, 59 titles were reported while 31 titles were discontinued or became inactive.

❖ Collaboration with Indian University Grants Commission (UGC) ✓ UGC provided a list of 73 suspicious titles ✓ Scopus validated all titles and sent 15 titles to re-evaluation. ✓ 12 titles were confirmed to be discontinued or inactive. ✓ Finally, the UGC Consortium for Academic and Research Ethics (UGC-CARE) approved list of journals includes all journals covered by Scopus. Scopus mandate and authority

Scopus is committed to creating a representative, curated dataset of scholarly content: • Overall journal selection based on journal-level data and performance • Monitoring and deselection of journals that are predatory or below standards

Scopus cannot interfere with editorial autonomy of journals: • Editorial decisions on quality of individual articles and conferences • (Scientific) content of the articles and abstracts included in the database • Plagiarism and other publication malpractice of individual articles • Authorship of the paper

Note: If publication malpractice is occurring knowingly and on a structural basis without policy to address and prevent such cases, Scopus will flag, re-evaluate and potentially discontinue titles “Whether or not a journal is predatory is not a binary decision; there is a broad spectrum of predatory journal behaviors. To assess whether a journal should be considered predatory, a number of parameters need to be considered, often in combination.”

– Prof. Jörg-Rüdiger Sack, Chair of CSAB and Subject Chair for Computer Sciences.

Prof Jörg-Rüdiger Sack, PhD “It is clearly important to remove journals that are predatory or adopt poor publishing practices from the Scopus database. It is less clear what to do with earlier content in the database, particularly as it may have arisen in periods when the journal was well managed and represents output from well conducted research.”

– Prof. Peter Brimblecombe, CSAB Subject Chair for Environmental Science Prof Peter Brimblecombe, PhD

https://www.elsevier.com/connect/the-guardians-of-scopus Thank you!

Updated Scopus Position Paper: https://www.elsevier.com/research- intelligence/resource-library/scopus-high-quality-content Scopus content info site: https://www.elsevier.com/solutions/scopus/how-scopus- works/content Scopus blog: http://blog.scopus.com Webinar series: http://blog.scopus.com/webinars Twitter: www.twitter.com/scopus Appendix Several flaws in two incriminating articles

• Nature published an article based on research looking into predatory publishing solely use Beall’s list as a definition for predatory journals. This article is based on a study from 2017

• Beall’s list has not been maintained since 2017 and is not authoritative about predatory publishing. It is not a good method to identify predatory journals

• All titles included in Beall’s list, were already re-evaluated by the CSAB, before 2017, as part of our ongoing re-evaluation program, 65% of them were discontinued from Scopus. The CSAB did not always come to the same conclusion as Beall.

• The title “Hundreds of ‘predatory’ journals indexed on leading scholarly database” is sensationalist and misleading. The articles do not use valid methodology to determine whether a journal is predatory.

• The article also does not acknowledge the rigorous evaluation and re-evaluation mechanisms that Scopus has in place

• The statement that Scopus indexing requires minimum quality criteria based either on bibliometrics or on what the journal declares about itself is untrue. There is an in-depth evaluation of each individual journal by the CSAB in all cases.

See also: https://blog.scopus.com/posts/the-importance-of-high-quality-content-in-scopus Selecting the right journal is important

The consequences of choosing a bad journal for good work: • Monetary costs for the author/institution • Reputational costs for the work and people involved • Negative impact on rankings and research assessment • Durability: no assurance of longevity of the paper Authors should submit their article to the right journal, for the right reasons • Delisting of the journal by indexing services • Possible legal consequences

Do your due diligence. Think Check Submit is a cross-industry initiative that provides simple guidelines for authors to assess a journal before submitting an article: https://thinkchecksubmit.org/ Other resources

• Position statement by Scopus and the CSAB on the importance of high quality content

• Webinar by Scopus and the CSAB on the importance of high quality content

• Scopus evaluation and re-evaluation process and criteria

• Scopus source list and Scopus discontinued sources list

• Consensus definition of predatory journals (article in Nature)

• COPE principles of transparency and best practice in scholarly publishing