F1000Research 2017, 6:208 Last updated: 04 MAR 2020 RESEARCH ARTICLE The impact factor of an open access journal does not contribute to an article’s citations [version 1; peer review: 2 approved] SK Chua1, Ahmad M Qureshi2, Vijay Krishnan3, Dinker R Pai4, Laila B Kamal1, Sharmilla Gunasegaran1, MZ Afzal1, Lahiru Ambawatta1, JY Gan 1, PY Kew1, Than Winn5, Suneet Sood 6 1Jeffrey Cheah School of Medicine and Health Sciences, Monash University Malaysia, Selangor, Malaysia 2Department of Public Health, Jeffrey Cheah School of Medicine and Health Sciences, Monash University Malaysia, Selangor, Malaysia 3All India Institute of Medical Sciences, New Delhi, India 4Ng Teng Fong General Hospital, JurongHealth, Jurong East, Singapore 5Department of Community Medicine, MAHSA University, Kuala Lumpur, Malaysia 6Department of Surgery, Jeffrey Cheah School of Medicine and Health Sciences, Monash University Malaysia, Selangor, Malaysia First published: 02 Mar 2017, 6:208 ( Open Peer Review v1 https://doi.org/10.12688/f1000research.10892.1) Latest published: 02 Mar 2017, 6:208 ( https://doi.org/10.12688/f1000research.10892.1) Reviewer Status Abstract Invited Reviewers Background 1 2 Citations of papers are positively influenced by the journal’s impact factor (IF). For non-open access (non-OA) journals, this influence may be due to version 1 the fact that high-IF journals are more often purchased by libraries, and are 02 Mar 2017 report report therefore more often available to researchers, than low-IF journals. This positive influence has not, however, been shown specifically for papers published in open access (OA) journals, which are universally accessible, and do not need library purchase. It is therefore important to ascertain if the 1 Eleftherios P Diamandis , University of IF influences citations in OA journals too. Toronto, Toronto, Ontario, Canada Methods 203 randomized controlled trials (102 OA and 101 non-OA) published in 2 Samiran Nundy , Sir Ganga Ram Hospital, January 2011 were included in the study. Five-year citations for papers New Delhi, India published in OA journals were compared to those for non-OA journals. Any reports and responses or comments on the Source papers were derived from PubMed. Citations were retrieved from Web of Science, Scopus, and Google Scholar databases. The article can be found at the end of the article. Thompson-Reuter’s IF was used. Results OA journals were found to have significantly more citations overall compared to non-OA journals (median 15.5 vs 12, p=0.039). The IF did not correlate with citations for OA journals (Spearman’s rho =0.187, p=0.60). The increase in the citations with increasing IF was minimal for OA journals (beta coefficient = 3.346, 95% CI -0.464, 7.156, p=0.084). In contrast, the IF did show moderate correlation with citations for articles published in non-OA journals (Spearman’s rho=0.514, p<0.001). The increase in the number of citations was also significant (beta coefficient = 4.347, 95% CI 2.42, 6.274, p<0.001). Conclusion It is better to publish in an OA journal for more citations. It may not be worth paying high publishing fees for higher IF journals, because there is minimal Page 1 of 10 F1000Research 2017, 6:208 Last updated: 04 MAR 2020 paying high publishing fees for higher IF journals, because there is minimal gain in terms of increased number of citations. On the other hand, if one wishes to publish in a non-OA journal, it is better to choose one with a high IF. Keywords bibliometrics, bibliometric analysis, information science, publications, literature based discovery, open access, Web of Science, Google Scholar Corresponding author: Suneet Sood ([email protected]) Competing interests: No competing interests were disclosed. Grant information: The author(s) declared that no grants were involved in supporting this work. Copyright: © 2017 Chua S et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. Data associated with the article are available under the terms of the Creative Commons Zero "No rights reserved" data waiver (CC0 1.0 Public domain dedication). How to cite this article: Chua S, Qureshi AM, Krishnan V et al. The impact factor of an open access journal does not contribute to an article’s citations [version 1; peer review: 2 approved] F1000Research 2017, 6:208 (https://doi.org/10.12688/f1000research.10892.1) First published: 02 Mar 2017, 6:208 (https://doi.org/10.12688/f1000research.10892.1) Page 2 of 10 F1000Research 2017, 6:208 Last updated: 04 MAR 2020 Introduction Articles were considered non-OA if the following three conditions A journal’s impact factor (IF) has long been used as a measure of were all fulfilled: the quality of a journal1. Today, the IF is used as a tool to assess 1. The publishing journal was not listed in PubMed’s OA subset researchers for employment, career promotion, and funding2–4. list; In the past, most libraries could possess only a limited number 2. The article was never made freely available by the journal; of journals, and librarians used the IF to decide which journals 3. The article was not self-archived (as determined by a careful web to buy3,5–7. Consequently, high IF journals were more likely to be search for the article). purchased, read, and cited. With low IF journals, availability was a constraint. Scientists, wanting a greater audience for their In other words, the non-OA article could, in theory, only be read research, preferred to publish in high IF journals. There was plenty by someone with a subscription. Within non-OA journals, we of evidence that publishing in a higher IF journal resulted in more excluded articles if their journals allowed free access to all articles citations8–13. any time after publication. We further excluded articles published in hybrid non-OA journals if over 20% of their articles were freely In contrast, at present, open access (OA) journals are universally available (for this, we counted 100 successive 2011 articles in that available. Libraries have no need to subscribe, and researchers can journal, and ensured that fewer than 20 were marked as freely access OA articles freely. Expectedly, OA publication is associated accessible). In other words, we attempted to ensure that the non- with increased citations14–19, so researchers are likely to prefer this OA journal was true non-OA, and its IF would properly represent path. What is not known is whether, within OA journals, increas- the IF of a non-OA journal (Figure 1). Finally, we also excluded ing IF is associated with increasing citations, as it is for non-OA articles if their journal did not have a measurable Web of Science journals. Yet this information is important, since cost of publishing IF for 2011. in an open access journal is high and increases with the journal’s IF. Should a researcher, or a sponsor, pay good money for publication The articles were scanned for citations as listed in Web of in a higher IF OA journal if the IF will not influence citations? Science, Scopus, and Google Scholar databases. The search period was extended up to 2016, allowing for five years of pub- We conducted a study to determine whether an OA journal’s IF lication time, with the assumption that citations over five years influences citations. provide a better estimate of the impact of a paper than citations over two years5. Only journal citations were included in the counts; Methods citations in books, theses, and government documents were We first conducted a pilot study to estimate required sample size. excluded to conform with the Web of Science policy21. We exported For this purpose, 57 randomized controlled trials (RCTs) were citation data from the three databases into .csv files, and imported extracted from PubMed, and scanned for citations as listed in the these into a Microsoft Excel sheet. Duplicates were excluded. Web of Science. PubMed was chosen to look for source articles Citations that appeared in two language versions of the same paper because most researchers start their search on PubMed20. Within were counted as one. this pilot group, for OA articles the mean citations were 12.0±8.81; for non-OA articles the mean citations were 7.14±6.89. The IBM® SPSS® Statistics (version 22.0) software was used to estimated sample size, at α = 0.05 and β =0.2, was 58 articles conduct the statistical data analyses on the dataset (Dataset 1, per group, which we rounded up to an intended 100 articles per doi: 10.7910/DVN/XR6MR922). OA journals were compared to group. non-OA journals for overall IF and citations over 5 years. Nor- mality for each independent variable and dependent variable was In order to have a 5-year follow up for citations, we chose 2011 as assessed using the “Kologorov-Smirnov” test, which showed that the publication year of articles included in this study, and restricted citations were not normally distributed (p< 0.05). Consequently, our source articles to those published in January 2011. We found non-parametric univariate analysis was carried out using the 3,742 RCTs, and saved them into a Microsoft Excel file. The IF of “Mann-Whitney” test. Linear regression was performed before their journals were derived from the Thompson Reuters’ Web of and after logarithmic transformation of the data. Science database. Results From these 3,742 articles, we extracted titles until at least 100 Citations for articles published in OA and non-OA journals articles met the criteria for OA, and 100 for non-OA.