Published online: 2021-06-30 THIEME Research Article e95 Examining the Relationship between Altmetric Score and Traditional Bibliometrics in the Ophthalmology Literature for 2013 and 2016 Cohorts Zachary C. Wiley, BS1 Carter J. Boyd, BS2 Shivani Ananthasekar, BS2 Nita Bhat, MD3 Shruthi Harish Bindiganavile, MD3 Andrew G. Lee, MD1,3,4,5,6,7,8 1 School of Medicine, Baylor College of Medicine, Houston, Texas Address for correspondence Andrew G. Lee, MD, Department of 2 School of Medicine, University of Alabama at Birmingham, Ophthalmology, Blanton Eye Institute, Houston Methodist Hospital, Birmingham, Alabama 6550 Fannin St, Houston, TX, 77030 3 Department of Ophthalmology, Blanton Eye Institute, Houston (e-mail: [email protected]). Methodist Hospital, Houston, Texas 4 Department of Ophthalmology, Neurology, and Neurosurgery, Weill Cornell Medicine, New York, New York 5 Department of Ophthalmology, University of Texas Medical Branch, Galveston, Texas 6 University of Texas MD Anderson Cancer Center, Houston, Texas 7 Texas A and M College of Medicine, Bryan, Texas 8 Department of Ophthalmology, The University of Iowa Hospitals and Clinics, Iowa City, Iowa J Acad Ophthalmol 2021;13:e95–e101. Abstract Background In this study, we reviewed a select sample of ophthalmology literature to determine if there was a correlation between Altimetric and traditional citation-based and impact factor metrics. We hypothesized that Altmetric score would more closely correlate with impact factor and citations in 2016. Methods Journal Citation Reports for the year 2013 was used to find the 15 highest impact factor ophthalmology journals in 2013. Then Elsevier’s Scopus was used to identify the 10 most cited articles from each journal for the years 2013 and 2016. Metrics for all identified articles were collected using the Altmetric Bookmarklet, and date of Twitter account creation was noted for journals with such an account. Altmetric scores, impact factor, and citation counts were tabulated for each article. Pearson’s correlation coefficient (r) determined correlation of independent variables (number of citations or impact factor) with dependent variable (Altmetric score). For our Twitter Keywords analysis, account age was the independent variable and calculated correlation ► citations coefficients (r) were the dependent variable. Proportion of variance was determined ► Twitter with a coefficient of determination (R2). ► ophthalmology Results This study included 300 articles, evenly split between 2013 and 2016. Within ► Altmetric the 2013 cohort, three journals had significant positive correlations between citation ► social media received DOI https://doi.org/ © 2021. The Author(s). October 21, 2020 10.1055/s-0041-1728658. This is an open access article published by Thieme under the terms of the accepted after revision ISSN 2475-4757. Creative Commons Attribution-NonDerivative-NonCommercial-License, February 22, 2021 permitting copying and reproduction so long as the original work is given appropriate credit. Contents may not be used for commercial purposes, or adapted, remixed, transformed or built upon. (https://creativecommons.org/ licenses/by-nc-nd/4.0/) Thieme Medical Publishers, Inc., 333 Seventh Avenue, 18th Floor, New York, NY 10001, USA Examining the Relationship between Altmetric Score and Traditional Bibliometrics in the Ophthalmology Literature e96 for 2013 and 2016 Cohorts Wiley et al. count and Altmetric score. For the 2016 cohort, both Altmetric score and citation count (r ¼ 0.583, p < 0.001) and Altmetric score and impact factor (r ¼ 0.183, p ¼ 0.025) revealed significant positive correlations. In 2016, two journals were found to have significant correlations between Altmetric score and citation number. Neither year revealed a significant correlation between the age of a journal’sTwitterprofile and the relationship between Altmetric score and citation count. In each year, Twitter accounted for the highest number of mentions. Conclusion The findings suggest that correlation between Altmetric score and traditional quality metric scores may be increasing. Altmetric score was correlated with impact factor and number of citations in 2016 but not 2013. At this time, Altmetrics are best used as an adjunct that is complementary but not an alternative to traditional bibliometrics for assessing academic productivity and impact. Introduction impact, include mentions of articles on various social media Bibliometrics is a method of quantifying and analyzing the sites, as well as Wikipedia, patent applications, and news impact of academic publications and ultimately is used for outlets.7 The term “Altmetric” is sometimes used to refer determining academic productivity, scholarly effort, journal generally to all alternative metrics of article reach, but here prestige and impact, and individual promotion and tenure. we use the term to refer to the article score provided by Traditionally, bibliometrics include quantity indicators, per- Altmetric.com. Rather than supplanting bibliometrics and formance indicators, and structural indicators of an article.1 peer review as markers of research quality, alternative Citations give credit for an idea or fact to an earlier article, and metrics are intended to be used along with these traditional the number of times an article is cited is generally understood means as part of a “basket of metrics” for quantifying impact as a marker of impact within an academic field.1 Citation in a particular field.5 The objective of this study is to examine analysis, then, involves the ranking of an article based on the correlation between bibliometric and Altmetric scores for a number of citations it receives to identify published work with subset of articles in ophthalmology literature, to answer the the greatest intellectual influence.2 Impact factor is a marker of questions of whether Altmetric data correlates with biblio- impact for journals that compares the number of citations for a metrics, and in what ways it may be weaker, stronger, or given year to the number of articles published. More citations complementary. It was hypothesized that there would be a per article equate to a higher impact factor.1 These metrics are stronger correlation between Altmetric scores and tradition- considered quality indicators and can have tremendous im- al bibliometrics in the later cohort as a reflection of social pact in funding decisions for individual researchers and entire media’s expanded influence across time compared with departments.3 articles published previously.8 In this study, we reviewed a Traditional bibliometrics allow the application of a stan- select sample of ophthalmology literature to determine if dardized, scientifically rigorous methodology to the determi- there was a correlation between Altimetric and traditional nation of the quality of individual publications, researchers, citation based and impact factor metrics. To our knowledge, and journals in aggregate, as well as to assess the productivity this is the first such study in the English language ophthalmic of departments, research teams, universities, and even entire literature. countries.4 The recent advent of newer information technolo- gies and platforms, however, has created opportunities for Methods dissemination of information among different audiences and across multiple platforms and formats.5 Traditional bibliomet- Using Journal Citation Reports for the year 2013, the 15 rics fail to capture the reach of this interconnected information ophthalmology journals with the highest impact factors web and the larger impact that publications may have for a were selected (►Table 1). Then, using Elsevier’s Scopus,9 the broader audience. Furthermore, citation time window has 10 most cited articles from each journal were identified for the been shown to have a large role in determination of elite, years 2013 and 2016. These particular years were selected to highly cited articles. The list of highly cited articles at 5 years ensure article metrics had sufficient time to mature after from publication is quite different from the list of highly cited publication, as well as to allow comparison of two distinct articles at 30 years.4 It is obvious there is much that traditional snapshots of a changing social media landscape. The Altmetric bibliometrics leaves out. Bookmarklet was used to collect metrics for all identified Recently, researchers have begun to question why articles, and journals with an active Twitter account were markers for research impact should not include broader identified and date of creation of the account was noted. impact among the general public.3,5,6 Altmetrics, so named Altmetric scores are calculated by an automated algo- with the intention of being alternative metrics of publication rithm. This algorithm is designed to take into account two Journal of Academic Ophthalmology Vol. 13 No. 1/2021 © 2021. The Author(s). Examining the Relationship between Altmetric Score and Traditional Bibliometrics in the Ophthalmology Literature for 2013 and 2016 Cohorts Wiley et al. e97 Table 1 Journal breakdown by citations, Altmetric score, impact factor, and age of Twitter account Journal Citations Altmetric Citations Altmetric Journal Journal Age of (median score (median score impact impact Twitter [range]) (median [range]) (median factor factor account (y) (2013) [range]) (2016) [range]) (2013) (2016) (2013) (2016) Progress in Retinal 112 [87–304] 4 [0–64] 88.5 [64–183] 2 [1–26] 9.897 11.587 N/A and Eye Research Ophthalmology 260.5 [188–441] 4 [0–124] 141 [99–432]