COMMENTARIES The Journal Impact Factor Denominator Defining Citable (Counted) Items Marie E. McVeigh, MS The items counted in the denominator of the impact fac- tor are identifiable in the Web of Science database by hav- Stephen J. Mann ing the index field document type set as “Article,” “Re- view,” or “Proceedings Paper” (a specialized subset of the VER ITS 30-YEAR HISTORY, THE JOURNAL IMPACT article document type). These document types identify the factor has been the subject of much discussion scholarly contribution of the journal to the literature and and debate.1 From its first release in 1975, bib- are counted as “citable items” in the denominator of the im- liometricians and library scientists discussed its pact factor. A journal accepted for coverage in the Thom- Ovalue and its vagaries. In the last decade, discussion has 6 son Reuters citation database is reviewed by experts who shifted to the way in which impact factor data are used. In consider the bibliographic and bibliometric characteristics an environment eager for objective measures of productiv- of all article types published by that journal (eg, items), which ity, relevance, and research value, the impact factor has been are covered by that journal in the context of other materi- applied broadly and indiscriminately.2,3 The impact factor als in the journal, the subject, and the database as a whole. has gone from being a measure of a journal’s citation influ- This journal-specific analysis identifies the journal sec- ence in the broader literature to a surrogate that assesses tions, subsections, or both that contain materials likely to the scholarly value of work published in that journal. These be considered scholarly works, and which therefore have misappropriated metrics have been used to assess indi- the potential to be cited. Among the features considered dur- vidual researchers, institutions, and departments. ing this review process are the following. Such evaluation practices put pressure on scholars to pub- Descriptive article titles: Most primary research and schol- lish in high-impact journals. To continue attracting manu- arly items highlight their content with highly descriptive titles scripts and authors, editors and publishers are pressured to denoting the subject of the study. This is also true of some, ensure a high impact factor for their journal.4 From a dis- but not all, reviews. cussion of uses and merits and improvements and expan- Named author(s) with author address: This information typi- sions of the impact factor, interest has turned to using pub- cally indicates that authors are affiliated with an academic, lishing and editorial practices to influence, alter, and even research, or other institution as opposed to paid contribu- manipulate the impact factor.5 tors (eg, a staff writer or reporter). The impact factor is calculated by considering all cita- Abstract or summary: Abstracts are another common fea- tions in 1 year to a journal’s content published in the prior ture of research and scholarly items and are used to pro- 2 years, divided by the number of substantive, scholarly items vide detailed information about the content and conclu- published in that journal in those same 2 years.1 The im- sions detailed in the body of the item and to guide readers pact factor is a ratio of journal citations to the number of regarding the relevance of the item to their needs. substantive, scholarly articles published by that journal, but Article length: Items less than a half page in length are un- not the mathematical average of the citations to all of the likely to provide significant scholarly contribution. Lengthy journal’s content. Although the calculation treats each jour- works are more suggestive of scholarly effort. nal as a whole rather than as a simple count of all pub- Data content: Whether primary research data or data col- lished items, the numerator and denominator of the im- lected and summarized from 1 or more prior works, the item pact factor ratio have different definitions. The numerator is more likely to be of interest to scholars in the field if a is the number of all citations to the journal from 1 year to figure or table was produced specifically for the current item. the previous 2 years. Any citation to any item represents an Cited references: An item that cites extensively to prior or acknowledgment of the journal. However, the denomina- concurrent works will have the purpose of evidentiary sup- tor is the number of substantive, scholarly articles most likely port of the author’s hypothesis, or of providing a novel syn- to be cited. Author Affiliations: Bibliographic Policy, Journal Citation Reports, Thomson Reuters, Philadelphia, Pennsylvania. See also p 1092. Corresponding Author: Marie E. McVeigh, MS, Thomson Reuters, 3501 Market St, Philadelphia, PA 19104 ([email protected]). ©2009 American Medical Association. All rights reserved. (Reprinted) JAMA, September 9, 2009—Vol 302, No. 10 1107 Downloaded From: https://jamanetwork.com/ on 10/01/2021 COMMENTARIES thesis of previous works. The presence of cited references ing items that are later cited and that contribute to the im- indicates a level of participation in the academic discourse pact factor. As an example of the effectiveness of the cur- and scholarly development of the subject. rent approach used to identify citable items, we identified Cited reference density: This is defined as the number of 1.1 million items published in 2000, and indexed for the cited references per text page of the item. A high density of Science Citation Index-Expanded and/or the Social Sci- references suggests a significant work of scholarship. ences Citation Index (ie, the 2 citation databases with jour- While no one of these characteristics is definitive, taken nals that appear in the Journal Citation Reports6 listings). in concert, these features have proven accurate in identify- For each item, we recorded the number of citations ac- crued in each year from 2000 through 2007. The 8-year time Table. Distribution of Citable (Countable) Content and Citations for window allowed the capture of a large proportion of total Items Published in 2000 and 2005 lifetime citation of the items. This count of citations in- Total Items Item Count (% of Total) cluded scholarly citations and citations in academic corre- (Citable ϩ spondence, such as a reply to a letter to the editor that cites Not Citable) Citable Not Citable the original letter. A total of 793 395 (72.8%) of the items Indexed in 2000 All in SCIE 1 090 333 793 395 296 938 (27.2) from 2000 were indexed as articles or reviews, and these and SSCIa (72.8) citable items accounted for 97.6% of all citations accrued Annu Rev Med 33 33 (100.0) 0 in the 8 years since their publication (TABLE). BMJ 3335 612 (18.4) 2723 (81.6) Journals with a simple array of published content, such JAMA 1785 377 (21.1) 1408 (78.9) as the Annual Review of Medicine, show 100% of accrued ci- Lancet 3395 821 (24.2) 2574 (75.8) tations are to citable items. Other journals, such as the BMJ, NEJM 1562 379 (24.3) 1183 (75.7) JAMA, Lancet, and the New England Journal of Medicine Total citations by end of 2007 (NEJM), have more complex content, but show a similar pat- All in SCIE 11 624 107 11 347 214 276 893 (2.4) tern of the majority of citations accumulating to the citable and SSCIa (97.6) (counted) items. The diverse content increases the relative Annu Rev Med 1768 1768 (100.0) 0 proportion of citations to items indexed as not citable (not BMJ 32 069 26 302 (82.0) 5767 (18.0) counted). Although these journals have a large number of JAMA 41 468 36 552 (88.1) 4916 (22.9) discrete sections, and vary in presentation and size, the pro- Lancet 63 246 55 255 (87.4) 7991 (12.6) portionate distribution of citations to citable items is con- NEJM 75 984 69 021 (90.8) 75 984 (9.2) sistent. Total citations in 2002b All in SCIE 1 708 953 1 661 819 47 134 (2.8) To focus on the contribution of published items to the and SSCIa (97.2) citation count for the impact factor, only citations to the cit- Annu Rev Med 254 254 (100.0) 0 able items in 2000 (ie, impact factor denominator) from BMJ 4681 3687 (78.8) 994 (21.2) works published in 2002 were considered. This represents JAMA 6214 5333 (85.8) 881 (14.2) one part of the numerator of the 2002 impact factor. Al- Lancet 9878 8400 (85.0) 1478 (15.0) though fewer items contributed citations to the 2002 im- NEJM 11 171 9921 (88.8) 1250 (11.2) pact factor (65.9% of published items) than were cited within Indexed in 2005 8 years after publication (42.1% of published items), the con- All in SCIE 1 277 290 918 215 359 075 (28.1) and SSCIa (71.9) centration of citations to citable items was even more marked; Annu Rev Med 30 30 (100.0) 0 97.2% of citations in 2002 were to items indexed as citable BMJ 2589 522 (20.2) 2067 (79.8) items in 2000 (Table). JAMA 1386 324 (23.4) 1062 (76.6) Because many changes to these journals have taken place Lancet 1730 422 (24.4) 1308 (75.6) in recent years, we considered another example using all NEJM 1798 308 (17.1) 1490 (82.9) items published in 2005. Four of 5 journals decreased both Total citations in 2007c the total number of items and the number of citable items All in SCIE 2 258 888 2 196 063 62 825 (2.8) a published from 2000 to 2005.