Alabama Law Scholarly Commons Essays, Reviews, and Shorter Works Faculty Scholarship 2000 Determinants of Citations to Articles in Elite Reviews Interpreting Legal Citations Ian Ayres Fredrick E. Vars University of Alabama - School of Law, [email protected] Follow this and additional works at: https://scholarship.law.ua.edu/fac_essays Recommended Citation Ian Ayres & Fredrick E. Vars, Determinants of Citations to Articles in Elite Reviews Interpreting Legal Citations, 29 J. Legal Stud. 427 (2000). Available at: https://scholarship.law.ua.edu/fac_essays/199 This Article is brought to you for free and open access by the Faculty Scholarship at Alabama Law Scholarly Commons. It has been accepted for inclusion in Essays, Reviews, and Shorter Works by an authorized administrator of Alabama Law Scholarly Commons. DETERMINANTS OF CITATIONS TO ARTICLES IN ELITE LAW REVIEWS IAN AYRES and FREDRICK E. VARS* ABSTRACT This article analyzes the determinants of citations to pieces published from 1980 to 1995 in Harvard Law Review, Stanford Law Review, and The Yale Law Journal. We also rank articles by number of citations using regressions controlling for time since publication, journal, and subject area. To summarize a few of our results: cita- tions per year peak at 4 years after publication, and an article receives half of its expected total lifetime citations after 4.6 years; appearing first in an issue is a sig- nificant advantage; international law articles receive fewer citations; jurisprudence articles are cited more often; articles by young, female, or minority authors are more heavily cited. Articles with shorter titles, fewer footnotes per page, and with- out equations have significantly more citations than other articles. Total citations generally increase with an article's length, but citations per published page peak at 53 pages. I. INTRODUCTION MANY legal citation studies have two shortcomings-they tend to focus too much on only the right-hand tail of the citation distribution and too little on the determinants of citation. The emphasis on the right-hand tail can be observed most clearly in Fred Shapiro's piece, "The Most-Cited Law Re- view Articles,"' which marked the path for later studies.2 This fight-tail * Townsend Professor, Yale Law School, and J.D., Yale Law School, respectively. For helpful comments, the authors thank John Donahue, Ted Eisenberg, William Landes, Jim Lindgren, Peter Siegelman, David Steglich, Charles Vars, participants at the AALS law and economics section, and participants at the Interpreting Legal Citations symposium. See Fred R. Shapiro, The Most-Cited Law Review Articles, 73 Cal. L. Rev. 1540 (1985). 2 See, for example, William M. Landes & Richard A. Posner, Heavily Cited Articles in Law, 71 Chi.-Kent L. Rev. 825 (1996); Deborah J. Merritt & Melanie Putnam, Judges and Scholars: Do Courts and Scholarly Journals Cite the Same Law Review Articles? 71 Chi.- Kent L. Rev. 871 (1996); Fred R. Shapiro, The Most-Cited Law Review Articles Revisited, 71 Chi.-Kent L. Rev. 751 (1996) (hereinafter Shapiro, Revisited); Fred R. Shapiro, The Most- Cited Articles from the Yale Law Journal, 100 Yale L. J. 1449 (1991). At least one other commentator has noted this bias. See Nancy Levit, Defining Cutting-Edge Scholarship: Femi- nism and Criteria of Rationality, 71 Chi.-Kent L. Rev. 947, 952 (1996) ("[P]erhaps the direc- tion of quantitative inquiry is fundamentally wrong. We might learn more about scholarship and the politics of the legal academy if we explore instead what works are not being cited and why"). [Journal of Legal Studies, vol. XXIX (January 2000)] © 2000 by The University of Chicago. All rights reserved. 0047-2530/2000/2901-0020$01.50 THE JOURNAL OF LEGAL STUDIES bias is not unique to studies of law review articles. A recent study in this journal ranked the top 32 law schools on the basis of citations to articles written by professors at each of the law schools.' There have, however, been exceptions to the right-tail bias in legal citations analysis. A pair of articles examined citations to judicial opinions in order to identify the least significant United States Supreme Court justice.4 And a sophisticated analy- sis of citations to opinions by judges on the federal courts of appeals in- cluded a complete ranking of the entire distribution of judges.' Notably missing from the literature is a comparable examination of the full distribu- tion of law review articles (including the least cited). Prior studies have also failed to delve very seriously into the determi- nants of citations. Studies of law review articles' citations have recognized the importance of opportunities to be cited but employ relatively crude sta- tistical methods to correct for this effect. For example, in response to the age-of-article criticism of his all-time 100 most-cited articles' list, Shapiro added top-10 lists for articles published in each year from 1982 to 1991.6 This methodology, however, does not allow for the simultaneous consider- ation of other variables.7 A more sophisticated statistical approach is that of William Landes and Richard Posner, who recreated Shapiro's top-100 rankings, adjusting for the effects of article age and age squared.' Landes and Posner, however, did not correct for journal and subject area effects, which may influence opportunities to be cited. More fundamental, because they relied solely on cross-sectional data, Landes and Posner could not model the flow of citations over time or isolate the effects of changes in the ' See Theodore Eisenberg & Martin T. Wells, Ranking and Explaining the Scholarly Im- pact of Law Schools, 27 J. Legal Stud. 373 (1998). See also James Lindgren & Daniel Selt- zer, The Most Prolific Law Professors and Faculties, 71 Chi.-Kent L. Rev. 781, 786 (1996) (ranking law school professors by the number of articles published in the 10 and 20 most- cited law reviews). Richard Posner has used citation analysis to examine the impact of Benja- min Cardozo and Learned Hand. See Richard A. Posner, Cardozo: A Study of Reputation (1990); Richard A. Posner, The Learned Hand Biography and the Question of Judicial Great- ness, 104 Yale L. J. 511 (1994) (reviewing Gerald Gunther, Learned Hand: The Man and the Judge (1994)). ' See Frank H. Easterbrook, The Most Insignificant Justice: Further Evidence, 50 U. Chi. L. Rev. 481 (1983); David Currie, The Most Insignificant Justice: A Preliminary Inquiry, 50 U. Chi. L. Rev. 466 (1983). Easterbrook argues that Thomas Todd was less significant than Gabriel Duval[l]. ' See William M. Landes, Lawrence Lessig, & Michael E. Solimine, Judicial Influence: A Citation Analysis of Federal Courts of Appeals Judges, 27 J. Legal Stud. 271 (1998). Con- sistent with the right-tall bias, however, much of the discussion centers on the most influen- tial judges. 6 See Shapiro, Revisited, supra note 2, at 773-77 (table 2). This criticism applies generally to Shapiro's comparisons between articles on his all-time and recent-year lists. See id. at 757-58. 8 See Landes & Posner, supra note 2, at 830-31. CITATION DETERMINANTS size of the data source.9 And, to our knowledge, no prior studies have re- gressed article citations on a richer set of article and author attributes. These two shortcomings are related because focusing on the "most cited" can obscure researchers' ability to identify the determinants of cita- tions more generally. Steven Jay Gould, for example, pointed out that fo- cusing solely on the decline of the .400 hitter in baseball led people to over- look explanations concerning the overall range of variation. ° With regard to legal citations, it is at least plausible to argue that, among articles in elite law reviews, receiving few citations is a stronger indicator of relative qual- ity than receiving many. Articles with many citations might be high quality or low quality (as other authors go out of their way to criticize the argu- ment). Articles in elite law reviews with few citations, however, are more likely to be of low quality. Finding that a Harvard Law Review article has many fewer cites than other articles in the same subject area is hard to square with a hypothesis of excellence." The data in our study consist of 979 articles and other nonstudent pieces published in HarvardLaw Review, Stanford Law Review, and The Yale Law Journal from 1980 to 1995.12 We use these data to make three types of esti- mates: Residual Rankings. Regressing article citations onto variables correlated with the opportunities the article had for citation (but independent of its quality), we rank the articles by how far their actual citations are above or below their expected citations (taking only opportunity variables into ac- count). With some trepidation, we then list the most- and least-cited articles in our sample on the basis of these residual rankings. Article and Author Attribute Regressions. By collecting panel data on the annual citations to particular articles, we are able to disentangle the expansion of our database, the Social Sciences Citation Index (SSCI), from the general tendencies of articles to be cited over time. We regress annual article citations on more than 60 regressors for article and author attributes. Least-Cited Article Regressions. Returning to our residual rankings, we Some analyses of citations in areas other than law review articles employ more sophisti- cated statistical methodologies. For two of the best examples, see Landes, Lessig, & Soli- mine, supra note 5, and Eisenberg & Wells, supra note 3. '0 See Steven Jay Gould, The Flamingo's Smile 218-21 (1985). " Others have noted more general problems with inferring high quality from large citation counts. See, for example, Posner, Cardozo, supra note 3, at 70-73. 12 We selected these three journals because these are generally considered to be the top three student-edited legal publications.