“Not that Smart”: Sonia Sotomayor and the Construction of Merit

Guy-Uriel Charles, Daniel L. Chen & Mitu Gulati Duke University School of Law

Abstract

The appointment of Sonia Sotomayor to the Supreme Court in 2009 was criticized as sacrificing merit on the altar of identity politics. According to critics, Sotomayor was simply “not that smart”. For some conservative critics, her selection illustrated the costs of affirmative action policies, in that this particular choice was going to produce a lower quality Supreme Court. For liberal critics, many were concerned that the President, by selecting Sotomayor, was squandering an opportunity to appoint an intellectual counterweight to conservative justices like Antonin Scalia, Samuel Alito and John Roberts. Using a set of basic measures of judicial merit, such as publication and citation rates for the years 2004-06, when Sotomayor was on the Second Circuit Court of Appeals, we compare her performance to that of her colleagues on the federal appeals courts. Sotomayor matches up well. She might turn out to be more of a force on the Court than the naysayers predicted.

“NOT THAT SMART”: SONIA SOTOMAYOR AND THE CONSTRUCTION OF MERIT

Guy-Uriel Charles Daniel L. Chen Mitu Gulati1

I. “NOT NEARLY AS SMART AS SHE SEEMS TO THINK SHE IS”

When President Barack Obama was considering whether to nominate to the Supreme Court Sonia Sotomayor, then a judge on the United States Court of Appeals for the Second Circuit, a prominent law professor, Laurence Tribe, wrote a letter to the President opposing Sotomayor’s potential nomination on the ground that “she’s not nearly as smart as she seems to think she is.”2 While Tribe’s assessment was intended as a private communication, others were saying something similar in public. 3 Jeffrey Rosen, another legal academic, wrote an article in the New Republic questioning Sotomayor’s merit. Based on anonymous sources, Rosen reported that there was widespread skepticism among the judges and academics familiar with Sotomayor’s work, regarding her capabilities for the job. The consistent theme was a concern that Sotomayor was simply “not that smart”.4 Even those on the Democratic side of the aisle, Rosen noted, appeared to have misgivings about Sotomayor’s intellectual capabilities.

It was not long before the conventional narrative became that Sotomayor was a mediocre legal mind. Commentators accused President Obama of having

1 Duke Law School faculty. For comments, thanks to Stephen Choi, Joseph Blocher, Al Brophy, Kim Krawiec, Daria Roithmayr and Angela Willig. For research assistance, we are grateful to Chris Battles, Heather Horst, Michelle Huang, Chris McCurdy, and Mark Wu. The Center for Race, Law and Politics at Duke provided research support. The analysis here builds on two blog posts done by Eric Posner at the time of Sotomayor’s nomination. See Eric A. Posner, Judge Sonia Sotomayor: What the Data Show, The Volokh Conspiracy (May 2009) (available at http://lists.powerblogs.com/pipermail/volokh/2009-May/017020.html); Eric A. Posner, Judge Sotomayor: More Data; And a New Conclusion (May 2009) (available at http://www.mail- archive.com/[email protected]/msg17205.html) 2 For discussion of the Tribe letter, see Ed Whelan, Tribe to Obama: “Not Nearly as Smart as She Seems to Think She is” (http://www.nationalreview.com/bench-memos/251301/tribe-obama- sotomayor-not-nearly-smart-she-seems-think-she-ed-whelan). For the letter, see http://www.eppc.org/docLib/20101028_tribeletter.pdf 3 See Countdown with Keith Olberman (interview with Jonathan Turley, May 2009) (available at http://www.msnbc.msn.com/id/3036677/vp/30950928#30950928); P.J. Gladnick, Liberal Jonathan Turley: Sotomayor Lacks Intellectual Depth, NewsBusters.Com, May 27th, 2009 (available at http://newsbusters.org/blogs/p-j-gladnick/2009/05/27/liberal-jonathan-turley-sotomayor-lacks- intellectual-depth) (quoting Turley as characterizing Sotomayor as lacking intellectual depth, but describing Diane Wood as “blazingly brilliant”). 4 See Jeffrey Rosen, The Case Against Sonia Sotomayor, The New Republic, May 4, 2009.

2

sacrificed merit for identity politics (“biography over brains,” in the words of Washington Post commentator, Dana Milbank).5 For many, she was not in the pool of the best-qualified candidates, even if it was the case that Obama wanted to pick a female justice. There were others, such as Diane Wood a former law professor at the University of Chicago and judge on the Seventh Circuit and Elena Kagan the former dean of the Harvard Law School and, at the time, Solicitor General of the United States, who they thought were smarter and more deserving. While supporters of Wood and Kagan often depicted them as brainy or brilliant, Sotomayor was lucky if her supporters described her as workmanlike and competent. 6

To us, a striking aspect of Sonia Sotomayor’s nomination was that this public and negative assessment of her merit was made without much factual support. One might even argue that the initial presumption should have been in her favor. After all, she graduated with honors from Princeton; was a graduate of the Yale Law School; and spent more than a decade on the Court of Appeals following stints as a trial judge and a federal prosecutor. It is hard to look at her credentials and conclude reflexively that she was unqualified for the Supreme Court.

That is, unless one applied a high discount to her achievements on the theory that her success was largely attributable to affirmative action. A theme running through much of the public discussion of her candidacy was that this appointment, more than most, represented the triumph of identity politics over merit.7 One could not escape the fact that she was going be the first Latina on the Supreme Court or that President Obama had considered and nominated her in part because she was Latina.8

Our goal in this Essay is to provide some data against which to test the claims of Sotomayor’s mediocrity. Prior to being nominated to the Court, she had been a judge on the Second Circuit Court of Appeals for roughly a decade. The pool of judges on the Courts of Appeals is the primary one from which Supreme Court justices are chosen these days. That means that we should be able to compare Sotomayor’s performance to that of her peers to get a rough sense of just how she matches up.

5 See Dana Milbank, Washington Sketch: Sonia Sotomayor in the Ballpark, Washington Post, May 27, 2009; see also Somin, infra note __. 6 See http://www.scotusblog.com/2009/07/over-1000-law-professors-join-letter-endorsing- sotomayor/; see also Richard Cohen, Sonia Sotomayor: A Safe Soporific Bet for the Supreme Court, Washington Post, July 20, 2009; Milbank, supra note 7 See Ilya Somin, Commentary: Sotomayor Pick Not Based on Merit, CNN.com (available at http://articles.cnn.com/2009-05-27/politics/shapiro.scotus.identity_1_judge-sotomayor-hispanic- supreme-court-sonia-sotomayor?_s=PM:POLITICS). 8 E.g., Joe Conason, Sotomayor is not Clarence Thomas, Salon.com, May 29, 2009 (available at http://www.salon.com/news/opinion/joe_conason/2009/05/29/clarence_thomas).

3

We use two categories of measures to evaluate Sotomayor’s performance relative to that of all the other active judges on the Courts of Appeals over the period 2004-06. The categories are publications and citations. As we explain in the next section, the nine measures we use within those categories should give us a rough sense of the skill and effort that Judge Sotomayor brought to her job, in comparison to her peers.

In constructing these comparisons, to test claims that were made about Sotomayor during her nomination, we are not writing on a blank slate. Other scholars have subjected the repeated claims that were made about Sotomayor being an “activist” judge to empirical testing. The claims were found wanting.9 And our analysis builds on a preliminary examination reported by Professor Eric Posner in two blog posts in 2009. He compared Sotomayor’s performance on the appeals court to that of the ten other judges whose names had been mentioned as potential candidates for the Court and found that she did well.10 We expand on that analysis in three ways: by comparing Sotomayor’s numbers to the full set of 136 judges who were active during the years 2004-06, by using additional measures, and by controlling for structural differences across the federal circuits.

The question of how best to construct objective measures of judicial performance has been the topic of debate in recent years. The measures we use, citations and publications, are among the more familiar ones in the literature.11 We should note though that these measures have come in for their share of criticism and other measures have been proposed.12 We will not rehash the debate here.13

If Sotomayor’s performance, while on the appeals court, had been mediocre, we would expect to find that in comparison to her peers, she would have published few opinions and that those opinions would have had little impact because they

9 See Corey Rayburn Yung, Flexing Judicial Muscle: An Empirical Study of Judicial Activism in the Federal Courts, __ Nw. U. L. Rev. __ (2011); Monica Youn, Judge Sotomayor’s Record in Constitutional Cases, Brennan Center Report (2009); Marcia Coyle, Sotomayor is No Activist Judge Says Author, National Law Journal __(2009 10 Posner’s analysis was done on the basis of data that was collected jointly with Stephen Choi and one of us. For a report on that analysis, see Assessing Sotomayor’s Influence, N.Y. Times, (NYTimes.com), May 28, 2009. 11 See William Landes, et al., Judicial Influence: A Citation Analysis of the Federal Courts of Appeals, 27 J. Legal Stud.__ (1998); Stephen J. Choi & Mitu Gulati, Choosing the Next Supreme Court Justice, 78 USC L. Rev. 23 (2004); Scott Baker, Should We Pay Federal Judges More? __ 88 B. U. L. Rev. __ ; Jake Dear & Edward Jessen, “Followed Cases” and Leading State Cases, 41 U.C. Davis L. Rev. __ (2007); Russell Smyth & Mita Bhattacharya, What Determines Judicial Prestige? A Citation Analysis For Judges of the Federal Court of Australia, 5 Amer. L. & Econ. Rev. 233 (2003). 12 E.g., Frank Cross & Stefanie Lindquist, Judging the Judges, __ Duke L. J. __ (2009); William P. Marshall, Be Careful What You Wish For, 78 USC L. Rev. 119 (2004); Kate Stith, Marin Levy & Jose Cabranes, The Costs of Judging by the Numbers, 28 Yale J. L. & Pol’y 2 (2010). 13 For interested readers, these measures have been debated in a number of symposia, including volume 32 (2005) of the Florida State Law Review, volume 58 (2009) of the Duke Law Journal, and volume 78 (2004) of the USC Law Review.

4

would not have often been cited by other judges and commentators. If, on the other hand, she had performed with distinction, we would expect to find that she does better than most of her colleagues in terms of output and citations. As we show below, Sotomayor compares well when judged against her peers. She is well above the mean in all of our measures and in the top quarter of all judges in the comparison pool on eight of the nine measures that we examine.

II. A HIERARCHY OF MEASURES

Before proceeding, it may help to clarify what we are, and are not, claiming to show. We are not challenging the merits of the reviews of Sotomayor by Tribe, Rosen or any of the others who either reviewed her opinions or collected anonymous reports. We are offering a different measure by which Sotomayor’s merit might have been judged, one that we concede has value primarily in terms of raising questions about the more sophisticated evaluations done by others such as Tribe and Rosen.

Our starting premise is that there are a variety of methods by which to measure an appellate judge’s merit, vis-à-vis a possible elevation to the High Court. The possible methods of evaluation all contain tradeoffs. Tribe’s evaluation, Rosen’s evaluation and our evaluation, all fall into different boxes in terms of using different methods to answer the same question.

The analogy to movie reviews may help. Someone interested in knowing whether to go to a particular movie has a variety of sources of information on that movie. One option is to look to the evaluations by Anthony Lane, Manhola Dargis or one of the other critics for the New Yorker or the New York Times. Here, the reviewers themselves are typically famous and conduct nuanced and erudite reviews. Most readers looking at these reviews know ahead of time that what they are getting is likely to be idiosyncratic – indeed, that is part of the value. Laurence Tribe’s review of Sotomayor might be thought of as falling into this category. It is a review done by a superstar academic who likely brings his idiosyncratic preferences to the table. At least a portion of those reading his review (and certainly, his former student, Barack Obama) are going to be able to read it in the context of who he is, what his preferences and motives are, and so on.

For a different category of views, our potential movie-goer might look online, at Rotten Tomatoes or Fandango, for aggregations of what are likely pseudonymous and perhaps less expert movie reviews. Here, one often will not have any idea who the reviewers are, what their motivations are, and so on (they could be jealous competitors, fawning acolytes, or seven-year olds from Kazakhstan). The fact of their anonymity has upsides and downsides. On the one hand, it allows for more honesty (no fear of repercussion from the author or her friends) and on the other hand it allows for more dishonesty (no fear of reputational loss if it turns out that there is widespread disagreement with the content of the review). Rosen’s review, relying on anonymous sources, might be thought of as falling into this category. It 5

provides a different type of information than the superstar review, but one that is just as legitimate and perhaps more relevant to some audiences.

Third, and even less sophisticated is market information. The potential movie-goer may want to know whether the movie is popular. That is, whether this is a movie that appeals to the broader public. If one is interested, one could even break down the market statistics in terms of which specific markets the movie has done well in. The fact that a movie has done well in Madison, Wisconsin, but done terribly in Cochin, India, conveys more information (if one knows anything about the audience preferences in those two locations) than simply knowing that the movie did well in Madison.

Our data on Sotomayor is like the market information about her performance. It is less nuanced than the above-mentioned sources, but it has a greater level of objectivity and it can be more easily verified. To the extent our findings point in the same direction as those of Tribe and Rosen, that should add to the level of confidence that one has in any of the evaluations. However, if the three sources do not line up as pointing in the same direction, that should raise questions. Whatever we find though, we cannot claim to be proving the others wrong. We are simply adding more information to the mix.

III. THE DATA

To examine Sotomayor’s performance relative to her colleagues on the federal appeals courts, we collected data from roughly the middle of her tenure on the Second Circuit. Her full first year on the Second Circuit was in 1999 and her last full year on the Second Circuit, prior to nomination, was in 2008.14 We examine her performance during three years falling roughly within the middle of that period, 2004-06. Our reasons for selecting these years were simple. We did not wish to look too early in her tenure, when she might have been learning the job and building a reputation. During this period, her numbers might have been biased downwards as a result of her inexperience and lack of a reputation. We also did not want to look too late in her tenure, when her name had begun surfacing as a possible candidate for elevation by President Obama. During this latter period, her citation numbers in particular may have been biased upwards by an anticipatory Supreme Court halo effect. That is, others citing her in order to invoke the extra authority that comes from citing a Supreme Court justice.

For the period 2004-06, we collected information at the individual case level on every case that was published for twelve circuit courts of general jurisdiction (The First Circuit to the Eleventh Circuit and the D.C. Circuit). We skipped the Federal Circuit because of its specialized caseload. We should note at the outset though that there is variation among the circuits in terms caseloads and case types.

14 See http://www.whitehouse.gov/the_press_office/Background-on-Judge-Sonia-Sotomayor/

6

For example, the D.C. Circuit’s diet of cases is much higher in administrative law than say the Fifth Circuit. Similarly, the Ninth Circuit likely sees many more immigration cases than the Eighth Circuit. Despite the variation in case types and caseloads across circuits, there is reason to think that judicial performances can be compared across the different federal courts. This is because our primary comparison metric derives from the choice made by individual judges to publish opinions. Two facts are important here. First, federal appellate judges, regardless of circuit, face hundreds, if not thousands, of cases every year. Many are easy and do not raise important legal issues. But there is also an abundance of cases that potentially do raise important legal issues and judges do not have enough time to deal with all of them with published opinions (there is now a large literature discussing this problem). Second, and relatedly, judges tend to publish only a small fraction of their cases. Some judges publish no more than a dozen opinions a year out of the hundreds that come before them. Therefore, if one assumes that every judge, regardless of circuit, gets more than enough cases that have issues that could be publication worthy, one can compare judges in terms of their published opinions.

To make adjustments for variations in case types, we also collected information for every case on the subject area that it fit under. The estimations of whether a case fell within a given subject area was necessarily a subjective one. The designation was made based on a combination of reading the Westlaw summary of the case and examining the Westlaw key cites that categorize every case as a function of the various subject areas that it covers. In total, we coded the cases as falling into twenty-one distinct subject areas, plus one general category for “other” cases. 15

We report on data for 136 circuit court judges for the twelve circuits mentioned above.16 These were all the judges who were on “active” status for the three years for which we collected data. Judges who were active for only a subset of the period did not get counted. The “active” designation means that the judge was taking a full caseload and had full administrative responsibilities (the alternative is senior status, where the judge can reduce his case load to as low as 25% of the full load).17 Our reason for focusing on the active judges is that our numbers are most illustrative in a relative rather than absolute sense, since what we are interested in is Sotomayor’s performance relative to that of her competitors for the top spot. We restrict the comparison to her “active” peers because that presumably constitutes a significant fraction of the pool of competitors for the slot on the Supreme Court. The comparison pool we have constructed is both under- and over-inclusive in terms of capturing the true pool of Sotomayor’s competitors’ vis-à-vis President Obama. On

15 The list of subject areas is reported in Appendix B. 16 There are 179 active positions on the federal courts of appeals. All of those positions, however, are not necessarily filled at any given time. 17 The assumption that senior judges do less work does not hold uniformly (in the course of this study we found indications that a few senior judges were producing as much, if not more, than some of their “active” colleagues).

7

the minus side, his pool likely did not include most Republican-appointed judges and judges above a certain age. On the plus side, Obama’s pool surely included some non-federal appeals court judges (senior government officials, academics, state judges, etc.). Our goal though is not to test whether Obama picked the best candidate, but rather to examine the hypothesis that his desire to further diversity goals led him to pick a mediocre candidate. For purposes of that hypothesis, comparing Sotomayor’s performance to a pool of candidates within which the candidates range from mediocre to exceptional should suffice. Recent history suggests that most nominations to the Court tend to be from the federal appeals courts, making it a reasonable comparison pool.18

Our analysis has little to say about an individual judge’s absolute merit. What we have are measures of relative performance. For example, saying that a judge publishes 100 opinions a year does not tell us much about their effort levels unless one also sees that 25% of the appellate judges publish fewer than 10 opinions a year.

Authorship: Here, we collected information on the identity of the writing judge and the identities of the two other judges on the panel. These variables allow us to determine both how many opinions Sotomayor authored and published herself and how many “for publication” opinions she was on the panel for. Prior research has tended to focus only on the primary author of the opinion. And that may be reasonable, given that the primary author is the one who generally puts in the major portion of the effort on any given opinion. However, the end product is supposed to be the joint product of three judges. Hence, the argument can be made that when one observes that an opinion was published or garnered a lot of citations, it is a team rather than individual effort that one is observing. To take this team- production effect into account, we attempt to estimate what the impact of Sotomayor sitting on a panel is, even when she is not the primary author.

In terms of the specific characteristics of individual opinions, we also measured the number of pages for each published opinion. The number of pages provides a different measure of judicial effort than just a count of the number of published opinions. Certain judges designate a large fraction of their opinions as worthy of publication and others are very judicious about such a designation. The number of pages could simply indicate a long-winded style of writing or may indicate how complex the issues were that the judge addressed. Thus, this measure provides us with an additional though different measure of judicial effort than the count of published opinions.

18 On the trend toward appointing federal appeals judges to the High Court, see Lee Epstein et al., The Norm of Prior Judicial Experience and its Consequences for Career Diversity on the U.S. Supreme Court, 91 Cal. L. Rev. 903 (2003).

8

Citations: Citations to an opinion can come from a variety of different audiences or users. Potentially, there is valuable information to be mined from the fact that a certain judge is more or less popular with certain audiences. Different audiences are likely to value different characteristics of a judicial opinion. The fact that one audience likes an opinion a great deal and another audience might provide information about the likely characteristics of that opinion.

Take for example, the complaint that judges and legal academics have grown increasingly farther apart in recent years.19 In this story, academics are more interested in interdisciplinary work that explores radical ideas whereas judges are more interested in careful doctrinal analyses. Assuming the foregoing, one would expect judges whose opinions are found more popular by academics to be making more radical arguments whereas judges whose opinions are popular among other judges may be producing careful and nuanced doctrinal syntheses. Finding out that Judge Sotomayor’s opinions are popular with academics, but unpopular with her fellow judges, might suggest that her opinions are innovative, but weak in terms of careful doctrinal parsing.

Along those lines, one might expect state judiciaries to be more interested in opinions that tackle basic issues in the types of areas that are more the province of state laws – subjects such as torts, contracts, family law and corporate law. A judge who finds herself more cited by the state courts may be writing high-quality opinions in the types of cases that fall under the purview of the local state courts. One might also distinguish local citations (from courts that a required to follow the decision as precedent) from outside circuit citations. Lots of local citations probably mean that the judge writes opinions that state the local law clearly. Lots of external citations may suggest something different, that the opinions are helpful to other judges in their analyses.

As described above, we look to four separate audiences to count citations. First, we examine citations by federal courts outside the circuit. Those are federal courts where the opinion would have no precedential value, but is presumably cited because it helps the analysis. Second, we aggregate citations within the circuit where the citations are often going to be used because the opinion has precedential value. Third, we report citations from the state courts. These are likely to be cites unrelated to the precedential effects of the case; since the state courts are not obligated to follow the federal courts in most matters, especially those relating to state law. Fourth, we display citations by legal academics in law journals and treatises.

For each of these citation measures, we used the Shepards citation reports. We counted these citations for every published opinion issued during 2004-06 for

19 E.g., Harry T. Edwards, The Growing Disjunction Between Legal Education and the Legal Profession, 91 Mich. L. Rev. 34 (1992).

9

the period until January 1, 2009. We did not collect citations beyond that point to avoid running into the halo effect that might have impacted Sotomayor’s circuit court opinions once her name became actively discussed in the debates over who President Obama might nominate.20

One frequently discussed measure that we do not use is the rate at which a judge is reversed by the Supreme Court. We do not use this measure for a couple of reasons. First, the numbers on this measure tend to be quite small for any given appeals court judge because the Supreme Court takes certiorari on only a few cases from each federal circuit every year. For the three-year window of cases for which we collected data, we would have been comparing a lot of zeroes and ones for the 136 judges in our pool. Second, this was the one quantitative measure regarding Sotomayor’s performance on the lower court that did receive attention during her nomination process. Some initially claimed that her reversal rate was unacceptably high. For example, The Washington Times ran a headline saying “Sotomayor Reversed 60% by High Court” and the Congressional Quarterly quoted Wendy Long, counsel to the Judicial Confirmation Network, as saying that Sotomayor "has an extremely high rate of her decisions being reversed, indicating that she is far more of a liberal activist than even the current liberal activist Supreme Court."21 Others countered that Sotomayor’s rate of reversal (reversed on three out of the five cases that were appealed) was quite acceptable when compared to the overall reversal rate of the High Court during the period in question (roughly in the region of 75%).22

IV. ANALYSIS

As a starting point, it helps to put the numbers that we are going to see within the context of the debate over Sotomayor’s merit. Her detractors characterized her as having a mediocre record as a lower court judge and therefore unqualified on her merits to sit on the Supreme Court. Of those other candidates, the federal judge whose name came up most often, and who was thought to have unimpeachable intellectual stature, was Diane Wood, of the Seventh Circuit.

Where would the Sotomayor-skeptics expect her to rank vis-à-vis the pool of possible alternative candidates for elevation to the High Court? Given the rhetoric

20 We are making an approximation here, since Sotomayor’s name was being actively discussed as a possible candidate for the Court at least as early as October 2008. See Ten Picks for Obama’s Supreme Court, Slate, November 2008 (available at http://www.salon.com/news/feature/2008/11/18/supreme_court ); Sarah Johnson, Granholm Placed on Short List for Supreme Court Nomination, Central Michigan Life, October 2008 (available at http://www.cm-life.com/2008/10/17/granholmplacedonshortlistforsupremecourtnomination/ ) 21 See Stephen Dinan, Sotomayor Reversed 60% by High Court, Washington Times, May 27, 2009. 22 E.g., Sam Stein, Sotomayor’s Reversals No Different from Alito or Souter, Huffington Post (5-27- 2009) (available at http://www.huffingtonpost.com/2009/05/27/sotomayors-reversals-no- d_n_208362.html ).

10

described at the outset from scholars such as Tribe, Turley and Rosen, we assume that the critics would predict Sotomayor to be in the bottom quarter of the distribution of all circuit court judges.

Publications

We propose publication numbers as a rough proxy for judicial effort. Judges themselves talk about how published opinions take greater effort than unpublished ones. Given that the judges have discretion in terms of which opinions they choose to designate as “for publication” and which ones they dispose with a short memorandum or maybe even a couple of words (e.g., “affirmed”), the relative numbers of publications for a judge gives us a rough measure of effort. There are numerous caveats that are in order here though, including the fact that the circuit rules generally tell judges that they are to publish important and precedent-creating opinions. So, judges in circuits that have a greater diversity in terms of disputes may get a lot more cases worthy of precedent-creating opinions. That said, only a small fraction of the cases that show up before judges result in published opinions. In other words, each judge has plenty of opportunities to publish. Some judges just choose to avail themselves of more of these opportunities than others.

A different caveat is that effort exerted in producing a published opinion may be taking away effort from other judicial tasks. This, while perhaps a concern at the district level where judges need to do things like run trials and decide preliminary motions, is less of a concern at the appeals level where the primary task is deciding appeals and explaining the reasons for those decisions. On the other hand, appellate judges may spend less time doing due diligence on the opinions of their fellow judges if the bulk of their time is taken up working on their own opinions.

Published Opinions

Here, we examine the number of opinions that each judge authored that were sent to Westlaw, with a “for publication” designation. We first look to see whether Sotomayor is in the bottom quarter of judges. The top number for the 0-25% range is 53 opinions and Sotomayor, with 90 opinions, is well above that.

The mean number of published opinions, in turn, is 86 opinions. As for the 75% level, the range starts at 109 opinions. At 90 opinions, Sotomayor is above the mean but not in the top 25%. Table I reports the top ten judges by publications. The Seventh Circuit’s Posner and Easterbrook show up at the top.

Table 1: Top Ten Judges; Published Opinions

Judge Circuit Gender Publications Posner 7 Male 276 Easterbrook 7 Male 240

11

Lynch 1 Female 215 Riley 8 Male 198 Smith 8 Male 196 Melloy 8 Male 195 Kanne 7 Male 189 Bye 8 Male 189 Colloton 8 Male 171 Wollman 8 Male 170

Histogram I: Distribution of Publication Rates

.015

.01 Sotomayor: 90 Percentile: 70.58 Density .005 0 0 50 100 150 200 250 Publications

Given the President’s interest in diversity, in appointing a female justice, and the empirical reality that most justices in recent years have been selected from the ranks of the sitting circuit judges, one might argue that the most relevant comparison set of judges for Sotomayor is the set of top performing women judges. When we break the data down in terms of the top ten women publishers, Sotomayor is number seven on that list. Diane Wood, of the Seventh Circuit, is number three.

Published Pages

Raw publication numbers standing alone may not tell us enough about judicial effort. Some circuits may simply have cultures of sending all of their

12

opinions, important or not, to be published. Or there might be individual judges who, for reasons of vanity, simply like seeing their opinions appear in print (judges sometimes refer to the West publications as the “vanity press for judges.”). Sotomayor, therefore, might simply be choosing to publish a lot of her very short opinions that others would not deign to publish. To test for this, we examine the number of pages she published vis-à-vis her competitors.

We begin with the distributional cut offs. The bottom 25% tops out at 455 pages over three years. The mean is 713 pages and the 75% level begins at 769 pages. Sotomayor published 888 pages during 2004-06, putting her in the top quarter of judges in terms of published pages. Table II reports the numbers for the top ten judges overall (two women judges, Rovner and Lynch, are in this group). Sotomayor, with 888 pages, along with Wood, is in the group of top ten women judges.

Histogram II: Distribution of Published Pages

Sotomayor: 888 Percentile: 83.09 .0015 .001 Density 5.0e-04 0 0 500 1000 1500 2000 Published Pages

13

Table II: Top Ten Judges; Published Pages

Judge Court Sex Pages Lynch 1 Female 2010 Ripple 7 Male 1649 Lipez 1 Male 1431 Clay 6 Male 1379 Posner 7 Male 1335 Rovner 7 Female 1315 Kanne 7 Male 1296 Selya 1 Male 1206 Torruella 1 Male 1172 Gilman 6 Male 1153

14

Citations

For citations, we look to four measures: citations by courts federal courts outside the circuit (that is, in regions where we are confident that the cases would not be constitute binding precedent); citations by courts within the circuit; citations by the state courts; and citations in law reviews. As described earlier, each audience here is likely using the citation for different purposes and, therefore, the fact of a citation reveals different information about the judge being cited.

If the accounts by Tribe, Rosen, and Turley have descriptive value in this domain, we’d have little reason to expect Sotomayor to do well with any of the audiences mentioned above. Her judicial opinions have at best been described as workmanlike and at worst been characterized as politically biased. Either way, one would not expect these to be the cases that other judges would cite to when looking for sources of authority. If possible, instead of citing to a Sotomayor opinion, they are more likely to cite to someone eminent, whose name carries weight, like Guido Calabresi or Richard Posner. Moreover, to the extent that citation counts measure a jurist’s impact, we expect undistinguished jurists to be at the bottom of the distribution of this measure.

In discussing the citation counts, as we did above, we begin in each case with a description of the distributions.

Law Journal Citations

This is the measure where we most expected to see Sotomayor scoring near the bottom. Even if her opinions were not as bad as characterized, the fact that many academics appeared to have a low opinion of her work at the outset should have resulted in her scores being low.

The distributional cut off for the bottom 25% of judges on this measure is 175 cites. The mean number of cites, in turn, is 261 cites. Sotomayor’s law journal citation count is 396. That is, more than twice the number at the top of the 25th percentile. She not only slots in well above the 75% level (the bottom number being 323 for the top quarter), but makes it into the top 10% (where the bottom score is 394).

Table III reports the top ten judges in terms of law journal citations. Sotomayor does not make the top ten overall. At the top, Easterbrook and Posner are ahead of the others. Two women judges make the top ten, Lynch of the 1st circuit and Wood of the 7th Circuit. Sotomayor is the third most cited woman judge. The histogram illustrates where she ranks vis-à-vis the rest of the active federal appellate bench.

Table III: Top Ten Judges; Law Journal Citations

Judge Circuit Gender Citations Posner 7 Male 1054 Easterbrook 7 Male 832 Randolph DC Male 675 Lynch 1 Female 575 Birch 11 Male 553 Lipez 1 Male 534 Selya 1 Male 502 Wood 7 Female 482 Thomas 9 Male 459 Reinhardt 9 Male 458

Histogram III: Distribution Pattern of Law Journal Citations

.0067

Sotomayor: 396 Percentile: 90.44 Density 0 0 200 400 600 800 1000 Law Rev/Periodical Citations

Outside Citations

Of the various citation measures used by scholars as proxies for quality or influence, outside-jurisdiction (non precedential) cites are the most commonly used ones. For the most part, courts tend to cite opinions from within their jurisdictions.

16

Citations to courts in other jurisdictions are less common and given out primarily when – so we presume – the opinion in question helps analyze a difficult question. Outside citations, therefore, provide a measure of how useful the analysis in an opinion is, divorced from precedential value.

In the numbers we report, we add the numbers of cites from outside appeals courts to those from outside district courts.23 The distributional cut off at the bottom end of the distribution for outside citations (for the bottom 25%) is 180 cites. The mean number of cites is 310. Sotomayor’s number is 538. At the top end, those with more than 372 citations fall in the top quarter. That puts her at more than 150 cites above the cut off for the top 25% of judges.

Table IV reports the top ten judges, by outside circuit citations. Posner and Easterbrook are at the top. Among the women judges, Lynch and Wood also show up again in the top ten overall. Sotomayor makes the top ten for women judges, slotting in at the third spot after Wood and Lynch.

Table IV: Top Ten Judges; Outside Circuit Citations

Judge Circuit Gender Citations Easterbrook 7 Male 1873 Posner 7 Male 1489 Lynch 1 Female 890 Melloy 8 Male 848 Lipez 1 Male 778 Jacobs 2 Male 734 Wood 7 Female 716 Selya 1 Male 692 Calabresi 2 Male 635 Kanne 7 Male 584

Given that our outside cite measure combines citations for both outside circuit courts and district courts, one might wonder about the breakdown between the two. Arguably, different information is revealed from the fact that a judge has a high district court citation count than from the fact that her high citation count is largely driven by appeals court citations. Below, we report separate histograms on the distributions of outside citations as broken down in terms of level of court. Sotomayor is on the high side of the distribution in both cases, but she does significantly better on citations from other appeals courts.

23 We also add in cites from bankruptcy and tax courts.

17

Histograms IV & V: Outside Circuit and District Citations

.008

Sotomayor: 263 Percentile: 95.59 .006 .004 Density .002 0 0 200 400 600 800 Outside Circuit Citations

18

.004

Sotomayor: 275 Percentile: 83.82 .003 .002 Density .001 0 0 500 1000 1500 Outside District Citations

Inside Citations

A relatively underused citation measure is the number of inside circuit citations. Here, the citations are primarily by lower courts within the circuit, for which the opinions by the circuit are precedential. This does not mean, however, that the sole reason for the citations is that the circuit’s opinion constituted precedent. Courts within the circuit will often have a wide range of cases from various judges that they can choose from to cite. The fact that one judge is consistently cited much more than others, therefore, potentially reveals something about the kinds of opinions that that judge is writing, as compared to his peers. A caveat here is that this measure is going to be significantly biased both in favor of, and against, judges from bigger circuits. In the bigger circuits, because there are many more appeals court decisions and many more district judges, the favored judges will get more cites. But, on the flip side, because the lower court judges have a greater degree of choice in terms of whom to cite, they can show more favoritism and more easily ignore the appellate judges they disfavor. Sotomayor, being on the Second Circuit, is on one of the larger circuits.

Once again, we begin with the distributional characteristics of the variable. At the bottom, at the 25% level, the highest number of citations is 349. The mean number of inside citations is 628. And, at the 75% level, the bottom number is 852.

19

Sotomayor’s number of inside citations is 6182. That puts her at the top for all judges and in the top ten for women judges (along with, yet again, Wood and Lynch).

Table V: Top Ten Judges; Inside Circuit Citations

Judge Circuit Gender Citations Sotomayor 2 Female 6182 Posner 7 Male 5098 Raggi 2 Female 5095 Gould 9 Male 4827 Kanne 7 Male 4670 Cabranes 2 Male 4519 Easterbrook 7 Male 4329 Wood 7 Female 4296 Calabresi 2 Male 4170 Ripple 7 Male 4158

To show the split in Sotomayor’s inside citations between cites from the lower courts and by other panels of the Second circuit, we report the histograms for inside circuit and inside district cites separately. As with the outside citations reported earlier, Sotomayor does better on the inside circuit citations than with the district citations.

20

Histograms VI & VII : Inside Circuit and District Citations

.001 8.0e-04 Percentile: 99.6 Sotomayor: 4106 6.0e-04 Density 4.0e-04 2.0e-04 0 0 1000 2000 3000 4000 Same Circuit Citations

21

8.0e-04

Sotomayor: 2076 Percentile: 95.2 6.0e-04 Density 4.0e-04 2.0e-04 0 0 1000 2000 3000 Same District Citations

State Court Citations

The state court citation measure is also a relatively underused measure. Like the insider circuit citation measure, it also has a bias in that federal courts that sit in more densely populated states are likely to have more state court decisions citing them (because many of the issues are likely similar; particularly, with diversity jurisdiction cases). With that caveat, the bottom quarter of judges, in terms of citations from the state courts tops out at 31 for the three-year period that we use. The mean number of citations is roughly 53 citations for the three years and the cut off for the 75% range is 67 cites.

Sotomayor has 158 cites from the state courts for the 2004-06. That puts her not only in the top quarter of all judges (that is, above the 75% cut off), but also at the very top. The only judge who does better than she does is Judge O’Scannlain, who has one cite more than she does. Table VI reports on the top ten judges in terms of state citations, illustrates. Sotomayor is the only woman judge to make the top ten list for state court citations.

Table VI: Top Ten Judges; State Court Citations

Judge Circuit Gender Citations O'Scannlain 9 Male 159 Sotomayor 2 Female 158

22

Posner 7 Male 157 Howard 1 Male 129 Easterbrook 7 Male 127 Selya 1 Male 118 Gilman 6 Male 116 Lipez 1 Male 115 Colloton 8 Male 111 Torruella 1 Male 103

In terms of the women judges, Sotomayor has the highest score. The histogram provides a graphical illustration.

Histogram VIII: Distribution of State Citations

.015

Percentile: 99.2 Sotomayor: 158 .01 Density .005 0 0 50 100 150 All State Citations

The Team Player Effect

A criticism that could be leveled against the prior measures is that they treat appellate judging as an individual act. But appeals court judges almost always operate in panels (typically three-judge panels, unless there is a rare en banc panel). To fully evaluate the amount a judge contributes in terms of quality, therefore, one should take into account not only the performance of the opinions where she was the primary author, but also the opinions where she was a secondary player.

23

To estimate this secondary author effect, we construct a measure that compares each of our 136 judges, for the three years for which we have data, in terms of citation counts to opinions where they were secondary actors in terms of producing the opinion.

A caveat here is that, at least as a matter of anecdote, overburdened federal judges in the modern era typically spend the vast majority of their time on the opinions where they are the primary author and a small amount of time on those where they are secondary authors. Moreover, the measure does not distinguish between potential ways at contributing to a team. A judge could do more due diligence or threaten to whistle-blow, i.e. dissent, from a poorly constructed majority opinion. A judge could also simply contribute by being present as a diverse input. Or, a judge could do nothing and let the primary author be solely responsible for the opinion. From our perspective, it is not necessary to be able to unpack the precise fashion in which an individual judge contributes to the team product. Since the question being asked involves estimating merit for elevation to the High Court, and since the High Court is more team production oriented than the lower appeals courts (nine justices as opposed to three judges), a judge who contributes positively to the team at the lower level may contribute similarly on the High Court.

In theory, we could generate these secondary author effects for all of our citation measures. However, because of space constraints, we only report it for the primary citation measure used in the literature – outside citations (the sum of citations from federal appeals courts in other circuits and outside federal district courts).24

The top of the range for the bottom 25% is 212 citations. And the mean number of citations is 363. Sotomayor has a count of 615, which is well outside the range of the bottom 25% and much above the mean. The cut off for the top 25% on this measure is 410.5. Sotomayor has a count of 615, which makes the cut off for the top 10%.

24 Other measures are available on request.

24

Histogram IX: Distribution of Team Citations

.004 .003 Sotomayor: 615 Percentile: 89.71 .002 Density .001 0 0 500 1000 1500 Outside

Table VII: Top Ten Judges; Team Citation Effect

Judge Circuit Gender Citations Rovner 7 Female 1436 Williams 7 Female 1433 Wood 7 Female 1394 Howard 1 Male 1113 Kanne 7 Male 1096 Ripple 7 Male 1014 Posner 7 Male 1004 Evans 7 Male 976 Manion 7 Male 908 Riley 8 Male 852

As the table for the top ten judges illustrates though, the problem with the team analysis is that it basically picks up strong circuit effects. That is, circuits that have cultures of high level of productivity will have their judges show up as high performers (of course – since we are measuring team effects). Also, there is likely a Posner-Easterbrook-Wood effect on the Seventh Circuit. The others on their circuits

25

may have high team numbers because there are inevitably sitting with one or two of the triumvirate of Posner-Easterbrook-Wood. The top ten table here, therefore, is dominated by Seventh Circuit judges.

In the top ten list for women, Sotomayor shows up as number 4. Wood (who was also in the top ten overall list) and Lynch also show up here.25

Circuit and Subject Area Controls

As others have noted, the various circuits can differ significantly in terms of court cultures. For example, some might develop cultures of publication, where the judges are under peer pressure to publish and others might, for whatever reason, disfavor publication. The Seventh Circuit, for example, has long had a culture of publishing many more opinions (and garnering many more cites) than the other circuits. Circuits can also differ in terms of the types of cases that they receive. The cases that show up in front of a federal court in Montana are going to be quite different than those that show up in New York City. The former, for example, is unlikely to see a case involving the resolution of the debt crisis in Argentina. Individual judges also will differ in terms of the types of cases they take on – judges, because of their prior backgrounds and the relative expertise levels of their colleagues may choose to take on opinions in particular subject areas and defer to their colleagues in others.

The measures presented thus far have not controlled for these effects. And whether they should or not is a complicated matter. Most of these circuits tend to be small and that means that the culture is, in significant part, likely to be the product of the preferences and collaborative instincts of the judges on the circuit. Put differently, having developed a culture of high productivity in a circuit is arguably an indication of merit in and of itself. The same argument can be made for subject areas. If a judge goes out of her way to pick her writing assignments for cases in subject areas that produce high levels of citations, that fact is itself arguably an indicator of merit. Those caveats aside though, circuit cultures can be a function of long past historical events and the subject areas in which a judge receives cases can be determined by location.

In this final section, therefore, we report on the rankings of judges on the outside citation measure and law journal citation measure, where we control for both circuit and subject area effects. We also control for year effects, since the date

25 One might expect that these measures should be roughly twice the citation count for when judges are authors of the opinions (after all judges write opinions on only about a third of the cases that they decide on). They are not. The reason has to do with the fact that we restrict our analysis to the 136 active judges who authored opinions throughout 2004-2006. Many of the panels also have non- writing judges, such as judges from district courts, senior judges, and judges who were promoted during the timeframe. About 40% of the slots available are assigned to judges who are senior, did not author an opinion, or left during 2004-2006.

26

of publication is likely an important determinant of the number of citations accrued by our cut-off date.26

Formally, our specification is: Outcomeijscy = ßj*Judgej + Yeary + Subjects + εijscy

where i denotes case, j denotes judge, c denotes circuit, y denotes year, and s denotes subject. We consider two outcomes, total outside citations and citations from law reviews and treatises.27 The coefficients of interest are the fixed effects coefficients on the dummy indicators for each judge. These coefficients give us an average citation count per published opinion written by the judges.

In all of these regressions, we omit the dummy indicator for Sotomayor. Consequently, the judge fixed effects indicate each judge’s performance in comparison to Sotomayor and whether the comparison is statistically significantly different.

Column 1 of the table in Appendix A reports unadjusted judge fixed effects without adjusting for year, subject, or circuit. To situate the reader and make these results comparable to the previous descriptive statistics, Histogram X shows the distribution of per capita outside citations and Histogram XII shows the distribution for law review citations. Column 2 controls for year fixed effects and Column 3 controls for year and subject fixed effects. Column 4 standardizes by the mean and standard deviation for the outcomes within each circuit and also controls for year and subject fixed effects. We could not directly include circuit fixed effects since our primary interest is in the judge fixed effects and if we included the circuit fixed effects, 11 judges would drop out due to collinearity. Histogram XI and XIII show the distribution of these standardized fixed effects.

We find that Sotomayor ranks #12 in outside citations per case among all judges (Table A). Only five judges have statistically significantly (at the 10% level) greater outside citations per case than her. Sixty-six judges have statistically significantly lower outside citations per case than her and this number increases slightly when adjusting for year and subject fixed effects. When the citations are normalized by the mean and standard deviation of citations within each circuit, Sotomayor still does better than most judges, as indicated by Histogram XI. None of the judge fixed effects are statistically significant in Column 4 of Table A.

On law reviews also, Sotomayor performs well. Only five judges have statistically significantly greater law review citations per case than her (Table B),

26 Since citations are calculated for all opinions from the 2004-06 period up to January 1, 2009, that means that opinions written in 2004 will garner more citations than those written in 2006, other things equal. 27 Other regressions are available on request.

27

while forty judges have statistically significantly lower law review citations per case than her.

Histogram X: Distribution of Outside Citations per Case

.3

Sotomayor: 5.977 Percentile: 91.18 .2 Density .1 0 0 5 10 15 Outside Citations Per Case

Histogram XI: Distribution of Normalized Outside Citations per Case 20

Sotomayor: 0 15 10 Density 5 0 -.08 -.06 -.04 -.02 0 .02 var1

Histogram XII: Distribution of Law Review Citations per Case

28

.4

Sotomayor: 4.4 Percentile: 83.82 .3 .2 Density .1 0 0 2 4 6 8 10 Law Review & Periodicals

Histogram XIII: Distribution of Normalized Law Review Citations per Case 15 Sotomayor: 0 10 Density 5 0 -.15 -.1 -.05 0 .05 .1 var1

V. OUTLIERS

A final test is to examine whether Sotomayor’s scores are being driven by outliers. For example, perhaps a large fraction of her cites come from just one or two big opinions? Further, what if those one or two opinions were in cases where her assignment to those cases was purely by random chance? Maybe those were

29

cases addressing topics of first impression that then got cited everywhere simply because those were the first cases of those types?28 To test for the outlier effect, we winsorize our citation data to address the possibility that outliers are driving the results. Specifically, we replace the top 0.5% extreme values with the 99.5%-ile value and replace the bottom 0.5% extreme values with the 0.5%-ile. We restrict our attention to the two primary forms of citation measures: outside citations and citations law review publications.

The results indicate that Sotomayor’s relative performance against the other judges is not being driven by the presence of outliers. Indeed, she performs slightly better against her colleagues when the data are adjusted for outliers. On the winsorized measures, Sotomayor is in the top 10 among all judges for outside citations. She is still at the 90.44%-ile for law review citations. Posner, Easterbrook, Wood and Lynch, all show up again, in both top ten tables.

TABLE VIII: TOP TEN JUDGES ON OUTSIDE CITATIONS (WINSORIZED)

Judge Circuit Gender Citations Posner 7 Male 1310 Easterbrook 7 Male 1270 Lynch 1 Female 883 Selya 1 Male 674 Wood 7 Female 674 Ripple 7 Male 576 Kanne 7 Male 568 Lipez 1 Male 559 Sotomayor 2 Female 498 McConnell 10 Male 492

28 The issue of outliers obviously does not apply to the measure of published opinions.

30

HISTOGRAM XIV: DISTRIBUTION OF OUTSIDE CITATIONS (WINSORIZED) .004

.003 Sotomayor: 498 Percentile: 93.38 .002 Density .001 0 0 500 1000 1500 W Outside

TABLE IX: TOP TEN JUDGES -- LAW REVIEWS & PERIODICAL CITATIONS (WINSORIZED)

Judge Circuit Gender Citations Posner 7 Male 946 Easterbrook 7 Male 768 Lynch 1 Female 575 Selya 1 Male 502 Wood 7 Female 457 Lipez 1 Male 440 Ripple 7 Male 394 Reinhardt 9 Male 389 Cabranes 2 Male 383 Smith 5 Male 381

31

HISTOGRAM XV: DISTRIBUTION OF LAW REVIEW & PERIODICAL CITATIONS (WINSORIZED)

.004

Sotomayor: 342 Percentile: 90.44 .003 Density .002 .001 0 0 200 400 600 800 1000 W Law Review & Periodical

Unreported here, we also examined the winsorized outside citation regressions (where, as described earlier, we control for matters such as circuit, year and subject area effects).29 Once again, Sotomayor performs better with the winsorizing process. Previously, in the raw data, six judges performed statistically significantly better than Sotomayor. Now, in the winsorized data, no judge performs statistically significantly better than Sotomayor and forty additional judges perform statistically significantly worse than her.

V. CONCLUSION

Sotomayor’s critics described her as a mediocre judge; one unworthy to take a place on the High Court. We compared her performance whilst on the Second Circuit against that of all the other active federal appeals court judges over a three- year period. Sotomayor was easily in the top 25% of all of the judges on the Court of Appeals in almost all of the categories that we examined. Moreover, in many categories she was in the top 10%. These results should at least bring into question the claims of her mediocrity. Indeed, she was among an elite group of judges on the Court of Appeals. Some might say that that a candidate for the Supreme Court should show up in the top 5% or 10% before being considered worthy of a Supreme

29 These results are available from the authors.

32

Court nomination. But of course we do not apply this standard to all candidates considered for elevation. We ask only whether the judge is in the pool of elite judges that ought to be considered for elevation.

Table X sets out a summary of her performance on the various measures.

Table X: Sotomayor’s Performance Vis-à-vis Nine Measures

Measure Bottom Above Top Top Ten Top Percentile 25% the 25% (Female) Ten Mean (Over all) Publications N Y N Y N 70.6% Pages N Y Y Y N 83.1% Law Journal Cites N Y Y Y N 90.4% Outside Circuit N Y Y Y N 95.6% Cites Inside Circuit N Y Y Y Y 99.6% Cites State Court Cites N Y Y Y Y 99.2% Secondary Author N Y Y Y N 89.7% Effect – Outside Cites Controlling for N Y Y Y N Outside Cites: Subject Area 91.2% Effects L. Rev Cites: 83.8% Normalizing for N Y Y N N Outside Circuit Effects Cites:82.4% L. Rev Cites: 55.9%

A related question is whether the treatment Sotomayor received was any different from what other candidates from the appeals court would have received. We cannot do an effective comparison here because so few appeals court judges get promoted to the High Court. But a couple of casual comparisons do suggest that the treatment Sotomayor received, in terms of questions about her intellectual capabilities as a judge, was different. One contemporaneous comparison is to Diane Wood, who performed at roughly the same level as Sotomayor and was a serious contender for the Court at least twice in recent years.30 Wood’s intellectual

30 Wood also performed at a high level in a prior study examining a time period roughly five years prior. See Choi & Gulati, supra note __.

33

capabilities were generally lauded, and certainly not challenged in anywhere near the fashion as Sotomayor’s were.

Justice Alito, who was elevated to the High Court some years prior, provides another imperfect comparison. In a prior study, done prior to Alito’s nomination, his performance had been ranked against that of his colleagues. Like we did for Sotomayor, the data on him was examined for a three-year period that came roughly towards the middle of his tenure on the bench.31 Overall, he was sixteenth out of a sample of roughly a hundred federal appellate judges, who were under the age of 65 – to our minds, a high level of performance.32 The nomination was contentious and Alito was attacked from many directions. But his intellectual capabilities did not receive serious challenge – if anything, there was seemed to be general agreement regarding his high level of intellect. With Sotomayor, however, things were different.

Our point though is not to complain about the treatment that Sotomayor received. It is to raise the question of whether, in cases like this, where there is a risk of bias with the subjective measures, more attention should be paid to objective measures.33 When there is a disconnect between the subjective conclusions and the objective measures, it may be worth looking deeper.

Ultimately, do our data tell us anything about Sotomayor’s future performance as a Supreme Court justice? Presumably, those who opposed her nomination on the ground that she was not that smart were also making a prediction with respect to her impact on the Supreme Court. On the basis of our measures, we are willing to wager that she surprises everyone by outperforming the majority of her colleagues on the Court.

31 Unlike in our current study though, Justice Alito’s performance was not the primary subject of the prior study. 32 Our current sample looks at a broader set of judges than the prior study did, since we look at all active judges and not just the ones under the age of 65. If we were to assume that the additional judges would have all performed in the bottom half of the distribution, then Alito would be roughly near the top ten per cent of all judges (16 out of 136). Another issue to note in the Alito comparison is that Alito’s scores were particularly high on a category that we did not estimate for this project, independence. Independence in the prior study was measured as a function of dissents against other judges with the same party affiliation (the overall measure being normalized for the relative numbers of judges from each party on the circuit in question). See Choi & Gulati, supra note __. 33 We should point out that our analysis cannot completely separate individual merit from a possible diversity effect. Perhaps being on the Second Circuit, Sotomayor got highly cited because of her “different” perspective vis-a-vis her colleagues. Nevertheless, a clean measure of individual merit is not necessary to ask whether Sotomayor was as meritorious as other candidate judges, since she would bring her diversity effect to the Supreme Court as well.

34

Appendix A

Table A: Outside Citations per Case (1) (2) (3) (4) Outside Citations Normalized Outside Citations Barry3 8.594 8.817 8.855 0.0253 [2.431]*** [2.426]*** [2.425]*** [0.240] Scirica3 4.308 3.986 3.799 -0.0863 [1.998]** [1.994]** [1.996]* [0.198] Batchelder6 4.258 4.149 4.192 -0.00702 [1.758]** [1.754]** [1.753]** [0.174] Anderson11 3.701 3.677 3.606 -0.0264 [2.171]* [2.166]* [2.166]* [0.215] Calabresi2 2.966 2.958 3.074 0.0000161 [1.592]* [1.589]* [1.588]* [0.158] Hull11 2.244 2.271 2.276 -0.00317 [1.978] [1.974] [1.976] [0.196] Jacobs2 2.178 2.161 2.127 -0.018 [1.495] [1.492] [1.492] [0.148] Easterbrook7 1.826 1.719 1.566 -0.0442 [1.240] [1.237] [1.238] [0.123] Marcus11 1.28 1.304 1.485 0.0304 [1.626] [1.622] [1.631] [0.162] King5 1.238 1.123 1.173 -0.00997 [1.633] [1.629] [1.629] [0.162] Wilkins4 0.815 0.59 0.464 -0.0326 [1.737] [1.733] [1.734] [0.172] Raggi2 -0.204 -0.344 -0.48 -0.0344 [1.737] [1.733] [1.732] [0.172] Wilson11 -0.248 -0.245 -0.0762 0.0281 [1.959] [1.955] [1.960] [0.194] Lipez1 -0.257 -0.337 -0.49 -0.0344 [1.363] [1.360] [1.360] [0.135] Paez9 -0.382 -0.571 -0.543 -0.0377 [1.805] [1.802] [1.802] [0.179] Traxler4 -0.503 -0.582 -0.769 -0.0285 [1.906] [1.902] [1.902] [0.189] Katzmann2 -0.539 -0.582 -0.579 -0.018 [1.890] [1.886] [1.887] [0.187] Posner7 -0.583 -0.654 -0.586 -0.0154 [1.218] [1.215] [1.216] [0.121] Carnes11 -0.589 -0.544 -0.598 0.00422 [1.586] [1.583] [1.584] [0.157]

35

Birch11 -0.621 -0.758 -0.816 -0.0537 [1.707] [1.704] [1.706] [0.169] Wilkinson4 -0.657 -0.757 -0.804 -0.0367 [1.536] [1.533] [1.534] [0.152] Sack2 -0.937 -0.924 -1.101 -0.0293 [1.580] [1.577] [1.575] [0.156] Shedd4 -0.955 -1.091 -1.219 -0.0267 [1.860] [1.856] [1.855] [0.184] Duncan4 -1.01 -1.222 -1.334 -0.0472 [2.089] [2.085] [2.085] [0.207] Reinhardt9 -1.043 -1.148 -1.324 -0.0619 [1.557] [1.554] [1.555] [0.154] McKeown9 -1.175 -1.216 -1.356 -0.0484 [1.626] [1.622] [1.624] [0.161] Cabranes2 -1.26 -1.403 -1.504 -0.0388 [1.487] [1.484] [1.484] [0.147] Williams4 -1.272 -1.208 -1.203 0.0064 [1.758] [1.754] [1.755] [0.174] Graber9 -1.281 -1.384 -1.538 -0.0456 [1.626] [1.622] [1.622] [0.161] Tjoflat11 -1.422 -1.65 -1.672 -0.0383 [1.586] [1.583] [1.587] [0.157] Sloviter3 -1.503 -1.635 -1.892 -0.0514 [1.552] [1.549] [1.548] [0.154] Davis5 -1.526 -1.594 -1.473 0.00784 [1.656] [1.652] [1.654] [0.164] Selya1 -1.598 -1.736 -1.858 -0.0402 [1.325] [1.322] [1.321] [0.131] Smith3 -1.608 -1.62 -1.693 -0.015 [1.818] [1.814] [1.814] [0.180] Wood7 -1.612 -1.75 -1.852 -0.0413 [1.316] [1.313] [1.313] [0.130] Melloy8 -1.629 -1.747 -1.81 -0.0226 [1.278] [1.276] [1.277] [0.127] Rendell3 -1.63 -1.764 -1.884 -0.0583 [1.818] [1.814] [1.815] [0.180] Thomas9 -1.705 -1.86 -1.871 -0.0516 [1.717] [1.714] [1.716] [0.170] Dubina11 -1.728 -1.74 -1.755 -0.0156 [1.793] [1.789] [1.789] [0.177] McConnell10 -1.773 -1.714 -1.754 0.0139 [1.406] [1.403] [1.405] [0.139] Tymkovich10 -1.809 -1.829 -1.988 -0.0107

36

[1.499] [1.496] [1.496] [0.148] Lynch1 -1.838 -1.973 -2.028 -0.0425 [1.259] [1.257] [1.256] [0.125] Sutton6 -1.893 -1.881 -1.978 -0.0159 [1.527] [1.523] [1.523] [0.151] Fisher3 -1.931 -1.786 -1.992 -0.0213 [1.860] [1.856] [1.855] [0.184] Black11 -1.978 -2.075 -2.029 0.00709 [1.874] [1.871] [1.871] [0.186] Ambro3 -2.023 -1.95 -2.071 -0.0176 [1.499] [1.496] [1.496] [0.148] Wiener5 -2.026 -2.139 -2 -0.000305 [1.656] [1.652] [1.652] [0.164] Boudin1 -2.064 -2.153 -2.159 -0.00979 [1.382] [1.379] [1.379] [0.137] Michael4 -2.069 -2.2 -2.238 -0.0301 [1.717] [1.713] [1.714] [0.170] Moore6 -2.088 -2.029 -2.224 -0.00775 [1.429] [1.426] [1.426] [0.141] Murphy10 -2.101 -2.166 -2.331 -0.024 [1.536] [1.533] [1.535] [0.152] Fletcher9 -2.167 -2.315 -2.401 -0.0573 [1.574] [1.571] [1.572] [0.156] Fuentes3 -2.182 -2.138 -2.257 -0.0127 [1.781] [1.777] [1.776] [0.176] Pooler2 -2.228 -2.266 -2.319 -0.0292 [1.586] [1.583] [1.582] [0.157] Gibbons6 -2.249 -2.248 -2.416 -0.0204 [1.599] [1.595] [1.594] [0.158] Hartz10 -2.282 -2.382 -2.425 -0.00494 [1.380]* [1.377]* [1.378]* [0.137] Cook6 -2.364 -2.487 -2.49 -0.0166 [1.845] [1.841] [1.841] [0.183] Fisher9 -2.369 -2.523 -2.622 -0.0424 [1.605] [1.602] [1.602] [0.159] Garland12 -2.39 -2.512 -2.258 0.02 [1.648] [1.644] [1.651] [0.164] Bybee9 -2.416 -2.426 -2.491 -0.00806 [1.698] [1.694] [1.694] [0.168] Ripple7 -2.426 -2.522 -2.523 -0.0198 [1.317]* [1.315]* [1.317]* [0.131] Berzon9 -2.437 -2.508 -2.595 -0.0344 [1.517] [1.514]* [1.514]* [0.150]

37

Smith5 -2.443 -2.401 -2.566 -0.00933 [1.409]* [1.406]* [1.406]* [0.139] Parker2 -2.444 -2.467 -2.546 -0.0199 [1.672] [1.668] [1.668] [0.165] Jones5 -2.497 -2.668 -2.819 -0.0502 [1.557] [1.554]* [1.554]* [0.154] Hawkins9 -2.521 -2.8 -2.865 -0.0748 [1.998] [1.994] [1.996] [0.198] Wesley2 -2.528 -2.56 -2.588 -0.0122 [1.906] [1.902] [1.903] [0.189] Boggs6 -2.557 -2.739 -2.918 -0.0414 [1.605] [1.602]* [1.602]* [0.159] Kelly10 -2.581 -2.683 -2.67 -0.00763 [1.580] [1.577]* [1.578]* [0.157] Higginbotham5 -2.615 -2.596 -2.549 0.0196 [1.541]* [1.538]* [1.538]* [0.152] Motz4 -2.69 -2.803 -2.775 -0.0148 [1.580]* [1.577]* [1.581]* [0.157] Straub2 -2.717 -2.811 -2.738 -0.012 [1.818] [1.814] [1.813] [0.180] Ginsburg12 -2.745 -2.912 -2.653 0.00539 [1.580]* [1.577]* [1.585]* [0.157] Gould9 -2.761 -2.917 -2.979 -0.0297 [1.468]* [1.465]** [1.467]** [0.145] Howard1 -2.765 -2.821 -2.886 -0.00756 [1.394]** [1.391]** [1.390]** [0.138] Pregerson9 -2.781 -2.789 -2.946 -0.042 [1.592]* [1.589]* [1.589]* [0.158] Lucero10 -2.793 -2.901 -2.74 0.0098 [1.487]* [1.484]* [1.484]* [0.147] Henry10 -2.884 -3.075 -3.025 -0.0249 [1.640]* [1.637]* [1.639]* [0.163] Clifton9 -2.887 -3.082 -3.214 -0.0554 [1.845] [1.842]* [1.841]* [0.183] Kanne7 -2.888 -2.953 -3.1 -0.0305 [1.285]** [1.282]** [1.284]** [0.127] Daughtrey6 -2.947 -2.954 -2.911 0.00073 [2.041] [2.037] [2.037] [0.202] O_Scannlain9 -3.001 -3.011 -3.031 -0.0111 [1.504]** [1.501]** [1.501]** [0.149] Manion7 -3.03 -3.108 -3.123 -0.0209 [1.409]** [1.406]** [1.408]** [0.140] Wardlaw9 -3.037 -3.024 -3.259 -0.052

38

[1.758]* [1.755]* [1.756]* [0.174] Luttig4 -3.079 -3.285 -3.219 -0.0239 [1.680]* [1.677]* [1.678]* [0.166] Rovner7 -3.106 -3.284 -3.338 -0.0389 [1.316]** [1.313]** [1.314]** [0.130] Flaum7 -3.123 -3.251 -3.28 -0.0326 [1.346]** [1.343]** [1.346]** [0.133] Briscoe10 -3.212 -3.259 -3.266 -0.00406 [1.536]** [1.533]** [1.533]** [0.152] Tacha10 -3.238 -3.361 -3.524 -0.0212 [1.580]** [1.577]** [1.577]** [0.156] Kleinfeld9 -3.258 -3.392 -3.601 -0.0424 [1.698]* [1.695]** [1.694]** [0.168] O_Brien10 -3.263 -3.219 -3.458 -0.0164 [1.874]* [1.870]* [1.872]* [0.186] Niemeyer4 -3.289 -3.365 -3.41 -0.0357 [1.557]** [1.554]** [1.555]** [0.154] Williams7 -3.308 -3.495 -3.57 -0.0413 [1.389]** [1.386]** [1.387]** [0.138] Clay6 -3.311 -3.327 -3.367 -0.000371 [1.371]** [1.368]** [1.368]** [0.136] Cole6 -3.32 -3.484 -3.455 -0.0159 [1.580]** [1.577]** [1.576]** [0.156] Jolly5 -3.378 -3.528 -3.496 -0.0225 [1.568]** [1.565]** [1.565]** [0.155] Sentelle12 -3.386 -3.568 -3.246 0.0204 [1.592]** [1.589]** [1.605]** [0.159] Gilman6 -3.393 -3.401 -3.52 -0.00464 [1.323]** [1.320]** [1.320]*** [0.131] Randolph12 -3.411 -3.519 -3.474 -0.0122 [1.619]** [1.615]** [1.622]** [0.161] Murphy8 -3.444 -3.494 -3.637 -0.0172 [1.365]** [1.362]** [1.364]*** [0.135] Silverman9 -3.448 -3.623 -3.857 -0.0778 [2.019]* [2.015]* [2.015]* [0.200] Rawlinson9 -3.457 -3.71 -4.048 -0.0888 [1.793]* [1.789]** [1.789]** [0.177] Loken8 -3.472 -3.493 -3.589 -0.0147 [1.322]*** [1.319]*** [1.320]*** [0.131] Colloton8 -3.51 -3.543 -3.566 0.00315 [1.306]*** [1.304]*** [1.305]*** [0.129] McKee3 -3.51 -3.957 -4.086 -0.0846 [1.805]* [1.802]** [1.802]** [0.179]

39

Barkett11 -3.522 -3.704 -3.828 -0.0467 [1.698]** [1.695]** [1.695]** [0.168] Benavides5 -3.544 -3.446 -3.411 0.0182 [1.527]** [1.523]** [1.522]** [0.151] Tatel12 -3.545 -3.752 -3.365 0.0148 [1.574]** [1.571]** [1.579]** [0.157] King4 -3.549 -3.612 -3.666 -0.023 [1.599]** [1.595]** [1.596]** [0.158] Kozinski9 -3.568 -3.647 -3.824 -0.0379 [1.664]** [1.660]** [1.661]** [0.165] Tallman9 -3.594 -3.823 -3.95 -0.0581 [1.672]** [1.669]** [1.668]** [0.165] Schroeder9 -3.599 -3.828 -3.767 -0.0397 [1.959]* [1.955]* [1.958]* [0.194] Rogers6 -3.641 -3.736 -3.733 -0.0059 [1.426]** [1.423]*** [1.423]*** [0.141] Clement5 -3.657 -3.773 -3.948 -0.0333 [1.536]** [1.533]** [1.533]** [0.152] Torruella1 -3.663 -3.733 -3.775 -0.0154 [1.314]*** [1.312]*** [1.311]*** [0.130] DeMoss5 -3.675 -3.957 -3.765 -0.02 [1.563]** [1.560]** [1.561]** [0.155] Prado5 -3.692 -3.773 -3.846 -0.026 [1.557]** [1.554]** [1.554]** [0.154] Barksdale5 -3.728 -3.796 -4.026 -0.0334 [1.845]** [1.841]** [1.844]** [0.183] Martin6 -3.731 -3.82 -3.727 0.00292 [1.351]*** [1.349]*** [1.348]*** [0.134] Wollman8 -3.754 -3.845 -3.91 -0.0131 [1.308]*** [1.305]*** [1.306]*** [0.130] Rymer9 -3.811 -4.058 -4.221 -0.0721 [1.727]** [1.723]** [1.724]** [0.171] Evans7 -3.815 -3.862 -3.904 -0.0185 [1.378]*** [1.375]*** [1.376]*** [0.136] Henderson12 -3.819 -3.921 -3.578 0.0306 [1.504]** [1.501]*** [1.517]** [0.151] Riley8 -3.892 -4.025 -4.09 -0.0223 [1.275]*** [1.273]*** [1.276]*** [0.127] Arnold8 -4.028 -4.081 -4.145 -0.00373 [1.323]*** [1.320]*** [1.323]*** [0.131] Gregory4 -4.072 -4.274 -4.361 -0.0542 [1.640]** [1.637]*** [1.637]*** [0.162] Edmondson11 -4.078 -4.493 -4.576 -0.0333

40

[3.344] [3.337] [3.334] [0.331] Smith8 -4.08 -4.18 -4.279 -0.0199 [1.277]*** [1.275]*** [1.277]*** [0.127] Dennis5 -4.122 -4.08 -4.15 0.00593 [1.527]*** [1.524]*** [1.523]*** [0.151] Bea9 -4.166 -4.222 -4.264 -0.0112 [1.737]** [1.733]** [1.733]** [0.172] Bye8 -4.348 -4.509 -4.444 -0.0133 [1.285]*** [1.282]*** [1.283]*** [0.127] Garza5 -4.362 -4.332 -4.324 -0.00327 [1.420]*** [1.417]*** [1.422]*** [0.141] Stewart5 -4.417 -4.348 -4.301 0.0131 [1.464]*** [1.461]*** [1.462]*** [0.145] Rogers12 -4.506 -4.58 -4.306 0.0309 [1.499]*** [1.496]*** [1.511]*** [0.150] Callahan9 -5.053 -5.133 -5.223 -0.0416 [1.737]*** [1.733]*** [1.733]*** [0.172]

Year 2005 -0.185 -0.232 -0.0658 [0.230] [0.230] [0.0228]*** Year 2006 -1.495 -1.514 -0.222 [0.230]*** [0.230]*** [0.0228]*** Subject Matter2 -1.919 -0.246 [1.433] [0.142]* Subject Matter3 0.129 0.0532 [1.128] [0.112] Subject Matter4 -0.0417 0.0418 [1.507] [0.149] Subject Matter5 1.243 0.142 [0.520]** [0.0516]*** Subject Matter6 0.215 0.121 [0.851] [0.0844] Subject Matter7 1.875 0.368 [0.665]*** [0.0660]*** Subject Matter8 0.718 0.154 [1.549] [0.154] Subject Matter9 0.119 0.0466 [0.840] [0.0833] Subject Matter10 1.782 0.369 [1.210] [0.120]*** Subject Matter11 0.879 0.209 [0.593] [0.0589]*** Subject Matter12 1.988 0.382

41

[0.932]** [0.0925]*** Subject Matter13 0.636 0.141 [0.557] [0.0553]** Subject Matter14 0.564 0.11 [0.881] [0.0874] Subject Matter15 -0.544 -0.026 [0.614] [0.0609] Subject Matter16 0.743 0.0865 [0.588] [0.0583] Subject Matter17 -0.683 -0.067 [1.183] [0.117] Subject Matter18 -0.849 -0.0783 [0.951] [0.0944] Subject Matter19 -0.0995 -0.00151 [0.697] [0.0691] Subject Matter20 1.006 0.106 [0.641] [0.0636]* Subject Matter21 3.582 0.528 [0.980]*** [0.0972]*** Constant 5.978 6.635 5.892 -0.01 [1.057]*** [1.065]*** [1.174]*** [0.116] N 11679 11679 11674 11674 R-sq 0.028 0.032 0.037 0.02 Standard errors in brackets ="* p<0.10 ** p< 0.05 *** p<0.01"

42

Table B: Law Review Citations per Case (1) (2) (3) (4) Law Reviews Normalized Law Reviews Randolph12 5.675 5.559 5.611 -0.0114 [1.194]*** [1.190]*** [1.184]*** [0.159] Birch11 5.475 5.344 5.411 -0.0175 [1.260]*** [1.255]*** [1.245]*** [0.167] Thomas9 3.945 3.75 3.592 -0.0556 [1.267]*** [1.263]*** [1.252]*** [0.168] Katzmann2 2.99 2.96 2.525 -0.0843 [1.395]** [1.390]** [1.377]* [0.184] Wilkins4 2.958 2.737 2.837 -0.048 [1.282]** [1.277]** [1.265]** [0.170] Wesley2 2.1 2.076 2.076 -0.0135 [1.407] [1.401] [1.388] [0.186] Scirica3 2.057 1.758 1.58 -0.0966 [1.475] [1.469] [1.456] [0.195] Reinhardt9 1.548 1.453 1.622 0.0125 [1.149] [1.145] [1.135] [0.152] Hull11 1.517 1.548 1.688 0.0133 [1.460] [1.454] [1.442] [0.193] Fisher9 1.165 0.985 1.008 -0.0312 [1.184] [1.180] [1.169] [0.157] Schroeder9 0.87 0.642 0.1 -0.128 [1.446] [1.440] [1.428] [0.191] Paez9 0.409 0.26 0.357 -0.0116 [1.332] [1.327] [1.315] [0.176] Ambro3 0.353 0.426 0.635 0.0469 [1.107] [1.103] [1.091] [0.146] Sack2 0.271 0.277 0.215 -0.0124 [1.166] [1.162] [1.149] [0.154] Ginsburg12 0.244 0.0729 0.581 0.0559 [1.166] [1.162] [1.156] [0.155] Sentelle12 0.22 0.0594 0.688 0.0809 [1.175] [1.171] [1.171] [0.157] Straub2 0.209 0.142 0.135 -0.019 [1.342] [1.337] [1.323] [0.177] King5 0.138 0.0195 0.15 -0.00312 [1.205] [1.201] [1.189] [0.159] Luttig4 0.0746 -0.161 -0.0869 -0.0335 [1.240] [1.236] [1.224] [0.164] Anderson11 0.0643 0.0154 -0.136 -0.0344 [1.602] [1.596] [1.580] [0.212]

43

Sutton6 0.0217 0.0319 -0.1 -0.0231 [1.126] [1.122] [1.111] [0.149] Cabranes2 -0.0739 -0.22 -0.0279 -0.00743 [1.097] [1.094] [1.082] [0.145] Edmondson11 -0.2 -0.572 -0.482 -0.0647 [2.467] [2.459] [2.433] [0.326] Marcus11 -0.203 -0.159 -0.0939 0.00438 [1.200] [1.195] [1.190] [0.159] Tjoflat11 -0.275 -0.517 -0.294 -0.0263 [1.170] [1.166] [1.158] [0.155] Raggi2 -0.343 -0.477 -0.394 -0.0268 [1.282] [1.277] [1.264] [0.169] Motz4 -0.4 -0.529 -0.637 -0.0523 [1.166] [1.162] [1.153] [0.155] Wilkinson4 -0.4 -0.504 -0.309 0.00723 [1.134] [1.130] [1.119] [0.150] Calabresi2 -0.442 -0.45 -0.288 0.026 [1.175] [1.171] [1.159] [0.155] Lipez1 -0.474 -0.553 -0.425 0.00178 [1.006] [1.002] [0.992] [0.133] McKeown9 -0.491 -0.55 -1.302 -0.125 [1.200] [1.195] [1.185] [0.159] Wardlaw9 -0.498 -0.546 -0.479 0.0116 [1.297] [1.293] [1.281] [0.172] O_Scannlain9 -0.502 -0.526 -0.667 -0.0183 [1.110] [1.106] [1.095] [0.147] Smith3 -0.509 -0.52 -0.265 0.0384 [1.342] [1.337] [1.323] [0.177] Jones5 -0.543 -0.719 -0.728 -0.0448 [1.149] [1.145] [1.133] [0.152] Wilson11 -0.562 -0.567 -0.762 -0.05 [1.446] [1.440] [1.430] [0.192] Fuentes3 -0.563 -0.517 -0.465 0.000937 [1.314] [1.309] [1.296] [0.174] Posner7 -0.581 -0.672 -0.384 0.0288 [0.899] [0.895] [0.887] [0.119] Barry3 -0.638 -0.466 -0.111 0.0976 [1.794] [1.788] [1.769] [0.237] Carnes11 -0.65 -0.604 -0.605 -0.00965 [1.170] [1.166] [1.156] [0.155] Kleinfeld9 -0.716 -0.868 -1.071 -0.0611 [1.253] [1.249] [1.236] [0.166] Clement5 -0.733 -0.85 -0.761 -0.0146

44

[1.134] [1.130] [1.118] [0.150] Wiener5 -0.739 -0.85 -0.569 0.0263 [1.222] [1.217] [1.205] [0.161] Davis5 -0.819 -0.889 -0.659 0.00993 [1.222] [1.217] [1.207] [0.162] Michael4 -0.873 -1.002 -0.817 0.00769 [1.267] [1.262] [1.250] [0.168] Pregerson9 -0.907 -0.915 -1.154 -0.0293 [1.175] [1.171] [1.160] [0.155] Easterbrook7 -0.933 -1.035 -0.83 0.00521 [0.915] [0.912] [0.903] [0.121] Hawkins9 -0.943 -1.198 -1.039 -0.0281 [1.475] [1.469] [1.456] [0.195] Batchelder6 -0.949 -1.026 -1.042 -0.0251 [1.297] [1.293] [1.279] [0.171] Gibbons6 -0.971 -0.968 -1.125 -0.0276 [1.180] [1.175] [1.163] [0.156] Shedd4 -0.981 -1.107 -0.89 0.00101 [1.372] [1.367] [1.353] [0.181] Cole6 -1.003 -1.166 -1.281 -0.0536 [1.166] [1.162] [1.150] [0.154] Rogers12 -1.018 -1.09 -0.564 0.0771 [1.107] [1.103] [1.102] [0.148] Dubina11 -1.046 -1.068 -1.093 -0.00119 [1.323] [1.318] [1.305] [0.175] Fletcher9 -1.049 -1.228 -1.629 -0.0896 [1.162] [1.158] [1.147] [0.154] Rendell3 -1.074 -1.198 -1.291 -0.0326 [1.342] [1.337] [1.324] [0.177] Tallman9 -1.083 -1.328 -1.469 -0.0719 [1.234] [1.229] [1.217] [0.163] Tacha10 -1.085 -1.206 -1.066 -0.0168 [1.166] [1.162] [1.151] [0.154] Black11 -1.09 -1.188 -1.123 0.000534 [1.383] [1.378] [1.365] [0.183] Pooler2 -1.094 -1.135 -0.894 0.0357 [1.170] [1.166] [1.154] [0.155] Daughtrey6 -1.097 -1.101 -1.02 0.0208 [1.506] [1.501] [1.486] [0.199] Smith5 -1.116 -1.078 -0.989 0.0167 [1.040] [1.036] [1.026] [0.137] Higginbotham5 -1.137 -1.084 -0.786 0.0547 [1.137] [1.133] [1.122] [0.150]

45

Murphy10 -1.141 -1.213 -1.241 -0.0302 [1.134] [1.130] [1.120] [0.150] Jacobs2 -1.211 -1.228 -1.228 0.00745 [1.103] [1.099] [1.089] [0.146] Boggs6 -1.212 -1.392 -1.536 -0.0579 [1.184] [1.180] [1.169] [0.157] Selya1 -1.223 -1.355 -1.243 -0.00828 [0.978] [0.974] [0.964] [0.129] Berzon9 -1.235 -1.311 -1.455 -0.0375 [1.120] [1.115] [1.104] [0.148] Gould9 -1.235 -1.377 -1.529 -0.0636 [1.083] [1.079] [1.070] [0.143] Cook6 -1.286 -1.415 -1.118 0.014 [1.362] [1.357] [1.343] [0.180] Jolly5 -1.373 -1.547 -1.314 0.00384 [1.157] [1.153] [1.142] [0.153] Benavides5 -1.4 -1.309 -1.242 0.0225 [1.126] [1.122] [1.111] [0.149] Traxler4 -1.4 -1.489 -1.726 -0.0483 [1.407] [1.401] [1.387] [0.186] Graber9 -1.415 -1.529 -1.546 -0.0167 [1.200] [1.195] [1.183] [0.159] Wood7 -1.461 -1.563 -1.328 0.0137 [0.971] [0.968] [0.958] [0.128] Barkett11 -1.47 -1.668 -1.478 -0.0088 [1.253] [1.249] [1.236] [0.166] Boudin1 -1.494 -1.581 -1.501 -0.00609 [1.020] [1.016] [1.006] [0.135] Kozinski9 -1.498 -1.573 -1.408 0.00837 [1.228] [1.223] [1.211] [0.162] King4 -1.514 -1.577 -1.68 -0.0304 [1.180] [1.175] [1.164] [0.156] DeMoss5 -1.532 -1.822 -1.522 -0.0105 [1.153] [1.149] [1.139] [0.153] Bybee9 -1.558 -1.546 -1.704 -0.0225 [1.253] [1.248] [1.236] [0.166] Garza5 -1.561 -1.531 -1.263 0.0328 [1.048] [1.044] [1.037] [0.139] Williams4 -1.576 -1.533 -1.168 0.0614 [1.297] [1.293] [1.281] [0.172] Barksdale5 -1.605 -1.699 -1.631 -0.0299 [1.362] [1.357] [1.345] [0.180] Niemeyer4 -1.608 -1.69 -1.677 -0.00806

46

[1.149] [1.145] [1.134] [0.152] McConnell10 -1.639 -1.569 -1.489 0.0156 [1.038] [1.034] [1.025] [0.137] Rawlinson9 -1.65 -1.909 -2.345 -0.11 [1.323] [1.318] [1.306]* [0.175] Rymer9 -1.659 -1.881 -2.221 -0.0868 [1.274] [1.270] [1.258]* [0.169] Henry10 -1.666 -1.847 -1.875 -0.0475 [1.210] [1.206] [1.196] [0.160] Parker2 -1.7 -1.72 -1.587 0.00934 [1.234] [1.229] [1.217] [0.163] Lynch1 -1.726 -1.858 -1.715 0.00254 [0.929]* [0.926]** [0.917]* [0.123] Martin6 -1.745 -1.848 -1.825 -0.0137 [0.997]* [0.994]* [0.983]* [0.132] Sloviter3 -1.746 -1.87 -1.703 -0.00727 [1.145] [1.141] [1.129] [0.151] Clifton9 -1.855 -2.066 -2.067 -0.0396 [1.362] [1.357] [1.343] [0.180] Kelly10 -1.866 -1.966 -1.755 0.0065 [1.166] [1.162]* [1.151] [0.154] Clay6 -1.877 -1.885 -1.733 0.0121 [1.012]* [1.008]* [0.998]* [0.134] Moore6 -1.923 -1.866 -2.023 -0.0193 [1.054]* [1.050]* [1.040]* [0.139] Silverman9 -1.929 -2.139 -2.584 -0.0952 [1.490] [1.485] [1.470]* [0.197] Ripple7 -1.983 -2.072 -1.899 -0.00146 [0.972]** [0.969]** [0.961]** [0.129] Manion7 -1.986 -2.06 -1.742 0.0277 [1.040]* [1.036]** [1.027]* [0.138] Loken8 -1.987 -1.999 -1.997 -0.00145 [0.975]** [0.972]** [0.963]** [0.129] Tymkovich10 -2.007 -2.015 -1.968 -0.00444 [1.107]* [1.102]* [1.092]* [0.146] Arnold8 -2.016 -2.072 -1.947 0.000588 [0.976]** [0.973]** [0.966]** [0.129] Fisher3 -2.051 -1.876 -1.934 0.0277 [1.372] [1.367] [1.353] [0.181] Evans7 -2.082 -2.14 -1.947 0.00995 [1.017]** [1.013]** [1.004]* [0.135] Lucero10 -2.172 -2.279 -2.083 0.000919 [1.097]** [1.093]** [1.083]* [0.145]

47

Garland12 -2.178 -2.3 -1.63 0.0871 [1.216]* [1.212]* [1.204] [0.161] Murphy8 -2.23 -2.25 -2.107 0.00238 [1.007]** [1.004]** [0.995]** [0.133] Stewart5 -2.257 -2.195 -1.968 0.0474 [1.081]** [1.077]** [1.067]* [0.143] Duncan4 -2.271 -2.491 -2.612 -0.0542 [1.542] [1.536] [1.521]* [0.204] McKee3 -2.272 -2.696 -2.515 -0.054 [1.332]* [1.328]** [1.314]* [0.176] Gilman6 -2.274 -2.271 -2.117 0.0184 [0.976]** [0.973]** [0.963]** [0.129] O_Brien10 -2.281 -2.231 -2.214 0.00113 [1.383]* [1.378] [1.366] [0.183] Rogers6 -2.282 -2.378 -2.385 -0.0201 [1.052]** [1.048]** [1.038]** [0.139] Torruella1 -2.333 -2.403 -2.321 -0.0107 [0.970]** [0.966]** [0.957]** [0.128] Henderson12 -2.343 -2.443 -2.016 0.0648 [1.110]** [1.106]** [1.107]* [0.148] Tatel12 -2.373 -2.573 -2.211 0.0305 [1.162]** [1.157]** [1.152]* [0.154] Hartz10 -2.416 -2.504 -2.355 0.0000778 [1.018]** [1.015]** [1.005]** [0.135] Briscoe10 -2.449 -2.502 -2.438 -0.00251 [1.134]** [1.130]** [1.119]** [0.150] Williams7 -2.456 -2.605 -2.267 0.0177 [1.025]** [1.022]** [1.012]** [0.136] Colloton8 -2.476 -2.486 -2.241 0.0271 [0.964]** [0.960]*** [0.952]** [0.128] Rovner7 -2.504 -2.689 -2.335 0.0126 [0.971]*** [0.968]*** [0.959]** [0.129] Prado5 -2.517 -2.613 -2.451 0.0133 [1.149]** [1.145]** [1.134]** [0.152] Howard1 -2.613 -2.667 -2.557 -0.0081 [1.029]** [1.025]*** [1.014]** [0.136] Smith8 -2.624 -2.712 -2.536 -0.00221 [0.943]*** [0.939]*** [0.931]*** [0.125] Flaum7 -2.662 -2.786 -2.47 0.0209 [0.993]*** [0.990]*** [0.982]** [0.132] Gregory4 -2.666 -2.879 -2.69 -0.00819 [1.210]** [1.206]** [1.194]** [0.160] Bye8 -2.691 -2.826 -2.74 -0.015

48

[0.948]*** [0.945]*** [0.936]*** [0.125] Kanne7 -2.717 -2.793 -2.739 -0.0157 [0.948]*** [0.945]*** [0.936]*** [0.125] Riley8 -2.718 -2.85 -2.64 0.0049 [0.941]*** [0.938]*** [0.931]*** [0.125] Wollman8 -2.724 -2.792 -2.703 -0.00513 [0.965]*** [0.961]*** [0.953]*** [0.128] Callahan9 -2.758 -2.823 -2.916 -0.0185 [1.282]** [1.277]** [1.265]** [0.169] Bea9 -2.796 -2.792 -2.885 -0.00788 [1.282]** [1.277]** [1.265]** [0.169] Melloy8 -2.949 -3.062 -2.848 0.00827 [0.943]*** [0.940]*** [0.932]*** [0.125] Dennis5 -3.364 -3.367 -3.165 0.0295 [1.126]*** [1.123]*** [1.111]*** [0.149]

Year 2005 -0.524 -0.54 -0.0892 [0.169]*** [0.168]*** [0.0225]*** Year 2006 -1.552 -1.521 -0.289 [0.169]*** [0.168]*** [0.0225]*** SubjectMatter2 3.223 0.615 [1.045]*** [0.140]*** SubjectMatter3 1.495 0.379 [0.823]* [0.110]*** SubjectMatter4 3.911 0.734 [1.100]*** [0.147]*** SubjectMatter5 1.395 0.296 [0.380]*** [0.0509]*** SubjectMatter6 3.998 0.674 [0.621]*** [0.0832]*** SubjectMatter7 1.13 0.231 [0.485]** [0.0650]*** SubjectMatter8 3.191 0.49 [1.130]*** [0.151]*** SubjectMatter9 2.984 0.568 [0.613]*** [0.0821]*** SubjectMatter10 0.179 0.0643 [0.883] [0.118] SubjectMatter11 0.479 0.109 [0.433] [0.0580]* SubjectMatter12 8.388 1.214 [0.680]*** [0.0911]*** SubjectMatter13 0.637 0.124

49

[0.407] [0.0545]** SubjectMatter14 1.36 0.29 [0.643]** [0.0861]*** SubjectMatter15 0.216 0.0352 [0.448] [0.0601] SubjectMatter16 2.142 0.317 [0.429]*** [0.0575]*** SubjectMatter17 -0.302 -0.0106 [0.863] [0.116] SubjectMatter18 0.901 0.187 [0.694] [0.0930]** SubjectMatter19 0.413 0.105 [0.508] [0.0681] SubjectMatter20 0.71 0.119 [0.468] [0.0627]* SubjectMatter21 2.406 0.419 [0.715]*** [0.0958]*** Constant 4.4 5.19 3.824 -0.106 [0.780]*** [0.785]*** [0.857]*** [0.115] N 11679 11679 11674 11674 R-sq 0.032 0.039 0.063 0.047 Standard errors in brackets ="* p<0.10 ** p<0.05 *** p<0.01"

50

Appendix B Subject Matter Codings

Administrative Law – 1 Review of Agency Decisionmaking (not in another subject matter category); APA; FCC Rates; FERC Rates; Freedom of Information Act actions; Social Security Entitlement; Medicare Campaign Finance – 2 Campaign Finance and any Election related issue. Capital Punishment – 3 Capital Punishment related actions. Church and State – 4 Establishment Clause; Pledge of Allegiance; Funding for Private Religious Schools; Prayer in School; Ten Commandments; etc. Criminal – 5 Sentencing Guidelines; Prisoners Rights; Drugs/Controlled Substances; Attorney-Client Privilege in Criminal Context; Grand Jury- related; RICO; Search and Seizure (Fourth Amendment); Prison Litigation Reform Act (PLRA); etc. *Excludes Capital Punishment cases. Environment – 6 National Park Service; Clean Air Act; CERCLA; Superfund; National Forest Management Act; Endangered Species Act; EPA; etc. Federal Business Law – 7 Bankruptcy; Antitrust; Federal Banking Laws; Unfair Trade Practices; Federal Debt Collections Act; Fair Debt Collection Practices Act; Truth in Lending Act; Deceptive Advertising under Lanham Act; Magnuson-Moss Warranty Act; etc.

Federalism – 8 State Rights; Federal Preemption; Commerce Clause Power. First Amendment – 9 First Amendment related issues *Excludes Church and State issues Government Actions – 10 Sovereign Immunity; False Claims Act; Government Forfeiture Action. Immigration – 11 Immigration related issues. Intellectual Property – 12 Patents; Copyright; Trademarks; Lanham Act (trademark related actions). Labor – 13

51

Employment issues (excluding employment contractual disputes); ERISA; National Labor Relations Board (NLRB); Occupational Safety and Health Act (OSHA); Fair Labor Standards Act (FLSA); Wrongful Discharge; Labor Management Relations Act (LMRA); and Medical Leave Act (FMLA); Employee Benefits; Worker’s Compensation claims; Retaliatory Discharge claims; etc. Other – 14 Indian Law; Maritime Law; Implicit Private Rights of Actions

Private Law – 15 Contracts; Insurance; Private arbitration; Creditor v. Debtor; Lessor-Lessee; Usury Laws; Franchise v. Franchisor; Employment Contractual Disputes; Corporate Law; Piercing the Corporate Veil. Rights – 16 Race Discrimination; Sex Discrimination; Affirmative Action; Civil Rights; Age Discrimination; Privacy; Abortion; Other Individual Rights, Writs of Habeas Corpus * Excludes Employment, 8th Amendment Capital Punishment Takings and Property – 17 Takings claims; Zoning issues; Property rights. Tax – 18 Internal Revenue Code and other tax-related matters. Torts – 19 Federal Tort Related Act; Medical Malpractice; Products Liability; Wrongful Death; Libel; etc. Courts – 20 Cases in which the court’s analysis is mostly or wholly absorbed with a discussion of civil procedure. Focus is on issues relating to summary judgment, removal, venue, etc.

Securities Law – 21 Securities

52