Tournament Selection Efficiency: an Analysis of the PGA TOUR's

Tournament Selection Eﬃciency: An Analysis of the PGA TOUR’s

FedExCup1

Robert A. Connolly and Richard J. Rendleman, Jr.

October 10, 2012

1Robert A. Connolly is Associate Professor, Kenan-Flagler Business School, University of North Carolina, Chapel Hill. Richard J. Rendleman, Jr. is Visiting Professor, Tuck School of Business at Dartmouth and Professor Emeritus, Kenan-Flagler Business School, University of North Carolina, Chapel Hill. The authors thank the PGA TOUR for providing the data used in connection with this study, Pranab Sen, Nicholas Hall and Dmitry Ryvkin for helpful comments on an earlier version of the paper and Ken Lovell for providing comments on the present version. Please address comments to Robert Connolly (email: robert [email protected]; phone: (919) 962-0053) or to Richard J. Rendleman, Jr. (e-mail: richard [email protected]; phone: (919) 962-3188). Tournament Selection Eﬃciency: An Analysis of the PGA TOUR’s

FedExCup

Abstract

Analytical descriptions of tournament selection eﬃciency properties can be elusive for realistic tournament structures. Combining a Monte Carlo simulation with a statistical model of player skill and random variation in scoring, we estimate the selection eﬃciency of the PGA TOUR’s

FedExCup, a very complex multi-stage golf competition, which distributes $35 million in prize money, including $10 million to the winner. Our assessments of efficiency are based on traditional selection efficiency measures. We also introduce three new measures of efficiency which focus on the ability of a given tournament structure to identify properly the relative skills of all tournament participants and to distribute efficiently all of the tournament’s prize money. We find that reasonable deviations from the present FedExCup structure do not yield large differences in the various measures of efficiency. 1 Introduction

In this study, we analyze the selection efficiency of the PGA TOUR’s FedExCup, a large-scale athletic competition involving a regular season followed by a series of playoff rounds and a “finals” event, where an overall champion is crowned. FedEx- Cup competition began in 2007. Each year, at the completion of the competition, a total of $35 million in prize money is distributed to 150 players, with those in the top three finishing positions earning $10 million, $3 million and $2 million, respectively.1 Research into selection efficiency highlights the importance of the criterion for assessing tournament properties.2 Most who study tournament competition emphasize the probability that the best player will be declared the winner (“predictive power”) as the critical measure of tournament selection efficiency. Largely maintaining the focus of the selection efficiency literature on a single player, Ryvkin and Ortmann (2008) and Ryvkin (2010) introduce two additional selection efficiency measures, the expected skill level of the tournament winner and the expected skill ranking of the winner. They develop the properties of these selection efficiency measures in simulated tournament competition. While we use these efficiency measures in our work, we also develop three new measures of selection efficiency that evaluate the overall efficiency of a tournament structure, not just the the mean skill and mean skill rank of the first-place finisher and the expected finishing position of the most highly-skilled player. Much of the existing literature (e.g., Ryvkin (2010), Ryvkin and Ortmann (2008)) assumes a specific set of distributions (e.g., normal, Pareto, and exponential) to describe competitor skill and random variation in performance. In this paper, we integrate an empirical model of skill and random variation in performance with a detailed tournament simulation to explore the selection efficiency of FedExCup competition. We do not specify the matrix of winning probabilities as in some studies; instead, it is generated naturally from the underlying estimated distributions of competitor skill and random variation and the tournament structure itself. In the next section of the paper we describe the characteristics of FedExCup competition. We develop tournament selection efficiency measures in Section 3. We present an overview of the statistical foundations of our work in Section 4, describe our simulation methods in Section 5, and present results and a discussion of practical implications of our work in Section 6. We summarize our findings in the final section. Appendix A describes the details of our simulation.

1See http://www.pgatour.com/r/stats/info/?02396. 2See Ryvkin and Ortmann (2008) for an excellent recap of existing work along these lines. 2 Characteristics of FedExCup Competition

2.1 Structure of FedExCup Competition

Under current FedExCup rules, similar in structure to NASCAR’s Sprint Cup points system, PGA TOUR members accumulate FedExCup points during the 35-week regular PGA TOUR season.3 As shown in the “Regular Season Points” portion of Table 1, points are awarded in each regular season PGA TOUR-sanctioned event to those who make cuts using a non-linear points distribution schedule, with the greatest number of points given to top finishers relative to those finishing near the bottom. At the end of the regular season, PGA TOUR members who rank 1 - 125 in FedExCup points are eligible to participate in the FedExCup Playoffs, a series of four regular 72-hole stroke play events, beginning in late August. In the Playoffs, points continue to be accumulated, but at a rate equal to five times that of regular season events. The field of FedExCup participants is reduced to 100 after the first round of the Playoffs (The Barclays), reduced again to 70 after the second Playoffs round (the Deutsche Bank Championship), and reduced again to 30 after the third round (the BMW Championship). At the conclusion of the third round, FedExCup points for the final 30 players are reset according to a predetermined schedule, with the FedExCup Finals being conducted in connection with THE TOUR Championship. The player who has accumulated the greatest number of FedExCup points after THE TOUR Championship wins the FedExCup.4

2.2 FedExCup Competition Objectives

It is clear that the objectives of FedExCup competition are multidimensional and complex. From the November 25, 2008 interview with PGA TOUR Commissioner Tim Finchem (PGA TOUR (2008)), it is possible to identify a number of these dimensions.

1. The points system should identify and reward players who have performed exceptionally well throughout the regular season and Playoffs. As such, among those who qualify for the Playoffs, performance during the regular season should have a bearing on ﬁnal FedExCup standings.

3The rules associated with FedExCup competition have been changed twice. Detail about the revisions is presented in Hall and Potts (2010). 4A primer on the structure and point accumulation and reset rules may also be found at http://www.pgatour.com/fedexcup/playoffs-primer/index.html. 2. The Playoffs should build toward a climactic finish, creating a “playoff-type feel,” holding fan interest and generating significant TV revenue throughout the Playoffs. 3. The points system should be structured so that the FedExCup winner is not determined prior to the Finals. (In 2008, Vijay Singh only needed to “show up” at the Finals to win the FedExCup. This led to significant changes in the points structure at the end of the 2008 PGA TOUR season.) 4. The points system should give each participant in the Finals a mathematical chance of winning. We note that Bill Haas, the 2011 FedExCup winner and lowest-seeded player to ever win, was seeded 25th among the 30 players who competed in the Finals.5 5. The points system should be easy to understand. Under the current system, any player among the top five going into the Finals who wins the final event (THE TOUR Championship) also wins the FedExCup. Otherwise, under- standing the system, especially during the heat of competition, can be very difficult.

We do not attempt to quantify the PGA TOUR’s objectives, as summarized above. Instead, we evaluate the optimal selection efficiency of FedExCup competition based on two decision variables. The first is the Playoffs points multiple. Presently, Playoffs points are five times regular season points. This has a potential impact on Commissioner Finchem’s objective points 1 and 2 above. Talking with PGA TOUR officials, we understand that the TOUR reassesses the FedExCup points structure at the end of every season and that this multiple is an important part of the discussion. Reflecting these discussions, we vary the multiple between 1 and 5 in integer increments. Our second decision variable is whether or not to reset accumulated FedExCup points at the end of the third Playoffs round. The present reset system is structured to satisfy objectives 3 and 4 and guarantee that any player among the top five going into the Finals who wins the final event will win the FedExCup (objective 5, at least in part). Although we are able to identify optimal competition structures evaluated in terms of our six efficiency measures, we find that the cost of deviating from optimal structure appears to be small. This finding suggests that the costs of the implicit constraints associated with the objectives listed above may not be high.

5Although confusing, we adopt the convention used throughout sports competition that a “low” seeding or ﬁnishing position is a higher number than a “high” position. For example, in a 10-player competition, the “highest” seed is seeding position 1, while the lowest seed is position 10. 3 Measures of Efﬁciency

In order to measure the selection efficiency of various FedExCup competition structures, we simulate entire seasons of regular PGA TOUR competition followed by four Playoffs rounds. In each simulation trial, we begin with a set of “true” player skills, or expected 18-hole scores. Throughout the regular season and Playoffs competition, each simulated score for a given player equals his expected score, as given by his true skill level, plus a residual random noise component. As the season pro- gresses, and throughout the Playoffs, each player accumulates FedExCup points according to a defined set of rules as described in Section 4.1. We then estimate the efficiency of the FedExCup points system using the criteria described below.

3.1 Ryvkin/Ortmann Selection Efﬁciency Measures

We use the following three measures of tournament selection efﬁciency, examined in detail by Ryvkin and Ortmann (2008) and Ryvkin (2010).

1. The winning (%) rate of the most highly-skilled player, also known as “predictive power.” 2. The mean skill level (expected 18-hole score) of the tournament winner. 3. The mean skill ranking of the tournament winner.

Note that these three criteria focus on a single player, either the most highly- skilled player (predictive power) or the tournament winner. No weight is placed on the finishing positions of other players other than through their effect on the finishing position of the most highly-skilled player or the mean skill ranking or skill level of the tournament winner. We propose three new measures of selection efficiency that capture the ability of a given tournament format to properly classify all tournament participants according to their true skill levels, not just the player who is the most highly skilled, and to properly allocate tournament prize money. Even if the most highly-skilled player in FedExCup competition wins most of the time, the FedExCup would surely lose credibility if the worst players in the competition could frequently finish near the top and win a significant portion of the prize money. Ideally, the FedExCup design would not only identify the single best player in the competition with high probability but would also place players in finishing positions relatively close to their true skill rankings. As such, tournament prize money would generally be the highest for the most highly skilled and lowest for the lowest skilled and, therefore, players would be rewarded in relation to their true skill levels. Our final three measures of selection efficiency take the form of loss functions that reflect these tradeoffs.

3.2 Mean Squared Rank Error (LRE)

Consider a tournament of N players, i = 1,2,...,N, ordered by true skill (or expected score) µi, with µ1 < µ2,... < µN. Let j(i) denote the tournament finishing position of player i. For example, if the most highly-skilled player finishes the tournament in 5th position, j(1) = 5. Then µ j(i) is the inverse transformation of true skill implied by player i’s tournament finishing position, j(i), which henceforth, we refer to as “implied skill.” Finally, let Mj(i) denote the monetary prize to player i if he finishes the tournament in position j(i), with M1 > M2,... > MN. Thus, Mi denotes what player i’s prize would have been if his tournament finishing position had equalled his true skill ranking and Mj(i) denotes player i’s actual prize. Our first loss function, the mean squared ranking error, LRE, measures the extent to which the tournament errs in identifying the true skill rankings of the N tournament participants.

N 1 2 LRE = N ∑ (i − j (i)) i=1 2 = 2σR (1 − ρ), (1) 2 2 where σR = N − 1 /12 is the variance of the ranking positions, i = 1,2,...,N, and ρ is the Spearman rank order correlation of the true skill ranks, i, and tournament finishing positions, j(i). Thus, a tournament scheme that maximizes the Spearman 6 rank, ρ, will minimize the mean squared ranking error, LRE. We note that LRE weights all ranking errors equally, regardless of the actual skill differences of the players who have been miss-ranked. Our final two efficiency measures reflect these differences.

3.3 Mean Squared Skill Error (LSE)

The mean squared skill error is deﬁned as follows:

N 1 2 LSE = N ∑ µi − µ j(i) i=1

6We note that Spearman’s footrule, another measure of ranking error, is equivalent to minimizing the sum of absolute ranking errors rather than squared ranking errors. 2 = 2σµ 1 − βµ . (2)

2 Here, σµ is the variance of true player skill, and βµ is the OLS slope coefﬁcient associated with a regression of true player skill µi on implied player skill, µ j(i), or vice versa. When true skill rankings and tournament ﬁnishing positions are per- fectly aligned, βµ = 1, and LSE = 0. Note that if µ is linear in skill rank, LSE = LRE. The mean squared skill error takes the form of a quadratic loss function, equivalent to the loss function underlying OLS regression and Taguchi’s (2005) loss function used in quality control.

3.4 Mean Money-Weighted Squared Skill Error (LWSE)

2 N Here we weight each value of µi − µ j(i) in (2) by wi = Mi/ ∑ Mi, where Mi i=1 denotes what the tournament prize would have been for the player with skill ranking i if his tournament ﬁnishing position had equalled his true skill ranking. Thus, the mean money-weighted skill error is computed as follows:

N 2 LWSE = ∑ µi − µ j(i) wi. (3) i=1 N Alternatively, we could weight by w j(i) = Mj(i)/ ∑ Mi, where Mj(i) denotes player i=1 i’s actual prize. If the money payout schedule were linear, a given difference between true and implied skill would be penalized the same, regardless of a player’s skill ranking. However, the actual FedExCup money payout schedule is highly convex, with the top three finishers earning $10 million, $3 million and $2 million, respectively, and the players in finishing positions 4-150 splitting the remaining $20 million of the prize pool, also in non-linear fashion.7 This non-linearity implies that the two possible weighting schemes, wi and w j(i), emphasize different types of implied ranking errors. Weighting by wi, rather than w j(i), assumes, implicitly, that tournament organizers are more concerned about under-performance by high-skill players than over-performance by low-skill players. Unlike the previous two loss functions as expressed in Equations (1) and (2), Equation (3) cannot be simpli- fied further without substantial restrictions on the functional form of the weighting function (and the consequent loss of generality).

7Details are provided at http://www.pgatour.com/r/stats/info/?02396. 3.5 Mean Squared Error Deﬂators

Inasmuch as the value of each mean squared error is difficult to interpret without a reference point, we deflate each by the corresponding variance of the variable whose error we are attempting to estimate (i.e., true skill rankings, true player skill and money-weighted true player skill.) Thus, the deflators for the ranking error, skill 2 2 error and weighted skill error are, respectively, DRE = N − 1 /12, DSE = σµ , and N N 2 2 DWSE = ∑ µi wi − ( ∑ µiwi) . i=1 i=1

4 Optimizing FedExCup Competition

4.1 FedExCup Points Distribution and Accumulation

The “Regular Season Points” section of Table 1 shows the distribution of regular season FedExCup points. WGC and Majors are allocated slightly more points than regular PGA TOUR events. “Additional events,” which are events held opposite of some WGC events and majors, are allocated half the points associated with each regular event finishing position. During the regular PGA TOUR season, players accumulate FedExCup points based on the regular season points schedule. At the end of the regular season, the top 125 players in accumulated FedExCup points qualify to participate in the Play- offs. Each participant in the Playoffs carries his accumulated FedExCup points into the Playoffs, but once in the Playoffs, FedExCup points are awarded and accumulated according to the schedule shown in the “Playoffs Points” section of the table. Note that the points distribution schedule for the first three rounds of the Playoffs is exactly five times the points distribution for regular PGA TOUR events conducted prior to the Playoffs. At the end of the first Playoffs round (The Barclays), only the top 100 players in accumulated FedExCup points are eligible to continue to the second round. After the second round (The Deutsche Bank Championship), only the top 70 players are eligible to continue to the third round. After the third round (The BMW Cham- pionship) only the top 30 players qualify for the FedExCup Finals (The TOUR Championship). Immediately prior to the Finals, points are reset for each of the Finals qualifiers according to the schedule shown in the “Finals Reset” column of the “Playoffs Points” section. Points are awarded during the Finals according to the schedule shown in the last column of Table 1. (Note that this is exactly the same distribution of points awarded to finishing positions 1-30 during the first three rounds of the Playoffs.) The points reset was put into place after the second Table 1: FedExCup Points Distribution and Reset Schedule. Points for regular events during regular season decreased by 0.02 points per finishing position past 70. * = includes THE PLAYERS Championship. Regular Season Points Playoffs Points Finishing Regular WGC Additional Finals Position Events Events Majors* Events Rounds 1-3 Reset Finals 1 500 550 600 250.0 2,500 2,500 2,500 2 300 315 330 150.0 1,500 2,250 1,500 3 190 200 210 95.0 1,000 2,000 1,000 4 135 140 150 70.0 750 1,800 750 5 110 115 120 55.0 550 1,600 550 6 100 105 110 50.0 500 1,400 500 7 90 95 100 45.0 450 1,200 450 8 85 89 94 43.0 425 1,000 425 9 80 83 88 40.0 400 800 400 10 75 78 82 37.5 375 600 375 11 70 73 77 35.0 350 480 350 12 65 69 72 32.5 325 460 325 13 60 65 68 30.0 300 440 300 14 57 62 64 28.5 285 420 285 15 56 59 61 28.0 280 400 280 16 55 57 59 27.5 275 380 275 17 54 55 57 27.0 270 360 270 18 53 53 55 26.5 265 340 265 19 52 52 53 26.0 260 320 260 20 51 51 51 25.5 255 310 255 21 50 50 50 25.0 250 300 250 22 49 49 49 24.5 245 290 245 23 48 48 48 24.0 240 280 240 24 47 47 47 23.5 235 270 235 25 46 46 46 23.0 230 260 230 26 45 45 45 22.5 225 250 225 27 44 44 44 22.0 220 240 220 28 43 43 43 21.5 215 230 215 29 42 42 42 21.0 210 220 210 30 41 41 41 20.5 205 210 205 .. .. 66 5 5 5 2.5 25 67 4 4 4 2.0 20 68 3 3 3 1.5 15 69 2 2 2 1.0 10 70 1 1 1 0.5 5 71-75 5 76-85 4 year of FedExCup competition to ensure that no single player could have won the FedExCup prior to the Finals event and also to give each participant in the Finals a mathematical chance of winning the FedExCup.

4.2 What We Evaluate

We limit our analysis of selection efficiency to the 125 players who qualify for the FedExCup Playoffs. Using all six efficiency measures, we evaluate the efficiency of the regular season points distribution system.8 For this same group of 125 players, we then evaluate each of the six efficiency measures at the end of each round of the Playoffs in an attempt to determine if each successive round of the Playoffs improves selection efficiency for this group of 125 players. We also evaluate selection efficiency over all six measures at the end of every Playoffs round, but only for those players who qualify to play in each round. Our concern is whether the points system improves efficiency incrementally at the end of each round for remaining participating players.

5 Statistical Foundations

5.1 Data

Our data, provided by the PGA TOUR, covers the 2003-2010 PGA TOUR seasons. It includes 18-hole scores for every player in every stroke play event sanctioned by the PGA TOUR for years 2003-2010 for a total of 151,954 scores distributed among 1,878 players. We limit the sample to players who recorded 10 or more 18-hole scores. The resulting sample consists of 148,145 observations of 18-hole golf scores for 699 PGA TOUR players over 366 stroke-play events. Most of the omitted players are not representative of typical PGA TOUR players. For example, 711 of the omitted players recorded just one or two 18-hole scores.9

5.2 Player Skill Estimation Model

We employ a variation of the Connolly and Rendleman (2008) model to estimate time-varying player skill and random variation in scoring for a group of profes-

8We take the regular season points distribution schedule as shown in Table 1 as given as well as the number of players who qualify for the Playoffs and each of its stages. 9Generally, these are one-time qualifiers for the U.S. Open, British Open and PGA Championship who, otherwise, would have little opportunity to participate in PGA TOUR sanctioned events. sional golfers representative of PGA TOUR participants during the eight-year period 2003-2010. As in Connolly and Rendleman (2008), we employ the cubic spline methodology of Wang (1998) to estimate skill functions and autocorrelation in residual errors for players with 91 or more scores. We employ a simpler linear representation without autocorrelation, as in Connolly and Rendleman (2012), for players with 10 to 90 scores over the full sample period.10 Simultaneously, we estimate fixed course effects and random round-course effects. We note that the model does not take account of specific information about playing conditions (e.g., adverse weather as in Brown (2011), pin placements, morning or afternoon starting times, etc.) or, in general, the particular conditions that could make scoring for all players more or less difficult, when estimating random round-course effects. Nevertheless, if such conditions combine to produce abnormally high or low scores in a given 18-hole round, the effects of these conditions should be reflected in the estimated round-related random effects.11 Mathematically, the model takes the following form:

s = P f (•) + Cβ + Rb. (4) 0 In (4), s = (s1,...,sm) is an N = 148,145 vector of 18-hole scores subdivided into player groups, i, with ni scores per player i and m = 699. Within each player group, 0 the scores are ordered sequentially, with si = (si 1,...,si ni ) denoting the vector of scores for player i ordered in the chronological sequence gi = 1, 2,...,ni. We refer to gi as the sequence of player i’s “golf times.” The usual error term is part of f (•). P f (•) captures time variation in skill for each of the m golfers in the sample. P is a matrix that identifies a specific player associated with each score. f (•) = 0 ( f1 (•),..., fm (•)) is a vector of m player-specific skill functions described in more detail in the next subsection. In (4), the N ×109 matrix, C, identifies the 109 individual courses on which tournament competition is conducted during our sample period, and β is a vector of estimated fixed course effects. The N × 1,673 matrix R identifies round-course interactions associated with each score, defined as the interaction between a regular 18-hole round of play in a specific tournament and the course on which the round

10We established the 91-score minimum in Connolly-Rendleman (2008) as a compromise between having a sample size sufficiently large to employ Wang’s (1998) cubic spline model (which requires 50 to 100 observations) to estimate player-specific skill functions, while maintaining as many established PGA TOUR players in the sample as possible. For further details see Connolly and Rendleman (2012). 11Interacted random round-course effects, with similar justification, are also estimated in Berry, Reese and Larkey (1999) and Berry (2001). We also estimate random round-course effects in our original 2008 model. However, we believe that the course component of a potential round-course effect is better modeled as fixed than random. is played. The vector of estimated random effects associated with each of the daily round-course interactions is denoted by b.

5.3 Player Skill Functions

Our skill function, as applied to individual player i, takes two forms depending upon the number of sample scores recorded by player i, and may be written as follows:

fi(•) = zi(gi) + θi zi(•) = hi(gi) f or ni ≥ 91 = li(gi) f or 10 ≤ ni ≤ 90. (5)

In (5), hi(gi) is Wang’s (1998) smoothing spline function applied to player i’s golf scores, reduced by estimated fixed course and random round-course effects, over his specific golf times gi = 1, 2,...,ni, for ni ≥ 91. (As noted above, gi counts player i’s golf scores in chronological order.) The vector of potentially autocorrelated random errors associated with player i’s spline fit is denoted θi with 0 2 −1 2 −1 θi = (θi 1,θi 2,....,θi ni ) ∼ N(0,σi Wi ) and σi unknown. In Wang’s model, Wi is a covariance matrix whose form depends on specific assumptions about depen- dencies in the errors, for example first-order autocorrelation for time series, com- pound symmetry for repeated measures, etc. (See Wang (1998, p. 343) for further detail.) li(gi), applied to players for whom 10 ≤ ni ≤ 90, is a simple linear function of player i’s golf times gi = 1, 2,...,ni. We note that for the 372 players for whom we estimate skill using Wang’s smoothing spline model, 158 of the spline fits turn out to be linear. 0 For any given player, i, f = ( f1,.... fn) denotes the vector of the player’s n sequentially ordered golf scores, reduced by estimated fixed course and random 0 round-course effects. If n ≥ 91, h = (h(t1),....h(tn)) denotes a vector of values from the player’s estimated cubic spline function evaluated at points t1,....,tn, which represent golf times g = 1, 2,...,n scaled to the [0, 1] interval. If 10 ≤ n ≤ 90, 0 l = (l (t1),....l (tn)) denotes a vector of values from the player’s estimated linear skill function evaluated at points t1,....,tn. In Wang’s model, as applied here, for each player, one chooses the cubic spline function h(t), the smoothing parameter, λ, and the first-order auto- 1 0 correlation coefficient, φ, embedded in W that minimizes n (f − h) W(f − h) + 1 2 λ R d2h(t)/dt2 dt. “The parameter λ controls the trade-off between goodness- 0 of-fit and the smoothness of the [spline] estimate” (Wang (1998, p. 342)). In (6) below, we break θi into two parts, ϕi + ηi, where ϕi represents the autocorrelated component of θi and ηi is assumed to be white noise.

θi = ϕi + ηi, with ϕi = 0 f or 10 ≤ ni ≤ 90 (6)

Inasmuch as there are likely to be gaps in calendar time between some adja- cent points in a player’s golf time, it is unlikely that random errors around individual player spline fits follow higher-order autoregressive processes (i.e., AR(k), k > 1). Therefore, we assume that for players with at least 91 scores, each θi follows a player-specific AR(1) processes with first-order autocorrelation coefficient φi. Oth- erwise, we assume residual errors are independent. As just described, when estimating player skill functions, we also obtain sets of player-specific residual scoring errors, denoted as θ and η. The θ errors represent potentially autocorrelated differences between a player’s actual 18-hole scores, reduced by estimated fixed course and random round-course effects, and his predicted scores. The η errors represent θ errors adjusted for estimated first- order autocorrelation, and are assumed to be white noise. We refer to a player’s skill estimate at a given point in time as an estimate of his “neutral” score, since estimated fixed course effects and random round-course effects have been removed.

6 Simulation of FedExCup Competition

6.1 Simulation Design

We structure each of 10,000 simulation trials so that the composition of the player pool is similar to what one might observe in a typical PGA TOUR season. As such, we do not include all 699 players from the statistical sample in each trial. Instead, the number of players per trial varies between 415 and 459 and reﬂects the actual number of players in the sample in each year, 2003-2010. We also structure the simulations so that the simulated distributions of player skill (mean neutral score per round), scoring, and player tournament participation rates during the simulated regular season closely approximate those observed in the actual sample. Simulation details are provided in the Appendix.

6.2 Simulation Results

Table 2 summarizes the simulation sample mean value of each of six efficiency measures at the end of the regular PGA TOUR season and at the end of each round Table 2: Efficiency Measures Computed for all 125 FedExCup Playoffs Qualifiers. Stage 0 = end of regular season; Stage 1 = end of first Playoffs round (Barclays); Stage 2 = end of second Playoffs round (Deutsche Bank); Stage 3 = end of third Playoffs round (BMW); Stage 4 NR = end of final Playoffs Round (TOUR Championship) with no points reset; Stage 4 R = end of final Playoffs Round (TOUR Championship) with points reset. “Weight” is the weighting of FedExCup points awarded per tournament finishing position during the Playoffs relative to those awarded during the regular season. Each efficiency measure reflects the mean value computed over 10,000 simulation trials, 1,250 per year for each year 2003-2010. For each efficiency measure except the first, a lower value is better. The best value per stage is shown in bold. Values in each stage that are not significantly inferior statistically in a one-sided test at the 0.05 level relative to the optimal value for the same stage are shown in italics. The optimal stage-4 value with no points reset is always significantly better at the 0.05 level than the optimal value with a points reset except where noted with an asterisk, in which case the optimal value with reset is better.

Panel A: First Place Rate of Best Player Panel D: Mean Squared Rank Error (Deﬂated) Stage Stage Weight 0 1 2 3 4 NR 4 R 0 1 2 3 4 NR 4 R 1 0.396 0.420 0.446 0.468 0.498 0.422 0.872 0.826 0.794 0.777 0.775 0.773 2 0.396 0.438 0.484 0.526 0.563 0.439 0.872 0.805 0.760 0.737 0.734 0.733 3 0.396 0.447 0.501 0.547 0.584 0.444 0.872 0.794 0.744 0.718 0.715 0.714 4 0.396 0.441 0.496 0.547 0.584 0.444 0.872 0.791 0.737 0.711 0.707 0.707 5 0.396 0.416 0.481 0.537 0.578 0.447 0.872 0.791 0.737 0.710 0.706 0.706*

Panel B: Mean Skill of Player in First Place Panel E: Mean Squared Skill Error (Deﬂated) Stage Stage Weight 0 1 2 3 4 NR 4 R 0 1 2 3 4 NR 4 R 1 69.009 68.926 68.853 68.790 68.725 68.825 0.774 0.725 0.693 0.672 0.661 0.669 2 69.009 68.873 68.763 68.670 68.600 68.767 0.774 0.704 0.659 0.630 0.618 0.632 3 69.009 68.847 68.728 68.637 68.571 68.750 0.774 0.696 0.647 0.616 0.603 0.620 4 69.009 68.850 68.734 68.640 68.575 68.750 0.774 0.696 0.647 0.615 0.601 0.617 5 69.009 68.880 68.759 68.658 68.586 68.750 0.774 0.703 0.653 0.619 0.604 0.619

Panel C: Mean Skill Rank of Player in First Place Panel F: Weighted Squared Skill Error (Deflated) Stage Stage Weight 0 1 2 3 4 NR 4 R 0 1 2 3 4 NR 4 R 1 10.625 8.242 6.724 5.864 5.416 5.426 0.656 0.570 0.509 0.465 0.433 0.456 2 10.625 7.196 5.504 4.633 4.253 4.478 0.656 0.528 0.444 0.386 0.351 0.402 3 10.625 6.687 5.033 4.196 3.846 4.108 0.656 0.505 0.416 0.358 0.326 0.383 4 10.625 6.422 4.861 4.052 3.702 3.960 0.656 0.495 0.411 0.353 0.321 0.378 5 10.625 6.298 4.797 4.004 3.631 3.879 0.656 0.499 0.416 0.358 0.322 0.377 of the Playoffs evaluated with respect to the 125 players who qualify for the Play- offs in simulated competition. Each of the six panels of Table 2 represents one of the six selection efficiency measures. For the efficiency measure shown in Panel A, “First Place Rate of Best Player,” higher values are better. In the remaining five panels, lower values indicate greater efficiency. Each efficiency measure is evaluated using a Playoffs points to regular season points weighting ratio that varies from 1 to 5. All efficiency measures shown in Panels D-F are the values computed from Equations 1-3, deflated by the corresponding variance of the variable whose error we are attempting to estimate. Efficiency for Playoff round 4 is evaluated without a points reset (NR) and with a reset (R). The points reset schedule is that given in Ta- ble 1 times the weighting ratio divided by 5. We denote the end of the PGA TOUR regular season as “Stage 0” and the end of Playoffs rounds 1-4 as Stages 1 through 4, respectively. In each panel, the best efficiency value is shown in bold for each stage 1- 4. Except for the few efficiency measures shown in italics, the measures shown in bold are statistically superior in a one-sided test at the 0.05 level relative to all other values shown for the same stage.12 Regardless of the points weighting or efficiency measure, efficiency improves during each stage of competition during Playoffs rounds 1-3 and from round 3 to round 4 when there is no points reset. In Panel C (mean skill rank of player in first place) and Panel D (mean squared rank error), efficiency measures improve from round 3 to round 4 when points are reset after round 3 for all weighting schemes. In Panels E and F, the same is true for weighting scheme 1 only. Thus, from the standpoint of pure mathematical efficiency (i.e., ignoring other factors that might argue for a reset), the evidence is mixed as to whether the 125 players in the Playoffs are ordered more efficiently with a reset after the third round of the Playoffs than after the final round. The optimal Playoffs points weight seems to vary by selection efficiency measure, but in general, optimal weighting falls between 3 and 5. While in the majority of cases (19 or 30) the present Playoffs points weight of 5 may not be optimal, we argue in Section 6.3 that these differences may have little practical significance. We note that with the exception of Panel D (mean squared rank error), efficiency measures as of the end of Stage 4 tend to be more favorable without a reset going into the final round. (Values in the Stage 4 no reset (NR) column tend to be higher than corresponding values in the Stage 4 reset (R) column in Panel A and lower in Panels B-F). Although not shown in the table, with two exceptions

12We estimate statistical significance using 1,000 bootstrap samples drawn from the simulated data generated by 10,000 trials. (Panel C, weight 1 and Panel D, all weights) differences between no-reset and corresponding reset values are favorable to no reset, and differences are statistically significant. The results summarized above in Table 2 evaluate efficiency for the entire 125-player Playoffs field at the end of each Playoffs stage. It is also informative to evaluate efficiency at the end of each Playoffs stage for only those players who actually participate in each stage. Table 3, organized similarly to Table 2, summarizes ratios of mean before and after efficiency measures at the end of each Playoffs stage for stage participants only. Stated differently, the focus in Table 3 is on selection efficiency computed incrementally for each round of the Playoffs for just those players participating in each specific round of the Playoffs. Each value shown in the table reflects the value over 10,000 simulation trials of the ratio of the mean efficiency measure computed at the end of the stage to the mean of the same efficiency measure computed at the beginning of the stage for stage participants only. In all but Panel A, a table entry less than 1 represents an improvement in efficiency from one stage of Playoffs competition to the next. Again, values shown in bold correspond to the best values per stage. Unless shown in italics, all other values in the same stage are are larger than the value shown in bold in more than 95% of bootstrap samples (except in Panel A, where we employ a less-than comparison). As in Table 2, optimal points weightings tend to vary by efficiency measure. Nevertheless, a weighting of 3 to 5 appears to produce the best efficiency measures, but in some cases, the current weight of 5 appears to be optimal. As in Table 2, values shown in Table 3 assume a constant playoffs weighting starting in stage 1. Therefore, it is inappropriate to infer from the Table 3 entries that efficiency can be improved by changing the weights from one Playoffs stage to another. For the first two efficiency measures, the first place rate of the best player (Panel A) and the mean skill level of the player in first place (Panel B), selection efficiency decreases from Stage 3 to Stage 4 with a reset for all weighting schemes. For the other four efficiency measures (Panels C-F), we generally observe more favorable efficiency outcomes when there is no reset compared with a reset (in 18 of 20 cases).

6.3 Practical Signiﬁcance

Despite finding optimal values for the Playoffs points weighting and the decision whether to reset FedExCup points going into the final Playoffs round, we believe that the practical differences are insignificant among efficiency outcomes based on optimal tournament design and those based on non-optimal design over the range Table 3: Ratios of Mean Before and After Efficiency Measures for Stage Participants Only. Stage 1 = end of first Playoffs round (Barclays); Stage 2 = end of second Playoffs round (Deutsche Bank); Stage 3 = end of third Playoffs round (BMW); Stage 4 NR = end of final Playoffs Round (TOUR Championship) with no points reset; Stage 4 R = end of final Playoffs Round (TOUR Championship) with points reset. “Weight” is the weighting of FedExCup points awarded per tournament finishing position during the Playoffs relative to those awarded during the regular season. Each efficiency measure reflects the mean value of the ratio of the efficiency measure computed at the end of the stage to the efficiency measure computed at the beginning of the stage for stage participants only over 10,000 simulation trials, 1,250 per year for each year 2003-2010. For each efficiency measure except the first, a lower value is better. The best value per stage is shown in bold. Unless shown in italics, all other values in the same stage are are larger than the value shown in bold in more than 95% of bootstrap samples (except in Panel A, where we employ a less-than comparison). The optimal stage-4 value with no points reset is lower (higher in Panel A) in more than 95% of bootstrap samples than the optimal value with a points reset except where noted with an asterisk.

Panel A: First Place Rate of Best Player Panel D: Mean Squared Rank Error (Deﬂated) Weight Stage 1 Stage 2 Stage 3 Stage 4 NR Stage 4 R Stage 1 Stage 2 Stage 3 Stage 4 NR Stage 4 R 1 1.061 1.061 1.051 1.062 0.898 0.947 0.944 0.940 0.934 0.893* 2 1.106 1.104 1.087 1.070 0.833 0.923 0.920 0.916 0.919 0.901 3 1.130 1.119 1.093 1.068 0.813 0.911 0.908 0.904 0.915 0.908 4 1.113 1.126 1.102 1.066 0.811 0.907 0.903 0.899 0.913 0.911 5 1.051 1.156 1.116 1.074 0.832 0.907 0.902 0.896 0.912 0.913

Panel B: Mean Skill of Player in First Place Panel E: Mean Squared Skill Error (Deﬂated) Weight Stage 1 Stage 2 Stage 3 Stage 4 NR Stage 4 R Stage 1 Stage 2 Stage 3 Stage 4 NR Stage 4 R 1 0.999 0.999 0.999 0.999 1.001 0.937 0.933 0.928 0.915 0.927 2 0.998 0.998 0.999 0.999 1.001 0.909 0.906 0.897 0.898 0.957 3 0.998 0.998 0.999 0.999 1.002 0.899 0.897 0.888 0.894 0.966 4 0.998 0.998 0.999 0.999 1.002 0.900 0.896 0.884 0.893 0.962 5 0.998 0.998 0.999 0.999 1.001 0.908 0.896 0.882 0.888 0.953

Panel C: Mean Skill Rank of Player in First Place Panel F: Weighted Squared Skill Error (Deflated) Weight Stage 1 Stage 2 Stage 3 Stage 4 NR Stage 4 R Stage 1 Stage 2 Stage 3 Stage 4 NR Stage 4 R 1 0.776 0.809 0.847 0.872 0.877* 0.870 0.874 0.879 0.864 0.897 2 0.677 0.760 0.821 0.877 0.951 0.805 0.821 0.824 0.838 0.967 3 0.629 0.748 0.818 0.882 0.968 0.769 0.804 0.816 0.837 0.999 4 0.604 0.752 0.820 0.885 0.969 0.755 0.809 0.818 0.839 0.995 5 0.593 0.757 0.823 0.879 0.957 0.761 0.814 0.819 0.834 0.976 of possible Playoffs design schemes that we consider. The entries in Table 4, which show the best and worst efficiency outcomes from the corresponding panels of Ta- ble 2 along with efficiency outcomes where the outcomes in each regular season and Playoffs event are determined randomly, provide support for this view.13 (We show results for random outcomes using a Playoffs weight of 3 only. With random tournament outcomes, the Playoffs weight has essentially no impact on any of the efficiency measures.) The outcomes in Panels A-C, based on the efficiency measures of Ryvkin and Ortmann, are the most straightforward to interpret. Panel A shows the rate at which the best player in the competition wins. At the end of the competition, the best and worst outcomes associated with non-random tournament competition fall between 58% and 42%. By contrast, with random tournament outcomes, the best player wins less than 1% of the time. Panel B shows the mean skill level (mean neutral score per round) of the first-place finisher. The best and worst outcomes at the end of non-random competition fall between 68.57 and 68.83 compared with 70.67 when tournament outcomes are determined randomly. Panel C shows the mean skill rank of the player who finishes the competition in first place. Here the best and worst outcomes at the end of non-random tournament competition fall between 3.63 and 5.43 compared with 66.61 when outcomes are determined randomly. Clearly, on the basis of these three measures, (non-random) regular tournament competition dramatically improves each of the three efficiency measures over what might have otherwise been obtained with random tournament outcomes. Whether tournament design is technically optimal appears to be of second-order importance relative to the general structure of the competition itself. Each of the efficiency measures in Panels D-F are the values computed from Equations 1-3, respectively, deflated by the corresponding variance of the variable whose error is being estimated. If we further divide the values in Panel D by 2, we obtain 1 − ρi, j(i) , where ρi, j(i) is the Spearman rank order correlation of the true skill ranks and tournament finishing positions. Best and worst values from non-random tournament competition fall between 0.706 and 0.775, which correspond to Spearman rank correlations of 0.647 and 0.613. By contrast, the 1.98 value for the same efficiency measure under random competition corresponds to a Spearman rank correlation of 0.01, essentially zero. Clearly the tournament competition, whether optimally designed in terms of Playoffs point weights and the reset, significantly improves the rank ordering of participating players. If we divide the values in Panel E by 2, we obtain 1 − βµ , where βµ is

13We maintain exactly the same simulation design as described in the appendix, but instead of basing tournament outcomes on scores, throughout the regular season and Playoffs, outcomes are based on random orderings of tournament participants, both before and after cuts. Table 4: Efficiency Measures with Random Tournament Outcomes. Stage 0 = end of regular season; Stage 1 = end of first Playoffs round (Barclays); Stage 2 = end of second Playoffs round (Deutsche Bank); Stage 3 = end of third Playoffs round (BMW); Stage 4 NR = end of final Playoffs Round (TOUR Championship) with no points reset; Stage 4 R = end of final Playoffs Round (TOUR Championship) with points reset. “Method” is the method by which values are computed, with “Optimal” denoting the optimal value from the corresponding panel of Table 2, “Worst” denoting the worst value from the corresponding panel of Table 2 and “Random” denoting the value when all tournament outcomes are determined randomly using a Playoffs weight of 3. Each efficiency measure reflects the mean value computed over 10,000 simulation trials, 1,250 per year for each year 2003-2010. For each efficiency measure except the first, a lower value is better.

Panel A: First Place Rate of Best Player Panel D: Mean Squared Rank Error (Deﬂated) Stage Stage Weight 0 1 2 3 4 NR 4 R 0 1 2 3 4 NR 4 R Best 0.396 0.447 0.501 0.547 0.584 0.447 0.872 0.791 0.737 0.710 0.706 0.706 Worst 0.396 0.416 0.446 0.468 0.498 0.422 0.872 0.826 0.794 0.777 0.775 0.773 Random 0.006 0.008 0.007 0.006 0.007 0.008 1.975 1.977 1.979 1.980 1.981 1.980

Panel B: Mean Skill of Player in First Place Panel E: Mean Squared Skill Error (Deﬂated) Stage Stage Weight 0 1 2 3 4 NR 4 R 0 1 2 3 4 NR 4 R Best 69.009 68.847 68.728 68.637 68.571 68.750 0.774 0.696 0.647 0.615 0.601 0.617 Worst 69.009 68.926 68.853 68.790 68.725 68.825 0.774 0.725 0.693 0.672 0.661 0.669 Random 70.693 70.682 70.675 70.677 70.674 70.672 1.971 1.971 1.975 1.976 1.976 1.976

Panel C: Mean Skill Rank of Player in First Place Panel F: Money-Weighted Squared Skill Error (Deflated) Stage Stage Weight 0 1 2 3 4 NR 4 R 0 1 2 3 4 NR 4 R Best 10.625 6.298 4.797 4.004 3.631 3.879 0.656 0.495 0.411 0.353 0.321 0.377 Worst 10.625 8.242 6.724 5.864 5.416 5.426 0.656 0.570 0.509 0.465 0.433 0.456 Random 67.563 66.940 66.653 66.658 66.605 66.617 3.366 3.336 3.331 3.331 3.330 3.329 the OLS slope coefficient associated with a regression of true player skill on skill implied by tournament finishing position. Best and worst values in Panel E fall between 0.601 and 0.669, which correspond to slope coefficients of 0.700 and 0.666. With random ordering, the 1.976 value for the same efficiency measure corresponds to a slope of 0.012, essentially zero. As in Panel D, the efficiency values from non- random competition, whether or not they reflect optimal tournament design, are substantially better than that obtained by a random ordering of players. The values for the money-weighted squared skill error, shown in Panel F, are not as readily interpreted. Nevertheless, best and worst values associated with regular competition fall between 0.321 and 0.456 compared with 3.330 with random tournament outcomes, suggesting that achieving exactly optimal tournament design is not critical. Finally, as summarized in the Panel A sections of Tables 2-4, it is clear that the reset substantially reduces efficiency, as measured by the winning rate of the best player. With a points reset, efficiency, as measured at the end of the competition, is approximately the same as efficiency measured at the end of Stage-1 of the Playoffs (see Table 2). Although it is not so easy for the average person following professional golf to appreciate all the dimensions of the other efficiency measures, we suspect that most would have an intuitive feel for the skill levels of the best players in golf.14 If the best players are not winning the FedExCup at a reasonably high rate, and in particular Tiger Woods over the 2003-2010 period of our study, it isn’t unreasonable to expect that the competition could lose credibility among those who follow professional golf. Otherwise, we see little cost in changing Playoffs points weights or the reset to satisfy PGA TOUR objectives that might not be easy to quantify.

6.4 Competitiveness and Excitement

It is clear from PGA TOUR Commissioner’s November 25, 2008 interview that the PGA TOUR strives to create a competitive and exciting playoffs system, building toward a climactic ﬁnish, that will hold fan interest throughout. While aiming to reward players who have performed exceptionally well throughout the regular season, the TOUR does not want the FedExCup winner to be determined prior to the Finals. Thus, the TOUR is seeking to achieve a ﬁne balance between player performance during both the regular season and Playoffs. This balance may not be

14In fact, the TOUR publishes player scoring averages and scoring averages adjusted for field strength throughout the PGA TOUR season. Although neither of these measures corresponds exactly to our mean neutral score, these averages, along with Official World Golf Rankings and other performance measures, make it relatively straightforward to identify the best golfers. easily quantified in terms of the tournament selection efficiency measures we have considered thus far. By construction, the points reset ensures that the ultimate winner of the FedExCup cannot be determined until the completion of the final Playoffs event. In the same simulations that underlie the results summarized in Tables 2 and 3, the winner of the competition would be determined prior to the Finals with probability 0.289 under the present Playoffs points weighting scheme (weight = 5) if there were no reset. With Playoffs points weights of 1, 2, 3 and 4, the probabilities would be 0.091, 0.140, 0.194, and 0.245, respectively. Clearly, if the FedExCup winner were determined prior to the FedExCup Finals, there would be little fan interest in the final event, THE TOUR Championship, which occurs during the middle of the professional and college football seasons. As such, we believe that the PGA TOUR would view these probabilities as being unacceptably high. (If Tiger Woods, or equivalently, a player with his scoring characteristics, is excluded from the simulations, the probabilities for Playoffs weights of 1-5 would be 0.048, 0.063, 0.078, 0.095, and 0.119, respectively.15 More detailed results for simulations that exclude Woods are provided in the online appendix.) Table 5 shows FedExCup winning percentages for 20 of the 125 players in the Playoffs. In Panel A, the players are ordered by their seeding positions, 1-20, at the beginning of the Playoffs. In Panel B, players are ordered by their skill rankings, 1-20, in relation to the field of Playoffs participants. We include an online appendix as a supplement to this paper, which shows results in both panels for players in all positions, 1-125. Without a reset and without giving more weight to FedExCup points earned during the Playoffs relative to the regular season, Table 5 shows that the player who finished the regular season in first place would win the FedExCup 79.6% of the time. (If Woods is excluded from the simulations, this estimate is 78.1%.) More- over, without a reset, and with a Playoffs weight of 1, all players in seeding positions 6-125 have less that a 1% probability of winning. (This is also the case if Woods in not included.) Clearly the winning rate of 79.6% for the top-seeded player and the very low winning rates associated with players seeded beyond position 5 are inconsistent with the TOUR’s objectives. Even with a Playoffs weight of 5, the top-seeded player going into the Playoffs wins the FedExCup almost 50% of the time with no reset, and no player beyond seeding position 10 has over a 1% chance of winning. (Without Woods, 33.8%, and no player past position 14 has more than a 1% chance of winning.) By contrast, regardless of the weight, the winning per-

15Since Woods was such a dominant player during the 2003-2010 period on which our simulations are based, projections based on the inclusion of a player of Woods’ skill may be misleading, at least at the top position, for future periods where there may be no equivalently dominant player. Table 5: FedExCup Winning Percentages by Playoffs Seeding Position and Relative Skill Rank. Based on 10,000 simulation trials (1,250 per year for years 2003-2010). “Weight” is the weighting of FedExCup points awarded per tournament ﬁnishing position during the Playoffs relative to those awarded during the regular season. In Panel B, skill rankings are relative to the 125 players in the Playoffs. If displayed to a precision of 0.1%, all entries below position 91 would equal zero.

Panel A: Position Based on Playoffs Seeding Position Panel B: Position Based on Relative Skill Rankings With Reset Without Reset With Reset Without Reset Position Weight = 1 Weight = 5 Weight = 1 Weight = 5 Weight = 1 Weight = 5 Weight = 1 Weight = 5 1 37.0 31.7 79.6 46.5 42.2 44.8 49.8 57.8 2 16.5 13.8 11.4 14.6 14.9 13.6 23.9 15.9 3 9.8 7.7 3.3 6.6 8.4 8.3 7.7 6.2 4 6.7 5.2 1.6 4.3 5.9 5.5 4.5 3.7 5 5.2 4.0 1.0 3.2 4.4 4.2 2.2 2.6 6 4.0 3.2 0.5 2.1 2.6 2.6 1.9 1.8 7 3.1 2.6 0.5 1.7 2.2 2.2 1.4 1.4 8 2.3 2.2 0.3 1.6 2.2 2.2 1.1 1.4 9 2.0 1.9 0.3 1.3 1.6 1.5 1.0 1.0 10 1.7 1.6 0.2 1.0 1.5 1.3 0.7 0.8 11 1.2 1.3 0.1 0.8 1.2 1.0 0.6 0.7 12 1.2 1.3 0.1 0.7 1.1 1.0 0.4 0.7 13 1.1 1.3 0.1 0.8 0.8 0.9 0.4 0.5 14 0.8 1.1 0.1 0.7 0.8 0.9 0.4 0.5 15 0.6 0.9 0.1 0.5 0.8 0.8 0.4 0.4 16 0.6 0.9 0.0 0.4 0.8 0.8 0.3 0.4 17 0.5 0.8 0.0 0.5 0.7 0.6 0.4 0.4 18 0.6 0.9 0.1 0.6 0.7 0.6 0.3 0.3 19 0.5 0.8 0.1 0.5 0.6 0.6 0.3 0.3 20 0.4 0.7 0.1 0.5 0.5 0.5 0.2 0.3 centage rate of the top-seeded player is substantially lower with a reset, 31.7% to 37.0%, but not so low that his performance during the regular season goes unre- warded, and many more players have a legitimate chance to win. (Without Woods, 21.6% to 30.8%.) From the “Stage 3” column of Table 2, we see that the most-highly skilled player in the competition is the number 1 Playoffs seed 47% to 55% of the time. Therefore, ignoring the remote possibility that this player would not make the Play- offs, the number 1 seed would be the most highly-skilled among the 125 Playoffs participants in 47% to 55% of FedExCup competitions.16 When this player is not the top seed, he is very likely be near the top. Panel B of of Table 5 shows that with a reset, the winning percentage rate of the most highly-skilled player in the Playoffs is greater than that of the number 1 seed for both weighting schemes (weights of 1 and 5). This suggests that even if the most highly-skilled player is not the number 1 seed, he still has a reasonably high chance of winning. By contrast, without a reset and with a Playoffs weight of 1, the most highly-skilled player does not win as often as the number 1 seed, but he does win more often with a Playoffs weight of 5. (These same relationships do not hold when Tiger Woods is excluded from the simulations. See the online appendix.) Table 6 shows percentage rates per FedExCup finishing position (through finishing position eight) for the top-eight-seeded players going into the Finals. (The online appendix shows the same results for all 30 players in the Finals over all 30 possible finishing positions.) Panels A and B indicate that the percentage rates per finishing position are hardly affected by the Playoffs weighting scheme when there is a points reset going into the Finals. With either a weight of 1 or 5, the top 5 seeds all have a reasonable chance to win, ranging from 5.3% to 44.1%. Although not shown, winning rates for players in seeding positions 11-30 are all less than 1% for both weighting schemes. Also, we estimate that a player seeded in position 25 or worse, the same as Bill Haas’ position going into the 2011 Finals, would win the FedExCup only 0.19% of the time under the present system with a reset and Playoffs weight of 5. Thus, Haas’ win was clearly a very rare event. Panels C and D show percentage rates per finishing position without a reset. With a Playoffs weight of 1, there is little remaining uncertainty about the ultimate winner and other top finishers; all are very likely to finish in the positions in which they started. This problem is mitigated somewhat with a Playoffs weight of 5. Nevertheless, the number 1 seed wins almost four out of five times. These same results tend to hold, but to a slightly lesser extent, when Tiger Woods is not included in the FedExCup competition. Although not entirely evident, since not all players and finishing positions

16Tiger Woods misses the Playoffs in 36 of 10,000 simulation trials. Table 6: Finishing Position Percentage Rates by Finals Seeding Position. Based on 10,000 simulation trials (1,250 per year for years 2003-2010). “Playoffs Weight” is the weighting of FedExCup points awarded per tournament ﬁnishing position during the Playoffs relative to those awarded during the regular season.

Panel A: With Reset, Playoffs Weight = 1 Panel C: Without Reset, Playoffs Weight = 1

Finishing Position Finishing Position Seed 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 41.6 37.7 17.9 2.6 0.2 0.0 0.0 0.0 91.4 8.0 0.6 0.0 0.0 0.0 0.0 0.0 2 17.7 20.0 38.6 20.6 3.0 0.1 0.0 0.0 6.2 77.8 14.3 1.7 0.2 0.0 0.0 0.0 3 10.3 7.9 18.5 39.7 20.4 3.1 0.2 0.0 1.5 8.7 68.6 18.0 2.9 0.3 0.0 0.0 4 7.2 5.1 5.1 21.3 40.5 18.3 2.4 0.1 0.6 2.8 8.3 60.5 22.6 4.6 0.6 0.0 5 5.5 3.1 3.1 5.6 20.5 38.1 20.4 3.6 0.2 1.3 3.6 8.7 54.2 25.5 5.6 0.9 6 3.6 2.5 2.1 2.9 4.5 15.2 36.0 27.0 0.1 0.6 1.8 3.9 7.9 49.3 27.7 7.1 7 2.7 1.1 1.9 1.6 2.8 3.6 14.9 37.3 0.1 0.3 0.8 2.2 3.8 7.2 45.0 30.1 8 2.4 1.1 1.0 1.8 1.6 2.2 3.1 13.0 0.0 0.2 0.8 1.5 2.4 3.6 7.9 40.8

Panel B: With Reset, Playoffs Weight = 5 Panel D: Without Reset, Playoffs Weight = 5

Finishing Position Finishing Position Seed 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 44.1 36.5 17.0 2.4 0.1 0.0 0.0 0.0 77.3 18.3 3.8 0.6 0.1 0.0 0.0 0.0 2 15.1 20.3 40.1 21.4 3.0 0.1 0.0 0.0 10.3 47.9 30.1 9.5 2.0 0.2 0.0 0.0 3 9.5 7.2 18.0 40.2 21.7 3.3 0.1 0.0 4.4 9.8 32.9 34.1 14.8 3.4 0.6 0.1 4 7.2 4.6 4.9 20.9 39.9 19.5 2.9 0.1 2.5 5.8 7.2 25.0 34.2 18.8 5.4 0.9 5 5.3 3.2 3.0 5.1 19.3 37.3 22.4 4.1 1.3 3.5 4.2 6.2 21.1 32.5 21.6 7.4 6 3.9 3.1 2.4 2.9 4.7 14.8 34.4 27.7 1.1 2.8 3.0 3.9 5.7 18.3 29.7 22.4 7 2.9 1.3 2.4 1.9 2.8 3.7 14.2 36.1 0.8 2.1 2.3 2.6 3.3 5.8 15.8 27.7 8 2.7 1.1 1.4 1.9 1.5 2.3 3.2 12.6 0.6 1.7 1.9 2.1 2.4 3.4 5.6 14.1 are shown, in panels A and B, except for the first and last seeds, each player’s most likely finishing position is worse than his initial seed. In Panel C, where there is no points reset and points per Playoffs event are the same as those of regular season events, the most likely finishing position for a player in seeding positions 1-11 and 22-30 is his Finals seeding position, and for those in seeding positions 12-19 the most likely finishing position is just one position worse. This suggests that a competition scheme with no reset and no differential weighting of Playoffs and regular season events leaves very little drama and potential for position changes in the Finals. By contrast, even without a reset, but with a Playoffs points weight of 5, players in Finals seeding positions 3-29 are most likely to finish worse than they started. (Note, all these results are provided in the online appendix.) Taken as a whole, we believe that the reset and the weighting of Playoffs points more heavily than those for regular season events plays a critical role in maintaining drama and potential fan interest throughout the Playoffs. From a pure efficiency standpoint, the reset tends to be suboptimal. Nevertheless, it is clear that without a reset, the PGA TOUR could not satisfy its objectives of conducting a meaningful regular season leading to playoffs with a climactic finish that both holds fan interest and has the potential to generate significant TV revenue.

7 Summary and Conclusions

In this paper we introduce several new tournament selection efficiency measures and apply these measures and several existing measures in a systematic evaluation of the selection efficiency of the FedExCup competition run by the PGA TOUR. Our new measures are defined on the full range of tournament outcomes, not just the characteristics of the top finisher or most highly-skilled player. Using simulation, we evaluate the efficiency characteristics of specific alternative tournament structures. Our simulations show that relative to random selection, every variation on the FedExCup tournament selection method that we consider produces significant improvements in selection efficiency. Beyond this result, perhaps the most important regularity is that the points reset impairs tournament efficiency for all efficiency measures except the mean squared ranking error. Despite the tendency for the reset to impair efficiency, an important aim of the reset is to ensure that the competition is in doubt until the last moment. We argue that the reset and weighting of Play- offs points more heavily than those of regular season events are critical elements in creating an exciting and dramatic set of Playoffs events. We acknowledge that our analysis of excitement and drama is much less scientific than our more direct mathematical assessment of tournament selection efficiency and believe that a more formal development of this aspect of competition could be an interesting area for future research. Appendix Simulation Methodology

A FedExCup Regular Season and Playoffs Competi- tion

In simulating the accumulation of FedExCup points during the regular PGA TOUR season and Playoffs, we make the following assumptions.

1. Between 415 and 459 players from our statistical sample participate for a full “regular season” prior to the FedExCup Playoffs in 35 4-round stroke play events.17 The average number of players per event varies by simulation year, reflecting the actual average number of sample players per TOUR event per year, ranging from 125 to 131. At the same time, the average number of actual players per event ranges from 129 to 140 over the same period, with the difference reflecting players who actually participated in PGA TOUR events who did not meet the 10-round minimum to be included in our statistical sample. By excluding these players from individual tournament competition, we assume, implicitly, that they would have had little, if any, impact on individual tournament outcomes and overall FedExCup standings. There is no “picking and choosing” of tournaments nor any qualifying requirements.18 The probability that any single player participates in a regular season event reflects his actual participation frequency on the TOUR in the year of simulated tournament competition being. Further details on player sampling are provided in Appendix Section B. 2. After the first two rounds of each regular season event, the field is cut to the lowest-scoring 70 players who then continue for two more rounds of tournament play.19

1735 regular season events reflects the number of weeks of regular season PGA TOUR competition prior to the FedExCup Playoffs during 2010. In three of the 35 weeks, two PGA TOUR sanctioned events were played simultaneously, but no single player could have participated in the two events at the same time. Therefore, to simplify the simulations, we treat these weeks as if a single event were held. 18A standard PGA TOUR event consists of 144 players. In the early and late parts of the PGA TOUR season, regular events tend to be reduced in size to 144 players due to limited daylight hours. The TOUR also conducts a few “invitationals” with smaller fields, along with a few smaller field select events, including tournaments in the World Golf Championship series. In addition, the Masters, one of the four “majors,” is a small field event, with 97 players participating in 2010. 19Generally, the lowest-scoring 70 players and ties make the cut in regular PGA TOUR events. As such, the number of players who make cuts will tend to exceed 70. However, by using fields of 3. FedExCup points are awarded for each tournament using the “PGA TOUR Regular Season events points distribution” schedule shown in Table 1, as- suming each of the 35 tournaments is a regular PGA TOUR event rather than a “major,” a World Golf Championship event or an “alternate” event held opposite tournaments in the World Golf Championship series. 4. At the end of the 35-event regular season, the Playoffs begin with the top 125 players in FedExCup points participating in The Barclays, the first of four Playoffs events. The Barclays employs a cut after the first two rounds, with the lowest-scoring 70 players advancing to the final two rounds. At the completion of play, FedExCup points are added to those previously accumulated for each of the 125 Playoffs participants according to the schedule of Playoffs points shown in Table 1. 5. After The Barclays, the top 100 players in FedExCup points advance to the Deutsche Bank Championship. The Deutsche Bank employs a cut after the first two rounds, with the lowest-scoring 70 players advancing to the final two rounds. FedExCup points are added to those previously accumulated for each of the remaining 100 Playoffs participants according to the schedule of Playoffs points shown in Table 1. 6. After the Deutsche Bank Championship, the top 70 players in FedExCup points advance to the BMW Championship, where there is no cut. FedExCup points are added to those previously accumulated for each of the remaining 70 Playoffs participants according to the schedule of Playoffs points shown in Table 1. 7. After the BMW Championship, the top 30 players in FedExCup points advance to THE TOUR Championship. 8. When simulating the present TOUR Championship structure, the number of FedExCup points for the 30 participating players is reset according to the reset schedule shown in Table 1. Players are then awarded additional FedExCup points according to their finishing position in THE TOUR Cham- pionship, a four-round stroke play event with no cut, using the points distribution schedule for the Finals as shown in Table 1. The FedExCup winner is the player who has earned the most FedExCup points, not necessarily THE TOUR Championship winner. players from our statistical sample, rather than all competing players, we feel that making a cut at 70 players in simulated competition is reasonable. It is almost certain that no ties will occur with our simulation methodology, but in the unlikely event that a tie does occur, the tie is broken randomly. B Player Selection

Players are selected for regular season tournament participation using the procedure described below. The procedure ensures that in a given year, each player participates in simulated competition at a rate that mimics his participation rate in actual PGA TOUR competition. It also ensures that no player is assigned to participate in the same event more than once in a given simulation trial. We note that this procedure will tend to create tournament ﬁelds that are less competitive than majors and other high-prestige selective events and more competitive than weaker ﬁeld events.

1. A single year from our statistical sample, 2003-2010, is selected, with each year being selected exactly 10,000/8 = 1,250 times. 2. All sample players who actually participated in the selected year become the regular season player pool. 3. We illustrate our procedure for assigning players to tournaments in simulated competition using the following hypothetical small-scale example. Assume there are 8 tournaments in a given year, with an average of 6 players per tournament. Thus, there are 8 × 6 = 48 tournament “slots” total. 12 players participate in the 8 tournaments. Appendix Table 1 summarizes how the players are assigned to each tournament in a hypothetical PGA TOUR season. Assume that we want to simulate 1,250 trials of tournament play in a given year, with 5 events per trial, each of which averages 6 players per event, the same number of players per event as in the hypothetical example. First, we sample the 48 slots in 5 × 1,250 = 6,250 groups of 6 players each, with replacement. Next, we group the sampled slots by player and number the simulated events 1 through 6,250, with the first five events in the sequence corresponding to the first trial, the next five to the second trial, etc. Assume that player “A” appears in the simulation sample 2,000 times. To ensure that he is never assigned to the same tournament more than once, we select the tournament numbers, 1-6,250 at random without replacement a total of 2,000 times and assign the randomly-selected tournament numbers to player “A,” doing the same for each of the remaining players, B-L. These assignments then define the tournament fields for each simulated event. With this procedure, each player will be represented in the simulation sample in approximately the same proportion of total slots as he is represented in the actual data. The average number of players per event will be exactly 6, but there is no guarantee that there will be exactly 6 players in each simulated event. Our actual sampling procedure differs from the example above only by scale. For example, in 2003, 427 sample players participated in 46 sample events and took up 5,991 tournament slots, an average of 5,991/46 ≈ 130 players per event. In simulated regular season play for 2003, we sample the 5,991 slots with replacement in 35 tournament groups, 130 players each. We then group the sampled slots by player and randomly assign simulation tournament numbers 1-35,000, sampled without replacement, for each slot within each player group. The mean number of players per event in our simulation of 2003 competition is 130, with a standard deviation of 8 and range of 96 to 166. In only 19 of 1,250 × 35 = 43,750 simulated events for year 2003 does the tournament size exceed 156, the standard maximum PGA TOUR field size during periods of maximum daylight.

C Simulated 18-Hole Scoring

The following procedure is used to generate 18-hole scores for players who could potentially compete in a given randomly selected PGA TOUR season.

1. A single mean skill level (mean neutral score) for each player is selected at random from the portion of his estimated spline-based skill occurring in the selected PGA TOUR season, 2003-2010. This becomes the player’s mean skill level for the entire season.20 2. For each player k, a single θ residual is selected at random from among the entire distribution of nk θ residuals estimated in connection with his cubic spline-based skill function. 3. For each player k, 166 η residuals are selected randomly with replacement from among the entire distribution of nk η residuals estimated in connection with his cubic spline-based skill function. 4. Using the initial randomly selected θ residual, the vector of 166 randomly- selected η residuals, and player k’s first-order autocorrelation coefficient as estimated in connection with his cubic spline fit, a sequence of 166 estimated θ residuals is computed. 5. The 166 θ residuals are applied to player k’s skill estimate to produce 166 simulated random 18-holes scores. The first 10 scores are not used in simulated competition but, instead, are generated to allow the first-order autocorrelation process to “burn in.” The next 156 are the scores required for a player who might be selected to play in every regular season tournament and who misses no cuts during the regular season (35 × 4 = 140) or during the four

20We assume that the level of effort for each player throughout the entire regular season and Playoffs is the same as that reﬂected, implicitly, in his estimated skill function. rounds of the Playoffs (4 × 4 = 16). We note that it is highly unlikely that all 156 scores would be used for any single player. 6. Starting with the 11th score, scores for each player k are applied in sequence as needed to simulate scoring during the regular season and Playoffs.21

21Suppose player 1 makes the cut in the ﬁrst regular season event and player 2 missed the cut. If both are selected to play in the second regular season event, then simulated scoring in the second event will start with scores 15 and 13 for players 1 and 2, respectively. References

Berry, S. M. (2001), How Ferocious is Tiger?, Chance, 14:3, 51-56. Berry, S. M., C. S. Reese and P. D. Larkey (1999), “Bridging Different Eras in Sports,” Journal of the American Statistical Association 94:3, 661-676. Brown, Jennifer (2011),“Quitters Never Win: The (Adverse) Incentive Effects of Competing with Superstar,” Journal of Political Economy 119:5, 982-1013. Connolly, Robert A. and Richard J. Rendleman, Jr. (2008), “Skill, Luck and Streaky Play on the PGA Tour,” The Journal of The American Statistical Association 103:1, 74-88. Connolly, Robert A. and Richard J. Rendleman, Jr. (2012), “What it Takes to Win on the PGA TOUR (If Your Name is “Tiger” or If It Isn’t),” Interfaces published online May 31. Hall, Nicholas G. and Chris N. Potts (2012), “A Proposal for Redesign of the FedEx Cup Playoff Series on the PGA TOUR,” Interfaces 42(2), 166-179. PGA TOUR, “FedExCup Q&A with Commissioner Finchem,” November 25, 2008. (http://www.pgatour.com/2008/fedexcup/11/25/tuesday.transcript.ﬁnchem/index.html) Ryvkin, Dmitry (2010), “The Selection Efﬁciency of Tournaments,” European Journal of Operational Research 206:3, 667-675. Ryvkin, Dmitry. and Andreas Ortmann (2008), “The Predictive Power of Three Prominent Tournament Formats,” Management Science 54:3, 492-504. Taguchi, G. et. al. (2005), Taguchi’s Quality Engineering Handbook “Part III : Quality Loss Function.” John Wiley & Sons, NJ., 171-98. Wang, Y. (1998), “Smoothing Spline Models with Correlated Random Errors,” The Journal of The American Statistical Association 93:1, 341-348. Appendix Table 1: Small-Scale Tournament Participation Example By Event By Player Slot Player Event Slot Player Event 1 A 1 1 A 1 2 B 1 13 A 3 3 C 1 25 A 5 4 D 1 37 A 7 5 E 1 2 B 1 6 F 1 14 B 3 7 G 2 43 B 8 8 H 2 34 B 6 9 I 2 3 C 1 10 J 2 15 C 3 11 K 2 38 C 7 12 L 3 4 D 1 13 A 3 16 D 3 14 B 3 35 D 6 15 C 3 44 D 8 16 D 3 5 E 1 17 E 3 17 E 3 18 F 3 26 E 5 19 G 4 39 E 7 20 H 4 6 F 1 21 I 4 18 F 3 22 J 4 27 F 5 23 K 4 36 F 6 24 L 4 45 F 8 25 A 5 7 G 2 26 E 5 19 G 4 27 F 5 28 G 5 28 G 5 40 G 7 29 H 5 8 H 2 30 I 5 20 H 4 31 J 6 29 H 5 32 K 6 46 H 8 33 L 6 9 I 2 34 B 6 21 I 4 35 D 6 30 I 5 36 F 6 41 I 7 37 A 7 10 J 2 38 C 7 22 J 4 39 E 7 31 J 6 40 G 7 47 J 8 41 I 7 11 K 2 42 K 7 23 K 4 43 B 8 32 K 6 44 D 8 42 K 7 45 F 8 12 L 3 46 H 8 24 L 4 47 J 8 33 L 6 48 L 8 48 L 8