Dynamic Decision Making and the Market for NFL Draft Picks

Home , Tackle (football move)

Michael Band, Carlos Moya, and Chris Yacu University of Chicago

Nick Kadochnikov Supervisor

Lander Analytics Sponsor

A Capstone Project

Submitted to the University of Chicago in partial fulfillment of the requirements for the degree of

Master of Science in Analytics

Graham School of Continuing Liberal and Professional Studies

March 2017

Abstract

In an environment driven by unique compensation constraints, the National Football

League Draft provides teams with the best opportunity to gain advantages in roster composition. This research explores the market value of draft picks, estimates the expected value of player performance as a function of draft order, and proposes a dynamic strategy to support trade negotiations in real-time. The researchers find significant differences between the market value and the expected performance value of draft picks. This discrepancy is critical to the evaluation of potential trades—an NFL team can maximize its expected return from trades by selling (trading down) at or above market price, and buying (trading up) at a price reflective of the targeted player.

Keywords: dynamic decision making, instance-based learning theory, efficient market hypothesis, competitive bidding, player personnel, professional football, NFL Draft

Executive Summary

For an individual team in the dynamic environment of the NFL Draft, a series of decisions can often determine the fate of the franchise. To effectively navigate trade opportunities in real-time, decision makers must have a support system in place to evaluate options instantaneously. The purpose of this research is to transform analytical insights into an actionable strategy that can adapt to a team’s specific intention when making trade decisions during the draft. Several models are developed to assign a value to each draft pick represented by the trade market, historical performance, and expected surplus. The results of the models power the memory of an application that can support the trade negotiation process and improve decision utility.

The research finds the market no longer abides by the market convention known as the chart. The trade market from 2009-2016 appears to behave more efficiently than the market from 1983-2008 (Massey & Thaler, 2012), though the discount rate for future draft picks remained consistent across periods—135% annually. As the market becomes more efficient, accumulating future picks becomes the more practical arbitrage-seeking strategy.

Through analysis of historical player performance metrics, the researchers build position- specific models to assign a value in salary cap dollars to the single-season performance of

NFL players. The models are trained by veteran performance metrics and compensation data to estimate the value of rookie performance on the unrestricted market. Regression is applied to the results aggregated by draft pick. The model finds performance declines monotonically as a function of draft order, but surplus does not. In fact, surplus increases as a function of draft order until its apex, the 19th overall pick, followed by a gradual decline. Despite decreases in salaries for rookies following the 2011 collective bargaining agreement, top picks are still paid at a disproportional figure relative to rest of the draft based on performance

expectations. However, since all draft picks are expected to yield positive surplus, the draft is the most efficient method to acquire new players since the cost of draft picks is cheaper than the cost of veterans with equivalent performance.

By comparing the performance model results to the trade market, the research finds the slope of the market declines faster than performance. However, the difference in values for the top 10 picks is minimal, which indicates the market effectively values top picks relative to the first overall pick. This validates the hypothesis that the trade market is becoming more efficient. The values of the performance model are used as the basis for a new pick valuation mechanism, the DC Chart.

Performance value varies by player position. The research finds the quarterback, edge defender, and offensive tackle positions are the premium positions in the draft—they are the only position groups expected to yield positive surplus value for the first overall pick.

Conversely, since performance value is significantly lower for the tight end, guard/center, and defensive safety positions, the research warns against using top picks on these positions. The research agrees with Massey & Thaler’s theory on draft-day trades, with an exception—never trade up for a top pick, unless it’s for a quarterback, and the price is reflective of the adjusted performance estimates. A quarterback is expected to outperform the average first pick by

115%, while all other positions yield less than 91%.

The results of the valuation models are used to evaluate trade opportunities in real-time.

The application proposed in this research can evaluate trade offers instantaneously, account for variations in value for the given situation, and identify optimal alternatives. A key feature of the application is an optimization algorithm that searches through all possible trade combinations between two trade partners to find terms that yield the most utility for the team within the limits of the market.

iii

Table of Contents

1. Introduction ...... 1 2. Background ...... 2 2.1. The Market Convention ...... 2 2.2. Player Compensation ...... 4 2.3. The Value of Player ...... 6 2.4. Dynamic Decision Making and Instance-Based Learning Theory...... 8 3. Research Hypotheses ...... 9 4. The Market Value of Draft Picks ...... 11 4.1. Data ...... 11 4.2. Methodology ...... 11 4.3. Results ...... 13 4.4. Discussion ...... 15 5. The Value of Player Performance ...... 16 5.1. Data ...... 17 5.2. Methodology ...... 18 5.2.1. The Starter Index ...... 19 5.2.2. Variable Selection ...... 21 5.2.3. Regression Model ...... 22 5.3. Results ...... 24 5.3.1. The Value of Veteran Performance ...... 24 5.3.2. The Value of Rookie Performance ...... 27 6. The Value of Draft Picks...... 28 6.1. Data ...... 29 6.2. Methodology ...... 29 6.3. Results ...... 31 6.3.1 Surplus Value of Draft Picks ...... 32 6.3.2. Relative Value of Draft Picks ...... 34 6.3.3. Variance by Player Position ...... 36 6.4. Discussion ...... 38 7. Dynamic Decision Making and the NFL Draft ...... 40 7.1. Methodology ...... 40 7.1.1. Recognition ...... 40 7.1.2. Judgement ...... 42 7.1.3. Choice ...... 43 7.1.4. Execution ...... 43 7.1.5. Feedback ...... 44 7.2. Discussion ...... 44

8. Conclusions ...... 44 9. Recommendations ...... 45 Appendixes ...... 46 Appendix A ...... 46 Table 9. The DC Chart ...... 46 Appendix B ...... 47 Figure 10. Performance Value by Position as a Function of Draft Order ...... 47 Appendix C ...... 49 Optimizer Methodology ...... 49 References ...... 51

List of Figures

Figure 1. “The Chart” ...... 3 Figure 2. Rookie Compensation Before & After the 2011 CBA ...... 5 Figure 3. Estimated Trade Market Value vs. "The Chart" ...... 15 Figure 4. Starter Index as a Function of Snaps Played (Quarterbacks) ...... 20 Figure 5. Distribution of Salaries expressed as (%) of the Salary Cap ...... 23 Figure 6. Estimated Performance Value as a Function of Draft Order ...... 32 Figure 7. Performance Value vs. Compensation as a Function of Draft Order ...... 33 Figure 8. Net Surplus Value as a Function of Draft Order ...... 34 Figure 9. The DC Value Chart vs. The Estimated Trade Market ...... 35 Figure 10. Performance Value by Position as a Function of Draft Order ...... 47

List of Tables

Table 1. Market Value of NFL Draft Picks: Regression Results ...... 14 Table 2. Variable Selection of Performance Metrics by Position ...... 22 Table 3. Veteran Performance Model: Regression Results ...... 25 Table 4. Top Veteran Quarterback Single-Season Performances (2005-2014) ...... 26 Table 5. Top Single-Season Performance in First Four Seasons by Position (2005-2014) ..... 28 Table 6. Performance of First Overall Picks (2003-2013) ...... 29 Table 7. Rookie Performance by Position: Regression Results ...... 36 Table 8. Position-Adjusted Performance Value (%) Above the Average Draft Pick ...... 38 Table 9. The DC Chart ...... 46

1. Introduction

The National Football League Draft is an annual event in which teams take turns selecting new players from a pool of eligible college football players. The selection order is determined by the reverse order of each team’s won-lost record from the previous season.

When a team is “on the clock” it can use its draft pick to select a player, or trade the pick for alternative picks in the current year’s draft, future year’s draft, an active player, or a combination of the aforementioned.

The quality of an NFL organization’s draft class is critical to the future success of the team. Accordingly, significant resources are devoted to the scouting evaluation of draft- eligible players on a year-round basis1. Despite this grand investment, team decision-makers continue to approach the draft with limited analytical validation. Through systematic data collection and advanced modeling techniques, learning opportunities in the draft valuation market are rich.

For an NFL team, new knowledge can have a profound impact on the team’s ability to generate consistent returns on its draft capital. Our research explores the value associated with each draft pick as it pertains to the selection and trade strategy from the perspective of individual team decision-makers. Our goal is to convert insights from analysis into actionable results that can be used to facilitate the trade negotiation process for an individual team in real-time.

The purpose of this research is (1) to explore the current value that the market assigns to picks in the National Football League Draft [and the influence of a market convention known

1 The 2016 draft class consisted of 253 players selected over seven rounds. Across all 32 teams, $1,112,384,507 was spent in total contract value of drafted players (Spotrac, 2016).

as the chart]; (2) to develop a valuation algorithm that assigns a value in salary cap dollars to the performance of draft picks based on the market cost of veteran players with equivalent performance; and (3) to establish a dynamic pick valuation strategy to maximize expected return from trade negotiations in real-time. The results of our models can be used not only to validate trades involving draft compensation, but also to facilitate learning by identifying arbitrage opportunities in an inefficient market. Insights from the analysis can assist team decision-makers in maximizing their return on draft capital. We intend to conduct this research through systematic data collection and various modeling decisions built upon methods established by Cade Massey and Richard Thaler (2005 & 2012), Kevin Meers

(2011), Steven Drake (2012), Chase Stuart (2012), and Brian Burke (2016).

2. Background

In their seminal research, The Loser’s Curse, Cade Massey and Richard Thaler (2005) studied the presence of several psychological factors that affect the judgement of team decision-makers during the annual player draft. Massey & Thaler concluded teams overvalue the top picks in the first round relative to all other picks, and the market for draft picks is inconsistent with rational expectations. Through the analysis of historical trades involving draft picks, the researchers concluded that the market value of picks resembles an existing market convention—known as the chart—as a system for valuing draft picks in trades with other teams (Massey & Thaler, 2005).

2.1. The Market Convention

“The Chart” was originally estimated in 1991 by Mike McCoy, then a minority owner of the Dallas Cowboys. McCoy estimated the value of draft picks (relative to the first pick) from a subset of trades that occurred from 1987 to 1990. His goal was merely to characterize past 2

trading behavior rather than to determine what each pick should be worth. When comparing the relative value of draft picks exchanged over the previous four years, McCoy found a [nonlinear] trendline sufficiently fit the data. These relative values were converted into a points system; the first overall pick in the draft was determined to be worth 3,000 points, while the

224th pick was worth 2 points.

Figure 1. “The Chart”

Figure 1 shows the values of each draft pick from the values determined by the chart. On the right axis, we convert the points of the chart into a percentage value relative to the first overall pick to facilitate the comparison of several valuation models. Per the chart, the fifth overall pick (1700 points) is worth just 57% of the value of the first overall pick (3000 points). This steep drop in value creates a premium valuation for top picks compared to all other picks.

Over the years, the chart passed from team-to-team and quickly developed into a market convention that guides most trade negotiations of draft picks. No evidence has been found to

support its validity, yet previous studies suggest the chart continues to influence the market for draft picks.

Alan James Kluegel (2015) studied the conditions that gave rise to the chart and proposes several sociological reasons for its continued use in negotiations. Kluegel noted market conventions are most prevalent when individual actors are uncertain of the preferences of other market actors. Since teams have limited time allotted to either select a player or negotiate a trade, the ability to reference the chart guides the negotiation process. (Kluegel,

2015).

2.2. Player Compensation

Before we consider alternative methods for valuing draft picks, we must first consider several factors that drive the unique economic environment of the National Football League.

Many of the conditions we consider are outlined in the collective bargaining agreement

(CBA) negotiated between league owners and the NFLPA2. Historically, the agreement has had a profound impact on player compensation. In 1993, the CBA established a salary cap which put a limit on the total compensation each team could allocate annually to its players3.

The agreement also granted free agency rights to players once their contract expires, subsequently creating an open market for the services of veteran players. Perhaps no previous

CBA has had a greater effect on the regulations of rookie compensation than the latest agreement, signed in 2011.

The 2011 CBA drastically changed the landscape of the draft. An amendment in the latest

2 The NFL Players Association (NFLPA) is the labor organization representing players, both past and present, to negotiate the compensation system set forth in the collective bargaining agreement (CBA).

3 Each team’s 51 highest valued player contracts count against the salary cap. In 2016, the league- wide cap ceiling was set at $155,270,000. The salary cap is calculated as a share of league revenue, originally negotiated in the 1993 CBA and reformed in the 2006 and 2011 agreements.

agreement, Total Rookie Allocation [Article 7, Section 1e], instituted a league-wide limit on the total amount of compensation each team could allocate to its drafted and undrafted rookie players (CBA, 2011). As part of the new compensation structure, players selected in the draft are bound to their team by four-year contracts at a fixed valuation dependent on draft order4.

Previously, teams would negotiate salaries in an uncapped environment leading to significantly higher salaries and more frequent contract holdouts. The amendment significantly reduced contract values for the top picks in the draft. Figure 2 shows the implied average salary by draft pick, expressed in percentage of the total roster salary cap.

Figure 2. Rookie Compensation Before & After the 2011 CBA

These values are normalized to account for inflation of the annual salary cap. The latest

CBA (blue line) shows a reduction in contract values for picks, most significantly affecting

4 The 2011 collective bargaining agreement (CBA) states that all rookies selected in the draft shall be signed to a fixed contract length of four years. There is a unique stipulation for first round selections; A team has the “unilateral right” to extend the rookie contract from four years to five years after the player’s third season.

the top of the draft order, with minimal effect on picks later in the draft. The expected compensation for the first overall pick decreased significantly from 2010 to 2011, by 56% in guaranteed money alone5.

Consequently, the reduction in rookie compensation increases the relative surplus value of picks, mostly affecting early draft picks. In other words, since rookie players are making less money, rookie players have more opportunity to outperform their contracts, which increases team performance per dollar spent, thus increasing the value of draft picks.

The latest CBA amendment requires new insight into the effects of the current rookie pay structure as it applies to the valuation of picks in the draft. The draft represents an arbitrage opportunity as teams can pay players on a rookie contract less than what they would have to pay a veteran for equivalent performance. Given the cap-constrained environment, this is a critical consideration for roster composition strategy. A major focus of our research is to estimate the value of each draft pick under the current compensation structure as a function of the value of player performance.

2.3. The Value of Player

On a macro-level, the value of a player is dependent on two general conditions: the player’s value relative to other players and the player’s value relative to the salary cap.

Under the constraints of the cap, the goal of any team is to maximize its performance per dollar spent (i.e. surplus value). To generate surplus value, teams can peruse the free agency market and sign players whom they predict will outperform their current value in the future.

This is difficult to achieve due to inflated prices in a competitive bidding environment;

5 2010 first overall pick Sam Bradford signed a six-year, $78 million-dollar contract ($50 million guaranteed); 2011 first overall pick Cam Newton signed a four-year, $22 million-dollar contract ($22 million guaranteed).

Richard Thaler (1988) referred to this effect as the winner’s curse. In football terms, as more teams compete to sign a player, the winning team is likely to overestimate the actual value of a player (Massey & Thaler, 2005, 2012; Thaler, 1988).

More commonly, teams rely on the draft as a means for maximizing roster surplus value.

Brian Burke (2016), in his review of Massey & Thaler’s research, noted that every draft pick provides positive expected surplus value, given the gap between expected performance value and rookie compensation. Teams can leverage surplus value by maximizing the roster’s cumulative performance per dollar spent. However, this rationale assumes that the team is spending all its available cap dollars (Burke, 2016).

In a recent study, Timothy Zimmer (2016) analyzed the effects of salary concentration on team success. He defines salary concentration as the amount a team spends as a percentage of the maximum salary cap available in each year. His findings show a significant positive relationship between salary concentration and team winning percentage. That is, the more a team spends up to the cap limit, the more likely the team is to win more games. Maximizing total performance value is equivalent to maximizing total surplus value, when a team allocates all its available cap dollars.

Often, a team with an early draft pick already has an excess of salary cap dollars, which deflates the short-term value of performance per dollar spent, placing more value on players with higher expected value despite higher compensation (i.e. top draft picks). This means teams will have fluid heuristic valuations of draft picks as each pick is selected, dependent on their current roster composition and the players available at the present pick. We propose a more dynamic approach to account for a team’s intention and its effect on the expected value of a draft pick in real-time.

2.4. Dynamic Decision Making and Instance-Based Learning Theory

Dynamic decision making (DDM) is characterized by multiple, interdependent, and real- time decisions occurring in an environment that changes independently and as a function of a sequence of actions (Gonzalez et al., 2003; Brehmer, 1990; Edwards, 1962).

Cleotilde Gonzalez, Javier Lerch, & Christian Lebiere (2003) proposed a learning theory to support the dynamic decision-making process called instance-based learning theory (IBLT).

IBLT suggests people learn with the accumulation and refinement of instances in a dynamic environment (Gonzalez et al., 2003). Through analysis of previous decisions, decision makers can refine their judgment strategy by learning from past instances. That is, accumulated knowledge can have a profound effect on the utility of future decisions.

The IBLT process is a continuous learning loop of five main steps of DDM: recognition, judgement, choice, execution, and feedback (Gonzalez et al., 2003):

1. Recognition retrieves similar decision situations from memory.

2. Judgement evaluates the utility of the decision using either a heuristic or the

aggregated utility value from past experiences.

3. Choice is determined by the decision maker’s aspiration level; to select best

alternative or to search for more alternatives.

4. Execution is often time-constrained, thus requiring an adaptive strategy.

5. Feedback uses the results from realized decisions to change decision memory.

For an individual team in the NFL draft, a series of instance-based decisions in a dynamic environment can often determine the fate of the franchise. We consider the process for instance-based learning and apply it to the trade decisions for a team in real-time. We leverage the findings from our analysis on the market for draft picks to build a dynamic web- based platform that can support the trade negotiation process.

3. Research Hypotheses

The market for draft picks is inefficient and teams can take advantage of it. That has been the consistent conclusion from nearly all studies on the value of draft picks, and we expect to find similar results in our research. We are concerned, however, that the market will become more efficient as more teams gain awareness of market mispricing. Massey & Thaler (2012) last explored the historical trade market from 1983 to 2008. Their findings suggest that although the chart was the driver for most trades, the market showed trends of mean reversion by comparing the values set by the market over time periods 1983-1992, 1993-2000, and

2001-2008. For example, the chart values the fifth overall pick at 57% the value of the first overall pick. Massey & Thaler found the market from 1983 to 1992 valued the fifth pick at

62%; from 1993-2000, 63%; from 2001-2008, 75%6. The first section of our research explores the market of draft-day trades from 2009-2016, by replicating the methodology set forth by Massey & Thaler. We believe the most recent trading period shows even more signs of efficient behavior. The more efficient the market, the more limitations to arbitrage exist.

Player value can be expressed as a function of his position, play time, and performance measures. We estimate the value [in salary cap dollars] of rookie players in their first four years of experience to identify the relationship between historical performance and draft order. To suggest performance decreases as a function of draft order is hardly a bold statement; rather, of far greater interest is measuring the rate at which performance decays in

6 Massey & Thaler (2012) estimated the market of draft picks through analysis of trades involving (1) picks from the same-year only, and (2) picks from the same-year and future years. In the comparison of time periods (in eight-year intervals), the researchers did not include trades that include future year picks. When analyzing all trades from 1983 to 2008, including trades with future year picks, their findings were relatively consistent; the fifth overall pick was determined to be worth 76% of the first overall pick.

comparison to player compensation, and the trade market. The difference between expected performance and compensation by draft pick estimates the expected surplus value for each pick. The difference between performance and market value identifies specific draft picks that may be undervalued by the market.

Given inconsistencies between the true value of each draft pick and its corresponding market valuation, an individual team can exploit market inefficiencies by negotiating trades that maximize its expected return under the constraints of implied market prices. However, variance unexplained by the market may affect the price of a pick in each specific negotiation situation. We consider covariates to the market value of draft picks, flexible to adjust the implied valuation in real-time. That is, if there are multiple bidders for the first overall pick— because multiple teams believe there is a franchise quarterback available in the draft—the value of the first overall pick should be reflected to consider an adjustment in value for the quarterback position, rather than the average across all positions. We may come to agree with the main conclusion of Massey & Thaler’s original research, with an exception: never trade up for a top draft pick, unless for a quarterback, and the price paid reflects the value adjusted for the position.

Due to the dynamically changing environment of the draft, where choice and preference for all teams changes as each selection is made, an NFL team must systemize trade negotiations in some form to facilitate the process. We propose an actionable strategy guided by the instance-based learning process of dynamic decision making. We leverage the results of our models to power the memory of our dynamic trade value chart, so that teams can evaluate trade offers instantaneously, account for variations in value in the real-time situation, and identify optimal trade offers based on the trade partner’s set of draft picks. Our final objective is to generate a tool that can support the time-constrained decisions of draft-day

trades.

4. The Market Value of Draft Picks

In this section, we replicate the methods of Cade Massey & Richard Thaler (2005 & 2012) to estimate the market value of draft picks as a function of draft order. The original research expressed the value of draft picks in terms of the value of other draft picks (i.e. relative value). The researchers wanted to know, “How much is the first overall pick worth relative to the 10th, 16th, or 32nd pick?” (Massey & Thaler, 2012). Critical to this analysis is the comparability of our results from historical draft-day trades made from 2009-2016 to the data collected and analyzed by Massey & Thaler from 1983-2008.

4.1. Data

We consider all trades involving the exchange of draft picks from 2009-20167. In total,

193 trades are observed: 148 (77%) involving current year picks [only] and 45 (23%) involving at least one future year pick8. When a team trades up in the draft, the average number of picks acquired was 1.2 (SD=.42, max=3). When a team trades down in the draft, the average number of picks acquired was 2.3 (SD=.71, max=6). The most common number of picks exchanged between trade partners was 2-for-1, accounting for 118 trades (61%).

4.2. Methodology

First, we analyze trades involving current year picks [only] under the assumption that the

7 We exclude trades involving NFL players as part of the assets exchanged. We do consider trades involving future year draft picks. This will require calculating the discount value the market applies to draft picks at time n to picks n+1 years in the future.

8 From 2009-2016, the ratio of trades involving current year picks to trades involving future year picks (77:23) is consistent with the trades observed from 1983 to 2008 by Massey and Thaler (76:24).

relative value of draft picks monotonically decays following a non-linear shape. To ensure reliability when comparing results, we replicate the original methodology of Massey & Thaler

(2012) using the shape of a Weibull cumulative distribution function (CDF) to describe the market9.

푈 퐷 Let 푝푖 represent the i-th pick acquired by the team trading up, and 푝푗 represents the j-th pick given by the team trading down. When estimating the market value associated with each draft pick10, we assume the value exchanged in each trade is equal,

푛 푈 푚 퐷 (1) ∑푖=1 푉(푝푖 ) = ∑푗=1 푉(푝푗 )

Where 푉( 푝푖 ) is the value of i-th pick relative the first overall pick. Since we are assuming the value of picks decays monotonically described by a Weibull CDF, the relative value of draft picks follows the shape,

(−휆(푝−1)훽) (2) 푉(푝푖 ) = 푒

Where 휆 and 훽 are unknown parameters. Through substitution of (2) into (1), expressed as the highest pick involved in the trade, we get the following expression.

1 퐷 훽 푈 훽 (3) 푃푈 = [ . 푙표푔( ∑푛 푒−휆(푃 −1) − ∑푛 푒−휆(푃 −1) )]1/훽 + 1 1 휆 푖=1 푖=2

푈 퐷 Where 푃푖 and 푃푖 are the values of the i-th pick received by the team trading up and trading down, respectively. Finally, we take the log of each side of (3) to account for normality assumptions and run a nonlinear least-squares regression to find values of λ and β.

The next step of our analysis considers trades involving future year draft picks. To

9 Weibull cumulative distribution function is defined as a two-parameter function (λ, β) in the exponential family. The shape of the function is flexible enough to represent a complete range of decaying trends.

10 The Efficient Market Hypothesis implies that assets exchanged in a rational market trade at fair value (Investopedia). That is, the total value of assets given up is equal to the total value of assets received. Under this assumption, we can estimate the implicit values of the market through analysis of historical trades of draft picks.

account for these trades, we modify our existing equation (3), which includes an annual discount rate to account for the future value of a draft pick,

퐷 훽 푈 훽 1/훽 1 푒−휆(푃 −1) 푒−휆(푃 −1) (4) 푃푈 = [ . 푙표푔( ∑푛 − ∑푛 )] + 1 1 휆 푖=1 (1+푑푟)푌푒푎푟푠푖 푖=2 (1+푑푟)푌푒푎푟푠푖 where dr is the annual discount rate and Yearsi is the number of years in the future of the i-th pick involved11. Again, we take the log of each side of (4) to account for normality assumptions.

4.3. Results

We compare the output of our models to the results of Massey & Thaler (2012) to assess patterns in the market. The Weibull CDF´s two-parameter shape fits both models exceeding well12; Current Year Trades Only (n=148), R2 =.994; and All Trades (n=193), R2 =.992.

Table 1 shows the results of the parameters λ and β, as well as the value of the 5th, 10th, 16th,

32nd and 64th pick relative to the first overall pick (%).

11 For trades involving current year picks only, parameters dr and Yearsi are set equal to zero.

12 Massey & Thaler (2005, 2012) used the phrase “exceedingly well” to describe their [1983-2008] model fit (R2 =.99). Our analysis yielded strikingly similar results. We believe the stability of the market value of picks that allows for such fit is driven by the prices of a market convention as the baseline for all trade negotiations (i.e. the chart).

Table 1. Market Value of NFL Draft Picks: Regression Results

Model: DCC (1) DCC (2) M&T (1) M&T (2) M&T (3) M&T (4) M&T (5) The Chart Years: 2009-2016 2009-2016 1983-2008 1983-2008 1983-1992 1993-2000 2001-2008 Future picks: No Yes No Yes No No No Parameter Estimates λ 0.075 0.126 0.146 0.0996 0.199 0.184 0.0994 β 0.796 0.699 0.698 0.745 0.642 0.662 0.764 dr 1.351 1.358 Implied Pick Values (relative to the first overall pick) (%) 5th pick 80 72 68 76 62 63 75 57 10th pick 65 56 51 60 44 45 59 43 16th pick 52 43 38 47 32 33 46 33 32nd pick 31 25 20 28 16 17 25 17 64th pick 13 10 7 11 6 6 9 9

N 148 193 313 407 70 108 135 R-Squared 0.99 0.99 0.99 0.99 0.98 0.99 0.99

Table 2 shows the parameter estimates of the Weibull-function and the market value of draft picks from our models (2009-2016 trades) compared to the results of Massey &

Thaler (1983-2008 trades). Also included is the relative pick values associated with the chart.

'dr' represents the discount rate of future year picks as a function of the market (Massey &

Thaler, 2012). Finally, we compared our estimations for the market value of picks against the chart.

Figure 3. Estimated Trade Market Value vs. "The Chart"

4.4. Discussion

If the chart overestimates the value of the top draft picks, and the market convention is inefficient, then it appears the market for picks is becoming more efficient. As we noted earlier, the chart values the fifth overall pick to be worth 57% of the value of the first overall pick. For [all] trades observed from 2009 to 2016 (Model 2), the fifth overall pick was estimated to be worth 72% of the first pick. This is a critical finding to our analysis and infers that while the chart has some effect on the market, overall, the market acts more efficiently than perception would indicate.

When considering only trades involving picks from the same year, it appears the market has become more conservative at estimating the rate at which value of picks decay; however, we believe this effect is due to the small number of trades involving top draft picks in this subset. When considering all trades, the trend does not hold. This phenomenon can be explained by the high valuation of top picks and the necessary inclusion of future year draft

picks; 50% of trades involving a top 10 draft pick includes at least one future year pick as part of the assets exchanged.

Another important insight from these results is both the astonishing magnitude and stability of the annual discount rate. Thaler & Massey (2012) measured the discount rate of future year draft picks to be 136% for all draft-day trades from 1983 to 2008. Remarkably, our results yield a discount rate of 135% for trades from 2009 to 2016. The consistency of these findings aligns with a well-known convention described as the one round per-year rule:

The market for future year picks is defined by a one-round [current-year] devaluation in price.

To account for uncertainty in next year’s draft order—as a rule of thumb—the median pick value is used as the baseline value of the future asset. For example, the 48th overall pick

(median pick of the second round) in the current year’s draft is equal to the 16th overall pick

(median pick of the first round) in next year’s draft.

The high discount rate highlights inefficiency in the market; teams undervalue future picks and overvalue current picks, which can be attributed to organizational pressures for immediate success. Since the market prices of draft picks is becoming more efficient in time, the discount rate and valuation of future year picks—unchanged across time periods—could be part of the optimal arbitrage-seeking strategy in trade negotiations.

5. The Value of Player Performance

In this section, we build multiple regression models to assign a value to the single-season performance of an NFL player. Armed with advanced measures of player performance, combined with historical contract information, our goal is to estimate the market price of statistics. We express the output of our models as the market price of performance in salary cap dollars. Due to constraints on rookie compensation, we define the market as any player

who had the opportunity to negotiate an unrestricted contract (i.e. veteran players). Since our main objective is to estimate the value of players selected in the draft, we leverage the veteran market (exp>5) to estimate the value of rookie players (exp<5) with equivalent performance metrics.

5.1. Data

We analyze player performance metrics and realized compensation representing the 2005-

2014 regular seasons of all individual players who participated in at least one play, specific to each of our 11 position models13. We evaluate the predictive power of several position- specific traditional statistics and advanced statistics compiled from our primary sources;

Football Outsiders, Pro Football Focus, and Pro-Football-Reference. The data collected from these resources expand our ability to more objectively measure the performance of every player.

Massey & Thaler (2005) expressed the value of performance as a function of categorical performance levels; pro bowl, starter, backup, did not play, and injured. Meers (2011), Stuart

(2012), Drake (2014) and Burke (2016) used Pro-Football-Reference’s Approximate Value

(AV) to measure the performance of drafted players14. We take a different approach in our analysis to consider several continuous predictors as inputs in our performance models. The player statistics that comprise our predictors fall into three categories: usage, aggregate, and

13 Player positions are grouped by quarterback (QB), running back (RB), wide receiver (WR), tight end (TE), offensive tackle (OT), offensive guard/center (IOL), edge defender (EDGE), defensive tackle (DT), linebacker (LB), cornerback (CB), defensive safety (DS), 11 positions in total. We exclude kickers, punters and long snappers from analysis.

14 Pro-Football-Reference’s Approximate Value (AV) is an attempt to put a single number on the seasonal value of a player at any position from any year since 1950. The metric is expressed as an integer and is calculated for each player as a share of the overall team total AV, dependent on the success of the team and the player’s contribution.

efficiency metrics. Aggregate statistics (e.g. total touchdowns, total yards, etc.) highly depend on how often a player plays (usage), while efficiency statistics (e.g. pass completion

%, QBR, etc.) measure performance on a per-event basis.

The output of our model is defined by the realized player compensation in salary cap dollars for each season of accrued experience. We collect historical contract information from Spotrac.com and OverTheCap.com spanning the same period as our performance measures. To account for the variance in annual salary cap inflation, we normalize our output from a value in millions of dollars to a percentage of the salary cap limit for the given year.

We define our training set by years of league experience (exp>5) to ensure that all players included are no longer bound to their rookie contract. We exclude veteran players from our training set who were placed on injured reserve, did not record statistics, or have missing contract information. We omit fifth-year players from our training set because, while many of these players are no longer subject to their rookie contracts, first round picks before the 2011

CBA were signed for five years in length, and [after 2011] were signed for four years, subject to a fifth-year team-option.

5.2. Methodology

Since the stability of efficiency metrics is dependent on usage, the insights from advanced metrics are subject to judgment bias15. Thus, the predictive power of efficiency metrics appears weak when considering the full sample of players (both high and low usage). This can be explained in a simplified example of quarterback valuation:

A typical NFL roster is comprised of two quarterbacks; the starter and the backup. The

15 The Law of Small Numbers states that judgement bias occurs when we derive insights from an insufficient sample size as a representative of the population (Tversky & Kahneman, 1971).

starting quarterback often plays every snap of every game leaving minimal opportunity for the backup to record statistics. Let’s assume the ‘starting quarterback’ plays in 16 games, throws

450 passes, completes 300 passes, and throws 25 touchdowns for a completion percentage of

67%. Meanwhile, the ‘backup quarterback’ plays in two games, throws 20 passes, completes

16 of them, and throws one touchdown for a completion percentage of 80%. When we compare the aggregate stats only, the starting quarterback threw 430 more passes and 24 more touchdowns, had far more of an impact on team performance than the backup. When we compare them by our efficiency statistic, completion percentage, the backup quarterback was deceivingly more accurate than the starter. This suggests that while efficiency statistics may be reliable measures of performance for a player with high usage, the data will significantly mislead for players with fewer opportunities16. This is especially true if we’re building a model with several predictors.

To keep efficiency statistics as possible predictors in our model, we create two blended models of starters and backups, weighted by the number of snaps played. Our method allows for comparison within similar groups, dependent on opportunity. For explanatory purposes, we will show examples from the quarterback model.

5.2.1. The Starter Index

To determine if a player is a true starter, true backup, or somewhere in between, we develop an algorithm that we call the Starter Index to create weights between our two models.

The algorithm consists of a 2-mean clustering of players based on the total number of snaps,

16 “The Jim Sorgi-Effect” was described by ESPN Senior Analyst, Brian Burke at the MIT Sloan Sports Conference in March, 2014 when explaining this issue on a panel discussion of the future of analytics in football. Jim Sorgi was the backup quarterback for the Indianapolis Colts from 2004-2009 behind starting quarterback Peyton Manning, throwing just 155 career passes. Sorgi finished with an 89.9 career QB Rating, in-line with NFL starters, however, Sorgi never started a game in his career.

subset by position. The algorithm uses the centroid of each cluster as limits for the label of true starter or true backup. For example, a player with more snaps than the higher-mean centroid is labeled as a true starter (starter index = 1) and a player with less snaps than the lower-mean centroid is labeled as a true backup (starter index = 0). For players who fall between these limits, we use a straight line to connect the two centroids. This makes the

Starter Index value more intuitive; for every increase in snaps, the player is incrementally more of starter than backup and vice versa.

Figure 4. Starter Index as a Function of Snaps Played (Quarterbacks)

Figure 4 shows the Starter Index algorithm for quarterbacks. We define a true starter as any quarterback who plays more than 949 snaps (SI=1), and a true backup as any quarterback who plays less than 290 snaps (SI=0). Any quarterback between those values is in both the starter and backup models, with a median of 620 snaps (SI=.5). The purpose of the starter index is to apply weights to our blended starter and backup models into a single value.

5.2.2. Variable Selection

For all positions, there is no single performance metric that fully captures the value of player performance. A major challenge to the variable selection process is that although each metric may represent different facets of performance, most of the metrics are highly correlated to each other. This high correlation between predictors creates higher variance in our estimates and, depending on the variables included, can affect the coefficient direction of predictors that would be counterintuitive through closer examination. In other words, the model has the make football sense! Since our objective is to represent performance in a single value using a series of predictors, we explore Principal Component Analysis (PCA) as a method to reduce the dimensions of our correlated predictors. Often a small number of principal components suffice to explain most of the variability in the data. If we find the first principal component accounts for enough variance of our data, and shows significant relationship with our response, we consider those series of variables as our final predictors in the model. Table 2 shows a list of the performance metrics included in the final model for each position.

Table 2. Variable Selection of Performance Metrics by Position

5.2.3. Regression Model

We find the distribution of compensation for veteran players shows signs of non- normality. This issue must be addressed to avoid significant bias in our estimates. Figure 3 is a histogram that shows the heavily skewed distribution of veteran compensation.

Figure 5. Distribution of Salaries expressed as (%) of the Salary Cap

We explore several different algorithms, and find that the flexibility of the beta regression helps capture the non-normality that exists amongst the higher tier of veteran players at each position, while a linear regression is more suitable for the backup model where there is less variance and a clearer linear relationship. Specifically, this suggests top players are paid exceedingly more for their services at a rate inconsistent with normal expectations of the full population. To combine the results of our starter and backup models into one score we use the starter index as the determinant for the weights applied to each model expressed in equation (5),

(5) 푃푒푟푓표푟푚푎푛푐푒 푉푎푙푢푒푖 = 푆퐼푖 ∗ 푆푡푎푟푡푒푟 푀표푑푒푙푖 + (1 − 푆퐼푖) ∗ 퐵푎푐푘푢푝 푀표푑푒푙푖 ,

Where 푃푒푟푓표푟푚푎푛푐푒 푉푎푙푢푒푖 is the predicted value in salary cap dollars for the i-th player,

푆퐼푖 is the starter index value corresponding to number of snaps, 푆푡푎푟푡푒푟 푀표푑푒푙푖 is the predicted value from the starter model and 퐵푎푐푘푢푝 푀표푑푒푙푖 is the predicted value from the backup model.

5.3. Results

For each of our 11 position models, we use our training set (veterans) to predict performance of players in their first four seasons of experience (rookies). We compare the results of our veteran performance model to a benchmark model to evaluate goodness-of-fit.

After selecting the best model for each position, we estimate the value of rookie performance in cap dollars based on the unrestricted veteran compensation market.

5.3.1. The Value of Veteran Performance

We evaluate the goodness-of-fit of our models against a commonly used model from previous research; performance as a function of [Pro-Football-Reference’s] Approximate

Value (AV). More specifically, we compare the fit of our blended model to the fit of a linear model using only AV as the predictor for all positions. Table 3 shows a comparison of the fit statistics.

Table 3. Veteran Performance Model: Regression Results

DCC Model Benchmark Model Position N Mean SD MAE RMSE MAPE MAE RMSE MAPE All 3100 3.37% 2.75% 1.58% 2.12% 46.79% 1.70% 2.28% 50.51% Position Models QB 226 6.05% 4.43% 2.57% 3.16% 42.48% 2.89% 3.82% 47.70% RB 231 3.06% 2.35% 1.33% 1.74% 43.59% 1.48% 1.86% 48.38% WR 400 3.61% 2.79% 1.72% 2.32% 47.61% 1.70% 2.40% 47.14% TE 224 2.54% 1.93% 1.13% 1.49% 44.34% 1.09% 1.52% 42.82% OT 218 3.38% 2.50% 1.62% 2.09% 47.78% 1.72% 2.16% 51.01% IOL 414 2.58% 1.72% 1.16% 1.51% 45.04% 1.62% 1.89% 62.74% EDGE 271 4.16% 3.00% 1.94% 2.57% 46.61% 2.00% 2.67% 48.08% DT 150 3.41% 2.20% 1.33% 1.81% 39.00% 1.44% 1.91% 42.42% LB 370 3.00% 2.30% 1.47% 1.92% 48.91% 1.58% 1.97% 52.91% CB 351 3.46% 2.71% 1.80% 2.40% 52.01% 1.70% 2.39% 49.27% DS 245 2.39% 2.01% 1.33% 1.73% 55.43% 1.49% 1.84% 62.22%

Our model outperforms the benchmark. Our model has a mean absolute error (MAE) of

1.58% compared to 1.70% for the benchmark model. We also present Root Mean Square

Error (RMSE) to compare each model’s fit of outliers, and Mean Absolute Percentage Error

(MAPE) as way to compare across positions. Using MAE, our model outperforms the benchmark model at all positions except for wide receiver, tight end, and cornerback.

However, for wide receiver and tight end the results are nearly identical and our model has a lower RMSE which suggests our models does a better job of fitting outliers, as we hypothesized when selecting the beta regressions as a component of our blended model.

Since compensation is agreed upon before performance is measured, we expected to find error between realized compensation and final performance measures. Our goal is to represent a series of performance measures by a distribution relevant to the compensation market for players more so than minimizing the mean absolute error of our models. For this research, we are most concerned with whether our model correctly values player's 25

performance ordinally. In other words, do the results of our models accurately rank players in by order of performance? Table 4 lists the top five single-season performances of veteran quarterbacks (experience>5).

Table 4. Top Veteran Quarterback Single-Season Performances (2005-2014)

Player Season Pred Cap (%) Actual Cap (%) Total TD DYAR QBR AV Tom Brady 2007 16.57% 13.08% 52 2702 87.1 24 Peyton Manning 2013 15.10% 14.23% 56 2446 82.9 19 Aaron Rodgers 2011 14.76% 6.46% 48 2130 87.1 23 Drew Brees 2011 14.10% 10.48% 47 2293 83.0 20 Peyton Manning 2006 13.25% 8.38% 35 2357 87.2 20 *Veteran is defined as league experience greater than five

The main predictors of the quarterback model are included in the table; total touchdowns, total DYAR, QBR, and Approximate Value. We find that Peyton Manning’s 2013 record breaking 56 touchdown season comes 2nd in our model behind Tom Brady’s 2007 season in which Brady finished with a higher QBR rating Approximate Value.

We also consider the predicted values and find our beta regression model is flexible to capture the actual salaries of top quarterbacks. The top five single-season contract values in percentage of the salary cap from 2005-2014: 18.88%, 17.86%, 16.06%, 15.34%, 14.93%. In our model, the top five estimated salaries: 16.57% (Brady, 2007), 15.10% (Manning, 2013),

14.76% (Rodgers, 2011), 14.10% (Brees, 2011), 13.25% (Manning, 2006). From the benchmark AV model: 13.95% (Brady, 2007), 13.44% (Rodgers, 2011), 12.44% (Brady,

2011), 12.44% (Rodgers, 2014), 11.94% (Manning, 2006). When we compare our model results to the benchmark model results of the top estimated quarterback performances, we find the benchmark model significantly underestimates the top tier of quarterbacks. Since top players are paid in a non-normal distribution, the flexibility of beta regression accounts for this effect. This validates our use of beta regression to account for the shape of the response

which limits bias in our estimates.

5.3.2. The Value of Rookie Performance

For the purpose this research, we are most interested in estimating the value of rookie performance as a function of draft order. We leverage the trained veteran models to estimate the value of rookie performance defined as the first four seasons of experience. Table 5 shows the top single-season performances from 2005-2014 in value for each position group.

The results of our model are used in the next section to find the expected value of performance for each draft pick.

Table 5. Top Single-Season Performance in First Four Seasons by Position (2005-2014)

6. The Value of Draft Picks

From the results of the previous section, we can estimate the value of draft picks as a function of expected player performance in salary cap dollars. Our goal is to estimate how quickly performance declines for each pick, which will become the basis of our proposed valuation system, the dynamic chart. We represent the expected value of draft picks in total performance value, surplus value, and relative value. We leverage the results of our model to 28

guide the learning mechanism of our dynamic valuation strategy that can support team decision-makers in real-time trade negotiations.

6.1. Data

We consider historical performance values for each player selected from the 2003-2013 draft classes. We calculate performance as the average of a player’s value in his first four seasons in the league. Since our performance metrics span the 2005-2014 seasons, we consider two seasons of performance values for 2003 & 2013, and three seasons for 2004 &

2012. To properly reflect the range of player performance, if a player did not record a statistic in a season, we assign a value of zero for that player-season in the model. Table 6 shows the average performance value for each of the (11) first overall picks from 2003-2013.

Table 6. Performance of First Overall Picks (2003-2013)

Player Pos Draft Pick Team Year 1 Year 2 Year 3 Year 4 4-Year Avg Carson Palmer QB 2003 1 CIN - - 9.30% 9.01% 9.16% Eli Manning QB 2004 1 NYG* - 8.17% 7.01% 6.27% 7.15% Alex Smith QB 2005 1 SF 1.97% 5.62% 4.52% 0.00% 2.26% Mario Williams EDGE 2006 1 HOU 4.35% 5.77% 5.60% 5.32% 5.46% JaMarcus Russell QB 2007 1 OAK 1.56% 5.14% 4.24% 0.00% 2.12% Jake Long OT 2008 1 MIA 5.77% 6.55% 6.11% 4.12% 5.12% Matthew Stafford QB 2009 1 DET 4.84% 3.32% 10.18% 8.30% 9.24% Sam Bradford QB 2010 1 STL 5.72% 4.49% 6.71% 4.90% 5.81% Cam Newton QB 2011 1 CAR 9.11% 8.08% 8.29% 6.90% 7.60% Andrew Luck QB 2012 1 IND 7.91% 8.48% 9.95% - 9.22% Eric Fisher OT 2013 1 KC 2.46% 3.42% - - 2.94% Pick Average: 6.01% In 2017 Salary Cap Dollars: $10,094,405

6.2. Methodology

We aggregate realized four-year performance values into a single estimate for each draft pick. We are careful to consider players who were drafted but did not play. Missing values, 29

where a player was drafted but did not participate in an NFL snap, are given a zero. We exclude kickers and punters17 from our analysis but do not penalize the aggregate value for picks where kickers and punters were selected. We aggregate the average performance value of a draft pick using the following equation (6),

1 푛푝푖푐푘 1 푦푟푠 (6) 퐴푃푉푝푖푐푘 = ∑ ∑푗=1 푃푒푟푓표푟푚푎푛푐푒푉푎푙푢푒푖푗, 푛푝푖푐푘 푖=1 푦푟푠

Where APVpick is the average performance value for each pick, n is the number of players selected at the pick, and Performance Valueij is the value of performance (in salary cap dollars) for the i-th player in his j-th season. To account for players with fewer than four seasons of measures (2003, 2004, 2012, 2013), yrs represents the number of seasons averaged for draft classes with incomplete data.

Next, we calculate surplus value by subtracting the fixed rookie compensation estimate from our fitted performance value curve. This is expressed in equation (7),

1 푛푝푖푐푘 1 4 (7) 푆푉푝푖푐푘 = ∑ ∑푗=1 푓(푃푒푟푓표푟푚푎푛푐푒푉푎푙푢푒푖푗) − 퐶표푚푝푒푛푠푎푡푖표푛푝푖푐푘,푗, 푛푝푖푐푘 푖=1 4

Where SVpick is the surplus value at position pick, Compensationpick,j is the compensation estimate for pick in season j, and 푓(푃푒푟푓표푟푚푎푛푐푒푉푎푙푢푒푖푗)is our estimated performance curve.

The fitted performance curve, discussed later in this section, is chosen based on model fit (r- squared) and validated through analysis of the residuals.

We noted earlier the chart represents the value of a draft pick relative to other picks. For use in trades, we are interested in finding the value of draft picks relative to the first overall pick (pick 1=1). Additionally, we consider the last pick in the draft to be relatively worthless

17 Only 25 kickers and 22 punters were selected during the 2003-2013 drafts. Of which, only five were selected before the round 4. Due to the specialty of the positions, we do not attempt to model performance for these positions, and thus we exclude these positions from the analysis without penalty to average value of the pick.

in trades since undrafted players are immediately subject to the open market at the draft’s conclusion. A team will rarely trade for a pick at the end of the draft knowing that they can sign a player without giving up a future draft pick (since there are no picks left to trade).

Thus, we assume the value of the last pick in the draft, irrelevant in many ways, is equivalent to an undrafted free agent on the trade market (pick 250=0). We express the transformation of our performance curve to estimate the relative value of draft picks in equation (8),

푓(퐴푃푉푝푖푐푘) 푓(퐴푃푉푝푖푐푘)−푓(퐴푃푉푚푖푛) (8) 푅푉푝푖푐푘 = ∗ , 푓(퐴푃푉1) 푓(퐴푃푉1)−푓(퐴푃푉푚푖푛)

Where RVpick is relative value of pick, f (APVpick ) is the fitted expected performance value of pick, APV1 represents the value of the first pick, and APVmin represents the value of the last pick in the draft. The results of equation (8) are used as the basis for the chart we will use to evaluate trades represented in relative performance value of the first overall pick. The final values of the dynamic chart will span 0 to 1, and multiplied by 100 to represent units between

0 and 100.

6.3. Results

We evaluate several fits to represent the estimated performance value of draft picks: linear regression, loess curve, and an ensemble method. We find that a linear regression with a log transformation fits our data best, R2 =.87. The loess curve adequately fit the data, R2 =.84, however, through examination of the residuals, we find bias in our estimates at the tails of our distribution. The ensemble of the two fits creates noise in the function, so we discard it.

Previous studies that used Approximate Value as the model output also found that a log regression best represented the data, validating our decision (Meers, 2011; Stuart, 2012;

Drake, 2012; Burke, 2016). Figure 6 is the estimated linear fit with a log transformation of the decline in player performance as a function of draft order.

Figure 6. Estimated Performance Value as a Function of Draft Order

The estimated performance value of the first overall pick is worth 5.93% of the salary cap, the second pick 5.27%, third pick 4.88%, fourth pick 4.60%, and fifth pick 4.38%. The last pick in our model, 250th overall, is worth 0.61% of the salary cap. By comparison, the contract value for the first overall pick is roughly 5.03% of the salary cap per season, 0.90% in estimated surplus value. We compare the results of our fit to the rookie compensation curve to find the expected surplus for each draft pick.

6.3.1 Surplus Value of Draft Picks

We interpret surplus value as a player’s level of performance above or below his compensation. Figure 7 compares our fitted value of performance to the rookie compensation distribution explored earlier in this paper.

Figure 7. Performance Value vs. Compensation as a Function of Draft Order

The steep drop in player compensation at the top of the draft declines much faster than our expected performance curve. For any draft pick where performance is greater than the compensation, surplus value is positive. As we can see from the graph, surplus value is positive through all picks in the draft, though we find surplus does not decline monotonically.

In other words, expected surplus value increases through the first round until peaking towards the end of the first round, followed by a gradual decrease in value. These findings align with

Massey & Thaler’s (2012) results, despite changes to the compensation structure. Figure 8 shows the relationship between surplus value and draft pick.

Figure 8. Net Surplus Value as a Function of Draft Order

We find the first overall pick generates less surplus value than all other first round picks: the first pick is worth +0.91% in surplus, while our surplus apex, the 19th pick, is worth

+1.44% in surplus. In 2017 salary cap dollars, $1.53 million in annual surplus excess for first pick, and $2.41 million for the 19th pick. Per Massey & Thaler’s estimates, the apex of surplus in their model was roughly the 35th pick, significantly later than the 19th pick

(Massey & Thaler, 2012). This notable difference can be attributed to the fixed rookie compensation scale; since top picks became cheaper, the ability to generate surplus increases relative to the reduction in price. We can deduce first round picks are more valuable in today’s NFL than in the period examined by Massey & Thaler in their original research.

6.3.2. Relative Value of Draft Picks

Now that we know the expected performance and surplus value for each draft pick, we transform the units of our performance curve to represent values that can be used as the basis of our dynamic chart to evaluate potential trade opportunities. Figure 9 shows the relative 34

value of our performance (1st pick=1 and 250th pick=0) compared to the trade market, estimated in Section 4.

Figure 9. The DC Value Chart vs. The Estimated Trade Market

By comparing the relative values of performance by draft pick (the DC Chart) to the estimated values of the trade market, we see the slope of the market declines more quickly than performance. We express the difference between these curves as the market mispricing of the expected value of draft picks, if maximizing total performance is the primary objective of trades. As the graph shows, the difference in relative value of the top 10 picks of the first round is minimal. That is, the market effectively values picks in the top 10 picks relative to the first overall pick. After the top of the first round, performance declines gradually while the trade market declines steeply. As we noted earlier, a team can maximize utility from trades by trading down at the price of the market (where later round picks are devalued), and trading up when benefit outweighs cost based on the expected return of performance. See

Appendix A for the final values for all draft picks based on the DC Chart.

6.3.3. Variance by Player Position

When a team trades up for a draft pick, their intention is to target a specific player available. Therefore, we consider the position of the targeted player as a covariate to the expected performance value of draft picks. We subset draft pick performance by position and fit a linear regression with a log transformation for each of the 11 models. To avoid problems associated with a smaller sample size, each individual player is a data point18. compared separate performance curves for each of the 11 positions. Table 7 shows the fit statistics for each position using a log regression.

Table 7. Rookie Performance by Position: Regression Results

Expected Performance Value by Draft Pick Position N R-Squared Top Pick 1st Pick* 5th Pick 10th Pick 16th Pick 32nd Pick All 2216 0.870 1st 5.94% 4.38% 3.71% 3.26% 2.59% QB 85 0.456 1st 6.82% 5.26% 4.58% 4.12% 3.45% RB 175 0.333 2nd 4.81% 4.07% 3.51% 3.13% 2.57% WR 286 0.277 2nd 4.83% 4.13% 3.59% 3.23% 2.70% TE 142 0.316 6th 4.05% 4.05% 3.60% 3.19% 2.58% OT 178 0.436 1st 5.15% 3.98% 3.48% 3.14% 2.64% IOL 178 0.236 7th 3.29% 3.29% 3.06% 2.77% 2.33% EDGE 218 0.319 1st 5.38% 4.14% 3.61% 3.25% 2.72% DT 239 0.358 2nd 4.72% 3.98% 3.42% 3.04% 2.48% LB 239 0.296 4th 4.52% 4.33% 3.73% 3.32% 2.72% CB 291 0.286 5th 4.17% 4.17% 3.67% 3.34% 2.84% DS 185 0.377 5th 3.34% 3.34% 2.93% 2.64% 2.23% *For positions without a first pick, we extrapolate our fit with the value of the "top pick" to compare across positions. For example, the highest drafted cornerback (CB) was selected 5th and has an estimated value of 4.17%. so the expected value of the 1st thru 4th pick is also set to 4.17%.

18 In contrast to using the average value for each pick. Note: this increases the dispersion of our sample, naturally causing smaller values of r-squared.

From analyzing the performance of 2003-2013 draft classes by position, we find quarterback, edge defender and offensive tackle are the premium positions of value at the top of the draft. Since the average annual compensation for the first overall pick is estimated to be 5% of the salary cap, roughly $8.4 million in 2017 dollars, QB, EDGE and OT are the only positions that are expected to yield positive surplus value. Conversely, since expected performance is significantly lower for the tight end, guard/center and defensive safety positions, these estimates advise against using top picks on those positions. Based on draft history, the market agrees this notion—there has not been a TE, OG, OC or DS selected with a top four draft pick in over 30 years19. For detailed plots of the position-specific performance curves, see Appendix B.

To reflect the variations in performance value by position, we compare each performance curve to the average for all positions for each draft pick. The result is a matrix of adjustment values for each draft pick and each position. For example, for the first overall pick, a quarterback is expected to yield 6.82% in performance value, while our performance curve

(Figure 6) estimates the average first pick is expected to yield 5.93% in value. We represent the adjustment values as a percentage—115% for quarterbacks selected with the first overall pick over the average. Table 8 shows the adjustment values for the top five picks, as well as the average of eight-pick intervals spanning the first, second and third rounds.

19 Per Pro-Football-Reference’s Draft Finder Tool, only five interior offensive linemen have been selected with a top four draft pick in the modern Super Bowl era, none since 1985. Between safeties and tight ends, none have been taken in the top four. For the purpose of segmenting players by position, we consider the most frequent position played in the first four years of experience. 37

Table 8. Position-Adjusted Performance Value (%) Above the Average Draft Pick

Position: QB RB WR TE OT IOL EDGE DT LB CB DS Top Five Picks Pick 1: 115% 81% 81% 68% 87% 55% 91% 80% 76% 70% 56% Pick 2: 117% 91% 92% 77% 88% 62% 92% 90% 86% 79% 63% Pick 3: 118% 92% 93% 83% 89% 67% 93% 90% 93% 86% 69% Pick 4: 119% 92% 93% 88% 90% 72% 94% 90% 98% 91% 73% Pick 5: 120% 93% 94% 92% 91% 75% 95% 91% 99% 95% 76% Average by Draft Pick Range Pick: 1-8 119% 91% 92% 87% 90% 72% 94% 89% 94% 89% 71% Pick: 9-16 125% 95% 98% 97% 95% 83% 98% 92% 101% 100% 80% Pick: 17-24 128% 97% 101% 98% 98% 86% 101% 94% 103% 105% 83% Pick: 25-32 132% 99% 103% 99% 101% 89% 104% 95% 104% 108% 85% Pick: 33-40 135% 100% 105% 100% 103% 91% 106% 96% 106% 111% 87% Pick: 41-48 138% 101% 107% 101% 106% 93% 108% 97% 107% 115% 89% Pick: 49-56 140% 103% 109% 102% 108% 95% 110% 98% 108% 118% 91% Pick: 57-64 143% 104% 111% 102% 110% 97% 113% 99% 110% 121% 93% Pick: 65-72 146% 105% 113% 103% 113% 99% 115% 100% 111% 124% 95% Pick: 73-80 148% 107% 115% 104% 115% 101% 117% 101% 112% 127% 97% Pick: 81-88 151% 108% 118% 105% 117% 103% 119% 102% 114% 130% 99% Pick: 89-96 154% 110% 120% 105% 119% 106% 121% 103% 115% 133% 101% Pick: 97-104 157% 111% 122% 106% 122% 108% 123% 104% 116% 136% 104%

Table 8 shows the 'Position Value Above Expected' as a function of top five picks and the average (%) in eight-pick intervals. The values included represent rounds 1, 2, and 3.

We can use the distribution of position adjustments to warn of the risk of trading up for a top draft pick to select a non-quarterback. If a team does attempt to trade for a top pick at a different position, they should only do so when the price is less than market value. We also note that by the second round, the adjustments encourage trading up because of the net position adjustment for the positions. This is an important finding and can be connected to our surplus value distribution of draft picks. Since surplus peaks towards the middle of the first round, and declines slowly through the second and third round, these draft picks are perhaps the most ‘valuable’ pieces to acquire through trade.

6.4. Discussion

A major part of our research was the replication of the methodology set forth by Cade

Massey and Richard Thaler (2005), to estimate surplus value by draft pick and to account for 38

the change to the rookie compensation structure. Despite the decrease in salaries for the top draft picks following the 2011 collective bargaining agreement, top picks are still paid at a disproportional figure based on the expected performance. In fact, we find there to be more surplus value obtained through draft picks since compensation either decreased (for top picks) or remained the same (for late picks) after 2011. This finding is insightful from a team building strategy; the draft is the most efficient method to acquire new players since the cost of draft picks is cheaper than the cost of veterans with equivalent performance.

By comparing the performance model results to the trade market, we find the slope of the market declines faster than performance. However, the difference in values for the top 10 picks is minimal, which indicates the market effectively values top picks relative to the first overall pick. This validates the hypothesis that the trade market is becoming more efficient.

We consider the player’s position as a covariate to our performance estimates. We find the quarterback, edge defender, and offensive tackle are the premium positions in the draft—they are the only position groups expected to yield positive surplus value for the first overall pick.

Conversely, since performance value is significantly lower for the tight end, guard/center, and defensive safety positions, the research warns against using top picks on these positions. The research agrees with Massey & Thaler’s primary conclusion, with an exception—never trade up for a top pick, unless it’s for a quarterback, and the price is reflective of the adjusted performance estimates. A quarterback is expected to outperform the average first pick by

115%, while all other positions yield less than 91%.

We leverage the results of the previous sections to support the memory of our proposed trade evaluation application. The result is a dynamic application that can facilitate team decision makers in any negotiation process involving draft picks that can adjust for the situation, search for alternatives, and provide instantaneous recommendations.

7. Dynamic Decision Making and the NFL Draft

The final objective of our research is to transform analytical insights into an actionable strategy that can adapt to a team’s specific intention during trade negotiations. Particularly, we consider whether the team wants to trade up or trade down, whether the team is targeting a specific player, the team’s [and trade partner’s] finite set of draft picks, and the team’s desired aspiration level to get the deal done under constraints of the clock.

7.1. Methodology

We recall the learning process for dynamic decisions follows a continuous learning loop: recognition, judgement, choice, execution, and feedback (Gonzalez et al., 2003). We describe the main steps of dynamic decision-making applied to the NFL draft and propose instance- based learning mechanisms to maximize the utility from time-constrained decisions.

7.1.1. Recognition

Time is of the essence in NFL team war rooms. For a team to effectively navigate trade opportunities in real-time, decision makers must have a support system in place to evaluate options instantaneously. The better equipped for the dynamically changing landscape of the draft board, the more leverage the team will have in negotiations. An instance-based learning mechanism called recognition-based retrieval compares the current situation to similar situations saved in memory (Gonzalez et al., 2003). To facilitate the retrieval process, we propose an application that can evaluate trades between two teams, search for alternatives, and provide immediate recommendations. We leverage the results of our estimated trade market value, performance value, surplus value, relative value, and position value of draft picks as the memory that guides trade evaluations.

The first step before any trade negotiation begins is to determine the team’s desired intention: Are we interested in trading up for a pick? Or, is our pick coming up and we are considering trading down? In either instance, one team is targeting a specific player, while the other accumulates future assets, uncertain at the time of the trade. The team targeting the highest pick in the trade is trade up actor, while its counterpart is the trade down actor. This is important to consider later in the DDM process (choice) when searching for trade alternatives—the highest pick in the trade scenario is fixed, while all other picks are potentially exchanged assets.

The recognition process begins with an environmental cue; either the team receives a phone call from another team with a trade offer, or the team makes the initial offer. When a team receives an offer, decision makers must immediately analyze the return on value based on their predetermined heuristic. We set up the application to account for the finite set of current and future year draft picks from any two teams. To evaluate an offer, the user selects the picks considered for the trade, while simultaneously updating the values exchanged. From the memory of our valuation models, the application provides instant feedback on the return from the trade in regards to the market value, performance value, and surplus value received from the deal.

Conversely, when a team initiates a trade, it is important to consider trades within the realm of consideration for their trade partner, while also maximizing utility. For this purpose, and the purpose of searching for counter offers, we develop an optimization feature for the application that considers all trade combinations between two teams, and evaluates the expected return based on the aggregated values of our historically-driven estimates.

7.1.2. Judgement

We express the value of draft picks by different measures throughout this paper: market value, performance value, surplus value, and positional value. Market value represents the relative value of picks to the first pick as a function of the trade market from 2009-

2016. Since we do not know the valuation mechanism of our trade partner, we assume their valuation behavior resembles the market. Thus, any trade we evaluate where the market return for the trade partner is greater than 100%, we assume they will consider the deal to be fair value. By comparing the market value return to performance value return (the DC Chart), a team may find trades where their market return is less than 100%, while performance return

[or surplus return] is greater than 100%. It is critical for the decision maker to determine which valuation mechanism to maximize for the given situation. The application we propose considers these variations in aspiration, and provides recommendations based on the priority of the team. A team can evaluate any trade by the value returned by the different measures. We can even consider the chart, if we find trade partners still abiding by its mispricing.

To convert our valuation system from a heuristic-based tool to an instance-based tool, we can adjust for the implied valuation of a draft pick to account for the specific player of interest when the pick is on the clock. For example, if a team owns the first overall pick, and a franchise quarterback is deemed worthy of the pick by scouts and executives, both the team and several competing teams will value the first pick higher than usual. As we discussed in

Section 6, quarterbacks selected with the first pick are expected to perform 115% better than the average first pick, while all other positions performance less than 91% of the average. A dynamic element of the application can account for the variance in the value of a pick based

on the positions of the available players. For this strategy, position adjustments will be most prevalent when a team trades up, because their intention is to target a specific player.

7.1.3. Choice

Once valuation constraints are established to account for the given draft scenario, we can evaluate trade opportunities and determine the best course of action. The next step to the

DDM process is to evaluate possible alternatives. For trade offers, this includes all possible counter offers based on both team’s set of draft picks. A key feature of our trade evaluation tool is an optimization algorithm that searches through all possible trade combinations between two trade partners to find trades that yield the most utility for the team. To stay within the bounds of realistic trade combinations, users of the application can adjust constraints in the model to find trades within the limitations of the market. This includes adjusting for the minimum and maximum return on market value (i.e. how big of a win/loss willing to take), setting a limit on the number of picks involved in the trade, as well as adjusting the value of future year picks based on a flexible discount rate.

The output of the optimization feature will generate the best alternative trades between two partners within the user-determined constraints. This can be used by a team to counter a trade offer or to formulate an initial offer, based on preference and the dynamically changing environment. See Appendix C for detailed methodology of the optimization algorithm.

7.1.4. Execution

Maximizing utility in any trade negotiation is restricted due to the time-constrained environment of the draft. Teams do not have the time to continuously negotiate trade terms; decisions to accept, reject or counter must occur instantaneously. Instance-based learning theory describes this urgency as the necessity level of a decision. In other words, necessity

level determines the number of alternatives that can be evaluated before a decision is executed

(Gonzalez et al., 2003). The trade evaluation tool we propose in this paper accounts for this decision urgency by finding optimal trade combinations based on the dynamic valuation mechanisms proposed in this paper to support the final human judgement.

7.1.5. Feedback

Decision makers can improve decision performance by refining the training set for the valuation models we propose in this research. With each passing draft class, we will have a new season of performance metrics, an additional draft class to evaluate, and a set of trades executed during the draft weekend. For this research, the feedback we refer to in our dynamic application is guided by the analysis from the 2003-2013 draft classes. We recommend updating the model next season to account for the results of the 2014 draft class, and trades executed during the 2017 draft.

7.2. Discussion

New knowledge can have a profound effect on decision utility in a dynamically changing environment. Draft-day decisions are made with significant uncertainty, under the constraints of the clock, where choice and preference changes with every pick. The methods described in this section are aimed to facilitate the decision-making process by converting insights into action in the form a user-friendly application.

8. Conclusions

Analytical insights from the study of the NFL Draft do not provide value until acted upon by a decision maker. In both academia and journalistic settings, several attempts aimed to re- create the chart to evaluate draft-day trades. Limitations to its use, however, are driven by the dynamic changing environment of the draft. That is, the value of a draft pick is not static and 44

therefore the value mechanism used in real-time should not be static. We explained the process of building upon previous methodologies to convert similar insights into an actionable strategy through the development of a dynamic application.

We are aware of the limitations to our valuation systems. We urge decision makers to interpret the values as estimates, and to use the tool only as a guide. While the application can adjust the historical performance of a specific position, it does not yet account for the live draft board and the value of players available, and thus its true dynamic feature does not represent the team’s true aspiration. Future iterations of the application could consider overlaying player grades onto the value of draft picks in salary cap dollars.

9. Recommendations

Discrepancies between the trade market value and the estimated performance value of draft picks suggest an NFL team can maximize its expected return from trades by selling (trading down) at or above market price, and buying (trading up) at a price reflective of the targeted player. To effectively gain advantages from an inefficient market in a highly dynamic and uncertain environment, decision makers can improve their decision utility through systematic evaluation of options.

Appendixes

Appendix A Table 9. The DC Chart

Round 1 Value Round 2 Value Round 3 Value Round 4 Value Round 5 Value Round 6 Value Round 7 Value 1 100.0 33 36.6 65 24.3 101 16.3 140 10.4 176 6.2 217 2.4 2 87.4 34 36.0 66 24.0 102 16.1 141 10.2 177 6.1 218 2.3 3 80.1 35 35.5 67 23.7 103 15.9 142 10.1 178 6.0 219 2.3 4 74.9 36 35.0 68 23.5 104 15.8 143 10.0 179 5.9 220 2.2 5 70.8 37 34.5 69 23.2 105 15.6 144 9.9 180 5.8 221 2.1 6 67.5 38 34.0 70 22.9 106 15.4 145 9.7 181 5.7 222 2.0 7 64.7 39 33.6 71 22.7 107 15.2 146 9.6 182 5.6 223 1.9 8 62.3 40 33.1 72 22.4 108 15.1 147 9.5 183 5.5 224 1.8 9 60.1 41 32.6 73 22.2 109 14.9 148 9.4 184 5.4 225 1.8 10 58.2 42 32.2 74 21.9 110 14.7 149 9.2 185 5.3 226 1.7 11 56.5 43 31.8 75 21.7 111 14.6 150 9.1 186 5.2 227 1.6 12 54.9 44 31.4 76 21.5 112 14.4 151 9.0 187 5.1 228 1.5 13 53.5 45 31.0 77 21.2 113 14.3 152 8.9 188 5.0 229 1.4 14 52.1 46 30.6 78 21.0 114 14.1 153 8.8 189 4.9 230 1.4 15 50.9 47 30.2 79 20.7 115 13.9 154 8.6 190 4.8 231 1.3 16 49.7 48 29.8 80 20.5 116 13.8 155 8.5 191 4.7 232 1.2 17 48.6 49 29.4 81 20.3 117 13.6 156 8.4 192 4.6 233 1.1 18 47.6 50 29.0 82 20.1 118 13.5 157 8.3 193 4.5 234 1.1 19 46.6 51 28.7 83 19.9 119 13.3 158 8.2 194 4.5 235 1.0 20 45.7 52 28.3 84 19.6 120 13.2 159 8.1 195 4.4 236 0.9 21 44.8 53 28.0 85 19.4 121 13.0 160 7.9 196 4.3 237 0.8 22 43.9 54 27.7 86 19.2 122 12.9 161 7.8 197 4.2 238 0.7 23 43.1 55 27.3 87 19.0 123 12.7 162 7.7 198 4.1 239 0.7 24 42.4 56 27.0 88 18.8 124 12.6 163 7.6 199 4.0 240 0.6 25 41.6 57 26.7 89 18.6 125 12.4 164 7.5 200 3.9 241 0.5 26 40.9 58 26.4 90 18.4 126 12.3 165 7.4 201 3.8 242 0.4 27 40.2 59 26.0 91 18.2 127 12.1 166 7.3 202 3.7 243 0.4 28 39.6 60 25.7 92 18.0 128 12.0 167 7.2 203 3.6 244 0.3 29 38.9 61 25.4 93 17.8 129 11.9 168 7.1 204 3.5 245 0.2 30 38.3 62 25.1 94 17.6 130 11.7 169 7.0 205 3.5 246 0.1 31 37.7 63 24.9 95 17.4 131 11.6 170 6.8 206 3.4 247 0.1 32 37.1 64 24.6 96 17.2 132 11.4 171 6.7 207 3.3 248 0.0 97* 17.0 133* 11.3 172* 6.6 208* 3.2 249* 0.0 98* 16.8 134* 11.2 173* 6.5 209* 3.1 250* 0.0 99* 16.7 135* 11.0 174* 6.4 210* 3.0 251* 0.0 100* 16.5 136* 10.9 175* 6.3 211* 2.9 137* 10.8 212* 2.8 138* 10.6 213* 2.8 139* 10.5 214* 2.7 215* 2.6 216* 2.5 The DC Chart (Draft Capital) shows the values of draft picks as a function of performance value in relative value to the first overall pick. The division of rounds on the chart above reflect the 2017 draft order.

* denotes compensatory draft picks awarded to teams based on the net loss of veteran players on the free agent market; these picks are 'add-on' selections starting at the end of the third round, continuing through the seventh round.

Appendix B Figure 10. Performance Value by Position as a Function of Draft Order

Appendix C Optimizer Methodology

The optimization algorithm uses integer programing to automatically generate trade recommendations to search for alternatives during the trade negotiation process. The program uses the results of our research, the team’s set of draft picks, the trade partner’s set of draft picks, as well as user-determined constraints, as follows:

C.1. Objective Function

The objective of any trade is to maximize the expected value returned within the limitations of the market. Therefore, we express the objective function (1) to be maximized as;

푛 푅 푅 푅 푚 퐺 퐺 (1) ∑푖=1 푋푖 ∗ 푃푉푖 ∗ 푃표푠퐴푑푗푖 − ∑푗=1 푋푗 ∗ 푃푉푖 ,

Where first term in the expression refers to the total expected performance value of picks the

푅 team will receive, where 푋푖 is a binary variable to indicate whether pick i will be included in

푅 푅 the trade, 푃푉푖 is the expected performance value of pick i and 푃표푠퐴푑푗푖 is the positional adjustment factor for cases when the team specifies the position of the player it expects to target. The second term in the expression refers to the picks given up in the trade.

C.2.1 Market Constraints

To ensure that trades recommended by the program are reasonable for the trade partner, we use the both the chart (2) and the estimated trade market (3) to represent the implied market value to be used as constraints within the optimizer.

푛 푅 푅 푚 퐺 퐺 (2) ∑푖=1 푋푖 ∗ 퐶ℎ푎푟푡푉푖 − ∑푗=1 푋푗 ∗ 퐶ℎ푎푟푡푉푖 ≤ 300, or

푛 푅 푅 푚 퐺 퐺 (3) ∑푖=1 푋푖 ∗ 푀푎푟푘푒푡푉푖 . − ∑푗=1 푋푗 ∗ 푀푎푟푘푒푡푉푖 ≤ 0.1,

Where 퐶ℎ푎푟푡푉푖 and 푀푎푟푘푒푡푉푖 represent the value of the i-th per the chart and the market respectively. Note equation (2) searches for trades in which the other team does not lose by

more than 300 points per the chart, while equation (3) prevents trades where the trade partner loses by more than 10% of the implied market value.

C.2.2. Salary Cap Constraints

We include a constraint in the optimizer to account for the salary cap and the fixed rookie allocation pool that limits the total amount a team can use to sign its draft picks;

푛 푅 푅 푚 퐺 퐺 (4) ∑푖=1 푋푖 ∗ 퐶표푚푝푒푛푠푎푡표푛푖 − ∑푗=1(1 − 푋푗 ) ∗ 퐶표푚푝푒푛푠푎푡푖표푛푖 ≤ 퐷푟푎푓푡퐶푙푎푠푠푆푎푙푎푟푦퐶푎푝,

Where that the first term of equation (4) represents the values in compensation for picks received by the team in the trade, while the second term represents compensation for the team’s set of draft picks not included in the trade. The total value of compensation cannot exceed the fixed limit determined by the rookie allocation pool.

C.2.3. Team-Imposed Constraints

The program allows the team to impose additional constraints to adjust for the instance-based scenario for any given trade opportunity;

a. Include or exclude specific draft picks:

푋푗 to 1 or 0

b. Limit number of picks given by the team:

푛 퐺 ∑푖=1 푋푖 ≥ 푀푎푥푁푢푚푏푒푟표푓푃푖푐푘푠 (푔푖푣푒)

c. Limit number of picks received by the team:

푛 퐺 ∑푖=1 푋푖 ≥ 푀푎푥푁푢푚푏푒푟표푓푃푖푐푘푠 (푟푒푐푒푖푣푒)

Additional user-determined constraints consider the desired discount rate of future year picks, the number of future year picks included in the trade, as well as the valuation mechanism used in the objective function—to maximize the trade utility using total performance, or net surplus.

References

Burke, B. (2016, January 15). The Value of Each Draft Pick: A Re-Examination of Massey-Thaler Surplus Value under the New CBA. Retrieved from http://www.advancedfootballanalytics.com/index.php/home/research/payroll-personnel/242- roster-construction-using-optimization Drake, S. (2012, November 13). What are NFL draft picks really worth (and do they help teams win)? Retrieved January 06, 2017, from http://www.sportsplusnumbers.com/2012/11/what-are-nfl- draft-picks-really-worth.html Football Outsiders. (2016) Player Performance Statistics (2005-2014). Retrieved from http://www.footballoutsiders.com/ Gilboa, I., & Schmeidler, D. (1997). Act similarity in Case-Based Decision Theory. Economic Theory, 9, 47–61. Gonzalez, C. (2003). Instance-based learning in dynamic decision making. Cognitive Science, 27(4), 591-635. Klein, G., Orasanu, J., Calderwood, R., & Zsambok, C. E. (Eds.). (1993). Decision making in action: Models and methods. Norwood, NJ: Ablex Publishing Corporation. Kluegel, A. J. (2015). On The Clock: The Life and Death of a Market Convention. Conference Papers -- American Sociological Association, 1-29. Massey, C. B., & Thaler, R. H. (2005, April 2). The Loser’s Curse: Overconfidence vs. Market Efficiency in the National Football League Draft. SSRN Electronic Journal. Massey, C. B. and Thaler, R. H. (2012, August 31). "The Loser's Curse: Decision Making and Market Efficiency in the National Football League Draft" Management Science 59(7):1479-1495. Meers, K. (2011, November 30). How to Value NFL Draft Picks. Retrieved from https://harvardsportsanalysis.wordpress.com/2011/11/30/how-to-value-nfl-draft-picks NFL Collective Bargaining Agreement (2011). Article 7 Rookie Compensation and Rookie Compensation Pool, ss. 3-8. Paine, N. (2014, May 8). No Team Can Beat the Draft. Retrieved from http://fivethirtyeight.com/features/no-team-can-beat-the-draft/ Pro Football Focus. (2016) Player Performance Statistics (2005-2014). Retrieved from https://www.profootballfocus.com/ Pro-Football-Reference. (2016) Player Performance Statistics and Player Information (2003-2014). Retrieved from http://www.pro-football-reference.com/play-index/psl_finder.cgi Simon, H. A., & Langley, P. (1981). The central role of learning in cognition. In H. A. Simon (Ed.), Models of thought (Vol. II, pp. 102–184). New Haven, London: Yale University Press. Spotrac. (2016) Player Contract Information (2005-2014). Retrieved from http://www.spotrac.com/nfl/contracts/ Staff, Investopedia. (2016, September 28). Efficient Market Hypothesis - EMH. Retrieved from http://www.investopedia.com/terms/e/efficientmarkethypothesis.asp Stuart, C. (2012, November 18). Creating a Draft Value Chart, Part II. Retrieved from http://www.footballperspective.com/creating-a-draft-value-chart-part-ii/ Thaler, R. H. (1988). Anomalies: The Winner's Curse. Journal of Economic Perspectives, 2(1), 191-202. Tversky, A., & Kahneman, D. (1971). Belief in the law of small numbers. Psychological Bulletin, 76(2), 105-110. Zimmer, T. E. (2016). The Impact of NFL salary cap concentration on team success. Choregia, 12(1), 53-66.