Dwar: Introducing a Method to Actually Calculate Wins Above Replacement

Home , Baseball Prospectus, Daniel Norris, George Kottaras, Huston Street, Jairo Labourt, Jason Motte

dWAR: introducing a method to actually calculate wins above replacement

Daniel J. Eck

March 3, 2019

1 Introduction

Wins above replacement (WAR) is meant to be a one-number summary of the total contribution made by a player for his team in any particular season. As stated by Steve Slowinski of Fangraphs, WAR oﬀers an estimate to answer the question, “If this player got injured and their team had to replace them with a freely available player of lower quality from their bench, how much value would the team be losing,” where this value is expressed in number of wins [Slowinski, 2010]. That being said, nobody actually calculates WAR in a manner that properly answers the above question as posed. This is not by any fault of the metric and those who calculate it. One problem is that it is impossible to simultaneously quantify the value of a player when the player is available and the value of a replacement to that player when the player is unavailable. The player in question is either available to play or unavailable to play, never both. Instead of confronting the problems raised in this factual-counterfactual world, people have attempted to calculate a hypothetical replacement player to implicitly compare every player with using the machinery of a proprietary black box [Baumer et al., 2015]. Three widely used versions of WAR that are calculated in this manner are Baseball Reference’s bWAR [Reference, 2010], Fangraphs’s fWAR [Slowinski, 2010], and Baseball Prospectus’s bWARP [Prospectus, 2019].

In this note, we propose a direct estimator of wins above replacement that confronts the diﬃculty of the factual-counterfactual real world. Note that there are numerous examples of seasons in which a player is available and unavailable for a substantial amount of time. When this is so, we can directly compare how the team performs when the player is available to how the team performs when the player is unavailable. This framework allows for a direct estimation of the wins that a player adds above a replacement player. This direct estimator is relatively simple to compute, available, easy to understand, and its interpretation is ﬂexible to the narrative of a season. We will refer to this direct calculation of WAR as dWAR, which is short for direct WAR, which is shorthand for the direct calculation of wins above replacement. The dWAR estimator has the potential to yield a much more natural and appropriate estimate of WAR than those which involve the calculation of a hypothetical replacement player via black box methodology. The validity of dWAR depends on the team and the competition faced by the team being similar during both player states. Very nuanced interpretations of what dWAR measures emerge when team makeup is confounded with the availability of the player in question.

1 We primarily focus on the 2014 Yadier Molina season to show the discrepancies between conventional calculations of WAR and dWAR. Our version of WAR gives much more value to Yadier Molina’s 2014 season than conventional versions. This result is far from surprising. Many note that conventional versions of WAR do not properly account for leadership, game management, pitch framing, and catcher defense, which are all aspects of baseball that Molina excels at [Fagan, 2015, Posnanski, 2015, Schwarz, 2015, Fleming, 2017, Womack, 2017]. That being said, a tangible numeric value of the additional Cardinals wins attributable to Molina as a result of these intangible traits has not existed until now. We caution against generalizing our ﬁndings beyond the 2014 season with certainty, but we hope that the point is taken and can be used to strengthen Molina’s case for the Hall of Fame. The point here being that conventional versions of WAR likely have underestimated the number of Cardinals wins attributable to Yadier Molina by a substantial amount.

Additional analyses are provided for Miguel Cabrera’s 2015 season with the Detroit Tigers and Mike Trout’s 2017 season with the Los Angeles Angeles. These specific players are chosen because of the 2012 most valuable player (MVP)race between them that is symbolic of the fight between those who favor new sophisticated analytics to value a player’s production and those who favor traditional analytics to value a players production. As noted in Baumer et al. [2015], sabermetricians from the new school advocated strongly for Trout while those that preferred traditional statistics advocated strongly for Cabrera. To the adherents of sabermetrics, the decision for who should win the 2012 MVP award was clear – point estimates showed Trout leading Cabrera by 3.2 fWAR and 3.6 bWAR. The openWAR metric in Baumer et al. [2015] provided far more sophistication to this debate. According to openWAR, the estimated difference between Trout and Cabrera is only 1.05 WAR in Trout’s favor. Moreover, there is substantial overlap of the interval estimates of Trout’s and Cabrera’s openWAR. We do not provide a dWAR estimate of these player’s WAR in 2012 because these players did not not miss a significant portion of the 2012 season, which voids comparisons to a suitable replacement player under the dWAR framework. However, both 2012 Cabrera and 2012 Trout were archetypically similar player in 2015 and 2017 respectively. Our dWAR estimates of WAR for 2015 Miguel Cabrera and 2017 Mike Trout give the opposite impression that conventional WAR and openWAR give for theses player’s respective value for their teams. In 2015, the Detroit Tigers were far worse when Miguel Cabrera did not play or was injured. However, the 2017 Los Angeles Angels were not terribly hindered by the absence of Mike Trout’s production. These findings are striking (especially for Trout) and they come with natural caveats arising from the context of those seasons. These caveats are explored.

2 Data Analyses

2.1 Yadier Molina in 2014

Yadier Molina played in 110 regular season games for the St. Louis Cardinals in the 2014 baseball season out of a possible 162. The St. Louis Cardinals won a total of 90 games and lost a total of 62 games. When Yadier Molina played, the Cardinals won 62 games and lost 48 games. When Yadier Molina did not play, the Cardinals won 28 games and lost 24 games [ESPN, 2019i]. The games in which he did not play are split among games in which he was healthy but did not enter the game and games in which he was injured and unavailable to play in any capacity. The former category

2 represents a strategic decision that involves a healthy baseball player, the latter category represents an unplanned incident in which team strategic decisions have to change due to the unexpected loss of a player. When Yadier Molina did not play but was available, the Cardinals won 7 games and lost 5 games. When Yadier Molina was injured, the Cardinals won 21 games and lost 19 games. Yadier Molina’s WAR in 2014 was 2.5 as estimated by BWARP, 2.9 as estimated by fWAR, and 3.1 as estimated by bWAR.

We now motivate three versions of a player’s WAR that directly answers the question as posed in the Introduction. We compare how the team did in games in which the player played to those in which the player did not play, denoted as dWAR\ on-oﬀ. This estimate of WAR is calculated as

dWAR\ on-off = (ˆpon − pôff) × G wherep ôn is the proportion of team games won when the player played,p ôff is the proportion of team games won when the player did not play, and G is the number of total games played. We then compare how the team did in games in which the player was available to play to those in which the player was unavailable to play, denoted dWAR\ avail-unavail. This estimate of WAR is calculated as

dWAR\ avail-unavail = (ˆpavail − pûnavail) × G wherep âvail is the proportion of team games won when the player played or was available to play andp ûnavail is the proportion of team games won when the player was unavailable to play. We finally compare how the team did in games in which the player played to those in which the player was unavailable to play, denoted dWAR\ on-unavail. This estimate of WAR is calculated as

dWAR\ on-unavail = (ˆpon − pˆunavail) × G.

These estimates of dWAR for Yadier Molina’s 2014 season are,

dWAR\ on-oﬀ = (62/110 − 28/52) × 110 = 2.77,

dWAR\ avail-unavail = (69/122 − 21/40) × 110 = 4.46,

dWAR\ on-unavail = (62/110 − 21/40) × 110 = 4.25.

The versions of dWAR that compare team success when Yadier Molina is available, and his playing time is subject to management’s decision, to team success when Yadier Molina is unavailable are drastically diﬀerent than conventional calculations of this metric. The discrepancy between these approaches is anywhere between 1.15 and 1.96 wins depending on which estimates of WAR are being compared. Interpretations of the discrepancy between these metrics are massive. Slowinski [2010] provided a rule-of-thumb guideline for interpreting WAR. According to these guidelines, a WAR between 2.5 and 3.1 corresponds to a player that is anywhere from a solid starting player to a good player, while a WAR between 4.25 and 4.46 corresponds to an all-star level player that performed near the top of the league.

The following are the notable injuries during the 2014 Cardinals season: Yadier Molina injured from July 10th until August 27th [ESPN, 2019i]; Jaime Garca will made his ﬁrst start on May 18 [of Communications, 2014]; Jason Motte made his return on May 21 [Langosch, 2014]; Jaime Garca announced on July 5 that he would have season-ending surgery [Wikipedia, 2019]. The following

3 are notable transactions made by the Cardinals in 2014: On July 11, the Cardinals claimed George Kottaras oﬀ waivers from the Cleveland Indians as a possible replacement for Yadier Molina; on July 26, the Cardinals signed A.J. Pierzynski as a free agent as a possible replacement for Yadier Molina; on July 29, the Cardinals released George Kottaras; on July 31, the Cardinals traded Allen Craig and Joe Kelly for Corey Littrell, John Lackey and cash [Reference, 2019b].

We do not ﬁnd that these injuries and transactions account for the massive discrepancies between our dWAR estimator and the conventional approaches. The Cardinals faced similar competition when Yadier Molina was unavailable and when Yadier Molina was available [Reference, 2019c].

We conclude that dWAR\ on-oﬀ, dWAR\ avail-unavail, and dWAR\ on-unavail are preferable to conventional versions of WAR when assessing the value of Yadier Molina to the Cardinals in the 2014 season.

2.2 Miguel Cabrera in 2015

We perform the same analysis for Miguel Cabrera’s 2015 season with the Detroit Tigers. Miguel Cabrera played in 119 games in 2015 and we estimatep ôn = 57/119,p ôff = 17/42,p âvail = 59/126, andp ûnavail = 15/35 [ESPN, 2019a]. Conventional estimates of WAR for Miguel Cabrera’s 2015 season are, bW AR = 5.2, fW AR = 4.6, BW ARP = 5.3. Our estimates of dWAR for Miguel Cabrera’s 2015 season are,

dWAR\ on-oﬀ = (57/119 − 17/42) × 119 = 8.83,

dWAR\ avail-unavail = (59/126 − 15/35) × 119 = 4.72,

dWAR\ on-unavail = (57/119 − 15/35) × 119 = 6.00.

We take a look at significant injuries and transactions made by the Tigers in 2015. The following are significant injuries: Miguel Cabrera injured from July 4th until August 12th [ESPN, 2019a]; Justin Verlander injured until June 13th [ESPN, 2019h]; Victor Martinez injured from May 14th until June 17th [ESPN, 2019c]; Jose Iglesias injured from September 4th until October 4th [ESPN, 2019b]; Anibal Sanchez injured from August 18th until October 4th [ESPN, 2019e]. The following are significant transactions: On July 6, the Tigers claimed Marc Krauss off waivers from the Tampa Bay Rays as a possible Cabrera replacement; on July 30, the Tigers traded David Price for Daniel Norris, Matt Boyd and Jairo Labourt; on July 30, the Tigers traded Joakim Soria for shortstop JaCoby Jones; on July 31, the Tigers traded Yoenis Cespedes for Michael Fulmer and Luis Cessa; on August 20, the Tigers acquired Randy Wolf from the Toronto Blue Jays for cash considerations [Reference, 2019d].

We ﬁnd that these injuries and transactions result in some key diﬀerences between in the makeup of the 2015 Tigers during the stretches when Cabrera is either available or unavailable. The strongest team makeup, excluding the availability of Cabrera, was from June 17th until the trades that began July 30th. Outside of this stretch, dWAR would maintain validity of the unavailability of Victor Martinez and Justin Verlander is equal to the balance of the departures of David Price, Joakim Soria, and Yoenis Cespedes and arrivals of Daniel Norris, Matt Boyd, Jairo Labourt, JaCoby

4 Jones, Michael Fulmer, and Luis Cessa. Perhaps this is so, in which case dWAR retains its validity. Perhaps this is not so, and a more thorough analysis is required.

Another striking feature of this analysis is the differences between dWAR\ on-off, dWAR\ avail-unavail, and dWAR\ on-unavail. These differences are partially explained by the 2015 Tigers winning 2 games and losing 5 when Cabrera was available to play but did not enter the game. Perhaps management picked poor spots to rest Miguel Cabrera or perhaps this abysmal performance is explained by the inherent volatility of the 2015 Tigers season [Mowery, 2015] and the small amount of games in which Cabrera was available to play but did not enter the game. We think that it is likely that dWAR\ avail-unavail underestimates 2015 Miguel Cabrera’s value to the 2015 Tigers and that dWAR\ on-off overestimates 2015 Miguel Cabrera’s value to the 2015 Tigers. In any event, Miguel Cabrera was a phenomenal hitter in 2015 and all estimates of WAR pick up on this. That being said, It appears that conventional estimates of WAR underestimated the number of wins that Cabrera’s hitting brought to the Detroit Tigers in 2015.

2.3 Mike Trout in 2017

We perform the same analysis for Mike Trout’s 2017 season with the Los Angeles Angels. Mike Trout played in 114 games in 2017 and we estimatep ôn = 57/114,p ôff = 23/48,p âvail = 59/118, andp ûnavail = 21/44 [ESPN, 2019g]. Conventional estimates of WAR for Mike Trout’s 2017 season are, bW AR = 6.7, fW AR = 6.9, BW ARP = 6.2. Our estimates of WAR for Mike Trout’s 2017 season are,

dWAR\ on-oﬀ = (57/114 − 23/48) × 114 = 2.38,

dWAR\ avail-unavail = (59/118 − 21/44) × 114 = 2.59,

dWAR\ on-unavail = (57/114 − 21/44) × 114 = 2.59.

The dWAR estimates that compare team success when Mike Trout is available, and his playing time is subject to management’s decision, to team success when Mike Trout is unavailable are drastically diﬀerent than conventional calculations of this metric. The diﬀerences between the two approaches are massive and are not in Mike Trout’s favor.

As before, we take a look at significant injuries and transactions made by the Angels in 2017. The following are significant injuries: Mike Trout injured from May 6th until May 10th and from May 28th until July 9th [ESPN, 2019g]; many key Angels players played very limited time in 2017, including Garrett Richards and Huston Street [ESPN, 2019d,f]. The following are significant transactions: On June 1, the Angels signed Michael Bourn as a free agent as a possible Mike Trout replacement; on July 2, the Angels released Michael Bourn; on August 31, the Angels traded Elvin Rodriguez and Grayson Long for Justin Upton [Reference, 2019a].

We do not ﬁnd that these injuries and transactions account for the massive discrepancies between dWAR and the conventional approaches. However, Reiter [2017] notes that the Angels’ relative success in Trout’s absence is mind-boggling, especially given that the Angels faced relatively tough

5 competition during that stretch. He attributes the team success in Trout’s absence to the rest of the players coming together and clicking the moment Trout got hurt. We find this explanation to be interesting and plausible. We also find it plausible that Mike Trout is not the type of leader that can instill motivation in other players when being looked upon to do so. We do not have enough information to properly distinguish causes for this very strange “Mike Trout effect.”

3 Discussion

We hope that dWAR can help strengthen Yadier Molina’s deserving Hall of Fame case. It is entirely within the realm of possibility that conventional calculations of WAR will have underestimated Yadier Molina’s career WAR by 10-30 wins when he retires.

The 2017 Mike Trout season was truly mind-boggling and it illustrates the collective shortcomings of all metrics, including dWAR. The dWAR estimator of Mike Trout’s value to the 2017 Angels is valid in the sense that the 2017 Angels performed almost as well without Mike Trout as they performed with him. That is what happened and it is not debatable. However, using this as a final determination of Mike Trout’s value is very suspect. Perhaps the “Mike Trout effect” is real and perhaps it is a result of either lackadaisical uninspired teammates, poor leadership on Mike Trout’s part, poor team management, or all of the above, or possibly some other explanation. Even if this effect exists, it has only been shown to exist with respect to the 2017 Angels season and it may not exist in any other year or on any other team. Conventional indirect estimates of WAR provide a one-number summary of how statistically dominant Mike Trout was and currently is. To what extent this raw context neutral statistical account of Mike Trout’s greatness can translate to actual wins above replacement is an honest conversation worth having by fans, historians, scouts, sabermetricians, the baseball media, and team executives alike.

As it currently stands, dWAR can only validly estimate WAR for a player during a season in which the player spent a substantial time in both the available and unavailable states. Since the calculation of dWAR is constrained to one season in isolation, it cannot be used to properly compare players across eras as noted in Eck [2018]. Valid era-invariant extensions to dWAR for all player seasons can be made with a suitable era-invariant model of the probabilities of team success that are central to the calculation of dWAR. This is not a trivial task and we speculate that strong structural assumptions in a partially Bayesian latent variable modeling framework would be required to achieve believable era-invariance.

We would like to emphasize that traditional considerations of value including leadership ability, winning attitude, clutch performances, great defense, and other intangibles are important sup- plements to statistical accounts of value. Our metric dWAR can contribute to this emphasis by measuring to what extent an aggregation of these previously unmeasurable accounts of value has on the additional number of wins a particular player produces for his team over that of a suitable and available replacement player. If intangible value actually helps a team win when a player is on the ﬁeld, then dWAR can ﬁnd it when conventional calculations of WAR such as bWAR, fWAR, and BWARP will not. These conventional calculations of WAR are not actually in the business of directly calculating wins above replacement, dWAR is.

6 References

Benjamin S. Baumer, Shane T. Jensen, and Gregory J. Matthews. Openwar: An open source system for evaluating overall player performance in major league baseball. Journal of Quantitative Analysis in Sports, 11 (2):1–27, 2015.

Daniel J. Eck. Challenging nostalgia and performance metrics in baseball. ArXiv Preprint, 2018. URL https://arxiv.org/abs/1810.08029.

ESPN. Miguel cabrera game-by-game stats, 2019a. URL http://www.espn.com/mlb/player/ gamelog/_/id/5544/year/2015.

ESPN. Jose iglesias game-by-game stats, 2019b. URL http://www.espn.com/mlb/player/ gamelog/_/id/30382/year/2015.

ESPN. Victor martinez game-by-game stats, 2019c. URL http://www.espn.com/mlb/player/ gamelog/_/id/5007/year/2015.

ESPN. Garrett richards game-by-game stats, 2019d. URL http://www.espn.com/mlb/player/ gamelog/_/id/30892/year/2017.

ESPN. Anibal sanchez game-by-game stats, 2019e. URL http://www.espn.com/mlb/player/ gamelog/_/id/6472/year/2015.

ESPN. Huston street game-by-game stats, 2019f. URL http://www.espn.com/mlb/player/ gamelog/_/id/6175/huston-street.

ESPN. Mike trout game-by-game stats, 2019g. URL http://www.espn.com/mlb/player/ gamelog/_/id/30836/year/2017.

ESPN. Justin verlander game-by-game stats, 2019h. URL http://www.espn.com/mlb/player/ gamelog/_/id/6341/year/2015.

ESPN. Yadier molina game-by-game stats, 2019i. URL http://www.espn.com/mlb/player/ gamelog/_/id/5986/year/2014.

Ryan Fagan. Which active mlb players are hall of fame bound? the case for 15, 2015. URL http://www.sportingnews.com/us/mlb/list/ baseball-hall-of-fame-2015-voting-players-pujols-cabrera-kershaw-trout-posey-beltre-beltran/ dzae73sytib612elvquapw62q.

John Fleming. How does yadier molina compare to 2017s hall of fame ballot catchers?, 2017. URL https://www.vivaelbirdos. com/st-louis-cardinals-sabermetrics-analysis/2017/1/20/14275092/ cardinals-yadier-molina-2017-hall-of-fame-catchers-ivan-rodriguez-pudge-jorge-posada-jason-varitek.

Jenifer Langosch. Motte impresses in return to the mound, 2014. URL https://www.mlb.com/ news/cardinals-jason-motte-impresses-in-return-to-the-mound/c-76353834.

Matthew B. Mowery. Numbers added up for an abysmal season for tigers in 2015, 2015. URL https://www.theoaklandpress.com/ sports/numbers-added-up-for-an-abysmal-season-for-tigers-in/article_ 15663b9d-d334-538d-901a-8654dec6f70f.html.

7 St. Louis Cardinals Department of Communications. Game notes, 2014. URL http://cardinals. mlb.com/documents/8/9/4/75903894/51814_Layout_1_5dhyviqx.pdf.

Joe Posnanski. Kids in the hall, 2015. URL https://sportsworld.nbcsports.com/ active-baseball-players-hall-of-fame-chances/.

Baseball Prospectus. View details for warp, 2019. URL https://legacy.baseballprospectus. com/glossary/index.php?search=WARP.

Baseball Reference. Baseball-reference.com war explained, 2010. URL https://www. baseball-reference.com/about/war_explained.shtml.

Baseball Reference. 2017 los angeles angels trades and transactions, 2019a. URL https://www. baseball-reference.com/teams/LAA/2017-transactions.shtml.

Baseball Reference. 2014 st. louis cardinals trades and transactions, 2019b. URL https://www. baseball-reference.com/teams/STL/2014-transactions.shtml.

Baseball Reference. 2014 st. louis cardinals schedule, 2019c. URL https://www. baseball-reference.com/teams/STL/2014-schedule-scores.shtml.

Baseball Reference. 2015 detroit tigers trades and transactions, 2019d. URL https://www. baseball-reference.com/teams/DET/2015-transactions.shtml.

Ben Reiter. Mike trout is injured, but the angels have only improved during his absence, 2017. URL https://www.si.com/mlb/2017/06/29/los-angeles-angels-winning-mike-trout-injury.

Joe Schwarz. An update on the pitch framing of yadier molina, 2015. URL https: //www.vivaelbirdos.com/st-louis-cardinals-sabermetrics-analysis/2015/6/25/ 8842869/an-update-on-the-pitch-framing-of-yadier-molina.

Steve Slowinski. What is war? fangraphs, 2010. URL https://library.fangraphs.com/misc/ war/.

Wikipedia. 2014 st. louis cardinals season, 2019. URL https://en.wikipedia.org/wiki/2014_ St._Louis_Cardinals_season.

Graham Womack. Yadier molina is a clear hall of famer to cardinals pitchers, 2017. URL http://www.sportingnews.com/us/mlb/news/ yadier-molina-hall-of-fame-cardinals-catcher-stats-pitch-framing/ xkxgjg96e1ck1eqaiqqlyw2a3.

c 2019 Daniel J. Eck

This note is the intellectual property of Daniel J. Eck and terms of its redistribution are subject to the Creative Commons license:

Attribution-NonCommercial-ShareAlike 4.0 International (CC BY-NC-SA 4.0)

For details on this license, see https://creativecommons.org/licenses/by-nc-sa/4.0/