Simulation-Based Projections for Baseball Statistics

Home , Baseball Prospectus, Hits per run, Isolated Power, PECOTA, Plate appearance, Times on base

A Thesis

Presented to the

Faculty of

California State Polytechnic University, Pomona

In Partial Fulfillment

Of the Requirements for the Degree

Master of Science

Computer Science

Daniel Adam Acevedo

2018

SIGNATURE PAGE

THESIS: SIMULATION-BASED PROJECTIONS

FOR BASEBALL STATISTICS

AUTHOR: DANIEL ADAM ACEVEDO

TERM SUBMITTED: Spring 2018

Computer Science Department

Dr. Yu Sun ______Thesis Committee Chair Department of Computer Science

Dr. Abdelfattah Amamra ______Department of Computer Science

Dr. Sampath Jayarathna ______Department of Computer Science

ii ACKNOWLEDGEMENTS

I would like to thank my family for their love, support, and for all of the sacrifices they’ve made so that I could get an education, resulting in a Master’s Degree. I would like to thank my advisor, Dr. Yu Sun, for his guidance and help throughout my time conducting this thesis. I would also like to thank my Dr. Sampath Jayarathna and Dr.

Abdelfattah Amamra for being members of my committee.

iii ABSTRACT

Baseball is an unpredictable sport. The introduction of sabermetrics established an opening for the application of computer science methods within the game’s evaluation.

Every Major League Baseball organization has developed their own method of measuring players’ results and making predictions as to what they should expect from a player entering a season. While most industry models use their own statistical analysis to perform predictions, this thesis introduces a new model that uses simulations in addition to statistical analysis in order to make predictions. The results of this thesis show that this model is comparable to some of the best projection systems available.

iv TABLE OF CONTENTS SIGNATURE PAGE ...... ii

ACKNOWLEDGEMENTS ...... iii

ABSTRACT ...... iv

LIST OF FIGURES ...... vi

1. INTRODUCTION ...... 1

2. ACQUIRING DATA AND WEIGHTS...... 4

2.1. Data Used ...... 4

2.2. Data Acquisition ...... 5

2.2.1. Setting up the Databases...... 5

2.2.2. Obtaining the Most Recent Four Year Period ...... 6

2.2.3. Obtain Weights ...... 7

2.2.4. Applying Weights and Regression to the Mean ...... 9

3. IMPLEMENTATION ...... 10

3.1. Explanation of a Simulation ...... 10

3.2. Creating a Prediction ...... 11

3.3. Apply Age Regression ...... 12

3.4. Prediction Example ...... 13

4. ANALYSIS ...... 19

4.1. Metrics for Evaluation ...... 19

4.2. Explanation of Industry Projections...... 19

4.3. Explanation of Metrics ...... 20

4.4. Predictions Comparison ...... 21

v 4.4.1. All Players ...... 22

4.4.2. Players with less than three years played ...... 26

4.4.3. Players with four or more years played ...... 30

5. CONCLUSION ...... 34

vi LIST OF FIGURES

Figure 1. Description of SQLite tables ...... 5

Figure 2. Description of statistics considered for predictions...... 6

Figure 3. Spinner Board example featuring BIP vs. Not a BIP ...... 10

Figure 4. Spinner Board example with numerous outcomes ...... 11

Figure 5. Wil Myers' Generalized Spinner Board ...... 14

Figure 6. Myers' Outcome Spinner Board ...... 15

Figure 7. Myers' Probabilities of outcomes given certain events occurring ...... 16

Figure 8. Myers' Resulting BIP vs Not a BIP ...... 17

Figure 9. Myers' Resulting Outcomes ...... 17

Figure 10. Myers' true rates and predicted values based on those rates ...... 18

Figures 11-15. Any Years Played MAE of Hit, Home Run, Runs Scored, RBI, and

WOBA Predictions ...... 23

Figures 16-15. Any Years Played RMSE of Hit, Home Run, Runs Scored, RBI, and

WOBA Predictions ...... 24

Figures 21-25. Any Years Played R of Hit, Home Run, Runs Scored, RBI, and WOBA

Predictions ...... 25

Figures 26-30. Less than Four Years Played MAE of Hit, Home Run, Runs Scored, RBI,

and WOBA Predictions ...... 27

Figures 31-35. Less than Four Years Played RMSE of Hit, Home Run, Runs Scored,

RBI, and WOBA Predictions ...... 28

Figures 36-40. Less than Four Years Played R of Hit, Home Run, Runs Scored, RBI, and

WOBA Predictions ...... 29

vi Figures 41-45. Four or More Years Played MAE of Hit, Home Run, Runs Scored, RBI,

and WOBA Predictions ...... 31

Figures 46-50. Four or More Years Played RMSE of Hit, Home Run, Runs Scored, RBI,

and WOBA Predictions ...... 32

Figures 51-55. Four or More Years Played R of Hit, Home Run, Runs Scored, RBI, and

WOBA Predictions ...... 33

vii 1. INTRODUCTION

Baseball is often called a game of failure – a player who hits the ball only three out of ten times is considered amongst the best players in the game. The prediction of baseball statistics could also be considered a game of failure, as prediction models try their best to utilize a large amount of data to project a player’s performance, but will ultimately never be able to perfectly predict a player’s performance consistently. Major

League Baseball (MLB) organizations have a substantial interest in the performance of their projection systems: teams pay salaries to players that are consistent with their performance, with the assumption that their success is reasonably sustainable in future years.

Most recently, in 2014, the Miami Marlins awarded Giancarlo Stanton with the largest monetary contract in MLB history, worth $325,000,000 over a thirteen-year period [1]. These large contracts are risks – Stanton played worse than his pre-contract average over the next two years, before obtaining the best statistical year of his career in

2017 [2]. The Marlins, under new ownership, traded Stanton to the New York Yankees, as the team could no longer afford such a large monetary contract. This highlights the importance of teams signing players to salaries that are consistent with their past performance and with an expectation that their performance will improve or stay the same, while simultaneously maintaining their budget.

A well-known example of the importance of maintaining a budget is portrayed in the film Moneyball. Based on a true story, it follows Oakland Athletics’ general manager

Billy Beane and his use of sabermetrics, the application of statistical analysis specific to baseball performance to evaluate and compare players. Beane would sign inexpensive

1 players who performed better than their basic statistics at a sustainable rate. Bill James, who coined the phrase sabermetrics in 1980, defined it as “the search for objective knowledge about baseball.” [3]

The use of sabermetrics has continually increased with time. Every MLB organization uses their own data analysis to place values on players. With money, revenue from fans, and the chase for a World Series championship always on the line, the use of sabermetrics continues to grow. Fantasy sports, with an estimated 57.4 million players in 2016 [4], have also raised the demand of projection systems outside of MLB franchises. Informed players constantly search for sources that can properly analyze sabermetrics, and use them to create their own projection systems. Many websites, such as Fangraphs, Baseball Prospectus, and ESPN, constantly produce baseball articles with in-depth sabermetric analysis, in addition to hosting their own projections.

The Player Empirical Comparison and Optimization Test Algorithm (PECOTA)

[5] is considered a top tier projection system. Developed by Nate Silver and hosted by

Baseball Prospectus to their paid users, PECOTA is a sabermetric system used to forecast player performance. The approach relies on features such as age, similar players, park factors, and past performance, amongst other considerations. Due to the proprietary nature of PECOTA, the exact calculations of the predictions are unknown.

Other models, such as Steamer [6] and ZiPs [7], are considered to be good projection models with great reputations, although not as in-depth as PECOTA. Another model, Marcel [8], is considered as the baseline for baseball predictions, due to its inherent simplicity. Each of these projection systems will be discussed in detail later in this thesis.

2 This projection system, denoted “Acevedo” for the purposes of this thesis, takes a different approach than these industry models. This simulation-based model derives from the approach presented in the book Curveball [9]. The book presents the idea of using sabermetrics and simulations in order to obtain a player’s true rates. The Acevedo model also accounts for age-related regression and regression to the league mean.

The purpose of this thesis is to create a projection system that takes a different approach for predicting player performance, while performing as well or better than other industry models. The model focuses strictly on offensive statistics, and outputs both the rates and predicted values based on its inputs. The predicted values include hits, home runs, runs scored, runs batted in, and stolen bases.

3 2. ACQUIRING DATA AND WEIGHTS

2.1. Data Used

All data comes from the Sean Lahman Baseball Database [10], which is publically available. The Lahman Baseball Database is used frequently in data analysis, as it contains player, franchise, and manager information and statistics dating from 1871 to 2017. Lahman updates this database every year to include the most recent year’s information.

Although data from 1871 is available, only data ranging back to 2003 is considered. While this decreases the amount of the data that is processed, the intent of the model is to reflect the way baseball has evolved in recent years. The era prior to 2003 is known as the “Steroid Era”, in which multiple players were accused of using performance-enhancing drugs to become better players. Since this era in baseball contains inflated power numbers that aren’t true in the game of baseball today, only data after the end of that era is considered such that the statistics are closer to way baseball is played today.

Pitchers were filtered out from the data set, as pitchers typically perform extremely poor when they are batting. As some positional players, players who play any non-pitcher position, are sometimes asked to pitch, the appropriate adjustments were made to ensure that these players are still included in the data set.

The statistics that derived from the Lahman Baseball Database include at bats, runs scored, hits, doubles, triples, home runs, runs batted in, stolen bases, caught stealing, strikeouts, base on balls, intentional base on balls, hit by pitch, sacrifice flies, and sacrifice hits. Using this information, the model is able to infer plate appearances, balls in

4 play, times on base, and all of the rates appropriate with those statistics, such as home runs per ball in play, or runs scored per time on base. The model utilizes a database which contains the same rate categories as above, but for two different situations: to fill in information for a player who has played less than four years, and to contain the difference in the rates between two consecutive ages. Players with no MLB experience are not included in the database. When all of the data is loaded into the database, the implementation begins.

2.2. Data Acquisition

2.2.1. Setting up the Databases

Using SQLite, the model parses the appropriate information from the Lahman

Baseball Database into its own database tables. In total, there are seven tables.

Table Name Description of Table stats Stores all hitters’ statistics from 2003-2017. historical Contains the difference in statistics between age and age+1. age Contains the league average statistics for a specific age from 2003 to 2017. projections Stores the Acevedo Projections. allProjections Stores projections for Acevedo, Marcel, Steamer, ZiPS, and Cairo. Also stores the actual result for the season. compareProjections Stores the Mean Absolute Error, Root Mean Square Error, and Correlation Coefficient between the five projection systems and the actual result for the season. Figure 1. Description of SQLite tables

The following information for each positional player from 2003 to 2017 is loaded from the Lahman Baseball Database and into the stats database:

5 From Lahman Baseball Database Derived from Lahman Baseball Database

Player’s ID Singles Player’s first and last name Batting Average Year On Base Percentage Player’s age Slugging Percentage At Bats On Base Percentage + Slugging Percentage Plate Appearances Isolated Power Hits Batting Average on Balls in Play Doubles Balls in Play (BIP) Triples Times on Base (TOB) Home Runs Strikeout Percent Runs Scored Base on Balls Percent Runs Batted In Intentional Walk Percent Base on Balls Balls in Play Percent Intentional Walks Singles per BIP Hit by Pitches Doubles per BIP Sacrifice Flies Triples per BIP Stolen Bases Home Runs per BIP Caught Stealing Sacrifice Flies per BIP Runs per TOB Runs Batted In per BIP Stolen Base Attempts per TOB Stolen Base Percent Figure 2. Description of statistics considered for predictions. 2.2.2. Obtaining the Most Recent Four Year Period

The first step in the implementation is obtaining each player’s statistics over a four-year period. There are two cases:

1. The player has played in all of the four previous seasons.

6 2. The player has not played in all four previous seasons, but has played in at least

one of these seasons.

In case one, the statistics from the previous four seasons within the stats data table are considered. In case two, only the statistics from the seasons in which a player played are considered, and the model fills in the remaining seasons using the league’s average statistics for the seasons in which the player has no data. For example, Kris Bryant, a third basemen from the Chicago Cubs, is entering the 2018 season with three seasons played: 2015, 2016, and 2017. In his case, the model calculates the league average statistics for the 2014 season, and uses that as the fourth year’s data.

2.2.3. Obtain Weights

Each statistic shares the same weights. The weights’ value is based off of the player’s plate appearances. There are four cases when determining the weights:

1. The player has played in each of the previous four seasons.

2. The player has played in three of the previous four seasons.

3. The player has played in two of the previous four seasons.

4. The player has played in one of the previous four seasons.

In each case, the weights are calculated differently.

2.2.3.1. Four Seasons Played

The model finds the average amount of plate appearances over the four-year period, and stores that into a variable. Then, the model places a higher weight on plate appearances from a player’s most recent season. The most recent season’s plate appearances are multiplied by 1.15, the following season is multiplied by 1.075, the third season is multiplied by .925, and the final season is multiplied by .85.

7 The plate appearances are then modified again based on the amount of times a player’s plate appearances were higher or lower than the average stored in the variable earlier. If the plate appearances were higher more often than they were lower than the average, a higher weight is placed onto the season with the highest plate appearances, and a lower weight is placed onto the season with the lowest plate appearances. If the plate appearances were lower more often than they were higher, the opposite of the previous scenario is calculated. The weight is then returned in the form of an array, where each index is calculated by the weighted plate appearances divided by the total plate appearances over the four-year period.

2.2.3.2. Less Than Four Years Played

When a player has less than four years played, the approach to applying the weights completely changes. In the case where a player only has one season played, the model takes that amount of plate appearances and multiplies it by .79, and sets the array’s first index equal to the result. Then, the league average’s plate appearances for the three missing seasons are used, each multiplied by .07, and then placed into the correct indexes. If the player has played two seasons, the most recent season is multiplied by .5, the second most recent season is multiplied by .3, and the two league average seasons are multiplied by .1; each calculation is then placed in the correct index within the array. In the final case, where the player has played three seasons, .35, .275, .225, and .15 is multiplied across the three most recent seasons and the league average season, respectively.

8 2.2.4. Applying Weights and Regression to the Mean

Using this array of weights, the model is able to manipulate each sabermetric value to prepare for the simulation. Strikeout Percent, Base on Balls Percent, Balls in

Play Percent, Singles per BIP, Doubles per BIP, Triples per BIP, Home Runs per BIP,

Sacrifice Flies per BIP, Runs per TOB, RBI per BIP, Stolen Base Attempts per TOB, and

Stolen Base Percent are the recipients of these weights.

After the weights have been applied, the values are summed together. They are passed into a function that regresses these values to the league mean. This is done by taking the current value and multiplying it by .9, and then taking the league average value of that statistic and multiplying it by .1. These two numbers are then added together into order to form the final value. These values represent the ranges in which an event occurs, which will be explained in detail in the Implementation section.

9 3. IMPLEMENTATION

3.1. Explanation of a Simulation

A simulation is best illustrated as a spinner board. The spinner board is the representation of all of the possible events that can occur as a result of a plate appearance.

For every plate appearance, there are two generalized events that can occur: the ball is hit into play, or the ball is not hit into play.

Figure 3. Spinner Board example featuring BIP vs. Not a BIP These two events can be broken down into more precise outcomes. When a ball is not hit into play, there are only two possible outcomes: the batter has struck out, or the batter has walked. Strikeouts could be broken down into a swinging strikeout and a looking strikeout, and a walk could be broken down into a walk on four balls thrown, an intentional walk, or a hit by pitch, but for the purposes of this simulation, each event is considered to be a general strikeout or walk.

When a ball is hit into play, there are many possible outcomes. A hitter can hit a single, double, triple, home run, sacrifice fly, or hit into an out. In the simulation, each

10 player is left with a small section of the spinner that is not defined by any category. The spinner does not consider this area when it lands on it, and it is classified it an error on defense.

Figure 4. Spinner Board example with numerous outcomes Each plate appearance includes the possibility of getting on base or hitting an

RBI. An overlapping area of the spinner is classified to take these possibilities into account. A stolen base can reasonably be predicted when a runner is on first or second base. A run can be scored as long as the player is on any base, or hits a home run. An RBI can only occur if there is already a runner on base, or the batter hits a home run.

3.2. Creating a Prediction

In order to create a reasonable prediction, a player’s performance is simulated multiple times. For each player, six hundred and fifty plate appearances, a reasonable amount of plate appearances over a full season of play, are simulated in order to obtain their season-long rates. This number is not the amount of plate appearances that is

11 predicted for the player, but rather is a metric of obtaining what a batter’s rates would be over the course of a full season.

The results from one season are not enough to reasonably obtain a player’s true rates. The results over any single season have an incredibly wide range – a player could produce anywhere between the worst season imaginable and the greatest season of all time. For this reason, a season is simulated multiple times.

Each player has ten thousand seasons simulated in order to obtain a wide variety of rates. The model records the result of every simulated season, and the amount of times each result occurred. Once the simulations have completed, the rates that occurred the most amount of times are considered. These rates are called the player’s true rates, rates that occurs a reasonable amount of times in a simulation that the model can confidently predict it is the most likely event.

Once the rates are obtained, the amount of plate appearances must be predicted.

For a player with four years played, this prediction is trivial: weights of .5, .25, .1, and .1 are placed on the amount of plate appearances of the most, second, third, and fourth most recent years, respectively. For players with less than four years played, higher weights are placed onto played years, and a value of fifty, two hundred, or three hundred is added, depending on the amount of years played. With all simulations complete, an age regression must be applied.

3.3. Apply Age Regression

The application of age regression is broken down into two steps. The first step is to retrieve a player’s expected change in each statistic based on their current age. For example, if a player is twenty-five years old entering the season, the model calculates the

12 change in statistics for all players from their age twenty-five season to age twenty-six season, and adds those values to the player’s predicted rates.

The second step is identical to how Marcel projections account for aging. For players under the age of twenty-nine, twenty-nine is subtracted from their age, and then multiplied by .006. For players over the age of twenty-nine, the same subtraction is performed, but is multiplied by .003. This value represents the age regression percentage that is applied to each true rate. This regression is positive if the player is under twenty- nine years of age, and is negative if the player is older than twenty-nine. This results in the expectation that younger players are expected to improve, while older players are expected to decline.

3.4. Prediction Example

As an example, Wil Myers, a first basemen and outfielder for the San Diego

Padres, will have his 2018 season predicted. Following along with the steps from the earlier sections, the first step is analyzing the percent of times that the simulation will expect a ball in play and a ball to not hit in play.

Figure 5. Wil Myers' Generalized Spinner Board The simulation breaks down these two events into numerous possible outcome probabilities for each of Myers’ plate appearances.

Figure 6. Myers' Outcome Spinner Board When predicting Myers’ runs, RBIs, and stolen bases, three separate conditions must occur. In order for a run to be scored, Myers must be on base, which occurs when he walks, hits a single, a double, or a home run. Myers can only steal a base if he is on first or second base, which occurs with a walk, single, or double. Stealing home base is an extremely rare event, so the projection system does not account for a player hitting a triple when calculating stolen bases. An RBI can occur when the ball is hit into play, regardless of if the outcome of the ball is a hit or an out.

Figure 7. Myers' Probabilities of outcomes given certain events occurring The red area of the graph represents the combined probabilities for the events described above. The blue area represents the probability of the event occurring. Once the simulation has ended, the following figures represent the final rates along with the predicted baseball statistics.

Figure 8. Myers' Resulting BIP vs Not a BIP

Figure 9. Myers' Resulting Outcomes

17 Name True Rate Predicted Value

Plate Appearances -- 566

Balls in Play .625 353

Singles .225 79

Doubles .078 27

Triples .005 1

Home Runs .065 22

Sacrifice Flies .011 3

Strikeouts .244 138

Base on Balls .122 69

Runs Scored .389 77

Runs Batted In .193 68

Stolen Bases .111 19

Figure 10. Myers' true rates and predicted values based on those rates Figure 10 displays Myers’ true rates after the simulation is complete. These rates are multiplied by either the predicted balls in play, or the predicted amount of times on base. The Predicted Value column shows this final result.

18 4. ANALYSIS

4.1. Metrics for Evaluation

In order to evaluate the performance of the model, it is compared against other well-known industry projections. These projections include Marcel, ZiPs, Cairo, and

Steamers. These models were chosen because they have good reputations and are available to the public for free. Three different metrics were chosen as a means of comparison: Mean Average Error (MAE), Root Mean Square Error (RMSE), and

Correlation Coefficient (R).

The statistics compared are hits, home runs, runs scored, runs batted in, and weighted on-base average (WOBA). WOBA is a sabermetric based on linear weights that measures a hitter’s overall contributions per plate appearance. WOBA’s weight calculation changes each year, and takes all hits and base on balls into consideration.

However, since WOBA’s calculations include hit by pitches and intentional base on balls, and many projection systems generalize these two statistics within base on balls, a slight adjustment is made to calculate WOBA. The equation for 2017 is:

. 693 ∗ 퐵퐵 + .877 ∗ 1퐵 + 1.232 ∗ 2퐵 + 1.552 ∗ 3퐵 + 1.979 ∗ 퐻푅 푊푂퐵퐴 = 퐴퐵 + 퐵퐵 + 푆퐹

4.2. Explanation of Industry Projections

Marcel, short for the Marcel the Monkey Forecasting System, uses “as little intelligence as possible,” [11] and is considered the baseline of all projection systems. It uses three years of statistics, and places a larger weight on the most recent years. It accounts for age regression and regresses players towards the league mean. The more information that is available for the player, the more reliable Marcel will project the player.

19 ZiPs, which stands for the sZymborski Projection System, uses growth and decline cycles based on player types to find trends [7]. For players between the ages of twenty-four and thirty-eight, the system uses the previous four years of statistics, with a higher weight placed on more recent seasons. For players older or younger than that range, the system uses three years of statistics, instead of four. ZiPs accounts of aging by comparing similar players’ past performance and their aging trends. It is regarded as one of the most accurate predictors in the industry [7].

Cairo uses Marcel as its baseline. In addition to the four most recent seasons, it uses a player’s age and position to account for regression. Minor league statistics, park and league factors, and an aging curve are also applied [11]. These projections are entered into Diamond Mind, a strategy-oriented baseball game [14], and are simulated fifty thousand times for the final results.

Steamer is substantially more complex than Marcel. It also uses weighted past performances regressed to the league mean [13]. The weights and extent of the regression varies between past statistics and a regression analysis of past players. Along with ZiPs, it is also widely regarded as one of the most accurate predictors in the industry [12].

4.3. Explanation of Metrics

Mean average error was chosen because represents a simple way to measure the errors of each projection. It measures each prediction equally, without penalizing errors.

MAE is also one of the easier calculations to evaluate. MAE is defined as:

푛 1 푀퐴퐸 = ∑ | 푥 − 푦 | 푛 푖 푖 푖=1 where n denotes the number of projections, xi denotes each projection, and yi represents the corresponding actual result. The closer to zero the result is, the better.

20 Root mean square error is very similar to mean average error. It differs from mean average error in that errors are penalized more. If a projection has a much higher root mean square error than its mean average error, it signals that the projection system is less consistent in making its predictions. RMSE is defined by:

∑푛 ( 푥 − 푦 )2 푅푀푆퐸 = √ 푖=1 푖 푖 푛

where n represents the number of projections, xi denotes each projection, and yi represents its corresponding actual result. As with MAE, the closer to zero the result is, the better.

Correlation coefficient, the final evaluation method, was chosen because it calculates the efficiency of a projection in a much different manner. It measures the correlation between two different numbers, in this case, a projection and the actual outcome. R is defined by:

푛 ∗ ∑푛 (푥 ∗ 푦 ) − ∑푛 (푥 ) ∗ ∑푛 (푦 ) 푟 = 푖=1 푖 푖 푖=1 푖 푖=1 푖 푛 2 푛 2 푛 2 푛 2 √(푛 ∗ ∑푖=1(푥푖 ) − ∑푖=1(푥푖) ) ∗ (푛 ∗ ∑푖=1(푦푖 ) − ∑푖=1(푦푖) ) where n denotes the number of projections, xi represents each projection, and yi represents the actual outcome. R’s outcome lands within the range of -1 to 1, in which -1 represents a negative correlation, 1 represents a positive correlation, and 0 represents no correlation.

4.4. Predictions Comparison

In this section, the performance of each projection model will be analyzed in three sections:

1. All players, regardless of the amount of years played, are considered.

2. Only players with less than four years played are considered.

21 3. Only players with four or more years played are considered.

4.4.1. All Players

When all players, regardless of the amount of years played, are considered, the

Acevedo model performs very well. When MAE is used for evaluation, Acevedo placed second in all but one category, in which it placed third. Steamer outperformed each model, placing in first each time. When RMSE is used, Steamer dominates again, placing first in all but one metric. Acevedo performs well again, placing second three times and third twice. When R is considered, Acevedo doesn’t perform quite as well. Acevedo places third twice and fourth three times. Steamer, again, sweeps the other models, placing first in each category. In general, when all players are considered, Acevedo performs in the upper half of these projections.

22 Figures 11-15. Any Years Played MAE of Hit, Home Run, Runs Scored, RBI, and WOBA Predictions

Actual Acevedo Steamer ZiPS Cairo Marcel

Rank 2 1 4 5 3

Actual Acevedo Steamer ZiPS Cairo Marcel

Home Runs 0 6 5.7 5.7 6.2 6.1

Rank 3 1 1 5 4

Actual Acevedo Steamer ZiPS Cairo Marcel

Runs 0 18.3 16.8 18.7 21.4 18.4

Rank 2 1 4 5 3

Actual Acevedo Steamer ZiPS Cairo Marcel

RBI 0 18.4 17.1 19.4 22.7 18.4

Rank 2 1 4 5 2

Actual Acevedo Steamer ZiPS Cairo Marcel

WOBA 0 .03999 .03875 .04007 .03967 .04004

Rank 3 1 5 2 4

23 Figures 16-20. Any Years Played RMSE of Hit, Home Run, Runs Scored, RBI, and WOBA Predictions

Actual Acevedo Steamer ZiPS Cairo Marcel

Rank 2 1 4 5 3

Actual Acevedo Steamer ZiPS Cairo Marcel

Home Runs 0 8.3 7.8 7.5 7.9 8.2

Rank 5 2 1 3 4

Actual Acevedo Steamer ZiPS Cairo Marcel

Runs 0 23.4 21.8 23.3 26.7 22.8

Rank 4 1 3 5 2

Actual Acevedo Steamer ZiPS Cairo Marcel

RBI 0 23.2 21.9 23.9 27.2 23.4

Rank 2 1 4 5 3

Actual Acevedo Steamer ZiPS Cairo Marcel

WOBA 0 .06693 .05371 .06678 .06666 .06827

Rank 4 1 3 2 5

24 Figures 21-25. Any Years Played R of Hit, Home Run, Runs Scored, RBI, and WOBA Predictions

Actual Acevedo Steamer ZiPS Cairo Marcel

Rank 3 1 2 5 4

Actual Acevedo Steamer ZiPS Cairo Marcel

Home Runs 1 .819 .853 .848 .835 .811

Rank 4 1 2 3 5

Actual Acevedo Steamer ZiPS Cairo Marcel

Runs 1 .801 .845 .833 .793 .813

Rank 4 1 2 5 3

Actual Acevedo Steamer ZiPS Cairo Marcel

RBI 1 .813 .843 .833 .786 .811

Rank 3 1 2 5 4

Actual Acevedo Steamer ZiPS Cairo Marcel

WOBA 1 .606 .678 .608 .609 .577

Rank 4 1 3 2 5

25 4.4.2. Players with less than three years played

When only the players who have played less than three years are considered, the

Acevedo model does not perform as efficiently. This is because the Acevedo model uses four years of player statistics, and fills in players with less than four years played with league average statistics. When MAE is used, Acevedo ranks fourth once, and third four times. Steamer performs best in four categories, but strangely performs worst in WOBA.

The overall results for RMSE are more mixed, but Acevedo performs decently. It scores second and third once each, and fourth three times. When using R, Acevedo has a subpar performance. It places fourth twice and fifth three times. In general, when only players with less than four years of experience are considered, the Acevedo model performs worse than if all players were being projected.

26 Figures 26-30. Less Than Four Years Played MAE of Hit, Home Run, Runs Scored, RBI, and WOBA Predictions

Actual Acevedo Steamer ZiPS Cairo Marcel

Rank 3 1 4 5 2

Actual Acevedo Steamer ZiPS Cairo Marcel

Home Runs 0 6.5 5.7 5.8 6.4 6.8

Rank 4 1 2 3 5

Actual Acevedo Steamer ZiPS Cairo Marcel

Runs 0 21.8 17.8 22.1 25.3 20.3

Rank 3 1 4 5 2

Actual Acevedo Steamer ZiPS Cairo Marcel

RBI 0 21.1 17.8 21.9 24.6 20.9

Rank 3 1 4 5 2

Actual Acevedo Steamer ZiPS Cairo Marcel

WOBA 0 .04085 .04798 .04042 .03940 .04248

Rank 3 5 2 1 4

27 Figures 31-35. Less Than Four Years Played RMSE of Hit, Home Run, Runs Scored, RBI, and WOBA Predictions

Actual Acevedo Steamer ZiPS Cairo Marcel

Rank 3 1 4 5 2

Actual Acevedo Steamer ZiPS Cairo Marcel

Home Runs 0 8.7 8.1 7.2 7.8 9.1

Rank 4 3 1 2 5

Actual Acevedo Steamer ZiPS Cairo Marcel

Runs 0 27.2 23.8 26.7 30.8 25.1

Rank 4 1 3 5 2

Actual Acevedo Steamer ZiPS Cairo Marcel

RBI 0 25.8 22.9 25.7 29.1 25.3

Rank 4 1 3 5 2

Actual Acevedo Steamer ZiPS Cairo Marcel

WOBA 0 .05657 .06627 .05530 .05711 .05945

Rank 2 5 1 3 4

28 Figures 36-40. Less Than Four Years Played R of Hit, Home Run, Runs Scored, RBI, and WOBA Predictions

Actual Acevedo Steamer ZiPS Cairo Marcel

Rank 5 1 2 3 4

Actual Acevedo Steamer ZiPS Cairo Marcel

Home Runs 1 .755 .842 .860 .837 .750

Rank 4 2 1 3 5

Actual Acevedo Steamer ZiPS Cairo Marcel

Runs 1 .655 .810 .791 .708 .707

Rank 5 1 2 3 4

Actual Acevedo Steamer ZiPS Cairo Marcel

RBI 1 .700 .821 .815 .756 .725

Rank 5 1 2 3 4

Actual Acevedo Steamer ZiPS Cairo Marcel

WOBA 1 .612 .488 .644 .614 .620

Rank 4 5 1 3 2

29 4.4.3. Players with four or more years played

When only players with four or more years of experience are considered, the

Acevedo model performs at its best. Acevedo scores first and second twice each, and third once when MAE is used as the valuation metric. It scores second three times, third once, and fifth once when RMSE is considered. Finally, when R is considered, Acevedo does very well, placing second in all but one category, where it placed in first. In general, when only players with four or more years of experience are considered, Acevedo consistently outperforms most of the other projections. This is due to the fact that the model is able to use four years of each player’s statistics when creating their predictions.

30 Figures 41-45. Four or More Years Played MAE of Hit, Home Run, Runs Scored, RBI, and WOBA Predictions

Actual Acevedo Steamer ZiPS Cairo Marcel

Rank 2 1 3 5 4

Actual Acevedo Steamer ZiPS Cairo Marcel

Home Runs 0 5.7 5.6 5.7 6.0 5.8

Rank 2 1 2 5 4

Actual Acevedo Steamer ZiPS Cairo Marcel

Runs 0 16.1 16.2 16.5 19.0 17.4

Rank 1 2 3 5 4

Actual Acevedo Steamer ZiPS Cairo Marcel

RBI 0 16.7 16.7 17.9 21.6 18.1

Rank 1 1 3 5 4

Actual Acevedo Steamer ZiPS Cairo Marcel

WOBA 0 .03945 .03279 .03984 .03983 .03867

Rank 3 1 5 4 2

31 Figures 46-50. Four or More Years Played RMSE of Hit, Home Run, Runs Scored, RBI, and WOBA Predictions

Actual Acevedo Steamer ZiPS Cairo Marcel

Rank 2 1 3 5 4

Actual Acevedo Steamer ZiPS Cairo Marcel

Home Runs 0 8.0 7.6 7.7 7.9 7.6

Rank 5 1 3 4 1

Actual Acevedo Steamer ZiPS Cairo Marcel

Runs 0 20.7 20.4 20.7 23.7 21.4

Rank 2 1 2 5 4

Actual Acevedo Steamer ZiPS Cairo Marcel

RBI 0 21.5 21.3 22.6 26.0 22.3

Rank 2 1 4 5 3

Actual Acevedo Steamer ZiPS Cairo Marcel

WOBA 0 .07265 .04373 .07327 .07196 .07275

Rank 3 1 5 2 4

32 Figures 51-55. Four or More Years Played R of Hit, Home Run, Runs Scored, RBI, and WOBA Predictions

Actual Acevedo Steamer ZiPS Cairo Marcel

Rank 2 4 1 5 2

Actual Acevedo Steamer ZiPS Cairo Marcel

Home Runs 1 .848 .858 .843 .835 .843

Rank 2 1 3 5 3

Actual Acevedo Steamer ZiPS Cairo Marcel

Runs 1 .863 .859 .868 .841 .862

Rank 2 4 1 5 3

Actual Acevedo Steamer ZiPS Cairo Marcel

RBI 1 .859 .849 .847 .803 .825

Rank 1 2 3 5 4

Actual Acevedo Steamer ZiPS Cairo Marcel

WOBA 1 .615 .793 .589 .606 .570

Rank 2 1 4 3 5

33 5. CONCLUSION

The goal of this thesis was to create to a baseball prediction system that was capable of competing with other industry models. Similar to these other models, Acevedo predicts most of the basic hitting statistics. The performance of projection systems is becoming more important to MLB organizations every year. This projection system performs in a way that is very different to most other models, due to its simulation-based background, but is very competitive with them.

The more data is available for a player, the better the Acevedo model performs. In fact, the Acevedo model consistently outperforms most of the other prediction models when more players have at least four or more years of data available. The approach used, while very different in its nature than other models, performed admirably well, proving that this projection system can be successful.

34 REFERENCES

[1] “Highest Paid Players.” Baseball Prospectus,

legacy.baseballprospectus.com/compensation/cots/league-info/highest-paid-

players/.

[2] “Giancarlo Stanton Stats.” Baseball Reference, www.baseball-

reference.com/players/s/stantmi03.shtml.

[3] “A Guide to Sabermetric Research.” Society for American Baseball Research,

sabr.org/sabermetrics.

[4] “Fantasy Sports Participation in North America Rises to 57.4 Million

Players.” FSTA, fsta.org/fantasy-sports-participation-in-north-america-rises-to-57-

4-million-players/.

[5] “PECOTA.” Baseball Prospectus | Compensation | Baseball Prospectus,

legacy.baseballprospectus.com/glossary/index.php?mode=viewstat&stat=476.

[6] “Steamer.” Steamer Projections, steamerprojections.com/blog/about-2/.

[7] “What Is a SZymborksi Projection System (ZiPS)? | Glossary.” Major League

Baseball, m.mlb.com/glossary/projection-systems/szymborski-projection-system.

[8] “What Is a Marcel the Monkey Forecasting System (Marcel)? | Glossary.” Major

League Baseball, m.mlb.com/glossary/projection-systems/marcel-the-monkey-

forecasting-system.

[9] “Prediction.” Curveball, by Jim Albert and Jay Bennett, vol. 2, Copernicus

Books, 2003, pp. 323-347.

[10] Lahman Baseball Database, www.seanlahman.com/baseball-database.html.

35 [11] Slowinksi, Steve. “The Projection Rundown: The Basics on Marcels, ZiPS,

CAIRO, Oliver, and the Rest | FanGraphs Sabermetrics Library.” FanGraphs,

www.fangraphs.com/library/the-projection-rundown-the-basics-on-marcels-zips-

cairo-oliver-and-the-rest/.

[12] “What Is a Steamer? | Glossary.” Major League Baseball,

www.mlb.com/glossary/projection-systems/steamer.

[13] Druschel, Henry. “A Guide to the Projection Systems.” Beyond the Box Score,

https://www.beyondtheboxscore.com/2016/2/22/11079186/projections-marcel-

pecota-zips-steamer-explained-guide-math-is-fun.

[14] “Simulation Baseball Game.” Diamond Mind Baseball, diamond-mind.com/.