Measuring Umpire Consistency

Home , Adam Hamari, Adrian Johnson (umpire), Alan Porter, Andy Fletcher (umpire), Bill Miller (umpire), Bill Welke

David J. Hunter

January 11, 2018

Assumptions/Motivations

I Consistency is more important than conformity.

I A consistent zone need not be rectangular, but should be convex.

I Consistency within a game is important.

I Diﬀerent zones for LH and RH batters are OK.

I One egregiously bad call is worse/as bad as several marginally bad calls. Assumptions/Motivations

I Consistency is more important than conformity.

I A consistent zone need not be rectangular, but should be convex.

I Consistency within a game is important.

I Diﬀerent zones for LH and RH batters are OK.

I One egregiously bad call is worse/as bad as several marginally bad calls. Assumptions/Motivations

I Consistency is more important than conformity.

I A consistent zone need not be rectangular, but should be convex.

I Consistency within a game is important.

I Diﬀerent zones for LH and RH batters are OK.

I One egregiously bad call is worse/as bad as several marginally bad calls. Assumptions/Motivations

I Consistency is more important than conformity.

I A consistent zone need not be rectangular, but should be convex.

I Consistency within a game is important.

I Diﬀerent zones for LH and RH batters are OK.

I One egregiously bad call is worse/as bad as several marginally bad calls. Assumptions/Motivations

I Consistency is more important than conformity.

I A consistent zone need not be rectangular, but should be convex.

I Consistency within a game is important.

I Diﬀerent zones for LH and RH batters are OK.

I One egregiously bad call is worse/as bad as several marginally bad calls. Deﬁne established ball and strike zones

gid_2017_07_19_sdnmlb_colmlb_1 gid_2017_07_19_sdnmlb_colmlb_1 vs. right−handed batters vs. left−handed batters 5 5 4 4 3 3 normedpz normedpz 2 2 1 1 0 0

−2 −1 0 1 2 −2 −1 0 1 2

px px Established Strike Zone: Convex Hull

Let Hl denote the open half-plane bounded by the line l. The convex hull of a set of points P is the set

\ c S = Hl {Hl |Hl ∩P=∅} When P contains the locations of all called strikes, S is the established strike zone. Established Strike Zone: Convex Hull

gid_2017_07_19_sdnmlb_colmlb_1 gid_2017_07_19_sdnmlb_colmlb_1 vs. right−handed batters vs. left−handed batters 5 5 4 4 3 3 normedpz normedpz 2 2 1 1 0 0

−2 −1 0 1 2 −2 −1 0 1 2

px px Established Ball Zone: α-Shape?

2 For α > 0, let Bx,α denote the open ball in R of radius α centered 2 2 at the point x. Given a set of points P ⊂ R , two points p1, p2 ∈ R are α-neighbors if p1 and p2 lie on the boundary of some Bx,α such that Bx,α ∩ P = ∅. The α-shape is the straight line graph formed by drawing line segments between α-neighbors.

I α-neighbors are close together but next to big empty regions.

I The α-shape is the outline of the points P.

I Does not need to be convex or even simply-connected.

I For large α, the α-shape is the boundary of the convex hull. Established Ball Zone: α-Shape?

gid_2017_07_19_sdnmlb_colmlb_1 gid_2017_07_19_sdnmlb_colmlb_1 vs. right−handed batters vs. left−handed batters 5 5 4 4 3 3 normedpz normedpz 2 2 1 1 0 0

−2 −1 0 1 2 −2 −1 0 1 2

px px Established Ball Zone: α-Hull

2 For α > 0, let Bx,α denote the open ball in R of radius α centered at the point x. Given a set of points P ⊂ R2, the α-hull of P is the set \ c X = Bx,α {Bx,α|Bx,α∩P=∅} When P contains the locations of all called balls, X is the established ball zone.

T c I Approaches convex hull S = {Hl |Hl ∩P=∅} Hl as α → ∞. I Intuitively: hole punch.

I May not be simply connected for smallish α. Established Ball Zone: α-Hull

gid_2017_07_19_sdnmlb_colmlb_1 gid_2017_07_19_sdnmlb_colmlb_1 vs. right−handed batters vs. left−handed batters 5 5 4 4 3 3 normedpz normedpz 2 2 1 1 0 0

−2 −1 0 1 2 −2 −1 0 1 2

px px Inconsistency Index Metric

S = established strike zone (convex hull of strikes) X = established ball zone (α-hull of balls)

balls in S + strikes in X inconsistency = total calls

I This is a per game metric.

I Compute separately for right-hand batters and left-hand batters and add. Inconsistency Index Metric

S = established strike zone (convex hull of strikes) X = established ball zone (α-hull of balls)

balls in S + strikes in X inconsistency = total calls

I This is a per game metric.

I Compute separately for right-hand batters and left-hand batters and add. Choosing α

I α too small: Ball zone has lots of components

I α too large: Seems unfair to umpires

I α way too large: Ball zone becomes simply connected

I α = 0.5539: 1/3 rule-book zone width Correlations: observations

Games ZoneSize Accuracy BBrate Krate Incon Games 1.00 -0.05 0.05 -0.03 0.30 -0.02 ZoneSize -0.05 1.00 -0.49 -0.39 0.09 0.34 Accuracy 0.05 -0.49 1.00 -0.01 0.27 -0.65 BBrate -0.03 -0.39 -0.01 1.00 -0.24 0.09 Krate 0.30 0.09 0.27 -0.24 1.00 -0.14 Incon -0.02 0.34 -0.65 0.09 -0.14 1.00

I Walk rate uncorrelated to inconsistency, accuracy.

I Smaller zones tend to be more accurate and consistent.

I Inconsistency is moderately correlated with accuracy. Correlations: observations

I Walk rate uncorrelated to inconsistency, accuracy.

I Smaller zones tend to be more accurate and consistent.

I Inconsistency is moderately correlated with accuracy. Correlations: observations

I Walk rate uncorrelated to inconsistency, accuracy.

I Smaller zones tend to be more accurate and consistent.

I Inconsistency is moderately correlated with accuracy. Accuracy vs. Inconsistency (r = −0.65)

Pat Hoberg●

Eric Cooper●

0.92 Sean Barber Vic Carapazza● ● Ben ●May Chris ●Segal Adam Hamari● Tom Woodring●

Jim ●Wolf John Tumpane●

Alan Porter● Jordan● Baker

Mark Ripperger● Dan Bellino● Chris Guccione● Quinn WolcottLance Barksdale● Todd Tichenor● Mark Wegner ● 0.91 Gabe● Morales Will Little ● ● Stu Scheurwater Manny Gonzalez Phil Cuzzi ● Chris Conroy ● ChadGarySam Whitson HolbrookCederstromTripp Gibson● Brian● III O'Nora ● ● Mark● ● CarlsonD.J. Reyburn● Chad Fairchild ● ● ● Scott ●Barry Mike Everitt Nic Lentz ● ● Mike Muchlinski Tony Randazzo ● Carlos● Torres Jeff Kellogg● Lance ●Barrett ● Bruce Dreckman Ryan Blakney Brian Gorman● ● ● David Rackley● GregBill Gibson Welke James Hoye ● ● Jerry Meals Jim Reynolds ● Jeff Nelson ● ● Alfonso Marquez ● Tim Timmons● Adrian● Johnson● CoryBill Blaser● Miller● Ted BarrettMike● Estabrook Paul Emmel● ● Ed Hickox● Tom Hallion Doug Eddings● ● Angel Hernandez Marty ●Foster 0.90 Ron Kulpa● ● Ramon De● Jesus Dana DeMuth Gerry Davis Marvin Hudson ● ● ● CB Bucknor● Fieldin Culbreth● Laz ●Diaz Paul Nauert● Clint Fagan● Dan Iassogna ● Andy Fletcher● Consensus Rule Book Accuracy Hunter Wendelstedt● Jerry ●Layne

Larry Vanover●

Mike Winters● Joe West● 0.89 Kerwin ●Danley

Rob Drake●

0.03 0.04 0.05 Average Inconsistency Index Principal Component Analysis

Standard deviations (1, .., p=5): [1] 1.43 1.21 0.87 0.66 0.54

Rotation (n x k) = (5 x 5): PC1 PC2 PC3 PC4 PC5 ZoneSize 0.495 -0.43 0.101 -0.642 0.39 Accuracy -0.626 -0.13 0.025 0.066 0.77 BBrate -0.089 0.68 -0.478 -0.536 0.10 Krate -0.179 -0.56 -0.784 -0.019 -0.21 Incon 0.568 0.16 -0.383 0.543 0.46 Plot of Components

● Andy Fletcher

● Alfonso Marquez 0.2 ● Sean Barber ● Jeff Kellogg ● Tony Randazzo ● ● James Hoye ● Dana DeMuthFieldin Culbreth ● Paul Nauert ● Bill Welke ● ● Pat Hoberg Larry Vanover ● Sam● Jordan Holbrook● D.J. Baker Reyburn ● Clint Fagan ● Jerry Meals ● ● ● ● David Rackley ● CarlosMark Torres Wegner Tom Hallion ● Jim Reynolds ● Paul EmmelMarvin Hudson ● Alan Porter ● Joe West ● ● Marty Foster ● Rob Drake ● Hunter Wendelstedt ● ● CB Bucknor Brian Gorman ● Lance Barksdale● Jerry Layne ● Manny Gonzalez ● John Tumpane ● Mike Winters ● ● Greg Gibson Will Little● Chad Whitson● MarkStu Carlson Scheurwater 0.0 ● ● ● Chris Conroy Bruce Dreckman Tom Woodring ●Jim Wolf ● Quinn Wolcott ● Tim Timmons ● Kerwin Danley ● Dan Bellino ● ● ● Gerry Davis ● ● Gary Cederstrom Brian O'Nora Ron Kulpa ● Gabe Morales ● Mike Everitt Todd Tichenor ● Dan Iassogna ● ● Chad● Fairchild ● ● Angel Hernandez ● Scott Barry Ed Hickox Jeff Nelson● Chris Guccione ● Tripp Gibson III Laz Diaz ● ● Nic Lentz ● Vic Carapazza ● Mark Ripperger Ramon De Jesus ●Ben May Hitter Friendliness

● ● ● ● Chris Segal Phil● Cuzzi Mike MuchlinskiLance Barrett ● Ryan Blakney ● ● Eric Cooper ● Adam Hamari Adrian Johnson Ted Barrett ● Cory Blaser

−0.2 ●Bill Miller ● Mike Estabrook

● Doug Eddings

−0.2 −0.1 0.0 0.1 0.2 Strike Zone Sloppiness References

I Data from MLBAM.

I R packages: alphahull, pitchrx, baseballr I Source code on GitHub: djhunter/inconsistency