Shot Quality 2005-06

Revisiting NHL quality for the 2005-06 Regular Season

Ken Krzywicki – October 2006

Abstract

This study revisits previous analyses regarding NHL shot quality. A logistic regression model using 2005-06 regular season data was constructed to predict the probability of a shot on going in. The data elements available for study during the 2005-06 regular season were more than those in 2003-04. For example, the NHL made available take- and give-aways, hits, missed shots, etc. that were not published in 2003-04. New rules, such as the elimination of the center red line for offside passes, the goalie “forbidden zone,” redrawn zone dimensions and tighter officiating standards made for a different appearing game than in the past. Far more power play opportunities were available the past season and the goals per game were up.

A logistic regression model was chosen to fit the 2005-06 regular season data, with the binary target outcome variable of goal versus . Five predictor variables—distance, rebound, situation, shot after opponent turnover and shot type—remained in the final model. Each shot on goal was then assigned a predicted probability of going in, i.e. shot quality for. One minus this value was the predicted probability of a save (shot quality against). The model fit the data well, as demonstrated below.

Certain inferences about actual performance compared to predicted performance were made and contained herein.

Background

Prior to building this new model, a model built using 2003-04 regular season data1 was examined and found to be rather predictive when applied to 2005-06. However, due to the availability of additional data elements and the changes made to the game, a new model was constructed. This might seem contrary to the reasons given in the paper Playoff Shot Quality – Examining 2003-04 NHL Playoff Shot Quality Using a Regular Season Model,2 for not rebuilding the model based on playoff data where scoring was down. The reason given at the time for not redeveloping the model was that the regular season algorithm fit the playoff data. We still stand by that reasoning, but wish to add the data available had but 389 goals on 4,816 shots. The low of goals would

1 Krzywicki – January 2005 2 Krzywicki – November 2005

2005-06 Regular Season Shot Quality Ken Krzywicki – October 2006 1 have made redevelopment of the model difficult at best; it would have been hard to construct a robust model that did not over fit a training dataset. We have, nonetheless, recently attempted to fit a new model to the 2003-04 playoff data and found this to be the case.3 Clumping of scores around a certain value, lack of a smooth distribution of predicted probabilities and over fitting issues were encountered.

The original 2003-04 model fit the 2005-06 data as well, but for reasons cited above regarding changes in the rules and zone dimensions, as well as the availability of additional data elements, we decided to rebuild the model using current data. While the new model was similar, it availed itself of an extra variable that was not previously obtainable (shot after a turnover) and, as we shall see further on, the trend for short- handed shots was opposite that from 2003-04.

2005-06 Regular Season Data

The data used for the model was collected from the NHL play-by-play (PBP) and game summary (GS) files, which are both available at www.nhl.com. The PBP and GS files were generated by the Real Time Scoring System (RTSS) and provided information on the events that occurred throughout the games. This information formed the building blocks for this study.

Some files were not available and others stopped short of recording the full game; the data did not tie out to the year-end figures published by the League, but any differences were statistically immaterial to this study. We had 7,426 goals on 73,570 shots for this analysis. All figures presented herein were extracted from this data.

Methodology

A binary target variable was created with a value of 1 for a goal and 0 for a save. The data was randomly split 75%/25% for model training, or development, and validation. Potential predictor variables were classed, or “binned” and considered for model inclusion. Variables considered for the model included:

3 Bootstrapping techniques were not attempted.

2005-06 Regular Season Shot Quality Ken Krzywicki – October 2006 2

• Distance in feet • Shot type • Situation o Even strength, short-handed or power play • Period • Rebound4 o A shot within two seconds of another shot with a distance less than 25 feet and no intervening event • Own rebound o Rebound shot, as defined above, taken by the same player as prior shot • Shot after face-off win o A shot where the shooting team won a face-off, taken within 5 seconds, with no intervening event • Shot after opponent shot block o A shot after the opponent blocks a shot with no intervening event • Shot after blocking an opponent’s shot o A shot after shooting team blocks an opponent’s shot with no intervening event • Shot after take-away o A shot after shooting team records a take-away with no intervening event • Shot after opponent give-away o A shot after opponent records a give-away with no intervening event • Shot after either type of turnover (defined above) • Shot after a missed shot o A shot after shooting team misses a shot with no intervening event

A logistic regression was constructed on the 75% training sample and validated on the 25% that was held out. This was done to ensure that the model did not over fit the development set. Only statistically significant variables remained in the final model.

4 As defined by Ryder in Shot Quality – A Method for the Study of the Quality of a Hockey Team’s Shots Allowed [January 2004]

2005-06 Regular Season Shot Quality Ken Krzywicki – October 2006 3 Model Results

Five variables from the list above, plus an intercept term, remained in the final model; the others were either too correlated to these or insignificant.

Marginal Variable Range Points Contribution Intercept Add to all records -2.0671 Less than 12 ft 0.5718 12 ft 0.5221 13 - 16 ft 0.4856 17 - 18 ft 0.3464 19 - 21 ft 0.2699 22 - 32 ft 0.0000 Distance 33 - 35 ft -0.4455 0.0484 36 - 37 ft -0.5130 38 - 40 ft -0.7515 41 - 44 ft -0.8876 45 - 52 ft -0.9855 53 - 59 ft -1.0885 60 ft or more -1.2802 Yes 1.3382 Rebound 0.0198 No -0.0743 Even Strength -0.1542 Situation Short-Handed -0.0582 0.0091 Power Play 0.3702 Shot after Yes 0.3917 0.0028 turnover No -0.0428 Wrap or Slap -0.0815 Wrist 0.0127 Shot Type Backhand 0.0227 0.0007 Snap 0.0289 Tip-In 0.1744

Table 1: Model Scorecard, Sorted by Marginal Contribution of Each Variable

Shot quality was defined by the model score, or predicted probability of a goal, using Table 1 as follows:

1 P(GOAL) = . (1) −ƒpoints 1+ e

2005-06 Regular Season Shot Quality Ken Krzywicki – October 2006 4 The marginal contribution5 of the shot type was not very high, but since it did add value to the model, was statistically significant and made sense, it remained. Shot distance and rebound contributed most to the model as evidenced by their relatively higher marginal contributions.

An interesting difference in the point assignments for situation versus the 2003-04 model was for short-handed shots. In 2003-04, short-handed shots had a positive trend, i.e., they were of higher quality. This was attributed to the fact that most short-handed shots on goal were probably on breakaways or odd-man rushes. This likely held true for 2005- 06 as well, but with the introduction of the shootout, goalies started practicing breakaways more often. This might explain the lower quality status of short-handed shots. That said, the point assignment was only -0.0582, which, while significant was not that strong relatively. This was another reason for not simply using the 2003-04 model; we wished to minimize misclassification, which would result from the short- handed shots receiving positive points when that clearly was not the trend during the 2005-06 regular season.

Other variables that were common to both models6 exhibited directionally similar trends.

In order to show the model did not over fit the training data, we wish the Kolmogorov- Smirnov (KS) statistic7 between development and validation datasets to be close and it was—a difference of 1.51 KS points, or 4.6% was observed. Details are shown in Table 2 below:

5 Reduction in max-rescaled R2 when variable removed from model, i.e., full model R2 minus R2 of model without variable in question. 6 All variables except shot after turnover were also in the original model. 7 See Glossary for definition.

2005-06 Regular Season Shot Quality Ken Krzywicki – October 2006 5 KS Report for Training Sample

Pred Probability (%) Totals Cuml % Save Int Rate Cuml % Goal Int Rate Cuml % KS Avg Scr (%) 19.64 100.00 5,560 10.09% 4,044 72.73% 8.16% 1,516 27.27% 27.38% 19.22 28.17 14.35 19.63 5,454 19.98% 4,509 82.67% 17.25% 945 17.33% 44.45% 27.20 16.24 13.09 14.34 5,615 30.17% 4,870 86.73% 27.07% 745 13.27% 57.90% 30.83 13.80 9.04 13.08 5,271 39.73% 4,655 88.31% 36.46% 616 11.69% 69.03% 32.56 11.60 8.18 9.03 5,554 49.81% 5,043 90.80% 46.63% 511 9.20% 78.26% 31.62 8.90 5.54 8.17 5,396 59.60% 5,012 92.88% 56.74% 384 7.12% 85.19% 28.45 6.67 4.47 5.53 5,888 70.28% 5,547 94.21% 67.93% 341 5.79% 91.35% 23.42 5.11 3.53 4.46 5,844 80.89% 5,633 96.39% 79.29% 211 3.61% 95.16% 15.87 3.91 3.10 3.52 5,122 90.18% 4,993 97.48% 89.36% 129 2.52% 97.49% 8.13 3.30 0.00 3.09 5,412 100.00% 5,273 97.43% 100.00% 139 2.57% 100.00% 0.00 2.69 Total 55,116 49,579 89.95% 5,537 10.05% 32.56 10.05

KS Report for Validation Sample

Pred Probability (%) Totals Cuml % Save Int Rate Cuml % Goal Int Rate Cuml % KS Avg Scr (%) 19.96 100.00 1,844 9.99% 1,362 73.86% 8.22% 482 26.14% 25.52% 17.29 28.84 14.38 19.95 1,774 19.61% 1,442 81.29% 16.93% 332 18.71% 43.09% 26.16 16.64 13.09 14.37 1,917 29.99% 1,692 88.26% 27.14% 225 11.74% 55.00% 27.86 13.85 9.04 13.08 1,738 39.41% 1,506 86.65% 36.23% 232 13.35% 67.28% 31.05 11.61 8.18 9.03 1,914 49.78% 1,744 91.12% 46.76% 170 8.88% 76.28% 29.52 8.91 5.54 8.17 1,855 59.84% 1,705 91.91% 57.05% 150 8.09% 84.22% 27.17 6.67 4.52 5.53 1,904 70.15% 1,786 93.80% 67.84% 118 6.20% 90.47% 22.64 5.14 3.54 4.51 1,751 79.64% 1,687 96.34% 78.02% 64 3.66% 93.86% 15.84 4.03 3.10 3.53 1,993 90.44% 1,927 96.69% 89.65% 66 3.31% 97.35% 7.70 3.33 0.00 3.09 1,764 100.00% 1,714 97.17% 100.00% 50 2.83% 100.00% 0.00 2.69 Total 18,454 16,565 89.76% 1,889 10.24% 31.05 10.14

Table 2: Model Training vs. Validation KS Report

The distribution of predicted probabilities, or model “scores,” for the entire dataset is found in Table 3, below:

Pred Probability (%) Totals Cuml % Save Int Rate Cuml % Goal Int Rate Cuml % KS Avg Scr (%) 19.70 100.00 7,286 9.90% 5,304 72.80% 8.02% 1,982 27.20% 26.69% 18.67 28.47 14.35 19.69 7,495 20.09% 6,182 82.48% 17.37% 1,313 17.52% 44.37% 27.01 16.35 13.09 14.34 7,383 30.13% 6,433 87.13% 27.09% 950 12.87% 57.16% 30.07 13.80 9.04 13.08 7,009 39.65% 6,161 87.90% 36.41% 848 12.10% 68.58% 32.18 11.60 8.18 9.03 7,468 49.80% 6,787 90.88% 46.67% 681 9.12% 77.75% 31.09 8.90 5.54 8.17 7,251 59.66% 6,717 92.64% 56.82% 534 7.36% 84.94% 28.12 6.67 4.48 5.53 7,437 69.77% 6,988 93.96% 67.39% 449 6.04% 90.99% 23.60 5.15 3.54 4.47 6,867 79.10% 6,632 96.58% 77.41% 235 3.42% 94.16% 16.74 4.03 3.10 3.53 8,198 90.25% 7,953 97.01% 89.44% 245 2.99% 97.45% 8.02 3.34 0.00 3.09 7,176 100.00% 6,987 97.37% 100.00% 189 2.63% 100.00% 0.00 2.69 Total 73,570 66,144 89.91% 7,426 10.09% 32.18 10.07

Table 3: KS Report for Entire Population

Recall that KS measures only the model’s ability to rank order and separate goals from saves; it does not speak to how well the predicted probabilities fit the actual scoring rates. There are many ways to help assess this fit; the c-statistic and fit chart were examined for this study. The c-statistic is the probability that an observed goal had a higher predicted probability than an observed save. Values less than 0.50 indicate that

2005-06 Regular Season Shot Quality Ken Krzywicki – October 2006 6 the model did no better than randomly assigning a goal to a particular observation. The c-statistic for our model was 0.723.

A fit chart measures the average score (predicted probability) per interval (see Table 3 above) against the actual goal rate in that interval. As shown in Illustration 1, the predicted values fit the actual values rather well; also note the overall average predicted probability of 10.07% compared to the actual goal rate of 10.09%.

30.00

25.00

20.00

t Goal Rate

c 15.00 P Avg Score 10.00

5.00

0.00 9 3 7 3 7 3 8 4 9 0 0 5 4 5 1 0 0 3 6 0 ...... 3 3 4 5 8 9 3 4 9 0

1 1 1 0 ------

1 - - -

0 0 4 8 4 8

- 0 1 5 4 5 1 4 9 5

...... 0 0 3 0 0 3 3 4 5 8 . . . 7 9 3 4 . 1 1 9 1 Score Range

Illustration 1: Model Fit Chart

Offense: Observations & Inferences

Now that we have chosen a suitable model to explain shots on goal, we may make certain observations and inferences regarding actual and predicted performance. The actual shooting percentage for the 2005-06 regular season (excluding empty net goals) was 9.9 and the predicted value (i.e., shot quality) based on the above model was 10.1.

2005-06 Regular Season Shot Quality Ken Krzywicki – October 2006 7 Overall, shooters scored at a slightly lower rate than predicted, given the quality of shots on goal. Two teams took average quality shots of 10.1, fourteen higher and fourteen lower shot quality.8

A unit of measuring performance relative to shot quality used throughout this paper applicable to both offense and defense was the shot quality index, or SQI, given by equation 2 below:

SQI = [Actual / Predicted] x 100. (2)

While we have data for each individual, a subset of those with at least 100 shots on goal was examined and ranked by SQI. This subgroup of 314 players took 50,996 shots, with a shot quality of 10.7, higher than the whole League value of 10.1, and scored at the 11.0 level (SQI of 102.80). Four players had an average shot quality (relative to this subset), 179 had higher and 131 lower. The median SQI was 99.61 and the average was 102.18.

Table 4, below, shows the breakdown of SQI as it relates to performance above, at, or below predictions.

Above SQI > 100 154 49.04% At SQI = 100 3 0.96% Below SQI < 100 157 50.00% Total 314 100.00%

Table 4: SQI Distribution for Players With 100+ Shots on Goal

Many of those outperforming predictions were defensemen—nine out of the top ten and fifteen of the top twenty. It makes sense that blueliners would have much lower shot quality due to taking a majority of their shots from further out (recall that distance contributed most to the model) and fewer opportunities for rebounds (another strong

8 These monikers are all in relation to the total average. Those over average were considered “higher” quality shots, etc.

2005-06 Regular Season Shot Quality Ken Krzywicki – October 2006 8 model variable). That some were able to score at a rate much better than predicted shows their offensive value.

Below find the top twenty players, ranked by SQI, along with some remarks on those 314 skaters with 100 or more shots on goal:

ACTUAL SHOTPCT SQ 9 SKATER TEAM SF GF ACTUAL PRED SQI COMP TOTAL 50,996 5,595 11.0 10.7 102.80

CAMPBELL_51 BUF 105 12 11.4 5.0 228.00Lower SCHNEIDER_23 DET 188 21 11.2 5.9 189.83Lower ZUBOV_56 DAL 141 13 9.2 5.0 184.00Lower BERARD_4 CBJ 126 12 9.5 5.2 182.69Lower BERGERON_47 EDM 143 14 9.8 5.4 181.48Lower BOUCHER_43 DAL 175 17 9.7 5.4 179.63Lower VISNOVSKY_17 LAK 152 17 11.2 6.3 177.78Lower DATSYUK_13 DET 146 28 19.2 11.0 174.55Higher BRISEBOIS_71 COL 107 10 9.3 5.6 166.07Lower PHANEUF_3 CGY 242 20 8.3 5.1 162.75Lower ALFREDSSON_11 OTT 245 43 17.6 10.8 162.96Higher MCCABE_24 TOR 207 19 9.2 5.8 158.62Lower KOVALCHUK_17 ATL 321 50 15.6 10.0 156.00Lower ZIDLICKY_3 NAS 113 12 10.6 6.9 153.62Lower SALO_6 VAN 140 10 7.1 4.6 154.35Lower MICHALEK_4 PHO 100 9 9.0 5.9 152.54Lower PRONGER_44 EDM 155 12 7.7 5.1 150.98Lower FOSTER_26 MIN 124 10 8.1 5.3 152.83Lower ARNOTT_44 DAL 167 32 19.2 12.6 152.38Higher PARRISH_37 NYI 102 24 23.5 15.5 151.61Higher

Table 5: Top 20 SQI (Minimum 100 Shots on Goal), excl. ENG

Brian Campbell of the finished first in this category with a remarkable SQI of 228.00. That is, Campbell’s predicted shooting percentage was 5.0, based on the model, and he finished the year with 12 goals on 105 shots for an actual shooting percentage of 11.4, exceeding predictions by 2.28 times.

Behind Campbell came other defensemen: Scheider, Zubov, Berard, et al—all who found the back of the net at a clip much higher than predicted. Ruslan Selei had the

9 SQ COMP column compares predicted shooting percentage for each player to the predicted value for all players in this subpopulation.

2005-06 Regular Season Shot Quality Ken Krzywicki – October 2006 9 lowest SQI for this group, coming in at 18.75. His shot quality was 4.8, quite lower than the 10.7 average for this group; he only scored one goal on 108 shots (0.9 actual). That said, the Ducks did not pay the big Byelorussian to produce goals, rather to prevent them.

Colorado’s Andrew Brunette had the highest shot quality of this group at 18.3; although he finished the season with an outstanding 18.0 actual shooting percentage, it was still below predictions (SQI of 98.36). Brunette’s SQI ranked 163rd of the 314 skaters in this subpopulation.

Brian Pothier had the lowest shot quality (4.2) and scored five goals on 131 shots (3.8 actual; 90.48 SQI). Petr Prucha of the had the highest actual shooting percentage of 23.6 (30 goals on 127 shots), well above his shot quality of 16.2 (SQI of 145.68), ranking 25th for this group of players.

We now give below some observations of offense examined on a team-wide basis. See Appendix for team-by-team results.

0 3 5 2 0 2 r o F

y t i l a 5 1 u Q

t o h S 0 1 5 0

ANA ATL BOS BUF CAR CBJ CGY CHI COL DAL DET EDM FLA LAK MIN MTL NAS NJD NYI NYR OTT PHI PHO PIT SJS STL TBL TOR VAN WAS

Illustration 2: Box Plot – Shot Quality For Distribution (excl. ENG)

2005-06 Regular Season Shot Quality Ken Krzywicki – October 2006 10 The box plot10 above depicts the distribution of predicted probabilities (i.e., shot quality) for all players on each team.11 As shown, some teams took a majority of high quality shots, while others did not (Florida had the lowest median predicted probability of 5.61; that is, half their shots had a 5.61 percent or less chance of going in and half were above that value).

Dallas: The Stars took mostly lower quality shots on goal—predicted 9.2 compared to the overall average of 10.1—but managed to find the back of the net at a 10.8 rate, for a SQI of 117.39, the highest of any team. This, despite a low median predicted probability of 6.36.

Detroit: The Red Wings took, on a whole, slightly lower quality shots on goal (9.4) and outperformed predictions, coming in at 10.4 shooting percentage (110.64 SQI). Interestingly, their median model score was 5.98 (third lowest in the League). Perhaps playing in a rather weak Central division attributed to their regular season success despite taking half their shots at the 5.98 level or lower.

St. Louis: While the Blues took higher quality shots on average (10.3 predicted) than the League as a whole (10.1), they failed miserably to achieve projected results, scoring at only an 8.1 pace. This translated to a League low SQI of 78.64. The chances to score were there for the Blues, but they could not capitalize on the opportunities and this was reflected by their place in the final standings—30th overall.

San Jose: The Sharks had the highest predicted shooting percentage (11.4), and thus highest shot quality, but fell short of that mark coming in at 10.5 (92.11 SQI). The Sharks also had the highest median predicted probability (8.98), indicating

10 The box plot depicts the 5th, 25th, median, 75th and 95th percentiles. 11 League median = 8.17.

2005-06 Regular Season Shot Quality Ken Krzywicki – October 2006 11 they consistently took quality shots. Interestingly, San Jose “allowed” the easiest shots against (see defense section of this paper) as well.

Ottawa: Ottawa is an example of a team with average shot quality (10.1) that well outperformed predictions. The Senators finished the regular season tied for first in shooting percentage at 11.1, for a SQI of 109.90.

Defense: Observations & Inferences

Just as we called the predicted probability of scoring “shot quality for,” we may also look at shot quality from a defensive point of view—one minus the probability of scoring was the predicted probability of a save, or “shot quality against.”

The overall shot quality from a defensive point of view was .899, compared to an actual of .901 (100.22 SQI). As a team, New Jersey and San Jose had the easiest shot quality against (.907), while the New York Rangers “allowed” the toughest shots (.885)—and somehow still made the playoffs, most likely, as we shall see due to their rookie .

2005-06 Regular Season Shot Quality Ken Krzywicki – October 2006 12 0 . 1 9 . 0 t s n i a g A

y t 8 . i 0 l a u Q

t o h S 7 . 0 6 . 0

ANA ATL BOS BUF CAR CBJ CGY CHI COL DAL DET EDM FLA LAK MIN MTL NAS NJD NYI NYR OTT PHI PHO PIT SJS STL TBL TOR VAN WAS

Illustration 3: Box Plot12 – Shot Quality Against Distribution (excl. ENG)

Ninety-six goaltenders13 were analyzed during the 2005-06 regular season and eight faced average shot quality, while 44 faced easier shots and 44 tougher shots. On a team-wide basis, only one team “allowed” average shot quality against and 16 faced easier shots, while 13 saw tougher shots. A detailed listing of goalies and teams is included in the Appendix.

Atlanta: Although the Thrashers had a goaltending carousel, rookie faced roughly half of their shots against. The shot quality when he was in net was .901, easier than the League average of .899, and he outperformed predictions, stopping shots at the .906 level (100.55 SQI). When the myriad others tended the Thrashers’ goal, the shot quality was tougher, indicating that the team “allowed” higher quality shots against. Did Atlanta, on the playoff bubble, feel

12 League median = .918. 13 A goalie was counted once for each team for which he played. For example, is counted twice—once for the Wild and once for the Oilers.

2005-06 Regular Season Shot Quality Ken Krzywicki – October 2006 13 extra pressure to score more goals in this situation and, thus, ease up on defensive responsibilities?

Calgary: The Flames, with a defensive style reputation, “prevented” tougher shots against (.903 shot quality). When Vezina winner played, the Flames saw a shot quality of .904 and their goalie well outplayed predictions (actual save percentage of .923; 102.10 SQI). When (acquired late in the year) played, the shot quality was tougher than average (.888) and he failed to stop shots at his predicted rate (96.17 SQI). Philip Sauve (dealt to Phoenix) saw average shot quality and came in below predictions. The Flames tied the Wild for the “easiest” median predicted probability of saving a shot at .940. That is, a full half of their shots against were predicted to be stopped at the .940 rate or higher, which speaks highly of Calgary’s defense.

Detroit: The oft-maligned faced slightly tougher shots (.898) and outperformed predictions, stopping them at the .915 level (101.89 SQI). Oddly— or not?—during 2003-04 the Wings “allowed” easier shots for Legace as opposed to starters Dominik Hasek or . In 2005-06, the Wings appeared to have tightened up defensively for Legace’s back-ups— saw .906 shots and faced .900 shots. Perhaps the Wings—consciously or not—have more faith in their goalies with the starter moniker or less in their back-ups.

Florida: The Panthers had the League’s busiest goaltender in . When he tended goal, they “allowed” .904 (easier) shots against; Luongo outplayed predictions, coming in at .914 (101.11 SQI). The Panthers also boasted a .934 median predicted probability of a save (near the top of the League). That they did so poorly might be attributed to their lack of offense—a League low 5.61 median predicted probability of scoring.

2005-06 Regular Season Shot Quality Ken Krzywicki – October 2006 14 Montreal: saw “easier” shots (.905) than average and stopped them at the .929 level (102.65 SQI), well above predictions. However, when Jose Theodore was in net (he faced roughly the same number of shots as Huet), the Canadiens “allowed” average quality shots against, predictions which he failed to meet (98.00 SQI). The Habs finished near the top of the League in median shot quality against with .934, attributed to their defense (perhaps instilled by ); that is, they “prevented,” for the most part, the high quality shots against.

New York Rangers: The Rangers were not known for their defensive prowess and their overall shot quality against of .885 points to this (toughest shots “allowed” in the League). was seemingly abandoned by the defense; shots came at him of the .885 quality. However, the rookie was more than up to the task—he finished the season with a .922 save percentage and SQI of 104.18, tops in the League for goalies facing a meaningful number of shots. Perhaps making all of those tough saves helped propel the Rangers to a playoff berth.

Ottawa: While he was injured during the Olympics, and incurred the scorn of Senators management, Dominik Hasek stood on his head when he played. Just as Detroit did in 2003-04, Ottawa “allowed” tougher shots against when Hasek played (shot quality of .895) and, true to form, he had a remarkable save percentage of .926 (103.46 SQI). When second stringer played, the defense appeared to tighten up, “allowing” easier shots (.900) that the goalie handled admirably at .902 (100.22 SQI). The Senators scored 166 goals14 with Hasek behind them; by contrast, the goal support afforded Emery was 124.15 By having an elite goaltender behind them, they apparently opened up—as evidenced by the higher

14 Due to data limitations, only includes goals where the goalie was actually on the ice (i.e., excludes goals where they were pulled for an ). 15 Even though Hasek played more minutes, the Senators scored 0.064 goals per minute he played compared to 0.057 per minute of play for Emery. Hasek’s goals per minute is understated, as the PBP file for game #234 vs. Florida was missing and the minutes played stat came from the League’s year-end numbers (Emery’s shots against and goals against tied out to League year-end figures).

2005-06 Regular Season Shot Quality Ken Krzywicki – October 2006 15 shot quality against—in order to outscore opponents more. Did the Senators err in letting Hasek go during the summer?

Phoenix: The Coyotes protected their first string goalie by “preventing” tougher shots against, .900, which Curtis Joseph stopped at the .902 rate (100.22 SQI), demonstrating his ability to still play well. When David LeNeveu tended goal, the Coyotes “allowed” average shot quality, which he failed to stop at predicted levels (.886 save percentage; 98.55 SQI). The other back-up net minders, Brian Boucher (traded late in the season) and Philip Sauve (acquired from the Flames) faced tougher shots than average and neither of them met predictions.

Conclusions

With a well-fit, robust model we were able to calculate predicted shooting and save percentages for shots on goal during the 2005-06 regular season. That is, we were able to assign shot quality, both for and against, to each shot. This data, examined at the skater, team and goalie level allowed us to make certain observations and inferences about how well a team (or player) did relative to shot quality.16

As noted here and in the previous study, some teams and individuals took higher quality shots than others. It is the ability to capitalize on these chances that is key to success and player measurement. Conversely, certain goalies faced much tougher shots than others (Henrik Lundqvist comes to mind). What is important is how they performed given their circumstances.

With this model and method of comparing actual to predicted performance, that is shot quality, a team can make accurate evaluations, spot various weaknesses in both shooters and , as well make adjustments to any employed system of play.

16 San Jose, for example, had the highest shot quality for at 11.4 and tied for the easiest shots against (.907), but both for offense and defense they failed to meet predictions. The chances were there, the defense was there but the goals for were not and the goals against were more than they should have been, given the shot quality.

2005-06 Regular Season Shot Quality Ken Krzywicki – October 2006 16 Epilogue

Allow me to drop the formalities of the analysis and write in the first person and less formally.

Why would a model built on regular season data from the “old” NHL work well in the playoffs and in the “new” NHL, too? While the games may have appeared (or actually been) different, the shot quality remained relatively stable (based on the model, at least). I think two main factors explain this: the models were necessarily parsimonious and they were dominated by basically shot distance and rebounds. The distribution of shot distances17 from one season to the next did not materially change (though in 2005-06 there were fewer shots at the 58-plus feet distance), hence the ability of the regular season 2003-04 model to fit the data from the playoffs of that season and the 2005-06 regular season.

25%

Regular Season 2003-04 20% Playoffs 2003-04 Regular Season 2005-06 15%

10%

5%

0%

t t t t t t t t t t f f f f f f f f f f re 0 2 4 6 2 1 6 8 4 7 o 1 1 1 1 2 3 3 3 4 5 m n ------r a 0 3 5 7 3 2 7 9 5 o h 1 1 1 1 2 3 3 3 4 t t f ss 8 e 5 L

Illustration 4: Shot Distance Distribution

Again, at the risk of sounding repetitive, I felt the need to rebuild the model after the 2005-06 regular season—mainly to re-calibrate the weight of the various variables

17 Distance bins shown are the same as those used for the 2003-04 Shot Quality Model.

2005-06 Regular Season Shot Quality Ken Krzywicki – October 2006 17 (some bins were different from the original) as well as to see how the newly available characteristics played out. In this case, turnovers (intuitive enough, but statistically relevant) entered the model. As well, short-handed shots became less dangerous than in the past.

Even the additional Real Time Scoring System variables available for 2005-06, while adding value, did not supplant the main two model drivers—shot distance and rebounds. I am not saying the other variables were unnecessary (I suspect, though have not tested this, using only distance and rebounds would not have yielded a robust, smooth distribution of shot quality).

Additional data elements such as man power (5-on-3, 5-on-4, etc.), odd-man rushes and breakaways, shot placement, place on ice from where the shot originated—all of which are not, but most of which could easily be made available by League—might help add more dimension to the issue of assessing shot quality and help us better understand the dynamics of shots on goal.

That said, I enjoyed working with the limited data available and do find it valuable for assessing shot quality, which I believe more telling than shooting or save percentage alone.

2005-06 Regular Season Shot Quality Ken Krzywicki – October 2006 18 Actual vs. Predicted Shooting Percentages 2005-06 Regular Season

ACTUAL SHOTPCT SQ TEAM SF GF ACTUAL PRED SQI COMP TOTAL 73,392 7,248 9.9 10.1 98.02

ANA 2,575 241 9.4 10.0 94.00Lower ATL 2,512 266 10.6 10.2 103.92Higher BOS 2,506 223 8.9 9.8 90.82Lower BUF 2,499 265 10.6 9.7 109.28Lower CAR 2,518 275 10.9 10.6 102.83Higher CBJ 2,247 212 9.4 10.2 92.16Higher CGY 2,293 210 9.2 9.7 94.85Lower CHI 2,478 207 8.4 8.9 94.38Lower COL 2,458 274 11.1 10.9 101.83Higher DAL 2,334 251 10.8 9.2 117.39Lower DET 2,787 291 10.4 9.4 110.64Lower EDM 2,436 243 10.0 10.1 99.01Average FLA 2,674 224 8.4 8.8 95.45Lower LAK 2,376 237 10.0 10.3 97.09Higher MIN 2,184 216 9.9 9.5 104.21Lower MTL 2,466 237 9.6 10.4 92.31Higher NAS 2,376 248 10.4 10.0 104.00Lower NJD 2,391 229 9.6 9.4 102.13Lower NYI 2,476 216 8.7 9.9 87.88Lower NYR 2,399 238 9.9 11.3 87.61Higher OTT 2,760 306 11.1 10.1 109.90Average PHI 2,540 251 9.9 10.4 95.19Higher PHO 2,312 236 10.2 10.6 96.23Higher PIT 2,297 243 10.6 11.0 96.36Higher SJS 2,479 260 10.5 11.4 92.11Higher STL 2,332 190 8.1 10.3 78.64Higher TBL 2,586 238 9.2 10.4 88.46Higher TOR 2,320 251 10.8 9.6 112.50Lower VAN 2,338 242 10.4 10.3 100.97Higher WAS 2,443 228 9.3 10.0 93.00Lower

• Excludes empty net goals. • SQ COMP column compares predicted shooting percentage for each team to the predicted shooting percentage for all teams.

2005-06 Regular Season Shot Quality Ken Krzywicki – October 2006 i Actual vs. Predicted Save Percentages 2005-06 Regular Season

ACTUAL SVPCT SQ GOAL GOALIE TEAM SA GA ACTUAL PRED SQI COMP SUPPORT TOTAL 73,392 7,248 .901 .899 100.22

BRYZGALOV ANA 730 66 .910 .903 100.78 Easier 72 GIGUERE ANA 1,695 150 .912 .902 101.11 Easier 163 TOTAL ANA 2,425 216 .911 .902 101.00 Easier

BERKHOEL ATL 255 30 .882 .891 98.99 Tougher 22 DUNHAM ATL 336 36 .893 .892 100.11 Tougher 60 GARNETT ATL 634 73 .885 .888 99.66 Tougher 66 LEHTONEN ATL 1,123 106 .906 .901 100.55 Easier 97 SHIELDS ATL 129 19 .853 .888 96.06 Tougher 15 TOTAL ATL 2,477 264 .893 .894 99.89 Tougher

RAYCROFT BOS 824 100 .879 .904 97.23 Easier 67 THOMAS BOS 1,213 101 .917 .906 101.21 Easier 94 TOIVONEN BOS 590 51 .914 .899 101.67 Average 57 TOTAL BOS 2,627 252 .904 .904 100.00 Easier

BIRON BUF 980 93 .905 .904 100.11 Easier 94 MILLER BUF 1,440 124 .914 .906 100.88 Easier 156 NORONEN BUF 77 12 .844 .911 92.65 Easier 8 TOTAL BUF 2,497 229 .908 .905 100.33 Easier

GERBER CAR 1,671 160 .904 .898 100.67 Tougher 179 WARD CAR 774 91 .882 .892 98.88 Tougher 87 TOTAL CAR 2,445 251 .897 .897 100.00 Tougher

DENIS CBJ 1,505 151 .900 .903 99.67 Easier 114 LECLAIRE CBJ 1,084 97 .911 .898 101.45 Tougher 78 PRUSEK CBJ 165 20 .879 .899 97.78 Average 10 TOTAL CBJ 2,754 268 .903 .901 100.22 Easier

BOUCHER CGY 103 15 .854 .888 96.17 Tougher 9 KIPRUSOFF CGY 1,951 151 .923 .904 102.10 Easier 177 SAUVE CGY 202 22 .891 .899 99.11 Average 19 TOTAL CGY 2,256 188 .917 .903 101.55 Easier

2005-06 Regular Season Shot Quality Ken Krzywicki – October 2006 ii

ACTUAL SVPCT SQ GOAL GOALIE TEAM SA GA ACTUAL PRED SQI COMP SUPPORT TOTAL 73,392 7,248 .901 .899 100.22

ANDERSON CHI 757 86 .886 .892 99.33 Tougher 60 CRAWFORD CHI 41 5 .878 .896 97.99 Tougher 6 KHABIBULIN CHI 1,379 157 .886 .895 98.99 Tougher 116 MUNRO CHI 234 25 .893 .906 98.57 Easier 17 TOTAL CHI 2,411 273 .887 .895 99.11 Tougher

AEBISCHER COL 1,233 123 .900 .897 100.33 Tougher 146 BUDAJ COL 864 86 .900 .891 101.01 Tougher 93 KOLESNIK COL 178 20 .888 .889 99.89 Tougher 14 THEODORE COL 133 15 .887 .881 100.68 Tougher 13 TOTAL COL 2,408 244 .899 .893 100.67 Tougher

HEDBERG DAL 473 48 .899 .902 99.67 Easier 59 TURCO DAL 1,623 166 .898 .895 100.34 Tougher 187 TOTAL DAL 2,096 214 .898 .896 100.22 Tougher

HOWARD DET 105 10 .905 .900 100.56 Easier 9 LEGACE DET 1,244 106 .915 .898 101.89 Tougher 171 OSGOOD DET 827 85 .897 .906 99.01 Easier 106 TOTAL DET 2,176 201 .908 .901 100.78 Easier

CONKLIN EDM 360 43 .881 .907 97.13 Easier 43 MARKKANEN EDM 871 105 .879 .899 97.78 Average 105 MORRISON EDM 362 42 .884 .900 98.22 Easier 40 ROLOSON EDM 497 47 .905 .910 99.45 Easier 46 TOTAL EDM 2,090 237 .887 .903 98.23 Easier

LUONGO FLA 2,447 210 .914 .904 101.11 Easier 189 MCLENNAN FLA 361 34 .906 .909 99.67 Easier 31 TOTAL FLA 2,808 244 .913 .905 100.88 Easier

GARON LAK 1,725 182 .894 .894 100.00 Tougher 161 HAUSER LAK 24 6 .750 .886 84.65 Tougher 1 LABARBERA LAK 688 69 .900 .895 100.56 Tougher 67 TOTAL LAK 2,437 257 .895 .894 100.11 Tougher

FERNANDEZ MIN 1,612 130 .919 .906 101.43 Easier 148 HARDING MIN 83 8 .904 .899 100.56 Average 7 ROLOSON MIN 759 68 .910 .901 101.00 Easier 56 TOTAL MIN 2,454 206 .916 .904 101.33 Easier

2005-06 Regular Season Shot Quality Ken Krzywicki – October 2006 iii

ACTUAL SVPCT SQ GOAL GOALIE TEAM SA GA ACTUAL PRED SQI COMP SUPPORT TOTAL 73,392 7,248 .901 .899 100.22

AEBISCHER MTL 240 26 .892 .894 99.78 Tougher 25 DANIS MTL 152 14 .908 .907 100.11 Easier 12 HUET MTL 1,085 77 .929 .905 102.65 Easier 89 THEODORE MTL 1,025 122 .881 .899 98.00 Average 107 TOTAL MTL 2,502 239 .904 .902 100.22 Easier

FINLEY NAS 41 7 .829 .911 91.00 Easier 3 MASON NAS 596 52 .913 .904 101.00 Easier 59 RINNE NAS 40 4 .900 .903 99.67 Easier 5 VOKOUN NAS 1,986 160 .919 .901 102.00 Easier 174 TOTAL NAS 2,663 223 .916 .902 101.55 Easier

BRODEUR NJD 2,105 187 .911 .907 100.44 Easier 200 CLEMMENSEN NJD 295 35 .881 .904 97.46 Easier 26 TOTAL NJD 2,400 222 .908 .907 100.11 Easier

DIPIETRO NYI 1,797 180 .900 .897 100.33 Tougher 163 DUBIELEWICZ NYI 145 15 .897 .898 99.89 Tougher 7 SNOW NYI 595 68 .886 .897 98.77 Tougher 44 TOTAL NYI 2,537 263 .896 .897 99.89 Tougher

HOLT NYR 2 0 1.000 .946 105.71 Easier 1 LUNDQVIST NYR 1,444 113 .922 .885 104.18 Tougher 141 WEEKES NYR 867 91 .895 .886 101.02 Tougher 92 TOTAL NYR 2,313 204 .912 .885 103.05 Tougher

EMERY OTT 1,045 102 .902 .900 100.22 Easier 124 HASEK OTT 1,175 87 .926 .895 103.46 Tougher 166 MORRISON OTT 96 12 .875 .893 97.98 Tougher 11 TOTAL OTT 2,316 201 .913 .897 101.78 Tougher

ESCHE PHI 1,099 113 .897 .903 99.34 Easier 120 NIITTYMAKI PHI 1,266 133 .895 .902 99.22 Easier 124 TOTAL PHI 2,365 246 .896 .903 99.22 Easier

BOUCHER PHO 268 33 .877 .891 98.43 Tougher 16 JOSEPH PHO 1,690 166 .902 .900 100.22 Easier 167 LENEVEU PHO 386 44 .886 .899 98.55 Average 37 SAUVE PHO 128 17 .867 .890 97.42 Tougher 6 TOTAL PHO 2,472 260 .895 .898 99.67 Tougher

2005-06 Regular Season Shot Quality Ken Krzywicki – October 2006 iv

ACTUAL SVPCT SQ GOAL GOALIE TEAM SA GA ACTUAL PRED SQI COMP SUPPORT TOTAL 73,392 7,248 .901 .899 100.22

CARON PIT 733 87 .881 .887 99.32 Tougher 79 FLEURY PIT 1,485 152 .898 .892 100.67 Tougher 131 SABOURIN PIT 14 4 .714 .887 80.50 Tougher 1 THIBAULT PIT 484 60 .876 .904 96.90 Easier 26 TOTAL PIT 2,716 303 .888 .893 99.44 Tougher

NABOKOV SJS 1,160 133 .885 .906 97.68 Easier 116 SCHAEFER SJS 138 11 .920 .917 100.33 Easier 13 TOSKALA SJS 878 87 .901 .907 99.34 Easier 124 TOTAL SJS 2,176 231 .894 .907 98.57 Easier

BACASHIHUA STL 515 52 .899 .897 100.22 Tougher 28 DIVIS STL 231 37 .840 .893 94.06 Tougher 15 LALIME STL 869 103 .881 .892 98.77 Tougher 67 SANFORD STL 884 81 .908 .893 101.68 Tougher 75 TOTAL STL 2,499 273 .891 .893 99.78 Tougher

BURKE TBL 763 80 .895 .894 100.11 Tougher 69 COLEMAN TBL 17 2 .882 .927 95.15 Easier 4 EKLUND TBL 19 3 .842 .868 97.00 Tougher 2 GRAHAME TBL 1,451 161 .889 .903 98.45 Easier 159 TOTAL TBL 2,250 246 .891 .900 99.00 Easier

AUBIN TOR 330 25 .924 .903 102.33 Easier 44 BELFOUR TOR 1,476 159 .892 .895 99.66 Tougher 145 TELLQVIST TOR 697 73 .895 .897 99.78 Tougher 58 TOTAL TOR 2,503 257 .897 .897 100.00 Tougher

AULD VAN 1,927 189 .902 .899 100.33 Average 185 CLOUTIER VAN 331 34 .897 .897 100.00 Tougher 35 NORONEN VAN 77 10 .870 .888 97.97 Tougher 4 OUELLET VAN 113 12 .894 .907 98.57 Easier 8 TOTAL VAN 2,448 245 .900 .899 100.11 Average

CASSIVI WAS 30 4 .867 .909 95.38 Easier 3 JOHNSON WAS 855 81 .905 .896 101.00 Tougher 67 KOLZIG WAS 1,986 206 .896 .902 99.33 Easier 152 TOTAL WAS 2,871 291 .899 .900 99.89 Easier

• Excludes empty net goals. • SQ COMP column compares predicted save percentage for each goalie/team to the total predicted save percentage for all. • GOAL SUPPORT only includes goals for where PBP file list the goalie as on the ice.

2005-06 Regular Season Shot Quality Ken Krzywicki – October 2006 v Glossary

• Author Contact: [email protected]

• Regular Season 2003-04 Paper: Shot Quality Model – a logistic regression approach to assessing NHL shots on goal [Krzywicki – January 2005] can be found at Alan Ryder’s web site: http://www.hockeyanalytics.com/Research_files/Shot_Quality_Krzywicki.pdf

• Playoffs 2003-04 Paper: Playoff Shot Quality Model – examining 2003-04 NHL playoffs shot quality using a regular season model [Krzywicki – November 2005] can be found at Alan Ryder’s web site: http://www.hockeyanalytics.com/Research_files/Playoff_Shot_Quality_2004_Krzywicki.pdf

• Kolmogorov-Smirnov (KS) Report: This is a measure of the model’s effectiveness in rank ordering performance (e.g., goal versus save). The KS statistic is calculated by subtracting the cumulative distribution of one performance category (goal) from another (save). The point where maximum separation occurs is the KS spread. It is desirable to maximize this separation – the higher the KS, the more predictive the score. As a model ages, the KS statistic can be expected to degrade over time. The KS Report can be used to compare the KS statistic from the development (baseline) population to that of another time period or validation sample.

• Acknowledgements: I wish to thank Carol Stoker and Peter Manikowski for proof reading; Denis Rouchouze for assistance with R programming to create the box plots; Alan Ryder and Graeme Johns for their prior work on shot quality and Alan Ryder for posting my papers on his web site: www.hockeyanalytics.com

2005-06 Regular Season Shot Quality Ken Krzywicki – October 2006 vi