How I learned to stop worrying and love the shootout

If you’ve read Jake Sundstrom’s recent research on goalie performance in shootouts, or the recaps of games that go to shootouts, you’re probably aware that a lot of people at Fear the Fin really hate deciding games this way. And I understand why: it makes no sense to use a glorified skills competition to decide games, and awarding a point to the losers encourages teams to play conservatively in close games, which makes the action less exciting for the fans. But do shootouts really matter? That is, leaving aside whether we like them or not, is it important for a team to be good at the shootout?

For me, the question of the shootout’s importance is one that can be evaluated statistically: if we can quantify how good teams are at shootouts, we can determine how many points they’d expect to gain in the standings by winning them. Fortunately, nhl.com provides a ton of data on teams’ shootout performance, going back to the skills competition’s introduction in 2005; you can find each team’s record of shootout wins and losses as well as their season shooting and save percentages. In order to measure a team’s skill in these situations, I added the shooting and save percentages to calculate a version of the PDO statistic; for the purposes of this analysis, let’s called it Shootout PDO or SPDO (you can call it “Speedo” if you don’t care to take it seriously). The table below presents data on each team’s SO performance and SPDO from the past eight NHL seasons (click to enlarge).

A few points to note regarding shootout performance:

• SPDO varies a lot more across teams and across seasons than traditional PDO, and the “heavy regression towards 1000” dynamic seen with PDO probably doesn’t apply here. This isn’t really surprising, insofar as regression to the mean requires sample sizes that don’t really exist when it comes to “shootout seasons”.
• The best SPDO team since the introduction of the shootout, Pittsburgh, had an incredible 1500 SPDO last season, but an awful 733 during the 2005-06 season. Lest you think this is just a function of talent, one of the worst SPDO seasons in the data comes from a decent 2011-12 Sharks squad, who managed just a 626.
• Unsurprisingly, SPDO seems to correlate strongly with shootout winning percentage.
• SPDO performance doesn’t appear to correlate strongly with team success, however. The worst shootout team in these data, Philadelphia, had an 835 SPDO in 2007-08 (a season that included a Conference Finals appearance) and an 827 SPDO during their outstanding 2010-11 campaign. On the other hand, the Avalanche haven’t had an SPDO lower than 1145 over the last three seasons, even though they’ve missed the playoffs in all of them. Similarly, the Islanders have had six seasons with SPDOs over 1000; 2013 was the first one that included a playoff appearance.

In order to study the strength of these associations more carefully, I matched my SPDO data with season point totals, and performed some panel-data linear regressions in R. These analyses are described in the table below.

Some key points from these results. First, the number of shootouts per season explains almost 17.5% of the variation in point totals. On one hand, that’s not all that interesting (i.e., “things that give you points” will tend to correlate pretty well with point totals), but on the other hand, that’s a lot of explanatory power for a single variable. While the number of shootout wins has a greater impact on point totals than shootout appearances, it’s interesting that the R^2 actually goes down. The betas might seem hard to interpret at first (i.e., “why is a shootout win associated with almost three standings points in this stupid analysis?”), but it’s important to remember that regression is an analysis of correlation, not causation. What these results tell us, basically, is that good teams end up in more shootouts; more important than actually winning the shootouts is being good enough that you’re in a lot of them. This is reinforced by the analysis of shootout win percentage; the variable is statistically significant, but it only explains 1.9% of the variation in point totals, so it’s probably irrelevant for practical purposes.

Second, being good in the shootout, as measured by SPDO or its components, is basically meaningless as a predictor of standings performance. SPDO explains just 1.6% of the variation in point totals, and the effect of the variable is tiny. Teams’ shooting percentage in shootouts isn’t even statistically significant as a predictor of point totals, so, you know, don’t sweat the missed shots the next time your team is in one. Save percentage has a stronger effect, but the components model explains just 3.1% of variation in point totals.

What all this means, basically, is that you should just enjoy the shootouts. If your team is in them a lot, it probably means they’re good, and that’s all that matters. A little counterintuitively, this matters more than actually winning the stupid things.