The Predictive Power of PythagenPuck

by Dirk Hoag on Aug 16, 2007 3:50 PM EDT in Statistical Analysis

Last season I made use of the PythagenPuck formula as outlined by Alan Ryder of HockeyAnalytics.com to make some (fortunately) successful predictions back in November that the Ottawa Senators weren't anywhere near as bad as their sub-.500 record at the time indicated, and that the Boston Bruins weren't exactly the playoff-caliber team they appeared to be according to the standings. The basic idea was that by looking at the ratio between each team's Goals For and Goals Against totals, you get a good idea of where their actual Winning Percentage will end up. The presumption there is that a team's overall ability to score and prevent goals is more constant than the bounces and breaks that can decide individual game results, and over the course of 82 games the "luck" will even out to a large extent.

This summer, I thought I'd revisit that analysis, to answer a few questions.

1. Did I just get lucky, or is PythagenPuck actually a better indicator than actual Winning Percentage of where teams stand during the early part of the season?

2. At what point does it make sense to do this kind of analysis? After 10 games, 20, 30...?

3. What kinds of events during the season helped make large differences in team performance and the Goals For/Against ratio? In other words, if a GM decides that action needs to be taken, what works, and what doesn't?

First, a few ground rules to clarify my use of PythagenPuck. Since it relates to Goals For & Against during regular action, I back out points from Shootout Wins from the Winning Percentage totals. I then took a snapshot at each game in the regular season, and recorded each teams Actual Winning Percentage, as well as Expected Winning Percentage as given by the PythagenPuck formula based on total Goals For & Against up to that point. For those interested in the nitty-gritty details, yes, I adjusted the exponent at each step along the way to account for the league-wide goals-per-game value as it changed during the season.

Finally, I ran correlation figures at each game along the 82-game journey between [Actual Win %(at Game X) & Final Actual Win %], and [Exp Win %(at Game X) & Final Actual Win %], and produced the following chart.

PythagenPuck correlation to actual winning percentage NHL 2006-7

What this demonstrates is that for most of the 2006-07 season, the blue line (Expected Win %) was actually more closely correlated with the final actual Winning Percentage than Actual Win % at that point in time. Only around Game 63 do actual results take over, which makes sense because at Game 82, the values are equal to each other by definition. So at least for last season, Question #1 would appear to be that yes indeed, PythagenPuck does indeed hold some value. I would need to see this play out over a few more seasons before I proclaimed it to be a crystal ball, however.

As to Question #2, we have an even more interesting answer. It looks like around the Game 15 mark, both lines reach a certain point that isn't improved upon for another 25 games or so. In other words, absent any major disruptive issues like a key player injury or coaching change, the picture you have around the 15-20 game mark is most likely a realistic one for the course of the season. If, as a GM, both Actual and Expected Winning Percentage aren't where you'd like them to be by mid-November, then something substantial would need to change, and the longer you wait, the less time you're left with to turn things around. If you have a case of mixed signals, such as last November when the Senators Expected Win % was much higher than Actual results, judgment would have to come into play as to the direction of the team.

To answer Question #3, I'll follow up with a series of posts over the next week looking at some of the major moves up or down that took place through the course of the season, and see how they relate to events such as injuries, coaching changes, and trades.

There are of course, a couple refinements I'd like to make to this analysis, which I may pursue before the start of the regular season. First, I would like to make some adjustment for the quality of opposition and it's impact on the numbers used above, and secondly, I'd like to exclude Empty Net goals from the analysis. Empty Net goals only serve to pad the Goals For of teams that have already pretty much locked a game up, and conversely inflate the Goals Against of teams that are about to lose. When I ran summary-level analysis and excluded Empty Netters, I did get a slightly more accurate result between Expected Win % and Actual, but I haven't had time yet to make that adjustment on my game-by-game figures that drove the analysis above.

Tweet Comment 8 comments | 0 recs |

Do you like this story?

Read More: Ottawa Senators, Boston Bruins

Comments

Now if you can just give me one of these at a player level, i'll clean up in Fantasy Hockey this year.

by Paul Nicholson on Aug 16, 2007 7:56 PM EDT reply actions

Thanks for doing this. I've always wondered how early the GF/GA estimates become useful. 15 game is roughly 18% of the total season so I guess that is when the sample size is large enough to become somewhat robust.

by The Falconer on Aug 17, 2007 12:21 PM EDT reply actions

I forgot to add this: Any thoughts on why the predictive power suffers a modest decline between games 15 and 40?

Perhaps it is just random and if you did this for several seasons the predictive power would be flat between games 15 and 40. I don't know how much time it took but I would be curious to see 2006-07 compared to 2004-05.

by The Falconer on Aug 17, 2007 12:24 PM EDT reply actions

I don't know how much time it took but I would be curious to see 2006-07 compared to 2004-05.

Just a guess, but I'd think that 06-07 would have more games played and less lockouts.

Just poking fun.

by Earl Sleek on Aug 17, 2007 1:24 PM EDT reply actions

I don't know man, those predictions look pretty hit and miss to me. Sure it worked for Ottawa and Boston and more than a few other teams (bang of for my Flames @ 96), but there's a lot of teams that didn't end up close (Jersey finished with 107 compared to the predicted 74, Columbus 73 points instead of 54, to name a couple, and I don't think they made any major changes and/or got that many shutout win points). It's a good indicator to look at goal differentials, and something I always do, but it doesn't seem accurate enough to me to put much stock in it, probably not much different than just looking at a roster on paper and saying "they can't be that bad (or good)?"

by Greg on Aug 17, 2007 3:00 PM EDT reply actions

Greg, this form of analysis isn't meant to be a precise crystal ball that projects where teams will end up at the end of the year, so much as providing a contrary indicator when you have a big spread in actual vs. expected winning percentage at that point in time.

At the end of the year, those two values are going to end up quite close together, and the implication here is that if we see a large spread between the two, it's more likely that the actual winning percentage will adjust, rather than the GF/GA ratio.

And as regards Columbus, they did fire Gerard Gallant and brought in Ken Hitchcock, so I would call that a pretty significant change!

Thanks for the note...

by The Forechecker on Aug 17, 2007 8:05 PM EDT reply actions

Falconer - hopefully I'll add a few more seasons to this analysis over the next couple weeks. It took quite a while to pull this together initially, but it shouldn't take more than a few hours to flesh it out.

Once that's done, I'll bet that the 15-40 game dip is probably an anomaly, but we'll see.

by The Forechecker on Aug 17, 2007 8:07 PM EDT reply actions