[Discussion] Analytics in Hockey

[Bob Long]

1 minute ago, Down By the River said:

Statistical assumptions are not the same as bias. Quite the opposite. Assumptions of ordinary least squares regression are there to ensure that bias does not occur, or, at the very least, does not impact your analysis in ways that create bias. 

 

Not sure how you can say that, all of these models rely on assumptions. You're talking about a specific method, but things like that Dom guys concepts like his points projects rely on assumptions beyond just what method they choose. See e.g. https://theathletic.com/4934305/2023/10/07/nhl-point-projections-reactions-2023-24/

 

[Bob Long]

Here's another example of bias in modelling. Doesn't mean they are wrong (or right) just they do have to make a lot of assumptions. 

 

 

https://evolving-hockey.com/blog/2023-2024-team-point-projections/

 

 

[Alflives]

9 minutes ago, Down By the River said:

Statistical assumptions are not the same as bias. Quite the opposite. Assumptions of ordinary least squares regression are there to ensure that bias does not occur, or, at the very least, does not impact your analysis in ways that create bias. 

Isn’t the data collection method important to the validity of the statistics? fancy stats have garbage input. The original data is based on human bias. 
The players care about plus/minus. 

[Down By the River]

6 minutes ago, Bob Long said:

Here's another example of bias in modelling. Doesn't mean they are wrong (or right) just they do have to make a lot of assumptions. 

 

 

https://evolving-hockey.com/blog/2023-2024-team-point-projections/

 

 

 

Maybe we are using the term "assumptions" differently. In formal stats language, "assumptions" are directly testable, meaning you can directly statistically evaluate whether there is bias in your analysis in particular ways that render your results unreliable (and thus should not be presented). For example, in OLS, there is an assumption that you do not have issues with multicollinearity, meaning the different covariates you included in your model are not highly correlated with one another. You can directly test this assumption to rule it out as an issue. 

 

The way you seem to be using the term is not with respect to stats but with respect to research design/data collection/measurement (e.g., the analytics community making assumptions that they are measuring shot quality, zone entries reliably, etc.). 

[Bob Long]

2 minutes ago, Down By the River said:

 

Maybe we are using the term "assumptions" differently. In formal stats language, "assumptions" are directly testable, meaning you can directly statistically evaluate whether there is bias in your analysis in particular ways that render your results unreliable (and thus should not be presented). For example, in OLS, there is an assumption that you do not have issues with multicollinearity, meaning the different covariates you included in your model are not highly correlated with one another. You can directly test this assumption to rule it out as an issue. 

 

The way you seem to be using the term is not with respect to stats but with respect to research design/data collection/measurement (e.g., the analytics community making assumptions that they are measuring shot quality, zone entries reliably, etc.). 

 

thats fair, I am looking at the modelling assumptions with respect to design. 

 

E.g., in the evolving hockey one above, the model bias is clear, and actually hilarious. 

Edited December 21, 2023 by Bob Long

[Down By the River]

3 minutes ago, Alflives said:

Isn’t the data collection method important to the validity of the statistics? fancy stats have garbage input. The original data is based on human bias. 
The players care about plus/minus. 

100%. Data collection and stats should never be separated in the process of conducting research, but for discussion purposes it is important to be clear about whether you're talking about data collection or stats because these two things can use the same words but those words carry different meaning. 

[Ghostsof1915]

All comes back to Yes, Prime Minister for me:

"So your statistics are facts, and my facts are merely statistics?"

 

I agree with The Hockey Guy, so far the only advanced stats that seem to be worth anything is goals above/below expected for goalies. But once again it doesn't tell you any more than save % and GAA.

 

You can't pigeonhole stats into hockey like you can with baseball. Baseball lives and breathes stats. But even with that. It's doesn't really factor hustle, drive, athletics.

Hockey things change within a game. You can struggle in the 1st period, and the game is basically over, even though you have an amazing 2nd and 3rd, you might have lost but the numbers don't tell the whole story.

Hockey is too dynamic, and it's a team game. It's not like Baseball where you're pitting a pitcher and batter head to head. Teamwork creates a dynamic all it's own.

 

Kudos to the OP for creating this thread. It's not that stats aren't interesting. It's just does it really reflect anything that's happening on the ice?

[elvis15]

38 minutes ago, Ghostsof1915 said:

All comes back to Yes, Prime Minister for me:

"So your statistics are facts, and my facts are merely statistics?"

...

Sort of. Some stats - a team's record for instance - are facts, but other stats are definitely subjective and can't be interpreted without looking at both other stats and the whole.

 

Like this tweet a few days ago:

PDO is known to be a stat that is meant to revert closer to 100 the larger the sample size gets. But to look at the above on its own and think it isn't indicative of each team's play is a big error. The Oilers deserve a lot of their resulting PDO, as do the Canucks, even if they both stats aren't sustainable.

 

The Oilers have 10 wins (2 OT/SO included) and 10 losses since that tweet, and are in 27th place in the league compared to 2nd for the Canucks since that tweet. (Edit: Canucks are 15-8 in that span with 1 win and 1 loss for OT/SO for comparison)

 

But the flip side is discounting stats and relying on the eye test. That's subjective as well, and relies on often small sample sizes and other people's opinions combined with your own scouts. So falling back on the old standard is regressive.

 

The trick is to look at it all. Teams have the capability and staff to do that, and in either the eye test or stats like PDO, those are indicators, not facts. If you see something you like (or don't like) in either, then you check more sources to validate it. You can exhaust both avenues and truly never end up with a fact as a player might not make the next level, not fit with a team/coach/other players, or just not be able to rise to the occasion when it counts.

 

So again, the only ones that count are the regular season record, and eventually, who wins the Cup.

 

Edited December 21, 2023 by elvis15

[DrJockitch]

Will be interesting to see when AI gets applied to statistical modelling where we end up. 
‘Hockey has such a high degree of randomness and so many things that can’t be calculated I think it probably takes a lot more statistical computing power to find true correlations if they exist. 
‘I think you run into too many issues with small sample size and too many variables such as day to day travel, moods, fatigue, injuries.  
What statistical model will tell us the effect of LV strip clubs on a team?

[Ghostsof1915]

 

[Shift-4]