Out of every update I've ever made, this might be the biggest and best yet. I've been working on it for about three weeks, and I'm so excited that it's finally finished!
For a quick overview, my model works by searching my database for similar players to whatever prospect it's projecting. Players were evaluated across multiple metrics such as production (the most important), height, league strength, and global location. Once a list of similar players is found, a cutoff is created, and the players above that point are used to calculate odds for the prospect.
The comp finding process worked well for what I wanted it to do, but some serious updates could be made. With the NHL offseason in its quiet era, I finally got around to it, and the results are exciting.
There were three main changes I made. The first is adding draft position/ranking as a variable to account for development opportunities a prospect will/is given. The second is an adjustment to how much weight each variable has to better match their correlation to model accuracy. The third is improving the cutoff creation process to more accurately include relevant comps.
This article will be broken up into these three parts, with a final section looking at the results. Let's get into it!
Part I - Adding Draft Position/Ranking
It's no secret that draft position plays a role in a player's development. Whether it's the fact that better prospects tend to be drafted higher or higher drafted players are given more development opportunities, it's clear that this information would improve my model's accuracy, which was previously blind to all this knowledge.
The tricky part to me was always how I would implement this. The first thought is to simply use a player's draft position, but then you run into the problem of how to handle D-1 and DY prospects before they're drafted. Then you run into the problem of how to create a draft ranking list to use. Another issue is how to account for the difference in draft pick value.
Here's how I handled all of these issues.
First of all, I will be using draft position in a prospect's D+1 and on. In my testing, draft position was more accurate than draft day rankings in predicting a prospect's NHL success. For players in their draft year, I'll be compiling scout lists to create a consensus scout ranking. This process will be the same as what I'm doing now for my draft board. Those rankings will be input into my database and used in a player's DY projection.
Using draft year ranking isn't just something I'm doing for the current class. Every player's draft year odds, whether they were drafted in 2026 or 2010, are based on their draft day ranking, not draft position. To create these lists, I used the scout lists I have stored in my historical draft boards. I mixed in draft position in these rankings for older years where the number of rankings I could find online was extremely limited.
I only created lists for the 2003 class and later. This means that I do not have odds anymore for any prospects whose draft years were 2002 or earlier. It sucks not to have these players, but I don't think many people were using them. It's also important to note that projections for players that old will always be inaccurate. The model is designed for the current era of prospects and the current NHL. Applying it to 90s/early 2000s prospects would never work well.
Another thing to handle was D-1 prospects. Ideally, I would have a database of draft rankings for prospects in their D-1, but unfortunately, I don't have that. As of now, D-1 odds are created using every variable except draft position. Maybe in the future I'll start building out these rankings to create more accurate odds in a prospect's D-1 year, but right now that isn't the focus and would take a lot of time to build out. I'd rather just get this update out and deal with these odds later on if the demand for it is there.
The final thing to account for is draft pick value. It would not be accurate to compare draft information between prospects linearly. The difference between a prospect ranked 1st and 15th is not the same as the difference between those ranked 100th and 114th. The solution is to compare prospects based on the difference in draft pick value. The curve I'm using for this is the PuckPedia Perri Pick Value Calculator. This results in the same difference in draft slot being worth very different amounts based on where the picks are.
Part II - Adjusting Variable Weights
As I stated earlier, my model uses a few different variables to calculate the similarity between a prospect and their comps. In a little more detail now, here's what they all are.
Production: The most important is production. This is calculated using NHLe and gets adjusted for team strength and a player's relative age to their age group. Each potential comp is given a similarity score based on production alone, and then penalties get applied to that score based on the other variables.
Height: Taller prospects get compared to taller players and shorter prospects likewise to smaller players.
League Strength: Prospects get compared to players from leagues of similar strength rather than much weaker/stronger leagues. This isn't as important a variable as production, draft position, or height, but still is a factor.
Global Location: North American prospects are given more North American comps while European prospects are given more Europeans. Earlier testing found that the model was underestimating North Americans and overestimating Europeans. This variable also isn't as important as production, draft position, or height.
Draft Position [NEW]: See Part I to see how this works.
Previously, these weights were balanced by hand. I did this years ago and never touched it again until this update.
To see how important each variable should be in the comp calculation process, I first built comps and odds for every player in my database using production only. This allowed me to measure the correlation between the model's accuracy and the variables I mentioned earlier. I did this process for each season age (DY, D+1, D+2, etc.) and position to see if certain variables needed to be adjusted as a player aged or if they should be treated for forwards vs defensemen.
Here's what I found, with the non-production variables listed in order of importance.
Height is the variable that correlated with model error the most in my testing. For forwards, the importance decreases with age, but for defensemen, it stays constant.
Draft Position is a very important variable, right up there with height. It's worth the same amount between forwards and defensemen, but as a player ages, the importance decreases, and production becomes more important.
League quality is a simple variable. Doesn't change with age or position.
Global Location is the least important variable but still accounted for model error, meaning it should remain included. The importance decreases with age but is the same between forwards and defensemen.
All of these new weights create a comp calculation process that is much improved from before. The results back it up.
Part III - New Cutoff Procedure
Creating odds is a two-step process. First, you have to get a list of a prospect's most similar comps and their similarity scores. Then, you have to decide which comps to include and which to exclude. That dividing line is the cutoff value.
Previously, I was using an equation based on the average similarity score of the top three comps. This would give me a value where anyone above it would be included and anyone below it wouldn't. After updating the comp creation process, it was clear to me that this methodology was flawed in some cases.
Here are Michael Misa's top comps for his D+1 season, plotted by their rank and similarity score. This chart shows the top twenty most similar players using the new and improved comp calculation methodology.
There are a few very similar players, and then it starts to drop off, and the gaps start to lessen between comps and the next best. This next chart shows the top sixty comps, and the y-axis now properly goes from 0 to 100. Check out how the curve starts to flatten towards the end.
Notice how this looks like a knee? If not, here's a more extreme example. The general pattern is that the knee is stronger with more elite players. Here's what Jack Hughes looked like in his draft year.
This pattern makes sense. For elite prospects, there aren't as many players who will score very highly in similarity compared to average prospects, where the pool of similar players is much larger.
Here's a comparison between Jack Hughes and Yaroslav Bryzgalov (I'm picking Bryzgalov since he was the first player selected in the 4th round this summer, an objectively "average" prospect).
Previously, I was handling the cutoff the same for both these cases, something now improved upon with this update.
The solution is known as the Kneedle Algorithm. The way it works is by creating a line between the best comp and the 40th-ranked comp. Then, it compares the gap between the line and each actual point to find the "knee gap" for each point. The point that has the largest knee gap is labeled the knee point and is the cutoff value.
Here's what this looks like on Jack Hughes with the knee point starred.
The knee point now serves as the new cutoff value. While this algorithm is great for players with strong knees, it doesn't work as well for more linear comp patterns, like Bryzgalov, where it can make comp pools unnecessarily small over the smallest differences.
To solve this, I am only applying the Kneedle algorithm to players where the max knee gap is greater than 3. Otherwise, I will use the old method, which works much better for prospects with a large number of similar comps.
Part IV - Results
Don't just take my word that these changes have improved the model. Take a look at these numbers, measuring the error in my draft rankings for each draft class before and after the update.
Each draft class saw some sort of improvement, significant improvement at that. Typically, when I update the model, the final result is an improvement of 0.5% - 3%. Improving by 12.25% is very significant and a result that I am absolutely thrilled about.
Another way to look at the results is to see the changes it made to today's prospects. Here are the most increased and decreased projections as a result of this update.
The biggest name on here is Matthew Schaefer from the decreased list. His odds went down a little bit due to now being compared to other highly drafted defensemen instead of just crazy productive D+1 guys. There's a decent track record of defensemen going near the top of the draft and not turning into 1st-pair guys. Schaefer's odds are still incredible but now a little more representative of the profile of an early-selected defenseman.
Other than Schaefer, I would say that everybody else is more or less moving towards where the consensus sees them.
Once again, I am very excited to be finally publishing this update. It's taken up a lot of my time recently, and I'm happy to be done with it and putting out a great product as a result.
For members, all membership perks are already updated with this update, including player cards, player rankings, and historical draft rankings. If you're curious what my 2026 Draft Rankings would have looked like with this update, then check out the historical draft board sheet. I don't update the year-specific draft board sheets after the draft. All ranking changes from updates are applied to the historical sheet.
If you read through this entire article, thank you. I hope everybody who uses Watch The Stats enjoys this update and has a great rest of their summer!
- Nick Iacoban