I don’t have a ton of time to blog at the moment with finals coming to an end, but just wanted to throw this up quickly with Ray Shero becoming the New Jersey Devils’ new General Manager and the questions about his seemingly poor draft record. Corey Pronman wrote a nice piece a while back about why Shero’s record in particular is underrated, but I wanted to more briefly examine a few more general reasons why I would be weary about being too reliant on such a history or lack of history of success.

1. Small Sample Size.

One of the central themes with regards to analytics in hockey is that we’re trying to maximize sample size in order to get the most accurate possible view of a player or team’s talent. This is no different with regards to drafting. The fact is, a GM can only draft on average seven players per season, meaning that over the course of, say, a five year tenure, that’s only 35 picks. Some may get hurt, some might lose their love for the game, some might develop better than others simply as a result of random variation. It’s very difficult to isolate real success based on 35 or so picks – which is one of the big reasons why drafting also appears to be so random based on studies in just about every sport.

Continue reading →