Two Reasons Why Evaluating a GM Based on His Draft Record is Dangerous and One Solution

I don’t have a ton of time to blog at the moment with finals coming to an end, but just wanted to throw this up quickly with Ray Shero becoming the New Jersey Devils’ new General Manager and the questions about his seemingly poor draft record. Corey Pronman wrote a nice piece a while back about why Shero’s record in particular is underrated, but I wanted to more briefly examine a few more general reasons why I would be weary about being too reliant on such a history or lack of history of success.

1. Small Sample Size.

One of the central themes with regards to analytics in hockey is that we’re trying to maximize sample size in order to get the most accurate possible view of a player or team’s talent. This is no different with regards to drafting. The fact is, a GM can only draft on average seven players per season, meaning that over the course of, say, a five year tenure, that’s only 35 picks. Some may get hurt, some might lose their love for the game, some might develop better than others simply as a result of random variation. It’s very difficult to isolate real success based on 35 or so picks – which is one of the big reasons why drafting also appears to be so random based on studies in just about every sport.

Continue reading

On Drafting for Skill and Trading for Size

Kyle Dubas had the following quote in Elliotte Friedman’s great 30 thoughts columns this week:

“Here’s the way I look at it,” he said. “Right now, we aren’t good enough to be picky about smaller players. We need as many elite players as we can. If we get into playoffs and are too small, or overwhelmed, it’s easier to trade small for size than draft for size and trade for skill.” (bolding my own)

The quote struck me as interesting because it takes a fundamentally different angle on the size debate than the one I personally ascribe to, and I wonder whether it is simply a matter of semantics, or whether there is actually more to this.

My sense was always that size is not easier to trade for than skill – assuming we mean top 6 size and not grinder size – but that the reason you want to draft for skill was simply that skill players have a higher success rate than big players who don’t score as much. You prefer guys who can score over guys with size because once you accumulate enough of them, you can overpay for the big players that have succeeded, and not bear the risk that they may be busts.

Continue reading

On the Chicago Cubs, Draft Inefficiencies, and the Last Mover’s Advantage

061312_epsteinThere’s a scene in Moneyball where Red Sox owner John Henry offers his vacant GM position to Billy Beane. “Anybody who’s not tearing their team down right now and rebuilding it using your model,” he says. “They’re dinosaurs.” The Oakland A’s were the first movers of modern sports analytics. They took a risk, and while there were stumbles along the way, they benefitted as a result. In hockey, it took a decade longer for any kind of true analytic implementation, and we’re still not quite in “tear down and rebuild using your model” territory. So why has it taken so long? I think the answer lies in the hockey world’s view of baseball. NHL executives are drawn to the differences between the two sports rather than their similarities. Yes, baseball is a stop-start game whereas hockey is fluid. And yes, baseball involves more one-on-one matchups and less team play. But beyond that, the games – and the strategies that result in building the best possible teams – are actually quite similar.

Continue reading