I don’t have a ton of time to blog at the moment with finals coming to an end, but just wanted to throw this up quickly with Ray Shero becoming the New Jersey Devils’ new General Manager and the questions about his seemingly poor draft record. Corey Pronman wrote a nice piece a while back about why Shero’s record in particular is underrated, but I wanted to more briefly examine a few more general reasons why I would be weary about being too reliant on such a history or lack of history of success.
1. Small Sample Size.
One of the central themes with regards to analytics in hockey is that we’re trying to maximize sample size in order to get the most accurate possible view of a player or team’s talent. This is no different with regards to drafting. The fact is, a GM can only draft on average seven players per season, meaning that over the course of, say, a five year tenure, that’s only 35 picks. Some may get hurt, some might lose their love for the game, some might develop better than others simply as a result of random variation. It’s very difficult to isolate real success based on 35 or so picks – which is one of the big reasons why drafting also appears to be so random based on studies in just about every sport.
Kyle Dubas had the following quote in Elliotte Friedman’s great 30 thoughts columns this week:
“Here’s the way I look at it,” he said. “Right now, we aren’t good enough to be picky about smaller players. We need as many elite players as we can. If we get into playoffs and are too small, or overwhelmed, it’s easier to trade small for size than draft for size and trade for skill.” (bolding my own)
The quote struck me as interesting because it takes a fundamentally different angle on the size debate than the one I personally ascribe to, and I wonder whether it is simply a matter of semantics, or whether there is actually more to this.
My sense was always that size is not easier to trade for than skill – assuming we mean top 6 size and not grinder size – but that the reason you want to draft for skill was simply that skill players have a higher success rate than big players who don’t score as much. You prefer guys who can score over guys with size because once you accumulate enough of them, you can overpay for the big players that have succeeded, and not bear the risk that they may be busts.
When Peter Thiel, co-founder of PayPal and Palentir and the first outside investor in Facebook, conducts interviews, he always asks one very difficult question.
“What important truth do very few people agree with you on?”
I’ll wait while you struggle to find an answer that suits you individually….no go ahead….okay maybe table that for later. While straightforward, it’s an incredibly difficult question both because most of the knowledge we accumulate – particularly when it comes to conventional education – is widely agreed upon, and because in an interview setting, answering it inherently involves voicing an opinion that the interviewer doesn’t share. It takes courage, and courage is something that Thiel feels is lacking.
On Sunday, the NHL’s website published my long read on the place of analytics in hockey and why they are so critical. As you know if you read it, it contained a lot of material, and good for you if you managed to understand it all. It was the product of months of revising and editing, and in the end a complete section needed to be cut. I made the decision to cut that section – which focussed on evaluating defense – in order to add substance to the other sections, and because I didn’t think it entirely fit. Here is that section:
EVALUATING DEFENSE WITH ANALYTICS
Evaluating defense is an area in which analytical findings have helped to shape perception. Conventional analysis dictates that the best players at preventing goals do so through proper positioning, good stick-work, blocking shots and hitting the opposition. Those things can be important when the other team has the puck, of course, but they don’t paint a full picture of a player’s effectiveness at preventing goals.
Analysts love to credit the Chicago Blackhawks for using analytics to survive the hardship of remaining competitive in a salary cap world. Purists prefer to blame analytics for the failures of the Edmonton Oilers, New Jersey Devils and Toronto Maple Leafs this season. Somewhere in the middle there is the Minnesota Wild, a team that willingly acknowledged its efforts to change from a trapping, dump-and-chase behemoth to a speedy carry-in assault, building on the principles of Eric Tulsky and others, who found that controlled entries on average lead to twice as many shot attempts as purposeful dump-ins.
The Wild are a curious case because last year, they started out as an analytic darling, with some of the best possession numbers in the league. But then some of the team’s core players got hurt, and in order to survive with a roster filled with replacements, the aggressiveness had to be abandoned – or at least put to one side – and the possession numbers, unsurprisingly, tumbled.
Jonathan Willis has a great piece at Bleacher Report today about the Chicago Blackhawks and the tough situation in which they find themselves following long extensions to Patrick Kane and Jonathan Toews, a number of other wealthy contracts, and the fact that the salary cap is unlikely to rise as much as originally hoped. This type of situation is, of course, the result of success, as GM Stan Bowman has had to pay his players high salaries thanks to the free agent demand for players who are well-developed, put in appropriate roles, and showcase their talents in the playoffs.
Sam Page has been doing a great series at SI.com on reconciling the views of the analytics community with conventional opinions on players based on the eye test. Most recently, he took a look at Shea Weber, a polarizing figure among analysts because of his mediocre possession numbers.
There’s an important difference between always taking the middle ground in an argument and recognizing nuance where many find none. Analytics are a case in which it is important to remember, whether it’s with corsi, or PDO, or fighting, or any other issue, that because of the imperfection of our metrics, our understanding of psychological factors at play, and our understanding of just what goes on behind closed doors, that what the numbers tell you isn’t always entirely accurate. This nuance is something that I’ve tried to emphasize with this blog over the past few months, and will continue to push. There isn’t a middle ground just because somebody says there should be…there’s a middle ground because of the number of factors in play that simply haven’t been taken into account by any model we have at our disposal right now.
My story of the day from yesterday turned into a more extended column, so I decided to post it this morning.
There has always been a conflict between the values of analytically inclined hockey people (let’s call them analysts) and old-school minds (purists). I’ve written before about how my experiences have allowed me to gain some insight into both schools of thought, and in many ways fuse them into my views on the sport, but I wanted to bring up a couple of issues with regards to the questions surrounding Michel Therrien and his coaching following an absolute drubbing — both in terms of possession numbers and the score — at the hands of the Calgary Flames. Friend of the blog Andrew Berkshire wrote a great recap of the game and criticized Therrien for the team’s possession struggles, which is justified. But some other analysts tend to simplify the game down to a variety of semi-predictive statistics without considering other circumstances. I wanted to use this situation to share some more general thoughts on the use of analytics.
There’s a show called Modern Family in which a couple adopts a child named Lilly. The show plays up the little girl’s typical curiosity, as at varying points she’ll go into a phase where she just asks “why” continually until her parents get fed up. For some reason I thought of that while reading this piece by the Star Tribune’s awesome Minnesota Wild columnist Mike Russo this morning. Parise has always been one the players I have most admired because I think he blends the old school abilities to work hard, go to the net, block shots, and play in all situations, with the more new school skills at puck handling, entering the offensive zone with control, and most importantly, scoring goals. I may be Canadian, but even I could appreciate how cool his game tying goal in the 2010 Olympic Gold Medal Game was (and let’s just not forget how that game ended).