Shootout Game Theory: Should Shooters Want Goalies Guessing?

Jack Han wrote a cool piece the other day about the shootout and game theory. He had a number of different ideas, but I want to address one in particular.

I believe his point was as follows.

“As a shooter in the shootout, if you are unpredictable, the goalie won’t know what is coming and will play you straight up. If, however, you have one prominent move and a lesser-used secondary option, the goalie is likely to know that and cheat, allowing you to score more often on your secondary option, which overall will increase your effectiveness.”

I want to look at this point within the unrealistic context of an NHL goalie having complete information on the shooter’s true shootout talent, ie their base rate, and the percentage of the time in which he uses a primary move relative to a secondary one.

So let’s say you’re a league average shootout performer with two moves (let’s say a backhand deke and a backhand-forehand deke). When the goalie plays reactionary, you score on 33% of your shots. You can, however, decide to adjust this rate by leading the goalie into guessing by using your primary move significantly more than your secondary move. The goalie, as I mentioned above, knows how much you use each move, just not in which cases you will use which.

In theory, this operates as an efficient market, which means that the goalie will do whatever it takes to maximize his odds of saving the puck, depending on what you do.

If we assume that every time the goalie guesses right, he makes the save, and every time he guesses wrong, you score, then if you use your primary move 80% of the time, the goalie will guess, and you will score on the other 20%. These numbers change depending on your tendencies, as the table below shows.

Screen Shot 2015-08-03 at 6.38.39 PM

In the goalie plays you efficiently, however, these percentages won’t hold up because once your success rate gets to the base rate, he will change to reading you, and the base rate will hold. This is shown in the table below.

Screen Shot 2015-08-03 at 6.38.50 PM

The point here is that as a league-average shootout performer, it doesn’t really matter how you vary your moves, because a smart goalie with the correct information will adapt based on the percentages.

Now the caveat is that we don’t operate in an efficient market, and this data is not available to goalies (or to anybody, because shootouts aren’t iterated/repeated games). There is variance worked into even the data we do have. So it is possible one could take advantage of a goalie in a one-time encounter, or a goalie that doesn’t understand this theory. I’m not sure over the long term that it can be that much of a help though.

A Note on Terminology, Ice Time and Production

The TSN Panel just had a conversation about Matt Beleskey and the kind of production a team can expect from him as a sought-after UFA this summer. Beleskey has been put into the same conversation as David Clarkson in 2013 as a player who will likely be overpaid due to a season in which he scored 22 goals on 15.2% shooting and had a playoffs that raised his profile even more with 0.5 goals per game on 17.8% shooting.

Ferraro made the point that Beleskey will only be a worthwhile signing if he is played in a top line role, with guys like Perry and Getzlaf, and McGuire added that he believes in such a situation the power winger could put up as many as 25 goals. But I think this type of discussion is missing the point. The goal for a general manager, after all, is to maximize team wins and thus team goals (both for and against but in this case we’ll focus on goals for).

Sure, if you put Beleskey in a first line role and give him 18 minutes per game (he averaged 14:29 this year), he’s more likely to put up 20 goals on say 180 shots, which is an 11.1 shooting percentage, something that would seem a lot more “sustainable”. But at what cost?

It makes sense to want to play a net-rushing garbage-goal winger with skilled players to maximize his skill set, but you can’t base your team structure around making a UFA deal you offered look like it paid off. If you find yourself in a position where you HAVE to play a guy in a top line role to make a deal seem worthwhile, you aren’t doing things for the right reasons.

You want to sign players who can play up and down the lineup and have an impact. When you paint yourself into a corner with a guy like Beleskey, you’re likely to be burned. They will inevitably have some regression, they will be bumped down the lineup, and before you know it they will be seen as a bust. Justin Williams, for example, could play on a third line and still have success. So could Joel Ward. So could Michael Frolik. I’m not saying those guys won’t be overpaid today, but at least they would be less likely to end up needing buy-outs in two years.

This is why statistics like points/60 minutes are so much more valuable than just points. As the NBA has begun to figure out, efficiency is crucial. Players are put in vastly different roles, and the ability to produce in the role you’re assigned and within your ice time is more important than simply the ability to produce overall. So obviously overall team goals are what is important, but on a player level you have to maximize goals/minute rather than just goals. The important thing with Beleskey isn’t to maximize his goals in your new system, it’s to maximize his efficiency, and his signing should be judged on his efficiency rather than his absolute output in the future.

AP Hockey Story of the Day: June 16 – FC Barcelona and the Benefits of Culture

Great piece in Harvard Business Review about FC Barcelona, who recently won soccer’s prestigious treble, and the way in which they attract players. In hockey, teams talk about “instilling a culture” or “fostering a winning culture” a lot, but it’s not usually specific beyond that. The Bruins, with their smashmouth play over the past decade are probably the closest recent example of a team searching for a specific kind of player and intimidating opposing teams with a specific identity, and obviously the Broad Street Bullies are an older illustration of this. Barcelona is successful because it is rich, but also because it has always been a team built on skill above all else, so skillful players want to play there, good players want to play there, and because of the success of La Masia, the team’s academy, kids want to play there as well. Coaches and players come and go, but the fans know what to expect, and they feel that something is special to them. It brings the best out in the team, and ultimately leads to success.

It is something that, depending on where I was in a rebuilding cycle, I might look to establish if I were an NHL team executive.

Two Reasons Why Evaluating a GM Based on His Draft Record is Dangerous and One Solution

I don’t have a ton of time to blog at the moment with finals coming to an end, but just wanted to throw this up quickly with Ray Shero becoming the New Jersey Devils’ new General Manager and the questions about his seemingly poor draft record. Corey Pronman wrote a nice piece a while back about why Shero’s record in particular is underrated, but I wanted to more briefly examine a few more general reasons why I would be weary about being too reliant on such a history or lack of history of success.

1. Small Sample Size.

One of the central themes with regards to analytics in hockey is that we’re trying to maximize sample size in order to get the most accurate possible view of a player or team’s talent. This is no different with regards to drafting. The fact is, a GM can only draft on average seven players per season, meaning that over the course of, say, a five year tenure, that’s only 35 picks. Some may get hurt, some might lose their love for the game, some might develop better than others simply as a result of random variation. It’s very difficult to isolate real success based on 35 or so picks – which is one of the big reasons why drafting also appears to be so random based on studies in just about every sport.

2. Delegation

General Managers have a lot on their plate. The fact is, it’s not all trades and draft picks. There’s a lot of paperwork, travel, scouting, meetings, staff and other decision-making that the public doesn’t see on a daily basis. Therefore, one of the big jobs of a GM is delegation. They delegate coaching to a coaching staff, legal and salary cap matter often to a lawyer or staff of cap experts, and they delegate scouting to a scouting staff. Now the GM still has final say generally, and they are held responsible for the actions of their delegates, but that doesn’t mean the decisions of the employees reflect perfectly on the capabilities of the executive. So if you combine that with the small sample size issue we’re confronting – and the fact that it can be difficult for a GM to realize that a scouting staff is underperforming in time to do anything about it – it’s a difficult thing to evaluate.

So How Can We Evaluate GMs or Scouting Directors?

There are a couple of keys here. The first thing a GM must ensure is that his scouting staff embodies his ideology on drafting. In the 21st century, that should include an analytic approach which features a combination of game viewership, statistics tracking, hindsight analysis on past drafts, and a formulaic approach to see which prospects, even based off qualitative characteristics, are most likely to succeed. But even if it’s not all that, in order to evaluate a GM based on their drafting, the GM should be sure they’re getting the information THEY want.

How would I evaluate a scouting staff if I was a GM? Or a GM based on scouting if I was a President or Owner? Simple. Beyond simply observing their process, I would take all of their draft lists and create a scale to evaluate those based on an objective ranking of players, say five years into the future. It’s not perfect, since there’s still luck involved and you need time for it to become relevant – which often an owner doesn’t have in evaluating a GM – but at least it increases the sample size, and uses information and data the public doesn’t have to come to the best possible decision on whether a change needs to be made in terms of scouting and drafting.

On Drafting for Skill and Trading for Size

Kyle Dubas had the following quote in Elliotte Friedman’s great 30 thoughts columns this week:

“Here’s the way I look at it,” he said. “Right now, we aren’t good enough to be picky about smaller players. We need as many elite players as we can. If we get into playoffs and are too small, or overwhelmed, it’s easier to trade small for size than draft for size and trade for skill.” (bolding my own)

The quote struck me as interesting because it takes a fundamentally different angle on the size debate than the one I personally ascribe to, and I wonder whether it is simply a matter of semantics, or whether there is actually more to this.

My sense was always that size is not easier to trade for than skill – assuming we mean top 6 size and not grinder size – but that the reason you want to draft for skill was simply that skill players have a higher success rate than big players who don’t score as much. You prefer guys who can score over guys with size because once you accumulate enough of them, you can overpay for the big players that have succeeded, and not bear the risk that they may be busts.

For example, over give years you could ascribe to plan A or plan B.

Plan A is that you draft ten players over five first/second rounds who you hope will be the next Milan Lucic. Let’s say two of them work out, you then trade one of them (plus prospects) for a smaller skilled player, say a Tyler Seguin-type.

Plan B is that you draft ten players over five first rounds who you hope will be the next Claude Giroux. Let’s say five of them work out (note these probabilities aren’t based on much, but bear with me, maybe they’re not all Claude Giroux but five of them are top 6 scorers). You can then trade two of them for a Lucic, or better, a Nash, and suddenly you have more pieces than you had in Plan A.

In essence, trading for size was actually harder than trading for scoring, but you ended up better off drafting for skill because those players tend to become stars far more often.

DC Hockey Analytics Conference Presentations

Thank you to all who attended the DC Hockey Analytics Conference (#DCHAC) and watched via livestream. While there were some technical difficulties that prevented us from recording all of the presentations, we did manage to salvage a good portion of them. Here they are, as well as all of the slides.

Arik Parnass – Opening Comments & Introduction to Analytics

Slides: Intro to Analytics

Analytics in Journalism Panel (Vogel, Press, Greenberg):

Rob Vollman – History of Statistical Analysis in Hockey:

Slides: History Of Stats In Hockey

Timo Seppa – Applying Analytics and Microstats to NCAA Hockey: An Updated Look:

Slides: Quinnipiac Presentation

Jack Han – Birds of a Feather: Implementing Analytics at the McGill Martlets Hockey Program:

Slides: Birds of a feather

Ryan Stimson – Shot Sequencing: Measuring Quality Possession Through Passing Metrics:

Slides: Pass Sequencing Presentation

Panel Session 1:

Arik Parnass – Playoff Series Score Effects and the Power of Loss Aversion:

Article: Playoff Series Score Effects

Micah Blake McCurdy – Does Ice Time Distribution Drive Score Effects?:

Slides: McCurdy Slides

Moneypuck – Evaluating NHL Draft Prospects: A Historical Cohort Based Approach:

Slides: Evaluating NHL Draft Prospects

Panel Session 2:

Josh Smolow – Skill, Coaching, or Randomness: What Goes Into Open-Play Performance in the Offensive, Defensive, and Neutral Zones:

Slides: Skill, Coaching, or Randomness Presentation

Muneeb Alam – Zone Shift and Finish Data: Value After Leaving the Ice

Slides: https://docs.google.com/presentation/d/115qxEDjoWdsdBX0lph6AabmrC_AhiO2_eQUhTJLASdo/edit?usp=sharing

Sam Ventura – The Highway to WAR: Defining and Calculating the Components for Wins Above Replacement:

Slides: Road to WAR Presentation

Arik Parnass – Closing Comments & Open Questions in Analytics

Slides: Closing Presentation

Zero to One: Why Attachment to Competition Hurts NHL Clubs

When Peter Thiel, co-founder of PayPal and Palentir and the first outside investor in Facebook, conducts interviews, he always asks one very difficult question.

“What important truth do very few people agree with you on?”

I’ll wait while you struggle to find an answer that suits you individually….no go ahead….okay maybe table that for later. While straightforward, it’s an incredibly difficult question both because most of the knowledge we accumulate – particularly when it comes to conventional education – is widely agreed upon, and because in an interview setting, answering it inherently involves voicing an opinion that the interviewer doesn’t share. It takes courage, and courage is something that Thiel feels is lacking.

“There is more genius out there than there is courage,” he said at a talk at Georgetown University Tuesday evening, the second time I had seen the venture capitalist talk (full disclosure, my sister is his executive assistant; it’s not a coincidence).

Now if you know anything about Thiel, you know he’s a controversial figure. He has advocated against the current American education landscape, and is known as an extreme libertarian. You kind of have to have some quirks to be as smart and successful as he is, he himself might say.

But going back to that first question, Thiel has his own answer. He believes that the two values that the American economy holds so dear, Capitalism and competition, are in fact not complements but – at least from the perspective of a business – opposites. We have an attachment to this idea of competition, and it leads people to go out and open restaurants, he jokes of essentially the least successful kind of business out there. But when you’re looking for success, you don’t want competition, you want a monopoly.

It’s something that the decision-makers in sporting organizations often don’t get. Largely composed of former players, the demographic has experienced nothing other than heated competition for most of its life, with the ideas implanted from an early age that hard work will pay off, that everybody will get their due, that it’s not about winning as much as it is about how you play the game. General Managers want to fit in more than they want to stand out. They want to be respected for their comportment more than they want to be admired for their achievements. “You always hope a trade works out for both sides,” they are known to say. They are a group not unlike that which goes to business school, which follows a path already set out for them, never quite sure if they’re doing what they want, or what somebody before them has decided is the norm.

“The word ‘ape’ comes from Shakespearian times,” Thiel mentioned in his talk. “It means both ‘primate’ and ‘to imitate.'”

Fans often accuse decision-makers of a fear of taking risks. It’s often a valid complaint; humans are risk-averse by nature. But I believe there’s another dynamic at play here as well. If I’m a GM and I look at Moneyball, I immediately tear down my entire operation, no matter the sport, and ask how we can rebuild it using Beane’s methods. As John Henry (Arliss Howard) says in the movie, “Anybody who’s not tearing their team down right now and rebuilding it using your model? They’re dinosaurs.” And yet it took more than a decade for each baseball team to take steps towards a buy-in, and every other sport is still lagging behind.

General Managers aren’t just afraid to take risks, they’re afraid to be great. They’re taught to win within the confines of the sport, when in reality it takes a complete break from convention to truly sniff greatness. Teams are now embracing analytics to varying degrees – and as I’ve mentioned a number of times before, analytics and statistics are not synonymous. But hockey is still waiting for somebody to take an aspect of the sport from 0 to 1 rather than from 1 to n, as Thiel would put it. It’s waiting for someone not to reinvent the wheel, but to create something better, something different.

It’s time for some of that courage.

AP Hockey Story of the Day: March 16 – Why Do Many Reasonable People Doubt Science?

For those of you that have been following me for a while, you’ll know that I don’t throw around the term “must-read” every day. I reserve it for an article that I truly feel every person with an interest in hockey, sports, analytics — whatever it may be — has to take a look at. Take my word for it, therefore, that when I classify an article as one of the top 10 I’ve ever read, I’m not exaggerating. I would produce a list if necessary. This recent piece by Joel Achenbach is absolutely in my top 10. It is a great read on the scientific process, but it can be applied to analytics and sports and really any aspect of life.

Here are just a few excerpts from the masterpiece. It’s not too hard to draw the necessary inferences when it comes to hockey analytics.

“Science is not a body of facts,” says geophysicist Marcia McNutt, who once headed the U.S. Geological Survey and is now editor of Science, the prestigious journal. “Science is a method for deciding whether what we choose to believe has a basis in the laws of nature or not.” But that method doesn’t come naturally to most of us. And so we run into trouble, again and again.

(Note: This relates back to what I discussed in my NHL.com article from last month. Analytics are not numbers. Analytics are a process by which one analyzes and scrutinizes data to draw conclusions.)

“Science will find the truth,” Collins says. “It may get it wrong the first time and maybe the second time, but ultimately it will find the truth.”

The idea that hundreds of scientists from all over the world would collaborate on such a vast hoax is laughable—scientists love to debunk one another.

The media would also have you believe that science is full of shocking discoveries made by lone geniuses. Not so. The (boring) truth is that it usually advances incrementally, through the steady accretion of data and insights gathered by many people over many years.

When we argue about [climate change], Kahan says, we’re actually arguing about who we are, what our crowd is. We’re thinking, People like us believe this. People like that do not believe this.

“We’re all in high school. We’ve never left high school,” says Marcia McNutt. “People still have a need to fit in, and that need to fit in is so strong that local values and local opinions are always trumping science. And they will continue to trump science, especially when there is no clear downside to ignoring science.”

(Note: Only now is there starting to be a downside for GMs to ignoring analytics.)

Liz Neeley, who helps train scientists to be better communicators at an organization called Compass, says that people need to hear from believers they can trust, who share their fundamental values.

(Note: This is why people like Kyle Dubas are so important. As smart as analytics consultants are — most of whom have dense professional and educational backgrounds in statistics or engineering — it is people who come from similar backgrounds as GMs and other executives who will convince them of their fallibility and the need to scrutinize decision-making.)

“Everybody should be questioning,” says McNutt. “That’s a hallmark of a scientist. But then they should use the scientific method, or trust people using the scientific method, to decide which way they fall on those questions.” We need to get a lot better at finding answers, because it’s certain the questions won’t be getting any simpler.

(Note: Amen.)

AP Hockey Story of the Day: March 4 – On the Chicago Cubs, Kris Bryant and the Human Element

I’ve written about the Cubs and their growing use of analytics before at this blog, and while I thought I had, it turns out I haven’t explicitly written about the Houston Astros and their extreme uses of analytics – in essence, treating their players as exclusively products and maximizing assets without regard for morale and the slippery slope that can result from that – but others have, so I would direct you here and here if you’re not sure what this paragraph is in reference to.

Anyway, Grantland had a nice piece recently on the Chicago Cubs and the decision they will have to face with regards to their top prospect Kris Bryant. They will need to decide whether a deserving player makes the major league roster right away, or is returned to the minors so that the team can gain a year of contract control down the line. It’s an issue that in hockey hasn’t been too much at the forefront since teams will generally appease their players at the expense of cost control, and sometimes that’s fine. I mentioned briefly in this piece how if I’m the team drafting Connor McDavid he’s on my roster next year, no matter the cost down the line. It’s not worth the black eye it would cause the organization and the potential for McDavid to get upset at management just for a few million dollars of savings for one year in the future.

It’s a nuanced issue because there are borderline situations where it is defensible and even optimal to keep a player in juniors for this reason. But, as the Astros have found out, it can be important to incorporate the human element into any risk/reward and opportunity cost calculations. So if I’m the Cubs, I would think long and hard before making the decision regarding Kris Bryant this March.

Myth of Net Neutrality Would Limit Internet’s Potential

With Net Neutrality the hot topic today, in particular this article in the New York Times, I thought I’d post an op-ed I wrote about the topic for a Media Law class last semester. I can’t say I’ve stayed up to date on how the issue has evolved since the time this was written (November), so if anybody has new information they’d like to contribute, feel free to post in the comments. Here it is:

———————————————————————————

We’ve all seen it; we’ve all been captivated by it; we’ve allowed ourselves not to question it because it’s the American journalistic dream: take boring old news and make it interesting to the average consumer. John Oliver, television’s newest comedy anchor, has been great for American awareness. Unlike his contemporaries, Oliver has managed to not only entertain, but also to incite protest. The foremost example has been Oliver’s rant on Net Neutrality, in which he compelled Internet commenters to send complaints to the Federal Communications Commission, urging it to reconsider legislation that would, according to Oliver, fix a problem that doesn’t exist. And comment they did, as the FCC’s website crashed as a result of a wave of traffic the very next day.

But there was a massive oversight on the part of Oliver and his staff that critically skewed the issue and that led many — including myself, initially — to come to the wrong conclusion regarding the importance of Net Neutrality, and the necessary course of action. The focus needs to be on the market conditions surrounding Internet Service Providers and creating new incentive structures and regulations to optimize use of the internet rather than on preserving an outdated and inconvenient ideal.

Let’s start back in 2009. By then, the myth of web traffic being evenly distributed over thousands of different content providers had become just that. That year, half of all internet traffic stemmed from less than 150 large content providers, and the internet simply couldn’t function in the way it long had; it could no longer be neutral. Large providers like Google, Facebook, and Netflix began making special arrangements with ISPs to bypass the back-and-forth communications that tend to slow down Internet accessibility. In simple terms, they developed their own fast lanes. And that was it. The idea of Net Neutrality that Oliver preached so vehemently for commenters to preserve hasn’t been a reality in half a decade. Nor could it be. Today, those 150 companies providing half the content have shrunk to 30. Without a fast lane, a Netflix or an HBO would never work. There simply isn’t enough bandwidth if big content providers are forced to share with everybody else. The average start-up doesn’t have access to this fast lane, but for the most part the average start-up doesn’t need it. How do I know? Well, start-ups seem to be doing fine for themselves right now, and did anyone — did John Oliver, or any of his viewers — even know that the status quo was so different from what it was five years ago?

There is a serious issue to confront here — because the concern becomes one of escalating power and incentives for ISPs and content providers — but it isn’t about Net Neutrality at all, because Net Neutrality is neither alive nor optimal. The question is how to adjust the system such that those companies we’re all supposed to hate so much — Comcast, Time Warner, Verizon, etc. — are forced to invest in increasing broadband, and are prevented from using the ability to prioritize certain content to their advantage, at the expense of consumers. “We should be asking ourselves how do we create abundance, not how do we manage and allocate scarcity,” says Blair Levin, a former FCC chief of staff. As Oliver points out in his rant, America is currently 31st in the world in download speeds, behind even Estonia, a country that Oliver characterizes as looking like they “still worry about Shrek attacks.”

But before once again painting ISPs as the bad guys with regard to this issue, consider that this is a capitalist country, and for-profit corporations like Comcast are around to make money — and more importantly, to maximize profits for shareholders. Comcast CEO Brian Roberts gets judged upon how much money he makes his company, not whether or not his customers are happy — note that those two ideas aren’t correlated in monopolies, which is one of the big problems with the current environment.

Consider the situation for Verizon, who recently invested $26 billion to provide its Fios service to 20 American cities. Only approximately 40 percent of eligible households signed up for the service, which led to a $1,350 investment from the company in each customer, something that will take years to earn back and is hardly worth it for the provider in an industry in which customers have few other cable and internet options to begin with. The situation certainly wouldn’t be any more favorable if such monopolies were forcibly dissolved; if anything, the incentives for companies like Verizon to invest in better technologies would be even fewer and farther between.

So what is the solution to this mess? It certainly doesn’t involve clinging to the “ideal” of net neutrality, which is neither the current reality nor optimal. ISPs need to be regulated ferociously, but maintaining monopolies may in fact save everybody money. The FCC needs to enact legislation that creates bidding between companies for monopolistic power in each area, with strict conditions for each winning ISP that involve heavy investment in new technologies, limits on the differences between fast and slow lanes, and price maximums for consumers. Will it be easy? No. Nothing in American politics seems to ever be so. Hopefully corporations will see this as a compromise to avoid the limits of full “net neutrality.” But this is necessary so that consumers can continue to use the Internet to its potential, and so that such potential can be expanded evermore.

Oliver was right about one thing: the system isn’t currently broken. But it is likely to run off track if we don’t take extreme action now to change the incentive structure for ISPs, who are monopolies just looking to maximize profits. Maybe a new rant is in order, John, and your “pretties” need to be made to fly once again — this time armed with the whole story.