A few years ago, I was talking to a young player who was really excited after a meeting with his coach. He'd been told he'd scored high on a stat the team kept: how often they retained possession whenever he was responsible for dumping in and chasing the puck.
So, I asked the coach about it. He paused, then smiled and said, "I have no idea what you're talking about." He just didn't want to say anything about this "private" statistic. The next day, the player said, "Please don't report that... I didn't realize I wasn't supposed to say anything."
So it was a secret -- until now.
I've always believed NHL teams keep much more detailed player evaluation tools than they let on. Last weekend, the Massachusetts Institute of Technology hosted its seventh annual Sloan Sports Analytics Conference in Boston. Nicknamed "Dorkapalooza" by ESPN's Bill Simmons, it has evolved into one the most important non-game events on the sporting calendar.
One of this year's panels was entitled "Revenge of the Nerds," about "the meteoric rise and dominance of probabilistic thinking using objective data in decision-making across all disciplines."
It featured Moneyball author Michael Lewis, Dallas Mavericks owner Mark Cuban, and Nate Silver, the statistical genius behind FiveThirtyEight.com. Silver's latest triumph was an astonishingly accurate prediction of the electoral vote count in the recent U.S. presidential election.
Looking for the 'Ah-ha!' moment
Four years ago, the Vancouver Canucks were the only NHL team to go to the conference.
"As we like to say, 'We don't know what we don't know,'" said Laurence Gilman, Vancouver's vice-president of hockey operations and assistant general manager. "What we're trying to figure out is what we don't know. This conference helps get us thinking in ways in which we weren't thinking."
Meanwhile, the Edmonton Oilers' Kevin Lowe and Craig MacTavish attended for the first time this year.
"It was thought-provoking," MacTavish said.
This year, Boston, Dallas, Edmonton, Tampa Bay, Vancouver and Washington were there. Technically, Anaheim attended too, although Brian Burke spent more time ripping statistical analysis than praising it. I couldn't stop laughing at his quotes ("Statistics are like a lamp post to a drunk: Useful for support but not for illumination"), picturing how much attendees wanted to strangle him.
"It's not only about analytics, but the critical decision-making process," said Don Fishman, the Capitals' assistant general manager and director of legal affairs, attending for the third time. "That's why I love it...You learn what other leagues and teams are going through to make the decisions they need to make."
"There was a panel talking about football decisions, in-game probabilities," he added. "They were going through plays -- if you have the lead and the ball on your own 40-yard-line with with under five minutes to go, the probabilities may be better to retain possession. [Former New York Jets and Kansas City coach] Herm Edwards was saying, 'Twenty years ago, you'd never think about it.'"
"Everyone wants a black-and-white answer: Will a trade work or not? Nate Silver said analytics won't provide that... You can't do that in sports. But they might let you know if there's a 60 per cent probability whether something will work. So, you can determine if there is a better chance the trade will succeed for you."
"There's a ton of data out there," MacTavish said. "How are you going to use or make it work for you? A little edge in today's game goes a long way. We're all looking for the 'Ah-ha!' moment, but no one's found it yet."
"If you're playing blackjack and you've got 15 while the dealer has a nine, you hit on that," said Stars assistant general manager Frank Provenzano, attending for the second time. "You might get a 10, but you still do it to shift the odds in your favour. I'm looking for the hockey equivalent of that."
"Can we uncover something of value? Something someone else doesn't have?"
Not the most statistic-friendly sport
The biggest problem for the NHL is the sport just doesn't have the statistical bent of others.
"We are third, behind baseball and basketball," Fishman said.
So teams are creating their own. Because there is no consensus, they are notoriously secretive. One thing I believe some teams do is remove "second assists" from players and see how many points are left over. But good luck trying to confirm that.
"We're looking to determine what is random, and what is true performance. Other than that, I'm not going to tell you what we do," Fishman laughed.
"This is a competitive business and sports analytics is untamed territory," Gilman says. "When people were out there discovering a new world, they didn't want anyone else to know what they were doing."
"But believe me when I tell you there are percentage results that allow you to coach and manage your team to hedge bets in certain events."
"Do we have a formula to evaluate players?" Provenzano pauses. "We do... And the question is do we have confidence in it to apply it yet? Not really ... We're working to get models that are more predictive at a higher confidence."
"Each variable has an element of error to it. That starts to add up, so it's full of uncertainty. The Holy Grail is to... find a model that brings you more goals and more wins."
More than just stats
One of the biggest issues with analytics is that its most hardcore followers tend to discount things like "heart" or "clutch performance" because they are not quantifiable. That's a bad idea in hockey, which is so highly dependent on one-on-one battles. Meanwhile, Oakland A's general manager Billy Beane, the hero of Moneyball, blamed his team's playoff failures on "luck." That's a cop-out.
"If you rely solely [on analytics], you do it at your own peril," says MacTavish, who played 1,093 NHL games. "Nobody's figured out a fail-safe method, not in baseball, football, hockey or soccer. But you're foolish not to be [looking at them], either."
Fishman and Gilman are lawyers. Provenzano has a masters' degree in business administration. They are trained to think logically and follow a process. But they aren't automatons. They agree the best way to succeed in the NHL is a marriage of statistical analysis and "feel," for lack of a better term. Provenzano's boss is a Hall of Famer who won three Stanley Cups and scored 564 goals. If anyone should understand what makes a hockey player, it's Joe Nieuwendyk.
Does he listen to analytical advice?
"He's open to it," the assistant says. "Like anyone sitting in [the GM's] chair, he will use it if what you're proposing makes sense."
"Can someone skate? Is he smart? These are things you can't capture numerically. But if he can do those things, then you plug his performance into a model and more good things happen than bad, that helps you."
Teams are trying to apply this to everything from what happens during a game to evaluating potential draftees to valuing free agents. Washington, for example, has a decision to make on Mike Ribeiro.
"You look at statistics, you look at age, you look at probabilities," Fishman says.
He does the contract work for the Capitals and it would be a stunner if every team in the NHL wasn't doing some variation of the same thing. The Oilers, for example, meet monthly with a volunteer "analytics advice group." It's led by Dan Haight, COO of Darkhorse Analytics and managing director for the University of Alberta's School of Business.
"It is amazing, the passion these people have for hockey and the Oilers," MacTavish said. "It's definitely led to philosophical conversations about what we should be doing. A bunch of people talking about different ways of doing things leads to innovation. It's good to surround ourselves with people who are rarely thinking like we are."
"I thought our guys would be at the top echelon of presenters [at the Sloan Conference]."
Edmonton runs a contest on the team website -- "Last Fan Standing." Basically, you choose a game on the schedule and answer five questions about it. The questions are pretty detailed:
"Will there be a penalty call for hooking?" "Will both teams combine to score six or more goals?" "Will the final goal be scored in the third period?"
The more correct answers, the more opportunity to enter prize draws. But is there something extra to this contest?
"You're always looking to flush out some brainpower," MacTavish says.
Back to accessibility links