Insight
Massey: Research shows job interviews predict interpersonal liking, not job performance
“I mean, we know from years of research on the academic side that job interviews don't predict a whole lot. The main thing they predict is whether I like you and whether we can get along interpersonally. That's not necessarily job-relevant information.”
Assertion Supported
Massey: Adjacent NFL draft picks outperform each other only 52% of the time
“Our 52% number is very much an outside view. It says, look, the base rate here is 52%, and that's across a lot of teams over a lot of years, and so I'm sure about that number.”
Insight
Massey: Intensive qualitative evaluation creates an illusion of predictive power
“These guys who believe in analytics and believe in the trade down philosophy and believe in uncertainty, it's a lot easier to believe in that stuff in January than it is in April, because between January and April, they're studying film. They're having debates…”
Insight
Massey: Correlating opinions drastically degrades group decision information value
“It's one of the simplest things you can do to improve group decision making is to build in more independence between opinions. The guys who do the stats on this will tell you know, it depends on how correlated these guys are, but two opinions, if they're uncor…”
Insight
Massey: Offensive lines drive rushing production, leaving little individual running back variance
“The most important thing is that we've come to appreciate how dependent their performance is on other components, and especially the offensive line. And the simplest way to think about it is you shouldn't think about a rushing yard as being, you know, attribut…”
Assertion Not checkable as stated
Massey: Google was ground zero for evidence-based people analytics
“The people analytics world really blew up in the last, I don't know, 10 years or so. Google was ground zero and the tech world really kind of bought into this. We need to improve HR decision-making by being more evidence-based.”
Opinion
Massey: Football is adopting analytics more slowly than baseball or basketball
“Baseball's been doing this for a long time, but then basketball caught fire, and then football is slowly getting there, and I end up working more with organizations and professional sports than any other industry.”
Prediction Held up
Massey: NFL teams will soon adopt motion tracking for personnel decisions
“Teams don't yet use a lot of motion tracking in football. Personnel decisions. Yet. I mean, they're gonna. It's just around the corner. They're beginning to build the base of it.”
Insight
Massey: Human judgments are more reliable when decomposed into component parts
“Instead of making a holistic judgment, you need to decompose it. So this is something we know from judgment and decision making that if your judgments are more reliable, if you break them down into the component parts.”
Insight
Massey: Most corporate performance evaluation relies on biased five-point supervisor ratings
“What's the modal performance measure in all industries? It's what your supervisor says on some like five point scale. That's the performance measure, which is not going to get us very far, and we're not going to be able to go out of analytics on it, and it's j…”
Prediction Not checkable as stated
Massey: Meeting analytics from video calls will become a major organizational tool
“I know the data aren't yet available. It's a lot of data. So take a lot of computational power to deal with it, but that is here. And that is going to be a thing. And we're going to learn a lot from it.”
Insight
Massey: Delaying information sharing preserves independent evaluation
“So postponing information sharing as far into the process as possible is something we talk a lot about, because eventually those guys are going to sit around at a table and argue about a player, and so everyone's going to know how everyone else feels about it,…”
Opinion
Massey: Investment firms should explicitly reward analysts who dissent
“Actually, the best I would hope would actually reward analysts who disagree with them. That should be part of the reward system is to disagree and poke holes.”
Insight
Massey: People abandon analytical models faster than humans when observing failure
“And what we found when we got into it was it's not that people are averse to models universally. It's that when they see models fail, They're harder on models than they are on people.”
Insight
Massey: Algorithmic buy-in requires giving users a small degree of control
“So it was the act of participating rather than the amount of influence that made the difference, and I think this is very general advice for people who are selling models or analytics. You got to let the users in a little bit. You got to let them kind of put t…”
Insight
Massey: The primary benefit of predictive models is consistency over human noise
“The main benefit of models is that they're more consistent than humans are. There's just so much noise in human judgment that it degrades the whole process and models ringing out that noise.”
Insight
Massey: Unquantified team contributors are under-hired and rarely promoted
“Because those are the kinds of things that don't typically get promoted. They might not be rewarded at all. They're probably under hired, and yet they're really important.”
Insight
Massey: Decision-makers must document predictions to counter hindsight bias
“You have to write things down. We trick ourselves. It is human nature to misremember what we thought once we know the outcome. We think we would have understood that. We think we predicted it right. There's just study after study that show this. And so unless …”
