Tetlock: Executives resist forecasting tournaments to protect their high-status judgment
Philip Tetlock · #6 Philip Tetlock: How to See the Future · Dec 8, 2015 · at 15:53
Psychologist Philip Tetlock explains institutional and sociological barriers preventing companies and intelligence agencies from adopting empirical forecasting tournaments.
0:00 / 0:30exact quote · 30.3s
720p mp4 · rendered on demand · StarZero watermark
“Even though I think we have shown that forecasting tournaments can appreciably improve probability estimates, there are a lot of reasons why organizations don't adopt them. One is the people at the top of the status hierarchy. They're not very enthusiastic. Bob, who's in the CEO suite is not that enthusiastic about being discovered that Bob in the mailroom is just as good as he is at anticipating trends relevant to the company's future. So you have the status hierarchy problem. People at the top don't want to be a second guess. They don't want their judgment process to be demystified.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Philip Tetlock
Tetlock: 50 minutes of debiasing training improves forecasting accuracy by 10%
“We got about, for average forecasters who are randomly assigned to an experimental condition in which they get Kahneman style debiasing exercises the improvement is in the vicinity of 10%. And that's a big effect when you consider that, you know, we're talking…”
Tetlock: Making explicit probabilistic judgments improves forecasting accuracy
“And I think one of the major takeaways from the forecasting tournaments we've been running is that when people make explicit judgments and they're fully self-conscious about what they're doing, they can learn to do it better.”
Tetlock: Geopolitical forecasting relies on 70% skill and 30% luck
“Well, what we find in the ARPA tournament is, is that there certainly is an element of chance in predicting geopolitical and geoeconomic outcomes but the skill luck ratio seems to be about 70 30.”
Tetlock: Superforecasters uniquely believe probability estimation is a cultivatable skill
“They tend to score higher on measures of fluid intelligence. They tend to score higher on measures of active open-mindedness. But if I had to identify one factor that I think best distinguishes super forecasters from other forecasters who are equally intellige…”
Tetlock: Corporate forecasting compromises accuracy with vague verbiage to avoid embarrassment
“Well, I think it's something you want to consider seriously that when people make forecasts inside organizations, most organizations today, accuracy is only one of the goals that they're pursuing. They're also interested in making forecasts that are going to b…”
Tetlock: Algorithmically extremizing independent forecasts produces more accurate predictions
“Each of you has very different reasons for believing .7. This leaves me to suppose that the answer is probably more extreme than .7, because if each of you knew the reasons the others had, you would probably become more extreme. And that's exactly what the bes…”