Tetlock: Modeling randomness is a good way to fail as a forecaster
Philip Tetlock · #6 Philip Tetlock: How to See the Future · Dec 8, 2015 · at 35:25
Philip Tetlock discusses problem selection and the limits of forecasting accuracy with host Shane Parrish.
0:00 / 0:25exact quote · 25.6s
720p mp4 · rendered on demand · StarZero watermark
“If you want to be a good forecaster, you don't spend very much time working on roulette wheel type problems. I mean, if, I mean, if you go to, if you visit casinos, you'll find lots of people who think they can detect patterns in roulette wheel spins. And they develop little algorithms even to help them, but what they're doing is they're essentially modeling randomness. So spending a lot of time modeling randomness is a good way not to become a super forecaster.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Philip Tetlock
Tetlock: 50 minutes of debiasing training improves forecasting accuracy by 10%
“We got about, for average forecasters who are randomly assigned to an experimental condition in which they get Kahneman style debiasing exercises the improvement is in the vicinity of 10%. And that's a big effect when you consider that, you know, we're talking…”
Tetlock: Making explicit probabilistic judgments improves forecasting accuracy
“And I think one of the major takeaways from the forecasting tournaments we've been running is that when people make explicit judgments and they're fully self-conscious about what they're doing, they can learn to do it better.”
Tetlock: Geopolitical forecasting relies on 70% skill and 30% luck
“Well, what we find in the ARPA tournament is, is that there certainly is an element of chance in predicting geopolitical and geoeconomic outcomes but the skill luck ratio seems to be about 70 30.”
Tetlock: Superforecasters uniquely believe probability estimation is a cultivatable skill
“They tend to score higher on measures of fluid intelligence. They tend to score higher on measures of active open-mindedness. But if I had to identify one factor that I think best distinguishes super forecasters from other forecasters who are equally intellige…”
Tetlock: Executives resist forecasting tournaments to protect their high-status judgment
“Even though I think we have shown that forecasting tournaments can appreciably improve probability estimates, there are a lot of reasons why organizations don't adopt them. One is the people at the top of the status hierarchy. They're not very enthusiastic. Bo…”
Tetlock: Corporate forecasting compromises accuracy with vague verbiage to avoid embarrassment
“Well, I think it's something you want to consider seriously that when people make forecasts inside organizations, most organizations today, accuracy is only one of the goals that they're pursuing. They're also interested in making forecasts that are going to b…”