Tetlock: Good Judgment Project won IARPA's 2011–2015 forecasting tournaments
Philip Tetlock · #6 Philip Tetlock: How to See the Future · Dec 8, 2015 · at 3:13
Philip Tetlock discusses the origins and results of the Good Judgment Project, funded by IARPA to improve intelligence analysis through university competitions.
0:00 / 0:13exact quote · 13.5s
720p mp4 · rendered on demand · StarZero watermark
“The tournaments ran from 2011 to 2015. They ended in June of this year. And the Good Judgment Project I am proud to say, was the winner of those forecasting tournaments.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Philip Tetlock
Tetlock: 50 minutes of debiasing training improves forecasting accuracy by 10%
“We got about, for average forecasters who are randomly assigned to an experimental condition in which they get Kahneman style debiasing exercises the improvement is in the vicinity of 10%. And that's a big effect when you consider that, you know, we're talking…”
Tetlock: Making explicit probabilistic judgments improves forecasting accuracy
“And I think one of the major takeaways from the forecasting tournaments we've been running is that when people make explicit judgments and they're fully self-conscious about what they're doing, they can learn to do it better.”
Tetlock: Geopolitical forecasting relies on 70% skill and 30% luck
“Well, what we find in the ARPA tournament is, is that there certainly is an element of chance in predicting geopolitical and geoeconomic outcomes but the skill luck ratio seems to be about 70 30.”
Tetlock: Superforecasters uniquely believe probability estimation is a cultivatable skill
“They tend to score higher on measures of fluid intelligence. They tend to score higher on measures of active open-mindedness. But if I had to identify one factor that I think best distinguishes super forecasters from other forecasters who are equally intellige…”
Tetlock: Executives resist forecasting tournaments to protect their high-status judgment
“Even though I think we have shown that forecasting tournaments can appreciably improve probability estimates, there are a lot of reasons why organizations don't adopt them. One is the people at the top of the status hierarchy. They're not very enthusiastic. Bo…”
Tetlock: Corporate forecasting compromises accuracy with vague verbiage to avoid embarrassment
“Well, I think it's something you want to consider seriously that when people make forecasts inside organizations, most organizations today, accuracy is only one of the goals that they're pursuing. They're also interested in making forecasts that are going to b…”