Sep 14, 2020 · 1h 38m · capital-allocators
Annie Duke – How to Decide (Capital Allocators, EP.156)
0:00 / 0:00
gold bands on the timeline = statements, start to end. Hover to read, click to jump. CC turns on captions
In this episode of Capital Allocators, host Ted Seides interviews cognitive psychologist and author Annie Duke on practical frameworks from her book 'How to Decide' to improve judgment under uncertainty. Duke outlines actionable methodologies for probabilistic scenario planning, calibrating decision speed, mitigating cognitive biases through pre-commitments, and restructuring investment committee meetings to eliminate groupthink.
How this conversation actually went
Every chapter scored 0–10 on four independent dynamics. Hover any point for the reasoning behind the score. Ted holds 11.6% of the talking time here. How this is scored →
Ted as informed peer 4.2 Guest teaching 4.8 Guest disagreement 1.3 Ted pushing back 1.3
speaking balance: gold is Ted, purple is the guest (3 minute bins)
Sharpest disagreement ▶ 1:33:20 Frustration with Top Executives Resisting Process Improvement
Annie sharply criticizes high-performing fund managers who swat away process coaching for themselves while demanding it for subordinates, calling out their headswell and refusal to consider missed value.
Hardest push from Ted ▶ 1:21:22 Host Challenges Committee Dynamics with the 'Nudge' ManeuverTed presents a realistic institutional scenario where a CIO uses pre-meeting lobbying as a runaround against an unhelpful committee member, directly challenging Duke's pure decision-science approach.
Biggest teaching moment ▶ 16:05 Dismantling 'I Don't Know' with the Table Weight PromptAnnie counters Ted's claim that assigning probabilities without perfect data is impossible by demonstrating through an interactive estimation that bounded knowledge always exists.
Ted holds their own ▶ 54:50 Ted Tests Category Shifting Against Value Investors Buying GrowthTed demonstrates deep investment expertise by pressing Annie on the fine line between rational model evolution and style drift when a traditional value manager rationalizes buying Google.
the scores for every segment, with the reasoning behind each
| Chapter | Topic | Ted as informed peer | Guest teaching | Guest disagreement | Ted pushing back | Why |
|---|---|---|---|---|---|---|
| Genesis and Practical Purpose of How to Decide | 3 | 3 | 1 | 0 | Ted prompts Annie to explain why she decided to write another book so quickly after her previous bestseller. Annie explains that readers of Thinking in Bets understood the concepts of uncertainty and luck but consistently asked for practical tools on how to actually make decisions in that environment. | |
| Paradox of Experience and the Six-Step Framework | 4 | 5 | 2 | 1 | Ted introduces the six-step framework from chapter four, but Annie gently corrects him by pointing out that the book begins with the paradox of experience and hindsight bias. She explains that experience alone is insufficient for learning and details how forward-looking counterfactual scenario planning forms the basis of the six steps. | |
| Managing Hidden Information and Foundation of Beliefs | 5 | 5 | 1 | 2 | Ted pushes back on the practical difficulty of assigning numerical probabilities to subjective real-world decisions. Annie walks Ted through an interactive estimation exercise regarding the weight of a desk to prove that people almost always possess bounded knowledge rather than total ignorance. | |
| Calibrating Decision Speed and the Happiness Test | 4 | 5 | 1 | 1 | Ted asks how to avoid analysis paralysis and determine when to apply rigorous analysis versus fast shortcuts. Annie introduces the happiness test and uses an interactive scenario about agonizing over ordering chicken versus fish to illustrate low-impact decision calibration. | |
| Evaluating Optionality and Quittuitiveness | 5 | 4 | 1 | 1 | Ted asks about the optionality side of speeding up choices, prompting Annie to explain quittuitiveness and two-way door decisions. Annie connects the ability to quit or run options in parallel to financial hedging and poker fold decisions. | |
| Escaping Analysis Paralysis in Close-Call Decisions | 4 | 6 | 2 | 1 | Ted asks how to know when one has gathered sufficient information to act. Annie explains that when two close options like Paris versus Rome feel impossible to separate, their expected values are abutting and extra research provides diminishing returns. | |
| Sponsor Message: Ridgeline Investment Management Technology | 3 | 4 | 1 | 1 | Following the sponsor read, Ted brings up mental models and the power of negative thinking. Annie uses an analogy from Star Wars and Darth Vader's reaction to overconfident commanders to illustrate the necessity of searching for points of failure. | |
| Mental Contrasting, Premortems, and Separating Luck from Skill | 5 | 5 | 1 | 1 | Ted references Gary Klein's previous appearance on the podcast to ask about practical negative thinking tools. Annie outlines how to conduct premortems by strictly bifurcating potential failure modes into luck factors versus internal decision execution. | |
| Guarding Competence with Category Decisions and Ulysses Contracts | 6 | 5 | 1 | 3 | Ted applies the concept of barriers to professional asset managers balancing investment style discipline against opportunism. Annie explains category decisions and circle of competence, warning that evaluating exceptions in real-time invites severe rationalization. | |
| Unmasking Rationalization with the Dr. Evil Game | 6 | 6 | 2 | 3 | Ted challenges Annie on how to distinguish a genuine regime shift from rationalizing drift (e.g. value investors buying Google). Annie introduces the Dr. Evil game and examines NFL 4th-down decision biases to show how individual rationalizations compound undetected. | |
| The Illusion of Group Intelligence and Suppressed Dispersion | 4 | 6 | 1 | 1 | Ted asks how negative thinking principles apply to investment committees and group dynamics. Annie cites academic studies by Zeckhauser and Levy showing that open group deliberations rapidly produce false consensus and suppress dissenting data. | |
| Eliminating Probabilistic Ambiguity and Structuring Pre-Work | 4 | 6 | 1 | 1 | Annie explains how natural language terms like 'real possibility' conceal wide numerical variance across team members ranging from 16% to 81%. She outlines an independent pre-work protocol to capture unbiased numerical forecasts and short rationales before meetings. | |
| Facilitating Productive Meetings Through Convey Versus Convince | 6 | 6 | 2 | 4 | Ted outlines a scenario where a CIO uses psychological runarounds on an unhelpful committee member ('nudge'). Annie pushes back, arguing that constructive friction and 'butt-why' questioning force clarity and elevate collective decision quality. | |
| Annie Duke's Future Plans, Odds on Writing, and Consulting Work | 3 | 3 | 1 | 0 | Ted asks Annie about her upcoming projects and whether she plans to write another book. Annie gives a calibrated 76% probability that she will not write another book and discusses her advisory work on meeting design. | |
| Personal Habits, Tennis Strategy, and Intellectual Nuance | 3 | 4 | 2 | 0 | Ted asks closing personal questions covering Annie's passion for tennis, daily habits, and organizational pet peeves. Annie expresses frustration with false dichotomies in public discourse and overconfident fund executives unwilling to examine their own decision processes. | |
| Reflecting on Leaving Academia and Championing Decision Education | 2 | 4 | 1 | 0 | Ted asks about the biggest mistake in Annie's life and the lessons derived from it. Annie reflects on leaving academia for professional poker by default rather than a conscious decision, highlighting why she co-founded the Alliance for Decision Education. |
Statements from this episode (29)
Duke: Uncertainty and luck untether outcome quality from decision quality
“Uncertainty, luck, hidden information, all of the stuff that makes our judgment so subjective, that untethers the quality of outcomes from the quality of decisions, that I can feel having this influence on the way I make decisions.”
Duke: Individual experience can actively interfere with decision learning
“And the paradox of experience is that experience is certainly necessary in order to become a better decision maker. But it's in no way sufficient. And in fact, the paradox is that any individual experience that you might have can actually interfere with learni…”
Duke: Counterfactual thinking should be applied forward, not backward
“Really, the best way to solve this is to not to try to look back and reconstruct all of that, but to actually do that looking forward as you're entering to new decisions.”
Duke: Decision-makers should never accept pure guessing since zero-knowledge states are rare
“What I try to argue in this book and what I try to give people tools for is that you should never accept that as an answer because there is almost nothing about which you know nothing, number one.”
Duke: Calibrate Decision Speed by the Penalty for Being Wrong
“The way that you figure out when you can speed up is by really thinking broadly, what's the penalty for being less right? And a little bit more wrong.”
Duke: Measure decision impact by the time horizon of a bad outcome
“The longer the time horizon in which the bad outcome is going to have actually had an impact. On whether you are happy, whether you are advancing toward your goals or not, that is going to tell you something about the impact of the decision.”
Duke: People Spend 6–7 Work Weeks Yearly Choosing Food, Clothes, and Shows
“The interesting thing about what to eat in a restaurant, what to wear, what to watch on Netflix, is that I think if you add just those three things together, I think it's something like between six and seven work weeks that we spend per year on those decisions…”
Duke: Rapid execution on repeating choices speeds predictive model accuracy
“So the more often that you're making a similar type decision, you want to go fast because it speeds up your ability to build models that will actually help your decision making going forward in terms of making those things more accurate.”
Duke: Sticking to things does not guarantee success due to survivorship bias
“I might argue that people are successful at things they stuck to, which is actually very different than if you stick to something, you'll be successful.”
Duke: High-impact decisions can be made faster when easily reversible
“But even for higher impact decisions, if it's easy for you to quit and reverse course and get back to another option, then you can actually speed your decision making up simply because there isn't as big a penalty for being wrong.”
Duke: Parsing close-call options is impossible and a waste of time
“All we've determined is that they're close, and even if we could determine it, it's a bad use of our time. Because they are so close that you're probably going to be kind of about the same happiness either way, at least in expected value, and so you should pro…”
Duke: Additional information rarely flips leading decision choices
“Is there a piece of information that I can find out That would actually make it so that some other option became clearly better than the option that I'm considering. And you'd be surprised. Most of the time the answer is no. In those rare cases when the answer…”
Duke: Assuming plans cannot fail impairs success; decision-makers must find vulnerabilities
“Believing that this thing that you've created is impenetrable to failure is not a really great way to make sure that you succeed, but rather that they needed to be combing over the plans as if they were the rebels to try to figure out if they were to fail, how…”
Duke: Positive thinking belongs in goal setting, not route execution
“We don't want to confuse the route planning with the destination planning. You figure out where you want to go, and then when you look at a paper map, it's like the power of positive thinking. Oh, look at these clear roads. I'm going to go on this road, and th…”
Duke: People underestimate luck when imagining their own success
“And we know that when we think about like illusion of control, for example, we think when we're imagining our own success that the role of luck will be less than it actually is.”
Duke: Pre-planning downside responses enables rational action during crises
“So by imagining in advance what your plan of action is, if there's a downturn, if the downside realizes, you're actually going to tend to be acting more rationally. You're going to be prepared, and that's going to allow you to be more nimble rather than to be …”
Duke: Decision-makers perform worst when facing immediate choices due to rationalization
“So when we're actually facing a choice, generally that's when we're at our decision-making worst, because that's when the most rationalization can happen.”
Duke: Investors should pair strict category rules with a secondary updating process
“Is make a category decision, hold that strongly, but then have a secondary process that's going on where you're really trying to think about the world and incorporate information in a rational way that will allow you to maybe alter those categories in the long…”
Duke: Investors should not stray outside core competency without new models or hires
“If you have competencies in a particular type of investing, just straying out of that category is just a really bad idea. If you specialize in REITs, you probably shouldn't be straying out of that unless you have a secondary stream, right? Unless you like hire…”
Duke: The idea that angel investors excel at Series C is absurd
“Like the idea that an angel investor would be great at series C is pretty absurd, except that what we see in the moment is that people feel like they are, because it's very easy to convince yourself when you see something in front of you that's really attracti…”
Duke: Group Decisions Are Not Better but Unjustifiably Boost Confidence
“What generally happens is that the decisions that come out of a group process are not really necessarily better. It's just that you have much more confidence in them because you have this perception that group decision making is higher quality. And that's actu…”
Duke: Harvard Study Showed Hand-Raising Artificially Created Supermajorities
“What they found with the group that was raising their hands in class was that super majorities formed very quickly. In other words, consensus formed. Once people sort of saw consensus start to form, people were sort of going with the consensus. I think partly …”
Duke: Surveyed probability interpretations for 'real possibility' ranged from 16% to 81%
“The biggest spread I ever got was on real possibility, where the range was 16% to 81%, and it wasn't like Three people in the group were like up toward 81 person was at 16. It was actually spread out across them.”
Duke: Qualitative probability terms create false consensus and hide substantive disagreement
“This is just one of these little ways where when we speak to each other, we may think we're agreeing with each other, because we may both agree that the term real possibility applies to the likelihood of a particular future occurring, but it turns out that we …”
Duke: The Goal of Group Meetings Should Be Informing, Not Consensus
“The goal of a really good group meeting is not to agree. And we think that the goal is agreement. We think that everybody has to walk out of that room on the same page. The goal of a great group meeting, rather, should be to inform, which is very different. So…”
Duke: Group Deliberations Waste Time Lingering on Consensus Points
“When you're in a group meeting, a lot of the time is spent on agreement. Oh, you know, Ted, you say something, and then I say, oh, I totally agree with Ted, and let me now say it in my own words, why that's such a smart idea. And then the next person says, yea…”
Duke: Decision-Makers Should Be Grateful for Dissenters and Skeptics
“If the person is in the role of the sort of person who's pushing back on it, You should kind of be grateful that that person exists because it forces you to be thinking about things from different perspectives. And like I said, it forces you to actually be abl…”
Duke: Successful Leaders Resist Process Improvement Because It Devalues Past Wins
“Particularly when people are very successful, it's particularly uncomfortable for them to imagine that there might be a ways that they could really improve their decision process. And I think that the reason is that then there's all of a sudden you open up a c…”