The Exchanges

Q is certainly one. You mentioned opportunity cost, the difficulty of assessing something that, it's okay, it's fine, it's working, there could be something better, there's an element of inertia. There's clearly sunk cost involved, part of the way up Everest, all the time you spent in that. What are some of those other threads and how do they weave together in the things that create the obstacles to effective quitting?

A Sure. Okay, so let me just list them off, and then you can tell me what you want to talk about. So there's sure loss aversion. Now, I just want to be clear, that's different than loss aversion. Both of these come from Kahneman and Tversky. Kahneman, obviously, Nobel laureate. Loss aversion is not wanting to start things for fear of the losses you might incur later. So let's think about loss aversion as something that stops you from starting. Sure loss aversion, as Kahneman says, talks about, is not wanting to turn a loss on paper into a realized loss. So loss aversion stops us from starting. Sure loss aversion stops us from stopping. Okay, so there's sure loss aversion. Then there's all these things that go under this rubric of kind of escalation of commitment. Escalation of commitment, simply put, is like this, we have the intuition that when we get bad news, that we'll stop doing what we're doing. And actually, we don't stop. When we get bad news, we actually double down. We escalate our commitment to the cause. And what goes under that umbrella is Which is more of a motivational explanation would be a lot of the cognitive explanations. We're going to put sunk cost under that rubric, and the endowment effect. Things we own, we value more than things we don't own. Omission commission bias. Status quo bias. Ambiguity aversion. Mental accounting actually becomes a thread that go…

AI assessment note: “So there's sure loss aversion... escalation of commitment... sunk cost under that rubric”

Q Which is, what's the best decision process you've gone through that led to quitting, regardless of how it turned out?

A Sometime around 2002, I got asked to give a talk about how poker might inform cognitive science to a group of options traders for a fund that was called Parallax Fund founded by Roger Lowe. And I really enjoyed it. I hadn't been in academics at that point for eight years. And I don't know, I think I had kind of forgotten how much I really enjoyed teaching. I hadn't been thinking as explicitly about the connection between the cognitive science that I had been doing when I was an academic. And poker and the way that those could inform each other. So I really loved it. So for the first two years, I just sort of started getting referrals. And then I decided to start building that business. And I very purposely kept two things going at once, which was my poker career and my speaking and consulting career. And I did that until I felt like I could spin off and actually be successful in that speaking and consulting. And then that actually ended up leading me back into academics. So I think that doing that in parallel And starting off with this sort of exploratory line, like, oh, this is an interesting kind of shiny object. Let me explore that without having to quit the other thing, which I think is something that people miss, is that you sometimes don't have to quit. You sometimes can just do some exploring so that you can gain more information before you actually make the switch. And …

AI assessment note: “doing that in parallel And starting off with this sort of exploratory line”

Q Oh, that's right. Yeah. Well, the combination of those two is just an extraordinary story. I can't let you leave without a couple of the new closing questions that I've had since the last time you were on the show. So here we go. Which two people have had the biggest impact on your professional life?

A I would remiss if I didn't say Lila Gleitman, who is for sure number one. She was my advisor in graduate school. She just passed last year, actually. I'm devastated about it, and she's just like an intellectual giant and kind of shaped the way that I think about the world, so for sure she's number one. Then the problem is, well, I've had so many different professional lives that it starts to get really hard, right? So I could say my brother, because he's the one who started me playing poker. I could say Eric Seidel for a similar reason. I can say Michael Mobison, who, to be quite honest, I sort of try to hopefully model myself in some small way after the way that he thinks about the world, his brilliance, just how incredibly nice he is as a human, how intellectually generous he is, and I hope that I can do about 10% of what he thinks in terms of thinking about my professional world. That aside, the person that I would have to name, like, today is Would be Phil Tetlock and Bart Mellors. Cause I started doing a bunch of work with them on forecasting and really starting to think about how do we actually, how do we actually improve our ability to make predictions, which that's all a decision is. Our predictions. And so currently I would say that would be the biggest. And I know that that wasn't too. And then I missed a whole bunch of people, but it's hard for me because first of al…

AI assessment note: “Lila Gleitman, who is for sure number one. She was my advisor in graduate school.”

Q What are some of the tools or techniques that you can use in decision making that are the equivalent of Waze?

A What we want to do is be engaging in something called mental contrasting. So let me explain what that is. This is something that's been very well studied, in particular by a woman named Gabrielle Uttingen, who's at NYU. And what she suggests is that if you can think about the obstacles that might be in your way ahead of time, that you will actually be more likely to succeed. So generally we want to be instantiating This idea of mental contrasting into our own decision making. So I know that you've had Gary Klein on this podcast. I really recommend that people go listen to that episode. And he talks a lot about premortems. That would be the kind of tool that we would be talking about. So just to remind people with a premortem. So I have a goal, let's say, and you imagine at some point in the future, and I failed to reach that goal. And you're asking yourself why, why was it that I didn't reach My goal. Now, notice your goal is still positive. I want to make that very clear, and I also want to make it clear that whether it's a premortem or mental contrast, you're not saying, I think I'm going to fail. You're saying, if I were to fail, Why would that happen so that I may have a higher probability of succeeding? With Gary Klein, you basically, you think about a goal. At some point in the future, I failed to reach that goal. Why did that happen? Now, I suggest in this book that you …

AI assessment note: “What we want to do is be engaging in something called mental contrasting.”

Q You can't think your way out of it, right? This is just hard wiring.

A Yeah. So there's really amazing work by a guy named Dan Kahan. He's at Yale. And what he's shown is people know a lot about confirmation bias. So confirmation bias is specifically you're, you're kind of noticing information that confirms you and you kind of don't pay attention or you don't notice information that is disconfirming. So Dan Kahan has done a lot of work in this sort of larger process called motivated reasoning, that not only do you have confirmation bias, but if I hand you information and force you to read it, so now you can't ignore it. Uh, something that disagrees with you, you will work incredibly hard to discredit it. So if I give you a scientific article that agrees with the belief you have, you'll go, yeah, sounds good. And if I give you a scientific article that disagrees, you'll be like, well, here's all the problems with the methodology and their end was too small. You know, I think they might've been P hacking and, you know, I mean, you will literally just come up with every reason why this isn't true. Okay. So that's really part of motivated reasoning is that our beliefs drive the way that we process the information, which then reinforces the belief. So it becomes this circular pattern. So what Dan Kahan showed is that being smart doesn't help. Because I think intuitively we think, well, I'm a smart person, and now you've told me about motivated reasonin…

AI assessment note: “what Dan Kahan showed is that being smart doesn't help”

Q Did you recognize at the time that your training in psychology would be so relevant for the game itself?

A At the time, I would have to say definitely no. There's language acquisition over here, and then there's Poker over here, and I didn't necessarily obviously see how those two things might relate to each other. Now, in retrospect, I can say it was certainly really helpful, but the first time that I actually thought about it in any kind of really explicit way that there was this really strong relationship between the two things was in 2002 when I got asked to give my first talk. A friend of mine named Eric Seidel, who's an incredible poker player, absolutely one of the legends of the game, Who I actually met when I was 16, long before I was playing poker through Howard. Eric got asked by a friend of his to speak to a retreat of options traders for this friend's hedge fund, and Eric knew him. His name's Roger Lowe. Eric knew him because Eric used to trade on the floor, and so Eric had been involved in the world of finance as well, so his friend said, hey, will you come and talk about what poker might teach us? And I think Eric had a tournament to play or something, and I was taking time off because I was Super pregnant at the time. I think I was about two weeks away from having my fourth child. I wasn't traveling anywhere, and so he said, I can't do it, but my friend Annie, you know, you should have her do it. I had to think about how am I going to explicitly talk about the relati…

AI assessment note: “At the time, I would have to say definitely no.”

Q What's your elevator pitch when you go on the Today Show to talk about quit?

A Yeah. So we think of grit as a virtue and quit as a vice, but that's not true. They're the exact same decision. By definition, if you choose to stick to something, you're choosing not to quit it. And if you choose to quit something, You're choosing not to stick with it. And we need to understand that all the skill is telling the difference between the two, because it's all about context. One is not a virtue and one is not a vice. They're the same decision. And that's the skill that we need to develop is when is it worthwhile to stick to things and when isn't it? And here's the thing that I want people to really, really deeply understand that usually if you quit at the moment that it's objectively correct, It will feel like you're quitting way, way, way too early, and that's the thing that we need to watch out for, because as human beings, we just generally stick to things too long, and life's too short for that.

AI assessment note: “we think of grit as a virtue and quit as a vice, but that's not true.”

Q And what are some of the simple things in each of those categories?

A So for your audience on the luck side, you can hedge. That's literally the simplest way to deal with bad luck that might lie on the horizon, but you want to get a really clear view of those points of luck that might really negatively affect you, because if you can get a hedge on, if you can get a hedge at the right price on, that's going to mitigate the impact of that bad luck occurring, why wouldn't you want to do it? And a lot of times I think that there are hedges that are available to us that we miss. Because we haven't actually gone through these processes of imagining the way that we might fail due to just things kind of not going our way. And we know that when we think about like illusion of control, for example, we think when we're imagining our own success that the role of luck will be less than it actually is. So we really want to think about how to get good hedges on. And then the other thing that you can do that is just really incredibly valuable is think about how would I react if this bad luck were to occur? Because in the moment of experiencing the bad luck, like if I were to lose a hand at poker, I'm not going to be at my decision-making best because I'm going to be pulled into the emotional parts of my brain, which is not where rational thoughts live. So by imagining in advance what your plan of action is, if there's a downturn, if the downside realizes, you're…

AI assessment note: “on the luck side, you can hedge. That's literally the simplest way”

Q A lot of the people listening are leaders of their organizations, and yeah, you could form a discussion group. It's probably a great thing to do. How do they go about communicating with their teams in such a way that improves the quality of the decision of the team?

A Let me give you two of the many, many, many ways that you can do that. The first has to do with really understanding that when you're in a leadership position in particular, there's this problem of contagion that becomes really exacerbated. So contagion is basically this. It's that it's not only that I will try to reason toward my own beliefs, but that if I tell you what my beliefs are, I have infected you with them, and you will now Without knowing it, also try to reason toward my beliefs. So most of us kind of want to be on the same page with the other human beings that we talk to, assuming that you're within tribe. If you're not within tribe, we have the opposite motivation, but that's, then I'm also infecting you in a different way, but let's assume we're in tribe. So given that you want to sort of be on the same page as I am, without knowing it unconsciously, once I've stated my belief, which could be a fact or a prediction, it doesn't matter, you're now going to start to Reason toward that. That's a particularly big problem if you're in a leadership position. So when you're trying to get high fidelity advice from your team, it's really important actually that you keep your beliefs to yourself as you're trying to work a decision and allow them to sort of speak freely. That's kind of number one. And it's actually really hard to do. Think about this for yourselves. Like when…

AI assessment note: “it's really important actually that you keep your beliefs to yourself”

Q And how did you start thinking about that difference between noise and signal?

A So obviously this was something that I was really thinking about when I was thinking about language acquisition, right? Like there's a lot of noise, and there's actual noise because it's language, but what's signal to the child? How are they picking the signal out? So I was kind of thinking about this just sort of trying to solve the problem for myself for about eight years. I wasn't really in any kind of explicit way thinking about it in an academic sense. I was thinking about how do I actually learn in this environment and figure out what's what? And then in 2002, I got asked by a hedge fund to come speak to a group of their traders. They were having a retreat for their traders to come and talk to them about risk. And I said, you know, I don't really want to talk about risk because I realized what I really want to talk about was this noise problem, and that was the first talk that I gave was about this noise problem and what it does. That's really where I ended up really thinking very explicitly about the way that this disconnection, this kind of Pulling apart this uncertain relationship between decision quality and outcome quality really gets in the way in so many ways of good decision making.

AI assessment note: “this was something that I was really thinking about when I was thinking about language acquisition”

Q How do you go about talking about what you do know and you don't know in a way that makes the people on your team more comfortable to follow through the process without the reliance on the outcome?

A So there's a few things that I think are really helpful in terms of getting people to that place. Thing number one is that as you're trying to work through it, really having people work through what are the possible scenarios, and have them start to try to really put probabilities on those. And when they say, well, the probabilities are unknown, you say fine, but we know it's not the whole range of zero to a hundred percent, so let's try to create a range on that. And once you do that, what you tell them is, I understand that I'm not asking you to be right. I'm asking you to start to narrow it down, and think in this probabilistic way, and give me sort of what you think is the best view of the future, And obviously, in order to start narrowing down what those ranges are and the probabilities, then you start to ask the right questions of what are the things that we could know that would allow us to actually get this to be an even narrower range. You start to get people really comfortable with being willing to not say, I think it's 55%. Being willing to say, I think it's somewhere between 40 and 65%. How can we get that better? So that's a good way to communicate that process matters to you, because even saying it's 75%, that's an outcome right there that people feel like they're sort of going And so they're afraid to give you anything at all. I think that's number one. Number tw…

AI assessment note: “what you tell them is, I understand that I'm not asking you to be right.”

Q And I don't really want to talk about the president. However, I thought it might be interesting to ask if you were hired in your business consulting seat to Consult President Trump about how he could improve his decision-making process. What advice would you give him?

A So, honestly, it would be the same advice I would give to anybody, and I think that you can see that here. One is, be curious. Be curious about what other people's opinions are and be open-minded to them. I think that that always makes you into a better decision-maker. That's true for anybody. Don't express things with such certainty. I just don't think that that's good for anybody. I don't think it's good for your listeners, and I don't think it's good for you. Specifically seek out dissent, and that's something really important. Be open-minded to dissent. Don't swat it away. Listen to it. And I think that this is one of the most important things that you can do in order to be a good decision maker.

AI assessment note: “One is, be curious... Don't express things with such certainty... Specifically seek out dissent”

Q which was her third book, but first for general audiences. So If what we're saying bores you, as we all have done, I've been sitting there too, and you take out your iPhone during this, go to your Amazon app, click it, it's fine, we don't mind, just click Thinking of Bets, buy a copy. Annie, what was it about poker that led you to start thinking about decision making?

A Oh gosh, I feel like it's so circular. So, what I was studying when I was in graduate school feels, I think to most people, like it was pretty far afield from poker. What I was actually focused on was how children learn their first language. And what you realize when you get at a poker table is actually this is an incredibly similar problem, which makes you think about the similarities kind of across all learning where there's noise. So if you think about the problem for a child trying to learn a language, there's all sorts of noises. The child has to pick the things that are language out of all the noises. So like crinkling papers don't count, right? So that's kind of hard. And if you can pick those out, now you've got to identify the word boundaries and the sentence boundaries, and that's also really hard. And then once you get a word boundary, And then mother says something like, Dax, what does Dax mean? Well, does it mean the act of pointing? Does it mean the thing that she's pointing to? Does it mean an aspect of what's being pointed to, like the color, like yellow, or soft? Could it be a state of mind, like think? Is it an action? I mean, this is a really hard problem, and it's incredibly noisy, and the feedback isn't great, and kids do it super fast. So that's what I was looking at, was how are there constraints that are sort of built into the way that we come into the w…

AI assessment note: “what you realize when you get at a poker table is actually this is an incredibly similar problem”

Q All right, last one. What's the biggest mistake that you've made, and what did you learn from it?

A Oh my gosh, how much time do you have? I make mistakes every single day, and some of them are quite large. So I will tell you this, it's probably shock people, but I think it was a huge mistake for me to leave academia and become a poker player. Honestly, I don't even know that I was, like, making an orderly decision in any way, shape, or form. At the end of graduate school, for those who don't know, I was suffering with some stomach issues that made me take time off, and then I sort of needed to fill in the gaps, money-wise. So I had a fellowship when I went to graduate school, and then when I went to take a year off, this was literally right at the end, I completed all my PhD work. I just needed something where I could have flexible hours, and I didn't want to reboot a new career, and blah, blah, blah. And so, like, I started playing poker, and then I never stopped. And I don't know that I ever, like, made a conscious decision to never stop. I think I just sort of didn't go back. And I didn't get my PhD, which, like, I was already done with. Literally, like, I did the research for my dissertation, and I didn't do it. And I look back at that, and I'm just like, that was a mistake. Like, that was a completely absurd mistake. I don't even want to call it a decision, because I can't remember making a decision, actually thinking about the decision to not go back to graduate school…

AI assessment note: “I think it was a huge mistake for me to leave academia and become a poker player.”

Q How about your biggest pet peeve from what you've seen inside investment organizations that you've worked with?

A I think that particularly when people are very successful, it's particularly uncomfortable for them to imagine that there might be a ways that they could really improve their decision process. And I think that the reason is that then there's all of a sudden you open up a counterfactual world where maybe they did even better. And in some ways, I think that makes them feel like they've turned a win into a loss, right? Like I thought I was so great and I won so much. And now you're telling me that maybe I left like half a bit on the table and then you're, that's gonna make me sad because it's gonna turn out that I wasn't the best decision maker in the world. And so a lot of times when I'm sort of working with teams and exploring a relationship with them, and I'm talking to kind of the decision makers, they aren't particularly open minded to the idea that they might be able to clean things up. And very often they'll bring me in and say, I want you to work with the people that are below me. But I'm fine. And then when you suggest to them that, well, maybe there are things that we could do that could actually make your decision process better, they sort of swat you away. And I have to say that that's like just a really big pet peeve for me, because I think that for one thing, it's a little headswell, like, how are you successful if you think that way? But I think that we all need to …

AI assessment note: “that's like just a really big pet peeve for me”

Q So a lot of the power of negative thinking and some of these decisions could be construed as based on an individual checking themselves. In a lot of investing, the decisions get made in groups, sometimes with a committee, sometimes just a group of peers. How do you think about making good decisions in groups?

A So I really dedicate almost a whole chapter to this question, chapter nine. Here's the danger with groups, is When we think about inside, outside view, what we're really talking about is how are you kind of getting outside of your own perspective in order to see the world from somebody else's perspective? And we can include in that, actually, how are you actually learning new things? And we know that a lot of the facts that you don't know live in other people's heads. And we know that a lot of the perspectives, that the ways of seeing the world are the same data that you're looking at that might be different. Lives in other people's heads. So we have this intuition that group decision making should be better than individual decision making. Why? Well, it's the old aphorism, two heads are better than one, and so now we have a group that's five heads or six heads, and we should have lots and lots of different perspectives colliding, and this should actually produce better decision making. But what we know is that generally isn't true. What generally happens is that the decisions that come out of a group process are, are not really necessarily better. It's just that you have much more confidence in them because you have this perception that group decision making is higher quality. And that's actually a really bad combination. Just more confidence in a decision that isn't really mu…

AI assessment note: “if you could surface all of that dispersion of opinion”

Q So how else are you spending your time in?

A Well, the newsletter takes a lot of time. So I just sold another book with a third book behind that, so I'm going to be working on a workbook, actually, to help people be able to instantiate some of the lessons in the book, and then another book that's a little bit more on the line of what we just talked about, actually, in terms of this real problem of, like, how do we actually get people away from being outcome-driven in their decision-making? Going back to do my PhD with Phil Tetlock. Trying to fit that in. It's been slow, but, because I'm a little busy. And outside of, like, my children, which take a lot of my time and other things, I'm doing a ton of this. I do a lot of keynoting, trainings, half-day, full-day trainings, coaching, and consulting.

AI assessment note: “Well, the newsletter takes a lot of time. So I just sold another book”

Q What are some of the other key remedies to help people quit better?

A Yeah, so another one that actually goes, I think, really well with monkeys and pedestals is what I call kill criteria. So let me just set this up by saying, I think Daniel Kahneman says this really well, that the worst time to make a decision is when you're in it. So what does he mean by in it? I like to describe it as you decided that you wanted to eat healthier, now there's a cupcake sitting in front of you. That's what it means to be in it. It's really hard to actually do that. Ok, so, Can we figure out a way to make these decisions when we're not in it? Because that should improve behavior. So kill criteria comes from this idea. Actually, it's from, I got the idea mainly from some work of Barry Staws. One of the things that doesn't help, and I think this is very, very important for financial professionals to hear, is one of the things that does not help is saying, just treat the decision like it's fresh. That's like intuitive, right? Well, if the problem is that I made the initial allocation, then what I should do is think, well, what if I were one of those people who was new to the decision? And I've actually heard people that I work with say this all the time. Oh, I tell my traders to say, what would you do if you had to buy it today? So it's sort of trying to do that mind trick of think about it as if you were new to the decision, as if you can somehow sweep the cognitiv…

AI assessment note: “another one that actually goes... really well with monkeys and pedestals is what I call kill criteria”

Q So here we are again. What was the path to coming up with the idea for Quit?

A So here's the thing. So after the last book, I really did say, like very much swear up and down that I wasn't going to write another book. So what I realized, though, was that sometimes you need to write a book, like you just have to. And I think that that was true with my first general audience book, Thinking in Bats. It was something that I'd been thinking about for a decade before I actually kind of made the decision that I was going to sit down and write it. And so I think that was something that had been brewing in me for a long time. When I wrote How to Decide, it was meant to give people a practical way to implement thinking in bets, but I wouldn't say it was a book like I had to write. It was a book that I felt was important to write for readers in relation to thinking in bets. So after that, I was really like, I'm never writing another book again, because it really, it's such torture to write a book. It's a lot of work. I swore that up and down to you. It was probably two months later that I asked you to get on a Zoom with me. To talk about this topic of quitting. So what happened? Right? That's the question. Like, how did that happen so quickly? So it's a good example of overconfidence. Basically, here's what happened. I'm doing podcasts like this one here for how to decide. And there's lots and lots of materials and how to decide. But as you kind of roll around to ch…

AI assessment note: “Basically, here's what happened. I'm doing podcasts like this one here for how to decide.”

Q So a lot of the monkey and pedestals and the kill criteria, you can imagine an individual decision maker working through that. How can you leverage a team that's working together to make better decisions?

A Yeah. So again, I said a lot of the problem is that we're trying to make these decisions when we're in it, and we know we're not very good at that. So I kind of think about two ways to not be in it. One is to think in advance. So that's to not be in it on your own timeline. Okay. Like I'm on my own timeline, and I'm thinking about it far in advance. And monkeys and pedestals would be doing that. It's think about what's the hard part of the problem first. Maybe I should do an engineering study, a feasibility study on the mountains before I start building any track. That's thinking in advance on how do I approach the project to figure out if I can solve for the bottleneck first. Kill criteria is the same thing. What basically saying to yourself in a way that feels counterintuitive, what are the signals that I might see in the future that would tell me that I ought to quit? And that's counterintuitive because we think, well, we have a thesis as we enter into, say, an investment. And obviously when the world's turning against our thesis, we're going to quit. So I don't need to do this advanced step, but you actually do. And so that's one way to do it. But another way to do it is to be on your own timeline, right? So in that moment in time, even when you might be in it, and talk to somebody who's not in it. In other words, just get yourself a quitting coach. And we all know this. We…

AI assessment note: “talk to somebody who's not in it. In other words, just get yourself a quitting coach.”

Q just a nudge. And I actually don't want your opinion because I don't value it. So I then, before the meeting, call you and say, hey, we're thinking about this decision. We're thinking of doing X. Does that work? I mean, it doesn't get to the point of making a good decision, objective truth, and enlisting that person's opinion, but it is sort of a psychological runaround if it works.

A So I think, first of all, we want to think about, are they actually a nudge, or are they just challenging us? Because a lot of times when a committee is coming to consensus, we don't like it when someone's pushing back at that. And if the person is in the role of the sort of person who's pushing back on it, You should kind of be grateful that that person exists because it forces you to be thinking about things from different perspectives. And like I said, it forces you to actually be able to give a rationale for why you believe what you do. You have to actually support it. You have to say, well, this is why I think the table weighs more than your computer. So I think there's a lot of value in those nudges. Now, obviously someone who's not doing that in good faith, that's just a cultural problem where perhaps you should not have them on your team. So let me just say that. But this idea of what I sort of call being butt-wide, I'm thinking about what Richard Feynman said, like, you better be able to explain it to an eight-year-old in terms that they can understand why you're going ahead with that decision, even if they don't agree with you, which was fine. So there's an exercise that I have in my book that really shows this butt-wying. So let me explain what butt-why is. My five-year-old, let's say, says to me, Mommy, why is the sky blue? And I went to graduate school, and I under…

AI assessment note: “You should kind of be grateful that that person exists because it forces you”

Q Other than suffering from resulting, confirmation bias, motivated reasoning, and just about anything else, Is there anything we can do to marginally improve the quality of our decisions?

A So we can, and the important word in there is marginally. I think that what people generally think is like, well, I'm super smart, and everybody's super smart because of the better than average effect, so we're all super, super smart, and now I know about this. Someone's told me about motivated reasoning. Someone's told me about confirmation bias. They've told me about resulting. They've told me about all this stuff, so yay me. I'm clearly not going to do it anymore. And the answer is, yes you will, and in fact it will be worse for you. So I'm just gonna give the little bad news before I get to the good news. So there's actually two really good pieces. One's in my book. It's a study by Dan Kahan that showed that people who are really numerate, really good with statistics, for example, being able to work correlational tables that are difficult because the raw numbers kind of point you in the wrong direction, but the correlation points you in the right one, that people are really good with those tables. When you give them tables that have to do with something that is politically motivated for them, like gun control, they're worse. They're better at spinning the data to fit their beliefs because they're better with data. Like, who do you put in a spin room? The dumb guy or the smart guy? No, you put the smart guy so that he can data mine for you. So that's number one. Then there w…

AI assessment note: “I'm just gonna give the little bad news before I get to the good news.”

Q Can we close this by you telling a poker story of how you can take advantage of being a woman?

A Oh yeah, sure. That's an interesting way to close it. Sure, absolutely. Okay, so to your point, I think that one of the problems that I have found when I've talked to women, just to give you an idea, in an average poker room, three percent of the people playing are women. Not too many of us. So you're sitting at the table, and by the way, there's no HR department. And there's this real sense, like, as someone says something completely outrageous to you at the table, and you look at the floor person who's supposed to be sort of controlling the game, it's this, well, you can leave. So, alright, ok, I guess I could. So it's this idea, like, you're here voluntarily, so therefore whatever abuse you're taking, you're choosing. A lot of what's happening is that what's coming at you is a real disrespect for your abilities and your intellect. I think that everybody has a need to be liked and respected, but there's a time and a place for that, and when you're sitting across the table from an opponent, it is not in any way, shape, or form to your advantage to be, prove that you should be liked and respected. As much as it doesn't feel good when someone says something really nasty to you and disrespects you, as an opponent, that's an advantage. So what I used to try to think about when I was playing was That in terms of the structure that I was sitting in, there wasn't a whole lot that I c…

AI assessment note: “As much as it doesn't feel good... as an opponent, that's an advantage.”

Q It'd be nice if we all could put numerical probabilities on the decision tree and actually have an expected value and be so easy to make the right decision. But most of the time, we don't really know what those probabilities are. So how do you start to think about that?

A Yeah, that's really most of the decisions that we make. We can think about that we're deciding in some way, sometimes more, sometimes less, behind a veil of ignorance. That we know some of the things that might occur, but not all of them, and we may not be able to assign probabilities to those things, at least not in terms of an exact point forecast. Like, if I flip a coin, I know exactly what the outcomes are. There are three heads, tails, or landing on the side. And then I know what the probabilities of those things happening are. So landing on the side is approaching zero and the rest is about fifty-fifty. So in this particular case, I actually have perfect knowledge despite the fact that there's an influence of luck. So I want to just sort of take a moment to think about the two sources of uncertainty because we're talking about the second source. The first source is luck, which is even if I know what all the possibilities are and I can assign probabilities to those, it doesn't mean that I know which Outcome I'm going to observe. So when I flip a coin, I don't know whether I'm going to observe heads or tails on that particular coin flip. I only know these things in the long run. And that's a problem in and of itself, right? That's really kind of the resulting problem, which is I think that if I call heads and it lands tails, that somehow that makes my decision wrong, which …

AI assessment note: “I want to just sort of take a moment to think about the two sources”

Q You've now written the book, and there's the definition of the problem, some prescriptions, and you go out and you start advising organizations who have read the book. What have you found?

A So it's two things. So when I was writing the book and I was talking about this disconnection between outcomes and decision quality, I was thinking about two problems that come from this. Problem number one is that you can make really, really good decisions and have them not work out. Even if it's 99% of the time it's going to work out, like sometimes that one percent hits. So there was definitely that side of it, but the obverse of that is that you can make really, really terrible decisions and have those work out just fine. I have run red lights in my life by mistake, and I'm alive. Thank you very much. And both of these are really huge problems for learning. Really, really big problems for learning, because in one case, obviously, you don't want to think that just because it worked out well that I should now go around running red lights. That's pretty bad. I mean, I've actually heard people say, like, I drive better when I'm drunk. It's like, great, because you got home safely once.

AI assessment note: “when I was writing the book and I was talking about this disconnection”