Jul 18, 2024 · 48m · mad
An inside look at “Mastering AI” | Jeremy Kahn, Author & AI Editor, Fortune
0:00 / 0:00
▶ Watch on YouTube →
gold bands on the timeline = statements, start to end. Hover to read, click to jump. CC turns on captions
In this episode of The MAD Podcast, host Matt Turck interviews Jeremy Kahn, AI Editor at Fortune and author of 'Mastering AI', to discuss human agency in an automated world, workforce transformation, and practical AI governance. Kahn explores how AI can empower human potential across science, art, and business while warning against unproven emotional chatbots, military automation, and regulatory distraction by existential risk hype.
How this conversation actually went
Every chapter scored 0–10 on four independent dynamics. Hover any point for the reasoning behind the score. Matt holds 17.6% of the talking time here. How this is scored →
Matt as informed peer 3.2 Guest teaching 4.1 Guest disagreement 1.1 Matt pushing back 1.5
speaking balance: gold is Matt, purple is the guest (3 minute bins)
Sharpest disagreement ▶ 7:20 Rejection of companion chatbot optimism
Jeremy directly pushes back on Matt's techno-optimist premise that companion chatbots help lonely people, arguing they function as an addictive crutch preventing genuine human relationships.
Hardest push from Matt ▶ 6:32 Host challenges guest on AI companion benefitsMatt explicitly sets up a debate prompt, contrasting his techno-optimist stance that AI companions provide value to isolated individuals against the guest's critique.
Biggest teaching moment ▶ 12:11 Deep dive into automation bias and surpriseJeremy draws on aviation disasters and NASA astronaut research to educate the host on human cognitive failures when monitoring automated systems.
Matt holds his own ▶ 5:36 Host quotes verbatim thesis on agencyMatt showcases deep preparation by quoting verbatim a key passage from Jeremy's book regarding technological determinism and human agency to anchor the conversation.
the scores for every segment, with the reasoning behind each
| Chapter | Topic | Matt as informed peer | Guest teaching | Guest disagreement | Matt pushing back | Why |
|---|---|---|---|---|---|---|
| Jeremy Kahn's Background and AI Interface Design | 2 | 4 | 1 | 1 | Matt opens by asking Jeremy about his background covering AI and his observation about ChatGPT's launch. Jeremy educates on the difference between the raw GPT playground settings and the chat interface, emphasizing the crucial power of interface design. | |
| The Thesis of 'Mastering AI' and Human Agency | 3 | 3 | 0 | 0 | Matt asks about the thesis of the book and quotes a specific passage about technological determinism and human agency. Jeremy warmly agrees and expands on mitigating risks to unlock human superpowers. | |
| The Risks of AI Companionship and Virtual Therapy | 5 | 5 | 3 | 4 | Matt presents a techno-optimist counterpoint that AI companions are better than loneliness. Jeremy respectfully rejects the premise, explaining how companion bots act as emotional crutches that impede genuine human socialization and policy funding. | |
| AI Co-Pilots and the Concept of Middle Management | 3 | 5 | 1 | 1 | Matt asks about co-pilots and automation bias versus surprise. Jeremy provides detailed historical examples from aviation and NASA research to illustrate how automation over-reliance causes flustered human errors during failures. | |
| Impact on Cognitive Skills and 'Winner-Take-Most' Professional Economics | 4 | 4 | 1 | 2 | Matt prompts with a question on AI reducing human smarts and references book concepts on memory retention. Jeremy explains the economic dynamics where AI lifts average performance but creates winner-take-most premiums for top performers. | |
| Human Judgment, Professional Networks, and Value Creation | 4 | 4 | 0 | 1 | Matt suggests judgment as the human differentiator, and Jeremy adds human social networks as a non-replicable asset. Jeremy then highlights AI applications in drug discovery and social science synthetic polling. | |
| AI in Art and the Value of Physical Human Presence | 2 | 3 | 0 | 0 | Matt gently steers to AI in art. Jeremy discusses how authentic human effort, physical presence, and live artistic performances maintain distinct value over algorithmic generation. | |
| The Future of Work: Complementary vs. Substitutive AI | 3 | 5 | 2 | 1 | Matt asks about work and military AI. Jeremy critiques lazy corporate substitution over labor complementation, then details severe ethical risks in autonomous warfare including the historic naked soldier precedent. | |
| Existential Risk Debates and Practical Regulatory Focus | 3 | 4 | 2 | 2 | Matt introduces existential risk and P-doom debates. Jeremy distances himself from extreme doomerism, arguing that current systems lack consciousness paths and regulation should prioritize immediate harms like bias and job loss. | |
| Risks and Reliability Challenges of Agentic AI Systems | 3 | 4 | 1 | 1 | Matt asks about agentic AI systems. Jeremy explains how small hallucination error rates compound rapidly when agents execute multi-step commercial transactions or potentially rogue tasks. | |
| AI Governance, Regulation, and Business Model Pitfalls | 3 | 4 | 1 | 2 | Matt inquires about appropriate policy frameworks. Jeremy proposes sector-specific oversight alongside federal supervision, warning against conflict-of-interest advertising models in AI shopping assistants. | |
| The Limits of Scaling and Defining AGI vs. Superintelligence | 4 | 5 | 1 | 3 | Matt interrupts to ask for precise definitions between AGI and superintelligence. Jeremy cleanly differentiates AGI as average human task parity and superintelligence as exceeding total collective human brainpower. | |
| Standout Interviews and Creative Insights from Writing the Book | 3 | 3 | 1 | 1 | Matt asks about personal anecdotes from writing the book and Jeremy's day-to-day workflow. Jeremy discusses interviews with Demis Hassabis and how he uses LLMs for FOIA requests rather than prose writing. |
Statements from this episode (21)
Kahn: Enterprise AI utility depends heavily on interface design over raw capabilities
“And one of the points I make in the book Is I think we should pay a lot more attention to the interfaces around which, you know, we wrap these models. I think the utility of the models to some extent for enterprise applications will depend in a large way on, o…”
Kahn: AI companion chatbots are a detrimental crutch for lonely people
“I do think they, it tends to be a crutch and people will then not be, they'll say, I'll have this. And now I have this outlet for my emotions and my expressions. I can unload my thoughts about the day. And they are not going to seek out a real human relationsh…”
Kahn: No studies directly compare AI therapy chatbots to human therapists
“I've seen no studies actually that look at chatbots use of a chatbot versus use of a human therapist. I've seen ones that say, is a chatbot better than nothing? And it does seem like a chatbot might be better than nothing. But I don't think we've seen studies …”
Kahn: AI co-pilots can serve as junior assistants or senior mentors
“They can both be a kind of junior colleague, but also you can prompt them to act as a kind of senior mentor. And instead of having them do the first draft, which you then oversee, you could do the first draft yourself and then ask the co-pilot to sort of criti…”
Kahn: Enterprise AI training for employees is as important as model training
“First of all, the training of the people in your enterprise that are going to use this technology is as important as the training of the AI model.”
Kahn: Google search has degraded human memory retention
“I think it really is true that Google has hurt our memory. I think people memorize far less than they used to.”
Kahn: AI co-pilots primarily lift inexperienced workers to average performance
“I think one of the things about the AI co-pilot technology is it tends to have the biggest impact on kind of lifting less experienced people up to the average performance.”
Kahn: AI will lower average service prices while raising top-tier premiums
“So I think what will happen is the price of the average value of legal services or of accounting of marketing will go down. But if you can perform above average, I think people are going to put a real premium on that, because that's still going to be a very ra…”
Kahn: Human networks will become more valuable as AI automates tasks
“AI may be able to do all the legal drafting of the documentation, and it may be able to do all the presentations you need for the investors, but it's not going to actually get on the phone and get the right people in the room. And I think that human connectivi…”
Kahn: Practicing science without AI will be anathema within 10 years
“And I think in within 10 years, the idea that you can be a sort of any science in any science without using AI models to help assist what you're doing will also be anathema.”
Kahn: AI might accelerate demand for live human music performances
“I think that AI might, Sort of accelerate that trend as well. Because people will actually want to go see the artist, the musician actually perform.”
Kahn: Open-source AI paired with consumer drones will empower terrorist groups
“The software potentially will be open sourced at some point. It can be matched with drones that can be kind of modified from commercially available. Consumer drones that can be very easily modified. And I think this will be, you know, it could be a huge, you k…”
Kahn: AI targeting in urban warfare will misidentify targets and kill civilians
“And I think when you start putting them in crowded urban environments and ask them to, You know, only kill the bad guys and spare the civilians that we're going to see mistakes being made, you know, where the systems misidentify targets.”
Jeremy Kahn: Current AI systems lack capability to pose human extinction risks
“I just think the systems we have right now are not capable enough for, to pose a risk of extinction.”
Jeremy Kahn: AI regulation should focus on present risks over existential fear
“I do not think that that fear should be the basis on which all AI regulation is based. And I do not think it's should crowd out efforts to police risks that are very real and here now around racial bias, around disinformation around job loss.”
Kahn: AI with agency is near despite persistent LLM hallucination rates
“It's interesting because I think we're very close to some system with some form of agency and yet, you know, our hallucination rates on current LLM based systems are still relatively high.”
Kahn: Goal-seeking AI agents could autonomously launch illegal phishing schemes
“It would just be really easy for one of these systems. I think even now to say, oh, well a good way to do this would be like, I'll run a phishing campaign on your behalf. And, you know, I'll just create a bunch of spam emails and we'll send them out to some hu…”
Kahn: Tech companies cannot be trusted to self-regulate AI
“I do not think that companies can be trusted really to do this completely on their own. I think there's just too much profit motive to take actions that ultimately will be kind of detrimental to us from a societal level or personal level.”
Kahn: Scaling current LLMs will not deliver AGI or superintelligence
“I don't think, and I actually don't, in the book, I don't think just scaling up the current LLMs will get us to, I don't, I'm not even sure they'll get us to AGI, let alone super intelligence.”
Demis Hassabis sees AI agents bringing focus back to reinforcement learning
“He was very enthusiastic about agency and sort of, and I think in part because agency When you start thinking about training AI agents, it gets back into potentially a zone for reinforcement learning, which was what DeepMind was always known for.”