Sep 13, 2024 · 56m · big-technology
Is OpenAI’s New “o1” Model The Big Step Forward We’ve Been Waiting For?
0:00 / 0:00
▶ Watch on YouTube →
gold bands on the timeline = statements, start to end. Hover to read, click to jump. CC turns on captions
Host Alex Kantrowitz and tech columnist Parmy Olson analyze OpenAI's newly released o1 reasoning model, evaluating its benchmark breakthroughs, enterprise readiness, and operational limitations. The discussion also examines the fierce corporate rivalries, governance controversies, and massive capital expenditures defining the broader artificial intelligence industry.
How this conversation actually went
Every chapter scored 0–10 on four independent dynamics. Hover any point for the reasoning behind the score. Alex holds 42.3% of the talking time here. How this is scored →
Alex as informed peer 6.1 Guest teaching 5.0 Guest disagreement 2.9 Alex pushing back 2.7
speaking balance: gold is Alex, purple is the guest (3 minute bins)
Sharpest disagreement ▶ 10:35 Olson dismisses safety rumor behind Altman firing
Parmy forcefully rejects Alex's suggestion that Ilya Sutskever and board members ousted Altman over existential safety panic, clarifying it was fundamentally about executive honesty.
Hardest push from Alex ▶ 15:18 Kantrowitz calls bullshit on Salesforce AI agentsAlex bluntly dismisses Parmy's mention of Salesforce Agentforce, refusing to accept vendor marketing claims that Salesforce is pioneering autonomous AI agents.
Biggest teaching moment ▶ 32:38 Olson explains the reality of capped-profit governanceParmy corrects Alex's characterization of OpenAI as a nonprofit, detailing how the capped-profit structure evolved and why both Altman and Hassabis failed to build sustainable governance.
Alex holds their own ▶ 1:08 Kantrowitz cites granular o1 benchmark jumpsAlex demonstrates commanding technical grasp of the new release by rattling off exact percentage point increases across math, code, and PhD-level science testing.
the scores for every segment, with the reasoning behind each
| Chapter | Topic | Alex as informed peer | Guest teaching | Guest disagreement | Alex pushing back | Why |
|---|---|---|---|---|---|---|
| Unpacking OpenAI's o1 Model and Reasoning Benchmarks | 8 | 2 | 1 | 1 | Alex opens by quoting exact benchmark figures from the OpenAI o1 release, highlighting jumps across competitive math, coding, and PhD-level science questions. Parmy responds collaboratively, framing the release around company pressure. | |
| The Gap Between AI Demos and Public Rollouts | 7 | 1 | 1 | 2 | Parmy highlights OpenAI's pattern of unfulfilled announcements like Sora and Advanced Voice. Alex matches her insight with his own subscription testing experience and quotes Sam Altman's snarky reply to Matt Paulson on X. | |
| Niche Utility Versus the Swiss Army Knife of AGI | 6 | 5 | 2 | 2 | Alex details technical preference data for o1 versus GPT-4o, while Parmy reframes the shift away from broad AGI Swiss army knives as a practical necessity for enterprises paralyzed by indecision. | |
| The Altman Ouster and Truth in AI Leadership | 4 | 7 | 5 | 4 | Alex inquires whether Ilya Sutskever was spooked by capabilities, but Parmy firmly corrects the effective altruist narrative, citing sources that Altman was ousted over truthfulness and executive trust issues. Alex questions if he is giving Altman too easy of a pass. | |
| Enterprise Adoption, AI Agents, and ROI Timelines | 7 | 5 | 4 | 7 | When Parmy introduces Salesforce Agentforce, Alex directly calls bullshit on Salesforce pioneering true autonomous agents. Parmy explains the bot versus agent distinction, while Alex counters with Spellbook Legal's real-world document manipulation workflow. | |
| Multi-Step Reasoning, Hallucinations, and Autonomous Agents | 7 | 3 | 3 | 3 | Alex cites OpenAI research lead Bob McGrew and Bloomberg's five-level roadmap to argue reasoning enables multi-step agents. Parmy warns about the severe stakes of hallucinations when autonomous agents execute business actions. | |
| Buy or Sell: Evaluating the o1 Hype and OpenAI's Strategy | 7 | 6 | 5 | 4 | Alex buys the o1 hype based on technical benchmarks, while Parmy contrarianly sells due to benchmark opacity and shares reporting on VC concerns regarding Altman's lack of focus. Alex pushes back by citing LMSYS Chatbot Arena leaderboards. | |
| OpenAI's $150 Billion Valuation and Capped-Profit Governance | 6 | 8 | 4 | 2 | Alex asks how a nonprofit can raise at a $150 billion valuation. Parmy clarifies the capped-profit corporate structure and draws on her book to explain the failed governance attempts of Altman and Hassabis amid soaring talent salaries. | |
| Sovereign Wealth Capital, Compute Burn, and Measuring AI ROI | 5 | 6 | 4 | 3 | Alex questions the ethics of courting Gulf sovereign wealth capital. Parmy reframes the dilemma between Middle Eastern and Western tech oligarchies, and uses an email analogy to illustrate why AI ROI is hard to quantify. | |
| Contrasting the Minds of Sam Altman and Demis Hassabis | 4 | 8 | 1 | 1 | Alex invites Parmy to share underreported insights about Altman and Demis Hassabis. Parmy provides deep biographical detail on Hassabis's background designing Theme Park at 17, chess mastery, and Nobel ambitions. | |
| Synthetic AI Avatars in Local News and Media Production | 6 | 4 | 2 | 1 | Alex brings up synthetic AI news anchors used by local outlets like The Garden Island. Parmy explains how audience exposure will eventually desensitize viewers to synthetic media noise across digital feeds. |
Statements from this episode (17)
OpenAI o1 Benchmarks Show Major Gains Across Math, Coding, and Science
“So if you think about, ah, competition math, ah, GPT-IV-O was getting a 13.4 accuracy score on competition math. But this thing and the way that it can think through the different problems is getting 83.3, ah, score on accuracy. Ah, with competition math. So i…”
OpenAI's Focus on Unreleased Demoware Creates User Cynicism and Frustration
“And I think that's kind of the trap that Sam and his company has kind of put themselves in by just releasing a lot of demo wear, which is very exciting, and if you start getting people too excited for too long a time, and then you don't give them anything then…”
Users Prefer GPT-4o for Writing But Require o1 for Coding
“If you look at the consumer preferences, like the, or the human preferences, like where they'd want to use these new O one preview models versus GPT four O they actually prefer GPT four O for personal writing. It's about even on editing text. But where GPT, ah…”
Selling General-Purpose AI to Enterprises Causes Customer Indecision and Paralysis
“And I think that actually has a big downside when you're actually trying to sell something packaged like that to businesses, which is that you're giving them this Swiss army knife of a tool that has all these general capabilities, and the risk is that your end…”
OpenAI Board Fired Sam Altman Because He Was Consistently Untruthful
“Well, the real reason was that the board was concerned that Sam was not telling the truth. He was He just was not a reliable source of information, and it wasn't just the board members who felt that, there were top executives who felt that as well.”
Kantrowitz: Salesforce will not be the leader or pioneer in AI agents
“I don't think Salesforce is going to be the leader in AI agents, or even the pioneer.”
Enterprise Generative AI Returns Will Take Significantly Longer Than Anticipated
“It's gonna take way more time than that. And that's been the case with every major tech revolution, whether it was search or desktop or mobile.”
OpenAI Notably Avoided Using the Word 'Agent' in o1 Announcement
“Open AI hasn't actually used the word agent once in its announcement, and it doesn't sound like any of its scientists have talked about that either.”
Kantrowitz: OpenAI's Blog Post States o1 Does Not Solve Hallucinations
“Yeah, and OpenAI even in its blog post says that it does not solve, this does not solve hallucinations, and then you can”
Developing AI Reasoning Capabilities Matters More Than Voice or Video Models
“If I was going to look at those three things, like the voice interface, Sora, and reasoning capabilities, Hands down, the bigger deal is reasoning capabilities.”
Silicon Valley Executives Report Growing Discontent Over Sam Altman's Disorganization
“Last week I talked to some people in the valley and A couple of times it came up that there are the, there's growing discontent about Sam Altman's general lack of focus and people are saying he's disorganized and they're trying too many things.”
Demis Hassabis Spent Years Trying to Separate DeepMind Into a Non-Profit
“Demis tried to break away from Google and also become something like a non-profit. He spent years trying to do that, and spent a lot of money and time trying to do that, and they both failed to create these structures of Responsible governance.”
Senior AI Executives Command Compensation Packages of $1.5 Million Annually
“One kind of mid-tier AI startup was asked, so how much would you pay for, like, a senior AI kind of executive, and they said, it's Basically around over the course of four years, six million dollars. I've heard elsewhere, you know, two to three million dollars…”
Olson: Big tech can sustain heavy AI spending for years
“They can totally afford this, and they can afford to keep spending like this for the next few years at least.”
Olson: AI will be essential like email but hard to measure ROI
“It will become a really important component of making a business more productive, but it will be really hard to measure exactly how it does that in numbers.”
Olson: Hassabis Was First to Launch an AGI-Focused Company in 2010
“Demis did it first. He was actually the first kind of put his neck on the line as a among the scientific community to do something that was really a fringe theory back in the, in 2010 and started a company that actually wanted to build human level AI.”