Friedman: Diode Failed on GPT-4 but Worked on o1 With Same Prompts
Jared Friedman · Why The Next AI Breakthroughs Will Be In Reasoning, Not Scaling · Y Combinator · Nov 14, 2024 · at 9:24
Jared Friedman is a Group Partner at Y Combinator discussing the performance differences between GPT-4 and OpenAI's o1 reasoning model for startup Diode Computer.
0:00 / 0:21exact quote · 21.3s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“The other thing I think is interesting about this example is like during the batch before a one came out, diode had tried to do this with GPT four. Oh, and it just flat out didn't work. And then they basically tried the same thing, the same prompts, but fed it to a one and boom, all of a sudden it worked. And so there really is a sort of like step function capability unlock.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Jared Friedman
Friedman: Worrying about security and privacy limits AI agent power
“It turns out that, like, the thing that was hampering the world was being worried about security and privacy and all the things that could go wrong, and when you, like, worry a bit less, you're like, oh my god, these things are unbelievably powerful.”
Friedman: Majority of Software Code Is Probably Already Written by AI Agents
“It's already probably the case that the majority of code being written is written by agents.”
Friedman: Tan replicated years of startup work in two weeks using Claude
“I feel like your real feel the AGI moment, Gary, was like getting Claude Code to build basically an entire startup for you, like replicating years of work of your previous startup in like two weeks, which is like insane.”
Friedman: Future SOTA Benchmarks May Come From Swarms of Cheaper AI Models
“It might be that like the next stuff that like is soda on benchmarks is not the most expensive newest foundation model with the most like GPU training. It's like a swarm of lower cost cheaper models working together just like humans do to solve a problem.”
Friedman: Cheap AI makes developers rewrite code rather than debug it
“Like it's wild how your coding style changes when actually writing the code becomes a thousand X cheaper. Like as a human, you would never just like blow away something that you'd worked on for a long time and rewrite it from scratch because you had a bug. You…”
Friedman: AI coding agents will bullshit founders just like human employees
“The AI agents, incidentally, will do exactly the same thing. They will absolutely, like, The AI agents will bullshit you just like a human employee will if you don't, like, if you're not technical enough to, like, call them out on their bullshit and be like, n…”