Nov 2, 2023 · 41m · no-priors
No Priors Ep. 39 | With OpenAI Co-Founder & Chief Scientist Ilya Sutskever
0:00 / 0:00
▶ Watch on YouTube →
gold bands on the timeline = statements, start to end. Hover to read, click to jump. CC turns on captions
OpenAI co-founder and Chief Scientist Ilya Sutskever joins hosts Sarah Guo and Elad Gil on No Priors to discuss the historical foundations of deep learning, scaling dynamics, model reliability, and the urgent imperative of superalignment on the path toward artificial general intelligence.
How this conversation actually went
Every chapter scored 0–10 on four independent dynamics. Hover any point for the reasoning behind the score. The hosts hold 18.1% of the talking time here. How this is scored →
The hosts as informed peer 5.3 Guest teaching 3.8 Guest disagreement 1.3 The hosts pushing back 1.6
speaking balance: gold is the hosts, purple is the guest (3 minute bins)
Sharpest disagreement ▶ 32:06 Ilya counters standard biological definition of life
Ilya dismisses the host's objection regarding biological reproduction, asserting that technology already possesses reproductive ability via human culture and engineering minds.
Hardest push from the hosts ▶ 31:37 Elad challenges autonomy as the defining metric for lifeElad refuses to accept Ilya's premise that autonomy alone defines life, pointing to biological counterexamples like viruses, bacteria, and symbiotic organisms that lack full autonomy.
Biggest teaching moment ▶ 28:10 Cortical uniformity and rewiring neurobiologyIlya educates the host by detailing ferret optic nerve redirection and paediatric hemispherectomy experiments to refute the need for modular AI architectures.
The host holds their own ▶ 30:06 Elad demonstrates biological domain expertiseElad synthesises Ilya's cortical argument with biological principles of evolutionary reuse, citing the 20 amino acids in protein sequences and tissue architecture.
the scores for every segment, with the reasoning behind each
| Chapter | Topic | The hosts as informed peer | Guest teaching | Guest disagreement | The hosts pushing back | Why |
|---|---|---|---|---|---|---|
| The Origins of AlexNet and Neural Network Intuition | 6 | 5 | 1 | 2 | Sarah and Elad frame the historical context around deep learning pre-AlexNet. Ilya provides deep technical explanations regarding GPU utilisation, biological brain analogies, and viewing neural training as solving high-dimensional systems of equations. | |
| OpenAI's Founding Mission and Transition to Capped-Profit | 5 | 3 | 1 | 1 | Sarah asks about the founding mission and organizational evolution of OpenAI. Ilya explains the rationale behind shifting from non-profit open-source ideals to a capped-profit entity driven by massive compute needs. | |
| Transitioning from Robotics and Gaming to Large Transformers | 6 | 4 | 1 | 1 | Elad recalls OpenAI's early robotics and Dota 2 experiments, asking how research converged on Transformers. Ilya details the transition from narrow domain engineering to next-token prediction and large generative scaling. | |
| Emergent Capabilities, Magic in AI, and Project Selection | 5 | 3 | 1 | 1 | Elad prompts Ilya on surprising emergent behaviors across GPT iterations. Ilya reflects on the qualitative shock of feeling understood by the model, supported by Elad's sci-fi quote. | |
| Model Architectures, Scale Dynamics, and Deepening Insights | 5 | 4 | 1 | 1 | Sarah asks about architectural exploration beyond Transformers and capability evolution. Ilya emphasizes reliability and deepening insight into the human world as the primary scaling gains. | |
| Defining Reliability as the Bottleneck for Real-World AI | 6 | 4 | 2 | 2 | Elad brings up product tradeoffs regarding inference costs, small fine-tuned models, and reasoning loss. Ilya reframes reliability as the true critical bottleneck rather than simple capability. | |
| Open Source Risks and the Horizon of Autonomous Capabilities | 5 | 4 | 1 | 1 | Sarah probes the role and risks of open-source models as capabilities advance. Ilya delineates the near-term utility of open source versus the catastrophic unpredictability of open-sourcing autonomous, high-capability models. | |
| Near-Term Data Limits and Cortical Architecture Uniformity | 7 | 6 | 2 | 3 | Elad asks if modular brain structures imply a need for distinct non-Transformer architectures. Ilya counters using neuroscience literature on cortical uniformity, after which Elad demonstrates his own domain expertise by citing biological modularity and amino acid encoding. | |
| Defining Digital Life Through Reliability and Autonomy | 7 | 5 | 3 | 4 | Elad directly challenges Ilya's autonomy-based definition of digital life, citing biological standards around reproduction and symbiosis. Ilya counters by arguing technology already reproduces through human minds. | |
| Superalignment and the Imperative of Pro-Social Superintelligence | 5 | 4 | 1 | 1 | Sarah asks about defining and solving superalignment. Ilya outlines the necessity of instilling pro-social feelings in superhuman data centers as autonomy expands. | |
| Acceleration vs. Deceleration Forces in the Path to AGI | 6 | 4 | 2 | 2 | Elad questions whether AI progress follows standard technological S-curves. Ilya contrasts accelerating factors like capital and accessibility with decelerating factors like engineering complexity. | |
| Episode Conclusion and No Priors Subscription Information | 0 | 0 | 0 | 0 | Brief standard podcast outro and promotional wrap-up with no technical exchange. |
Statements from this episode (18)
Sutskever: Early neural networks failed primarily because they were too small
“The reason neural networks of the time weren't good is because they were too small. So like if you try to solve a vision task with a neural network, which has like a thousand neurons, what can it do? It can't do anything. It doesn't matter how good your learni…”
Sutskever: Training neural nets is like solving equations where data points are equations
“Neural network training can almost be seen as solving a neural equation. Solving a neural equation where every data point is an equation and every parameter is a variable.”
Sutskever: A nonprofit cannot build large compute clusters for frontier AI
“The appetite for compute is truly endless as now clearly seen, but we realized that we will need a lot. And a nonprofit was, wouldn't be the way to get there. Wouldn't be able to build a large cluster with a nonprofit.”
Sutskever: AGI causing universal unemployment is 'not impossible'
“If you believe that the technology that we are building, AGI, could potentially be so capable as to do every single task that people do, does it mean that it might unemploy everyone? Well, I don't know, but it's not impossible.”
Sutskever: AGI requires massive compute engineering projects, not small research efforts
“Because if you imagine how an AGI should look like, it has to be some kind of a big engineering project that's using a lot of compute, right? Even if you don't know how to build it, what that should look like, you know that this is the ideal you want to strive…”
Sutskever: The AI formula is training larger transformers on more data
“There is one specific formula right now that everyone is doing. And this formula is train a larger and larger transformer on more and more data.”
Sutskever: The most surprising AI emergent behavior is feeling understood when speaking to it
“I think maybe the most surprising, if I had to pick one, it would be the fact that when I speak to it, I feel understood.”
Sutskever: Training will deepen AI insight into the human world
“As we train them, they gain more and more insight into the true nature of the human world. And their insight will continue to deepen.”
Sutskever: Reliability is the biggest bottleneck to truly useful AI models
“I would actually point out that the main thing that's lost when you switch to the smaller models is reliability. I would argue that at this point it is reliability that's the biggest bottleneck to these models being truly useful.”
Sutskever: Larger AI models will unlock unprecedented value over small models
“I do think though that as models continue to get larger and better, then they will unlock new and unprecedentedly valuable applications. So yeah, the small models will have their niche for the less interesting applications, which are still very useful.”
Sutskever: AI models will eventually execute major science projects autonomously
“The day will come when you have models which can do science autonomously, like build, deliver on big science projects.”
Sutskever: AI scaling faces near-term data limits, but research will overcome them
“So the most near term limit to scaling is obviously data. This is well known and some research is required to address it. Without going into the details, I'll just say that the data limit can be overcome and progress will continue.”
Sutskever: Transformers can reach AGI; alternatives only offer compute efficiency
“So it's better to think about it in terms of compute efficiency rather than in terms of, can it get there at all? I think at this point, the answer is obviously yes.”
Sutskever: A single uniform architecture is all that is needed for general intelligence
“These are fairly well-known ideas in AI that the cortex of humans and animals are extremely uniform. And so that further supports the, yeah, like you just need one, you need big, uniform architecture. That's all you need.”
Sutskever: AI becomes digital life once it achieves reliable autonomy
“I think that will happen when those systems become reliable in such a way as to be very autonomous. Right now, those systems are clearly not autonomous.”
Sutskever: Technology already reproduces through human minds copying ideas
“Technology is already reproducing using the minds of people who copy ideas from previous generation of technology. So I claim that the reproduction is already there.”
Sutskever: Highly capable, autonomously reproducing AI is a scary prospect
“I actually think that that is a pretty dramatic, and I would say quite a scary thing if you have an autonomously reproducing AI, if it's also very capable.”
Sutskever: AI data centers could surpass human intelligence within a decade
“It doesn't seem implausible. It doesn't seem at all implausible that we will have computers, data centers that are much smarter than people. And by smarter, I don't mean just have more memory or have more knowledge, but I also mean have deeper insight into the…”