Steinberger: A mediocre post-AGI future does not exist; outcomes are bimodal
Eric Steinberger · No Priors Ep. 79 | With Magic.dev CEO and Co-Founder Eric Steinberger · Aug 30, 2024 · at 33:51
Magic.dev CEO Eric Steinberger discusses why navigating AI safety and avoiding catastrophic failure modes is the central imperative of developing AGI.
0:00 / 0:03exact quote · 3.3s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“Because I can't come up with, like, a mediocre AGI future. It doesn't exist.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Eric Steinberger
Steinberger: It will eventually be responsible to replace human CEOs with AI
“There'll be some CEO system, and then, like, the responsible decision will be to, you know, like, have that thing be the CEO, and then, so that will happen at some point.”
Steinberger: AGI can be reduced to an AI that writes code and tests ideas
“If you your end goal is to have a system that can do everything you can reduce that to building a system that can build that system. And so that minimal system is a system that writes code and comes up with ideas and can validate those by writing code and runn…”
Steinberger: Long context windows are fundamentally superior to RAG
“Retrieval selects a subset of data for one completion. Our model sees all the data all the time. Clearly, a subset of data for the whole completion is a subset of all the data all the time. So if retrieval was optimal, our system could learn it. And it just tu…”
Steinberger: Pure training compute will fail to solve frontier intellectual problems
“Even the best mathematicians in the world, for the frontier of mathematics, require a long time to solve the problem. So, so, so, like, I would love to have a Terence Tao in my computer, but I would then still need to run Terence Tao for a year of human thinki…”
Steinberger: AI alignment is only solvable via recursive automated models
“The only way to sort of reasonably approach this is to iteratively ask your model to solve alignment and safety at that stage, not, not, you know, surely you can also ask it to solve your product level problems, but like that, that's nice, but that's not the f…”
Steinberger: AI coding tool adoption will be a bimodal step-function
“I just genuinely think that there is a gigantic market that gets unlocked in a step function moment where users decide that they're no longer going to use VS code to write code and send it to their colleagues. They're going to use magic or whoever ends up doin…”