Reinforcement Learning Fails Without Initial SFT to Seed Rewardable Behaviors
Misha Laskin · Solve coding, solve AGI [Reflection.ai launch w/ CEO Misha Laskin] · Mar 7, 2025 · at 11:38
Reflection AI CEO Misha Laskin explains why supervised fine-tuning (SFT) and reinforcement learning must be combined when training autonomous coding agents.
0:00 / 0:26exact quote · 26.3s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“It can potentially work otherwise, but practically it only works when the agent has interacted with a reward, right? It's received a positive reward for what it's done. Maybe one out of 10 times, one out of 50 times, but if it's getting zero reward, then you don't really know what behavior to amplify. And so that's why it's really important in the kind of initial mixture for it to already be doing things out of the box that are somewhat sensible, even if they're unreliable.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Misha Laskin
Solving Autonomous Coding Is the Direct Path to AGI
“Our core belief is that if you solve this problem, you solve the autonomous coding problem and build a super intelligent coding agent, that that thing will lead to super intelligence more broadly.”
Superintelligence Cannot Be Trained Entirely From Scratch
“In the era of language models, I don't think you'll be able to train superintelligence from scratch.”
Future UIs Will Be Built as Programmatic Interfaces for AI Models
“Over the coming years, there'll be more kind of AI friendly or language model friendly UIs. And what's friendly to a language model is, is code. So the way a model will be doing work, not just for coding and software engineering, Is by basically making functio…”
Frontier Labs May Hoard Superintelligent Models and Release Nerfed Versions
“You can imagine you know, the world converging on a few companies have really powerful coding models. They basically release a nerfed version of that to the public at large, and basically have a competitive advantage by having, you know, a super intelligent co…”
Gemini 1 Proved GPT-4-Level Models Can Bootstrap Reinforcement Learning
“Giannis and I led a lot of the work for post-training and kind of RL check for Gemini, and Giannis being my co-founder, and when we shipped Gemini One, we just realized that the models, like, models that were basically at GPT-IV level or above, were capable en…”
Software Engineering Is Ergonomic for LLMs, Making It the Ideal Wedge
“Our belief as a company is that the correct wedge in, the correct starting point to this entire problem is decoding agent, because it's already, you know, software engineering is already what I would call kind of ergonomic for a language model.”