Dubois: Larger AI models achieve higher efficiency by thinking through weights
Yann Dubois · OpenAI's Yann Dubois: Why AI Progress Suddenly Feels Real · May 21, 2026 · at 24:34
Yann Dubois, Post-Training Frontiers Co-Lead at OpenAI, discusses the relationship between pre-training model scale, explicit thinking tokens, and inference efficiency.
0:00 / 0:39exact quote · 39.2s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“If you have larger models the amount of thinking time, so the amount of tokens they will think for will usually decrease. And the way that you can think about it is that metaphorically, the model already thinks through its weights when it generates a certain token. So you can decrease the number of, like, tokens that it needs to generate for thinking by kind of, like, increasing the size of the model. That you are training. So, so oftentimes if you just increase the model size, if you basically train pre-train larger models you will get better efficiency.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Yann Dubois
Dubois: AI progress feels discontinuous because OpenAI crossed a reliability threshold
“Even though the, in my mind, everything, the progress is actually pretty continuous, you need to reach this level of reliability. To really make any of these AI tools very useful, and I think we just crossed that probably December last year, at least at OpenAI…”
Dubois: Test-time compute scaling exhibits logarithmic, diminishing returns
“We, we've seen again and again, the longer the model think for the better answers we will get. The problem is that this, these curves that we're talking about are not, are definitely not linear, and like they, there's some plateauing effect, and they kind of l…”
Dubois: AI frontier labs have successfully bypassed internet data walls
“There were a lot of conversation about hitting data walls, and it seems like we did not quite hit it. So the larger the model is, the more data it needs to ingest to be trained. And it seems like different companies kind of found different ways to overcome the…”
Dubois: Simulations will never fully eliminate the need for real-world AI training
“The problem is simulations are always going to be really hard and are not going to be truthful. So I think there will always need to be a certain, a little bit of training that will need to happen in the real world to make sure that the model realizes kind of …”
Dubois: RL becomes effective once base models possess strong world priors
“It seems that after crossing a certain scale of models that know basically everything about the world, and what we call, like, good priors about the world, It seems that reinforcement learning just started to work, and this is not only with LMS. Robotics seems…”
Dubois: AI model knowledge calibration generalizes across all domains
“When you have hallucination of LMs, if a model is really bad at saying that it doesn't know, that usually happens in every single domain. You won't have, like, one domain where the model is extremely calibrated about its knowledge, and another domain where it'…”