OpenAI co-founder Greg Brockman explains the rationale behind hiding raw reasoning traces in reasoning models.
0:00 / 1:04exact quote · 64.1s
720p mp4 · rendered on demand · StarZero watermark
“So there's two reasons. One is to think about distillation, but the second, in some ways more important, is that we had this insight when we first developed the reasoning paradigm that it gives us a interpretability mechanism we had not been anticipating, because you can really read the model's thoughts. You can see exactly how it got to an answer, so you can interpret How, like, what was actually motivating that answer? Now, the problem is, if you train the model to have a chain of thought that looks good, then you lose all the faithfulness, right? It's just going to be like, the model knows that part of the answer that is desired is for the chain of thought to look a certain way, and so it may not be representative of how it actually arrived at that answer anymore. And so we were, we made an early decision to say we want to avoid any temptation to train these chain of thoughts to look at Favorable to look like something you could present to a user, and so that really made us lean out for multiple reasons, for competitive reasons, for safety reasons, from the idea of showing these intermediate thoughts.”
quote is from the automated transcript, cleaned for reading:
filler sounds and stutters are removed, nothing is rephrased. names can be misheard
(the analysis reads context, assessments check outside sources). how →
More from Greg Brockman
AssertionSupported
Brockman: Elon, Sam, Ilya, and I Agreed OpenAI Needed For-Profit Structure
“Elon, Sam, Ilya, and I all agreed. That the only path forward for OpenAI, the only path to achieve the mission, was to create a for-profit entity associated with OpenAI of some form. And so we were committed to that direction, and that is something that we kne…”
Brockman: OpenAI board gave zero explanation when firing Altman and removing Brockman
“I was told that the board has decided That Sam would be removed, and effectively the message that I got was the same messaging that was in the public post, and I asked if I could have any more information. I was told no, not right now, and I pressed on that ma…”
Brockman: Predicting Einstein's Next Words Requires Intelligence Equal to Einstein
“On the one hand, just predicting what comes next sounds like a pedestrian task, but if you really can predict the next word out of Einstein's mouth, you are at least as smart as Einstein.”
Brockman: OpenAI models recently resolved an open quantum physics problem
“If you look at math and physics, we now are solving open math problems. We're solving Open physics problems and actually have resolved this particular physics problem recently in quantum physics in the opposite way that the community expected.”
Brockman: Claims that OpenAI data centers consume massive water are misinformation
“Like a good example is data centers and water usage. Like that, that's something that people talk about a lot, but actually our data centers use incredibly little water, right? That's actually misinformation that they use a lot.”
This entire site, over 200 episodes transcribed, diarized, checked and made playable,
runs on the StarZero media pipeline. Drop in your own episode and the podcast clipper finds the
moments worth sharing, cuts them, captions them, and reframes them for every feed.
We use essential cookies to make the site work. With your permission we
also use analytics cookies (Google Analytics and Mixpanel) to understand
usage and improve StarZero. See our Cookie Policy.