Brockman: LLMs consistently generalize to untrained preferences
Greg Brockman · Greg Brockman on OpenAI's Road to AGI · Aug 15, 2025 · at 38:56
OpenAI President Greg Brockman discusses reinforcement learning feedback and deep learning's core ability to generalize across subjective human preferences.
0:00 / 0:15exact quote · 15.1s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“In order to get them to be able to operate according to different preferences and values, we just need to show that to them during training, and they are able to sort of generalize to different preferences and values that we didn't actually train against, and that's something that we've seen very consistently across different model generations.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Greg Brockman
Brockman: AI models are reaching parameter counts comparable to human synapses
“It's a hundred T synapses, which kind of corresponds to the weights of the neural net. And so there's some sort of equivalence there. And so we're starting to get to the right numbers. Let me just say that.”
Brockman: Physicists say GPT-5 re-derived research insights taking months of work
“We've seen physicists starting to kick the tires on GPT-V and say that, like, hey, this thing was able to get, this model was able to re-derive an insight that took me many months worth of research to produce.”
Brockman: AGI will be a menagerie of models, not a single model
“The flip side, though, is that I think that the evidence has been away from having the final form factor, the AGI itself being a single model. But instead thinking about this menagerie of models that have different strengths and weaknesses.”
Brockman: Open-source AI creates tech stack dependency beneficial to OpenAI and the US
“Another thing at a very practical level that we've thought about with open source models is that people building on our open source model are kind of building on our tech stack, right? If you are relying on us to help improve the model that you're relying on u…”
Brockman: AI developer productivity gains will increase demand for engineers
“The productivity impacts of people being able to do more means we actually want more people, right? It's like we are so limited by the ability to produce software, so limited by the ability of our team to actually clean up tech debt and go and refactor things,…”
Brockman: AI infrastructure buildout dwarfs Apollo program and New Deal
“The engineering project that we collectively as a country, as a society, as a world are undergoing right now, right? It's like projects like the New Deal, like pale in comparison, you know, the Apollo program pale in comparison to what we're doing right now.”