Pachocki: AI reward modeling will evolve toward simpler, human-like learning
Jakub Pachocki · From Vibe Coding to Vibe Researching: OpenAI’s Mark Chen and Jakub Pachocki · Sep 25, 2025 · at 15:26
Jakub Pachocki is Chief Scientist at OpenAI. He addresses how enterprises and domain experts should view the evolution of reinforcement learning and reward modeling techniques.
0:00 / 0:22exact quote · 22.4s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“I expect this will evolve quite rapidly. I expect it will become simpler, right? Like I think, you know, maybe like two years ago, we would have been talking about like, what is the right way to craft my fine tuning data set? And I don't think we are like at the end of that evolution yet. And I think we will be inching towards more and more human-like learning which, you know, RL is still not quite.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Jakub Pachocki
Pachocki: OpenAI's primary research target is building an automated researcher
“The big thing that we are targeting is producing an automated researcher. So automating the discovery of new ideas, the next set of evals and milestones that we're looking at will involve actual movement on things that are economically relevant.”
Pachocki: Standard AI evaluation benchmarks are close to saturation
“One thing is that indeed for like these e-files that we've been using for the last few years, they're indeed pretty close to saturated.”
Pachocki: AI models will soon conquer the hardest math and programming problems
“If you look at things like Well, I guess the IMO problem six, or maybe some very hardest programming competitions problems. Like, I think there's still a little bit of headway to go for the models, but I wouldn't expect that to last very long.”
Pachocki: AI progress will remain compute-constrained rather than data-constrained
“I haven't really bought that much into the, like, will be data constraint claim. And yeah, I don't expect that to change.”
Pachocki: AI progress over the next year will dwarf current gains
“But I expect that well, now as we're seeing you know, these models like actually able to automate, well, yes, like we're saying solving contest problems over, over longer time horizons. I expect that that is, well, that's, that, that was quite small compared t…”