McGrath: DeepSeek Math's real breakthrough is verifiable reward trust, not GRPO
Josh McGrath · [State of Post-Training] From GPT-4.1 to 5.1: RLVR, Agent & Token Efficiency — Josh McGrath, OpenAI · Dec 31, 2025 · at 12:42
OpenAI post-training researcher Josh McGrath evaluates the impact of DeepSeek Math and verifiable rewards compared to academic focus on optimization algorithms.
0:00 / 0:27exact quote · 27.6s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“As you said, it came out in the deep seek math paper, and like, it's an interesting optimization method, but it's like the more interesting thing that they have a new reward signal that they sort of like re that we can really, really trust. Like when, you know, you find the answer to a math problem, it's a lot less debatable than like, oh, well, is this thing that the human preferred actually what we want to do? Like you want to be right at math. And so I think in some ways that's underappreciated in I would say what's getting published.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Josh McGrath
McGrath: Specialized and Frontier Reasoning AI Models Will Eventually Converge
“You know, I think if you look at like deep research, the original one and GPT-Five thinking on like high reasoning today, I think you'll see that like eventually the models all sort of converge in their capabilities.”
McGrath: RLHF and RLVR differ by data quality, not optimization math
“Really, at the end of the day, like, RLHF, RLVR,
They're both policy gradient methods, but the, what's different is just like the input data.”
McGrath: RL runs have far more infrastructure failure points than pre-training
“The issue with RL is, like, you're doing tasks, and each task could have, like, a different grading setup, and each one of those different grading setups, that's, like, more infrastructure, and so, You know, when I'm staying up late trying to figure out what's…”
McGrath: Design specs let Codex complete hours of coding in 15 minutes
“If I spend, like, you know, 30, 40 minutes writing something that looks like a design doc or something, Codex can do more work than I can do in a few hours in, like, 15 minutes.”