Fulford: Training reasoning models on math and coding generalizes to writing
Isa Fulford · No Priors Ep. 112 | With OpenAI Deep Research, Isa Fulford · Apr 24, 2025 · at 7:23
OpenAI researcher Isa Fulford discusses AI model generalization and reinforcement fine-tuning with Sarah Guo on No Priors.
0:00 / 0:26exact quote · 26.4s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“So I think in general you will always get a model better, better at a specific task if you train on that task, but we also see a lot of generalization from training on one kind of task to, you know, other domains. So you can train a reasoning model on mostly math, coding, other reasoning kind of problems, and it will be good at writing, but if you know, trained on that specific task, it would be better at it.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Isa Fulford
Fulford: Information Synthesis Is a Prerequisite for Scientific AI Discovery
“Secondly, I think the overall goal for OpenAI is to create an AGI that can make new scientific discoveries, and we kind of felt that a prerequisite to that is to be able to synthesize information. You know, if you can't write a literature review, you're not go…”
Fulford: RFT is only worthwhile for out-of-distribution or make-or-break tasks
“I think if you have a very specific task that you think is so different to anything that the model was likely trained on and you try it a bunch of times yourself and you've tried a lot of different prompts and it's just really not good at it. So maybe it's gen…”
OpenAI: Deep Research learned upfront planning without explicit instruction
“We didn't teach it
To plan up front, but sometimes we'll see it does end up making a plan up front before starting its research.”
Fulford: OpenAI Deep Research attempts reward hacking around tool restrictions
“Sometimes the model will do smart things and try to get around restrictions you put on it.
So you have to make sure that it's not hacking, you know, and trying to use a different search engine other than the search engine that you gave it or something like tha…”