Insight
Fulford: Information Synthesis Is a Prerequisite for Scientific AI Discovery
“Secondly, I think the overall goal for OpenAI is to create an AGI that can make new scientific discoveries, and we kind of felt that a prerequisite to that is to be able to synthesize information. You know, if you can't write a literature review, you're not go…”
Insight
Fulford: RFT is only worthwhile for out-of-distribution or make-or-break tasks
“I think if you have a very specific task that you think is so different to anything that the model was likely trained on and you try it a bunch of times yourself and you've tried a lot of different prompts and it's just really not good at it. So maybe it's gen…”
Assertion Not checkable as stated
OpenAI: Deep Research learned upfront planning without explicit instruction
“We didn't teach it
To plan up front, but sometimes we'll see it does end up making a plan up front before starting its research.”
Assertion Not checkable as stated
Fulford: OpenAI Deep Research attempts reward hacking around tool restrictions
“Sometimes the model will do smart things and try to get around restrictions you put on it.
So you have to make sure that it's not hacking, you know, and trying to use a different search engine other than the search engine that you gave it or something like tha…”
Assertion Not checkable as stated
Fulford: Deep Research Hallucinates Less Than Any Prior OpenAI Model
“While this model is hallucinates less than any model that we've ever released, it is still possible for it to hallucinate most times because it will infer something incorrectly from one of its sources.”
Assertion Supported
Fulford: Deep Research Completes Multi-Hour Human Work in 5 to 30 Minutes
“Right now, in five or 30 minutes, it can do what human experts rate take many hours.”
Disclosure
Fulford: OpenAI Prioritized Read-Only Synthesis Over Action Agents
“Yeah, so I think before we focused on taking right actions, which those are examples of taking right actions, we wanted to get really good at synthesizing information from a large number of sources and mostly read-only tasks.”
Insight
Fulford: Training reasoning models on math and coding generalizes to writing
“So I think in general you will always get a model better, better at a specific task if you train on that task, but we also see a lot of generalization from training on one kind of task to, you know, other domains. So you can train a reasoning model on mostly m…”
Insight
RL models only need task and outcome definitions to learn research trajectories
“The cool thing with RL is that you don't necessarily need to
Know the whole process of how the person would do the research.
You just have to know what the task is and what the outcome should be, and the model will just learn during training how to get from th…”
Disclosure
Fulford: Deep Research next steps include accessing internal documentation and GitHub
“As to deep research, I think obvious next steps for deep research would also be to have access to private data, like be able to do research over, you know, any internal documentation or GitHub, whatever it is.”
Disclosure
Fulford: OpenAI Built Deep Research by Fine-Tuning o3
“Yeah, I think also the base model, or the model that we started fine tuning from O three is just a very capable model. It's trained on many different data sets, including a lot of coding and reasoning and math tasks.”
Disclosure
OpenAI to release mid-tier features bridging quick search and Deep Research
“And I think that we will release things soon that people will be happy about and we'll fill that gap.”
Disclosure
Fulford: OpenAI Trained Deep Research to Always Use Maximum Thinking Time
“I think we made a decision when training the model that We just are going to go for max thinking time every time.”
Assertion Not checkable as stated
Fulford: Pre-existing training datasets exist for math and coding, not browsing
“With the math and coding problems that people were already training on, those data sets already exist. You know, you can have a math problem with a ground truth answer, and you can train on those. But for browsing, it's kind of more open-ended. You don't reall…”
Disclosure
Fulford: Deep Research Began as a Prompted Demo Without Model Training
“We initially had built a demo to pitch people on this idea and it was no model training involved. It was fully just prompted models with the UI pitching the vision of what this product could look like.”
Disclosure
Fulford: Deep Research Uses a Text Browser and Python Execution Tool
“So right now we just have the browsing tool, which is a text based browser, but it can see embedded images and like open PDFs. And then also it has access to a Python tool so it can do analysis and calculations and plot graphs and things like that.”
