Prediction Not checkable as stated
Pachocki: OpenAI's primary research target is building an automated researcher
“The big thing that we are targeting is producing an automated researcher. So automating the discovery of new ideas, the next set of evals and milestones that we're looking at will involve actual movement on things that are economically relevant.”
Assertion Not checkable as stated
Pachocki: Standard AI evaluation benchmarks are close to saturation
“One thing is that indeed for like these e-files that we've been using for the last few years, they're indeed pretty close to saturated.”
Prediction Not checkable as stated
Pachocki: AI models will soon conquer the hardest math and programming problems
“If you look at things like Well, I guess the IMO problem six, or maybe some very hardest programming competitions problems. Like, I think there's still a little bit of headway to go for the models, but I wouldn't expect that to last very long.”
Prediction Not checkable as stated
Pachocki: AI progress will remain compute-constrained rather than data-constrained
“I haven't really bought that much into the, like, will be data constraint claim. And yeah, I don't expect that to change.”
Prediction Not checkable as stated
Pachocki: AI progress over the next year will dwarf current gains
“But I expect that well, now as we're seeing you know, these models like actually able to automate, well, yes, like we're saying solving contest problems over, over longer time horizons. I expect that that is, well, that's, that, that was quite small compared t…”
Assertion Not checkable as stated
Pachocki: Current OpenAI models can perform one to five hours of reasoning
“And so now, as we kind of, like, get to a level of near mastery of this high school competitions, let's say, I would say, like, we get to, like, maybe on, on the order of one to five hours, Of reasoning.”
Insight
Pachocki: Long-horizon AI research blurs verifiable and open-ended domains
“I think if you actually truly want to extend to research and, you know, finding, discovering ideas that, that meaningfully advanced technology on the, on, you know, the scale of like months and years, like I think the, these questions like stop being so differ…”
Prediction Not checkable as stated
Pachocki: AI reward modeling will evolve toward simpler, human-like learning
“I expect this will evolve quite rapidly. I expect it will become simpler, right? Like I think, you know, maybe like two years ago, we would have been talking about like, what is the right way to craft my fine tuning data set? And I don't think we are like at t…”
Insight
Pachocki: Researchers fail by trying to prove ideas instead of seeking truth
“I think a trap many people fall into is going after the way to like, to prove that it works. Right. Which is quite different from, you know, like I think like believing in your idea and significant is extremely important, right? And you want to persist, persis…”
Assertion Not checkable as stated
Pachocki: Many of OpenAI's best researchers came from non-AI backgrounds
“A lot of our most successful researchers have started their journey with deep learning at OpenAI and have worked in other fields like physics or computer science, theoretical computer science or finance in the past.”
Opinion
Pachocki: AI research ideas succeed more often today because deep learning works
“Currently the pace of progress is very fast. Maybe also the ideas tends to work out a little bit more often than they did in the past. Because yeah, deep learning just wants to learn”
Disclosure
Pachocki: OpenAI does not base long-term research on short-term product reception
“So we generally, like, have some pretty strong convictions about the future, and so we don't tie them that closely to, like, the short-term reception of our products, right?”
Prediction Not checkable as stated
Pachocki: Energy and robotics physical constraints will soon become major AI focuses
“I think more broadly than compute, there is physical constraints of, well, energy, but also like, you know, at some point, not too far, like robotics will become a major focus. And so so I think thinking about like the physical constraints is, is, is going to …”
