Sep 25, 2025 · 53m · a16z
From Vibe Coding to Vibe Researching: OpenAI’s Mark Chen and Jakub Pachocki
0:00 / 0:00
▶ Watch on YouTube →
gold bands on the timeline = statements, start to end. Hover to read, click to jump. CC turns on captions
OpenAI leaders Jakub Pachocki and Mark Chen join The a16z Podcast to discuss the evolution of reasoning models, the future of AI-driven scientific discovery, and the strategic management behind frontier AI research.
How this conversation actually went
Every chapter scored 0–10 on four independent dynamics. Hover any point for the reasoning behind the score. How this is scored →
The host as informed peer 3.0 Guest teaching 3.0 Guest disagreement 0.7 The host pushing back 1.0
speaking balance: gold is the host, purple is the guest (3 minute bins)
Sharpest disagreement ▶ 24:39 Jakub rejects host's tension premise
Jakub explicitly rejects the host's framing that research conviction and truth-seeking exist in zero-sum tension, clarifying that deep belief in an idea can coexist with objective progress tracking.
Hardest push from the host ▶ 36:33 Host introduces Google's Nano Banana counter-exampleThe host directly challenges OpenAI's research priorities by raising Google's Nano Banana image model, questioning whether OpenAI risks ignoring valuable media generation breakthroughs.
Biggest teaching moment ▶ 2:44 Jakub reframes eval metrics and RL reasoningJakub educates the host on how pre-training benchmark evaluations have reached saturation, explaining how RL-driven domain reasoning fundamentally changes progress measurement.
The host holds their own ▶ 24:11 Host draws on bioinformatics grad school experienceThe host cites her own background as a bioinformatics researcher in graduate school to frame a sophisticated question about the psychological trap of going native on a research problem.
the scores for every segment, with the reasoning behind each
| Chapter | Topic | The host as informed peer | Guest teaching | Guest disagreement | The host pushing back | Why |
|---|---|---|---|---|---|---|
| GPT-V and Bringing Reasoning to the Mainstream | 3 | 4 | 1 | 1 | The host demonstrates familiarity with eval saturation percentages and AtCoder benchmark rankings. Jakub educates the host on how RL training in specific reasoning domains differs from traditional pre-training generalization. | |
| Lightbulb Moments and Daily Utility in Hard Sciences | 1 | 3 | 0 | 0 | The host prompts the guests for surprising lightbulb moments during internal testing. The guests explain how physicists and mathematicians experienced breakthrough utility in formula derivations. | |
| The Research Roadmap: Building the Automated Researcher | 4 | 3 | 1 | 2 | The host exhibits technical depth by questioning whether multi-step agentic tool use creates quality regressions compared to single-step execution. Jakub and Mark clarify that core reasoning capability provides the necessary stability across long horizons. | |
| Applying Reasoning to Open-Ended and Unverifiable Domains | 3 | 4 | 1 | 2 | The host brings up industry skepticism regarding RL plateaus, synthetic data mode collapse, and eval saturation. Jakub reframes the issue by detailing OpenAI's historical approach to RL environments and language pre-training integration. | |
| Reward Modeling and the Evolution of AI Learning | 2 | 3 | 0 | 0 | The host asks practical questions about enterprise reward modeling and latency presets for coding models. Jakub advises shifting mindsets toward simpler, more human-like learning paradigms. | |
| From Competitive Coding to Vibe Coding and Vibe Researching | 4 | 2 | 0 | 0 | The host demonstrates domain context by comparing AI coding adoption to Lee Sedol's retirement after losing to AlphaGo. The guests reflect collaboratively on personal coding shifts and the rise of high school vibe coding. | |
| Research Mindset, Problem Selection, and Overcoming Obstacles | 5 | 4 | 2 | 2 | The host leverages her bioinformatics research background to ask about the tension between research conviction and truth-seeking. Jakub politely rejects the premise, asserting that conviction and truth-seeking are not in zero-sum tension. | |
| Recruiting, Retention, and Fostering Research Culture | 3 | 3 | 1 | 1 | The host introduces external commentary by citing Elon Musk's tweet on the researcher versus engineer distinction. Mark nuances the topic by outlining distinct research archetypes, such as ideators versus rigorous experimenters. | |
| Aligning Research Strategy with Product Vision | 2 | 2 | 0 | 1 | The host explores how leadership protects fundamental research while integrating top product executives. Mark and Jakub outline structural mandate clarity and shared company-wide alignment. | |
| Compute Allocation and Portfolio Prioritization | 4 | 3 | 1 | 2 | The host challenges research priorities by citing Google's viral Nano Banana image model as a potential distraction from pure reasoning. Mark responds by emphasizing strict compute portfolio management and clear focus on winning core bets. | |
| Academia, Frontier AI, and the OpenAI Residency | 3 | 2 | 0 | 0 | The host contrasts the historical role of university research labs with modern corporate frontier AI orgs. Mark discusses the OpenAI Residency program designed to accelerate PhD-level AI intuition. | |
| Navigating External Perception and Long-Term Research Conviction | 2 | 3 | 1 | 1 | The host asks whether short-term external product reception influences long-term research roadmaps. Jakub explains that core research operates from strong internal conviction rather than external perception cycles. |
Statements from this episode (24)
Pachocki: OpenAI's primary research target is building an automated researcher
“The big thing that we are targeting is producing an automated researcher. So automating the discovery of new ideas, the next set of evals and milestones that we're looking at will involve actual movement on things that are economically relevant.”
Chen: The future of AI will center on reasoning and agents
“So we think the future is about reasoning, more and more about reasoning, more and more about agents.”
Pachocki: Standard AI evaluation benchmarks are close to saturation
“One thing is that indeed for like these e-files that we've been using for the last few years, they're indeed pretty close to saturated.”
Chen: Competitive coding and math benchmarks predict future research success
“I mean, I think it is important to note that these evals like you know, IOI, AtCoder, IMO are actually real world markers for success in future research.”
Chen: GPT-5 Pro automates tasks that take physics students months
“And you know, we see physicists, mathematicians kind of repeating this experience over and over where they're trying GPT-Five Pro and saying, wow, this is something that the, you know, previous version of the models couldn't do. And it is a little bit of a lig…”
Pachocki: AI progress over the next year will dwarf current gains
“But I expect that well, now as we're seeing you know, these models like actually able to automate, well, yes, like we're saying solving contest problems over, over longer time horizons. I expect that that is, well, that's, that, that was quite small compared t…”
Pachocki: Current OpenAI models can perform one to five hours of reasoning
“And so now, as we kind of, like, get to a level of near mastery of this high school competitions, let's say, I would say, like, we get to, like, maybe on, on the order of one to five hours, Of reasoning.”
Chen: Reasoning provides essential robustness for long-horizon AI agents
“Reasoning is core to this ability to operate over a long horizon, because, you know, you imagine kind of yourself solving a math problem, where you try an approach, it doesn't work, and, you know, you have to think about, you know, what, what's the next approa…”
Pachocki: Long-horizon AI research blurs verifiable and open-ended domains
“I think if you actually truly want to extend to research and, you know, finding, discovering ideas that, that meaningfully advanced technology on the, on, you know, the scale of like months and years, like I think the, these questions like stop being so differ…”
Pachocki: AI reward modeling will evolve toward simpler, human-like learning
“I expect this will evolve quite rapidly. I expect it will become simpler, right? Like I think, you know, maybe like two years ago, we would have been talking about like, what is the right way to craft my fine tuning data set? And I don't think we are like at t…”
Chen: Earlier Codex models spent too little time on hard problems
“What we found is the latest, the previous generation of the codex models, they were spending too little time solving the hardest problems and too much time solving the easy, easy problems. And I think that, that is actually just probably out of the box what yo…”
Pachocki: AI models will soon conquer the hardest math and programming problems
“If you look at things like Well, I guess the IMO problem six, or maybe some very hardest programming competitions problems. Like, I think there's still a little bit of headway to go for the models, but I wouldn't expect that to last very long.”
Chen: The future of scientific discovery will be 'vibe researching'
“I do think, you know, the future hopefully will be vibe researching.”
Pachocki: Researchers fail by trying to prove ideas instead of seeking truth
“I think a trap many people fall into is going after the way to like, to prove that it works. Right. Which is quite different from, you know, like I think like believing in your idea and significant is extremely important, right? And you want to persist, persis…”
Chen: Direct reports at OpenAI are unaffected by tech talent wars
“Like when I look at my direct reports they haven't been affected by the talent wars.”
Pachocki: Many of OpenAI's best researchers came from non-AI backgrounds
“A lot of our most successful researchers have started their journey with deep learning at OpenAI and have worked in other fields like physics or computer science, theoretical computer science or finance in the past.”
Chen: OpenAI historically prioritized algorithmic advances over product research
“I think historically we've put a little bit more on just the core algorithmic advances versus kind of the product research.”
Chen: AI labs risk second-place outcomes without strict strategic prioritization
“I think the danger is you end up like second place at everything and, you know, not like, you know, clearly leading at anything. So I think prioritization is important, right? And you need to make sure there's some things you're clear eyed on. This is the thin…”
Pachocki: AI progress will remain compute-constrained rather than data-constrained
“I haven't really bought that much into the, like, will be data constraint claim. And yeah, I don't expect that to change.”
Chen: No research leader at OpenAI feels they have enough compute
“There's no one who's like, ah, you know, I have all the compute that I need.”
Pachocki: AI research ideas succeed more often today because deep learning works
“Currently the pace of progress is very fast. Maybe also the ideas tends to work out a little bit more often than they did in the past. Because yeah, deep learning just wants to learn”
Pachocki: OpenAI does not base long-term research on short-term product reception
“So we generally, like, have some pretty strong convictions about the future, and so we don't tie them that closely to, like, the short-term reception of our products, right?”
Pachocki: Energy and robotics physical constraints will soon become major AI focuses
“I think more broadly than compute, there is physical constraints of, well, energy, but also like, you know, at some point, not too far, like robotics will become a major focus. And so so I think thinking about like the physical constraints is, is, is going to …”
Chen: Jakub Pachocki should be ranked as the top AI researcher
“I think, you know any of these rank lists, like he should be number one. Like just his ability to, you know, take any very difficult technical challenge and almost like personally just kind of think about it for two weeks and just crush it.”