Apr 15, 2025 · 45m · latent-space
GPT 4.1: The New OpenAI Workhorse
0:00 / 0:00
▶ Watch on YouTube →
gold bands on the timeline = statements, start to end. Hover to read, click to jump. CC turns on captions
OpenAI researchers Michelle Pokrass and Josh join the Latent Space podcast to detail the GPT-4.1 model family, highlighting advances in post-training, software engineering, one-million-token context reasoning, and developer tooling. They provide actionable guidance on prompt engineering, benchmark evaluations, model selection, and cost optimization for production AI workflows.
How this conversation actually went
Every chapter scored 0–10 on four independent dynamics. Hover any point for the reasoning behind the score. The hosts hold 36.2% of the talking time here. How this is scored →
The hosts as informed peer 4.8 Guest teaching 3.4 Guest disagreement 0.8 The hosts pushing back 1.7
speaking balance: gold is the hosts, purple is the guest (3 minute bins)
Sharpest disagreement ▶ 24:20 Dismissing the claim that JSON is deprecated for XML
Michelle immediately rejects Swyx's playful assertion that JSON is now bad and everyone must switch to XML, clarifying the distinct separation between prompt structuring inputs and parsed outputs.
Hardest push from the hosts ▶ 37:41 Challenging the GPU reclamation rationaleSwyx directly challenges OpenAI executive messaging about reclaiming GPUs via deprecations, pointing out that concurrent model hosting over three-month windows actually increases immediate resource usage.
Biggest teaching moment ▶ 39:28 Differentiating RFT from preference fine-tuningMichelle and Josh correct Swyx's mistaken assumption that preference fine-tuning is restricted to reasoning models, clearly explaining the difference between paired preference tuning and reinforcement fine-tuning.
The host holds their own ▶ 14:13 Referencing CogEval literature on graph walksSwyx demonstrates deep domain knowledge by connecting OpenAI's newly released synthetic graph benchmarks to prior research in the CogEval paper at NeurIPS regarding agent planning.
the scores for every segment, with the reasoning behind each
| Chapter | Topic | The hosts as informed peer | Guest teaching | Guest disagreement | The hosts pushing back | Why |
|---|---|---|---|---|---|---|
| Unveiling the GPT-4.1 Model Family and Launch Lore | 4 | 1 | 0 | 1 | Swyx opens with conversational context regarding OpenAI's pre-release code names on OpenRouter and the Waterloo engineering connection. Michelle and Josh explain the high-level release details and the inclusion of the nano tier. The dynamic is friendly and collaborative. | |
| Decoupling Model Names and Post-Training Research Architectural Insights | 5 | 4 | 1 | 2 | Swyx probes the naming logic, model size comparisons with GPT-4.5, and Omni architecture lineage. Josh and Michelle clarify that versioning reflects user capability improvements and post-training gains rather than linear pre-training size scaling. | |
| Achieving One Million Context Windows Through Graph Reasoning Benchmarks | 6 | 3 | 1 | 2 | The hosts and guests discuss 1M context evaluation, where Josh details synthetic graph walk tasks. Swyx cites previous research literature on graph traversals for agent planning (CogEval at NeurIPS) to frame the discussion. | |
| Real-World Developer Instruction Following and API Data Evaluation | 5 | 3 | 0 | 1 | Swyx asks about Shen Yu's work and internal API instruction-following evals. Michelle explains the limitations of open-source evals with programmatic grading versus real-world developer prompts with complex negative constraints. | |
| Effective Prompt Engineering Guidelines and Agentic Workflow Persistence | 6 | 4 | 2 | 3 | Swyx challenges the prompt guide recommendation of placing instructions at both the top and bottom of the context because of conflicts with prompt caching. Josh and Michelle clarify how caching can still work depending on prompt structure and dispute the idea that JSON is deprecated in favor of XML. | |
| Navigating Model Selection Between GPT-4.1 and Reasoning Models | 4 | 3 | 0 | 1 | Alessio and Swyx ask how developers should choose between GPT-4.1 chain-of-thought prompting and dedicated reasoning models like o1. Michelle outlines a tiered heuristic based on latency, cost, and planning horizon. | |
| State-of-the-Art Coding Capabilities and Internal OpenAI Engineering Workflows | 5 | 2 | 1 | 1 | Swyx asks about SWE-bench benchmarks and breakdowns across diff generation and full-repo exploration. Michelle shares internal developer metrics and anecdotes about massive PR commits completed by 4.1. | |
| Multimodal Vision Advancements and Infrastructure Optimization | 5 | 3 | 1 | 3 | Swyx questions the premise that deprecating GPT-4.5 reclaims GPUs when OpenAI supports old models concurrently for months. Michelle explains the balance between compute reclamation and developer API reliability commitments. | |
| Day-One Fine-Tuning Offerings and Upcoming Model Horizons | 4 | 6 | 1 | 1 | Michelle highlights preference fine-tuning for style steering, which Swyx confuses with reinforcement fine-tuning (RFT). Michelle and Josh politely correct him, clarifying that RFT is strictly for reasoning models. | |
| Developer Ecosystem Feedback, Caching Pricing Reductions, and Concluding Remarks | 4 | 5 | 1 | 2 | Swyx asks about blended pricing and assumes all models received a price drop. Michelle corrects the pricing assumption regarding mini and highlights the newly increased 75% prompt caching discount. |
Statements from this episode (24)
OpenAI launches GPT-4.1 model lineup featuring 1M-token context window
“Yeah, I'll just say we released three new models today, GPT-Fort.one, GPT-Fort.one mini, and GPT-Fort.one data, and the real focus on these were just making the models that were great for developers so we improved instruction following, coding, and shipped our…”
OpenAI stealth-tested GPT-4.1 models on OpenRouter before official release
“Yeah yeah, we really wanted to get as much developer feedback as possible on this model to make sure it worked well in the real world, and so we tested it kind of through Open Router and it was super cool to see people latch on to the names and get the theorie…”
OpenAI has no current plans to add GPT-4.1 to Realtime API
“I don't think we don't have any current plans to release 4.1 in the real time API, but you know, things, things may change.”
GPT-4.1 Nano and Mini are new pre-trains; base 4.1 is mid-train
“Nano is obviously a new pre-train. We also have a new pre-train for Mini, and then, ah, the larger version is, ah, a new mid-train.”
Pokrass: AI model gains now driven by post-training, not larger pre-trains
“We find that actually a significant amount of the gains come from new post-training techniques. So I think in the past the narrative is that you need to pre-train these larger and larger models to get better performance, and we're finding that we're able to sq…”
Pokrass predicts developers will abandon RAG vector stores for direct long-context
“So we do expect a lot of developers to start, you know, uploading their full context more directly to the model. So for smaller tasks, you maybe don't need The whole vector store.”
GPT-4.1 powers OpenAI API while enhanced memory remains ChatGPT-exclusive
“So, 4.1 is powering the API, whereas the enhanced memory is, is ChatGPT only.”
Pokrass warns against close collaboration between AI evaluators and model developers
“Honestly, I think it's best when eval authors and model developers don't collab too much because you want things you know, as objective as possible, not trying to game any evals.”
Open-source AI benchmarks omit critical tasks because they are hard to grade
“And these are useful instructions, but we find that many of the really interesting instructions are actually challenging to grade. And so the open source evals often don't have them.”
OpenAI uses its own models to categorize anonymized developer prompts
“Well, I will say we do use our own products internally where we can, and so we're not manually by hand reading every prompt. After they're like anonymized, we scrub them with any identifying data, then we use our models to take passes to categorize them.”
GPT-4.1 reduces extraneous edit rate to 2%, down from GPT-4o's 9%
“And we found that from four O, which got nine percent, which is pretty crazy, nine percent of the time making an extraneous edit is a lot. 4.1 is at two percent, so it's a pretty big improvement.”
Pokrass: Use XML for structuring LLM inputs and JSON for parsing outputs
“I do think XML is very helpful for structuring prompts, whereas for parsing outputs maybe the story is a bit different. Like sometimes it's really useful to get outputs in JSON, so you can plug them directly into your application. But I do think the models wor…”
GPT-4.1 significantly improves chain-of-thought planning over previous non-reasoning models
“We have found that 4.1 is a lot better at doing planning and thinking through its steps in COT when prompted than our previous non-reasoning models.”
Pokrass: Prototype with GPT-4.1, then downscale for latency or upscale for reasoning
“I think the answer is always going to be the fastest model that accomplishes your task, right? So maybe you start prompting 4.1 as a starting point if it does your task super well, Then maybe you could drop down a 4.1 mini and save latency, or even nano. Where…”
Pokrass: Pair reasoning models for planning with smaller models for execution
“I do think reasoning models for planning and using kind of more targeted models to execute is definitely a good architecture.”
GPT-4.1 excels at exploring repositories, while reasoning models dominate targeted file changes
“Basically, where GPT, 4.1, can it kind of explore, go through a repo? It's been trained to do that particularly well. Whereas you know, to just get some code and produce a change, a reasoning model might do better because it can kind of reason over the entire …”
GPT-4.1 Mini significantly outperforms 4o Mini, nearing original GPT-4o performance
“4.1 mini is actually quite significantly better than four o mini and not that far away from the old four o.”
OpenAI researcher uses GPT-4.1 for 49 of 50 commits on massive PR
“I was actually just talking to one of the researchers on the team who worked on something over the weekend. And he said that this model, GBT, 4.1 was able to like get 49 out of 50 of his commits on this massive PR done.”
GPT-4.1's multimodal vision improvements stem from pre-training, not post-training
“We talked about like coding instruction following long context, a lot of gains coming from post training, but in particular multimodal, like basically everything you're seeing, the gains are there from pre-training.”
Pokrass: Developers are sleeping on preference fine-tuning for model style steering
“One thing I will say is that I think people have slept on the preference fine tuning offering or the, I think that's what we call the product. So SFT is, people know it pretty well. It's the original fine tuning we had, whereas this preference fine tuning is s…”
OpenAI currently restricts reinforcement fine-tuning exclusively to its reasoning models
“No, that's reinforcement fine tuning is only for reasoning models.”
OpenAI is working to inject GPT-4.5's humor and nuance into future models
“We're working on incorporating kind of those improvements into the models more generally. People loved about 4.5 is like the humor, the green text, the nuance. So we've heard that feedback and I know, yeah, there's lots of folks working on that and trying to b…”