GPT-4
also referred to as: gpt-iv
includes GPT 4 Turbo, GPT 4 With Vision, GPT 4 Vision
43 statements across 32 episodes · 17 bullish · 11 bearish · 29 people on the record · first statement Jun 20, 2023 by George Hotz · said 323 times in 87 episodes since 2023 · across every show →
Mentions by year, the whole family
brought up most by Shawn Wang (45), Nathan Lambert (22), Michael Royzen (15), Alessio Fanelli (14), Thomas Scialom (10), Dylan Patel (10), Alistair Pullen (9), Jerry Liu (6)
tap a year for its mentions
2026 21 mentions in 11 episodes 2 per episode
- Artificial Analysis: The Independent LLM Analysis House — with George Cameron and Micah Hill-Smith
- 🔬 From Red Teaming GPT-4 to Automating Drug Discovery: The Future of AI in Science — Andrew White
- Measuring Exponential Trends Rising (in AI) — Joel Becker, METR
- ⚡️ Reverse Engineering OpenAI's Training Data — Pratyush Maini, Datology
- ⏭️ Forward Deployed: Voice AI on what works in 2026
- Next 100x in AI: Inference, Networking, & Self-Optimizing Models — Philip Kiely & Ali Taha, Baseten
- Notion’s Sarah Sachs & Simon Last on Custom Agents, Evals, and the Future of Work
- ⚡️ The best engineers don't write the most code. They delete the most code. — Stay Sassy
- 3 more episodes that year, every mention in 2026 →
2025 76 mentions in 33 episodes 2 per episode
- Greg Brockman on OpenAI's Road to AGI
- 2024 Year in Review: The Big Scaling Debate, the Four Wars of AI, Top Themes and the Rise of Agents
- ⚡️Ranking Agentic LLMs — Pratik Bhavsar, Galileo
- ⚡ Inside GitHub’s AI Revolution: Jared Palmer Reveals Agent HQ & The Future of Coding Agents
- Why RL Won — Kyle Corbitt, OpenPipe (acq. CoreWeave)
- ⚡️Factorio Learning Environment: the ultimate Game Agent Eval — Jack Hopkins
- Information Theory for Language Models: Jack Morris
- Beating OpenAI and Anthropic by Looking At Data: the new #1 on SWE-Bench w/ W&B CTO Shawn Lewis
- 25 more episodes that year, every mention in 2025 →
2024 161 mentions in 34 episodes 5 per episode
- The Origin and Future of RLHF: the secret ingredient for ChatGPT - with Nathan Lambert
- Building AGI in Real Time (OpenAI Dev Day 2024)
- Agents @ Work: Dust.tt — with Stanislas Polu
- Training Llama 2, 3 & 4: The Path to Open Source AGI — with Thomas Scialom of Meta AI
- The Winds of AI Winter (Q2 Four Wars of the AI Stack Recap)
- [LLM Paper Club] Llama 3.1 Paper: The Llama Family of Models
- Supervise the Process of AI Research — with Jungwon Byun and Andreas Stuhlmüller of Elicit
- Is finetuning GPT4o worth it?
- 26 more episodes that year, every mention in 2024 →
2023 65 mentions in 9 episodes 7 per episode
- Beating GPT-4 with Open Source Models - with Michael Royzen of Phind
- The State of Silicon and the GPU Poors - with Dylan Patel of SemiAnalysis
- The "Normsky" architecture for AI coding agents — with Beyang Liu + Steve Yegge of SourceGraph
- RAG is a hack - with Jerry Liu of LlamaIndex
- The End of Finetuning — with Jeremy Howard of Fast.ai
- Ep 18: Petaflops to the People — with George Hotz of tinycorp
- Powering your Copilot for Data - with Artem Keydunov from Cube.dev
- RWKV: Reinventing RNNs for the Transformer Era
- 1 more episode that year, every mention in 2023 →
Everything said about GPT-4, oldest first
Jun 20, 2023 neutral
Oct 12, 2023 negative
Nov 3, 2023 positive
Royzen: Users switch to Phind when ChatGPT-4 fails on code
“What really shocks us is that a lot of the people who do that they're coming from ChatGPT. So they tried it in ChatGPT with ChatGPT-IV. It didn't work. Maybe it required like some multi-step reasoning. Maybe it required to like, Some internet context or someth…”
Nov 3, 2023 bearish
Nov 3, 2023 positive
Royzen: Training on code unlocked general spatial and temporal reasoning
“We've seen emerging capabilities in the find model, whereby training it on high quality code, it can actually, like, reason better. It went from not being able to solve like, World problems where like riddles where like with like temporal and like low, like pl…”
Nov 3, 2023 negative
Royzen: GPT-4 was trained on HumanEval, proving data contamination
“GPT-IV itself has been trained on human eval, and we know this because GPT-IV is able to predict the exact doc string in many of the problems. I've seen it predict, like, the specific example values in the doc string, which is extremely improbable for it to ju…”
Dec 5, 2023
Dec 5, 2023 neutral
Dec 5, 2023 bullish
Jan 11, 2024 neutral
Jan 11, 2024 positive
Lambert: GPT-4 achieves 80% preference labeling agreement versus 70% for humans
“Essentially, people also think that synthetic data is, like, GPT-IV is more accurate than humans at labeling preferences, so if you look at these diagrams, like, humans are about 60 to 70% agreement, or, like, that's what the models get to, and if humans are a…”
Jan 11, 2024 positive
Feb 7, 2024 negative
Apr 6, 2024
Apr 11, 2024
Byun: GPT-3 was a qualitative shift, while GPT-4 was an extension
“I think GPT-III was a big change because it kind of said, oh, now is the time to build to you that we can use AI to build these tools. And then GPT-IV was maybe a little bit more of an extension of GPT-III. It felt less like a level, GPT-III over GPT-II was li…”
Apr 24, 2024 negative
Apr 27, 2024 positive
Claude 3 can bypass its assistant persona to expose the underlying simulator
“Instead of having this entity, like GPT-IV, that's an assistant that just pops up in your face that you have to kind of, like, punch your way through and continue to have to deal with as a headache, instead, there's ways to kindly coax Claude into having the a…”
Jul 5, 2024 negative
Yi Tay: Distilled open-source model variants disappeared after failing to climb LMSYS
“When people realize that, like, this, like, turning on the GPT-IV tab and running some DPO is not going to give them the reward signal that they want anymore, right? Then all these variants gone, right? You know, there was this era where there's, wow, there's …”
Jul 23, 2024 positive
Scialom: Llama 3 405B is the best open-source model ever released
“At a high level, it's the best open source model ever. It's Better than GPT-IV. I mean, what version? But, by far, compared to the version originally released even now, I think there's maybe the last cloud Sonya FF-V and GPT-IV-Zero that are performing it.”
Aug 28, 2024 negative
Carlini: GPT-4 would exist identically without adversarial machine learning research
“Nothing about GPT-IV would be at all different if the field of, like the entire field of Everson machine learning disappeared. Like everything to do with Everson examples, like all of the, like for the most part, like GPT-IV would exist identically.”
Sep 20, 2024 neutral
Schulhoff: GPT-4 Fails to Output Reasoning on 1 in 100 to 1,000 Prompts
“I remember I did a lot of experiments with GPT-IV, and especially when you look at it at scale, so I'll run thousands of prompts against it through the API, and I'll see, you know, every one in a hundred, every one in a thousand outputs no reasoning whatsoever…”
Sep 27, 2024 negative
Oct 4, 2024 bullish
Oct 19, 2024 neutral
Oct 19, 2024 positive
Jesse Hu: SWE-bench gains over baseline GPT-4 come entirely from agent scaffolding
“The diff between that and something like Devin is all in like, sort of like the agent scaffold or the agent code, right? So that's, what's really exciting about this stuff. It shows off what you can do just from prompting and just from adding tools.”
Nov 11, 2024
Nov 29, 2024 bullish
Eugene Yan: Ensembled LLMs outperform standalone GPT-4 for evaluation tasks
“So in this paper here by Kohir, what they did was they have a reference model, and this reference model is GPT-IV. And then essentially what they did was the ensemble command R, Haiku, and GPT-IV. And I can't remember what the I think the ensemble was just maj…”
Jan 1, 2025 bearish
Swyx: AI models hit a 2T parameter wall, won't reach 10T
“As far as everyone is concerned, Claude, you know, Opus 3.5 is not coming out. GPT 4.5 is not coming out. And Gemini two, like we don't have pro whatever we've hit that wall, whatever that wall is. Maybe I'll call it like the two trillion parameter wall. Like …”
Jan 28, 2025 neutral
OpenAI o1 struggles with multi-step agentic tasks compared to GPT-4
“O-One is very good at programming, but it's kind of, the agent part was the harder part to get it to do here.
I think it's like less trained
To take the next step in like an agentic task, whereas GPT-IV for like the last two years has been really, you know, pr…”
Feb 1, 2025 negative
Nguyen: AI model card benchmark numbers are never apples-to-apples across labs
“None of the numbers are, like, apples to apples. So you actually need to, like, go back to, like, I don't know, like, GPT-E for model card and, like, read the appendix just to, like, make sure that, like, The settings were the same as you're running the settin…”
Mar 7, 2025 positive
Gemini 1 Proved GPT-4-Level Models Can Bootstrap Reinforcement Learning
“Giannis and I led a lot of the work for post-training and kind of RL check for Gemini, and Giannis being my co-founder, and when we shipped Gemini One, we just realized that the models, like, models that were basically at GPT-IV level or above, were capable en…”
Apr 27, 2025 neutral
Claude 3.5 wrote fire-and-forget code while GPT-4 used defensive programming
“Claude, for instance, the Sonnet 3.5 was very much fire and forget. It would write code in a kind of Pythonic way, just like, let it fail. Don't be careful about it. Whereas GPT four would use defensive programming, use self assertions.”
Jul 5, 2025 bullish
Jul 11, 2025 neutral
Hsu: Real-World Inertia Leaves AI's Daily Impact Outside Bay Area Near Zero
“And I think if you, like, go to another state outside of the Bay Area, probably even in California, outside of the Bay Area, and then you ask somebody how much their life has materially changed, it's, like, pretty close to zero. Real-world inertia is enormous.…”
Jul 11, 2025 neutral
Hsu: Early Speak GPT-4 role-play users submitted questionable custom scenarios
“In 2023, when we first launched our AI role plays using GPT-IV, back then people were way more concerned about safety, right? And obviously the models now are much better at like refusals and line sharper between what's appropriate and not. But we did see a lo…”
Jul 31, 2025 positive
Jul 31, 2025 bullish
Lambert: Custom personality fine-tuning is open source AI's winning turf
“If open models are to win, part of it could be just, like, everybody can have exactly the model they want. We're serving GPT-IV. It's kind of its thing. You can prompt it, but if fine-tuning is more effective than prompting, everybody can have the model. That …”
Aug 15, 2025
Brockman: GPT-4 handled multi-turn chat without being trained on it
“We actually did a instruction following post-train on it, so it was really just a data set that was, here's a query, here's what the model completion should be, and I remember that we were like, well, what happens if you just follow up with another query? And …”
Nov 22, 2025 positive
Jan 9, 2026 negative
Hill-Smith: Google used unpublished 32-shot CoT to claim Gemini beat GPT-4
“Back when I'm Googled a Gemini one when I ultra and needed a number that would say it was better than GPT four. And Like, constructed I think never published, like, chain of thought examples, 32 of them in every topic in MLU to run it, to get the score.”
Jan 9, 2026 positive
Jan 28, 2026 neutral
Apr 7, 2026 positive
Lopopolo: Reasoning models eliminate need for rigid state-machine scaffolding
“And this I think is like the fundamental difference between reasoning models and the four ones and four O's of the past where these models could not think. So you kind of had to put them in boxes with a predefined set of state transitions. Whereas here we have…”