Assertion Supported AI assessment confidence: 88% certainty 4/5 debate potential 2/5

Patel: GPT-4-Level Training Costs Have Dropped 10x to 100x

Dwarkesh Patel · Dwarkesh Patel: AI Continuous Improvement, Intelligence Explosion, Memory, Frontier Lab Competition · Jun 18, 2025 · at 22:46

Dwarkesh Patel argues that algorithmic efficiency and cheaper compute have rapidly reduced the training costs of frontier models.

0:00 / 0:28exact quote · 28.3s

▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark

“If you look at what it costs to train GBT for originally, I think it was like 20,008, 100 over the course of a hundred days. So I think it costs on the order of like half a million to a hundred million dollars, somewhere in that range. And I think you could train an equivalent system today. I mean, deep seek, we know it's trained on five million dollars supposedly, and it's better than GPT four, right? So you've had literally multiple orders of magnitude decrease, like 10 to a hundred X decrease in the cost to train a GPT four level system.”

quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →

The claim, plainly

The cost to train a GPT-4-level frontier AI model has dropped by 10x to 100x since GPT-4 was originally trained.

a quote alone can be ambiguous, so this restates it with the references and context of the conversation around it resolved. AI-written; the tape below the player is the source.

Assessment

Dwarkesh Patel claimed in June 2025 that the direct cost to train a GPT-4-level frontier AI model had dropped by 10x to 100x compared to GPT-4's original training run. Multiple credible industry and academic sources corroborate this range. OpenAI's original GPT-4 pre-training compute cost (trained across 2022-2023) was estimated by Epoch AI and the Stanford AI Index at $41 million to $78 million (with total training and development cited by then-CEO Sam Altman at over $100 million). By late 2024 and early 2025, models matching or exceeding GPT-4's benchmark performance were trained at a fraction of that cost: DeepSeek-V3 reported direct pre-training compute rental costs of $5.5 million to $5.6 million (an ~8x to 18x reduction), while competitive 70B-parameter models (such as Llama 3/3.1 70B and Qwen 2.5 72B) were estimated at $1.2 million to $6.0 million to train (up to an ~80x reduction). Furthermore, Epoch AI's research on algorithmic progress confirmed efficiency gains of 10x to 60x annually when replicating fixed past frontier performance levels.

Epoch AI and Stanford AI Index estimated original GPT-4 direct compute costs at $41M-$78M, with Sam Altman stating development costs exceeded $100M.
DeepSeek-V3 technical report (December 2024) disclosed direct pre-training compute costs of ~$5.5M-$5.6M on H800 clusters for a model matching or exceeding GPT-4 performance.
Independent analysis from Artificial Analysis and Epoch AI found 70B-class models matching GPT-4 performance cost between $1.2M and $6.0M to pre-train by early 2025.

Checked against

[1] statista.com [2] alicelabs.ai [3] arxiv.org [4] forbes.com [5] pristren.com [6] reddit.com [7] intuitionlabs.ai [8] localaimaster.com

DeepSeek V3 training cost 6 million GPT 4 level cost to train GPT-4 level model decline 10x 100x cost to train GPT-4 vs DeepSeek V3 training cost Epoch AI "Epoch AI" estimated training compute cost of GPT-4 algorithmic progress compute cost GPT 4 level model Epoch AI

assessed as of 2026-09-02

This assessment is an AI reading of a public statement, made against the cited references with the confidence shown. It can be wrong. The quote, the tape and the references are all on this page; if something is off, use report an error below. full disclaimer →

Mentioned

DeepSeek GPT-4 training costs

⚑ Spot an error? Report it

More from Dwarkesh Patel

Prediction Not checkable as stated

Patel: Individuals will eventually train superintelligences in basements

“The cost of training, the systems is declining so fast that Literally you will be able to train a super intelligence in a basement at some point in the future.”

Dwarkesh Patel Jun 18, 2025 ▶ 56:49 Dwarkesh Patel: AI Continuous Improvement, Intelligence Explosion, Memory, Frontier Lab Competition

Opinion

Dwarkesh: AGI requires further algorithmic progress, not just current model scaling

“I don't think we're just right around our corner from AGI and it's just a little additional dash of something. That's all it's going to take. I think, you know, people often ask if all AI progress stopped right now and all you could do is collect more data or …”

Dwarkesh Patel Jun 18, 2025 ▶ 1:56 Dwarkesh Patel: AI Continuous Improvement, Intelligence Explosion, Memory, Frontier Lab Competition

Insight

Dwarkesh: Lack of continual learning prevents LLMs from replacing human labor

“I think a big bottleneck these models have is their inability to learn on the job, to have continual learning. Their entire memory is extinguished at the end of a session. There's a bunch of reasons why I think this actually makes it really hard to get human-l…”

Dwarkesh Patel Jun 18, 2025 ▶ 2:21 Dwarkesh Patel: AI Continuous Improvement, Intelligence Explosion, Memory, Frontier Lab Competition

Prediction Not checkable as stated

Patel: Reinforcement learning may not generalize beyond verifiable domains

“I still think I I'm like, I'm not confident that this will generalize to domains that are not so verifiable or text-based.”

Dwarkesh Patel Jun 18, 2025 ▶ 7:34 Dwarkesh Patel: AI Continuous Improvement, Intelligence Explosion, Memory, Frontier Lab Competition

Prediction Not checkable as stated

Patel: Online continual learning is not imminent for current AI architectures

“And the reason I don't think that's around the corner is just because there's not, there's no obvious way, at least as far as I can tell, to just slot in this online learning into the models as they exist right now.”

Dwarkesh Patel Jun 18, 2025 ▶ 8:11 Dwarkesh Patel: AI Continuous Improvement, Intelligence Explosion, Memory, Frontier Lab Competition

Assertion Not checkable as stated

Patel: Pre-training scaling is seeing diminishing returns

“Pre-training, which is this idea that you just make the model bigger that has had diminishing returns.”

Dwarkesh Patel Jun 18, 2025 ▶ 9:13 Dwarkesh Patel: AI Continuous Improvement, Intelligence Explosion, Memory, Frontier Lab Competition

Patel: GPT-4-Level Training Costs Have Dropped 10x to 100x

More from Dwarkesh Patel

Turn any episode into a week of clips.