Nick Joseph
Head of Pre-training, Anthropic · 1 appearance on the record.
computed by AI from the episodes · how this works → · full disclaimer →
engineerscientistexecutivefounder@nickevanjoseph ↗anthropic.com ↗
Nick Joseph is a co-founder and Head of Pre-training at Anthropic, overseeing the infrastructure and training runs behind the Claude models. Before co-founding Anthropic in 2021, he worked at Vicarious and OpenAI on safety and code models.
38statements → 7claims → 2claims resolved → 3.63/5average certainty → 2.13/5average debate potential →
1 supported 0 partly supported 1 contradicted 1 not yet assessed 4 not checkable as stated how the 7 claims stand · each chip opens the sources
2 predictions · 5 assertions · 1 opinion · 24 insights · 6 disclosures · every statement was checked. The predictions and assertions are the 7 claims: statements the public record can support or contradict. 2 are resolved, 1 is not yet assessed, and 4 name no date, number or outcome precise enough to check. Everything else (opinions, insights, what ifs, disclosures) can never be settled by the record, so it carries no assessment.
The record, in short
What the tape says about how Nick argues and how the claims held up. Everything they said, and everything said about them, is in the tabs below.
Their most notable supported claim
Joseph: Public estimates placed GPT-3 training cost at $5 million
“Like the public estimates for GP three, I remember, were that it cost five million dollars to train, which you're like, on the one hand, five million is kind of a lot, but it's like a lot for an individual person. It's not really a lot from like a company pers…”
Their most notable contradicted claim
Expressed certainty vs assessment result
weighted support: a fully supported claim counts one, a partly supported claim counts half. Each filled bar is clickable and opens exactly those claims; "none yet" means nothing said at that certainty level has resolved yet
Everything Nick Joseph said on the Y Combinator Startup Podcast that made the record, most notable first. Filter by type, assessment or year in the ledger →
Anthropic's Joseph: Compute matters far more than pre-training objective details
“I think that, like, the one sort of general intuition I have is, like, compute is the thing that matters. So, like, I think if you throw enough compute at any of these objectives, you're gonna get something that's probably pretty good, and can kind of be fine …”
Joseph: LLM training requires collaborative infrastructure work over publishable research papers
“And to do a project like training a large language model requires a lot of people to collaborate on like a really complicated piece of infrastructure that isn't going to be a paper, right? Like you're not going to publish like, oh, I got a slightly, I got five…”
Joseph: Sparsely linked long-tail data may be most valuable for frontier AI
“And it might be that like, that data ends up more valuable because you, everything that's linked to a lot, you've already got. Like at some point, you're maybe like going for the tails, or you're going for the stuff that no one's ever, like, you know, it's onl…”
Joseph: Training purely on raw LLM generations cannot produce a better model
“Theoretically, I shouldn't be able to train a better model than that. Like, I'm just going to get the same thing out. So I think that's-”
Nick Joseph: Third parties can steer frontier AI labs by publishing evals
“Like, it is the case that, like, the labs right now are really driven by getting good eval scores. And it's hard to make them, and anyone can do it. There's no comparative advantage to having the model to making an eval. So I do think it's actually, like, an i…”
Anthropic's Joseph: Certain AI Alignment Pieces Will Move to Pre-Training
“I do think at some point there will be, like, some pieces of alignment that, like, you do want to export back into pre-training because that might be a way to, like, Put them in with more strength, like, more robustness, kind of, or more core to the intelligen…”
Anthropic's Joseph: Scaling alone likely will not achieve AGI without further paradigm shifts
“Like I think the sort of shift towards more RL is like one paradigm shift in the field, and I think it's, I think there will probably be more. I think a lot of people sort of argue about like, oh, it's like, you know, current paradigm's enough to get us to EGI…”
Anthropic's Nick Joseph: Frontier pre-training teams primarily need engineers, not researchers
“The thing we, like, most need is engineers. Almost always, like, throughout, like, the entire history of this field. It's, like, the case that you throw more compute, the thing kind of works. The challenge is, like, actually doing that.”
Nick Joseph: Scaling standard models is easier and more reliable than inventing novel architectures
“It's just that scale is easier, and it's more reliable, and I think you, we're still seeing really big gains to that.”
Anthropic built custom distributed training to scale beyond Facebook's infrastructure
“We don't want to outsource this to some package because A, we're about to go to a bigger scale, like PyTorch, for instance, they had a package for doing this. But we were going to go to a bigger scale than Facebook had been to. And you don't want to have a dep…”
Joseph: Reinforcement learning exhibits scaling laws where compute yields better models
“You can get pretty big wins from RL. You sort of have another set of scaling laws. It's like you put more and more compute into RL, you can get better and better models out of that.”
Joseph: Anthropic avoids pre-training and post-training rivalry seen at other labs
“Yeah, I think we managed to avoid this, and it's pretty collaborative. Like, we're basically all producing one model and kind of can, but I do think at other places there's been some, from what I've heard, there's been some amount of, like, friction between, b…”
Joseph: Adversaries actively publish web data designed to poison AI models
“There are people who are, like, trying to put stuff out that is, like, as damaging as possible for the model, you know, how can I make it past the filter and get into the model would be totally like secretly useless.”
Nick Joseph: Solving coding interview questions proved to be shockingly narrow, not AGI
“I used to think that if you had an AI that could solve coding interview questions, it would probably be AGI. I was like, that's what I did to get my job, I could probably do the job. And it turns out like, nope, nope, you solve those, it's shockingly narrow, a…”
Joseph: Technical AI steerability must precede debates over whose values to embed
“I think that's a really hard problem. I think it's, like, actually kind of downstream of being able to pick any. I think of it almost, I think one analogy I've heard that I like is, like, putting a steering wheel on a car. It's like, if you don't have a steeri…”
Anthropic's rate limits are caused by short-notice compute shortages
“Anthropic has rate limits constantly, and people complain about it a lot, and like the reason is like, there's only so much compute we can get on short notice, so you, like, making your inference more efficient is like the way you can serve more users.”
Anthropic: Frontier AI models are first-shot attempts due to chip limits
“But I do think the change is massive, and I think people, like, don't realize how chip-limited AI, like, research is, or something right now, like, the models that everyone uses, right? If you're using, like, Cloud Sonic four, Cloud Opus four, it's like, it's …”
Anthropic's Joseph: Startups shouldn't over-invest in scaffolding future models won't need
“I think maybe the thing to watch out for is things where, like, they work now with a huge amount of work, like, to build up a scaffold, but the next generation, you're not gonna need the whole scaffold you built up.”
Joseph: AI scaling laws predictably quantify loss reductions from compute and data
“There's this idea of scaling laws, which is that you can actually quantify, like, as you put in more compute, more, more data, more parameters, you get models in a very, you got a lower loss, a better prediction of the next word in a very predictable way.”
Joseph: AI progress relies on a revenue-to-compute positive feedback loop
“There's this positive feedback loop where you can train a model, You can use it to make something useful and sell that and get more money, use that to buy more compute, and then use that to train a better model. And we've sort of run that cycle over and over a…”
Anthropic reverse-engineered cloud provider GPU physical room layouts to fix latency
“We're using a cloud provider, but I think it's kind of, it's not actually that different, because one of the things that was surprising to me is you actually have to understand the literal layout. Like, I remember at 1.1 of my co-workers running a clustering a…”
Joseph: Original scaling laws paper spanned 11 orders of magnitude
“Like, you know, the scaling, I think the original scaling laws paper had, like, 11 orders of magnitude, and there was, like, this intense debate on whether it would continue for, like, another point.”
Joseph: Too many specialists forces managers to connect cross-domain optimizations
“I think if you get too many people who are specialists, you end up with a lot of effort has to come from the manager, from like the lead to connect everything, and to notice something like, Ah, if we change the architecture here, that would make this, like, ef…”
Joseph: Using multiple AI chip architectures multiplies engineering workload
“The downside of having multiple chips is that you have to write the thing multiple times. In theory, you could have abstractions across them, but they're different enough that it's pretty hard to do that. So you can sort of end up, if you do all the workloads …”
Appearances (1)
| Episode | Date | Speaking time |
|---|---|---|
| Anthropic Head of Pretraining on Scaling Laws, Compute, and the Future of AI · Y Combinato | Sep 30, 2025 | 40m |