Emmanuel Ameisen
Interpretability Researcher, Anthropic · 1 appearance on the record.
computed by AI from the episodes · how this works → · full disclaimer →
scientistengineerauthor@mlpowered ↗LinkedIn ↗mlpowered.com ↗
Emmanuel Ameisen conducts mechanistic interpretability research at Anthropic, focusing on understanding the computational graphs and internal representations of large language models. Previously an ML engineer at Stripe and Head of AI at Insight Data Science, he authored the O'Reilly book Building Machine Learning Powered Applications.
30statements → 17claims → 13claims resolved → 92%fully supported → 3.9/5average certainty → 1.83/5average debate potential → 3said about them ↓
12 supported 0 partly supported 1 contradicted 4 not checkable as stated how the 17 claims stand · each chip opens the sources
2 predictions · 15 assertions · 2 opinions · 7 insights · 4 disclosures · every statement was checked. The predictions and assertions are the 17 claims: statements the public record can support or contradict. 13 are resolved, and 4 name no date, number or outcome precise enough to check. Everything else (opinions, insights, what ifs, disclosures) can never be settled by the record, so it carries no assessment.
The record, in short
What the tape says about how Emmanuel argues and how the claims held up. Everything they said, and everything said about them, is in the tabs below.
Their most notable supported claim
Ameisen: Deceptive Backward Reasoning Exists in Base Pre-Trained Models
“I bet, I don't know how much I bet a hundred bucks. So somebody can like, they would get a hundred bucks from me if they prove that I'm wrong, that this behavior for a model that does a drink fine tuning, it also does it post pre-training.”
Expressed certainty vs assessment result
weighted support: a fully supported claim counts one, a partly supported claim counts half. Each filled bar is clickable and opens exactly those claims; "none yet" means nothing said at that certainty level has resolved yet
Everything Emmanuel Ameisen said on Latent Space that made the record, most notable first. Filter by type, assessment or year in the ledger →
Ameisen: Stochastic Parrots Label Ignores Complex Multi-Step LLM Reasoning
“It's, like, activating many different distributed representations, like, combining them, and sort of, like, doing something pretty complicated. And so, yeah, I think it's funny, because in my opinion, that's like, yeah, like, oh god, stochastic parrots is not …”
Ameisen: LLMs plan future tokens rather than operating purely myopically
“Language models are next token predictors is like a fact. Like that is what they do. They are trained to predict the next token. However, that does not mean that they myopically only consider the next token When they choose the next token, you can work on brea…”
Ameisen: Deceptive Backward Reasoning Exists in Base Pre-Trained Models
“I bet, I don't know how much I bet a hundred bucks. So somebody can like, they would get a hundred bucks from me if they prove that I'm wrong, that this behavior for a model that does a drink fine tuning, it also does it post pre-training.”
Ameisen: Current LLM Chain of Thought Is Unfaithful and Untrustworthy
“So I think there's like a sense in which right now the chain of thought is, is unfaithful, or at least you can't read the chain of thought and trust that that's how the model did it.”
Ameisen: Multi-Hop Reasoning Circuits Are Extremely Similar Across Small and Large Models
“The way the circuit looks in Gemma, like a really small model is extremely similar to the way that it looks like a huge model, which that in itself is, I think like a pretty novel discovery. It's like, oh, you have these models that are like super different. Y…”
Ameisen: Anthropic's circuit tracing tool ignores attention heads and only decomposes MLPs
“These are just MLPs. So the model has both attention heads and multi-layer perceptions MLPs. We don't just do it. Like we completely ignore attention or like we don't try to decompose it at all. So there's some prompts where like all of the interesting stuff i…”
Ameisen: Swapping Internal Features Proves Single-Pass LLM Multi-Step Reasoning
“We claim that this is like the Texas representation. Let's get another one and replace it. And we just change like that feature in the middle of the model and we change it to like California. And if you change it to California, sure enough, it says Sacramento.…”
Ameisen: Larger language models share more concept representations across languages
“If you look inside the model, if you look at the middle of the model, which is the middle of this plot here, models share more features. They share more of these representations in the middle of the model, and bigger models share even more. And so the, like, t…”
Ameisen: LLMs use internal circuits to backwards-plan rhyming poetry lines
“And two, this plan doesn't just control, like, what you're gonna rhyme with. It's also doing what's called like backwards planning, where it's like, well, because I need to finish with green, I'm not going to say illuminating the peaceful night, because then I…”
Ameisen: LLMs Execute Parallel Sub-Processes During Math and Hallucinations
“So I think one example of this is like math where the model is like independently computing the like last digit and then the like order of magnitude and then kind of like combining them at the end or like hallucinations are also that where like, there's one si…”
Ameisen: Interpretability researchers lack good methods for analyzing attention layers
“So like, I think that right now we have some pretty good solutions for like understanding what's in the residual stream, understanding what's, is it in MLPs? We don't have good solutions for like attention.”
Ameisen: Anthropic publishes interpretability research to recruit more researchers
“The reason for publishing this is that we think interpretably is important. We think it's tractable, and we think more people should work on it. And so publishing it helps us like accomplish with these goals all these goals, which we think are just like crucia…”
Ameisen: Anthropic Trained a Misaligned Model With Hidden Goals for Detection
“A team at Anthropic trained a model to have like weird hidden goals and then gave it to a bunch of other teams and said, Figure out what's wrong with it”
Ameisen: Mechanistic interpretability methods successfully scaled to production models
“And it turns out scaling it. I don't want to say it just worked because it was a lot of work. I don't mean to apply. There was an effort, but it worked. And now we're in the phase where it's like, oh, cool. These methods work on the models that we care about.”
Ameisen: Validated circuit models enable predictable steering via feature swapping
“If you understood the circuit well, and if you identified where it's thinking about Huskies or where it's thinking about like kind of like breeding two different breeds, then you should be able to like swap these in and out and get it to kind of like say whate…”
Ameisen: Interpretability research has lower entry barriers and low compute needs
“I think for Interp in particular, there's like another thing that makes it easier to transition to, which is maybe two things. One, you can just do it without huge access to compute. Like, there are open source models. You can look at them. A lot of Interp pap…”
Ameisen: Golden Gate Claude Was Created by Clamping a Bridge Feature
“That means that, like, if that's true, then you can, like, set that feature to zero, or artificially set to a hundred, And you'll change model behavior. That's what we did when we did Golden Gate Claude, in which we found a feature that represents the directio…”
Ameisen: Naive model pruning fails because superposition distributes critical representations
“Well, right, and it's like, on, on each example, maybe this neuron is like at the bottom of, like, what matters, but actually it's participating, like, five percent to, like, understanding English, like, doing integrals and, you know, like, whatever, like, cra…”
Ameisen: Model internal representations show measurable bias toward English logits
“And it does seem like Does sort of like inner representations have a higher connection to like the output logits for English logits. And so there's like some bias towards English at least in the model we studied here.”
Ameisen: Sparse autoencoder feature interpretability can and will be automated
“There's been a lot of work in sort of like automated feature interpretability. And it's something that we've invested in and that like other labs have invested in. And I think basically the answer is we can definitely automate it and We're definitely going to …”
Ameisen: Anthropic Released Circuit Tracing Code Built by Fellows
“And even more recently, we released some code in partnership with the Anthropic Fellows program. It was mostly built by Anthropic Fellows that lets people play with the research basically.”
Ameisen: Anthropic's Open Tool Traces Internal States in Gemma 2 2B
“And then the release this week sort of lets anyone do it for a set of open source models. So notably maybe the most easy one here is like Gemma two to be. So you can sort of like think of some prompt and you kind of like can explain any like token that the mod…”
Ameisen: Circuit tracing notebooks run entirely on free Google Colab
“The notebooks themselves They can all be run on Google Colab and all of the code, as far as we can tell, we've like tested on the notebooks, just like runs on Colab. And so that means that like, you don't need on a free tier to be clear, like you don't need li…”
Ameisen: Circuit tracing diagnoses model failures by exposing incorrect internal representations
“Like you, you're not limited to studying what the model can do, right? Like if the model's failing at something like, you know, counting the number of letters in strawberry or whatever you could just try that and try to figure out the circuit for like, well, i…”
The other half of the tape: Emmanuel Ameisen's own voice is left out of every number here. Other people bring the name up 3 times in 3 episodes on Latent Space. every mention, with the transcript →
Who brings them up most Shawn Wang 2
Every mention by year
tap a year for its mentions
2026 1 mention in 1 episode
2025 2 mentions in 2 episodes 1 per episode
Appearances (1)
| Episode | Date | Speaking time |
|---|---|---|
| The Utility of Interpretability — Emmanuel Amiesen | Jun 6, 2025 | 1h 14m |