attention heads
1 statements across 1 episodes · 0 bullish · 0 bearish · 1 people on the record · first statement Jun 6, 2025 by Emmanuel Ameisen · across every show →
Everything said about attention heads, oldest first
Jun 6, 2025 neutral
Ameisen: Anthropic's circuit tracing tool ignores attention heads and only decomposes MLPs
“These are just MLPs. So the model has both attention heads and multi-layer perceptions MLPs. We don't just do it. Like we completely ignore attention or like we don't try to decompose it at all. So there's some prompts where like all of the interesting stuff i…”