Merullo: LLM memorization spans a gradient from reasoning to rote recall
Jack Merullo · [State of MechInterp] SAEs in Production, Circuit Tracing, AI4Science, "Pragmatic" Interp — Goodfire · Dec 31, 2025 · at 6:02
Jack Merullo, researcher at Goodfire, discusses findings from Goodfire's research paper on disentangling internal model memorization from reasoning capabilities.
0:00 / 0:17exact quote · 17.9s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“You can actually see, like the way that we, like, disentangle memorization, you can kind of see this like, gradient of memorization in between both mechanistically and behaviorally with, like, logical reasoning tasks being quite distinct from rote memorization, and then, like, factual recall is kind of somewhere in the middle.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Jack Merullo
Merullo: Current machine unlearning techniques merely suppress data rather than removing it
“I would describe it more as not unlearning, but maybe suppression. I think there's, like, really, like, I guess, guarantees that you've fully removed information from a model is, is, I don't think it's been convincingly showed anywhere yet”