Mechanistic Interpretability

“The problem with McInterp is it's a lot, it's been about sort of testing small hypotheses. Hypothesis. And you know, you have a hypothesis, you'll find some small thing, you'll test that in isolation. But I don't think it's really become a science yet.”

“Most fascinating things about coding agents actually is they can do a lot of experimentation in an automated fashion. Yeah. They will give new hope. They'll breathe new life into mechanter research.”

“The hope is that you kind of really learn the underlying basis for how it's making the predictions, and so you open up the black box and you can actually understand kind of the biology that the model is representing.”

“We believe that understanding the internal weights and activations, what is the internal structure, the mathematical structure of these systems is going to be at least part of the solution.”

“But as we're starting to have models that are much smarter than us, at least in some important ways, we think that we'll be able to start tracking mechanistic capability much more effectively.”

“I think that we actually might finally be able to make more what I would consider a science of this through essentially leveraging mass research by agents deployed for this problem.”

“We at this point have like pretty sophisticated technology to understand what's happening in the neurons to trace it. And so for example, like if there's a neuron related to deception, we can start, we're starting to get to the point where we can monitor it an…”

“Bring interpretability to training, which I don't think has been done all that much before. A lot of this stuff is sort of post-talk poking at models as opposed to actually using this to intentionally design them.”

“I think that if we further that research direction two, three years in the future, we will be able to understand why models say what they'd say.”

“There's no gap for scale. Like, they've shown that even for the biggest open source models, you like, even like DeepSeq's big models, you, they can do it. And then in general, like, scaling is not the bottleneck.”

“If you understood the circuit well, and if you identified where it's thinking about Huskies or where it's thinking about like kind of like breeding two different breeds, then you should be able to like swap these in and out and get it to kind of like say whate…”

“Like you, you're not limited to studying what the model can do, right? Like if the model's failing at something like, you know, counting the number of letters in strawberry or whatever you could just try that and try to figure out the circuit for like, well, i…”

“I think for Interp in particular, there's like another thing that makes it easier to transition to, which is maybe two things. One, you can just do it without huge access to compute. Like, there are open source models. You can look at them. A lot of Interp pap…”

“There is a large swath of research on modeling, on how these models behave, on evaluation, on inference, on mechanistic interpretability that could not happen at all. If you didn't have open models.”