Francois Chauvard discusses how recursive models solve problems like Sudoku by iterating in latent memory rather than using step-by-step chain-of-thought data.
Assertion Partly supported
Chaubard: HRM scored 70% on ARC Prize 1 without pre-training
“There is no pre-training at all. This starts from, like, literally Tagula-Rasa weights, and it can outperform at that time, if we go back, you know, we had O-three, if you remember back, way back when. And it, O-three gets zero. Literally zero, and this got, l…”
Prediction Not checkable as stated
Chaubard predicts running recursive reasoning models inside giant LLM latent spaces
“What you can imagine is we found mapping from token space or from vision, from pixels, Some really cool latent space where, like, things are just nicely semantically separated, and we can, you know, makes it really easy for downstream tasks to do, but now in t…”
Opinion
Chaubard: AI models are recursively distilling into one another, Claude into Kimi
“What I think is kind of happening right now, but like clawed kind of mother birds into, Kimmy too, and then now Kimmy too is post-training thinking machines, and so it just keeps going and going.”
Opinion
Chaubard: Hierarchical Reasoning Models Offer Little Novelty Over Standard RNNs
“The, this is directly in the lineage of RNNs. There's not that much novel from, like, the RNN standpoint at least in my opinion.”
Insight
Chaubard: Chain of thought trained on bubble sort won't discover merge sort
“If you chain of thought it on all the bubble sort input and output, it will only do bubble sort. In fact, it won't even do bubble That well.”
Assertion Contradicted
Chaubard: 7M parameter TRM scored 87% on ARC Prize 1
“And so it's a twenty-eight million parameter model for HRM. Now she brings it down to a seven million parameter model. It actually gets from 70% to 87% on on ArcPrize one. And does actually quite well on ArcPrize two as well.”