Francois Chaubard explains that chain-of-thought prompting fails to achieve algorithmic discovery beyond its training data on YC Decoded.
Assertion Partly supported
Chaubard: HRM scored 70% on ARC Prize 1 without pre-training
“There is no pre-training at all. This starts from, like, literally Tagula-Rasa weights, and it can outperform at that time, if we go back, you know, we had O-three, if you remember back, way back when. And it, O-three gets zero. Literally zero, and this got, l…”
Insight
Chaubard: Recursive models discover problem-solving strategies without human teacher forcing
“That's the most important part, is that if we had Sudoku, and we know how to solve Sudoku, because like we were just, you know, dumb homo sapiens that didn't know how to solve Sudoku, like it would just have solved it. And that's why it's cool, because it actu…”
Prediction Not checkable as stated
Chaubard predicts running recursive reasoning models inside giant LLM latent spaces
“What you can imagine is we found mapping from token space or from vision, from pixels, Some really cool latent space where, like, things are just nicely semantically separated, and we can, you know, makes it really easy for downstream tasks to do, but now in t…”
Opinion
Chaubard: AI models are recursively distilling into one another, Claude into Kimi
“What I think is kind of happening right now, but like clawed kind of mother birds into, Kimmy too, and then now Kimmy too is post-training thinking machines, and so it just keeps going and going.”
Opinion
Chaubard: Hierarchical Reasoning Models Offer Little Novelty Over Standard RNNs
“The, this is directly in the lineage of RNNs. There's not that much novel from, like, the RNN standpoint at least in my opinion.”
Assertion Contradicted
Chaubard: 7M parameter TRM scored 87% on ARC Prize 1
“And so it's a twenty-eight million parameter model for HRM. Now she brings it down to a seven million parameter model. It actually gets from 70% to 87% on on ArcPrize one. And does actually quite well on ArcPrize two as well.”