Assertion Partly supported
Chaubard: HRM scored 70% on ARC Prize 1 without pre-training
“There is no pre-training at all. This starts from, like, literally Tagula-Rasa weights, and it can outperform at that time, if we go back, you know, we had O-three, if you remember back, way back when. And it, O-three gets zero. Literally zero, and this got, l…”
Insight
Chaubard: Recursive models discover problem-solving strategies without human teacher forcing
“That's the most important part, is that if we had Sudoku, and we know how to solve Sudoku, because like we were just, you know, dumb homo sapiens that didn't know how to solve Sudoku, like it would just have solved it. And that's why it's cool, because it actu…”
Prediction Not checkable as stated
Chaubard predicts running recursive reasoning models inside giant LLM latent spaces
“What you can imagine is we found mapping from token space or from vision, from pixels, Some really cool latent space where, like, things are just nicely semantically separated, and we can, you know, makes it really easy for downstream tasks to do, but now in t…”
Opinion
Chaubard: AI models are recursively distilling into one another, Claude into Kimi
“What I think is kind of happening right now, but like clawed kind of mother birds into, Kimmy too, and then now Kimmy too is post-training thinking machines, and so it just keeps going and going.”
Opinion
Chaubard: Hierarchical Reasoning Models Offer Little Novelty Over Standard RNNs
“The, this is directly in the lineage of RNNs. There's not that much novel from, like, the RNN standpoint at least in my opinion.”
Insight
Chaubard: Chain of thought trained on bubble sort won't discover merge sort
“If you chain of thought it on all the bubble sort input and output, it will only do bubble sort. In fact, it won't even do bubble That well.”
Assertion Contradicted
Chaubard: 7M parameter TRM scored 87% on ARC Prize 1
“And so it's a twenty-eight million parameter model for HRM. Now she brings it down to a seven million parameter model. It actually gets from 70% to 87% on on ArcPrize one. And does actually quite well on ArcPrize two as well.”
Prediction Not checkable as stated
Chaubard: Massive chip specialization will split training and inference data center specs
“We just see that at the chip level, there's going to be massive specialization. The specs that you would need for a training data center are going to be very different than the specs for a inference data center.”
Assertion Not checkable as stated
Chaubard: CPU-based environment simulation bottlenecks on-policy reinforcement learning rollouts
“The, it's amazing how much of simulators, when you call environment.step, is still run on the CPU, and so that's usually the bottleneck for a lot of your on-policy rollouts”
Prediction Not checkable as stated
Chaubard: Chip specialization and ASICs will proliferate across AI workloads
“From my vantage point, we're gonna see this proliferation, and where we have sufficient demand now, because there's so much demand for tokens, Where it makes sense to specialize at the chip level that will pay out and allow you to go through the full new produ…”
Assertion Supported
Chaubard: Inference stacks split between Nvidia for pre-fill and Cerebras for decode
“We already do see it a little bit on where a lot of the stack peep for inference. People will go to Nvidia for pre-fill and then they'll go to cerebris for decode engine.”
Insight
Chaubard: Standard LLMs sacrifice latent time compression unlike RNNs
“What you actually paid for that you have to give up is this latent reasoning thing and this compression in the time direction. There is no compression in LMs. Every single decode that I do, I still have to retain the entire, you know, Shakespeare novel just to…”
Insight
Chaubard: Transformers cannot sort lists longer than layer count in one pass
“In a one-shot basis. It's like literally that we know a theoretical lower bound that for comparison sort, you can't do better than n log n steps. And if I have a list that's 31 characters or elements long, and my transformer is 30, I run out of steps to do com…”
Assertion Partly supported
Chaubard: Sudoku, mazes, and rolling sums are incompressible reasoning problems for LLMs
“In HRM and TRM, they use Sudoku as an incompressible problem. Similarly, and so are mazes. Those are incompressible problems. Rolling sum, incompressible problem.”
Assertion Supported
Chauvard: Chain of thought makes LLMs Turing complete at test time
“And so it's completely true that at test time, they are turn complete. And you can simulate all turn computable functions at test time.”
Insight
Chaubard: Chain of thought fails on unsolved problems lacking human traces
“Unless you're training it on human labeled traces for which there's a lot of problems like the millennial prize problem. We don't have the trace for it.”
Assertion Supported
Chaubard: Recursive models tested on one step retain nearly full performance
“If you actually train on 16, and you test on only one, you get, like, seven eighths of the performance, or, like, almost all the performance. So it's actually quite interesting that this is just overdone, too much compute, and it doesn't actually help you all …”
Assertion Supported
Chaubard: MLP outperformed attention on Sudoku but scored zero on mazes
“Yeah, on Sudoku, MLP actually outperformed the attention, It was it scored zero on the maze, ah, the MLP scored zero on the maze, and so there's, it's not clear, it's not obvious that, ah, the transformer is always better.”
Assertion Not checkable as stated
Chaubard: Prior to 2016, Researchers Believed RNNs Were Necessary for AGI
“An RNN is just a model that you recursively call again and again and again on itself, and we, We're very much in the belief that this was required to get to AGI peak RNN use was probably until 2016 with Alex Graves NeurIPS keynote, which is just fantastic, and…”
