HRM

“There is no pre-training at all. This starts from, like, literally Tagula-Rasa weights, and it can outperform at that time, if we go back, you know, we had O-three, if you remember back, way back when. And it, O-three gets zero. Literally zero, and this got, l…”

“If you actually train on 16, and you test on only one, you get, like, seven eighths of the performance, or, like, almost all the performance. So it's actually quite interesting that this is just overdone, too much compute, and it doesn't actually help you all …”

“In HRM and TRM, they use Sudoku as an incompressible problem. Similarly, and so are mazes. Those are incompressible problems. Rolling sum, incompressible problem.”

“One of the things that's really interesting about these TRMs and HRMs is they're not general purpose models, right? These were Task specific models, right? The model trained to do Sudoku cannot do ArcPrize inherently. It has to be trained on the ArcPrize set t…”

“And so it's a twenty-eight million parameter model for HRM. Now she brings it down to a seven million parameter model. It actually gets from 70% to 87% on on ArcPrize one. And does actually quite well on ArcPrize two as well.”