Gupta: Chain of Thought Operates in Token Space, Not Model Recursion
Ankit Gupta · Recursion Is The Next Scaling Law In AI · Y Combinator · May 1, 2026 · at 19:01
Y Combinator's Ankit Gupta discusses the architectural constraints of Chain of Thought prompting in standard feed-forward LLMs versus true recursive latent models.
0:00 / 0:29exact quote · 29.4s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“There already exists some type of recursion that people are used to in LLMs, which is a chain of thought we mentioned earlier, but that is a recursion that's happening in the token space of the model's outputs, not inherent to the model itself. That's sort of the fundamental limitation is that the model can only do a feed forward one shot output, and then we basically just have this hack that if you keep letting it output things, then it can read its outputs and do somewhat intelligent seeming things with it.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Ankit Gupta
Gupta: Machine learning progresses by abandoning bio-plausibility for computational efficiency
“I think machine learning tends to have a long history of people starting with bio-plausible arguments, and then realizing that there's some variant of them that seems highly bio-implausible that actually works better.”
Gupta: Recursive Latent Hidden States Function Like a Turing Machine Tape
“I kind of think of this set of hidden states or carry as akin to a Turing machine tape or akin to the radix sort, ah, memory bank, where you can basically train a model to use this memory cache in an intelligent way in a single forward pass so that you can get…”
Gupta: Recursive architectures achieve compute depth without parameter depth
“Recursion advantage now gives you a bunch of advantages over transformers where rather than having, you know, 500 or a thousand or a million or whatever transformer layers and having tons and tons of parameters, you get compute depth basically without this par…”
Gupta: Current TRMs and HRMs are task-specific, not general-purpose
“One of the things that's really interesting about these TRMs and HRMs is they're not general purpose models, right? These were Task specific models, right? The model trained to do Sudoku cannot do ArcPrize inherently. It has to be trained on the ArcPrize set t…”