Jan 23, 2026 · 1h 32m · latent-space
Captaining IMO Gold, Deep Think, On-Policy RL, Feeling the AGI in Singapore — Yi Tay
0:00 / 0:00
▶ Watch on YouTube →
gold bands on the timeline = statements, start to end. Hover to read, click to jump. CC turns on captions
In this conversation, Google DeepMind Senior Research Scientist Yi Tay breaks down the algorithmic breakthroughs behind Gemini's IMO Gold victory, the mechanics of on-policy reinforcement learning, and the establishment of GDM Singapore. He offers deep insights into Transformer longevity, data efficiency, AI-augmented engineering workflows, and the enduring necessity of algorithmic innovation on the path to AGI.
How this conversation actually went
Every chapter scored 0–10 on four independent dynamics. Hover any point for the reasoning behind the score. How this is scored →
The hosts as informed peer 5.6 Guest teaching 4.2 Guest disagreement 1.6 The hosts pushing back 2.3
speaking balance: gold is the hosts, purple is the guest (3 minute bins)
Sharpest disagreement ▶ 1:16:00 Ranting on recommendation systems and IR conferences
Yi strongly dismisses the IR and recommender systems subfield, calling its modeling dynamics weird, rude, and fundamentally lagging behind mainstream ML.
Hardest push from the hosts ▶ 48:59 Refusing the sequence-to-sequence paradigm for 200M contextsThe host rejects the guest's assumption that attention models are all we need, arguing that massive context scaling requires continual learning rather than standard seq2seq inference.
Biggest teaching moment ▶ 5:26 Explaining on-policy RL distillation mechanicsYi gives a precise conceptual breakdown of how on-policy LM reinforcement learning differs fundamentally from off-policy supervised imitation learning.
The host holds their own ▶ 8:30 Host maps learning rate theory to Bayesian belief updatingThe host demonstrates deep analytical synthesis by explaining how Bayesian priors fail when single counter-examples require aggressive learning rate adjustments.
the scores for every segment, with the reasoning behind each
| Chapter | Topic | The hosts as informed peer | Guest teaching | Guest disagreement | The hosts pushing back | Why |
|---|---|---|---|---|---|---|
| Emergent Utility in AI Coding and Image Generation | 6 | 3 | 1 | 2 | The host opens by connecting Yi's previous research on UL2 and T5 to modern RL modeling, while probing into Google DeepMind team structure. | |
| Contrasting On-Policy Reinforcement Learning with Supervised Imitation | 5 | 6 | 2 | 2 | Yi breaks down the technical distinction between on-policy reinforcement learning and off-policy supervised fine-tuning using human sports and learning analogies. | |
| Translating Machine Learning Optimization Principles to Human Learning | 7 | 2 | 2 | 3 | The host drives the thesis that ML optimization dynamics, specifically learning rate adjustments and Bayesian updates, map directly onto human cognitive shifts. | |
| Parallel Trajectories, Self-Consistency, and Chain of Thought Reasoning | 6 | 4 | 1 | 2 | Host and guest discuss the nuances between parallel test-time sampling, chain of thought, and self-consistency mechanisms in frontier models. | |
| Subsuming Specialized Symbolic Engines into General-Purpose Parameters | 6 | 5 | 2 | 3 | Yi argues that general connectionist neural parameters will inevitably subsume specialized symbolic engines like Lean or external calculators. | |
| Multi-Timezone Captaincy and Live Olympiad Verification Dynamics | 4 | 6 | 1 | 1 | Yi details the collaborative mechanics of co-captaining the IMO run across multiple international timezones while monitoring live human participant benchmarks. | |
| Reflecting on Five Years of AI Progress and the IMOCAT Run | 5 | 4 | 1 | 2 | Discussion reflects on the pace of AI advancement over five years and clarifies the origin of the IMOCAT experiment config name. | |
| Evaluating Long-Horizon Agent Planning Through Pokemon Benchmarks | 6 | 4 | 1 | 2 | Host and guest evaluate Pokemon as a contamination-resistant long-horizon benchmark testing spatial reasoning, state understanding, and deep research. | |
| Novel Knowledge Discovery vs. Synthesis in the AI Scientist Paradigm | 6 | 4 | 3 | 4 | The host challenges whether guide-following is true intelligence versus inventing novel concepts from scratch like the Transformer architecture. | |
| Demystifying Reasoning: Discrete Chains of Thought vs. Latent Thinking | 6 | 4 | 2 | 2 | Yi demystifies reasoning into post-training RL elicitation and discusses latent thinking representations versus discrete token generation. | |
| Web Contamination, Synthetic Reasoning Loops, and Code Generalization | 6 | 3 | 2 | 2 | The host questions web data contamination with synthetic reasoning tokens, leading into Yi sharing how AI coding tools accelerated his ML workflows. | |
| AI as an Augmentation Aura and Managing Model Laziness | 5 | 4 | 1 | 2 | Yi characterizes AI as an augmentation aura that saves time across an engineering organization, while acknowledging occasional model laziness. | |
| Compounding Datasets and the Collective Nature of AI Scaling | 6 | 4 | 1 | 2 | Discussion centers on whether Transformer attention remains sufficient for AGI or if fundamental architecture replacements will emerge. | |
| Scaling Context Lengths, Continual Learning, and Local Minima | 7 | 4 | 2 | 4 | The host challenges the sequence-to-sequence assumption for massive 200M context windows, prompting Yi to discuss gradient learning paradigms and local minima traps. | |
| The Sweet Lesson: Research Ideas and the Expanding Closed-Lab Gap | 5 | 5 | 2 | 2 | Yi proposes the Sweet Lesson to highlight that research ideas and algorithmic innovations remain decisive, widening the gap between closed labs and open-source. | |
| Memory vs. Compute Constraints in Next-Generation Hardware | 6 | 3 | 2 | 3 | Host outlines memory bandwidth and networking bottlenecks across hardware generations, while Yi clarifies his focus stays strictly on algorithmic modeling. | |
| Categorizing World Models and Optimizing Compute Allocation Per Token | 7 | 4 | 2 | 3 | The host synthesizes three paradigms of world models, while Yi reframes learning efficiency into spending higher FLOPs per token. | |
| The Economics and Market Demand for Specialized RL Environments | 6 | 3 | 2 | 3 | Host probes the market valuations of proprietary RL simulation environments before transitioning into DSI and recommendation systems. | |
| Comparing Modeling Dynamics in Information Retrieval and Language Tasks | 6 | 6 | 3 | 2 | Yi candidly describes the modeling dynamics of IR and recommendation systems as counter-intuitive and disconnected from pure ML progress. | |
| Launching the GDM Singapore Symposium and Meeting Government Leadership | 4 | 5 | 1 | 1 | Yi recaps organizing the DeepMind Singapore symposium with Jeff Dean and Quoc Le, including their briefing with Singapore government leadership. | |
| Establishing Frontier AI Research in Singapore vs. Silicon Valley | 5 | 4 | 1 | 2 | Discussion contrasts building frontier research teams in Singapore versus the hyper-saturated monoculture of the San Francisco Bay Area. | |
| Hiring Philosophy at GDM Singapore: High-Stat Talent and Research Taste | 5 | 5 | 1 | 2 | Yi details hiring criteria focused on high general cognitive stats and independent research taste over formal RL specialization. | |
| Physical Biohacking, Health Metrics, and Sustaining Research Energy | 4 | 4 | 2 | 2 | Yi shares his health transformation metrics and how physical conditioning sustains stamina for intense frontier AI research. |
Statements from this episode (34)
DeepMind Singapore Explicitly Adds AGI to Job Postings
“I think that, like, one reason why we work on these models is that we want to get to AGI, and, like, this was a right thing that we added AGI to the job posting, yeah. There is no, like, formal name of the team yet, but it's basically the Gemini theme Singapor…”
Yi Tay: I had almost no RL background before returning to DeepMind
“I spent a lot of my past life, I call it the past art, working on like architectures and pre-training, but I think now I more, I have like transitioned more into RL. I'm not like old school RL, but the games RL and the old school RL, and to be honest, I had al…”
Yi Tay: Reinforcement learning is the primary AI modeling toolset today
“So I think RL is basically the main modeling tool set that we play around with these days.”
Yi Tay: On-policy RL is more generalizable than imitation fine-tuning
“So I think on policyness is basically this idea of like model training on its own outputs and letting the model like generate its own trajectories and then letting some reward verify it and then the model train its own outputs. I think this is more generalizab…”
DeepMind Abandoned AlphaProof to Run Gemini End-to-End for IMO Math
“We wanted to try to, like, use, actually use Gemini as an end-to-end model. Basically, no, no second system with alpha proof. No second system. In, text out.”
Gemini's IMO Model Checkpoint Required Only One Week of Training
“The training process of this IMO model itself was, like, maybe a week or so.”
DeepMind Shipped Full Gemini IMO Config Only to Select Mathematicians
“So the inference time config was like the one serve to most people is different, but And the full IMO, like, inference config was presented, like, shipped to some mathematicians just because of the inference cost, right? But that was good enough to be a genera…”
Tay: Most specialized tools will be subsumed directly into model parameters
“Then the most I can see in the future is there'll be a model then that, that is, there's something that really cannot be subsumed by a model. Then you just use a tool or something, right? But my prediction is that I think most things can be subsumed by the mod…”
Yi Tay: Four captains across three locations trained DeepMind's IMO model
“So I think there were four captains for the IMO, two from London. Jonathan was from Mountain View. I was from Singapore. So I think four of us basically trained this model together.”
Tay: Present AI Milestones Would Have Been Viewed as AGI Five Years Ago
“If you just look at the AI progress now and five years ago, I think people would think that we already reached like AGI.”
Tay: DeepMind did not optimize Gemini specifically for Pokémon
“There's actually nothing specifically done for Pokemon.”
Yi Tay: Today's Models Likely Couldn't Invent the Transformer from Pre-2015 Data
“Even today's models, they might not even be able to invent the transformer. Like, if you freeze the time at a certain time, and even you bring the time, I mean, the model is a transformer, so I just say there's no, assuming there's no leakage.”
Yi Tay: 'Reasoning' Technically Just Means Post-Training RL with Thinking Trajectories
“So I think the actual, like, technical definition of reasoning is making models better with thinking and post-training. Ok? Yeah. So basically, like, RL-ing the model to think better.”
Yi Tay: AI model thoughts do not need to resemble human thoughts
“Generally, I'm not really, I don't really believe that model thoughts have to be the same with human thoughts. I'm actually like, generally in ML, I'm more of the school of thought of let the model do whatever it wants.”
Yi Tay Fixes ML Bugs Automatically Using AI Without Reading Error Traces
“I think AI coding has started to become the point where I run a job, I get a bug. I almost don't look at the bug. I paste it into, like, anti-gravity, and, like, I throw it, that will fix the bug for me. And then I relaunched the job. And, like, beyond, like, …”
Yi Tay: AI Tools Act as a Team Productivity Aura Rather Than Replacing Engineers
“These things are not like going to replace one person as it is, but more like a passive aura that buffs everybody.”
Yi Tay: AI Model Laziness and Edge Flaws Will Disappear via General Scaling
“I don't think there's anything that to be done to specifically like focus fire. These things is more like general capability improvements. The models just get better over time and then these things will just like go away.”
Yi Tay: AI progress is driven by compounding small incremental changes
“I think that it's true that sometimes a lot of progress on the whole is just a series of small incremental changes that, yeah, that push. I think that's accurate. That's true. There's also, it also feels that there's also a lot of like small, like seemingly mi…”
Yi Tay: The Architecture That Achieves AGI Will Still Be a Transformer
“It will be a transformer, I think. Like people, it depends on what you call it, but I think unless the paradigm shifts completely, which is, I mean, as a scientist, you cannot like completely say no to like that, this would never happen. But my feeling is that…”
Yi Tay: Efficient attention research consistently failed to eliminate self-attention
“There was this whole big era, which I was also involved in this era, where people try to like undermine the attention as much as possible. Like they try to remove it, simplify it, make it efficient, like this whole Like, efficient attention era. At the end of …”
Yi Tay: Gradient descent learning paradigm is AI's bottleneck, not architecture
“It's not architecture itself. That's, that there's a problem that we, that is more of like the learning paradigm itself rather than the architecture itself. I think the architecture is just basically like the interface between the learning algorithm and the to…”
Yi Tay: The AI Industry Is Stuck in a Transformer Local Minimum
“So now we are like in this local minima of like transformers, everything, everything, right? Maybe it's not easy to like get totally out Of this, because also a lot of people's investment optimization have been done. So the things that play well needs to play …”
Yi Tay: The 'Bitter Lesson' Is Overapplied; Architectural Ideas Fundamentally Matter
“The bitter lesson gets used too much in, like, too conveniently used around, but actually there's also a little bit of a, not a bit, there's also a sweet lesson where it's like, ideas matter.”
Yi Tay: AI Research Has Not Entered Diminishing Returns on Ideas
“The number of ideas that actually work is not decreasing compared to the last, like, we're not in the era of diminishing returns yet.”
Yi Tay: Gap Between Closed AI Labs and Open-Source Is Increasing
“I think the gap is definitely increasing.”
Yi Tay: AI Labs Prioritize Data Efficiency Because the World Lacks Tokens
“I think in general, the, like learning more, like extracting more from varied data points is definitely valuable, but I think that's what related to the fact that we're like running out of Tokens in the world.”
Yi Tay: The term 'world models' is not well-defined in AI
“I don't think about world models that often. I think because world models are just not really well defined in the first place.”
Yi Tay: Data-Bound AI Must Spend More Compute Per Token to Scale
“So if you are, you come to a point where you are Very data bound, but not compute bound at all. You just find algorithms that spend a lot of compute on every token.”
Yi Tay: IR and RecSys research lags significantly behind NeurIPS and ICML
“Also the IR community and the retrieval community is also like always behind the mainstream. And then now it's just probably gotten even more worse because of ILM and stuff. So, okay, I'm getting into Hottick territory, but it's just, like, certain conferences…”
Yi Tay: Top Southeast Asian AI talent requires prominent leadership to unlock
“I feel like the talent we can get from the region is really, really good, but it's only It's only because it's us, we can unlock this talent. Otherwise, might join some other place.”
Yi Tay: AI research benefits from escaping the Bay Area monoculture
“I do think that to some extent, if you want to do research in, you need a little bit of peace and quiet somewhere, right? So this island may be good for that, but then you can, you're still, like, able to, like, be connected, right?”
DeepMind Singapore is keeping its Gemini RL team small to maximize compute per capita
“We're hiring, like, my team will work on like RL and reasoning for Gemini and Gemini deep thing. I think we care more about like talent density now. So we're not like also like growing that big, this small team first, just because compute per capita is probabl…”
Yi Tay: ML and RL Knowledge Can Be Learned Easily by Engineers
“ML. ML can be learned easily. Our knowledge can be learned easily.”
Yi Tay: Independent research taste is a stronger hiring signal than execution
“If somebody comes up with something and then does something that you feel that is very tasteful and it aligns with what Like researchers in the labs, like one, and they come up with that independently, you know that the function that is good, right? Like if yo…”