Yi Tay
Senior Staff Research Scientist, Google DeepMind. On 1 show, 2 appearances. The Shows tab opens the full record on each.
scientistfounderexecutiveengineer@YiTayML ↗LinkedIn ↗yitay.net ↗
Yi Tay leads reasoning and RL-driven post-training research at Google DeepMind Singapore, serving as model co-lead for the Gemini Deep Think team. He previously co-founded foundation model startup Reka AI as Chief Scientist and co-led major AI projects at Google Brain, including PaLM 2, UL2, and Flan.
1shows
2appearances
68statements
3resolved
2supported
1contradicted
67%fully supported
6said about them ↓ Everything Yi Tay said on any show that made the record, most notable first. Each card names its show and opens the statement there.
Yi Tay: Gap Between Closed AI Labs and Open-Source Is Increasing
“I think the gap is definitely increasing.”
Yi Tay: IR and RecSys research lags significantly behind NeurIPS and ICML
“Also the IR community and the retrieval community is also like always behind the mainstream. And then now it's just probably gotten even more worse because of ILM and stuff. So, okay, I'm getting into Hottick territory, but it's just, like, certain conferences…”
Tay: Frontier AI researchers cannot maintain standard nine-to-five work-life balance
“You cannot be, like, checking out on, like, Friday, Saturday, Sunday, and, like, work at, like, nine to five if you want to, like, Make progress, or like, some people are just so good at detaching, like, ok, like, you know, like, eight pm, I'm not going to, my…”
Tay: Long context architecture is the future of AI over RAG
“And, yeah, I mean, I think long context is definitely the future, rather than rec. But I mean, they could be used in conjunction, like,”
DeepMind Abandoned AlphaProof to Run Gemini End-to-End for IMO Math
“We wanted to try to, like, use, actually use Gemini as an end-to-end model. Basically, no, no second system with alpha proof. No second system. In, text out.”
Tay: Most specialized tools will be subsumed directly into model parameters
“Then the most I can see in the future is there'll be a model then that, that is, there's something that really cannot be subsumed by a model. Then you just use a tool or something, right? But my prediction is that I think most things can be subsumed by the mod…”
Yi Tay: Today's Models Likely Couldn't Invent the Transformer from Pre-2015 Data
“Even today's models, they might not even be able to invent the transformer. Like, if you freeze the time at a certain time, and even you bring the time, I mean, the model is a transformer, so I just say there's no, assuming there's no leakage.”
Yi Tay Fixes ML Bugs Automatically Using AI Without Reading Error Traces
“I think AI coding has started to become the point where I run a job, I get a bug. I almost don't look at the bug. I paste it into, like, anti-gravity, and, like, I throw it, that will fix the bug for me. And then I relaunched the job. And, like, beyond, like, …”
Yi Tay: The Architecture That Achieves AGI Will Still Be a Transformer
“It will be a transformer, I think. Like people, it depends on what you call it, but I think unless the paradigm shifts completely, which is, I mean, as a scientist, you cannot like completely say no to like that, this would never happen. But my feeling is that…”
Yi Tay: Gradient descent learning paradigm is AI's bottleneck, not architecture
“It's not architecture itself. That's, that there's a problem that we, that is more of like the learning paradigm itself rather than the architecture itself. I think the architecture is just basically like the interface between the learning algorithm and the to…”
Yi Tay: The AI Industry Is Stuck in a Transformer Local Minimum
“So now we are like in this local minima of like transformers, everything, everything, right? Maybe it's not easy to like get totally out Of this, because also a lot of people's investment optimization have been done. So the things that play well needs to play …”
Yi Tay: The 'Bitter Lesson' Is Overapplied; Architectural Ideas Fundamentally Matter
“The bitter lesson gets used too much in, like, too conveniently used around, but actually there's also a little bit of a, not a bit, there's also a sweet lesson where it's like, ideas matter.”
Yi Tay: AI Research Has Not Entered Diminishing Returns on Ideas
“The number of ideas that actually work is not decreasing compared to the last, like, we're not in the era of diminishing returns yet.”
Tay: Newly provisioned GPU clusters are highly unreliable and require node filtering
“Usually when, like, a provider, like, provisions new notes, or they would, like give us... Yeah, it's usually, like, bad, like, dog shit, like, at the start. And then it gets, like, better as you go through the process of, like, returning notes, like, and, you…”
Yi Tay: Llama 3 shows Meta may have caught up to Google
“So I think I don't really follow, like, fine much, but I think that, like, Lama Tree actually shows that, like, kind of, like, Meta got a pretty, like, a good stack around training these models you know, like, oh, and I've even started to feel like, oh, they a…”
Tay: Zero-shot benchmark scores at 1B model scale are random chance
“Every time some people propose like this, they run like some zero-shot score on like some LM event harness or something like that, and you know like at one B scale, all the numbers are random, basically. Like all your bull kill, they're all like random chance …”
Tay: Mixture-of-Experts is fundamentally the right architecture for scaling
“Fundamentally, I just think that MOEs are just, like, the way to go in terms of, like, floppyram ratio, they bring the benefit from the scaling curve, if you do it right, if you, they bring the benefit from the scaling curve, right, and then, Like, that's, lik…”
Tay: Hugging Face's Open LLM Leaderboard is a major problem
“The open LM leaderboard is, like, probably, like, the, a big, like, Problem, to be honest.”
Yi Tay: On-policy RL is more generalizable than imitation fine-tuning
“So I think on policyness is basically this idea of like model training on its own outputs and letting the model like generate its own trajectories and then letting some reward verify it and then the model train its own outputs. I think this is more generalizab…”
Gemini's IMO Model Checkpoint Required Only One Week of Training
“The training process of this IMO model itself was, like, maybe a week or so.”
Tay: Present AI Milestones Would Have Been Viewed as AGI Five Years Ago
“If you just look at the AI progress now and five years ago, I think people would think that we already reached like AGI.”
Yi Tay: AI model thoughts do not need to resemble human thoughts
“Generally, I'm not really, I don't really believe that model thoughts have to be the same with human thoughts. I'm actually like, generally in ML, I'm more of the school of thought of let the model do whatever it wants.”
Yi Tay: AI Tools Act as a Team Productivity Aura Rather Than Replacing Engineers
“These things are not like going to replace one person as it is, but more like a passive aura that buffs everybody.”
The other half of the tape: Yi Tay's own voice is left out of every number here. Other people bring the name up 6 times in 4 episodes across the shows. every mention, with the transcript →
Who brings them up most Shawn Wang 3Swix (Shawn) 2
Every mention by year
tap a year for its mentions
Latent Space 6
2026 3 mentions in 2 episodes 2 per episode
2024 3 mentions in 2 episodes 2 per episode
One line per show, most statements first. The link opens Yi's full record on that show: the calibration, argument clarity, speaking style and every statement made there.
| Show | Role there | Eps | Statements | Record | |
|---|---|---|---|---|---|
| LATENT SPACELEDGER | Senior Staff Research Scientist, Google DeepMind | 2 | 68 | 67% 2/3 | full record on Latent Space → |