Transformer
also referred to as: transformers
15 statements across 11 episodes · 7 bullish · 3 bearish · 11 people on the record · first statement Jan 22, 2020 by Clement Delangue · said 8 times in 5 episodes since 2015 · across every show →
Mentions by year
brought up most by Yann Dubois (2), William Falcon (2), Matt Turck (2), Mike Knoop (1), Jake Porway (1)
tap a year for its mentions
2026 2 mentions in 1 episode
2025 1 mention in 1 episode
2024 2 mentions in 1 episode
2023 2 mentions in 1 episode
Everything said about Transformer, oldest first
Jan 22, 2020 neutral
Jan 22, 2020 positive
May 1, 2023 positive
Falcon: Modern generative AI would not exist without open-source research
“None of this wouldn't exist if Google hadn't put published transformers and put the paper out there, right? None of this would have existed if Ah, the attention stuff would have been out there if the Anno hadn't been created, if TensorFlow hadn't been created.”
Oct 10, 2024
Jun 5, 2025 bearish
Gomez: Discrete diffusion models will not replace the Transformer
“Now there are these discrete diffusion models, which do diffusion, which has been super popular for, like, image understanding, image generation. It's doing that same process for language models, but I still don't see that replacing the transformer.”
Jun 5, 2025
Jun 5, 2025 neutral
Gomez: Hardware lock-in raises the bar to replace Transformer architectures
“They built so much infrastructure specialized to the transformer. And so it's like we dug ourselves into this. Well like we now have chips that are being optimized explicitly to that architecture. And so to move architecture, it requires so much effort, energy…”
Oct 2, 2025 bullish
Nov 26, 2025 positive
Dec 18, 2025 positive
Jan 15, 2026 bearish
Apr 2, 2026
Dehghani initially thought Google's Transformer architecture was random and would die
“And I was like, I don't know if I want to go with this team. It's just like, they're doing something random. Like who, like everybody's doing LST. I'm like, why should I go and work with like a group of people who are working on this like random architecture, …”
Apr 2, 2026 positive
Unified Transformer architectures simplified the training of natively multimodal AI models
“Even if this is not, like, the only architecture that would be, like, in a multi-model, but it made it really simple to train these models, like, natively, because you have, like, a single architecture and you can have all the modalities in, during training.”
Jul 2, 2026 positive
Jul 16, 2026 negative
Katti: Turbine and transformer manufacturers face multi-year lead times to expand capacity
“Those industries have historically have not added much capacity for the last decade or so ago, and they've suddenly experienced a demand shock. And it takes years before you can add capacity to produce more turbines and transformers.”