StarZero All shows Try the podcast clipper →

Transformer

also referred to as: transformers

8 statements across 8 episodes · 0 bullish · 4 bearish · 8 people on the record · first statement May 15, 2023 by Yann LeCun · said 7 times in 5 episodes since 2024 · across every show →

Mentions by year

brought up most by David Luan (2), Aidan Gomez (2), Mikey Shulman (1), Demis Hassabis (1), Cliff Weitzman (1)

tap a year for its mentions
002142202420252026episodesmentions
012202420252026episodes it came up in
001122202420252026episodesmentions per episode
2026 2 mentions in 2 episodes 1 per episode
2025 1 mention in 1 episode
2024 4 mentions in 2 episodes 2 per episode

every mention, scene by scene, with the transcript →

Everything said about Transformer, oldest first

May 15, 2023 negative
Assertion Contradicted
LeCun: All Google Transformer Paper Authors Have Left for Startups
“So you look at the original paper from Google about BERT or Transformers, right? The thing that revolutionized NLP, all of them have left. Okay. They're all in startups.”
Yann LeCun May 15, 2023 ▶ 1:02:04 Yann LeCun: Meta’s New AI Model LLaMA; Why Elon is Wrong about AI; Open-source AI Models | E1014 · 20VC with Harry Stebbings
Aug 18, 2023
Insight
Socher: GPU incompatibility prevents exploration of non-transformer AI models
“Transformers are mostly amazing because they can be trained on GPUs. There are lots of other models that we're currently not exploring because they're not trainable on GPUs.”
Richard Socher Aug 18, 2023 ▶ 25:31 Richard Socher: The 3 Biggest Barriers to Building AGI; AI Startups vs Incumbents | E1050 · 20VC with Harry Stebbings
Nov 15, 2023 bearish
Opinion
Traynor: Google squandered its early AI leadership despite inventing Transformers
“Feels like Google, who kind of owned all this at the start, bear in mind with the original paper on Transformers, feels like they have, like, rescued defeat from the jaw of victories, in a sense.”
Des Traynor Nov 15, 2023 ▶ 41:04 Des Traynor: How to Survive and Thrive in a World of OpenAI | E1082 · 20VC with Harry Stebbings
Sep 4, 2024
Insight
Kolter: AI model architectures and Transformers no longer matter
“I, for a lot of my career, was thinking that model architectures really mattered, and by having clever, complex architectures and sub-modules inside architectures, you would have, that was the route to sort of better AI systems. For the most part, I don't beli…”
Zico Kolter Sep 4, 2024 ▶ 59:25 Zico Kolter: OpenAI's Newest Board Member on The Biggest Questions and Concerns in AI Safety | E1197 · 20VC with Harry Stebbings
Mar 24, 2025 bearish
Prediction Open · timeframe Mar 2030
Feldman: AI will rely less on transformers in 3 to 5 years
“We won't be as dependent on transformers in three years or five years as we are now.”
Andrew Feldman Mar 24, 2025 ▶ 34:05 Andrew Feldman, Cerebras Co-Founder and CEO: The AI Chip Wars & The Plan to Break Nvidia's Dominance · 20VC with Harry Stebbings
Jun 2, 2025 neutral
Assertion Supported
Varun Mohan: AI model architectures converged almost entirely on transformers
“There were going to be hundreds of different model architectures and we were wrong. All of them ended up being transformers.”
Varun Mohan Jun 2, 2025 ▶ 3:03 Windsurf CEO & Co-Founder, Varun Mohan: AI's Biggest Acquisition to Date! · 20VC with Harry Stebbings
Sep 1, 2025 negative
Assertion Not checkable as stated
Frosst: Google failed to quickly commercialize or scale the Transformer
“It wasn't then commercialized very quickly within Google. It wasn't scaled up very quickly within Google. A lot of that work had to be done elsewhere and years later.”
Nick Frosst Sep 1, 2025 ▶ 2:41 Cohere Founder, Nick Frosst: How To Compete with OpenAI & Anthropic, and Sam Altman’s AI Disservice · 20VC with Harry Stebbings
Aug 2, 2026
Insight
Angelopoulos: Data is the hardest part of model training as algorithms commoditize
“The data is really the hardest part of model training. Because you need to source it. It's so dirty. Nobody wants to do that shit. Nobody wants to hire all these people to generate data and then, you know, turn that into basically data plus GPUs equals model. …”
Anastasios Angelopoulos Aug 2, 2026 ▶ 46:24 Arena CEO: There Will be a $100BN US Open-Source Model & Data is a Trillion Dollar Market
Made with StarZero

Turn any episode into a week of clips.

This entire site, over 1,200 episodes transcribed, diarized, checked and made playable, runs on the StarZero media pipeline. Drop in your own episode and the podcast clipper finds the moments worth sharing, cuts them, captions them, and reframes them for every feed.

Try the podcast clipper Learn more