Transformer, every mention
89 scenes across 22 shows · ← back to Transformer
tap a year for its mentions
Acquired 33
No Priors 13
the MAD Podcast 8
20VC 7
How I Built This 6
Big Technology 6
Latent Space 5
All-In 414 more shows
every year every show Acquired 33 No Priors 13 the MAD Podcast 8 20VC 7 How I Built This 6 Big Technology 6 Latent Space 5 All-In 4 
WTF is with Nikhil Kamath 3 
the Y Combinator Startup Podcast 2 
In Depth 2 
Innovators & Investors 2 
the Knowledge Project 2 
the a16z Podcast 2 
TBPN 2 
Founder's Journal 1 
Cheeky Pint 1 
We Live to Build 1 
the Startup Ideas Podcast 1 
Invest Like the Best 1 
Sourcery 1 
Top Founders 1
Verbatim, from the transcripts: passages where Transformer comes up on Acquired, No Priors, the MAD Podcast, 20VC, How I Built This
How to Build Your Own Data Center & Why Every Startup Should Do It
- ▶ 2:07 Cliff Weitzman So if you look at it, the Transformer was invented inside of Google in 2017.
OpenAI's Bots Break Containment and Hack Hugging Face Autonomously — With Alex Stamos
- ▶ 1:59 Alex Kantrowitz Just to put it in context, it's the first time, this is from Transformer, the breach appears to be the first known example of a misaligned AI escaping containment and autonomously carrying out a cyber attack on a third party.
Kimi K3 Reactions, The Odyssey Hits $264M in Debut, Netflix Embraces AI | Diet TBPN
- ▶ 28:09 John Coogan But Marvel, Fast and the Furious, and Transformers have all stumbled at the box office recently, as audiences have grown tired of them.
Apple Sues OpenAI, Experts Sign Another AI Warning Letter, Paramount Threatens CA Exit
- ▶ 30:38 John Coogan I'm pretty sure SpaceX accidentally built the Transformer, at least that's what it sounds like here.
Harvey Co-Founder Gabe Pereyra on the Token Pricing Reckoning Coming for AI · Sourcery with Molly O'Shea
- ▶ 11:26 Gabe Pereyra And I think part of the outcome there is you had someone like Noam Shazir and Ashish Viswani and kind of the rest of the folks that invented transformers.
AI Pioneer Geoffrey Hinton: AI Is Conscious, Superintelligence is Coming, And We Should Be Worried
- ▶ 8:06 Geoffrey Hinton We've also seen new ideas, but, but mainly since Transformers, it's been much better hardware, many more resources, um, better engineering, and many more talented people.
OpenAI's Yann Dubois: Why AI Progress Suddenly Feels Real
- ▶ 13:32 Yann Dubois So that was, uh, uh, right when Transformers started. 2 times in the scene
Demis Hassabis: Why AGI is Bigger than the Industrial Revolution & Where Are The Bottlenecks in AI · 20VC with Harry Stebbings
- ▶ 7:01 Demis Hassabis So one of our groups, um, if you think of like AlphaGo and reinforcement learning, and of course Transformers,
The history and future of AI at Google, with Sundar Pichai
- ▶ 1:08:04 John Collison We're talking a lot about initiatives that started small at Google, like the Transformer, which is not Google's main priority, you know, when that initiative started.
AI for Atoms: How Periodic Labs is Revolutionizing Materials Engineering with Co-Founder Liam Fedus
- ▶ 12:46 Liam Fedus But Transformers have been very powerful.
Moonlake: Interactive, Multimodal World Models — with Chris Manning and Fan-yun Sun
- ▶ 21:20 Chris Manning Um, you know, although that's true, all of the weights of the model that are internal to the Transformer, they are a joint model of the model's understanding of the world, and so I think you can think of the weights of the model as a form…
World's Top Researcher on AI, LLMs, and Robot Intelligence · Invest Like The Best
- ▶ 12:46 Patrick O'Shaughnessy Are there hash mark equivalents on the timeline of, like, AlexNet or the Transformer?
Google DeepMind CEO Demis Hassabis: AI's Next Breakthroughs, AGI Timeline, Google's AI Glasses Bet
- ▶ 3:15 Demis Hassabis At maximum, with maximum, uh, force, both, you know, scaling up, uh, the current paradigms and ideas, and, and, and when I say scaling up, that also involves innovation, by the way, um, pre-training especially, I think we're very strong…
Captaining IMO Gold, Deep Think, On-Policy RL, Feeling the AGI in Singapore — Yi Tay
⚡ Open Model Pretraining Masterclass — Elie Bakouch, HuggingFace SmolLM 3, FineWeb, FinePDF
- ▶ 1:11 Elie Bakouch So I think like most of the people know Hugging Face for like the, the hub or like the transformer library.
Google Part III: The AI Company. Google is amazingly well-positioned... will they win in AI? (Audio) · Acquired
- ▶ 1:52:14 David Rosenthal Because this is the Transformer paper.
- ▶ 1:52:17 Ben Gilbert Okay, so where did the Transformer come from?
- ▶ 1:53:26 Ben Gilbert And this was the year before the Transformer paper would come out.
- ▶ 1:53:29 David Rosenthal Before the Transformer paper, yes.
- ▶ 1:57:09 David Rosenthal They decide that they're gonna call this new technique the transformer.
- ▶ 1:57:27 David Rosenthal That's not not why they named it the transformer.
- ▶ 1:57:30 David Rosenthal And it's taking in the giant corpus of text and storing it in a compressed format, right?
- ▶ 1:58:05 David Rosenthal And it's a good thing he did, because before Gnome joined the project, they had a working implementation of the transformer, but it wasn't actually producing any better results than LSTMs.
- ▶ 1:58:16 David Rosenthal Gnome joins the team, basically pulls a Jeff Dean, rewrites the entire code base from scratch, and when he's done, the transformer now crushes the LSTM-based Google Translate solution.
- ▶ 2:00:27 David Rosenthal Yeah, there's an elegance to the Transformer.
- ▶ 2:01:45 David Rosenthal It is a false narrative out there that Google did nothing with the transformer after the paper was published.
- ▶ 2:03:08 David Rosenthal And also, of course, within a couple years, all eight authors of the Transformer paper had left Google to either start or join
- ▶ 2:03:48 David Rosenthal With the Transformer, yes.
- ▶ 2:03:53 Ben Gilbert And does anyone think the Transformer's a big deal over there?
- ▶ 2:04:02 David Rosenthal Right after Google publishes the Transformer paper, in September of 2017, Elon gets really, really fed up with what's going on at OpenAI.
- ▶ 2:05:56 David Rosenthal The sort of panic that this throws OpenAI into is the catalyst that makes them reach for the Transformer and say, all right, we gotta figure things out.
- ▶ 2:06:11 Ben Gilbert I don't know if during this personal tension between Elon and Sam, if they had already decided to go all in on Transformers or not.
- ▶ 2:06:25 Ben Gilbert Transformers, language models, we're going all in on that.
- ▶ 2:07:33 David Rosenthal So in June of 2018, OpenAI releases a paper describing how they have taken the transformer and developed a new approach
- ▶ 2:08:14 Ben Gilbert Which we should say is right around the same time as BERT and right around the same time as another large language model based on the Transformer out of here in Seattle, the Allen Institute.
- ▶ 2:20:33 David Rosenthal So, Noam Shazir, incredible engineer, re-architected the Transformer, made it work, one of the lead authors of the paper, storied career within Google, has all of this sway, should have all of this sway within the company.
- ▶ 2:20:48 David Rosenthal After the Transformer paper comes out, he and the rest of the team are like, guys, we can use this for a lot more than Google Translate.
- ▶ 2:20:59 Ben Gilbert Are you about to read the Transformer paper?
- ▶ 2:21:07 David Rosenthal We plan to extend the transformer to problems involving input and output modalities other than text
- ▶ 2:21:33 David Rosenthal Hey, I think this is going to be so big, the transformer, that we should actually consider just throwing out the search index
- ▶ 2:21:42 David Rosenthal And the 10 blue links model and go all in on transforming all of Google into one giant transformer model.
- ▶ 2:21:51 David Rosenthal And then Gnome actually goes ahead and builds a chatbot interface to a large transformer model.
- ▶ 2:25:17 David Rosenthal Gnome knew, and people in Google knew, that you could make a chatbot interface to a transformer-based LLM, and that was a really compelling product.
- ▶ 2:57:56 Ben Gilbert 2017, obviously, the Transformer comes out.
- ▶ 2:57:58 Ben Gilbert They incorporate some learnings from the Transformer, especially around prediction and planning.
Anthropic Head of Pretraining on Scaling Laws, Compute, and the Future of AI · Y Combinator
- ▶ 55:00 Ankit Gupta One is, uh, not using Transformer as an architecture. 2 times in the scene
Google Part II: Alphabet (Audio) · Acquired
- ▶ 3:40:43 David Rosenthal Andrew Ng from Stanford, Kwok Lee, Oriol Vinales, and oh yeah, in addition to all of those people, the authors of the Transformer paper because Google invented the Transformer and published the paper in June of 2017.
- ▶ 3:41:10 David Rosenthal When I was talking to folks in the research for this episode and AI came up, one of them said, you know, I have to remind people when I'm talking to partners out in the ecosystem that the T in chat GPT stands for transformer and that we…
EP10: Robots on the Move: Disrupting Delivery with Ottonomy ft Ritukar Vijay
- ▶ 32:30 Ritukar Vijay And, you know, pun intended, I mean, the Transformer movie also came in with Autobots. 2 times in the scene
Netflix Founder Reed Hastings on Scaling High-Trust Culture & Bold Judgment
- ▶ 42:04 Shane Parrish Sometimes I see these recommendations, you know, when they were younger, and I'm like, Transformer. 2 times in the scene
Google DeepMind CTO: Advancing AI Frontier, New Reasoning Methods, Video Generation’s Potential
- ▶ 26:45 Alex Kantrowitz The transformer came from Google. 2 times in the scene
Nikhil Kamath x YouTube CEO, Neal Mohan | People by WTF Ep. 9 · Nikhil Kamath
- ▶ 37:01 Nikhil Kamath Uh, the one thing that stood out to me is you grew up on things like Transformers and baseball and Star Wars. 3 times in the scene
Chasing Real AGI: Inside ARC Prize 2025 with Chollet & Knoop
- ▶ 42:55 Mike Knoop One, we were trying to emulate a bit more of what the academic environment around AI research looked like during the 2010 to 20 twenties, where, you know, if you look at even where we got to with language models, that was built on, you…
NVIDIA VP Rev Lebaredian Talks Plan To Build AI That Understands The Real World
- ▶ 22:07 Alex Kantrowitz Now you mentioned that the transformer model was an important development on this path.
Mikey Shulman, CEO @Suno: The Future of Music, What is Gonna Happen? | E1244 · 20VC with Harry Stebbings
- ▶ 12:45 Mikey Shulman I think, um, yeah, I mean, uh, I'm very happy we use Transformers because it lets us kind of, uh, get that to you quicker.
The $4.5B Platform Driving the Open Source AI Revolution | Clem Delangue, CEO, Hugging Face
- ▶ 0:10 Matt Turck HuggingFace has had a remarkable rise in the AI space, from creating the Transformers library to becoming a key platform where the AI community collaborates on models, datasets, and applications, most recently valued at 4.5 billion dollars.
- ▶ 34:58 Matt Turck Was it Transformers?
No Priors Ep. 80 | With Andrej Karpathy from OpenAI and Tesla
- ▶ 8:11 Andrej Karpathy okay, they're being reconfigured from a car, like a transformer, the movie.
Aidan Gomez: What No One Understands About Foundation Models | E1191 · 20VC with Harry Stebbings
- ▶ 34:36 Aidan Gomez Uh, in the Transformer times, around that time, um, because he was big in the Toronto scene.
- ▶ 46:49 Aidan Gomez Well, they weren't in the sense that, like, two weeks after we released the Transformer paper, we started training language models.
David Luan: Why Nvidia Will Enter the Model Space & Models Will Enter the Chip Space | E1169 · 20VC with Harry Stebbings
- ▶ 1:36 David Luan Like, the people who invented Transformer, the people who invented the diffusion model, people who did, um, all of these new optimization techniques that we all take for granted today, they were all a brand at the same time.
- ▶ 7:14 David Luan DeepMind was that the next phase of AI after Transformer was not going to be about, you know, research paper writing.
Watch this if you want free $1m-$100m business ideas (with Kevin Rose)
- ▶ 29:17 Kevin Rose And, you know, that meant, um, you know, the A-Team, MacGyver, um, you know, Scooby-Doo, Voltron, Transformers, uh, I'm talking about the original ones, you know, way back in the day, the original Nintendo, like, that's kind of my…
AI is smarter than you think with Shane Legg of Google DeepMind
- ▶ 18:06 Shane Legg We, at Google, had invented an extremely powerful algorithm called Transformer.
The Bard blueprint | Creating value, shipping fast, and advancing AI | Jack Krawczyk (Google)
- ▶ 10:29 Jack Krawczyk The first version of the Transformer was written about in 2017. 2 times in the scene
What is the Future of Hollywood?
- ▶ 2:01:01 Marc Andreessen And you could deploy it into another Marvel movie or another Transformers movie, right?
No Priors Ep. 32 | With NEAR’s Illia Polosukhin
- ▶ 0:15 Sarah Guo This week, Ilad and I are joined by Ilya Palosukhin, the co-founder of NIR, and co-author of the landmark Transformers paper, to discuss the interaction of blockchain and AI technologies, what we should expect from AI agents, how to handle…
- ▶ 0:37 Sarah Guo You are one of the authors of the original Transformers paper.
- ▶ 1:53 Illia Polosukhin You know, we have transformers coming out.
Nvidia Part III: The Dawn of the AI Era (2022-2023) (Audio) · Acquired
- ▶ 45:14 Ben Gilbert Google Brain's Transformer had blown open a new frontier where AI could improve endlessly.
Your Kids Signed Contracts With No Expiration Date and No Way Out
E141: State of Series A's, VC dry powder, IPO window opens + more with Bill Gurley & Brad Gerstner
- ▶ 21:51 Jason Calacanis It's not the Transformers or a Marvel film, but it's well worth it if you can get your ADHD under control.
20,000 AI Lip Sync Vids Created, Now Raising $2m on $10m Valuation
- ▶ 4:51 Matthias Spitzer Uh, he has the lead of effects of, uh, Guardians of the Galaxy or Transformers.
E139: Recapping Chamath's wedding, VC surplus, unions vs Hollywood, room-temp superconductors & more
- ▶ 40:53 Jason Calacanis For Transformers?
No Priors Ep. 5 | With Huggingface’s Clem Delangue
Build and Deploy AI with Pytorch | Lightning AI Founder & CEO, William Falcon
- ▶ 13:25 William Falcon And in the, in what we did, and this is 2016, 2017, before Transformers, um, we gave you suggestions.
- ▶ 17:37 William Falcon But, um, you know, like none of this wouldn't exist if Google hadn't put published transformers and put the paper out there, right?
No Priors Ep. 12 | With Noam Shazeer
No Priors Ep. 13 | With Jensen Huang, Founder & CEO of NVIDIA
- ▶ 16:05 Jensen Huang And of course, the, the Transformer was a big deal. 4 times in the scene
How 2 Personal Losses Led To Building The #1 Faith-Based Media Company
- ▶ 29:14 Steve Gatina So like Mission Impossible, Transformers, CSI, Law and Order, all the major motion pictures in TV shows.
Spin Master/PAW Patrol: Ronnen Harary
- ▶ 43:33 Ronnen Harary It was like marbles meets transformers.
- ▶ 54:47 Ronnen Harary And we said, what if we took the magical aspects of transformation that kids loved in Bakugan, and they love in Transformers, and why don't we bring it to a preschool audience?
- ▶ 59:00 Ronnen Harary You look at Transformers, they did films, and they were really innovative with their films.
E57: Understanding Omicron, tech stocks plummet, VC's great resignation, Jack Dorsey's departure
- ▶ 12:59 unnamed speaker It's not a transformer. 2 times in the scene
a16z Podcast | Tech's Biggest Ideas and How They Take Hold -- With Marc Andreessen and Dan Siroker
- ▶ 10:56 Marc Andreessen And, uh, it's sort of this very dark kind of, you know, science fiction movies, you know, it's Terminator, and it's, you know, it's, you know, Terminator, I guess, you know, Terminator, Transformers, depending on how old you are.
Melissa & Doug: Melissa And Doug Bernstein (2016)
Melissa & Doug: Melissa And Doug Bernstein
Jake Porway, DataKind // Data For The Greater Good (FirstMark / Data Driven NYC)
- ▶ 2:25 Jake Porway But if we're truly in this age of reason, doesn't it seem like we could do more with that data than just make sure that never again does Netflix recommend Transformers to you when you really wanted to see Pacific Rim?