BERT
4 statements across 4 episodes · 3 bullish · 0 bearish · 4 people on the record · first statement Jan 1, 2025 by Shawn Wang · said 110 times in 27 episodes since 2023 · across every show →
Mentions by year
brought up most by Jack Morris (7), Ankur Goyal (7), Jeremy Howard (6), Varun Mohan (4), Swix (Shawn) (4), Yi Tay (3), Shawn Wang (3), William Beauchamp (2)
tap a year for its mentions
2026 7 mentions in 4 episodes 2 per episode
- 🔬 Training Transformers to solve 95% failure rate of Cancer Trials — Ron Alfa & Daniel Bear, Noetik
- 🔬Causal Models Need Causal Data - Xaira’s X-Cell model (Bo Wang & Ci Chu)
- The AI Frontier: from Gemini 3 Deep Think distilling to Flash — Jeff Dean
- Captaining IMO Gold, Deep Think, On-Policy RL, Feeling the AGI in Singapore — Yi Tay
- every mention in 2026, scene by scene →
2025 20 mentions in 10 episodes 2 per episode
- Information Theory for Language Models: Jack Morris
- 🕰️ The Oral History of Windsurf (ft. Varun Mohan, Scott Wu, Jeff Wang, Kevin Hou, Anshul R)
- Outlasting Noam Shazeer, Crowdsourcing Chai AI w/ 1.4m DAU — with William Beauchamp, Chai Research
- SAM 3: The Eyes for AI — Nikhila & Pengchuan (Meta Superintelligence), ft. Joseph Nelson (Roboflow)
- The Future of Email: Superhuman CTO on Your Inbox As the Real AI Agent (Not ChatGPT) — Loïc Houssier
- ⚡ Inside Google Labs: Building The Gemini Coding Agent — Jed Borovik, Jules + AIE CODE Preview
- Building Jamba 3B: the tiny Hybrid Transformer State Space Reasoning Model - Barak Lenz, CTO of AI21
- Personalized AI Language Education — with Andrew Hsu, Speak
- 2 more episodes that year, every mention in 2025 →
2024 81 mentions in 12 episodes 7 per episode
- [Paper Club] BERT: Bidirectional Encoder Representations from Transformers
- Answer.ai & AI Magic with Jeremy Howard
- Production AI Engineering starts with Evals
- [Paper Club] Embeddings in 2024: OpenAI, Nomic Embed, Jina Embed, cde-small-v1 - with swyx
- Best of 2024: Synthetic Data / Smol Models, Loubna Ben Allal, HuggingFace [LS Live! @ NeurIPS 2024]
- The 10,000x Yolo Researcher Metagame — with Yi Tay of Reka
- [Paper Club] Upcycling Large Language Models into Mixture of Experts
- Is finetuning GPT4o worth it?
- 4 more episodes that year, every mention in 2024 →
2023 2 mentions in 1 episode
Everything said about BERT, oldest first
Jan 1, 2025 positive
Jul 2, 2025 positive
Morris: CycleGAN mapping aligns disparate model embeddings without paired data
“We took it and we applied it to model embeddings where instead of zebras and horses, we have like BERT embeddings and GPT embeddings, or like two completely different models with different architectures. So I think these are GTR, which is a T five based retrie…”
Jul 28, 2025 positive
Mohan: Enterprises should fine-tune off-the-shelf models over custom architectures
“For a vast majority of enterprises, they should probably be using something off the shelf, fine tuning BERT models. If it's a vision, they should be fine tuning resonant or using something like clip, like the less work they can do the better.”