Llama 3
also referred to as: llama-3
part of Llama includes Llama 3 405B, Llama 3 70B, Llama 3 8B, Llama 3 Tokenizer
12 statements across 5 episodes · 9 bullish · 0 bearish · 5 people on the record · first statement May 31, 2024 by Mark Huang · said 118 times in 32 episodes since 2023 · across every show →
Mentions by year, the whole family
brought up most by Shawn Wang (24), Alessio Fanelli (15), Thomas Scialom (8), Soumith Chintala (4), Philip Kiely (3), Mark Huang (3), Loubna Ben Allal (3), Yi Tay (2)
tap a year for its mentions
2026 7 mentions in 4 episodes 2 per episode
- Next 100x in AI: Inference, Networking, & Self-Optimizing Models — Philip Kiely & Ali Taha, Baseten
- 🔬 "The Most Innovative Diffusion Research Is Happening in Drug Discovery, Not Image Generation"
- The Agent Cloud: Databricks’ Bet on the Future of AI — Matei Zaharia and Reynold Xin
- Agent Inference at the "Speed of Light" — How NVIDIA moves like a $4.3 Trillion Startup
- every mention in 2026, scene by scene →
2025 9 mentions in 8 episodes 1 per episode
- 2024 Year in Review: The Big Scaling Debate, the Four Wars of AI, Top Themes and the Rise of Agents
- SAM 3: The Eyes for AI — Nikhila & Pengchuan (Meta Superintelligence), ft. Joseph Nelson (Roboflow)
- How Zyphra went all-in on AMD + Why Devs feel faster with AI but are slower — with Quentin Anthony
- ⚡️Math Olympiad gold medalist explains OpenAI and Google DeepMind IMO Gold Performances
- ⚡️Ranking Agentic LLMs — Pratik Bhavsar, Galileo
- The Shape of Compute (Chris Lattner of Modular)
- The Agent Network — Dharmesh Shah, Agent.ai + CTO of HubSpot
- The Unreasonable Effectiveness of Reasoning Distillation: using DeepSeek R1 to beat OpenAI o1
- every mention in 2025, scene by scene →
2024 101 mentions in 19 episodes 5 per episode
- Training Llama 2, 3 & 4: The Path to Open Source AGI — with Thomas Scialom of Meta AI
- The Winds of AI Winter (Q2 Four Wars of the AI Stack Recap)
- [LLM Paper Club] Llama 3.1 Paper: The Llama Family of Models
- How to train a Million Context LLM — with Mark Huang of Gradient.ai
- The 10,000x Yolo Researcher Metagame — with Yi Tay of Reka
- LLM Asia Paper Club Survey Round
- Open Source AI is AI we can Trust — with Soumith Chintala of Meta AI
- Best of 2024: Synthetic Data / Smol Models, Loubna Ben Allal, HuggingFace [LS Live! @ NeurIPS 2024]
- 11 more episodes that year, every mention in 2024 →
2023 1 mention in 1 episode
Everything said about Llama 3, oldest first
May 31, 2024 positive
Jul 23, 2024 bullish
Scialom: Llama 3 Scaled Pre-Training to 15 Trillion Tokens
“It's the same recipe done in terms of architectures and training than LAMA-II, but we put so much effort on scaling the data and the quality of data. There's now 15 triant tokens compared to two triants, so it's another magnitude there as well, including for t…”
Jul 23, 2024
Meta changed Llama 3's pre-training data mixture mid-training run
“What happened is we changed the data mix during the training of Lama three with some findings that happened in the... Training is long, so you have to do something while it's training. And what the team did, I was working on my side of motion post-training, bu…”
Jul 23, 2024 positive
Scialom: Llama 3 405B is the best open-source model ever released
“At a high level, it's the best open source model ever. It's Better than GPT-IV. I mean, what version? But, by far, compared to the version originally released even now, I think there's maybe the last cloud Sonya FF-V and GPT-IV-Zero that are performing it.”
Jul 23, 2024 positive
Meta's Llama 3 post-training uses almost entirely synthetic data
“So what we did is that we generated all the data on the prompts with LAMA-II, and we applied, like, basically the last round of LAMA-II we had to kick off and start LAMA-III post-training. So Lama-free post-training doesn't have any, like, human-written answer…”
Jul 23, 2024 positive
Scialom: Meta Used Llama 2 to Filter and Tag Llama 3 Pre-Training Data
“LAMA was the best, at the time, before LAMA Free, the best model we had access to legally, to labelize the web and select what are the good tokens and the bad tokens. The additional thing is that it also enabled to have a topic tag, Like, is it about law? Is i…”
Jul 23, 2024 positive
Scialom: Llama 3 natively supports state-of-the-art tool calling
“We have that from day one. Good news for the community. We are state of the art there. I think the model will be pretty good at that, we have a lot of gems about tools in the paper, but the model is fine-tuned to do tool usage, to zero-shot function calling”
Jul 23, 2024
Jul 23, 2024 positive
Scialom: Meta expanded Llama 3's vocabulary size to support multilingual capabilities
“Lama III compared to Lama II is multilingual, has multilingual capabilities. We worked on that. And so, because you have languages that are not just Latin languages like English, there's a lot of different characters you want to include them to represent, like…”
Aug 22, 2024 positive
Wang: Natural language-to-code translation is ripe for synthetic data generation
“I think that translation between natural language, English versus code and back and forth, I think is actually actually a really ripe source of synthetic data and Lama three specifically called out that, that they trained on that.”
Dec 23, 2024 bullish