Mixtral
includes Mixtral 8x7B, Mixtral 8x22B
2 statements across 2 episodes · 0 bullish · 1 bearish · 1 people on the record · first statement Oct 29, 2024 by Ethan He · said 46 times in 15 episodes since 2024 · across every show →
Mentions by year, the whole family
brought up most by Ethan He (9), Nathan Lambert (3), Yi Tay (1), Vipul Ved Prakash (1), Shawn Wang (1), Pranav Reddy (1), George Cameron (1)
tap a year for its mentions
2026 7 mentions in 3 episodes 2 per episode
- Mistral: Voxtral TTS, Forge, Leanstral, & Mistral 4 — w/ Pavan Kumar Reddy & Guillaume Lample
- Inside xAI: Building Grok Imagine in 3 Months, Videogen vs World Models, and Video Agents— Ethan He
- Artificial Analysis: The Independent LLM Analysis House — with George Cameron and Micah Hill-Smith
- every mention in 2026, scene by scene →
2025 6 mentions in 3 episodes 2 per episode
2024 33 mentions in 9 episodes 4 per episode
- The Four Wars of the AI Stack - Dec 2023 Recap
- [Paper Club] Upcycling Large Language Models into Mixture of Experts
- A Brief History of the Open Source AI Hacker - with Ben Firshman of Replicate
- The Origin and Future of RLHF: the secret ingredient for ChatGPT - with Nathan Lambert
- Best of 2024: Open Models [LS LIVE! at NeurIPS 2024]
- The Winds of AI Winter (Q2 Four Wars of the AI Stack Recap)
- The 10,000x Yolo Researcher Metagame — with Yi Tay of Reka
- The State of AI Startups in 2024 [LS Live @ NeurIPS]
- 1 more episode that year, every mention in 2024 →
Everything said about Mixtral, oldest first
Oct 29, 2024 negative
He: Mixtral's top-k before softmax routing hurts MoE upcycling performance
“We actually found the mix-throughs approach didn't work as well as expected, because the original model, the original switch transformer from Google uses a softmax and topk for a reason. And because of upcycling, if you switch to topk, then softmax, it actuall…”