Mixtral 8x7B
part of Mixtral includes Mixtral 8x7B Instruct
1 statements across 1 episodes · 0 bullish · 1 bearish · 1 people on the record · first statement Oct 29, 2024 by Ethan He · said 28 times in 8 episodes since 2024 · across every show →
Mentions by year, the whole family
brought up most by Ethan He (8), Nathan Lambert (3), George Cameron (1)
tap a year for its mentions
2026 5 mentions in 2 episodes 3 per episode
2025 1 mention in 1 episode
2024 22 mentions in 5 episodes 4 per episode
- The Four Wars of the AI Stack - Dec 2023 Recap
- [Paper Club] Upcycling Large Language Models into Mixture of Experts
- The Origin and Future of RLHF: the secret ingredient for ChatGPT - with Nathan Lambert
- Best of 2024: Open Models [LS LIVE! at NeurIPS 2024]
- The Winds of AI Winter (Q2 Four Wars of the AI Stack Recap)
- every mention in 2024, scene by scene →
Everything said about Mixtral 8x7B, oldest first
Oct 29, 2024 negative
Ethan He: Hugging Face's sequential GEMM loop for Mixtral is inefficient
“Let's also look at the implementation of Mixtro eight by seven on Hagen-Phys transformer. You will soon notice the, in the expert operation there, You would iterate over all of the experts and compute each of the gem operations one by one. We found that this i…”