SGLang
also referred to as: sglangs
10 statements across 5 episodes · 5 bullish · 1 bearish · 5 people on the record · first statement Nov 1, 2024 by Anastasios Angelopoulos · said 67 times in 11 episodes since 2024 · across every show →
Mentions by year
brought up most by Yining Zhang (6), Shawn Wang (4), Philip Kiely (4), Kyle Kranen (3), Mark Bissell (2), Chris Lattner (2), Ali Taha (2), Stefano Ermon (1)
tap a year for its mentions
2026 13 mentions in 4 episodes 3 per episode
- Next 100x in AI: Inference, Networking, & Self-Optimizing Models — Philip Kiely & Ali Taha, Baseten
- Agent Inference at the "Speed of Light" — How NVIDIA moves like a $4.3 Trillion Startup
- The Future of AI Infra: from Kubernetes to Agent Sandboxes — Akshat Bubna, Modal CTO
- Goodfire AI’s Bet: Interpretability as the Next Frontier of Model Design — Myra Deng & Mark Bissell
- every mention in 2026, scene by scene →
2025 52 mentions in 5 episodes 10 per episode
- DeepSeek V3, SGLang, and the state of Open Model Inference in 2025 (Quantization, MoEs, Pricing)
- The Shape of Compute (Chris Lattner of Modular)
- [State of Research Funding] Beyond NSF, Slingshots, Open Frontiers — Andy Konwinski, Laude Institute
- ⚡️Mercury: Ultra-Fast Diffusion LLMs — Estefano Ermon, CEO Inception Labs
- Information Theory for Language Models: Jack Morris
- every mention in 2025, scene by scene →
2024 2 mentions in 2 episodes 1 per episode
Everything said about SGLang, oldest first
Nov 1, 2024 neutral
Angelopoulos: Chatbot Arena is decoupling from LMSYS as co-creators shift focus
“Sort of Chatbot Arena has, of course, like, kind of become its own thing, and Lianmin and Ying, who are, you know, created LMSYS, have kind of, like, moved on to working on SGLang, and now They're doing other projects that are sort of originating from LMSS. An…”
Jan 19, 2025 positive
Jan 19, 2025 positive
Jan 19, 2025 positive
Jan 19, 2025 negative
Jan 19, 2025 positive
Zhang: SGLang achieves higher cache hit rates using block size of one
“Redix cache, I think it's the technology of the prefix caching, and it is a special case for something like block size is one, you know, for VLM or for other frameworks, they use something block size, 32, and SGLAN use the block size one. I think if you use th…”
Jan 19, 2025 neutral
Jul 2, 2025 bullish
Aug 4, 2025 neutral
Ermon: Inception Labs built proprietary engine for production inference traffic
“So just like you would normally serve an LLM using a VLLM or SGLang or a Tensor or TLLM, we have built our own inference engine. And so we are supporting production traffic already with our own inference engine. We support continuous batch and quantization.”
Aug 3, 2026
Open-source inference engines often receive model weights before official launches
“Getting to the point of I can make a token out of this model is not that hard because generally the open source inference engines, your VLMs, SGLangs of the world oftentimes even receive weights ahead of time maintainers do, or the people making the model merg…”