Bubna: Batch compute demand comes mainly from non-LLM workloads like computational biology
Akshat Bubna · The Future of AI Infra: from Kubernetes to Agent Sandboxes — Akshat Bubna, Modal CTO · Jul 8, 2026 · at 41:15
Modal CTO Akshat Bubna discusses customer usage patterns for latency-insensitive, off-peak compute tiers on Modal's cloud platform.
0:00 / 0:20exact quote · 20.9s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“The demand that we see for something like that is actually not for LLMs. Although sometimes people want to run evals and do synthetic data prep and there it makes sense. But it's from a lot of non LLM companies like people who are doing computational bio, like they haven't run really big bad jobs and they don't care about when they get it back.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Akshat Bubna
Bubna: Speculative decoding accept length delivers multiplicative speedups over kernel tuning
“People talk a lot about, we made these kernels faster and whatnot, but improving kernel only give you like a few percentage points of improvement and increasing except length literally is a multiplicative decrease.”
Bubna: Kubernetes lacks burstiness support and has terrible developer experience
“Kubernetes is hard to manage. It's not built for burstiness and custom images and has a terrible developer experience.”
Bubna: System Observability Is Becoming More Important Than Reading Code
“You still need humans to go interpret what's going on and you know, make judgment calls and whatnot. And that's, I feel like maybe more important now than looking at the code itself.”
Bubna: Open-source DFlash matches proprietary speculative decoding performance
“Recently we shared our work on dflash, which is a block-based speculator, and we've open sourced all of it, so you can get, by using open source dflash, you can get the same performance as you would with one of the proprietary providers.”