Bubna: Transferring RL weights is fundamentally an OS memory problem
Akshat Bubna · The Future of AI Infra: from Kubernetes to Agent Sandboxes — Akshat Bubna, Modal CTO · Jul 8, 2026 · at 31:40
Modal CTO Akshat Bubna discusses why operating systems researchers and principles are critical for optimizing AI inference and reinforcement learning pipelines.
0:00 / 0:17exact quote · 17.4s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“Like the way you move around your KV cache and how efficiently you can do it, how efficiently you move your weights from your training GPUs to your inference GPUs in RL is, there's a lot of degrees of freedom, and it is basically a systems problem of Moving memory around. Scheduling.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Akshat Bubna
Bubna: Speculative decoding accept length delivers multiplicative speedups over kernel tuning
“People talk a lot about, we made these kernels faster and whatnot, but improving kernel only give you like a few percentage points of improvement and increasing except length literally is a multiplicative decrease.”
Bubna: Kubernetes lacks burstiness support and has terrible developer experience
“Kubernetes is hard to manage. It's not built for burstiness and custom images and has a terrible developer experience.”
Bubna: System Observability Is Becoming More Important Than Reading Code
“You still need humans to go interpret what's going on and you know, make judgment calls and whatnot. And that's, I feel like maybe more important now than looking at the code itself.”
Bubna: Open-source DFlash matches proprietary speculative decoding performance
“Recently we shared our work on dflash, which is a block-based speculator, and we've open sourced all of it, so you can get, by using open source dflash, you can get the same performance as you would with one of the proprietary providers.”