Firshman: GPU demand is not currently outpacing supply at Replicate
Ben Firshman · A Brief History of the Open Source AI Hacker - with Ben Firshman of Replicate · Feb 28, 2024 · at 1:05:09
Ben Firshman, CEO of Replicate, addresses whether industry GPU hardware crunches are constraining inference workloads at Replicate.
0:00 / 0:07exact quote · 7.7s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“From our point of view, demand is not outpacing supply of GPUs. Like we have enough, from our point of view, we have enough GPUs to go around, but that might change for sure.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Ben Firshman
Firshman: Academic ML researchers make poor software users due to six-month cycles
“And they were like, what was funny about that is they were like, not very good users. Like, they were doing great work, obviously, but the way that research worked is that they just made, like, one thing every six months, and they just fired and forgot it. Lik…”
Firshman: Early 2021 Discord AI bots originated Midjourney's collaborative interface
“It was the start of, it was the start of mid-journey, and, you know, it's where that kind of user interface came from. Like, what's beautiful about the user interface is, like, You could see what other people are doing, and that you could riff off other people…”
Firshman: Llama 2 costs $25M to train but $50 to fine-tune
“Lama II as a base model is that, like, yeah, it costs twenty five million dollars to train to start with, but then you can fine tune it for, like, 50 bucks.”
Firshman: Building Fig and Docker Compose in Python was a mistake
“We used Python, which was a big mistake, where Python's really hard to get booting up fast, because you have to load up the whole Python runtime before it can run anything.”
Firshman: CLIs are a more natural fit for LLMs than GUIs
“It's almost more natural to a CLI than it is in a graphical user interface, because it feels like there's back and forth with the computer. Yeah. Almost funnily like a language model. So I think there's some interesting intersection of like CLIs and language m…”