Firshman: Dockerfiles are too complex for ML researchers to manage
Ben Firshman · A Brief History of the Open Source AI Hacker - with Ben Firshman of Replicate · Feb 28, 2024 · at 50:38
Ben Firshman, CEO of Replicate, explains why standard Dockerfiles failed to serve machine learning developers, motivating the design of Cog.
0:00 / 0:21exact quote · 21.7s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“Dockerfiles are hard enough for software developers to write. I'm saying this with love as somebody who works on Docker and, like, works on Dockerfiles but it's really hard to use, and you need to know a bunch about Linux, basically, because you're running a bunch of CLI commands. You need to know a bunch, bunch of Linux and best practices and, like, how apt works and all this kind of stuff. So we're like, okay, we can't dip to that level. We need to, we need something that machine learning researchers will be able to understand”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Ben Firshman
Firshman: Academic ML researchers make poor software users due to six-month cycles
“And they were like, what was funny about that is they were like, not very good users. Like, they were doing great work, obviously, but the way that research worked is that they just made, like, one thing every six months, and they just fired and forgot it. Lik…”
Firshman: Early 2021 Discord AI bots originated Midjourney's collaborative interface
“It was the start of, it was the start of mid-journey, and, you know, it's where that kind of user interface came from. Like, what's beautiful about the user interface is, like, You could see what other people are doing, and that you could riff off other people…”
Firshman: GPU demand is not currently outpacing supply at Replicate
“From our point of view, demand is not outpacing supply of GPUs. Like we have enough, from our point of view, we have enough GPUs to go around, but that might change for sure.”
Firshman: Llama 2 costs $25M to train but $50 to fine-tune
“Lama II as a base model is that, like, yeah, it costs twenty five million dollars to train to start with, but then you can fine tune it for, like, 50 bucks.”
Firshman: Building Fig and Docker Compose in Python was a mistake
“We used Python, which was a big mistake, where Python's really hard to get booting up fast, because you have to load up the whole Python runtime before it can run anything.”
Firshman: CLIs are a more natural fit for LLMs than GUIs
“It's almost more natural to a CLI than it is in a graphical user interface, because it feels like there's back and forth with the computer. Yeah. Almost funnily like a language model. So I think there's some interesting intersection of like CLIs and language m…”