On Device AI
topic on 2 shows · 3 statements across 3 episodes
3 statements about On Device AI, every show
Sanseviero: Embedding offloading suits edge devices; larger models require MoEs or dense architectures
“This is really optimized and designed for, like, on-device. And when I say on-device, I mean, like, running in a phone, Android, Raspberry Pi, and so on, right? When you go larger, you usually want to come back more You want to have more, like, dense architect…”
Ravi: SAM 2 will soon run on-device and inside web browsers
“Like, I'm pretty sure soon we'll see like an on-device SAM-II or, you know, maybe even running in the browser or something. So I think that could definitely unlock some of these edge use cases.”
Appenzeller: AI compute will bifurcate between local and cloud tasks
“There's probably going to be a little bit bifurcation there on quality and parameters, right? You can run things locally but you can probably run them a lot better in the cloud, right? Because you have a much bigger server there. So it probably depends on what…”