Goel: Cartesia intends to train an on-device multimodal model
Karan Goel · No Priors Ep. 70 | With Cartesia Co-Founders Karan Goel & Albert Gu · Jun 27, 2024 · at 27:54
Cartesia co-founder Karan Goel responds to Elad Gil asking whether Cartesia plans to train its own multimodal foundation model.
0:00 / 0:26exact quote · 26.8s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“Yes. But, you know, we have our own sort of set of techniques that we're developing in order to be able to do that effectively. I think that I will maybe leave for another podcast. But I think, yeah, I think that is the intention at the end of the day is build a great multimodal model, but then make it really, really easy to run on device and make it really cheap to run and really focus on kind of the audio piece and making that as good as possible”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Karan Goel
Goel: AI models generating real-time data will replace traditional graphics engines
“In the future, like you will be essentially replacing, you know, graphics and rendering with essentially models that are outputting streams of data. In real time.”
Goel: 3B parameter models are still too small and lack strong capabilities
“So the three V models are interesting, I think, but they aren't like, they're still small and not very capable.”
Goel: Current AI speech models fail to capture profession-specific vocal nuances
“And that's sort of the nuance that I don't think any models really capture well, which is like, you know, if you're a nurse, you need to talk in a different way than if you're a lawyer, or if you're a judge, or if you're a venture capitalist, you know, very di…”
Goel: Cartesia's Sonic TTS engine reduces typical voice latency by 150ms
“Even with what we've done with Sonic, we're already kind of shaving off, like, a 150 milliseconds off of you know, what they typically use.”
Goel: Cartesia aims to cut another 600ms of voice latency this year
“And so, you know, the roadmap is let's, let's get to the next 600 milliseconds and try to shape those off in the, over the course of the year.”
Goel: Cartesia aims to build edge-oriented infrastructure for SSMs
“I think in general for what we're trying to build is, is sort of the infrastructure to be able to train these models, make them run fast, and then bring them closer and closer to kind of be, you know, very edge oriented rather than cloud oriented.”