Shafqat: NotebookLM Audio Combines DeepMind Voice Tech With Gemini 1.5
Usama Shafqat · How NotebookLM Was Made · Oct 25, 2024 · at 14:28
Usama Shafqat, AI engineer on NotebookLM, describes the underlying model architecture powering the product's podcast-style audio summaries.
0:00 / 0:23exact quote · 23.0s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“Like we work with the DeepMind audio folks pretty closely. So they're always cooking up new techniques to like get better, more human-like audio. And then Gemini, 1.5 is really, really good at absorbing long context. So we sort of like generally put those things together in a way that we could reliably produce the audio.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Usama Shafqat
Shafqat: Longer thinking time consistently yields better AI results
“More thinking time equals just better results consistently. And that holds true for probably every single time that I've tried to build something.”
Shafqat: Engaging audio dialogues require tension and gradual information reveal
“I think one thing we find often is if there's just too much agreement between people, like that's not Fun to listen to. So there needs to be some sort of tension and build up, you know, withholding information, for example, like as you listen to a story unfold…”
Shafqat: NotebookLM does not directly prompt AI hosts for humor
“Humor is contextual also, like super contextual is what we're realizing. So we're not prompting for humor, but we're prompting for maybe a lot of other things that are bringing out that humor.”
Shafqat: NotebookLM will theoretically cover most languages soon
“So I guess high level, like we're definitely working on adding more languages. That's like top priority. We're going to start small, but like theoretically we should be able to cover like most languages pretty soon.”
Shafqat: Adding user parameters to AI products requires redoing quality evaluations
“The knobs are not as easy to add as simply like, I'm going to add a parameter to this and it's going to make it happen. It's like, you kind of have to redo the quality process for everything.”
Shafqat: NotebookLM Audio Feature Began as Independent Content Transformation Demo
“We didn't actually start out thinking this would live in notebook, right? Like notebook was sort of, we, Built this demo out independently, tried out like a few different sort of sources that the main idea was like go from some sort of sources and transform it…”