Shafqat: Deep Dive audio pairs text signals with intonation modeling
Usama Shafqat · How NotebookLM Was Made · Oct 25, 2024 · at 26:28
Google Labs AI engineer Usama Shafqat explains how NotebookLM generates conversational speech features like laughter and pauses.
0:00 / 0:16exact quote · 16.4s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“The audio model is definitely trying to mimic like certain human intonations and like sort of natural, like, you know, breathing and pauses and like laughter and things like that. But yeah, in generating like the text, we also have to sort of give signals on like where those things maybe would make sense.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Usama Shafqat
Shafqat: Longer thinking time consistently yields better AI results
“More thinking time equals just better results consistently. And that holds true for probably every single time that I've tried to build something.”
Shafqat: NotebookLM Audio Combines DeepMind Voice Tech With Gemini 1.5
“Like we work with the DeepMind audio folks pretty closely. So they're always cooking up new techniques to like get better, more human-like audio. And then Gemini, 1.5 is really, really good at absorbing long context. So we sort of like generally put those thin…”
Shafqat: Engaging audio dialogues require tension and gradual information reveal
“I think one thing we find often is if there's just too much agreement between people, like that's not Fun to listen to. So there needs to be some sort of tension and build up, you know, withholding information, for example, like as you listen to a story unfold…”
Shafqat: NotebookLM does not directly prompt AI hosts for humor
“Humor is contextual also, like super contextual is what we're realizing. So we're not prompting for humor, but we're prompting for maybe a lot of other things that are bringing out that humor.”