Jul 31, 2026 · 46m · sourcery
AssemblyAI Now Handles 4x YouTube's Daily Volume
0:00 / 0:00
▶ Watch on YouTube →
gold bands on the timeline = statements, start to end. Hover to read, click to jump. CC turns on captions
Dylan Fox, founder and CEO of AssemblyAI, joins host Molly O'Shea on Sourcery to discuss how his company scaled to process four times YouTube's daily audio volume. Fox details breakthroughs in context-aware speech models, AssemblyAI's pure-play developer infrastructure approach, and the future transition toward ambient voice interfaces across hardware and robotics.
How this conversation actually went
Every chapter scored 0–10 on four independent dynamics. Hover any point for the reasoning behind the score. Molly holds 18.2% of the talking time here. How this is scored →
Molly as informed peer 3.2 Guest teaching 2.9 Guest disagreement 0.5 Molly pushing back 0.3
speaking balance: gold is Molly, purple is the guest (3 minute bins)
Sharpest disagreement ▶ 41:28 Dylan rejects assuming everything is AI
When Molly suggests users should just assume all calls are automated AI, Dylan directly counters with a medical nurse helpline scenario to demonstrate why deceptive anthropomorphic agents generate significant distrust.
Hardest push from Molly ▶ 42:46 Molly questions the novelty of contextual voice modelsMolly challenges Dylan's presentation of environmental context as a novel feature, questioning how existing commercial drive-through voice systems could possibly lack basic environmental awareness.
Biggest teaching moment ▶ 33:20 Dylan exposes deceptive open-source speech benchmarksDylan educates Molly on model evaluation, explaining that public benchmarks are easily optimized for marketing vanity, whereas enterprise utility requires fine-grained filtering like distinguishing between ordering customers and backseat screaming.
Molly holds their own ▶ 1:09 Molly demonstrates operational voice workflow expertiseMolly illustrates her practical domain fluency right at the outset, detailing how she captures audio data to synthesize automated investor calls and open-source memos.
the scores for every segment, with the reasoning behind each
| Chapter | Topic | Molly as informed peer | Guest teaching | Guest disagreement | Molly pushing back | Why |
|---|---|---|---|---|---|---|
| AssemblyAI's Unprecedented Scale and Rapid Volume Inflection | 4 | 2 | 0 | 0 | Molly opens by explaining her own use case of voice transcription for VC investment memos before prompting Dylan to quantify AssemblyAI's scale. Dylan explains the platform's massive growth metrics, including handling over four times YouTube's daily volume, while Molly reacts receptively. | |
| Y Combinator Origins and the Autonomous Vehicle Parallel | 3 | 3 | 0 | 0 | Molly asks whether Dylan anticipated current voice adoption during AssemblyAI's early YC days. Dylan explains the YC AI batch origins and articulates his autonomous vehicle thesis, noting that market adoption unlocks progressively as core model thresholds are met. | |
| Three Macro Drivers Accelerating Voice AI Adoption | 2 | 4 | 0 | 0 | Molly asks an open prompt about macro trends driving voice inflection. Dylan systematically educates the audience on three core drivers: model capability, adjacent AI tooling, and coding agents expanding the developer TAM. | |
| Pure-Play Voice Infrastructure and Scaling Engineering | 4 | 3 | 0 | 0 | Molly inquires about competitive positioning against full-stack voice providers like ElevenLabs and Sierra. Dylan clarifies AssemblyAI's dedicated pure-play infrastructure focus across inference and orchestration layers rather than consumer application interfaces. | |
| Sponsor Spotlight: Brex Agentic Finance Platform | 3 | 1 | 0 | 0 | Following an ad read for Brex, Molly asks how Dylan internally restructured AssemblyAI with AI tooling amid the SaaS shift. Dylan describes internal workflow automations, including personal executive assistants and autonomous web deployment. | |
| Sovereign AI and Enterprise Privacy Deployments | 4 | 3 | 2 | 1 | Molly introduces an investor perspective suggesting mice and keyboards will soon be entirely obsolete. Dylan gently counters this maximalist view, pointing out that touchscreens did not replace keyboards and predicting voice will serve as an additive dimension rather than a wholesale replacement. | |
| Humanoid Robotics and Acoustic Disambiguation Challenges | 4 | 4 | 1 | 0 | Molly inquires about humanoid robotics applications and suggests models distinguish speech via listening data rather than visual feeds. Dylan elaborates that even with visual sensors, acoustic disambiguation of overlapping speakers remains a core unsolved technical bottleneck. | |
| Navigating Cultural Nuance in Multilingual Voice AI | 3 | 4 | 0 | 0 | Molly asks Dylan about the complexities of multilingual translation and whether AssemblyAI would acquire local specialized vendors. Dylan explains that localized performance depends heavily on cultural policy alignment and native nuance rather than pure underlying architecture. | |
| Sponsor Spotlight: MongoDB for AI Applications | 2 | 2 | 1 | 0 | After ad reads for MongoDB and AssemblyAI, Molly playfully asks whether technology will enable animal translation, citing sci-fi. Dylan humorously debunks the voice premise, noting mind-reading interfaces or sub-vocal scans would be needed over acoustic translation. | |
| Training Regimes, Data Alignment, and Real-World Evals | 3 | 4 | 1 | 0 | Molly asks how models are trained across millions of audio hours. Dylan details the reality of speech engineering, explaining that public benchmarks are easily gamed and real performance requires extensive custom evaluation suites and domain-specific background noise handling. | |
| AssemblyAI Team Structure, Founder Roots, and Infrastructure DNA | 3 | 1 | 0 | 0 | Molly prompts Dylan to explore AssemblyAI's internal team breakdown and personal technical roots. Dylan recounts learning to code in college, participating in IRC communities, and defining the company's core identity around developer infrastructure. | |
| Future Outlook: On-Device Hardware and the Voice Agent UX Dilemma | 4 | 4 | 2 | 2 | Dylan outlines the UX dilemma of voice agents attempting to deceptively mimic humans, while Molly counters that users will simply assume everything is AI. Dylan pushes back with a healthcare counterexample, illustrating why deceptive AI interaction induces user discomfort in sensitive settings. | |
| Final Technical Reflections: Context-Aware Voice Models | 3 | 3 | 0 | 1 | Molly asks if any topics were missed, and Dylan highlights new context-aware models. Molly expresses surprise that voice systems have lacked environmental context until now, and Dylan confirms that AssemblyAI only recently engineered this breakthrough. |
Statements from this episode (19)
Fox: AssemblyAI weekly conversation volume grew 800 percent over three years
“One stat I was just looking at before I came over here was like the amount of weekly conversations that Assembly handles through our APIs every week is up over 800% over the last three years.”
Fox: AssemblyAI processes four times YouTube's daily audio volume on peak weeks
“And so now, you know, on a given week, on a peak week, there'll be something like over a hundred and twenty million conversations, voice conversations going through our platform, over two million hours of voice, which as of December of this past year was four,…”
Fox: AssemblyAI handles nearly 100 million daily API calls from developers
“There's, you know, almost a hundred million API calls a day coming against our API, about a million developers, a little over a million developers on the platform now.”
Fox: Voice AI Follows Self-Driving Cars in Tech-Threshold Adoption
“I always viewed this journey as similar to self-driving cars, where You know, it wasn't a question of like, oh, is the product market fit going to come? It was like, no, the technology just sucks. And as it gets better and better, you're going to continuously …”
Fox: Lawn care chains use Lovable and Cursor to build on AssemblyAI
“We saw this, like, small business, like, lawn care chain sign up, and, you know, we're like, what do they do with the API? And a lot of these small businesses are automating parts of their back office using Lovable, or Cloud Code, or Cursor, or whatever, Repli…”
Fox: Coding Agents Have Expanded AssemblyAI's TAM by 100x
“So I think about this as like our TAM has just increased by a hundred X because we're not just selling to engineering teams within product companies. It's now like any, anyone.”
Fox: Most AI note-takers use AssemblyAI as voice infrastructure
“And so the reason, you know, most AI note takers are using assembly as the voice infrastructure is because our models are the most scalable, right?”
Fox: Big tech AI labs build models disconnected from actual customer needs
“We have an amazing team of engineers, of researchers that just operate so closely to customers that they really understand, like, how this stuff is being deployed. I think that's the biggest difference between us and, like, a lab at a bigger company. So, you k…”
Fox: AssemblyAI used Claude to rebuild and migrate its entire website
“Like, you know, the most recent example is we had Cloud rebuild our whole website off of Webflow and just deployed on Vercel.”
Fox: Customers abandon fine-tuned open-source voice models due to maintenance burdens
“We've seen some customers maybe six months ago or a year ago, they'll take an open source model and fine tune it or something. And then, you know, they're now like, okay, this is like, shit, this is like really behind and it's like nightmare to maintain and I …”
Fox: Voice AI became reliable data capture only in the past year
“Like for the first time probably ever in the past year, it's like a reliable form of data capture.”
Fox: Voice will augment, not replace, screens over the next couple years
“And so I don't think that means that computers are just going to be like a screen and you just talk to it, because actually we'd get tired of talking, but I think it's going to be a dimension that's added to everything, and so that over the next couple years w…”
Fox: Multi-speaker acoustic disambiguation remains a major hurdle for humanoid robots
“One of the main problems the humanoid robots face today, because a lot of them are using our APIs If you have three people standing next to the robot, it doesn't know who to listen to. And it has a hard time disambiguating who's saying what.”
Fox: In five years, kids will expect all devices to understand voice
“Across humanoid robots, consumer electronics, like, you're gonna see voice as this dimension that you just, like, expect And what, the example I think about is, you know, when you see pictures of, like, kids, like, trying to, like, swipe on TVs or something, b…”
Fox: Local voice AI vendors beat global generalists through cultural nuance
“If you look at, like, local voice AI vendors in, you know, certain countries, like, they actually are typically the best, because it's, you know, they speak the language, they understand it, and they can more quickly identify, like, which data is good and stuf…”
Fox: Training data drives roughly 75 percent of AI model performance
“It's really, probably, like, 75% of it is, like, the data that you're training on. Like, I would say for any AI model, it's like, there's always, like, you know, like, these, like, step functions and, like, algorithms and architectures and stuff, but the data …”
Fox: Consumer voice hardware and software applications will surge within a year
“I think over the next year, we'll see a lot of, a lot more consumer applications, hardware and software, where voice is a core dimension. So, toys, games consumer electronics”