Mar 14, 2025 · 1h 17m · latent-space
Snipd: The AI Podcast App for Learning — with CEO Kevin Ben-Smith
0:00 / 0:00
▶ Watch on YouTube →
gold bands on the timeline = statements, start to end. Hover to read, click to jump. CC turns on captions
Shawn 'Swix' Wang interviews Snipd CEO and co-founder Kevin Ben-Smith to explore how their AI-first podcast player is engineered to transform passive spoken audio into structured, retrievable knowledge. They discuss Snipd's technical infrastructure, pragmatic consumer AI design principles, and the future of voice-driven conversational learning.
How this conversation actually went
Every chapter scored 0–10 on four independent dynamics. Hover any point for the reasoning behind the score. The hosts hold 31.8% of the talking time here. How this is scored →
The hosts as informed peer 5.8 Guest teaching 3.5 Guest disagreement 1.5 The hosts pushing back 2.6
speaking balance: gold is the hosts, purple is the guest (3 minute bins)
Sharpest disagreement ▶ 26:29 Clarifying ad skipping vs dynamic audio resyncing
Kevin corrects Swyx's assumption that Snipd automatically skips ads, explaining how dynamic ad insertion breaks audio timestamps and requires proprietary byte-level resyncing.
Hardest push from the hosts ▶ 1:10:00 Swyx pushing YouTube and creator focus over audiobooksSwyx strongly challenges Kevin's roadmap priorities, insisting YouTube video podcasting should take precedence over audiobooks and pitching tools tailored directly to creators.
Biggest teaching moment ▶ 1:02:00 Explaining user triggers and active synthesis over chatKevin reframes why voice AI is uniquely suited to podcast learning retention by comparing consumer trigger mechanics to Duolingo, convincing Swyx of the product strategy.
The host holds their own ▶ 34:33 Swyx contrasting Perplexity vs Exa positioningSwyx demonstrates his deep grasp of AI developer infrastructure by categorizing Perplexity as consumer-facing high-margin software versus Exa as core search infrastructure.
the scores for every segment, with the reasoning behind each
| Chapter | Topic | The hosts as informed peer | Guest teaching | Guest disagreement | The hosts pushing back | Why |
|---|---|---|---|---|---|---|
| NYC Summit Impressions and the Zurich AI Landscape | 5 | 2 | 1 | 2 | Swyx opens with friendly banter about the outdoor recording and AI Engineer Summit. Swyx brings domain knowledge regarding team movements from Google to OpenAI in Zurich, while Kevin outlines the local AI and university ecosystem. | |
| Defining Snipd: From Social Clips to Knowledge Management | 6 | 3 | 1 | 1 | Kevin explains how Snipd pivoted from a social clipping feed into a personal knowledge management tool. Swyx shares his past experience running a manual podcast mixtape clipping workflow to validate the product need. | |
| Kevin Ben-Smith’s Path from Quantitative Finance to AI | 6 | 3 | 1 | 2 | Kevin and Swyx bond over their mutual backgrounds in quantitative finance and mathematics at ETH and trading desks before transitioning into machine learning. | |
| HackZurich Origins, Team Structure, and the Apple Watch App | 4 | 3 | 1 | 2 | Kevin explains the HackZurich origin story winning with Elon Musk audio search and why a four-person team built an Apple Watch companion. Swyx questions the utility of a watch podcast app. | |
| Platform Friction and Upgrading the Legacy Podcast UX | 6 | 2 | 1 | 3 | Swyx complains about Substack's lack of support for Apple Watch podcasting and critiques Overcast's stagnant pace of innovation compared to modern AI tooling. | |
| Core AI Features and Solving Dynamic Ad Sync Challenges | 5 | 4 | 2 | 3 | Kevin details the Snipd pipeline including guest bio extraction, AI chapters, and book citations. When Swyx assumes dynamic ads are skipped, Kevin clarifies that they instead built fuzzy byte-level matching like Shazam to dynamically resync transcripts. | |
| Infrastructure Architecture: Flutter, Wav2Vec, and LLM APIs | 7 | 4 | 2 | 4 | Swyx and Kevin engage in a technical breakdown of Flutter vs React Native and discuss early audio model history like Wav2Vec. Swyx pushes on search API economics comparing Perplexity to Exa and Google Grounding. | |
| Optimizing Speaker Diarization and Consumer Product Principles | 6 | 5 | 2 | 3 | Kevin explains how Snipd uses podcast-specific structural heuristics and LLMs to refine pyannote speaker diarization clustering. Swyx compares Snipd favorably to heavily-funded competitors like Descript. | |
| Invisible AI, Timestamp Citations, and Vibe Evals | 6 | 4 | 2 | 3 | Swyx requests custom prompt summaries, prompting Kevin to explain consumer UX beyond raw chat boxes. They discuss invisible AI, streaming constraints, regex formatting cleanups, and 'vibe evals'. | |
| LLM-as-a-Judge and Workload Architecture: Batch vs. Real-Time | 6 | 4 | 1 | 2 | Kevin explains using LLM-as-a-judge for Snipd Wrapped quote curation and book entity extraction, alongside dividing architecture between predictable batch queues and serverless real-time user calls. | |
| Frontier Multimodality and Conversational Content Discovery | 6 | 3 | 1 | 2 | Swyx and Kevin discuss end-to-end multimodal audio models like Gemini 1.5/2.0 Flash and conversational recommendation algorithms that let users steer discovery beyond engagement traps. | |
| Voice AI Companions and Driving Active Learning Retention | 6 | 5 | 2 | 3 | Kevin presents the concept of a post-podcast voice AI companion to prompt active synthesis and retention, referencing Duolingo's trigger mechanics. Swyx admits he usually gets trapped in passive consumption and validates the product framing. | |
| Voice Cloning Normalization and Video Podcasts on YouTube | 7 | 3 | 2 | 4 | Swyx pushes hard on creator-centric features over listener features, advocating that YouTube video podcasts are dominant and urging Snipd to prioritize creator workflows over audiobooks. |
Statements from this episode (26)
Swix: Google's entire SigLIP vision team left to join OpenAI
“I think the most recent notable move, I think the entire vision team from Google Lucas Beyer and all the other authors of Siglip left Google to join OpenAI”
Ben-Smith: Podcast apps today are basically repurposed music players
“Like podcast apps today, they're still, they're basically repurposed music players, but we actually look at podcasts as one of the largest sources of knowledge in the world.”
Swix: My fund was among the first combining social data with quantitative trading
“We were sort of the first to combine like social data with quantitative trading. And I think I think now it's very common”
Ben-Smith: Snipd operates with a four-person, all-technical team
“We're just four people. We're just four people. Yeah. Like four, we all technical. Basically two on the backend side. So one of my co-founders is this person who got me into machine learning and startups, and we won the hackathon together. So we have two peopl…”
Ben-Smith: Substack-hosted podcasts cannot play on Apple Watch
“So we found out that all of the podcasts hosted on Substack you cannot play them on an Apple Watch.”
Ben-Smith: Snipd indexes 99% of all podcasts in-house
“Yeah, we have a search engine that is powered by ListenNotes, but I mean, in the meantime, we have a huge database of, like, 99% of all podcasts out there ourselves.”
Swix: Overcast will basically never have searchable transcripts
“I should have a podcast that has transcripts that I can search. Very, very basic thing. Overcast will basically never have it.”
Ben-Smith: Snipd Uses Perplexity API to Fetch Book Metadata
“Then we use perplexity API together with various other LLM orchestration to go out there on the internet, find everything that there is to know about the book.”
Ben-Smith: Podcast Ads Create Unique Dynamic Audio Files per Listener
“Like the way that ads get inserted into podcasts or into most podcasts is actually that every time you listen to a podcast, you actually get access to a different audio file. And on the server, a different ad is inserted into the MP three file automatically.”
Ben-Smith: Snipd Built Audio Fuzzy Matching to Re-Sync Dynamic Podcast Ads
“So it's actually not we're actually not doing exact matches, but we're doing fuzzy matches. To identify the moment. It's basically we basically built Shazam for podcasts. Just as a little side project to solve this issue.”
Snipd builds on Python, GCP, and Flutter for cross-platform clients
“So the general tech stack is our entire back end is, or 90% of our back end is written in Python. Hosting everything on Google Cloud platform, and our front end is written with, well, we're using the Flutter framework.”
Snipd has processed more than 1 million podcasts
“So we have more than a million podcasts that we've already processed.”
Ben-Smith: Perplexity web search API costs roughly $5 per 1,000 queries
“If you use the web search, the price is like five dollars per a thousand queries.”
Ben-Smith: Scaling AI products requires matching sub-tasks to the cheapest viable intelligence
“Like for us, it's not just about taking the best model for every task, but it's really getting the best, like identifying what kind of intelligence level you need, and then getting the best price for that to be able to really scale this and provide us yeah, le…”
Ben-Smith: Snipd uses LLMs to recalibrate speaker diarization switching points
“Another thing is that we actually combine it with LLMs. So the transcripts, LLMs and the speaker diarization, like bringing all of these together to recalibrate some of the switching points.”
Swix: Descript still sucks despite major funding and OpenAI backing
“Descript is so much funding. They had OpenAI invested in them, and they still suck.”
Ben-Smith: Consumer AI apps must move beyond chat boxes
“If you're building a consumer app, you have to move beyond the chat box. People do not want to always type out what they want.”
Ben-Smith: Snipd relies on regexes to format streaming LLM responses
“For this specific feature, like, we actually also have, like, countless regexes. That, that, they're just there to correct certain things that the LLM is doing, because it doesn't always adhere to the format correctly, and then it looks super ugly on the front…”
Swix: Spotify is not very good at podcasting
“Let's just say Spotify is not very good at podcasting. I have a documented dislike for their podcast features.”
Snipd CEO: Claude is the best model at phrasing and personality
“Like, in my opinion, Claude is the best one when it comes to the way it formulates things.”
Ben-Smith: End-to-end multimodal LLMs will replace modular audio transcription pipelines
“In the future that would just be put everything into a big multimodal LLM. And it will output everything that you want.”
Ben-Smith: Multimodal LLM audio transcription remains vastly costlier than self-hosted pipelines
“The big difference right now is still, like, the cost difference of doing speaker diarization this way, or doing transcription this way, is a huge difference to the pipeline that we've built up.”
Ben-Smith: AI will enable natural language steering of recommendation algorithms
“I think what actually AI will enable is not that you bring your own algorithm, but you will be able to talk. You will be able to communicate with the algorithm.”
Ben-Smith: Duolingo is the only successful app lacking a natural trigger
“There's basically only one app, one super successful app that has been able to do that without this natural trigger, and that is Duolingo.”
Swix: YouTube is the best podcast platform over Spotify and Apple
“YouTube is the best podcasting platform. It is not MP threes. It is not Apple podcasts. It is not Spotify. It's YouTube. And it's just the social layer of recommendations and the existing habit that people have of logging onto YouTube and getting, getting that…”
Swix: Most podcasts are bad because creators lack real-world experience
“The reason that most podcasts or YouTube videos are shit is they're made by people who don't have life experience, who are not that important in the world. They're not doing important jobs. And so what you want to actually enable is CEOs to each of them make t…”