Feb 29, 2024 · 1h 2m · mad
How Intercom transitioned to being AI-first | Des Traynor, Co-Founder of Intercom
0:00 / 0:00
▶ Watch on YouTube →
gold bands on the timeline = statements, start to end. Hover to read, click to jump. CC turns on captions
Des Traynor, Co-founder and Chief Strategy Officer of Intercom, joins host Matt Turck on The MAD Podcast to discuss Intercom's rapid pivot to becoming an AI-first customer support platform, detailing the creation of their AI chatbot Fin, value-aligned pricing models, and lessons in executive leadership.
How this conversation actually went
Every chapter scored 0–10 on four independent dynamics. Hover any point for the reasoning behind the score. Matt holds 19.2% of the talking time here. How this is scored →
Matt as informed peer 3.7 Guest teaching 3.5 Guest disagreement 1.3 Matt pushing back 1.2
speaking balance: gold is Matt, purple is the guest (3 minute bins)
Sharpest disagreement ▶ 59:15 Des slams EU regulatory tendencies and cookie banners
Des forcefully expresses frustration with European technology policy, calling cookie banners mindless crap and warning that over-regulation will hinder AI development in Europe.
Hardest push from Matt ▶ 40:05 Matt presses Des on per-seat revenue cannibalizationMatt directly challenges Intercom's economic alignment by invoking the Innovator's Dilemma, forcing Des to address how resolution pricing eats into legacy seat revenue.
Biggest teaching moment ▶ 36:50 Des details the extreme financial costs of LLM featuresDes educates Matt on the stark economic realities of building on LLMs, explaining that blanket summarization of Intercom's message volume would bankrupt the company.
Matt holds his own ▶ 21:00 Matt challenges AI defensibility citing Sierra AIMatt demonstrates sharp market knowledge by citing Bret Taylor's newly launched Sierra AI and pressing Des on how any wrapper maintains a moat when relying on identical underlying models.
the scores for every segment, with the reasoning behind each
| Chapter | Topic | Matt as informed peer | Guest teaching | Guest disagreement | Matt pushing back | Why |
|---|---|---|---|---|---|---|
| The Catalyst Behind Intercom's AI Pivotal Shift | 2 | 3 | 1 | 0 | Matt opens by asking what enabled Intercom to move so quickly onto generative AI in early 2023. Des shares the internal timeline following ChatGPT's release, including weekend experimentation and executive alignment to launch Inbox AI and Fin. | |
| Why Customer Service is in the AI 'Kill Zone' | 3 | 4 | 1 | 1 | Matt asks why customer service is squarely in AI's kill zone, and later references Intercom's published manifesto. Des explains that LLMs excel at conversational problem solving, disambiguation, and quick procedural tasks like password resets. | |
| Intercom's Strategic Evolution to an AI-First Support Platform | 3 | 3 | 1 | 0 | Matt provides accurate historical context on Intercom's founding in 2011 and asks what the core offering was before the AI shift. Des explains the strategic narrowing from a general communication platform to customer support, then AI-first support. | |
| Defining the 'AI-First' Paradigm & Product Mindset | 3 | 5 | 1 | 1 | Matt asks whether 'AI-first' refers to development effort or a complete shift in customer service interaction logic. Des delivers an extended breakdown on prioritizing outcome automation over step automation and rethinking UI for natural interaction. | |
| Symbiotic Relationship Between Humans and AI in Support | 2 | 4 | 1 | 1 | Matt inquires about UX lessons and offering choices between AI and human agents. Des outlines the symbiotic relationship where bots resolve direct queries, assist human reps, and learn from human corrections. | |
| Overcoming Chatbot Trauma and Changing User Behavior | 4 | 3 | 1 | 1 | Matt highlights the persistent user PTSD created by poor legacy chatbots. Des acknowledges historical bot generations and describes how public expectations and user input styles ('bot speak') are evolving. | |
| The 'Thick Wrapper' Philosophy & Model Independence | 5 | 4 | 2 | 2 | Matt references Intercom's early integration with OpenAI models and cites Des's concept of a 'thick wrapper'. Des distinguishes basic PDF QA apps from complex enterprise support platforms requiring reporting, guardrails, and multichannel support. | |
| Building Moats, Brand, and Model Agnosticism | 6 | 4 | 2 | 3 | Matt presses Des on defensibility, citing Bret Taylor's newly launched Sierra AI and asking how moats exist when competitors use identical foundational models. Des argues that technical moats are fleeting and defensibility comes from product momentum and brand. | |
| Model Evaluation, Benchmarking, and Optimization Strategy | 3 | 4 | 1 | 1 | Matt asks about evaluating model options like GPT-4, Claude, Llama, and Mistral. Des explains Intercom's automated and manual torture tests across thousands of queries, prioritizing trust before cost optimization. | |
| Mitigating Hallucinations and Managing Knowledge Sources | 4 | 4 | 2 | 2 | Matt identifies hallucination as the core barrier for enterprise generative AI adoption. Des clarifies that poor answers usually stem from stale help center data rather than model invention, criticizing competitors who trade accuracy for fake coverage numbers. | |
| Per-Customer Customization & Ingesting Enterprise Data | 4 | 4 | 1 | 1 | Matt asks about per-customer customization and tracking individual user histories across visits. Des details RAG data ingestion across tools like Notion and Confluence, CDP integration, and the L1 to L5 framework of support automation. | |
| Economic Reality: AI Costs, Gross Margins, and Resolution Pricing | 5 | 4 | 1 | 2 | Matt raises unit economics and gross margin pressures associated with AI features. Des candidly reveals that summarizing all 500 million monthly messages would bankrupt the company, forcing strict value-aligned pricing at $0.99 per resolution. | |
| Navigating the Innovator's Dilemma & Usage-Based Pricing | 5 | 4 | 2 | 3 | Matt challenges Des on the Innovator's Dilemma, asking how Intercom manages tension between traditional seat-based revenue and Fin's usage-based resolution pricing. Des embraces the disruption, asserting that SaaS is naturally shifting away from seat-based models. | |
| Measuring Resolution Rates and Customer Satisfaction (CSAT) | 4 | 3 | 1 | 2 | Matt asks how 'resolution' is defined and whether buyers need education. Des breaks down involvement rates versus resolution criteria, noting that Matt's philosophical observation about holding AI to higher standards than humans is spot on. | |
| Global Adoption, Resolution Rates, and Autonomous AI Actions | 4 | 3 | 1 | 1 | Matt brings up specific resolution rate stats around 41% and asks about autonomous actions. Des confirms the resolution figures and previews upcoming capabilities like third-party API lookups and automated actions. | |
| Customer Adoption Spectrum & 2024 as the Year of AI Adoption | 3 | 3 | 1 | 1 | Matt cites Intercom's customer footprint of 25,000 organizations and inquires about adoption dynamics. Des contrasts risk-averse legacy institutions with aggressive SF startups, declaring 2024 as the definitive year of market AI adoption. | |
| Adapting Intercom: Executive Leadership Changes and AI Focus | 4 | 3 | 1 | 1 | Matt asks how Intercom adapted internally during the tech downturn, referencing leadership changes such as Owen returning as CEO. Des discusses cost right-sizing, strategic refocusing on customer support, and advising startups to be everything to somebody. | |
| The Founder Dynamic: 13 Years of Collaboration and Mission | 3 | 2 | 0 | 0 | Matt asks about the 13-year co-founder dynamic among the four Irish founders. Des attributes their longevity to explicit operational hierarchy, clear ownership, and deep personal friendships. | |
| Navigating Distributed Teams and Hybrid Work Culture | 3 | 2 | 1 | 0 | Matt asks about managing teams across San Francisco and Dublin and Des's current stance on remote work. Des voices strong support for in-person hybrid work, noting that face-to-face contact provides essential social richness. | |
| European Tech Landscape: AI Regulation and Growth Ambitions | 4 | 4 | 3 | 1 | Matt asks Des for his perspective on the European startup landscape compared to Silicon Valley. Des sharply criticizes European regulatory tendencies like cookie banners, while highlighting standout exceptions like Mistral and AMO. |
Statements from this episode (29)
Traynor: Intercom's AI chatbot Fin handles millions of customer resolutions
“Like we have like thousands of people using it to do millions of full resolutions for people”
Traynor: Customer service is extremely vulnerable in AI's 'kill zone'
“I think genuinely customer service is very much in like the kill zone of AI or whatever. I think it's really you know, it's a very vulnerable space if you don't move with the times.”
Traynor: For routine support, customers value speed over personal tone
“And people actually care about instancy more than they care about like you know, let's just say the personal tone in those types of things.”
Traynor: Intercom had no choice but to rapidly adopt an AI strategy
“I think we moved fast, but I genuinely think that, like, there was no other option. I think, were we sitting here without an AI strategy today, we'd be deeply worried.”
Traynor: AI builders must automate complete outcomes rather than individual minor tasks
“We shouldn't be approaching AI from just simply, how can we automate a task? Or how can we automate a little step? Or like, you know, how can we summarize this paragraph? Or you have to actually look at how can we automate the outcome?”
Traynor predicts broad adoption of natural language input for AI chatbots
“I think you're seeing a bit more people just saying, like, they've gone back to the whole, I will tell you what you need to know, and I'll trust when I enter, you're going to figure it out, and I think that has taken a while, but I think we're going to see a k…”
Traynor: Public quickly accepted that modern AI chatbots work
“The biggest learning for me has been how quickly the public have gotten on board with the idea that chatbots work now, you know.”
Traynor: GPT-4 crossed the hallucination threshold required for customer support bots
“So Finn's built on GPT-IV, by the way, we've, we tried, we wanted to build on a three, 3.5, but it didn't, it's still, I remember back when we used to talk about hallucinations, like four was the sort of the perceptual change for us in terms of trust and relia…”
Traynor: Many startups in his angel portfolio build thin AI wrappers
“When I like, you know, I also invest in companies and I talk to companies in my portfolio and a lot of them are kind of building these thin wrapper solutions.”
Traynor: Intercom is model-agnostic and will drop GPT-4 if a better model emerges
“I think all of our work is pretty model agnostic. We use four because it's best, but like the very second there's a better model out there, we'll do it.”
Traynor: The era of SaaS is an era of "right-click view source"
“The era of SaaS is an era of like right click view source. You know, ultimately like if there's a single thing you've done, other people can do it. The job is to build a brand around having the best product. And to do that, you often, you have to have the fres…”
Intercom evaluates AI models using a benchmark suite of thousands of scenarios
“We have a torture test of like thousands of questions and scenarios that we run any given model through.”
Traynor: AI builders should explore capabilities before optimizing for cost
“I would encourage anyone who's trying to do the You know, it's basically one of those explore then exploit. We are still deeply exploring how much we can transform the nature of CS with AI. When we were confident we finished that exploration, there will be an …”
Traynor: Operating multiple AI models carries significant ongoing maintenance work
“That's why we wouldn't necessarily be running with a bank of 10 of them, because to own any given one of them in any period, it doesn't ongoing work to it. It's not for free.”
Traynor: AI chatbot errors usually stem from bad content, not hallucinations
“A lot of the times when Finn gives bad answers, it's not hallucination. It's actually usually just bad content and it's stale content on our customers help centers”
Traynor: AI support lets human teams answer a specific question only once
“The way I think about Finn in the lives of our customers who are customer service teams is that it should move them to being the, they answer questions for their first time because Finn has never seen it before, but also the last time, because once they've ans…”
Traynor: Higher AI resolution rates come at the cost of worse answers
“If you're willing to tolerate worse answers you can hit a higher resolution rate, but that's a very conscious trade-off that I think people aren't honest enough about.”
Traynor: Intercom uses RAG rather than per-customer model fine-tuning
“We're not yet building models per customer or tuning models per customer. We're doing rag.”
Traynor: Intercom plans to push customer support automation from 50% to 90%
“Does it like, you know, we have genuinely a sort of full spectrum of how we're going to automate support to get from our current, say, 40, 50% total automation up to like maybe seventies, eighties, nineties, depending on the business.”
Traynor: AI robots will inevitably handle phone-based customer support
“Like, voice from a point of view of robots answering the phone, and hearing your query, and answering it as best as they can, and if not, failing over to a human voice. I think that's all certainly going to happen. Don't have a stated time frame for it yet.”
Traynor: Intercom powers 500 million conversations a month
“Summarize every conversation Intercom powers every month. That would be summarizing five hundred million conversations a month.”
Traynor: AI will accelerate the death of seat-based SaaS pricing
“I do think separate to all this, SaaS is going to be a bit more usage-based and metered anyway. I think the days of, like, seats above all are starting to fade away. AI is going to accelerate it, but I think generally speaking, people are kind of realizing tha…”
Traynor: Intercom charges for AI Fin only when no follow-up occurs
“We basically charge when we gave the customer an answer that they saw and they didn't have any follow-up questions, which is exactly how CS reps are measured as well.”
Traynor: Any company enabling Intercom's Fin gets 25% to 30% resolution rate
“On average, I would be confident any business who turns on fin will get at least like 25, 30.”
Traynor: Test AI safely on free users, unstaffed hours, and unsupported languages
“What we encourage those people to do is just work out a way to dip their toe in. So like, hey, how about if Fin only works with your free customers? How about if Fin only works in the languages you don't speak? How about if Fin only works on the errors that yo…”
Des Traynor: Startups must be everything for somebody, not something for everybody
“The framing I would use with a lot of the startups I've invested in or spoken to is like, you have to be everything for somebody and not something for everybody.”