Jan 31, 2024 · 38m · no-priors
No Priors Ep. 49 | With Shopify VP of Core Product Glen Coates
0:00 / 0:00
▶ Watch on YouTube →
gold bands on the timeline = statements, start to end. Hover to read, click to jump. CC turns on captions
Shopify VP of Core Product Glen Coates joins Sarah Guo and Elad Gil on No Priors to discuss engineering resilience, founder-driven leadership, and the architectural principles behind deploying dependable AI assistants across global e-commerce.
How this conversation actually went
Every chapter scored 0–10 on four independent dynamics. Hover any point for the reasoning behind the score. The hosts hold 22.5% of the talking time here. How this is scored →
The hosts as informed peer 4.6 Guest teaching 3.3 Guest disagreement 0.6 The hosts pushing back 0.4
speaking balance: gold is the hosts, purple is the guest (3 minute bins)
Sharpest disagreement ▶ 26:47 Demystifying LLM Hallucinations
Coates playfully reframes popular complaints about AI hallucinations, pointing out that generating tokens is inherently hallucination and pushing past superficial industry talking points.
Hardest push from the hosts ▶ 17:32 Pressing on Shopify's Risk ToleranceSarah Guo challenges Coates on whether Shopify possesses an unusually aggressive risk orientation for its scale given that suggestions can still impact live merchant storefronts.
Biggest teaching moment ▶ 8:35 Anatomy of Stack RedundancyCoates gives a masterclass on architectural debt, explaining how separate feature teams accidentally rebuild duplicate pricing engines beneath the checkout stack.
The host holds their own ▶ 35:50 Gil Demonstrates Search System ExpertiseElad Gil draws directly on his tenure at Google and leading Search at Twitter to explain navigational query intent and argue that LLMs should act as intent routers rather than end-to-end engines.
the scores for every segment, with the reasoning behind each
| Chapter | Topic | The hosts as informed peer | Guest teaching | Guest disagreement | The hosts pushing back | Why |
|---|---|---|---|---|---|---|
| Background and Journey to Shopify Leadership | 3 | 1 | 0 | 0 | Sarah Guo opens the episode with a warm introduction and invites Glen Coates to outline his background founding Handshake and his transition into leading core product at Shopify. | |
| Navigating Code Reds and Organizational Structure | 4 | 3 | 1 | 0 | Elad Gil probes how Shopify retains acquired founders and restructures around centralized execution. Coates details Tobi Lutke's decisive, anti-consensus management style. | |
| Architectural Cohesion and Eliminating Stack Redundancies | 5 | 6 | 1 | 0 | Gil draws an analogy to historical Google infrastructure bottlenecks. Coates provides an educational technical breakdown of how duplicate business logic engines emerge at the same stack layer. | |
| AI Philosophy and Lowering Barriers for Entrepreneurs | 4 | 3 | 0 | 0 | Gil asks about balancing rapid AI adoption with unified platform architecture. Coates outlines Shopify's core mission of turning complex software controls into intuitive AI co-pilots for merchants. | |
| Safety Frameworks and Human-in-the-Loop AI Deployment | 5 | 4 | 1 | 2 | Guo questions how Shopify ships non-deterministic AI tools so rapidly despite enterprise quality risks. Coates articulates their strict human-in-the-loop paradigm where models only propose actions. | |
| Core Upgrades: Product Taxonomies, AI Imagery, and Search | 6 | 5 | 0 | 0 | Coates deep-dives into product taxonomy overhauls, breaking changes in pagination, and semantic search upgrades. Gil engages on the technical mechanics of multimodal weights and fine-tuning embeddings. | |
| The Production Challenge: Bridging the 75% to 95% AI Gap | 5 | 5 | 2 | 1 | Gil asks what capabilities are missing in current AI. Coates offers a contrarian take on hallucination, stressing that generating plausible text is an LLM's inherent function, making production reliability difficult. | |
| Evaluating Interaction Interfaces: GUI Agents versus APIs | 6 | 3 | 0 | 0 | Coates highlights Rabbit's GUI interaction model as a compelling alternative to API-based agents. Guo matches his analysis by sharing insights from her direct discussions with Rabbit's founder Jesse Lyu. | |
| The Future of Search: Information Retrieval versus Direct Synthesis | 8 | 3 | 1 | 1 | Coates initiates a discussion on Perplexity versus ChatGPT and Google. Gil leverages his background leading Search at Twitter and working at Google to explain navigational intent and LLM query routing. | |
| Episode Conclusion and Distribution Channels | 0 | 0 | 0 | 0 | Sarah Guo delivers the standard closing housekeeping remarks and subscription information. |
Statements from this episode (17)
Coates: Shopify's 2020 checkout 'code red' took 200-300 people for a year
“At least in my case, checkout code red in 2020 was, hey, the checkout is failing in, I don't know, three, four, five different ways. And the checkout is a pretty important part of Shopify, obviously. So let's go fix that. So we, me and, you know, somewhere bet…”
Coates: Shopify consolidated from 12-15 business units down to 3-4
“There used to be 12 to 15 of these kind of fairly small fractured business units, and now there is actually only like three or four, which is actually truer to what the product is, but of course, each of those units is bigger than the ones that were before, so…”
Coates: High percentage of Shopify R&D leaders are former founders
“When I look at like my management team, the people who report to me, who each of these sort of nine or 10 people run R and D orgs that are like a couple hundred people each and almost like a very high percentage of those people are also founders, right?”
Gil: Google's identity layer build froze company-wide shipping for 18 months
“Like at Google, there was a famous sort of identity layer that was built, and for a year and a half, people just didn't ship things, right, in other areas.”
Coates: Redundant systems at the same stack layer cause compounding upstream delays
“The real answer to how do you solve this problem is one is developing extremely good intuitions for when you are accidentally creating duplicates at the same layer of the stack. So you just never get into the problem in the first place. And then the second par…”
Coates: Companies trying to serve both SMB and enterprise usually fail
“The world is littered with the corpses of, you know, the companies that tried to serve SMB and enterprise at the same time.”
Coates: AI copilots remove UI complexity barriers for entrepreneurs
“AI brings us into the world of like, okay, well, what if we can give you the driver in the car who knows how all these switches work and can help you along without having to learn all the switches? It's probably one of the most powerful opportunities for more …”
Coates: Shopify's AI rule is models can propose but never commit
“One of the principles that we have for all of the Shopify magic features today is that they're allowed to propose changes, but not commit changes, right? So they can like generate text, but they're not going to save it without you actually reading it and savin…”
Coates: Human-in-the-loop AI generates a three-part feedback loop for models
“You actually get a three part signal. You get which suggestions are accepted clean, which suggestions were accepted, but then with minor edits and then which suggestions were outright rejected. And that's An amazing loop to be able to improve things through.”
Coates: Shopify accounts for roughly 10% of all US e-commerce
“The fact that you can get to like 10% of all us e-commerce on like a fairly simplistic data model is actually kind of amazing.”
Coates: Shopify storefronts will soon feature semantic search capabilities
“Very soon Shopify stores will actually understand the meanings of things as you, so today Shopify search is very, very literal keyword searchy.”
Coates: AI agents reach 75% in 10 minutes, but 95% is brutal
“Well, look, I mean, I think the exciting and frustrating thing about LLMs right now is that you can get an LLM like agent thing, like a sidekick-esque thing. You can get it to like 75% in like 10 minutes. And then it's like this brutal hill climb to get it to,…”
Gil: Solving the final 5% of AI reliability will take five years
“Yeah, the last five percent, I think, is the next five years of tech or something.”
Coates: Factual LLM outputs are merely the hallucinations that users prefer
“Yeah, you know, that's its job, right? You know that every time it emits a token, it's hallucinating, right? It's just that you like some of the hallucinations more than the others, right?”
Coates: GUI AI agents instantly turn the open web into an app store
“If it actually works, the strategic brilliance of that is they instantly have the world's largest app store because the world's largest app store is just the web, right?”
Coates: Perplexity sits exactly between ChatGPT's interface and Google's search
“The sort of the triangle of like perplexity, Google, ChatGPT of like, it's basically like LLM native search and like ChatGPT really good at the LLM interface thing, not amazing at search yet. Google amazing at search, not really figured out the LLM thing yet. …”
Guo: Specialized AI will fracture search from a single unified market
“We're at this very, I think, special moment where the technology and perhaps the like user behavior, the expectation, if we take out like slices of it, where the expectation is very different and get people to think about pieces of search Because it's not at a…”