Jun 2, 2023 · 44m · a16z
Embedded AI: The Questions Every CEO is Asking
0:00 / 0:00
▶ Watch on YouTube →
gold bands on the timeline = statements, start to end. Hover to read, click to jump. CC turns on captions
This episode of the a16z Podcast explores how corporate leaders and technology founders are navigating the transition from consumer AI hype to enterprise integration. Featuring executives from Cresta, Hex, and Sourcegraph, the discussion examines strategic product moats, context retrieval techniques, UX design, and enterprise data security.
How this conversation actually went
Every chapter scored 0–10 on four independent dynamics. Hover any point for the reasoning behind the score. How this is scored →
The host as informed peer 4.9 Guest teaching 4.2 Guest disagreement 1.1 The host pushing back 2.0
speaking balance: gold is the host, purple is the guest (3 minute bins)
Sharpest disagreement ▶ 27:24 Barry candidly acknowledges UI copying by competitors
Barry directly concedes the host's skeptical premise that UI elements are easily copied by competitors, admitting Hex has already seen competitors copy their design.
Hardest push from the host ▶ 26:48 Host challenges AI SaaS defensibility and moatsThe host refuses to accept standard startup pitching points and directly confronts Barry on whether UI features, data cleaning, or model wrappers offer any true defensibility against fast copycats.
Biggest teaching moment ▶ 14:00 Byung explains fuzzy human-like memory in LLMs vs computer memoryByung educates the host on why LLMs naturally hallucinate by contrasting fuzzy human-like associative memory in language models with exact computer memory, showing why codebase graph lookup is required.
The host holds their own ▶ 16:07 Host drills guest on specific market competitorsThe host demonstrates strong technical market awareness by explicitly contrasting Sourcegraph Cody against named competitors like GitHub Copilot and Replit Ghostwriter to force a granular comparison.
the scores for every segment, with the reasoning behind each
| Chapter | Topic | The host as informed peer | Guest teaching | Guest disagreement | The host pushing back | Why |
|---|---|---|---|---|---|---|
| The Strategic Questions Facing CEOs Implementing AI | 4 | 5 | 1 | 1 | The host synthesizes Zed Inam's points on turning unstructured contact center call audio into actionable product insights. Zed educates the host on moving past lazy automation to creative AI and gives the Intel Andy Grove historical analogy. | |
| Introduction to Hex and Data Analytics (Barry McArdle) | 3 | 3 | 1 | 1 | The host introduces Hex Magic and prompts Zed on customer trust and transparency around AI bot disclosures versus brand risk. The dynamic is exploratory and collaborative across both clips. | |
| Introduction to Sourcegraph and Cody (Byung Liu) | 6 | 5 | 1 | 2 | The host demonstrates strong market context by asking Byung Liu how Cody specifically differentiates context retrieval compared to GitHub Copilot and Replit Ghostwriter. Byung delivers an educational breakdown of LLM fuzzy memory versus codebase graphs. | |
| The Open Source Differentiation and Broader Ecosystem Context | 5 | 3 | 1 | 1 | The host extends Byung's open-source context argument with an apt conceptual analogy about writing an article with five open browser tabs versus deep search. The tone remains highly collaborative. | |
| Proprietary Datasets, Customization, and UI as Differentiators | 3 | 4 | 1 | 1 | Zed Inam and Barry McArdle explain how proprietary schemas, interaction histories, and workflow data generate superior prompts without running unvetted code. The host guides transitions smoothly between guests. | |
| UX Design Principles, Latency, and Context Windows in Hex | 5 | 5 | 1 | 2 | Barry highlights the lesson that overwhelming LLMs with too much context degrades output quality. The host contributes an informed anecdote about a founder linking fifteen distinct AI models together. | |
| Evaluating Moats and Competitive Advantage in AI SaaS | 7 | 5 | 2 | 7 | The host directly pushes Barry on whether AI startup UIs and wrappers offer any genuine moat when competitors can effortlessly replicate features. Barry candidly acknowledges competitor copying while defending deep workflow integration. | |
| Enterprise Security, Multi-Model Approaches, and Search Engines | 6 | 4 | 1 | 2 | The host cites Bing's overnight 5x price hike to highlight vendor dependency risks for AI developers. Byung explains why retrieval engines remain essential alongside LLM reasoning engines. | |
| Data Retention Trade-Offs, Risk Management, and Organizational Knowledge | 5 | 4 | 1 | 1 | Zed paints a vision of future contact centers where background AI eliminates administrative burden, letting agents focus on relationship building. The host recontextualizes the customer service agent into a strategic account manager role. |
Statements from this episode (14)
Zed Inam: Contact center employee net promoter scores are often negative
“Employee net promoter score for contact centers often less than zero.”
Zed Inam: End-to-end automation of existing processes is lazy AI
“When you look at artificial intelligence, I think there's really two ways to look at it. There's one way to look at it, which is lazy artificial intelligence. And that's basically, I have an existing process that my business does right now. I'm going to take t…”
Zed Inam: Brands deploying AI agents must default to transparency
“I think at the end of the day, like if you are a brand you are fundamentally building trust with your subscriber or customer base. Right. Your brand value is like, sort of, Hey, I can like call up this company and like, they'll take care of me or like, I can t…”
Byung Liu: LLM output quality depends primarily on provided context
“Context is king. So the context that you provide to a language model really dictates the quality of the output of the model, especially if you're asking about things that are kind of outside the training corpus of that model.”
Byung Liu: Sourcegraph's Cody is the only AI fetching full codebase context
“As far as I know, Cody is, is the only AI enabled editor assistant or coding tool today that, that fetches context as kind of like as broadly as we do.”
Byung Liu: Sourcegraph's code graph achieves compiler-level accuracy across languages
“Like that's the code graph that we spent the past 10 years building. And we can do that essentially in any language, any code base at compiler level accuracy.”
Barry McArdle: Hex enforces a rule to never automatically run AI code
“We have a rule as we're building these features. It's like, we don't run the code for you. Like we will generate the code. We will show it to you, but we want to keep the human in the loop.”
Barry McArdle: Passing too much context into LLM prompts confuses the model
“I think we've been, we were tempted early on to try to shove as much context as we could in. You kind of figure like the more I can tell this model, the better. And you realize you can pretty easily confuse a model in terms of the amount of context you're pass…”
Barry McArdle: Context building and model chaining are AI's hardest work
“And I think that is really actually the hard work in many ways of Incorporating AI into your product, right? Like it feels really simple. Like you can go get an API. Anyone listening to this today could go to the open air website and get an API key. It's every…”
Barry McArdle: AI foundation models and API calls are already commodities
“I think the models themselves and being able to send a request to one of them is, is already a commodity.”
Barry McArdle: Hex does not use customer code to train AI models
“We're not using it to train models. I should be very clear. Like that's one thing we've been very upfront with our customers about, like we're not training models where they would expect to ever have like some code they've written be a completion for someone e…”
Byung Liu: Sourcegraph will offer a self-hostable LLM within six months
“And while we don't have one that's like completely self-hostable yet, I think, you know, with all the interesting things happening around, you know, llama, alpaca and that sort of thing, I think like within six months we'll, we'll have like a viable language m…”
Byung Liu: LLMs increase search engine value instead of replacing them
“It is not true that language models make search engines unnecessary. If anything, they make the search engines more valuable because now all that data that you can search Becomes like 10 x more powerful because you can use that to, you know, get to get to your…”