Mar 12, 2026 · 47m · mad
Everything Gets Rebuilt: The New AI Agent Stack | Harrison Chase, LangChain
0:00 / 0:00
▶ Watch on YouTube →
gold bands on the timeline = statements, start to end. Hover to read, click to jump. CC turns on captions
In this episode of The MAD Podcast, host Matt Turck interviews LangChain CEO Harrison Chase about the architectural evolution of the modern AI agent stack, spanning agent harnesses, context engineering, memory systems, and enterprise infrastructure. The conversation provides actionable guidance on how developers and enterprises can build durable, production-grade autonomous agents while navigating shifting model capabilities.
How this conversation actually went
Every chapter scored 0–10 on four independent dynamics. Hover any point for the reasoning behind the score. Matt holds 16.9% of the talking time here. How this is scored →
Matt as informed peer 3.4 Guest teaching 4.4 Guest disagreement 0.9 Matt pushing back 0.9
speaking balance: gold is Matt, purple is the guest (3 minute bins)
Sharpest disagreement ▶ 28:01 Pragmatic resistance to MCP market consensus
When the host asks if Model Context Protocol has normalized as the industry-wide standard, the guest candidly cool-downs the hype with 'MCP is fine' and redirects attention toward lower-level primitives.
Hardest push from Matt ▶ 6:26 Host questioning framework vs model dominanceThe host directly pushes the guest on whether foundation model labs will swallow the framework layer or if frameworks will commoditize the underlying models.
Biggest teaching moment ▶ 11:59 Correcting misconceptions about agent planning toolsThe guest corrects the host's assumption that agent planning tools strictly enforce step-by-step DAG execution, explaining how modern LLMs perform better using loose context scratchpads.
Matt holds his own ▶ 34:07 Host displaying deep insider knowledge of Kensho talent factoryThe host steps in to list a series of notable AI startup founders originating from Kensho, showcasing deep knowledge of the guest's former workplace.
the scores for every segment, with the reasoning behind each
| Chapter | Topic | Matt as informed peer | Guest teaching | Guest disagreement | Matt pushing back | Why |
|---|---|---|---|---|---|---|
| Episode Overview: Deep Dive into the AI Infrastructure Stack | 1 | 4 | 1 | 0 | The host sets up the podcast intro and asks an open question contrasting early AI agents with recent advances. The guest provides a detailed historical timeline covering ReAct, AutoGPT, and modern harness primitives. | |
| Conversational Agents vs. Long-Horizon Coding Agents | 3 | 4 | 1 | 1 | The host asks whether conversational agents will eventually become coding agents as they deepen. The guest outlines the split between low-latency chat agents and long-horizon agents, explaining why coding primitives predominate. | |
| The Strategic Dynamics Between Models, Harnesses, and UIs | 4 | 5 | 2 | 2 | The host poses a strategic question on whether foundation models will eat framework layers or vice versa. The guest reframes the dynamic around harnesses and UI coupling, gently pushing back on pure model dominance. | |
| The Role of Detailed System Prompts in Driving Agent Architecture | 3 | 4 | 1 | 1 | The host checks if planning tools enforce strict step-by-step execution. The guest corrects this intuition, detailing how modern LLM agents perform better with unconstrained scratchpads rather than rigid state-machine constraints. | |
| Sub-Agents, Context Isolation, and Inter-Agent Communication | 2 | 5 | 1 | 0 | The host prompts an explanation of sub-agents and prompt creation. The guest breaks down how context window isolation improves performance while introducing inter-agent communication failure modes. | |
| Virtual File Systems as Context Engineering Tools | 3 | 5 | 1 | 2 | The host asks if virtual file systems in agent architectures are literal POSIX disks or database abstractions. The guest explains how virtualizing databases into file paths matches LLM training distributions. | |
| Synthesizing Core Components & Introducing Execution Sandboxes | 4 | 4 | 0 | 1 | The host synthesizes four core primitives and asks if the list is comprehensive. The guest expands the list to include code execution sandboxes and progressive disclosure skills. | |
| Context Compaction and Agent-Driven Summarization | 2 | 5 | 0 | 0 | The host asks for a technical breakdown of context compaction. The guest explains window truncation, message summarization, and an upcoming tool for agent self-compaction. | |
| Demystifying Agent Memory: Short-Term vs. Long-Term Architecture | 3 | 6 | 1 | 1 | The host seeks clarification on what 'memory' actually means across different components. The guest educates the host by breaking down short-term context vs semantic, episodic, and procedural long-term memory. | |
| Architectural Trends: Single Mega-Agent vs. Sub-Agent Fleets | 4 | 5 | 2 | 1 | The host asks whether architecture is moving toward one mega-agent or fleets of sub-agents. The guest advises enterprise builders not to get attached to topology scaffolding, but rather focus on domain tools and prompts. | |
| Model Context Protocol (MCP) and Infrastructure Stability | 5 | 4 | 2 | 2 | The host asks if protocols like MCP have standardized the industry and probes sandbox security models. The guest downplays MCP hype as simply 'fine' while detailing API key proxying outside execution sandboxes. | |
| Harrison Chase's Career Journey: Kensho, Robust Intelligence, and LangChain's Founding | 6 | 2 | 0 | 0 | The host demonstrates deep ecosystem domain knowledge by naming multiple successful startup founders coming out of Kensho. The guest warmly confirms the strong engineering culture of his former employer. | |
| The Evolution of LangChain: From V0 Chains to LangGraph and CreateAgent | 3 | 5 | 1 | 0 | The host asks the guest to contrast early LangChain with modern versions. The guest candidly explains why early v0 abstractions failed in production, driving the pivot to LangGraph runtimes. | |
| LangSmith Platform: Observability++, Deployments, and No-Code Agent Builder | 4 | 4 | 0 | 1 | The host asks about continuous evaluation loops and the proper abstraction level for no-code builder tools. The guest details how LangSmith links traces, human feedback, prompt optimization, and no-code memory. | |
| LangChain's $125M Series B Funding & Strategic Roadmap | 4 | 4 | 1 | 1 | The host asks where enterprise value accrues if agent harnesses and models commoditize. The guest reiterates that differentiation resides in domain instructions and customized tool integrations. |
Statements from this episode (23)
Chase: High reliability AI agents require structured, graph-like scaffolding workflows
“So people ended up building scaffolding around the models to make them do things in a more predictable and reliable way. And that's why we at link chain, we built lane graph, which was another framework really aimed at that kind of like. Graph like workflows a…”
Chase: Major AI labs' RL focus on code makes coding agents perform best
“All the big model labs have been RL and code and bash and editing files into those models. And so that is the stuff that works the best.”
Chase: LangChain is debating building a separate harness for conversational agents
“We talk a bunch about this internally because we're debating whether we should build like a different type of agent harness for these types of agents.”
Anthropic's Claude Code uses custom harness tools over model-level RL tools
“It doesn't actually use the tools that are RL into the model. So like anthropic models have some like file editing tools. They have a completely different set of tools in, in the actual harness.”
Chase: Agent harnesses matter more for performance than underlying models
“I, the, so I don't know what happens, but I do know the harness is really, really important. Like, I think this is the thing that matters.”
Harrison Chase: Most current agent harnesses feature built-in sub-agents and skills
“Most harnesses today have sub agents built in. They have skills built in.”
Chase: Human standard operating procedures should guide AI agent system prompts
“If you have a standard operating procedure for how a human should do things like that, that should influence a lot of what the system prompt is.”
Harrison Chase: Modern AI agent harnesses don't strictly enforce execution plans
“And so now what most things do is they just have that plan in the text file, and the main agent can, like, use that to help guide its actions, but there's nothing that says I'm explicitly doing this step, or I'm explicitly doing another step.”
Harrison Chase: Inter-agent communication is the hardest part of working with AI agents
“Communication is the hardest part of life, by the way, it's the hardest part of startups, hardest part of relationships, hardest part of working with agents is getting them to communicate.”
Chase: Virtual file systems let LLMs manage their own context windows
“And the way that I think about a file system is it basically lets the LLM manage its own context window. So it can decide what to read from files.”
LangChain's Deep Agents offloads large tool outputs into virtual files
“Is if you call a tool and it comes back with like 60,000 tokens, we don't show that all to the LLM because that's a ton of tokens. Rather, we actually put that in a file and then say, hey, here are the first like thousand tokens. If you want to read the rest, …”
Manus, Claude Code, and Deep Research share four core architectural primitives
“We saw Manus, we saw Cloud Code, we saw Deep Research, they all had these four things and we were like, okay, that's pretty common, let's put it into a Python package and make it easy for people to build their own versions of that.”
Chase: AI agent context compaction must retain recent messages for flow
“You want to keep around like the last N messages, let's say the last like 10 or so messages, because if you compact everything, it actually like throws it off completely.”
LangChain adds tools for AI agents to autonomously trigger context compaction
“One interesting thing there, actually, that we haven't yet released as of this recording, but will probably be released by the time it comes out, is we actually give the agent a tool to trigger its own compaction.”
Chase: DeepAgents enables agent learning by editing procedural memory files
“So one of the things that we do in deep agents is we represent those all as files. And so the agent can update those as they go along so it can learn things. And so when we say agents kind of like can learn with deep agents, what that really means is it can mo…”
Harrison Chase: AI agents will evolve into synchronous interfaces orchestrating asynchronous sub-agents
“Like, I do think we'll get to a place where we have this kind of like synchronous conversational agent kicking off kind of like longer running asynchronous agents in the background.”
Chase: Basically all AI agents will write code
“You know, if agents never write any code, then okay, maybe they're not useful, but I think it's trending where Basically all agents will write code, so that's a very interesting piece, I think.”
Chase: AI agents will clearly be long-running and stateful
“So I think, like, pretty clearly agents will be long-running and stateful, and so I think we have a deployments product. I think a lot of the I think deployments products that let you build long-running stateful things will be kind of, like, interesting no mat…”
Developers are buying Mac Minis as primitive physical sandboxes for AI agents
“That's why people are buying Mac minis as a, you know primitive way of sandboxing them and keeping them in a contained environment.”
LangChain sees a 50-50 split in how developers deploy agent sandboxes
“In practice, we see people doing about fifty-fifty between each of these.”
Harrison Chase: AI evals and prompt optimization are closely tied, unlike memory
“I guess evals and prompt optimization are pretty closely tied, but like evals and memory are actually not at all tied.”
Chase: LangChain does not maintain a one-year product roadmap
“I don't think we have a one year roadmap.”
Chase: AI builder differentiation lies in domain knowledge, not harness infrastructure
“I think a lot of the differentiation is in like the instructions and the tools and the skills and that basically, yeah, knowledge of how to do a process that you encode into natural language and give the agent and then the tools and the skills that you let it …”