Everything Harrison Chase said on any show that made the record, most notable first. Each card names its show and opens the statement there.
Chase: Open-source models still lag behind Claude 3 and GPT-4
“Like there's, I think we see increasingly interest in open source, but the reasoning abilities are still just like lagging behind Cloud three or GPT four. And I think like for a lot of the applications that it kind of, it probably depends on the types of appli…”
Chase: Agent harnesses matter more for performance than underlying models
“I, the, so I don't know what happens, but I do know the harness is really, really important. Like, I think this is the thing that matters.”
Chase: Basically all AI agents will write code
“You know, if agents never write any code, then okay, maybe they're not useful, but I think it's trending where Basically all agents will write code, so that's a very interesting piece, I think.”
Chase: Nobody in the AI industry knows how to properly solve memory.
“I don't think anyone knows how to deal with memory, and so I think all these different approaches are...”
Chase: Long context windows will not replace chaining and AI agents
“There are also things where it requires iterations. You need to like decide what to do, interact with the environment, get that back. So this whole idea of chaining and agents, I don't like,
That's less around context windows and more around interacting with t…”
Chase: Needle-in-a-haystack benchmarks fail to represent real RAG reasoning
“That, that actually really doesn't reflect a lot of RAG use cases in, in my opinion, because like that's the needle in the haystack is like, okay, given this long context, can I find a single information point? But oftentimes RAG is about seeing multiple infor…”
Anthropic's Claude Code uses custom harness tools over model-level RL tools
“It doesn't actually use the tools that are RL into the model. So like anthropic models have some like file editing tools. They have a completely different set of tools in, in the actual harness.”
LangChain adds tools for AI agents to autonomously trigger context compaction
“One interesting thing there, actually, that we haven't yet released as of this recording, but will probably be released by the time it comes out, is we actually give the agent a tool to trigger its own compaction.”
Harrison Chase: AI agents will evolve into synchronous interfaces orchestrating asynchronous sub-agents
“Like, I do think we'll get to a place where we have this kind of like synchronous conversational agent kicking off kind of like longer running asynchronous agents in the background.”
Chase: AI builder differentiation lies in domain knowledge, not harness infrastructure
“I think a lot of the differentiation is in like the instructions and the tools and the skills and that basically, yeah, knowledge of how to do a process that you encode into natural language and give the agent and then the tools and the skills that you let it …”
Harrison Chase says coding AI agents are not yet a proven success
“There's a bunch of people doing coding stuff.
We've already talked about that.
I think that's a little bit, I wouldn't say that's a success yet, but there's a lot of excitement and stuff there.”
Chase: AI agent memory is extremely nascent and lacks interesting developments
“I feel it's like a field that's just, like, super, super nascent. Like, I don't, I actually am underwhelmed at the amount of, like, really interesting stuff that's going on there.”
Chase: Successful production AI agents operate as controlled state machines
“And I think the things that we see making it into production and informed a lot of the development of laying graph is, or is something in the middle where it's like this controlled state machine type thing.”
Chase: Developers only implement model fine-tuning after reaching critical scale
“We see people experimenting with it. I think the only real place where they're doing it is when they've reached like really critical scale which I still don't think is that many applications to date.”
Chase: High reliability AI agents require structured, graph-like scaffolding workflows
“So people ended up building scaffolding around the models to make them do things in a more predictable and reliable way. And that's why we at link chain, we built lane graph, which was another framework really aimed at that kind of like. Graph like workflows a…”
Chase: Major AI labs' RL focus on code makes coding agents perform best
“All the big model labs have been RL and code and bash and editing files into those models. And so that is the stuff that works the best.”
Harrison Chase: Most current agent harnesses feature built-in sub-agents and skills
“Most harnesses today have sub agents built in. They have skills built in.”
Chase: Human standard operating procedures should guide AI agent system prompts
“If you have a standard operating procedure for how a human should do things like that, that should influence a lot of what the system prompt is.”
Harrison Chase: Modern AI agent harnesses don't strictly enforce execution plans
“And so now what most things do is they just have that plan in the text file, and the main agent can, like, use that to help guide its actions, but there's nothing that says I'm explicitly doing this step, or I'm explicitly doing another step.”
Harrison Chase: Inter-agent communication is the hardest part of working with AI agents
“Communication is the hardest part of life, by the way, it's the hardest part of startups, hardest part of relationships, hardest part of working with agents is getting them to communicate.”
Chase: Virtual file systems let LLMs manage their own context windows
“And the way that I think about a file system is it basically lets the LLM manage its own context window. So it can decide what to read from files.”
Chase: AI agent context compaction must retain recent messages for flow
“You want to keep around like the last N messages, let's say the last like 10 or so messages, because if you compact everything, it actually like throws it off completely.”
Chase: AI agents will clearly be long-running and stateful
“So I think, like, pretty clearly agents will be long-running and stateful, and so I think we have a deployments product. I think a lot of the I think deployments products that let you build long-running stateful things will be kind of, like, interesting no mat…”
Developers are buying Mac Minis as primitive physical sandboxes for AI agents
“That's why people are buying Mac minis as a, you know primitive way of sandboxing them and keeping them in a contained environment.”