LangChain CEO Harrison Chase discusses different technical approaches to agent memory with Elad Gil on No Priors.
Opinion
Chase: Open-source models still lag behind Claude 3 and GPT-4
“Like there's, I think we see increasingly interest in open source, but the reasoning abilities are still just like lagging behind Cloud three or GPT four. And I think like for a lot of the applications that it kind of, it probably depends on the types of appli…”
Prediction Not checkable as stated
Chase: Long context windows will not replace chaining and AI agents
“There are also things where it requires iterations. You need to like decide what to do, interact with the environment, get that back. So this whole idea of chaining and agents, I don't like,
That's less around context windows and more around interacting with t…”
Insight
Chase: Needle-in-a-haystack benchmarks fail to represent real RAG reasoning
“That, that actually really doesn't reflect a lot of RAG use cases in, in my opinion, because like that's the needle in the haystack is like, okay, given this long context, can I find a single information point? But oftentimes RAG is about seeing multiple infor…”
Opinion
Chase: AI agent memory is extremely nascent and lacks interesting developments
“I feel it's like a field that's just, like, super, super nascent. Like, I don't, I actually am underwhelmed at the amount of, like, really interesting stuff that's going on there.”
Insight
Chase: Successful production AI agents operate as controlled state machines
“And I think the things that we see making it into production and informed a lot of the development of laying graph is, or is something in the middle where it's like this controlled state machine type thing.”
Assertion Not checkable as stated
Chase: Developers only implement model fine-tuning after reaching critical scale
“We see people experimenting with it. I think the only real place where they're doing it is when they've reached like really critical scale which I still don't think is that many applications to date.”