Opinion
Chase: Open-source models still lag behind Claude 3 and GPT-4
“Like there's, I think we see increasingly interest in open source, but the reasoning abilities are still just like lagging behind Cloud three or GPT four. And I think like for a lot of the applications that it kind of, it probably depends on the types of appli…”
Prediction Not checkable as stated
Chase: Long context windows will not replace chaining and AI agents
“There are also things where it requires iterations. You need to like decide what to do, interact with the environment, get that back. So this whole idea of chaining and agents, I don't like,
That's less around context windows and more around interacting with t…”
Insight
Chase: Needle-in-a-haystack benchmarks fail to represent real RAG reasoning
“That, that actually really doesn't reflect a lot of RAG use cases in, in my opinion, because like that's the needle in the haystack is like, okay, given this long context, can I find a single information point? But oftentimes RAG is about seeing multiple infor…”
Opinion
Chase: AI agent memory is extremely nascent and lacks interesting developments
“I feel it's like a field that's just, like, super, super nascent. Like, I don't, I actually am underwhelmed at the amount of, like, really interesting stuff that's going on there.”
Insight
Chase: Successful production AI agents operate as controlled state machines
“And I think the things that we see making it into production and informed a lot of the development of laying graph is, or is something in the middle where it's like this controlled state machine type thing.”
Assertion Not checkable as stated
Chase: Developers only implement model fine-tuning after reaching critical scale
“We see people experimenting with it. I think the only real place where they're doing it is when they've reached like really critical scale which I still don't think is that many applications to date.”
Insight
Chase: AI agents fundamentally require cyclical graphs, not linear architectures
“So, you know, all these agents are basically running an LLM in a loop. You need cycles and so lane graph helps with that.”
Insight
Chase: Working AI agents require hardcoded domain structure, not LLM autonomy
“I think when we see people building agents that work right now, it's often breaking it down into a bunch of smaller components and kind of like imparting their domain knowledge about how information should Flow through these components. Because I think the ele…”
Insight
Chase: Few-shot prompting effectively builds procedural memory for AI agents
“So on the procedural side, I think the main thing that we see people doing and that we think is pretty effective is few-shot prompting and maybe fine-tuning for how to use for how to use tools, because that's basically what it comes down to. What's the right w…”
Opinion
Chase: Passive background insight extraction will drive AI personalization memory
“I also think one thing that I'm bullish on is a more kind of, like passive background process that kind of looks at conversations and almost, like, extracts insights. And then you can use those insights in kind of, like, future conversations.”
Prediction Not checkable as stated
Chase: LLM prompts will likely converge as models become more intelligent
“I do think
the prompts will probably start to converge in the sense that if you think the models are getting more and more intelligent than like, hopefully these small idiosyncratic sees don't matter as much.”
Opinion
Chase: New AI startups should build applications leveraging long-term memory
“If I wasn't doing LinkedIn, if I was starting a company right now, I'd probably start something at the application layer, and it would probably be something that really takes advantage of, like, long-term memory.”
Insight
Chase: Few-shot example datasets are faster and cheaper than model fine-tuning
“Building up few shot example data sets and really using those. I think it's much faster and cheaper than fine tuning models. It's easier to do than trying to like. Programmatically change the prompt in some way.”
Insight
Chase: Code execution gives AI coding agents a direct feedback loop
“But those type like coding, coding problems in general, we see a lot of people working on. I think there's a really nice feedback loop that you can get by just like executing the code and seeing if it works.”
