Harrison Chase says TypeScript yields better LLM tool-calling performance than JSON
Harrison Chase · Language Agents: From Reasoning to Acting — with Shunyu Yao of OpenAI, Harrison Chase of LangGraph · Sep 27, 2024 · at 46:42
Harrison Chase (co-founder and CEO of LangChain) discusses whether developer interfaces and tool definitions should be optimized specifically for language models.
0:00 / 0:06exact quote · 6.5s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“I saw some paper that used TypeScript notation instead of JSON notation for tool calling and it got a lot better performance.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Harrison Chase
Chase: Nobody in the AI industry knows how to properly solve memory.
“I don't think anyone knows how to deal with memory, and so I think all these different approaches are...”
Harrison Chase says coding AI agents are not yet a proven success
“There's a bunch of people doing coding stuff.
We've already talked about that.
I think that's a little bit, I wouldn't say that's a success yet, but there's a lot of excitement and stuff there.”
Harrison Chase says production AI agents rely on three main defaults
“And there's such a long tail of other ones, but in practice, like, when people go to production, they generally have their own tools, or maybe one of those three, maybe some other ones, but, like, very, very few other ones.”
Chase: LangChain and other frameworks lack off-the-shelf Reflexion implementations
“I don't think we have like an off the shelf kind of like implementation of reflection and kind of like the general sense. I think the concepts like absolutely we see used in different kind of like specific cognitive architectures, but I don't think we have one…”
Harrison Chase says ReAct is the most popular agent prompting framework
“I would say like reacts probably like the most popular. I think there's aspects of reflection that Get used. Tree of thought, probably like the least so.”
Chase: Few-shot prompting works better than detailed instructions for agent trajectories
“I'm pretty bullish on it, to be honest, for a few reasons. Like, one, I think it can maybe help for more complex things, but then also, two, like, it's a form of prompting, and prompting is just Communicating with the model what you want it to do. And sometime…”