why aren't all 10 resolved? a statement only gets an assessment when the public
record can support or contradict it. opinions and what-ifs never can, and 0 checkable
ones are still open, waiting for their date. predictions held up or didn't;
assertions are supported or contradicted. on every card:
▮▮▮▮▮ certainty ·
▮▮▮▮▮ debate potential. speakers are clickable
Opinion
Ganatra: Existing AI tool-calling benchmarks are not production-ready
“Tool calling has a lot of different evals. You have seen the function tool calling evals. You have multi-function, like, multi-tune tool calling evals. But, like, none of them are actual, like, production quality. Like, none of them are actually something that…”
Assertion Not checkable as stated
Ganatra: 95% of Composio integrations are built and maintained by agents
“So, 95% of all our integrations are completely built using agents, maintained using agents.”
Assertion Not checkable as stated
Ganatra: Poor API documentation is the main source of tool-calling errors
“Is this Salesforce tool getting transformed into the right request payload so that it is hitting the right endpoint in Salesforce, right? And that is like the major source of error today, because a lot of documentations are not great. A lot of like ad cases ar…”
Disclosure
Vaidya: Composio evolves agent skills by analyzing usage patterns
“We are kind of self evolving skills now for agents. So people building or using agents want to like basically make Agents interact with their apps. They can use Composio. We manage all the authentication and user account related stuff for them so that they don…”
Assertion Supported
Ganatra: MCP Lacks Client Post-Processing for Long Context Responses
“MCP let's just say you have a tool that has really long context, a really long sort of response. Currently, if you use MCP for those kinds of tools, it ends up sort of messing up, right? Because let's say I'm trying to fetch like a hundred emails on Gmail and …”
Assertion Not checkable as stated
Ganatra: Over 100,000 developers have built with Composio
“In terms of developers, we have, like, more than 100,000 developers who have been building on top of course and, like, trying our product out.”
Assertion Supported
Ganatra: Server-Controlled Schemas in MCP Complicate Client-Side Customization
“Some of the other things that we have observed is like, hey, maybe I want to sort of change the tool descriptions, but like in MCP, the tool descriptions are controlled by the server itself. And so essentially for me as a developer who's sort of using an MCP, …”
Assertion Supported
Ganatra: Composio supports more than 500 applications
“So we have at this point more than 500 different applications on top of Composio.”
Assertion Not checkable as stated
Ganatra: Composio's meta-tool became highly used after a year of zero adoption
“We built it, like, a year back. Nobody used it for a very long time. Today, it's probably one of the most used tools.”
Disclosure
Karan Vaidya: Composio runs an internal agent to build and evaluate skills
“We have an agent essentially, which builds a lot of these skills and kind of like does a lot of like testing as well as eval. So it's like, you can say like agent as an eval where our internal agent uses these tools and kind of like we figured out where it is …”