Neubig: Anthropic's MCP Duplicates Existing APIs With Little Added Value
Graham Neubig · Best of 2024 in Agents (from #1 on SWE-Bench Full, Prof. Graham Neubig of OpenHands/AllHands) · Dec 25, 2024 · at 41:29
Graham Neubig, co-founder of AllHands AI and CMU professor, answers an audience question about whether OpenHands is adopting Anthropic's Model Context Protocol (MCP).
0:00 / 0:46exact quote · 46.0s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“We already have an API for GitHub. So why do we need an MCP for GitHub, right? You know, like GitHub has an API. The GitHub API is evolving. We can look up the GitHub API documentation. So it seems like kind of duplicated a little bit. And also they have a setting where it's like you have to spin up a server to serve your GitHub. Stuff and you have to spin up a server to serve your like, you know, other stuff. And so I think it makes sense if you really care about like separation of concerns and security and like other things like this. But right now we haven't seen that to have a lot more value than interacting directly with the tools that are already provided.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Graham Neubig
Neubig: Arbitrary Python Execution Beats Individual Agent Tool Calls
“And the method that we adopt in open hands instead is we provide these tools, but we provide them by just giving a coding agent the ability to call arbitrary Python code. And in the arbitrary Python code, it can call these tools. We expose these tools as APIs …”
Neubig: SWE-bench Scores Are Inflated By Training Data Contamination
“Sweebench is on popular open source repos and all of these popular open source repos were included in the training data for all of the language models. And so, the language models already know these repos. In some cases, the language models already know the in…”
Neubig: GPT Loops On Errors While Claude Tries New Approaches
“So, like, GPT doesn't have very good air recovery ability. And so, because of this, it will go into loops and do the same thing over and over and over again, whereas Claude does not do this.”
Neubig: Claude Is The Best Agent Model, Open Models Lag Behind
“I still am under the impression that Claude is the best. The other closed models are, you know, not quite as good, and then the open models are a little bit behind that.”
Neubig: Single Agents Adapt Better Than Rigid Multi-Agent Systems
“If you have a really, really good instruction following agent it will follow the instructions as long as things are working according to your plan, but let's say you need to deviate from your plan, you still have the flexibility to do this, and if you do expli…”