Graham Neubig
Associate Professor, Carnegie Mellon University. On 1 show, 1 appearance. The Shows tab opens the full record on each.
academicscientistfounderexecutive@gneubig ↗phontron.com ↗
Graham Neubig is an Associate Professor at Carnegie Mellon University's Language Technologies Institute and the co-founder and Chief Scientist at All Hands AI. He directs the NeuLab, focusing on natural language processing, and is a creator of the open-source autonomous AI coding agent platform OpenHands.
1shows
1appearances
25statements
3resolved
3supported
0contradicted
100%fully supported
11said about them ↓ Everything Graham Neubig said on any show that made the record, most notable first. Each card names its show and opens the statement there.
Neubig: Anthropic's MCP Duplicates Existing APIs With Little Added Value
“We already have an API for GitHub. So why do we need an MCP for GitHub, right? You know, like GitHub has an API. The GitHub API is evolving. We can look up the GitHub API documentation. So it seems like kind of duplicated a little bit. And also they have a set…”
Neubig: Arbitrary Python Execution Beats Individual Agent Tool Calls
“And the method that we adopt in open hands instead is we provide these tools, but we provide them by just giving a coding agent the ability to call arbitrary Python code. And in the arbitrary Python code, it can call these tools. We expose these tools as APIs …”
Neubig: SWE-bench Scores Are Inflated By Training Data Contamination
“Sweebench is on popular open source repos and all of these popular open source repos were included in the training data for all of the language models. And so, the language models already know these repos. In some cases, the language models already know the in…”
Neubig: GPT Loops On Errors While Claude Tries New Approaches
“So, like, GPT doesn't have very good air recovery ability. And so, because of this, it will go into loops and do the same thing over and over and over again, whereas Claude does not do this.”
Neubig: Claude Is The Best Agent Model, Open Models Lag Behind
“I still am under the impression that Claude is the best. The other closed models are, you know, not quite as good, and then the open models are a little bit behind that.”
Neubig: Single Agents Adapt Better Than Rigid Multi-Agent Systems
“If you have a really, really good instruction following agent it will follow the instructions as long as things are working according to your plan, but let's say you need to deviate from your plan, you still have the flexibility to do this, and if you do expli…”
Neubig: Every Major LLM Trainer Will Focus On Agents By Mid-2025
“My prediction is every large LM trainer will be focusing on training models as agents. So every large language model will be a better agent model. By mid 20, 25.”
Neubig: Agents Solve 80 To 90 Percent Of Tasks With Feedback
“Actual ability of models is maybe closer to 30 to 40%. So 30 to 40% of the things that I want an agent to solve on my own repos, it just solves without any human intervention. 80 to 90% it can solve without me opening an IDE, but I need to give it feedback.”
Neubig: Coding Agents Should Use Standard Developer Tools, Not Custom Protocols
“We're already developing things for programmers, you know, How is an agent different from a programmer? And it is different, obviously, you know, like agents are different from programmers, but they're not that different at this point, so we can kind of intera…”
Neubig: RAG From Natural Language To Code Does Not Work Well
“At least in my experience, rag from language to code doesn't work super well.”
Neubig: Browsers, Terminals, And Code Editors Constitute The Core Agent Toolset
“Let's say I gave you a web browser and a terminal or a file system and the ability to edit text or code. What could you do with that? Everything. Yeah, probably a lot of things. This is like 99% of my, you know, daily daily life, I guess when I'm working. So I…”
Neubig: OpenHands Equips Its Agent With Only Five Or Six Tools
“We're kind of extreme. And we're only giving the agent five tools or maybe six tools.”
Neubig: Coding Agents Excel At Using The GitHub API Directly
“The agents are super good at using the GitHub API also, so they can do, you know, things on GitHub, like finding all of the, you know, comments on your issues or checking GitHub actions and stuff.”
Neubig: Nobody Has A Good Answer For Human-Agent Interface Design
“I don't think anybody has a good answer to this, and I don't think we have a good answer to this”
Neubig: OpenHands Uses A Single Prompt With No Multi-Agent Systems
“So in open hands, we do very light planning. We have a single prompt. We don't have any multi agent systems.”
Neubig: High Agent Running Costs Will Plunge Within Six Months
“Right now, actually, agents are somewhat expensive to run in some cases, but I expect that that won't last six months. I bet we'll have much better agent models in six months.”
Neubig: AI Agents Including Claude Are Ineffective At Asking For Help
“I think it, my impression is that agents are not very good at asking for help, even Claude. So like when they ask for help, they'll ask for help when they don't need it and then won't ask for help when they do need it.”
Neubig: AI Models Are Poor At Pixel-Based Web Navigation
“The first way is this, the simplest way and the newest way, but it doesn't work very well, which is you take a screenshot of the website and then you click on a particular pixel value on the website and like models are not very good at that at the moment.
Like…”
Neubig: Hybrid Screenshot And Text Summaries Work Best For Web Agents
“And then the third way is kind of a hybrid where you present the screenshot, but you also present like a textual summary of the output.
And that's the one that I think will probably work best.”
Neubig: Inadequate Information Gathering Is The Biggest Agent Failure Mode
“So I think actually probably the biggest thing that it fails at is. Or that our agent plus Claude fails at is insufficient information gathering before trying to solve the task, and so if you provide all, if you provide instructions that it should do informati…”
Neubig: Most APIs Lack The Fine-Grained Authentication Needed For Agents
“For other things, they're totally not prepared to give that sort of fine-grained control. Like, most APIs don't have something like a fine-grained authentication token, and that goes into my, like, comment that we're gonna need to prepare the world for agents,…”
Neubig: Agent Workflow Memory Boosts WebArena Performance By 22.5 Percent
“And this we demonstrated that this leads to a 22.5% increase on web arena After 40 examples.”
Neubig: All Hands AI Is Releasing A Unified Agent Benchmark
“We don't have benchmarks that test whether agents can code and do web navigation. But we're working on that and hoping to release something in the next week or two.”
Neubig: Web Agents Achieve Much Higher Accuracy Using APIs Over UIs
“We had a paper on API based web agents. And basically what we showed is if you take a web agent and the agent interacts not with a website, but with APIs, the accuracy goes way up just because APIs are way easier to interact with.”
The other half of the tape: Graham Neubig's own voice is left out of every number here. Other people bring the name up 10 times in 4 episodes across the shows. 1 statement on the record names them. every mention, with the transcript →
Who brings them up most Shawn Wang 5Swyx (Marcos Swix) 4
Statements about Graham Neubig, by other people (1)
Every mention by year
tap a year for its mentions
Latent Space 10
2025 8 mentions in 2 episodes 4 per episode
2024 2 mentions in 2 episodes 1 per episode
One line per show, most statements first. The link opens Graham's full record on that show: the calibration, argument clarity, speaking style and every statement made there.
| Show | Role there | Eps | Statements | Record | |
|---|---|---|---|---|---|
| LATENT SPACELEDGER | Associate Professor, Carnegie Mellon University | 1 | 25 | 100% 3/3 | full record on Latent Space → |