Ensembling
topic on 1 show · 2 statements across 1 episodes
2 statements about Ensembling, every show
Better ensembling typically adds only a few percentage points on SWE-bench
“Ensembling is usually not the kind of thing that just solves a benchmark, just also in other domains, you typically can get like, yeah, and this, and first we bench probably a few percentage points is my guess.”
Ensembling coding agents will transition from a cost to a UX challenge
“I think it will become less of over time as cost will come down. I expect it will become less of a cost question and more of a UX question because with ensembling, well, what we found is that users really want to see what the agent is doing and follow along. A…”