Ubl: Vercel publishes evals to influence OpenAI and Anthropic models
Malte Ubl · The Great Evals Debate — Ankur Goyal & Malte Ubl · Dec 7, 2025 · at 28:12
Vercel CTO Malte Ubl explains the strategy behind releasing open-source Next.js evaluation suites to steer frontier model training.
0:00 / 0:10exact quote · 10.5s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“I'm Vercel and I publish at Eval. That I want OpenAI and Anthropic to use to make sure when they ship the next model that they're better at the stuff that I care about.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Malte Ubl
Ubl: Anthropic's Boris Power Vibe-Codes From a Position of Privilege
“I think he comes from a particular position of extreme unusual privilege, which is that he works at an AI lab where like people in the office next door are like writing the evals and are like training the model like every day in exactly that way.”
Malte Ubl: When Vibes and Eval Data Disagree, Vibes Are Right
“I think that the common quip that if the vibes and the data disagree, the vibes are probably right. It's true, right? So you have to like, be honest with yourself, like, do they agree and kind of iterate On them over time.”
Ubl: Vercel's Composite Models Are Faster Than Agentic Loops
“Basically what we do is we have this like composite model architecture. We run the frontier model and then we run the fine tune model after to fix its errors. That doesn't perform better than an agentic loop, but it's orders of magnitude faster, right?”
Ubl: Vercel leads the truly open-source deployment business model
“Vercel maybe has not invented this, but certainly kind of is the most successful at a model where you say, okay, I have this software library and it's truly open source. Everyone can run it. It comes with like adapters for every place on the planet, and that m…”
Future app platforms must extract auth and authorization completely
“Auth cannot be part of the app, because they're not going to get that right, right? So, Auth has to be extracted from the app. In fact, which data you can see, they also cannot be under control of the app, because again, you're going to get it wrong, right? So…”