Nadella: Many open-weight models score well on benchmarks but fail in practice
Satya Nadella · Satya Nadella on AI: @NoPriorsPodcast x Latent Space Crossover Special at Microsoft Build 2026 · Jun 3, 2026 · at 3:11
Satya Nadella, CEO of Microsoft, discusses model pre-training data quality and the limitations of publicly benchmarked open-weight models during Microsoft Build.
0:00 / 0:07exact quote · 7.7s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“In fact, that's one of the challenges of a lot of the open rate models is they look great on one benchmark or two, but they're not great on practice.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Satya Nadella
Nadella: Public AI evals are maxed out; every company will use private evals
“Most importantly, you'll have private evals, because we know all the evals out there are good, interesting, but they're not really that critical. At this point, because they're all can be maxed. And so the point is each company will have its own private eval.”
Nadella: Managing multi-agent coding requires completely rebuilding IDE interfaces beyond chat
“Coding has worked so well that we now have to rebuild the IDE, right? I mean, it's kind of nuts to see what we launched is like, oh my God, I have these hundred agent sessions. I, the cognitive load, it transfers back to me as a human is so excessive that now …”
Nadella: In six months users will routinely have AI autopilots working overnight
“I'm positive that six months from now, we'll all be saying, oh, wow, like all through night, the night, there was a bunch of stuff that all these autopilots that I have working on my behalf with my delegated authority, so to speak, right? I can sort of given e…”
Nadella says private evals and execution traces are a company's biggest IP
“So that's why I would say every company having private evals may be the biggest IP, right? I think about it like what's that private eval that you can then use even a frontier model to hill climb on and not leak the traces. Maybe one of the biggest drivers of …”
Nadella predicts AI agent knowledge traces will become corporate balance sheet assets
“Which is when a company says it should in fact go onto the balance sheet is how I think about it, right? That's so, in fact, there may be like human capital was never possible to go put on a balance sheet because you didn't know how to capture the tacit knowle…”