May 24, 2023 · 54m · no-priors
No Priors Ep. 18 | With Kevin Scott, CTO of Microsoft
0:00 / 0:00
▶ Watch on YouTube →
gold bands on the timeline = statements, start to end. Hover to read, click to jump. CC turns on captions
In this episode of No Priors, Microsoft CTO Kevin Scott joins Sarah Guo and Elad Gil to discuss Microsoft's foundational AI bets, the engineering architecture of the Copilot stack, and the societal impacts of generative models on labor, education, and governance.
How this conversation actually went
Every chapter scored 0–10 on four independent dynamics. Hover any point for the reasoning behind the score. The hosts hold 13.7% of the talking time here. How this is scored →
The hosts as informed peer 4.1 Guest teaching 3.2 Guest disagreement 0.8 The hosts pushing back 0.3
speaking balance: gold is the hosts, purple is the guest (3 minute bins)
Sharpest disagreement ▶ 32:50 Kevin dismisses superficial LLM integration as 'fairy dust'
Kevin forcefully argues against lazy adoption strategies, criticizing companies that try to sprinkle LLM fairy dust on legacy software rather than solving hard, non-obvious problems.
Hardest push from the hosts ▶ 14:50 Elad questions the non-obvious timing of the OpenAI betElad challenges Kevin on why Microsoft committed massive capital to OpenAI during the GPT-2 era before scaling was proven, pressing on traditional buy-versus-build alternatives.
Biggest teaching moment ▶ 31:00 Kevin reframes product strategy around impossible-to-hard shiftsKevin instructs founders and enterprise leaders on product design, explaining that raw models and infrastructure are not products and warning that easy initial use cases rarely become durable businesses.
The host holds their own ▶ 39:37 Sarah defines Benjamin Bloom's two-sigma tutoring studySarah demonstrates sharp domain recall by jumping in to precisely define Benjamin Bloom's educational study, citing its 98th percentile mastery and variance reduction findings.
the scores for every segment, with the reasoning behind each
| Chapter | Topic | The hosts as informed peer | Guest teaching | Guest disagreement | The hosts pushing back | Why |
|---|---|---|---|---|---|---|
| Kevin Scott's Journey from Rural Virginia to Engineering Leadership | 2 | 1 | 0 | 0 | Sarah opens the interview asking Kevin for his personal backstory. Kevin delivers an extended personal monologue on his transition from rural Virginia to compiler research and early Google engineering leadership. | |
| Academic Cultures in Tech: Early Google Compared to OpenAI | 4 | 3 | 0 | 0 | Elad draws a comparison between early Google's academic density and OpenAI's hiring culture. Kevin validates the observation and explains how early transfer learning breakthroughs convinced him to concentrate Microsoft's GPU budget. | |
| Strategic Rationale and Conviction Behind the OpenAI Partnership | 5 | 4 | 1 | 1 | Elad probes why Microsoft invested in OpenAI during the non-obvious GPT-2 era rather than building internally. Kevin explains Satya Nadella's 'no regrets' investing framework and empirical compute scaling laws. | |
| Building Hyperscale AI Supercomputers with Nvidia | 4 | 3 | 0 | 0 | Elad asks about supercomputing infrastructure and the co-development with Nvidia. Kevin breaks down the timeline of the first supercomputing cluster built for GPT-3 in 2019 and recent Hopper deployments. | |
| The Coexistence of Open Source and Proprietary AI Models | 5 | 5 | 2 | 1 | Elad asks about open source versus proprietary models and B2B enterprise AI architectures. Kevin pushes back against framing open vs closed models as a binary choice, and walks through Microsoft's Copilot stack including RAG, orchestration, and meta-prompts. | |
| Rapid Research Realignment and the Disorienting Pace of AI | 4 | 2 | 0 | 0 | The hosts and guest reflect on the disorientation caused by the rapid pace of AI breakthroughs since ChatGPT's release. Kevin observes that legacy ML practitioners often struggle more with the paradigm shift than new founders. | |
| Product Strategy: Tackling Problems from Impossible to Hard | 3 | 5 | 2 | 0 | Sarah asks for organizational product advice for enterprises adopting AI. Kevin warns against treating raw models as products or adding superficial LLM features, contrasting transient 'impossible-to-easy' apps with enduring 'impossible-to-hard' platforms. | |
| Democratizing AI and Opportunities for Public Good | 6 | 3 | 1 | 0 | Sarah asks about Kevin's book on AI democratization. When Kevin references Sal Khan and the two-sigma tutoring problem, Sarah interjects with domain expertise defining Benjamin Bloom's original study metrics. | |
| Future Careers, Physical Labor, and Human-Centric Value | 4 | 4 | 2 | 1 | Elad asks what career advice to give kids over a 20-year horizon given cognitive automation. Kevin highlights durable physical and craft labor (surgeons, machinists) and argues humans fundamentally demand human-centered storytelling, rejecting leisure society predictions. | |
| Emerging AI Frontiers: Multimodal Models and Trillion-Dollar Startups | 4 | 2 | 0 | 0 | Elad prompts Kevin on upcoming technological frontiers. Kevin forecasts foundation model rollouts, multimodal expansion with GPT-V, the founding of the next trillion-dollar startup, and regulatory safety frameworks. |
Statements from this episode (26)
Scott: Early Google hiring lacked direction, leading engineers to zero-impact projects
“We were hiring these brilliant, brilliant people at the time, and the way that we did hiring was kind of crazy. It's like, all right, well, if you're smart, just come work here. And like, we have no idea like what exactly it is you're going to do. And you like…”
Kevin Scott: OpenAI's size and energy resemble early Google
“When I go sit In OpenAI, it really reminds me of early Google days, and it's about the same size Google was when I joined, and so, like, I couldn't figure it out for a while, and I was like, wow, this is, like, really giving me, like, you know, early Google no…”
Gil: OpenAI Is Hiring String Theorists into CS Roles Like Early Google
“It's the first time I've seen, like, string theorists getting hired again. Yeah. Into computer science roles. Yeah. You know, since Google days.”
Scott seized Microsoft's entire GPU budget to stop peanut buttering resources
“At one point I seized the entire GPU budgets for the whole company. And I was like, we will no longer peanut butter these resources around. Like we will focus them because it's all capital intensive. It's like, we will just allocate these things to things wher…”
Kevin Scott: OpenAI Was the Highest-Ambition AI Partner Available
“And when we looked around, like OpenAI was clearly the highest ambition partner that was in the field, you know, and I think still their ambition is just breathtaking in what it is that they're trying to accomplish.”
Kevin Scott: Microsoft Had Significant Internal Dissent Over OpenAI Deal
“So there was a huge diversity of opinions inside of the company on the wisdom of doing doing this.”
Scott predicts the AI industry will never have enough compute capacity
“I just had super, super high confidence that we were never going to get to the point where we're like, all right, we got enough compute.”
Scott: Microsoft built the AI supercomputer that trained GPT-3 in 2019
“So we built our, the first thing that we called an AI supercomputer. I think we started working on it in 2019 and we deployed it at the end of that year. And it was the computing environment that GPT three was trained on”
Scott: Microsoft Copilots use a portfolio of AI models
“If you look at things like Bing chat or Microsoft three 65 copilot or GitHub copilot, you end up using a portfolio of models to do the work and like you use it for performance and cost optimization reasons, and you use it for you know, just sort of precision a…”
Scott: AI will create value through assistive tools, not full autonomy
“We started with this assumption that AI is going to be a platform and the way that people are going to most usefully
Make or the way that people are going to make most use of the platform is by building tools that assist people with jobs. So it's like less abo…”
Scott: No single company can build all useful AI platform tools
“We'll build some of this stuff and like the community is going to build a tremendous amount of it because like there's never been a platform or ecosystem where one company builds all of the useful things. Like that's just nonsense. Like it's just never happene…”
ChatGPT launched as a 10-month-old model with RLHF as a practice test
“ChatGPT was a 10 month old model with a little bit of RLHF on top of it. And, you know, like by, you know, admission, like You know, not a beautiful user interface. It was just sort of a way to get something out there because you know, you needed some practice…”
Scott: Generative AI transition is harder for ML veterans than new entrepreneurs
“I think it's honestly harder for some, Machine learning people than it is you know, for like a brand new entrepreneur who's, you know, just looking for an interesting thing to go do because it is a very different way for a machine learning team to do its work.…”
Kevin Scott: AI Models and Infrastructure Are Not Products
“Models aren't products and infrastructure isn't a product.”
Kevin Scott: The Best Products Move Problems from Impossible to Hard
“Probably the place where the most interesting products are where you've made the phase change from impossible to hard. So like something that like literally you couldn't do at all before this technology exists has become hard now because like the things that h…”
Scott: A 6-month ML project from 2003 takes a high schooler four hours today
“The first project that I did, which was you know, like a machine learning classifier thing in 2003 2003, 2004. Like that was, you know, stacks of like super technical, you know, research papers and, you know this elements of statistical machine learning, you k…”
Scott: AI is the only realistic way to deliver universal 1-on-1 tutoring
“Every learner in the world, ah, deserves to have access to that individualized, high-quality instruction at no cost. Which seems like a reasonable thing. And then when you think about how you go realize that in the world, like the only way that you can realist…”
Scott: Tech has a cognitive exponential today, not a robotics exponential
“We do not have a, Yeah. Sort of robotics exponentials right now. Like we've got a cognitive exponential.”
Scott: Society will probably need many more physical-world jobs than today
“So like I think all of the, like the world is just sort of full of these jobs where you know, really, you know, affecting change on a physical system, like doing something in the physical world, like all of those things like we will need probably many, many mo…”
Scott: Society must rebuild entire power generation and distribution system
“Especially because we're going to have to rebuild our entire power generation and distribution system, like in our children's lifetimes.”
Scott: Past predictions of four-hour leisure workweeks were bull crap
“All of the, you know, the industrial revolution predictions about, you know, four hour work weeks and we're all going to live lives of leisure is bull crap. Yeah, like just hasn't happened.”
Scott: Microsoft software interactions will fundamentally change by the end of 2023
“It will touch all of Microsoft's product portfolio. Like the way that you will interact with our software will be substantially different by the end of this calendar year than it was coming in.”
Scott and Altman suspect the next trillion-dollar company is founded in 2023
“You know, one of the things that Sam Altman and I have talked about a lot is like, I suspect that this year, like the next trillion dollar company gets founded.”
Scott: OpenAI will release GPT-4 Vision to wide distribution soon
“OpenAI doesn't have GPTV and wide distribution, but, like, it'll get to wide distribution at some point in the not too distant future, and so, like, you'll have these, like, very powerful multimodal models”
Scott: AI Governance and Safety Should Focus Primarily on Deployments
“I think most of the stuff that people ought to be thinking about is deployments, making sure that as you deploy the technology, getting the requirements and the expectations right there is the most important thing.”
Scott is not worried about major AI competitors behaving unsafely
“The thing that I will say is we fiercely compete with a whole bunch of these folks. But, like, one of the things that I don't do is, like, look at any of those companies that you just named and, like, worry that they're going to do something that like, is, lik…”