Aug 29, 2024 · 42m · no-priors
No Priors Ep. 78 | With AWS CEO Matt Garman
0:00 / 0:00
▶ Watch on YouTube →
gold bands on the timeline = statements, start to end. Hover to read, click to jump. CC turns on captions
AWS CEO Matt Garman discusses the evolution of cloud computing, outlining AWS's architectural philosophy, custom silicon investments, and generative AI strategy. He explains how Amazon Bedrock, modular infrastructure primitives, and serverless inference will drive the next wave of enterprise modernization and startup innovation.
How this conversation actually went
Every chapter scored 0–10 on four independent dynamics. Hover any point for the reasoning behind the score. The hosts hold 24.7% of the talking time here. How this is scored →
The hosts as informed peer 5.4 Guest teaching 3.6 Guest disagreement 1.8 The hosts pushing back 1.3
speaking balance: gold is the hosts, purple is the guest (3 minute bins)
Sharpest disagreement ▶ 15:07 Garman refutes colo return premise
Garman directly corrects Sarah's framing that GenAI puts developers back in the on-prem colo DGX era, explaining that modern liquid cooling and cluster sizes make cloud hosting far more practical.
Hardest push from the hosts ▶ 36:40 Sarah challenges platform consolidation assumptionsSarah presses Garman on whether current enterprise AI platform building reflects genuine divergence or another cycle of DIY platform skepticism before eventual managed cloud adoption.
Biggest teaching moment ▶ 11:39 Garman explains intelligence agency validationGarman educates the hosts on how winning the secret intelligence community contract and the subsequent public IBM lawsuit provided the decisive technical credibility stamp for AWS in enterprise IT.
The host holds their own ▶ 5:59 Elad draws on early startup and Twitter scaling historyElad demonstrates domain expertise by detailing his personal experience using AWS in 2007 and contrasting server setup friction with the scaling issues Twitter later faced.
the scores for every segment, with the reasoning behind each
| Chapter | Topic | The hosts as informed peer | Guest teaching | Guest disagreement | The hosts pushing back | Why |
|---|---|---|---|---|---|---|
| Episode Preview: Matt Garman on AWS Philosophy | 4 | 2 | 1 | 1 | Elad Gil opens by grounding the interview in Matt Garman's early start at AWS as an intern. Garman explains the early founding days and vision under Andy Jassy in an open, collegial manner. | |
| AWS Founding Philosophy: Building Blocks Over Complex Paradigms | 6 | 4 | 2 | 2 | Elad shares his first-hand perspective as an early 2007 startup customer on AWS and Twitter's later infrastructure challenges. Garman details the building-block philosophy contrasted with competitors' prescriptive paradigms. | |
| Overcoming Enterprise Skepticism and the Intelligence Community Contract | 6 | 5 | 2 | 1 | Sarah and Elad highlight enterprise skepticism and historical financial metrics. Garman recounts winning over Wall Street banks and the landmark secret CIA/intelligence community deal that IBM contested. | |
| Blockers to Modernizing Remaining On-Premises Workloads | 5 | 4 | 3 | 2 | Sarah probes on whether GenAI reverts computing back to on-prem colo DGX setups, and Garman immediately pushes back, noting most buy cloud instances and clusters. He walks through the three foundational hypotheses behind AWS Bedrock. | |
| Model Ecosystem Strategy and Commitment to Open Source | 6 | 4 | 2 | 1 | Sarah inquires about first-party model strategy versus competitor lock-in, citing Alyssa Henry. Garman breaks down Titan, Claude, Llama 3.1 open weights, and Amazon's philosophy against vendor lock-in. | |
| Key AI Infrastructure Primitives: RAG, Guardrails, and Agents | 5 | 4 | 1 | 1 | Elad asks about emerging AI primitives beyond core models, such as RAG, evals, and agentic workflows. Garman details Bedrock's knowledge bases, guardrails, and ecosystem partnerships with Scale AI and LangChain. | |
| Managing Compute Capacity, Custom Silicon, and Power Infrastructure | 5 | 4 | 2 | 1 | Elad and Sarah explore physical datacenter scaling, Trainium, power acquisition, and long-term capital commitments. Garman explains balancing fungible power/land investments against near-term hardware supply chain bottlenecks. | |
| Infrastructure Advice for AI Startups and Value Capture | 6 | 3 | 2 | 2 | Sarah and Elad examine value capture across the AI stack, noting value accrual to application layers rather than pure compute. Garman offers advice on startup runway discipline and monetizing practical enterprise workflows. | |
| Enterprise DIY AI Platforms vs. Managed Cloud Infrastructure | 6 | 3 | 2 | 1 | Sarah questions whether enterprises will build DIY internal AI platforms. Garman explains why running custom GPU management doesn't deliver core enterprise value and predicts managed abstractions like SageMaker will dominate. | |
| The Three-to-Five Year Vision: Inference as a Core Primitive | 5 | 3 | 1 | 1 | Elad and Sarah ask about AWS's 3-to-5 year trajectory and persistent startup commitment. Garman outlines inference evolving into a fundamental compute primitive alongside storage and database services. |
Statements from this episode (19)
Garman says Google and Microsoft forced new app paradigms in early cloud
“I think if you look at some of those others like Google and later Microsoft they kind of went at this space for first, we were the first ones out there that had anything like this, but even soon after that, I think they went after the space, like that was goin…”
Jeff Bezos mandated internal move to modular services in 2003
“If you scroll back all the way to 2003 or so, Jeff Bezos basically mandated across the company that in order to move from a big monolithic stack that wasn't going to scale anymore for Amazon, we had to move everything to services.”
AWS grew revenue from $500M in 2010 to $90B in 2023
“I think you guys went from something like five hundred million or so in 2010 to about ninety billion last year in terms of revenue for AWS.”
Garman estimates 85% of enterprise workloads still run on-premise
“Now that we're at a hundred billion run rate you look at, you still go out there, and I think, 85% of workloads are still running on-prem today, by most estimation, somewhere in that range, you know, pick your number, whether it's 80 to 90, whatever it is, lik…”
Secret intelligence contract win against IBM marked AWS inflection point
“One of the big Inflection points we saw is we went after the intelligence agencies for the U S government, and we won that contract and it was secret. And it, you know, we pushed really hard to go in that. It was against all the incumbents, HPs and IBMs and Or…”
AWS is building an easy button to migrate legacy mainframes
“If I had an easy button and by the way, we're trying to build an easy button, but if I had an easy button that would just migrate mainframes to a modern cloud architecture today, almost everyone will push that button, but it doesn't quite exist today.”
Garman predicts liquid cooling will make on-prem AI clusters too difficult
“Increasingly, I think that's going to get harder and harder as you move to liquid cooling and larger clusters”
AWS began developing proprietary AI processors five to six years ago
“We've been investing in AI broadly for the last 10 years, and that's why we started five or six years ago investing at the infrastructure layer and building our own processors”
Amazon Titan is by far the most popular embeddings model on Bedrock
“In fact, the Titan embeddings model, Is by far the most popular embeddings model that we have inside of Bedrock today for people that are building search indices and thinking about things like that.”
Garman calls Anthropic Claude models the best performing in the world
“I think people love using Anthropix Claude models. Those are fantastic, and right now, those are the best performing models in the world which is fantastic.”
Adobe built its Firefly AI models entirely on top of AWS
“Where we see folks like Adobe building Firefly is a all built on top of AWS purpose built for the own thing, their own thing that they're building.”
Garman predicts foundation models will command less attention over time
“Today the models is the front and center thing that everybody pays attention to, but I think increasingly it'll become a smaller percentage of the thing that people pay attention to”
Next generative AI leap requires integrating action-taking agentic workflows
“The next generation of and the next step forward and what we can get out of AI systems is going to depend a lot on, How well we can integrate agentic workflows and actually get these AI systems to do things, not just kind of summarize and tell us information.”
Multi-year fab lead times will keep AI hardware supply chains constrained
“Look, I think we're probably going to be in a constrained world for the next little bit of time. Just, you know, that some of these things are, they take time. Like, look how long it takes to build a semiconductor fab. Like, it is, it's not a short lead time, …”
Nvidia runs its own AI training clusters on AWS infrastructure
“NVIDIA actually runs their AI training clusters in AWS, because we actually have the most stable infrastructure of anyone else, and so they actually get the best performance from us”
Inference workloads must dominate training for AI capital investments to pay off
“Inference is, is one of those workloads that today it's, you know, fifty-fifty maybe of training in Inference, but in order for the math to work out, Inference workloads have to dominate, otherwise all this investment in, in these big models isn't really gonna…”
Garman's first startup burned $27 million in 18 months before folding
“My very first startup. We raised, I think at the time was a lot of money. It was twenty seven million dollars. We ran out of money in like 18 months, and then, you know, the 2000 came around, and there wasn't any more funding, and we went out of business.”
Most enterprises will eventually stop building their own AI models
“Most companies are not going to build their own models over time. They might tweak them a little bit, but I think a lot of companies are going to build, are going to use the applications that use the software and the models underneath.”
AI inference will become a core cloud computing primitive like storage
“As you move forward, generative AI honestly becomes one of the compute building blocks that you think about. You're going to need storage, you need compute, you need databases, you need inference, if you will, for your application, largely. And I think that's …”