Jul 10, 2024 · 51m · big-technology
Amazon's Longterm AI Vision — With AWS VP Matt Wood
0:00 / 0:00
▶ Watch on YouTube →
gold bands on the timeline = statements, start to end. Hover to read, click to jump. CC turns on captions
In this episode of the Big Technology Podcast, host Alex Kantrowitz interviews AWS VP of AI Products Matt Wood to assess the real-world state of enterprise generative AI, exploring Amazon Bedrock's model choice strategy, the rise of autonomous agents, and the organizational shifts required to scale AI in production.
How this conversation actually went
Every chapter scored 0–10 on four independent dynamics. Hover any point for the reasoning behind the score. Alex holds 23.5% of the talking time here. How this is scored →
Alex as informed peer 5.6 Guest teaching 5.4 Guest disagreement 2.5 Alex pushing back 5.1
speaking balance: gold is Alex, purple is the guest (3 minute bins)
Sharpest disagreement ▶ 22:02 Wood defends the reality of AI agents in production
Wood pushes back against Kantrowitz's assertion that autonomous AI agents remain purely vaporware without production examples.
Hardest push from Alex ▶ 34:29 Kantrowitz challenges domain-specific LLMs with Wharton studyKantrowitz directly cites research showing general frontier models beat BloombergGPT to challenge AWS's specialized model strategy.
Biggest teaching moment ▶ 5:22 Wood deconstructs the Gartner proof-of-concept metricWood contextualizes the 21% proof-of-concept rate by demonstrating that high experimental volume makes a 20% conversion rate exceptionally healthy.
Alex holds their own ▶ 34:29 Kantrowitz deploys empirical benchmark data on BloombergGPTKantrowitz demonstrates technical depth by referencing specific comparative benchmark data from academic research on domain models versus frontier LLMs.
the scores for every segment, with the reasoning behind each
| Chapter | Topic | Alex as informed peer | Guest teaching | Guest disagreement | Alex pushing back | Why |
|---|---|---|---|---|---|---|
| Enterprise AI Adoption in Regulated Industries | 5 | 6 | 2 | 4 | Kantrowitz challenges Wood with Gartner data showing only 21% of enterprise AI proofs-of-concept reach production. Wood reframes the premise by explaining how regulated industries leverage long-standing data governance to adopt AI faster than expected. | |
| Practical Utility vs Hype of Emergent Behaviors | 6 | 6 | 3 | 6 | Kantrowitz pushes back on high compute costs, change management hurdles, and the gap between emergent reasoning claims and mundane document-scanning tasks. Wood counters that a 20% conversion rate is healthy when the experimental denominator is massive. | |
| Transforming Boring Workloads and Scientific Step Functions | 5 | 5 | 2 | 4 | Kantrowitz asks whether AI value will expand beyond unexciting operational tasks. Wood embraces the concept of boring workloads, drawing historical parallels to early AWS cloud adoption and citing computational biology breakthroughs. | |
| Cultural Readiness and Change Management in AI Adoption | 4 | 5 | 1 | 3 | Kantrowitz explores enterprise cultural readiness and employee friction. Wood offers an analogy about alien discovery to illustrate that major technological step functions often feel gradual and incremental. | |
| Philosophical Aside on Extraterrestrial Life | 5 | 5 | 2 | 5 | Kantrowitz asks whether generative AI risks falling into a trough of disillusionment due to Wall Street quarterly pressure. Wood argues we remain at the flat bottom of the technology S-curve rather than an inflection peak. | |
| Real-World Applications of AI Agents with Amazon Q and NinjaTech | 6 | 5 | 4 | 7 | Kantrowitz directly questions whether AI agents actually exist in production today after a year of industry talk. Wood responds with concrete examples including NinjaTech and Amazon Q's automated developer workflows. | |
| The Naming of Amazon Q and Mid-Episode Break | 6 | 5 | 3 | 6 | Kantrowitz pivots to Bedrock's selection, noting criticism that omitting flagship models like GPT-4o and Gemini weakens Amazon's pitch of model choice. Wood defends Bedrock's approach by arguing against Swiss Army knife foundation models. | |
| Specialized Domain Models vs General Frontier LLMs | 8 | 6 | 4 | 7 | Kantrowitz cites Ethan Mollick's Wharton findings showing general GPT-4 outperformed BloombergGPT on financial benchmarks to challenge specialized model utility. Wood counters that domain depth and retrieval grounding matter more than general web-scale training. | |
| Expectations for Next-Generation Frontier Models | 5 | 5 | 1 | 3 | Kantrowitz queries Wood on what to realistically expect from upcoming models like GPT-5 and Claude 4. Wood describes incremental reasoning upgrades combined with proprietary internal organizational grounding. | |
| 18-Month Outlook: Best and Worst Case Scenarios | 6 | 5 | 3 | 6 | Kantrowitz demands clear best- and worst-case scenarios over an 18-month horizon, pressing Wood when he initially gives a corporate answer. Wood highlights plateauing S-curves and lagging enterprise culture as the true downside risks. | |
| Modernizing Amazon Alexa with Large Language Models | 6 | 6 | 2 | 5 | Kantrowitz asks whether Alexa's legacy hard-coded intent mapping necessitates a complete architectural rewrite for LLM integration. Wood explains that Alexa is retaining its deterministic intent mapping while layering generative LLMs on top for dialogue. |
Statements from this episode (20)
Matt Wood: Regulated industries are adopting GenAI faster due to existing data governance
“There is a group which is moving slightly faster than the average, which is somewhat Counterintuitive. And that group is actually the regulated industries. And so it's folks like in financial services and insurance and healthcare and life sciences and manufact…”
Wood: AWS AI and Machine Learning Business Exceeds Multibillion-Dollar ARR
“Bedrock, which is the service that we make available to customers to build generative AI applications. That's one of our fastest growing services ever. And all up, AI machine learning at AWS is already a multi-billion dollar business. In terms of ARR.”
Wood: Generative AI Functions as a Discovery Exhibiting Emergent Reasoning
“It's much more like a discovery than it is an invention. Ah, we discovered that if you build these very sophisticated mathematical models, that there is emergent behavior within them that resembles reasoning, that resembles intelligence.”
Wood: AWS worked with EvolutionaryScale on generative AI drug discovery
“Pfizer or the work we've done with a startup called evolutionary scale to be able to use generative AI to be able to design entirely to entirely new molecules to design entirely new antibodies that are manufacturable that can go on and find new drug targets. L…”
AWS's Wood: Near-Term AI Progress Will Feel Much More Incremental Than Expected
“I suspect that whilst there will be these step function changes over the long period, I think in the shorter term, in the shorter outlook, it's gonna feel a lot more incremental than we're probably used to.”
AWS's Wood: AI Success Requires 50% Cultural and 50% Technical Investment
“I'd actually say it's more like 50% technical, 50% cultural in terms of the weighting of the, ah, elements of investment that are going to be required to be successful.”
AWS's Wood: 25% to 35% of Enterprise Workforce AI-Ready Today, 100% Long-Term
“I would guess if I had to put a number on it, I would say it's probably 25%, 35% in most large-sized enterprises. But over time, you know, if you look three years out, five years out, 10 years out, whatever it might be, with that long-term horizon, my guess is…”
AWS's Wood: Generative AI has not reached its hockey-stick inflection yet
“My guess is that we're, it's probably more likely that we're at the bottom left hand corner. I don't think we've hit the kind of hockey stick inflection point yet. Of what this, ah, what this technology is capable of. It's still very, very, very early.”
Wood: Most enterprises will fine-tune existing models rather than build foundation models
“There isn't one model to kind of rule them all. Each different model has, you know, different sweet spots, and it's my expectation that most customers will invest in not building the foundation models, but will invest in fine tuning and improving those individ…”
AWS's Wood: AI agents will be the apps of the generative AI era
“I think agents have a good chance of being the apps for the generative AI world and the generative AI era. And that as we add more of those and we find ways to orchestrate multiple agents together, and there's already customers that are building multi-agent sy…”
Wood: AI Startup NinjaTech Has Hundreds of Thousands of Monthly Active Users
“I think Ninja Tech is seeing, you know, remarkable growth. They have hundreds of thousands of monthly active users.”
Wood: Amazon Q Code Acceptance Rate Is 35% to 50%
“We've seen some customers get, you know, in terms of just the amount of code that is automatically generated that they accept. It's usually between 35 and 50%. It's higher on Q than any other comparable service”
Wood: Claude 3.5 Haiku Outperforms All Other Models on the Planet
“Claude III.V Haiku outperforms all other models on the planet.”
Wood: Amazon Bedrock Has Tens of Thousands of Active Customers
“Bedrock is our, one of our fastest growing services ever. We have tens of thousands of customers that are using it today.”
AWS's Matt Wood: Web-trained AI models have a six-month shelf life
“Today it looks like models have a shelf life of probably about six months if you're training on kind of open, open web data”
Wood: Smaller specialized models beat frontier models on deep enterprise tasks
“I am absolutely positive that the inverse is also true. That you can find older, smaller, specialized models, ah, that, ah, will, ah, will offer much better, higher quality, lower hallucination results on specific tasks at the depth that most organizations nee…”
AWS's Wood predicts foundation AI models will consolidate to 12-24 major providers
“I think that there's not going to be hundreds of world model providers. I think that there's likely to be maybe a dozen, two dozen, something of that order of magnitude. I think that, you know, Anthropic will be one, Meta will be one, Amazon will be one, there…”
AWS's Wood: Foundation AI models will diversify rather than become commodities
“My guess is that these models will not, you know, kind of commodify. My guess is that they will diversify increasingly over time, and that the idea that there's, these models will become commodities Defined as, you know, you can hot swap them and that their ec…”