Jan 2, 2019 · 22m · a16z
a16z Podcast | The Storage Renaissance
Peter Levine · 7m spoken Mike Matchett · 6m spoken Haoyuan 'H.Y.' Li · 5m spoken Sonal Chokshi · 2m spoken
0:00 / 0:00
▶ Watch on YouTube →
gold bands on the timeline = statements, start to end. Hover to read, click to jump. CC turns on captions
In this episode of the a16z Podcast, industry experts discuss 'The Storage Renaissance,' exploring how falling memory costs, machine-generated data, and unified virtualization layers are transforming cloud architecture, edge computing, and real-time artificial intelligence.
How this conversation actually went
Every chapter scored 0–10 on four independent dynamics. Hover any point for the reasoning behind the score. The host holds 11.3% of the talking time here. How this is scored →
The host as informed peer 3.7 Guest teaching 5.4 Guest disagreement 1.4 The host pushing back 2.1
speaking balance: gold is the host, purple is the guest (3 minute bins)
Sharpest disagreement ▶ 7:06 Mike pushes back on total in-memory dominance
Mike Matchett tempers Peter Levine's optimism for all-memory storage by citing vendor predictions of global hardware chip and drive shortages, forcing a more nuanced architectural view.
Hardest push from the host ▶ 2:40 Host challenges need for new storage paradigmsSonal Chokshi refuses to accept that more data automatically requires a new storage architecture, asking why existing systems cannot simply be scaled bigger and better.
Biggest teaching moment ▶ 4:10 Peter redefines data and storage economicsPeter Levine educates the panel on how autonomous vehicle sensors fundamentally alter what constitutes data beyond human typing, and how mobile supply chains rewrite data center cost structures.
The host holds their own ▶ 15:02 Host introduces system abstraction conceptSonal Chokshi displays technical insight by correctly identifying that bridging legacy systems with modern data movement requires a unified software abstraction layer.
the scores for every segment, with the reasoning behind each
| Chapter | Topic | The host as informed peer | Guest teaching | Guest disagreement | The host pushing back | Why |
|---|---|---|---|---|---|---|
| Why Storage Matters and the Concept of Data Gravity | 4 | 5 | 1 | 3 | Host Sonal probes why storage matters, framing data growth as a potential difference of degree vs kind. Guests introduce fundamental data gravity concepts to educate the host on why scaling old storage methods fails. | |
| Evolution of Storage Formats and Machine-Generated Data | 2 | 6 | 1 | 1 | Peter Levine reframes the definition of data from human input to sensor/machine data, while explaining how mobile supply chains lower enterprise cost curves. Sonal listens and offers light humor. | |
| In-Memory Storage Architectures and Cost Reductions | 3 | 6 | 3 | 2 | Guests debate the limits of in-memory architecture. Mike Matchett offers mild combativeness by pointing out hardware supply constraints that counter pure in-memory optimism, prompting Peter to clarify edge computing models. | |
| Multi-Cloud Ecosystems and In-Memory Tiering | 5 | 5 | 1 | 3 | Sonal asks precise questions about enterprise multi-cloud storage sprawl and forces H.Y. Li to explain why in-memory tiering specifically solves the performance and cost problems. | |
| Machine Learning and In-Memory Processing Acceleration | 2 | 6 | 1 | 1 | Peter Levine and Mike Matchett deliver detailed technical explanations connecting machine learning iterative compute demands with in-memory architectures like Spark, with minimal host intervention. | |
| Virtualizing Storage via Unified Abstraction Layers | 6 | 5 | 1 | 3 | Sonal demonstrates strong technical domain synthesis by framing H.Y.'s explanation of data silos as the need for a unified virtualization abstraction layer, which H.Y. strongly confirms. | |
| The Changing Role of IT and Predictive Analytics | 4 | 5 | 2 | 2 | Mike and Peter discuss the shifting role of IT toward real-time predictive analytics. Mike gently interrupts Peter to push the predictive timeline from 'tomorrow' to real-time micro-decisions. |
Statements from this episode (13)
Falling memory costs will unify storage and compute memory
“With the cost of system memory decreasing, memory for both storage and compute will be the exact same thing.”
Storage is the most complex infrastructure element due to data gravity
“Storage is really the most complex part of that equation. It takes a lot of effort to protect data, to manage data. Data has gravity. It has momentum. It has Weight in history. So storage is really the critical piece to get right.”
Sensor and machine data volume will expand by orders of magnitude
“It will be literally orders of magnitude and in the exact mathematical sense, orders of magnitude, more data that needs to be stored in process.”
Disk drives, tape drives, and SSDs will all go away
“We will have much more in-memory data, systems that literally live in real memory, and the notion of disk drives and tape drives and even SSDs will all go away.”
Storage vendors forecast a global hardware shortage within 3 to 5 years
“They're forecasting that there just simply isn't going to be enough storage for all the data we're collecting in a midterm horizon, like three to five years that we're creating so much data that There won't be enough chips. There won't be enough hard drives. T…”
Edge endpoints will process data locally instead of centralized streaming
“If you think about the new world of distributed computing, the data that gets collected in a self-driving car or some endpoint is going to be, the information will be processed at that endpoint. It won't be translated back to a central storage pool. The inform…”
Autonomous vehicles generate 10 gigabytes of data per mile driven
“Self-driving car collects 10 gigabytes of data a mile, right?”
Computer memory costs drop 50 percent every eight months
“The cost is decreasing very fast. It's about every eight months, the cost decreased by 50%.”
Machine learning cannot function effectively without in-memory processing
“Unless we get to in memory processing and in-memory data structures, machine learning doesn't really work.”
Manually moving data across storage environments takes weeks and degrades value
“Today, if people want to really see the global data, they move the data manually from one storage to another storage to put them together to analyze it. Even though the data could be in memory in a final storage, But because of this manual process or this proc…”
IT departments will lead corporate machine learning adoption
“So IT departments are really going to be looked at as the point of the sword in bringing these advances to the company and not just being seen as reactive operators of infrastructure on the back end. And that's a big shift.”
Storage-class memory will ship within a couple quarters
“Well, I think we mentioned the storage class memory that's coming out Intel, Some other folks making a very fast, close to the chip memory that's actually storage. So it'll be faster again than flash, maybe a little bit slower than today's DRAM, but it's going…”
Companies will face 'data poverty' within two years
“I think there's going to be some companies kind of surprised in a year or two to find that they're not going to have access to the data that they're going to really want to have that's relevant to what they need to do to make the predictions to optimize their …”