Jan 2, 2019 · 21m · a16z
a16z Podcast | The Cool Stuff Only Happens at Scale
0:00 / 0:00
▶ Watch on YouTube →
gold bands on the timeline = statements, start to end. Hover to read, click to jump. CC turns on captions
Host Chris Dixon sits down with Herman Narula and Vijay Pandey to discuss how distributed computing architectures and large-scale agent-based simulations are transforming software engineering, scientific discovery, economics, and real-world disaster management.
How this conversation actually went
Every chapter scored 0–10 on four independent dynamics. Hover any point for the reasoning behind the score. How this is scored →
The host as informed peer 4.8 Guest teaching 2.8 Guest disagreement 1.0 The host pushing back 2.0
speaking balance: gold is the host, purple is the guest (3 minute bins)
Sharpest disagreement ▶ 6:31 Questioning existing tech infrastructure
Herman Narula politely challenges conventional wisdom that tech giants will maintain dominance over large compute, arguing scaling web services differs fundamentally from distributed simulation.
Hardest push from the host ▶ 4:04 Challenging necessity of volunteer computeHost Chris Dixon presses Vijay Pande on whether volunteer distributed computing projects are obsolete in the modern era of commercial cloud infrastructure.
Biggest teaching moment ▶ 4:11 Explaining cloud cost and compute scaleVijay Pande educates the host on the massive scale of Folding at Home by pointing out that purchasing Amazon's entire cloud capacity of 300,000 boxes would be cost-prohibitive compared to distributed networks.
The host holds their own ▶ 9:26 Articulating pet theory on simulation historyHost Chris Dixon showcases high expertise by comparing current fringe agent-based simulation techniques to 1980s machine learning enclaves that later became mainstream AI.
the scores for every segment, with the reasoning behind each
| Chapter | Topic | The host as informed peer | Guest teaching | Guest disagreement | The host pushing back | Why |
|---|---|---|---|---|---|---|
| Distributed Computing Applications in Science and Healthcare | 3 | 4 | 1 | 2 | Host Chris Dixon asks foundational questions about distributed computing in biology and probes whether modern cloud platforms like AWS supersede projects like Folding at Home. Guest Vijay Pande educates the host on the economics and sheer scale of petaflop compute compared to buying out cloud capacity. | |
| Frameworks, Fault Tolerance, and Academic Innovation | 6 | 2 | 1 | 3 | The host presents an insightful theoretical framework explaining why academia leads foundational distributed computing breakthroughs while industry focuses on depth search. Guests fully agree and build upon this framework without any conflict. | |
| Understanding World Complexity Through Simulation Platforms | 7 | 2 | 1 | 2 | Host Chris Dixon demonstrates deep domain knowledge by drawing a detailed historical parallel between 1980s machine learning enclaves and modern agent-based simulation modeling. Guests validate the host's pet theory and extend it with examples of emergent complexity. | |
| Biological Cell Simulation and Model Validity | 2 | 4 | 1 | 1 | Guests drive the discussion around cell simulations and address common preconceptions about model accuracy. The host takes a backseat while Herman Narula and Vijay Pande unpack how imperfect models still yield actionable scientific insights. | |
| Simulating Smart Cities and Infrastructure Integration | 5 | 2 | 1 | 2 | The host actively co-constructs a thought experiment with guest Herman Narula regarding hybrid city simulations blending IoT sensor feeds with modeled entities. The conversation remains entirely collaborative with no hostility. | |
| Disaster Recovery, Contingency Planning, and Conclusion | 6 | 3 | 1 | 2 | The host provides an informed industry perspective on disaster recovery, contrasting aviation safety feedback loops with rare black swan events like terrorist attacks. Guests discuss real-time decision making during earthquakes and spatial dynamics. |
Statements from this episode (19)
Dixon: Distributed computing will dominate software engineering for decades
“Over the next few decades, distributed computing will be a particularly important topic because we're now with things like AWS awash in Computing resources you know, compute is becoming, approaching zero, storage, networking, but most of this is on, you know, …”
Pandey: Domain-specific frameworks, not universal languages, will solve distributed computing
“And I think what we're going to see is verticals moving in, in certain directions that can do different things. It's hard to imagine the magic language that solves all of these problems. And so picking things that can sort of attack certain key domains actuall…”
Narula: Distributed computing's next phase will target hard-to-split problems
“I think the next few years are going to be about attacking problems which are naturally harder to split, harder to scale. And how we go about that, I mean, a whole other layer of reliability is going to be required as well for computations that are more challe…”
Pandey: Folding@home achieves 40 petaflops across 400,000 volunteer processors
“Folding at Home is a large-scale distributed computing project where people go to our website folding.snayfer.edu, they download the software, and right now we have about 40 petaflops worth of performance out of maybe about, you know, maybe 400,000 processors.”
Pandey: Work requiring 10,000 GPUs today will soon run on small clusters
“What we are doing now with 10,000 GPUs, people will probably do in the future with maybe a small GPU cluster.”
Narula: Fundamental approaches to distributed systems haven't changed since the 1980s
“Most of the ideas that are being used now, like actor paradigms, for example, which some people may be familiar with. I mean, these are from the eighties, right? Nothing's really changed in how we attack distributed systems.”
Pandey: Traditional supercomputing standard MPI is extremely fault-intolerant
“And MPI, which has been the standard in supercomputing, HPC, is very fault intolerant. You know, the whole job will crash, you know, things like that.”
Dixon: Industry excels at iterative depth, while academia excels at fundamental rethinkings
“And I think that the theory we have is that that's because you kind of, you know, industry is very good at a kind of a depth search, right? Like, you know, continuing to iterate on something. But when you need to go back and fundamentally rethink how you do so…”
Narula: Elite UK university academics use costly supercomputers unaware of distributed alternatives
“And we see a lot of academics that we speak to in kind of Cambridge, Oxford, using supercomputer methods right now, unaware that actually a distributed systems approach might be more cost effective or even easier to think about from that perspective.”
Narula: Historical data cannot answer what-if questions about unprecedented events
“You can answer questions, right? Answer those what if questions? What happens if, you know, a disease is released in this crowd? What happens if we shut down this tube station? Questions which governments and companies want to answer, but which, you know, you …”
Dixon: Mainstream Macroeconomic Equation Models Have a Poor Track Record
“If you look at, like, macroeconomics, for example, they use, you know, whatever. They have their set of equations, and they have a very, very poor track record in predicting the future.”
Narula: Improbable partners with Oxford to model the UK housing economy
“So, I mean, for example, we're dealing with a group called the Institute for New Economic Thinking at Oxford, and these are some amazing scientists, and they want to model the UK housing economy.”
Pandey: Simulation Is the Dominant Paradigm in Physics and Chemistry
“So I think actually in physics and chemistry, this is, simulation is, is a dominant paradigm.”
Narula: Emergent complexity in simulations challenges preconceived domain assumptions
“Analytical methods, they come with a certain degree of certainty and trust, right? Like, I can exactly explain to you why, how and why this works, but emerging complexity is unpredictable. It's scary. The results may not be what you What you expect, and it has…”
Pandey: Simulations do not need 100% accuracy to deliver valuable insights
“The simulation doesn't have to be perfect to be provocative. A, there's, you know, the ideal is something where it's perfect, and it gives quantitative predictions, whether of the stock market or of traffic or something like that. But a lot of times things are…”
Narula: Simulations should function as operating platforms, not static offline models
“At Improbable, we don't really believe that a simulation should be this, like, standalone box where knowledge comes out of. We see it as an operating platform, somewhere that you can actually make decisions and build applications that consume that simulation.”
Narula: Dominant simulation platforms must rewrite low-level infrastructure software
“I mean, I think ultimately the companies and solutions are going to start dominating the space. They're going to have to redo a lot of stuff at quite low layers in order to make it effective. So there may be a need to throw out some of the work that's gone bef…”
Narula: Improbable models power grid failures to reveal catastrophic risk accumulation
“For example, when we look at cascading failures in power grids, which is an area that, you know, we've explored it with improbable, how those failures arise can often be the accumulation of many, many disparate, like, seemingly irrelevant events and slight vul…”
Narula: Adding spatial parameters to Prisoner's Dilemma simulations radically alters outcomes
“When you just run prisoner's dilemma style games between participants, okay, you get one outcome. But when you start to introduce geographic components to those simulations, suddenly it all changes again.”