Oct 3, 2024 · 28m · a16z
AI at the Intersection of Bio | Vijay Pande, Surya Ganguli & Bowen Liu
0:00 / 0:00
▶ Watch on YouTube →
gold bands on the timeline = statements, start to end. Hover to read, click to jump. CC turns on captions
In this episode of the 'Raising Health' podcast, Dr. Vijay Pande hosts AI experts Dr. Surya Ganguli and Dr. Bowen Liu to explore how deep learning, self-supervised foundation models, and generative AI are transforming drug discovery, target identification, and clinical development.
How this conversation actually went
Every chapter scored 0–10 on four independent dynamics. Hover any point for the reasoning behind the score. How this is scored →
The host as informed peer 5.4 Guest teaching 4.8 Guest disagreement 1.4 The host pushing back 3.5
speaking balance: gold is the host, purple is the guest (3 minute bins)
Sharpest disagreement ▶ 14:54 Surya rejects the universal superiority of foundation ML models
Surya explicitly frames his argument as controversial, claiming that deep learning models like AlphaFold 3 fail at out-of-distribution generalization and that simple physics-based docking outperforms them on rare ligands.
Hardest push from the host ▶ 9:20 Host challenges the practical value of protein structure predictionThe host interrupts the guest's overview to directly challenge the utility of structure prediction, demanding to know what specific value it adds to actual drug discovery.
Biggest teaching moment ▶ 14:54 Surya breaks down AlphaFold 3 failure modes with empirical dataSurya educates the host on model generalization limits by citing data from Inductive Bio, demonstrating that physics docking beat AlphaFold 3 by 8 percent on ligands outside the top 50 common data bank examples.
The host holds their own ▶ 16:28 Host uses classical physics to reframe extrapolation vs interpolationThe host draws on his background as a physicist to challenge Surya's definition of extrapolation, showing how Newton's jump from falling apples to orbiting planets was actually interpolation within the correct underlying latent space.
the scores for every segment, with the reasoning behind each
| Chapter | Topic | The host as informed peer | Guest teaching | Guest disagreement | The host pushing back | Why |
|---|---|---|---|---|---|---|
| 'Aha Moments' in AI and Biological Sciences | 4 | 4 | 1 | 1 | The host opens the episode asking about 'aha moments' and framing the shift from traditional computational chemistry to deep learning. He offers a helpful analogy comparing representations to Arabic vs. Roman numerals, while guests detail the evolution from physics-based models to self-supervised learning. | |
| Data Scale and Self-Supervised Learning Across Modalities | 4 | 5 | 1 | 2 | Surya provides a deep quantitative breakdown of dataset sizes across language, proteins, 3D structures, and chemical spaces. The host interjects briefly to add nuance about sequencing bias and deep learning architectures acting as complex physics models. | |
| Solving Labeled Data Scarcity in Drug Discovery | 6 | 5 | 1 | 6 | The host demonstrates strong knowledge of the drug discovery pipeline and explicitly pushes back on Bowen to clarify what protein structure prediction actually accomplishes for practical drug design. The guests respond by explaining binding mechanics, multi-objective optimization, and the economic burden of Eroom's law. | |
| Generative AI and Inverse Molecular Design | 6 | 5 | 2 | 6 | When Bowen notes that validating generated ideas in science is hard, the host explicitly pushes back by citing in silico AUC benchmarks. Bowen clarifies that wet lab synthesis remains the true bottleneck, leading the host to discuss target hit rates and experimentalist trust dynamics. | |
| Foundation Models vs. Specialized Models and Generalization Limits | 7 | 6 | 3 | 6 | Surya takes a contrarian stance that specialized physics models beat foundation models like AlphaFold 3 on out-of-distribution targets. The host counters Surya's framing of extrapolation by using Newton's laws of motion to demonstrate how finding the right latent space turns apparent extrapolation into interpolation. | |
| AI in Target Discovery and Disentangled Cellular Latent Spaces | 6 | 5 | 1 | 4 | The host probes the biological target discovery space, asking critical questions about cellular phenotypes and interpreting latent spaces versus physics formulas. Surya explains disentangled latent spaces using variational autoencoders and face generation analogies. | |
| Revolutionizing Clinical Trials and Countering Eroom's Law | 5 | 4 | 1 | 2 | The discussion turns to clinical trials, where the host calculates that shifting trial success from 20% to 30% represents a dramatic 50% increase in drug output. Surya details patient selection via EMR databases and strategies to counter Eroom's law. | |
| The 'Digital Human' Foundation Model and Future Outlook | 5 | 4 | 1 | 1 | The host synthesizes the discussion into a vision of an end-to-end 'digital human' foundation model over a 10-year timeline. The guests agree and outline the multi-modal biological hierarchy data currently being gathered to build it. |
Statements from this episode (18)
Ganguli: ESMFold proves language models can learn protein structure from sequences
“I was actually really impressed by ESM fold, this average scale modeling where you could kind of do the same thing that you do for language, but do it for sequences. And then you learn representations that know about the structure of proteins.”
Vijay Pande: Proper data representation makes computation natural
“If I asked you, like, 25 plus 17, that's really easy to do. If I gave you that same problem in Roman numerals, you'd probably have to think about that back into Arabic and then do the computation and put back into Roman. Some representations make computation n…”
Ganguli: ESM-3 model trained on 2.8 billion amino acid sequences
“ESM-III with evolutionary scale modeling did the same language modeling, but now on amino acid sequences of about 2.8 billion sequences, right?”
Ganguli: Protein Data Bank contains roughly 200,000 solved structures
“We have about 200,000 or so solved protein structures in the protein data bank.”
Bowen Liu calls labeled data scarcity the core bottleneck in scientific AI
“Yeah, I think you touched on, like, probably the core problem of, like, you know, ML applied to science. While we have a lot of, like, unlabeled data, like, there's just not that much label data out there. And a lot of it is because, like, it's very experiment…”
Liu: 100 active compounds is a drug candidate but tiny for ML
“Usually in a drug discovery project, you know, if you have a hundred actives, you should be close to a drug, but a hundred data points is, like, tiny for machine learning, right?”
Liu: Protein structure prediction is essentially solved for many common proteins
“In the past, like, four or five years, this problem of protein structure prediction went from, you know, something that was, you know, quite far away from being solved, to now you could argue that, It's pretty much been solved for a lot of, like, common kind o…”
Ganguli: Bringing a drug to market costs $2.5B and takes 10–15 years
“The cost of drug design, I mean, I know in the industry this is well known, but it's worth, worth emphasizing, it's 2.5 billion dollars per drug in 10 to 15 years, right?”
Ganguli: 90% of drug candidates fail to achieve FDA approval
“90% of drug candidates don't get FDA approval.”
Ganguli: Drug output per billion dollars of R&D halves every nine years
“There's this law that the number of drugs brought to market per billion dollar spend is going down by half every nine years, right?”
Ganguli: FDA-approved drugs target only 800 of roughly 20,000 human genes
“FDA approved drugs they target only 800 of the 20,000 known genes that we have, 20 to 25,000 known genes that we have.”
Liu: Generative AI in science makes validation much harder than idea generation
“In science, you know, it's actually the inverse. Like, it's actually, it's probably easier to generate ideas, but way harder to validate.”
Ganguli: Specialized Models Beat Fine-Tuned Foundation Models for Specific Tasks
“You really, I think you're best off with a specialized model trained on the data that's very relevant to the thing, the task you want to solve. Your second best bet is to start with a foundation model that understands the broad space and fine tune it. Again, o…”
Bowen Liu: Small Molecules Currently Lack Self-Supervised Learning Frameworks
“Because like with protein sequences and biological sequences, like you can actually do self-supervised learning, right? Because there is this like complicated generative process with like evolutionary pressure that you can learn from. There's no equivalent for…”
Pande: 80% of late-stage drug failures stem from flawed biology
“What is it, like, 80% of drugs fail in phase two or three in trials, and that's not because it's toxic, it's because we screwed up the biology.”
Ganguli: Evolutionary robustness proves low-dimensional structure controls biological function
“Biological systems have survived for almost four billion years of evolution. They've tolerated all sorts of insults, competition, and so forth, so they're extremely robust. Because they're robust, their function can't depend on all of the details. That means t…”