Rives: Amino acid sequence patterns allow protein language models to learn biology
Alex Rives · 🔬 The Bitter Lesson is Coming for Proteins - Alex Rives, BioHub · May 27, 2026 · at 15:24
Alex Rives, Head of Science at Chan Zuckerberg Biohub, adapts Zellig Harris's distributional structure theory of linguistics to explain why protein language models learn biological meaning.
0:00 / 0:33exact quote · 33.8s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“The contexts in which an amino acid can occur are really determined by, you know, the structure, the function of the protein, its biological roles, you know, these I mean, very complex phenomenon both the intrinsic biology of the protein and its relation to all of the other proteins and the function and evolution. And so, but those are what determine the context sets. And so you would imagine that then those statistical patterns in the use of amino acids, they directly reflect those underlying hidden variables. And so the model is going to learn something about those hidden variables.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Alex Rives
ESMC model search generates novel antibodies achieving therapeutic-grade binding affinity levels
“What we're able to see is that, you know, you can search ESMC and you can actually find antibodies that are reaching the level of affinity that are, I should say, are really at the level of affinity that is needed for therapeutic function and activity.”
Rives: ESMC is state of the art among open models for multimer prediction
“Yeah, I mean, I think we're state of the art for open models.”
AI cell simulations will enable parallel reasoning across millions of biological hypotheses
“We're gonna have increasingly capable and accurate digital representations of molecules, genomes, cells, ultimately physiology. That's where you want to get. We're gonna have to go up that, that complexity scale, the levels of biological complexity that requir…”
Rives: Curing disease requires personalized computational models, not conventional pills
“What is, you know, what is the cure to disease look like, right? It's not a pill, right? It's not a medicine in the conventional sense. You know, it's going to have to be a system that is capable of modeling and understanding, you know, the underlying physiolo…”
Rives: Current Virtual Cell Models Cannot Predict Novel Interventions
“I think with, you know, kind of the current generation of models that are being called virtual cells, they are good representations of the underlying data, but, you know, they have a very limited ability to predict what will happen when you make a novel interv…”
Scaling protein models by orders of magnitude unlocks emergent biological capabilities
“So our team has really explored that idea over a number of different years, and we've really kind of, I think, seen the scaling curve and really seen as we have increased models by an order of magnitude kind of in each generation that, you know, there's this e…”