ESMC

“At the time that this podcast comes out, we will have announced ESMC and this world model for protein biology. It's gonna be open source. It's gonna be MIT licensed, and we want people to use it.”

“Having a vast diversity of sequences across a wide range of protein families is, you know, really critical for the emergence of this kind of structure Prediction capability, because I think kind of large diversity is what trains the model to understand, to dev…”

“Yeah, I mean, I think we're state of the art for open models.”

“What we're able to see is that, you know, you can search ESMC and you can actually find antibodies that are reaching the level of affinity that are, I should say, are really at the level of affinity that is needed for therapeutic function and activity.”

“We've been able to use this to actually now go and design many protein binders, but I think sort of most excitingly, we've been able to use this to actually design antibodies, SCFVs, and we're seeing really, I think, exciting success rates and a small number o…”

“And then, you know, what we saw basically is, is, is there are no longer diminishing returns to scale. So that's really saying that ESM two was kind of data limited rather than compute limited for ESMC.”

“ESM-II is trained on Uniref. And for ESMC, we added metagenomics. So we added billions more sequences to the training data.”

“You know, what we found basically is that the model has a kind of a single feature for this nucleophilic elbow and is activating across these like very evolutionarily diverse families, you know, really completely different structural topologies, proteins that …”

“So there's actually three models in that family. There's a three hundred million parameter model, a six hundred million. Parameter model and a six billion parameter model.”