Sergei Yudinov explains the data scarcity bottlenecks in training molecular machine learning models.
Opinion
Edunov: LLM architectures are boring and fundamentally unchanged since 2017
“And honestly, LLM architectures are relatively boring. I don't know, probably alienate half of your audience. But it's like, it's a transforming layer in the end, like paper was published in 2017, and you go to any LLM lab today, you will see very, very simila…”
Insight
Edunov: Small molecules can be modeled with physics to generate training data
“In small molecular space, you can actually model your small molecules with physics. You can model their behavior, and that allows you to create more data that you can train model on. Something which is not necessarily possible in protein to protein.”
Assertion Not checkable as stated
Edunov: Physics-guided inference-time scaling significantly boosts molecular model performance
“We are doing very similar thing with our models where a model is forced to think, except it's not thinking in language tokens. It's thinking in terms of crystal structures, not fully materialized crystal structures, but some sort of a crystal structure represe…”
Insight
Edunov: Predicting molecular poses is necessary to validate drug discovery models
“Yes, it's an abstraction, but it's a very useful abstraction. It helps us to build up confidence that a particular model output is actually valid. Whether you did not just straight up hallucinate at something, because yes, ultimately what matters is binding af…”
Insight
Edunov: Predicting ADMET properties is as critical as protein structure prediction
“Predicting all of those properties is also just as important as, or maybe even more important than predicting the crystal structure itself.”
Insight
Yudinov: Only data, infrastructure, and evals matter in AI development
“Where three things matter in AI it's data, infrastructure, and evals. Right. So you can only improve what you measure. And once you are very careful about measuring, What matters? And you have really talented people on the team. We're going to figure out how t…”