Edunov: LLM architectures are boring and fundamentally unchanged since 2017
Sergei Yudinov · 🔬 "The Most Innovative Diffusion Research Is Happening in Drug Discovery, Not Image Generation" · Jun 30, 2026 · at 1:46:23
Sergey Edunov (Genesis Therapeutics CTO, formerly leading Llama pretraining at Meta) contrasts the lack of architectural innovation in LLMs with 3D molecular AI.
0:00 / 0:25exact quote · 25.8s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“And honestly, LLM architectures are relatively boring. I don't know, probably alienate half of your audience. But it's like, it's a transforming layer in the end, like paper was published in 2017, and you go to any LLM lab today, you will see very, very similar pieces, ah, in their models. Yes, there is a little bit of flavor of MOEs, but otherwise, architectures are fundamentally very similar to what was discovered back in 2017, right?”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Sergei Yudinov
Edunov: Small molecules can be modeled with physics to generate training data
“In small molecular space, you can actually model your small molecules with physics. You can model their behavior, and that allows you to create more data that you can train model on. Something which is not necessarily possible in protein to protein.”
Edunov: Physics-guided inference-time scaling significantly boosts molecular model performance
“We are doing very similar thing with our models where a model is forced to think, except it's not thinking in language tokens. It's thinking in terms of crystal structures, not fully materialized crystal structures, but some sort of a crystal structure represe…”
Edunov: Predicting molecular poses is necessary to validate drug discovery models
“Yes, it's an abstraction, but it's a very useful abstraction. It helps us to build up confidence that a particular model output is actually valid. Whether you did not just straight up hallucinate at something, because yes, ultimately what matters is binding af…”
Edunov: Predicting ADMET properties is as critical as protein structure prediction
“Predicting all of those properties is also just as important as, or maybe even more important than predicting the crystal structure itself.”
Yudinov: Only data, infrastructure, and evals matter in AI development
“Where three things matter in AI it's data, infrastructure, and evals. Right. So you can only improve what you measure. And once you are very careful about measuring, What matters? And you have really talented people on the team. We're going to figure out how t…”