Neural operators overcome PINN limitations by combining data with physics constraints
Anima Anandkumar · 🔬 Why Transformers Hit a Wall the Moment Physics Shows Up — Anima Anandkumar, Caltech · Aug 26, 2026 · at 13:25
Caltech professor Anima Anandkumar explains how neural operators resolve optimization bottlenecks in physical simulation by blending data-driven supervised learning with physical equations.
0:00 / 0:55exact quote · 55.8s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“Our idea of neural operators came as a way to overcome this, right? So saying, you know, we can't rely just on physics constraints alone to come up with answers. We have lots of data available. You know, I'll talk about the weather example where we even collect data, right? So we don't just solve equations and Have synthetic data, but we also have real data by observing the weather as one example. So why not make use of all of the data available? So we don't just rely on trying to solve partial differential equations and other physical problems from scratch, because it's really the data driven approach that makes it possible to get quick answers. And so with neural operators, we can bring both of them together. We can Have all the data that's available, we can utilize it, we can add physical constraints, and then that overcomes the limitations that pins face.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Anima Anandkumar
Transformers will never scale to high-resolution 4D physics simulations
“So forget ever having a transformer for anything of this scale. All of the world's compute will not be enough. And first of all, they all have to be co-located to be able to ever do this. So that's why we need other architectures.”
Neural operators are the only AI architecture that works for climate emulation
“This is where the Allen AI Institute has now built climate models based on our neural operator architecture. And that's the only one that works As an AI emulator, right? None of the other architectures work for climate because climate requires us to assume the…”
AI models predict fusion reactor plasma disruption one million times faster
“You know, I talk about plasma and fusion reactor. You know, we barely have a few thousand samples, but we are able to accurately predict events like disruption very well. And we are able to do that a million times faster than what traditional simulations were …”
Anandkumar: Multi-physics models outperform single-physics models of equivalent parameter size
“And in fact, I was going to add that it turns out that having the model of the same size with multiple areas of physics does better than giving all of those parameters to each single physics. So if you had separate models and made them big enough as the origin…”
Anandkumar: Standard Transformers cannot scale to 5-trillion context lengths for physics
“On the other hand, if you think about using transformer architectures that have worked so well for language, that just wouldn't be able to support a five trillion context length. No matter all the compute in the world is thrown at it. So that kind of quadratic…”
Anandkumar: Dense physics feedback enables better AI self-improvement than sparse LLMs
“And the difference there is compared to language where self-improvement needs something like human feedback or other reward signals that are very sparse. They just tell you yes or no, thumbs up or down. We have dense feedback because the physics laws, there's …”