Physics-Informed Neural Networks fail on chaotic, time-dependent differential equations
Anima Anandkumar · 🔬 Why Transformers Hit a Wall the Moment Physics Shows Up — Anima Anandkumar, Caltech · Aug 26, 2026 · at 12:24
Caltech professor Anima Anandkumar discusses why Physics-Informed Neural Networks (PINNs) break down when applied to dynamic or turbulent physical systems like fluid mechanics.
0:00 / 0:55exact quote · 55.9s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“Optimization ends up being usually very difficult, especially for problems that are time-dependent, meaning it's not just stationary, you also have time, and the time component in many cases could be turbulent, like in the case of fluid dynamics, you know, you kind of, like, if you Run it long enough. It can become chaotic. So you really, you know, have like very small fine scale effects matter. And so in those cases, just trying to solve a partial differential equation at all times is just hopeless. Like, you know, this is not an optimization landscape that, you know, I think we'll, you know, we can have any handle on. And this is where the idea that from scratch we would be able to Solve these equations using a neural net is not possible. So pins don't work everywhere.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Anima Anandkumar
Transformers will never scale to high-resolution 4D physics simulations
“So forget ever having a transformer for anything of this scale. All of the world's compute will not be enough. And first of all, they all have to be co-located to be able to ever do this. So that's why we need other architectures.”
Neural operators are the only AI architecture that works for climate emulation
“This is where the Allen AI Institute has now built climate models based on our neural operator architecture. And that's the only one that works As an AI emulator, right? None of the other architectures work for climate because climate requires us to assume the…”
AI models predict fusion reactor plasma disruption one million times faster
“You know, I talk about plasma and fusion reactor. You know, we barely have a few thousand samples, but we are able to accurately predict events like disruption very well. And we are able to do that a million times faster than what traditional simulations were …”
Anandkumar: Multi-physics models outperform single-physics models of equivalent parameter size
“And in fact, I was going to add that it turns out that having the model of the same size with multiple areas of physics does better than giving all of those parameters to each single physics. So if you had separate models and made them big enough as the origin…”
Anandkumar: Standard Transformers cannot scale to 5-trillion context lengths for physics
“On the other hand, if you think about using transformer architectures that have worked so well for language, that just wouldn't be able to support a five trillion context length. No matter all the compute in the world is thrown at it. So that kind of quadratic…”
Anandkumar: Dense physics feedback enables better AI self-improvement than sparse LLMs
“And the difference there is compared to language where self-improvement needs something like human feedback or other reward signals that are very sparse. They just tell you yes or no, thumbs up or down. We have dense feedback because the physics laws, there's …”