Neural Operators
topic on 1 show · 5 statements across 2 episodes
5 statements about Neural Operators, every show
Anandkumar: Neural operators mix physical scales internally to boost data efficiency
“Neural operators have this flexibility because they can allow you to mix across different scales within the model rather than be prescribed externally like a lot of other Hybrid machine learning for physics do, and that, you know, allows us to be a lot more da…”
Anandkumar: Standard Transformers cannot scale to 5-trillion context lengths for physics
“On the other hand, if you think about using transformer architectures that have worked so well for language, that just wouldn't be able to support a five trillion context length. No matter all the compute in the world is thrown at it. So that kind of quadratic…”
Neural operators accurately model non-local phenomena like atmospheric rivers
“These are like thousands of miles wide, so you really need non-local models that capture these very large span phenomena and do that accurately, and that's what our neural operators are able to do.”
Neural operators can evaluate at arbitrary resolutions at inference time
“Neural operators enable because they model inputs and outputs as continuous functions that can be infinitely resolved, that can have infinite discretization. And now we can have You know, at inference time, you can give it now inputs and ask for outputs at any…”
Neural operators overcome PINN limitations by combining data with physics constraints
“Our idea of neural operators came as a way to overcome this, right? So saying, you know, we can't rely just on physics constraints alone to come up with answers. We have lots of data available. You know, I'll talk about the weather example where we even collec…”