Peter Battaglia, research director at Google DeepMind, discusses the underlying machine learning approaches powering modern AI weather forecasting models.
Opinion
Battaglia: Data pipelines matter more than neural network architectures today
“I don't feel that neural network architectures these days tend to be the sort of exciting part. It's usually more of like the training and the sort of data how you handle the data and that kind of thing.”
Disclosure
DeepMind uses transformers to model spatial interactions in weather, not time
“We actually use transformers not to model the spatial, the interactions in weather over time, like the sequence of text, but in space.”
Insight
Battaglia: AI weather models leverage macroscopic spatial ranges unlike traditional simulations
“In a traditional model, the way it simulates that is it, in very fine detail, it kind of figures out, like, what's the pressure, and the temperature, and the wind, and the moisture, and what are those things? What's going to happen next is determined strictly …”
Assertion Supported
Battaglia: Early ML in weather improved downstream calibration, not base models
“So actually, the first advent of like, AI and machine learning in weather forecasting, or at least some of the earliest, was not, like, trying to overhaul the whole weather forecast process itself, but making, you know, using more and more, like, statistical m…”
Insight
Battaglia: Precipitation and wind are harder to forecast due to fine-scale dynamics
“So that type of very high resolution, complex, you know, patterns of precipitation, for example, Wind as well. Those are much harder to predict because you're effectively predicting a lot more information. You can't just sort of summarize it by saying, oh, eve…”
Insight
Battaglia: Weather is Markovian unlike text sequences in language models
“Weather is actually different from text in a fundamental way. In fact, most physical processes are. They are what's called Markov, in that the most recent state of the system determines the subsequent state.”