The Exchanges

Q And so, that brings us to this question of interpretability. Like, if something goes wrong, um, historically, how do regulators work through a problem and then With AI, like, how do you, how does it change the way you work through a problem and ask who is responsible for, for a problem if it's AI related?

A Yeah, it's a great question. So, historically, um, a lot of, uh, prediction algorithms on the power grid were, um, based on rule-based systems. So, you try to figure out what electricity demand looks like by writing down a set of rules associated with, um, You know, is it, um, you know, a weekend or a weekday? Um, is it a holiday? Is there a kind of, you know, very famous TV show on where everyone's going to turn their tea kettle on right afterwards? You actually write down that set of rules and use it to try to create some kind of, um, prediction. And so historically, then, if something went wrong on the power grid, if there was a mismanagement of the grid, and that was in part due to a misprediction, the idea is that the regulator would ask Ask your, your system operator to go back and say, what went wrong in your rule-based prediction? Which rule didn't hold? And how do you actually improve that for the future? Um, the kind of thing that's happening as we increasingly use AI and machine learning for these kinds of predictions is that the same kind of regulatory practice continues to be applied. So if there's a misprediction, the, the regulator will often want to know, well, what went wrong in the internals of your predictive model? To cause that to have happened. And I think there's genuine debate about whether it is sort of on the technology to become more interpretable in …

AI assessment note: “historically, then, if something went wrong... the regulator would ask... what went wrong”

Q Yeah, so talk about the limitations of machine learning in a physical system, complicated physical system like the grid, where you're controlling, you know, sensitive electronics, you're controlling power plants, you're dispatching, like, what are those physical limitations that we need to start to work through?

A Yeah, so how machine learning works is basically it's analyzing a large amount of data, and it's trying to find what the predominant patterns in that data are. In some sense, they're the average thing that would happen in that data. And so, When you're doing something like, you know, um, predicting what an image is, if you are most of the time right, because you've gotten those sort of averages in the data right, that's great, and if every now and then you're wrong, okay. On a power grid, where you have to do things like make sure that you're not asking equipment to do something it can't actually do, or make sure that your power lines are not being asked to carry too much power, or you're maintaining voltages or currents or stability constraints or whatever have you, Averages aren't often enough. The, the fact that you can get a machine learning algorithm that learns to do something nuanced from data and does it right most of the time doesn't help you in those times when it does something wrong that one time that really blacks out your grid. So this is where kind of machine learning can help you find the predominant patterns, help you figure out what to do in the average case when things are going as usual, but in these extremal cases or, or kind of, you know, anomalous cases that may only show up You know, little or not at all in your underlying data. That's when you need ofte…

AI assessment note: “On a power grid, where you have to do things like make sure that”

Q So on that front, then, what does it mean to create a safety-critical bias-free model?

A Yeah, so, I mean, I think this is very, you know, context-dependent, right? But let's think about, for example, a couple of power and energy problems here. So when we talk about a kind of safety-critical algorithm for power grid control, we want to make sure that if we have an algorithm that's controlling our devices on the power grid, for example, That the outputs are yielding power flows on the grid that, you know, meet what the grid can actually handle. Again, you're not overflowing lines, you're not causing voltages to be out of whack, um, all things like that. Um, and you also want to make sure that you're meeting certain, you know, stability constraints, making sure your grid is staying near equilibrium. So there are, you know, fields of study that actually try to, you know, write these criteria down, right, in electrical engineering and in control theory. And so if we have those criteria, engineering an AI system that meets those criteria. So what some of my work does is actually say, can we actually construct those criteria in a way that looks like a layer in a deep neural network and actually embed that layer within your neural network so that the output has some kind of guarantee on it. So I would say creating safety critical AI is, you know, really understanding what is the metric that needs to be met or the kind of requirement that needs to be met in our safety crit…

AI assessment note: “creating safety critical AI is, you know, really understanding what is the metric”

Q to answer here are, like, technological pathways, and then we want to talk about, like, integration inside companies. And so I wonder, you know, as, as there are companies in this room who are thinking about how to build AI teams, um, how to integrate them inside their companies, like, how do you Think about hiring personnel. What do they need to be thinking about to build infrastructure around AI?

A Yeah, so there, yeah, there, there are kind of a couple of different kinds of personnel that you, you need to sort of integrate AI with the use case. So one of them is, um, somebody who's obviously doing the business case and business scoping, right? For a particular use case, we're thinking of AI. How does it fit into the broader business? How does it get integrated with everything else across the company? So that, that aspect of things. But you also need, um, you know, there's a difference between, you know, data engineers, software engineers, and data scientists. So data engineers are the ones who will actually, you know, work closely with the underlying data to clean it, to make sure it's of high quality, to make sure that it actually is usable for large-scale analysis. Then your data scientists are the ones who are actually trying to glean insight from that data in order to kind of enable whatever kind of insight is needed for your use case. And then a software engineer is the one who actually helps to maybe integrate those Those algorithms with maybe the broader product or workflow, um, that they're in that, um, writes things like unit tests and integration tests, things that make sure that the software is, is, is performing the way it should and is interacting with the rest of the ecosystem the way it should. Um, and then of course you, yeah, so these are often the diffe…

AI assessment note: “there are kind of a couple of different kinds of personnel that you need”

Q often beating their head against the wall at how slow it can be. Um, so with that in mind, and given how fast a lot of these technologies are Are accelerating. What is something that you think will astonish us in the coming years, and what is, what are maybe some of the physical limitations that you think potentially could hold these technologies back? Priya, do you want to start?

A Yeah, I mean, I think the thing I'm excited to be astonished by is the capability of, um, AI and machine learning, uh, algorithms to, um, enable kind of automatic optimization and control of electric power grids. I think that once we set up the correct infrastructure to enable that kind of thing, this is going to accelerate really quickly, and so this is things like, um, controlling distributed devices on the power grid based on local sensor data, or speeding up centralized optimization algorithms, like, Unit commitment, you know, security-constrained optimal power flow, the boring stuff nobody really talks about in ways that enable us to actually more dynamically optimize power grids. I think simultaneously there is, there is something really critical holding that back, which is that there aren't really good pathways to deployment for a lot of techniques, AI machine learning or otherwise, that are actually interacting with critical operations and, you know, critical, um, functions on a power grid. And this is because, you know, We have historically not allowed for huge innovation on power grid optimization because we're worried about the power grid blacking out, and so really thinking about how do you create the correct simulators, test beds, metrics that are really agreed upon by practitioners, by the industry, and really create these pathways to deployment, as well as the ki…

AI assessment note: “the capability of, um, AI and machine learning, uh, algorithms to, um, enable”

Q is coming next. So, you heard GPT introduce the panelists. Let's bring them back in. Um, Priya. For years, you've been talking about, you've been focused on machine learning, um, to, to, to, uh, as a helpful tool for climate solutions. Talk about the moment today. Like, what is different? All of a sudden, everyone is talking about it. Um, how would you characterize the sudden interest in the space?

A Yeah, so I'd say that, um, machine learning has really in and of itself accelerated over the last several years, last decade, and this has been due to a combination of kind of widespread data availability, um, increased computational power, and also improvements in algorithms. Um, and then, of course, in parallel, the climate conversation has accelerated a lot, and I actually attribute a lot of that to Um, improvements in, uh, attribution science, right? When we were able to really start saying this specific event is due to climate change, I think that really kind of galvanized the public conversation. But as Stephen mentioned, kind of, you know, over the last, uh, you know, for, for a long time, people didn't put these things together. Um, and so, um, I think many trends came together, but, but one thing was that, um, my organization, Climate Change AI, we, um, You know, wrote this paper called Tackling Climate Change with Machine Learning, where we kind of sat down, really looked at the literature, and really surveyed stakeholders over, you know, several months, uh, you know, and, and really tried to understand where is it that AI and machine learning can play a role. Um, and we saw a lot of different kinds of use cases, including kind of turning raw data into actionable insights, so taking satellite imagery and turning it into insights about where the solar panels are, Or, u…

AI assessment note: “I think many trends came together, but, but one thing was that”

Q a lot of companies, um, sort of hyping this up as well. So like, how do you think about Where the real use cases are? What kind of new and emerging and interesting use cases are you seeing? And, and do you see any froth, uh, you know, companies that are maybe just, um, potentially just using the words AI, uh, artificial intelligence and don't actually have a good solution?

A Yeah, so I'd say there are a lot of, you know, really exciting use cases out there. So I mentioned some earlier, this idea of, you know, creating better forecasts for power system operators. For example, the non-profit Open Climate Fix, they used AI and machine learning in combination, you know, with satellite imagery, historical data, to cut the error of the forecast that National Grid, the UK power system operator, is creating by a third to a half. And this is really improving the way in which power system operators are actually operating their grids. You also see many use cases of, um, again, scientific discovery, which is an example I gave earlier. So, for example, the startup Aionics is using AI to actually analyze, um, when I am, you know, trying to synthesize a battery for energy or for transport or for something else, these are going to have different properties from each other. But, and I can only synthesize so many batteries, so can I somehow learn from the success or failures of my past experiments in order to better understand both what battery I should try to synthesize next that might be more successful, or where there is a part of the search space that I should learn more about in order to then inform my future experiments. So there are lots of use cases like this. I would say that, kind of, if you can get past a couple of key questions, right, so what is the act…

AI assessment note: “Open Climate Fix, they used AI and machine learning in combination”

Q and then a machine learning use case for decarbonization with Google's Savannah Goodman. And with that, I want to start our opening conversation with Priya Danti, who is the co-founder and executive director of Climate Change AI. She's an assistant professor at MIT, and she's one of the leading thinkers on this subject. So Priya, come on up. So more profound than electricity or fire. What's your reaction to that?

A Yeah, I mean, I think AI is undoubtedly shaping, you know, many things across our society. It shapes how we, you know, interact with our phones, how we interact with information online, with each other. Um, but I think echoing some of the points made in the intro there, transformation shouldn't happen just for the sake of transformation. It needs to be shaped in a way that is aligned with our societal values. So thinking about some of the other transformations that were mentioned, right? Electricity. Something like seven hundred million people around the world don't have Electricity, right? Medicine. Bunch of people in the US can't afford insulin. Um, and with technologies like generative AI, there's a lot of, you know, flashiness and sort of jumping on the hype train, but I, I do sincerely worry that this will lead to an AI winter if a lot of those, you know, if we see a lot of, you know, hype and unscrupulous promises made there that then kind of, um, reflect poorly on maybe other uses of AI that are, uh, very principled and do have the ability to do things like transform our electric grids. So I think fundamentally, yes, AI will play a really large role and already is, but sort of developing the technologies and policies and social structures in a way that actually shapes that transformation in the way we want is going to be critical.

AI assessment note: “thinking about some of the other transformations that were mentioned, right? Electricity.”

Q around large language models is that, like, a lot of the researchers, like, don't exactly know how the system is making the decisions it is. Is that a, do you see that as a real risk now? I mean, going back to what you just outlined, like, is this a real tangible problem today in that people who are developing these systems don't exactly know how decisions are being made?

A I think it is a problem, and especially it becomes a problem when you start to look at, like, safety-critical systems. When we think about, you know, power grids as a safety-critical system, right? Again, something where you really cannot break the system. It has huge economic consequences and, you know, consequences for loss of lives. But I think we actually, I mean, to, to bring this to the, you know, the public conversation about, about GPT, we also need to think about the ways in which language is a safety-critical system, right? In what cases is it, for example, Shaping someone's opinion, or how they consume information, or, or, um, serving information that is meant to be accurate, and it's a combination of both, do you know what's going on in the model to produce it, but also, do you know what it means for an output to be good? Um, and I think having answers to this latter part, do you know what it means for an output to be good, leads to two different Potential directions, one of which is engineer your underlying model or your data to kind of bias it towards outputs that meet that standard of quality, or take the output of your model and then change it in a way that is, um, kind of reflective of, you know, looking at whether it's meeting those standards of quality. So I think it's, again, just understanding what does it mean for an output to be good or safe or meeting th…

AI assessment note: “I think it is a problem, and especially it becomes a problem”

Q Yeah, and we're going to dig into some of those ethical questions a little bit deeper later today. Um, so on the security side, can you talk about, like, What are some worst case scenarios?

A Yeah, so, I mean, right now, I would say that for, for power grids, the, um, kind of explosion of sensing and, and data, um, creates certain kinds of risks. It increases our attack surface when you have sensors that can be tampered with, when you have autonomous algorithms that can be tampered with, um, and tampering can either be, you know, literally messing with the algorithm or somehow messing with the input data stream in a way that's adversarial and such that you know the algorithm is going to do something Incorrect. And so, I mean, in a worst case scenario, right, you, you can really have kind of major, um, you know, cyber hacking of the grid and cyber attack, and there's been some kind of preliminary studies that also show that, um, often the kind of biggest, um, risk case for cyber attacks is when the grid is already weak for some other reason, so you have some kind of extreme event or something, um, That is, that is natural, that is happening, and the grid is already operating in a bit of a kind of recovery or resilience mode in order to deal with that, and then on top of that, then you get some other kind of attack. So I think that is the worst case scenario, and one that we need to, again, kind of design our policy, social, and technology systems to be robust to. So there are ways, for example, you can, um, engineer parts of the grid to have provable guarantees acros…

AI assessment note: “often the kind of biggest, um, risk case for cyber attacks is when the grid is already weak”

Q And where do you fall on that question?

A Yeah, I mean, I think that it is sort of a, I think that we do need algorithms that are, um, interpretable and robust, I think, right? I think many algorithms today are very brittle to kind of noise or anomalies in the underlying data in a way that isn't always visible, and kind of building in robustness and interpretability or kind of more principled ways of making sure you know what your algorithm is doing. I think services, a lot of these real-world use At the same time, I think we shouldn't necessarily be wasting our time fiddling with how do we understand the exact, you know, weight within a deep neural network to, to, for the purposes of an audit, if that's not actually what we need, if instead maybe the right question is, what was it about the underlying data that didn't reflect the scenario that we saw, and how do we either adapt the algorithm or adapt our practices around using its output, because you don't have to use its output wholesale, you can do modifications and steps to the output. I think, again, having that conversation between these two is really important.

AI assessment note: “I think that we do need algorithms that are, um, interpretable and robust”

Q So to close this wrap up, uh, we have a cross section of investors, utilities, tech companies, startups, uh, in this space. What are, what is a provocative or some provocative questions they need to be asking themselves or each other at an event like this?

A Yeah. So I think thinking through, you know, What are the kinds of transformations that we are having, that we must, uh, must happen, but that we're having difficulty in practice kind of getting off the ground? So where is there maybe a mismatch between our vision, for example, the transformation of the grid and, and the, the practical implementation towards that vision? Um, and I think within that, then the question is, what is it that is, what is in there that maybe could Be enabled by AI and analytics, and then that leads to, of course, the questions around how and what, but what is it that it, where it is something else, where it is some kind of financial incentive, where it is some kind of, um, you know, policy incentive, um, and I think really teasing out what is the technology? What is the policy? What is the society aspect? How do they interact with each other? And then how do we move forward in kind of a holistic and integrated way on these? I think these kinds of questions very practically is what we need to kind of match our practice to our vision.

AI assessment note: “Where is there maybe a mismatch between our vision... and the practical implementation”