The Exchanges

Q Yeah. So how does Carbon Aware Compute work and how has it evolved since when we caught up with Anna, with Anna, it was sort of at its early stages and it's been a couple of years. So how does it work? Where are you at?

A Yeah, so Carbon Aware Compute is a really exciting platform that allows us to shift our flexible workloads, both in time and in space now, uh, in order to have those workloads be run when the wind is blowing or where the sun is shining. Um, so we're now rolling this out across our global data center fleet. Um, and the way that it works, one of the primary inputs that we need is a day ahead, uh, carbon intensity forecast. And we get this data from a partner of ours called Electricity Maps. They are continuously running machine learning models, uh, looking at the historical data they've collected, looking at weather data to generate these predictions. Those forecasts, in addition to load forecasts, which are created internally from statistical models, are put through a robust optimization model, and that optimization model essentially outputs a schedule That minimizes the carbon footprint of our flexible workloads across the fleet.

AI assessment note: “allows us to shift our flexible workloads, both in time and in space”

Q Got it. And so where does that technology fit into the other machine learning and data science tools that you're developing, that you're working on specifically?

A Yeah, so we, um, are looking at, uh, AI and machine learning across a number of different, uh, technologies. So the Carbon Aware compute platform is really internal to data, Google's, um, data center operations, but we are looking at opportunities beyond, you know, Google's footprint. So let me walk through three different examples where we found AI to be particularly useful in the energy space. Uh, so the, one of the, the problems that, you know, we were trying to solve, Um, was to make our wind and solar assets more economically efficient, and, um, in order to do this, we created a solution called Grid Intelligence, and that leverages ML and AI models to generate solar and wind production forecasts using lots of historical data, again, weather data, other factors, and we run this, again, through an optimization model, so you can kind of see there's a pattern, machine learning inputs, optimization model, and that outputs an optimal dispatch Schedule, essentially, or, um, delivery commitments for the wind and solar assets, and that can therefore increase market participation revenue, making the projects, uh, more economic. So we're using that for our own renewable energy fleet, but we're also, what's really exciting is we've started to work with Engie to extend that technology such that they can use it for their own portfolio of assets. Um, the second problem I wanted to highli…

AI assessment note: “let me walk through three different examples where we found AI to be particularly useful”

Q you thinking about them to serve industry? Because one of the big goals, obviously, is if you're going to start decarbonizing the broader system, you have to work with partners, as you said. So, uh, you're working with Anji. I mean, how do you scale from one partner to many, and then across these three different areas, how much does the product development roadmap include getting it out to industry?

A Yeah, no, it's a, it's a really great question. So a lot of, what's really great about Google is we have the opportunity to incubate solutions internally, learn, you know, what works, what doesn't work, work out the kinks, and then bring it to market through our different platforms, like through Google Cloud, or even through some of our, um, geospatial, you know, Google Earth tools, for example. Um, so let me, let me give a few specific examples, actually. One is on the supply side. So we use, We aggregate data sets. We use tools like Google Earth and Earth Engine to help project developers, ah, site their new energy assets, and Google Earth is a open source free tool, and so we've actually seen a lot of adoption on the developer side. Almost all major developers, um, are using Google Earth to help site their renewable energy projects, and so we're aggregating data sets. We're looking to aggregate even more data sets to make that process easier for them, and that can help You know, increase the supply, right, of, of clean energy. So we're aggregating not just the, you know, solar wind potential, we're also looking at geothermal, so subsurface data. We're looking at aggregating transmission line data, load centers, carbon-free electricity that's already on the grid, and bringing those together in a single view, which can help facilitate, um, their development processes and plann…

AI assessment note: “incubate solutions internally, learn, you know, what works... and then bring it to market”

Q Yeah. So we just had a really good conversation about data. I think that is just a theme that is going to come up time and time again. So what are some examples of bad or limited data that you're working with and what would be your optimal data sets?

A Yeah, great question. So, um, I think the good news is there's a lot of data already out there. It's more about standards to make that data easier to use and accessible. Um, and I, I'll give some examples specific to twenty-four-seven. So, one example is we're working with a partner, Energy Tag, to develop a new data standard for granular certificates, which are essentially like hourly recs. At Google, we call them TEKS because we like to have wonky terminology. Stands for Time-Based Energy Attribute Certificates. And TEKS are a really exciting standard because it enables consumers like Google and others to actually make third-party verifiable, twenty-four-seven matching claims. But it also creates, this new data set can create additional market signals for incentivizing new technology development like, like energy storage. So that's, um, you know, one of the data sets that we're working to create through data standards with partners. Uh, for 24 seven, from a consumer perspective, again, another key data set is that hourly electricity consumption data. And, um, there's a lot of challenges with getting access to this data, even, even for Google. Um, in some, you know, in some utility regions, the data is often there, but they just don't give consumers access. In regions where consumers do have access, it's often a manual download of CSVs, so really, really not scalable, especial…

AI assessment note: “another key data set is that hourly electricity consumption data. And, um, there's a lot of challenges”

Q systems, and, you know, thanks to GPUs and TPUs, we have much better, better and faster training. Um, I have been just working on this project on data centers the last few years. I've become completely enamored, enamored with how they work and how they're evolving. Can you talk to the data infrastructure, the data center infrastructure generally, and what it is allowing us to do in training these models?

A Yeah, so we've seen, um, you know, a really exciting uptick in demand for AI and machine learning, and we are, you know, constantly improving and evolving the way that we build and manage data centers to account for this increase in demand. Um, the good news is research has shown so far that despite this uptick in demand, we're not seeing an increase in, a correlating Increase in energy consumption as we had initially forecasted. Um, and there's a lot of reasons for this. Uh, part of the, you know, reason is on the hardware side. So just one example, um, TPU version four is actually twice as efficient as TPU version three. So that means, you know, the amount of machine learning compute that we can run is substantially more for less energy. And just to add another kind of metric there, um, compared to, Five years ago, we are now delivering the same amount of compute, or sorry, five times the amount of compute for the same amount of electricity. So that's a really, really substantial increase in efficiency and energy efficiency is really at, you know, Google's core. And so it's not, you know, this isn't a new problem we're trying to solve. We've always tried to have, um, efficient compute, but, uh, having this in uptick in demand for machine learning is just sort of accelerating those efforts and creating additional innovation. Uh, for us to continue to, to meet those needs.

AI assessment note: “we are now delivering the same amount of compute, or sorry, five times the amount”

Q I want to go back to carbon aware computing for a second. It's a concept that's probably familiar to many in this people, many people in this room. Demand response. Uh, what kind of loads can you shift inside data centers and why?

A Yeah, that's a great question. So not all of our, our workloads are flexible. You know, when you put something in Google search, you obviously expect an immediate response. So there's, you know, some things like search serving load, which we aren't touching. That being said, we are, there are a number of flexible workloads. So one example that you can think about is if you're a, uh, creator on YouTube and you upload a video, there's a lot of processing that has to be done. Um, that is a workload that could be shifted in space and time because it's not really, it's not as sensitive Right? To, um, uh, the, you don't have to, you know, display something right away, for example. And so there's a lot of number of different workloads. A lot of them actually are on the ML and machine learning, or ML and AI side. So we're right now really taking a lot of time to evaluate the amount of flexibility for ML and AI workloads. And, um, we're continuing to see how we can increase the amount, the proportion, right, of our workloads that are flexible, such that we can Increase the capacity of this flexible demand, not just to minimize our carbon footprint, but to also support the grid through things like demand response.

AI assessment note: “YouTube and you upload a video, there's a lot of processing that has to be done”

Q So we have these inherent, um, physical and business model constraints that we've heard, uh, in different forms today. Uh, but you're sitting inside, you know, one of the largest tech companies working on really sophisticated tools. You have this window into how this will transform industry potentially. How transformative for energy do you think these tools are generally?

A Yeah, so I think there's a lot of opportunity for transformation here, and I can maybe walk through a couple of specific examples, again, from the corporate energy buyer perspective. So we think about energy data and software into three core use cases. The first one is, again, what we've talked about, right? Measuring emissions and tracking progress towards your goals. There's a lot of opportunity here to work with the, you know, amazing and brilliant startups that are out there to help aggregate those data sets, Automate those calculations and make it more scalable. Um, I think that I mentioned already to the opportunity to streamline the corporate reporting process as well as the third party auditing. That is right now actually a huge pain point. I think transparency is really important in this industry as we move towards net zero. And so, um, you know, that's a really big opportunity for tools to really transform and, and, and change the, the measuring and tracking piece. Um, the second kind of core use case is planning for your energy portfolio, keeping in mind the changing grid forecasts. And this is something that companies usually will pay consultants to do. They don't really have a good handle on. And I think there's a really unique opportunity here because there's so, there's been a lot of development actually in open source grid models. So for example, Gen X that was …

AI assessment note: “I think there's a lot of opportunity for transformation here, and I can maybe walk through”

Q would you characterize the trajectory of AI in the applications that you're working on? So everyone obviously is looking at this AI arms race and large language models and, uh, amongst the tech companies, but you're working on some of these other, you know, specific concrete tools, um, that are relevant to the people in this room. How would you characterize where those are going in terms of their sophistication?

A Yeah, I mean, I think we're going to see a lot of improvements in these tools, whether it's, like we've talked about, improved forecasts, you know, going back to the Project Sunroof example, increased coverage, right? We know electricity markets are so fragmented, but, and we know that in order to run a lot of the exciting applications, you need good data, and so ML and AI has the potential to bridge that gap, right, where there's, where the data isn't necessarily available or accessible, It can really help spread these applications to new geographies that we hadn't initially thought would be possible without the data. Um, so I think, yes, there's, there's a lot of potential, but we are, you know, carefully evaluating where it makes the most sense. I think LLMs in particular are very, very hot right now. Um, I do think there could be some interesting opportunities. For example, one, one, uh, pain point that's very specific to corporates, I would say, is Third party auditing of our sustainability reporting. It's a very manual process. There's very rigid rules. It's, you know, if you read even our environmental reports, there's a lot of language in them, and it feels like there's a way that that could be streamlined and maybe, ah, yeah, made more automatic with, with LLMs.

AI assessment note: “I think we're going to see a lot of improvements in these tools”

Q So in terms of, um, some of the applications, let's think about forecasting, for example, with many of the projects that are serving corporate campuses or data centers, um, do you have a forecasting problem, or is it just you want to improve it? What, what, what kind of improvements do you imagine, and what are you actually trying to solve relative to what exists in the market today?

A Yeah, that's a great question. So I think one thing we want to keep in mind, um, is that in the, for, for Google, energy is a huge part of our, you know, operating margins, right? Like we spend, we think a lot about energy, we spend a lot of money on energy, that's why we have a twenty-four-seven goal. So even small improvements, right, in the way that we manage our data centers and we manage our energy operations can have a really big implication On the bottom line. Um, and I think in the grid intelligence solution that I mentioned earlier, the same thing can be said too for solar and wind projects, right? So even a 10% increase in revenue because of the improved forecasts can make or break, you know, new projects that then become economically feasible.

AI assessment note: “even small improvements, right, in the way that we manage our data centers”

Q in the nineties, people thought the data centers or the internet generally was going to consume 10% of, um, of power, but it was going to account for 10% of power demand, but like in the reality is around two percent, um, because of some of the improvements in, in PUE and data centers. And so, um, what do you make of the question around energy use of AI generally?

A Yeah, I mean, I think, like I said, overall, we, we do see that AI can be more helpful than harmful in the, the energy transition, and the reason being, you know, there's numerous applications that we've walked through. I think there's lots of applications that haven't been fully figured out yet, and because there's a lot of benefits that are directly, you know, reducing energy consumption or reducing the carbon footprint, um, you know, for consumers or different applications within the grid, The, and that paired with the drastic improvements in energy efficiency, um, you know, we, we really think that this is just an opportunity here, and that, again, AI can be more helpful than harmful for the energy transition.

AI assessment note: “we do see that AI can be more helpful than harmful in the, the energy transition”

Q So how do you separate, like, machine learning from other data science tools, and, like, why would you choose machine learning over other forms of data science?

A Yeah, and I think in this case, you know, we, we want to be very critical about what we use machine learning for versus other data science tools. We see it as a tool, you know, a tool within the broader tool set, um, including other things like optimization, data engineering, those kind of things, and so we're looking for use cases that are particularly well suited. Um, some of those, you know, what we've heard a lot today, too, is not only does the use case have to make sense, there has to be enough data. Um, you know, as one of the, the great axioms of modeling, right? Garbage in means garbage out. So if you don't have the appropriate data or a high enough volume of data or clean enough data, machine learning may not be the best tool for you to use in that instance.

AI assessment note: “if you don't have the appropriate data or a high enough volume of data”

Q So, 2016, Google says we're an AI first company. Um, what does that mean for product development as you think about achieving these goals? Like, what does that do to the, to the questions that you have to answer as a team, if, in an AI first company?

A Yeah, so we're, we're really excited at Google, of course, about the potential that AI has across many different industries, including energy. Uh, that being said, we want to avoid AI being a hammer trying to find a nail. And so, for us, it's really important to find those use cases that are well-suited for these particular tools. Uh, one thing, too, I want to emphasize is it's, it's not just about having really good technology, it's also about having a solid go-to-market strategy. Um, you know, Not only do we need to make sure that these AI tools are solving real pain points and needs, we also need to make sure that the technology can be embedded within an organization's, um, existing infrastructure. So that does take, you know, a broader, uh, digital transformation. And that's why, you know, efforts, that's why AI and Google Cloud, those teams are working so closely together at Google, because you really need that digital transformation piece. To actually bring the technology to markets and, and enable the, the real value.

AI assessment note: “we want to avoid AI being a hammer trying to find a nail”