The Exchanges

Q Yeah, so what does that mean that the data integrity is not there and that's the main thing holding them back?

A I think the big thing, right? I mean, I mean, I'm, I'm probably aging myself. This is the same thing that happened in the 19 nineties and, you know, the, the digital transformation, the biggest blocker that we see with digital transformation is where data is very siloed within an organization. It is not really, really available. And then who has the proper definition of the single source of truth, uh, for that data that exists in an organization. So once you're able to align as an organization on what that is, Models can get better about grounding themselves and providing that level of, uh, insight and analysis that I think agents can then act upon and that provides better outcomes for our clients. And that's what, what we see, you know, uh, when in, even internally, like as a, as a, as a COO at heart still, I find that to be the biggest thing that, because we want to be able to act on it in the most, uh, you know, efficient manner possible. So.

AI assessment note: “biggest blocker that we see with digital transformation is where data is very siloed”

Q Yeah. Very interesting, especially the range of use cases, just in that gold bond example, AI for workspace order, uh, optimization, um, you know, trying to facialize products. Do you have, are the models today sort of equally good at all different things, or do you have a sense as to like what the models are best at and where they could improve?

A I think the models today, I'll, I'll zoom out and give a general answer of what I think the models do very well today, right? Maybe that's probably specific, so I don't upset any product managers anywhere, but, but what I would say the, you know, the models are very good at reasoning, understanding of given context, and it's, it's, it's, they're very good at that. Where I think they can be a little bit better is, and, you know, it's obviously we want to always limit hallucinations and, you know, It's the idea of becoming a little bit more, uh, it's just like people, I would say, is becoming more self-aware, right? As in, you don't need to give an answer. If you don't know something, say you don't know. It's something we tell our kids and people in meetings, right? So don't feel like the idea of, like, you have to provide an answer needs to probably be minimized a little bit, and that's probably where I think the models in general probably need to get better at in my mind.

AI assessment note: “models are very good at reasoning... Where I think they can be a little bit better”

Q Right. And so then as the AI models have gotten better, that's allowed you to do more of that or what exactly have the model improvements translated to?

A Like one of the things I will say here, right? An example of this would be your, uh, you know, uh, we are talking about before about how we have, uh, you know, internally to our organization, we are asking our people to be. 30% more effective, right? I don't, I never use the word efficient because efficiency means like all kinds of weird things and people are like, oh, what's about my job? It's not about it, right? I mean, like the idea is, okay, if you got 30% of your time back, you know, let's take our finance team in our organization. We know they cannot take PTO the first week of the month because they're closing, right? It's impossible for our CFO and our financial controllers and our AFP and A team. So what we have challenged them is how can we work in a way in which a lot of the tedious, mundane tasks that's working in our own bespoke billing system that we have in there, How can that be automated so you can actually take PTO off, right? So that to me is what, you know, the agents that we are building and, uh, you know, sitting on top of Looker or BigQuery, you know, that can action the mundane and gives time back. Whether we use it for PTO or taking time off that much deserved or do something else, investing in your career is what we talk about when we talk about the gains that we want to make in people's growth and in our company's growth. That's what, that's what inte…

AI assessment note: “the agents that we are building and, uh, you know, sitting on top of Looker”

Q a little flat footed at the beginning. Yes. Everyone knows. I think theoretically that things have gotten a lot better, but from your perspective, just talk to us about the tangibility of this stuff. And, uh, when we hear, okay, it's gotten better. What have you seen on the ground? The models have gotten better at handling complex, uh, queries. They're hallucinating less. Talk a little bit about the progression.

A I think the progression, the big progression, right, it, I think, is going from a novelty, like, you know, hey, give me answer some questions, do some basic stuff, to where we are right now, I feel like with, with the advent of agents at this point, where you can do agent assembly lines, the models have gotten very good, you know, you know, very close to being good at, ah, discerning, discerning, there's some, still some work to be done in terms of hallucination, And, but Google's done a very good job of grounding, you know, the models providing visibility into it. And that's the reason why we are super excited to partner with them. And look, there's still some work to be done here, right? I mean, you know, like, and, and this is not just a Google only race, right? I mean, today the models are at a point where next, you know, you're going to get an announcement with open AI. There's going to be the, you know, there's going to be leaps that are happening back and forth, which overall, I think like one of the things we like to say internally is, This is as worse as these technologies is going to be. It's only going to get better as things go, and a lot of this still we feel like is grounded in the fact that organizations still struggle with data integrity, and that is fundamentally what is the primary blocker, right, in terms of like, you know, really AI, I mean, models and agent…

AI assessment note: “going from a novelty... to where we are right now... with the advent of agents”

Q What does that mean? That they're making better decisions?

A Well, they're getting better decisions based, based on the information that's available. Right. And, you know, and I would say that is, you know, especially with Google, uh, you know, where, with, you know, with the models have gotten very efficient, very, you know, and then there's a level of transparency that's in there, which allows, you know, have a vision to help the thinking process that happens in that we are able to see it in, you know, inside what is happening. What the decision is, and you're able to tune how much of the hallucinations, you know, you can tolerate, what you don't want to tolerate. I'm obviously, you know, speaking in very basic terms here, right? There's a lot of, you know, you can, data sovereignty is a big thing, so you don't want to go outside, you know, like things that you don't, you know, want to base decisions on, and that level of, you know, um, you know, control that is given to the user, Has gotten, at least the Google models have gotten to be very, very effective. At least internally, that's what we see, and that's what our clients are saying.

AI assessment note: “they're getting better decisions based, based on the information that's available.”

Q you know, survive this moment. And I think that, that to me shows that there's, there's been this moment where there's companies like yours have taken the AI models, used the improvements, and translated that into Real business outcomes. That's happening now. So I just want to hear your perspective. When the model gets better, how does that translate into your ability to do more with a, with a customer?

A Well, um, I, our CTO, John Pettit and I, we spent a lot of time thinking about this and talking about this. And, um, one of the things we believe, right? The idea of the AI platform, right? It's getting closer and closer. I mean, I know, um, you know, I'm going to coach Satya, right? We'll talk about, you know, the idea like a few years ago, I talked about the idea of, you know, These, uh, platforms like CRM platforms that sit on top of data that exist in your organization, and that's what we talk about, uh, AI platforms that are on top of it, and how we work is going to fundamentally change, you know, in the, I really, I think we are, and I'm not trying to, like, um, you know, scare anybody to think that PowerPoint will go away, but you are seeing PowerPoint, the people whose alliance on PowerPoint is less than it used to be, right? And, you know, I, we feel like, like, as a, as an operator of companies, most people, we don't want a plethora of AI tech tools out there. Neither do we want to have a IT sprawl with a whole bunch of, you know, um, platforms that exist. What we want is the ability to action and do our work, you know, you know, with the data that's already exist in our organization, and how do we interact with it, and how do we make decisions on it? And I feel like that's where we are right now, and, you know, I don't know what the timeframe is, but I feel like it'l…

AI assessment note: “What we want is the ability to action and do our work”