The Exchanges

Q AI lab leaders, um, like Dario from Anthropic, something that he said is like, you know, we are starting to, um, mirror open source in a way where that customization is possible and the bringing the data in is possible with our, our off-the-shelf models, meaning clawed at Anthropic. So why go open source and customize? Is it just a greater degree of control or What do you get there?

A Well, I mean, when we say greater degree of control, we're actually saying a greater ability to improve performance. It's, it's not control for control sake in that sense. It's really about the performance. Let's go back to chat concierge for, for a moment, right? Like terms of it, what are the different, uh, steps that like something like chat concierge, uh, uh, goes through, right? Um, even simple interactions, you want to confirm, uh, the needs with the user. You then want to simulate The plan that you're going to work against. You want to validate that plan, make sure that it's, it's right, etc. Each one of these steps, ah, is a combination of both the, the data that is required to improve the execution of the, of the step, but also, ah, kind of the reasoning capability and the interaction with the user. How does the business want to execute it? You may have certain things that you say, if these happen, I want With this to be, this to go to my, to my human in the loop, ah, in that thing. In other, some other people may have different things. So there's a fair bit of customization both of the UX and of the, ah, performance itself, right? In this context, ah, what are things we care, one of the things we care about most is speed or latency, right? You know this from your favorite home assistant that you use. If it takes three seconds to respond to you, it's way less satisfyin…

AI assessment note: “when we say greater degree of control, we're actually saying a greater ability to improve performance.”

Q So are you building ground up or are you using this data with off the shelf models? Because my impression is this was a ground up build.

A Yeah, yeah, yeah. We're definitely building ground up. Like I said, first we went all in on the cloud. So we, we basically build on top of the cloud, but when it comes to, uh, building platforms, we, we use the, the core services on the cloud, the elastic compute, uh, the elastic storage and all of that, and many other Um, you know, baseline services, but then we build our platforms on top of that, of those core cloud services. So in the case of before AI, in the case of just classical machine learning models, um, we built our own, uh, enterprise platforms for all of our modelers and data scientists across the company to build their models, and these platforms work in the context of our data, uh, platforms and our software platforms, uh, to unlock You know, modeling at scale. Uh, we've brought that same thinking. So we've built our own platforms. We've built our own reusable services on top of that platform. Uh, and these reusable services are both like patterns that developers across the company can leverage to build AI powered applications. But there are also other important things. I mean, we are a bank. We lead with risk management as a central thing. In fact, hey, let me show you. Look at this bottle. I drink water out of the risk tech bottle, right? And so the, um, we lead with that. So all kinds of observability and monitoring, uh, capabilities are built in the platform.…

AI assessment note: “We're definitely building ground up. Like I said, first we went all in”

Q I really like. I wonder if I qualify for financing. And so I hop onto chat concierge and tell it, uh, you know, a lot of my attributes, like how much I make, where I live, uh, you know, whether I've gone bankrupt in the past, and then it will, you know, tell me, tell me whether I qualify for a loan or what is, what does it do exactly?

A We live in a very online world, as you know, Alex, today, right? So our customer experience doesn't start when we walk into some place. Our customer experience starts when we, when we look that business up on the web and, and say, what do they have? So chat concierge starts, uh, online, uh, when customers are looking for cars, they can say, what kind of vehicle are they looking for? And cars are so central. One of the things I learned from, uh, Sanjeev Who heads our, I mean, inspiring leader who leads our auto division, was how central cars are to people being able to live full, uh, economic and successful, productive lives in the U.S. It's just incredible. So I think there's a mission focus here on making sure we can, you know, help people get in their cars. Uh, but the experience starts online. Uh, so people will go to a dealer website, And they'll say, these are the kinds of cars I want, and they might interact and say, okay, what's the size of your family, what kind of trips, this, that, and then you come. All of that experience is now fronted by Chat Concierge. Because it can, it can actually engage with you in an interactive understanding of your needs. You know, usually we think of it as just what is the customer's intent, like in the classical chatbot, and then let's go fulfill the intent. We're now in this paradigm where you say, we would really want to understand your…

AI assessment note: “All of that experience is now fronted by Chat Concierge. Because it can, it can actually engage”

Q uh, help bring its data, uh, to the models to be able to sort of, uh, improve the function of these models overall. So let's go back to this experience again with the chat concierge, uh, where people are chatting with it, uh, on dealer websites, maybe to find the type of car they want. Where does the data that Capital One has come into play on, on this experience?

A So on this, it's really, uh, you know, as I said, we started off with a, with, with an application where kind of the risk surface is relatively small, uh, and the interaction is large. So as you're interacting with the users, what are the kinds of questions they ask, right? Um, and where, where do some of these conversations, especially historically, where have they been frictionful, right? Etc. Uh, and how would we address them In a more flexible, more capable paradigm. That was one, um, there was one set of data, uh, that we have. The other set of data, of course, is actually all the inventories for the cars and what's available, et cetera. So in that sense, uh, because this was the initial beachhead, uh, for us to prove out, uh, this capability, we were able to bring both of those. There's also an important learning, uh, aspect to all of this, uh, Alex, which is, You know, building something in the, ah, in your engineering environment and then taking it to production is one step of it. Ah, what I've come to recognize is that's the first step in the AI stairway to heaven. Right? A lot of the action, a lot of the learning is actually in ascending through the rest of the staircase. Uh, you know, what are your post-production learnings? What do your observability and monitoring tools tell you about where the customer might be experiencing friction? That data comes in. How do we …

AI assessment note: “The other set of data, of course, is actually all the inventories for the cars”

Q So, okay. You mentioned that this is a beachhead. Um, so talk a little bit about what the shore is like, uh, what's the ambition here? If this works well, then what comes next?

A The ambition, so what comes next will be really a slow and steady, uh, again, we slope through risk. I mean, our ambition is, of course, to use this as broadly and as effectively as possible, uh, across a wide range of, uh, across a wide range of, uh, application areas. But, um, our initial sets of ambitions around these things are how do we empower our associates To deliver kind of the best, uh, services we can to our customers. One of the, uh, first applications, for example, actually the first application we developed started developing back in, uh, was Agent Assist, right? We have a lot of, uh, customer servicing, uh, interactions that happen on a daily basis, and as you know, when you call in, There's always a little bit of a wait time, right? And, and when you call in to the call center and you've been waiting for, the agent, the human agent who picks up the call is very aware of the fact that you've been, you've been waiting and that you're in a heightened state of, you know, anticipation of a quick and correct answer. And so, but then they have to deal with the systems they have at the time to find the answers Sometimes the answers come from multiple different sources. They have to synthesize the answer and, and give it to you. And then, you know, we being customers and, you know, impatient are often like, you know, I need the answer quickly. What happens, therefore, is…

AI assessment note: “our ambition is, of course, to use this as broadly and as effectively as possible”

Q Last question for you. So you were on the ground, like I mentioned in the, in our opening, uh, with Amazon Alexa, uh, the very beginning there. Um, just tell us a little bit about what it's been like watching AI from there to where it is today. And where do you think it's heading?

A Yeah. Uh, Alex, if it's okay, I'll take a further step back. Because I do think, uh, we have to acknowledge, uh, the tremendous contribution of DARPA and the community that it sponsored through decades of its existence, because almost all of the foundational work in AI and machine learning, uh, huge fraction of it has happened with DARPA sponsorship, and so I, having spent a lot of my life in a world that was, that benefited from DARPA, I won't acknowledge it. Whether it's speech recognition, machine translation, even AI. You know, DARPA was investing in explainable AI before other folks were talking about it. DARPA was talking about trustworthy AI before, um, other folks. So I think there's that part to be acknowledged. What I've seen, though, as a trend, and especially where I think the work at Alexa was kind of, um, uh, took it to a new level at the time that it, um, that it arrived, was We were building all these component technologies one at a time, and we're trying to put them together in certain applications, you know, but the scale at which it, it came into a retail experience in everybody's home at one time, and, and became kind of a consumer AI that not just adults, but children could interact with, right? That was the magical transformation. At the time that it happened, right? Which is, you know, I, I used to see my daughters, like, uh, like, you know, they have the…

AI assessment note: “What I've seen, though, as a trend, and especially where I think the work”