The Exchanges

Q Okay, great. So maybe bring it to life for us. If I'm an agent and I interact with ASAPs technology, what do I do? What's happening?

A I'll give you an interesting example in the messaging side. For example, if you're chatting with a company, it takes on most industries, the agent roughly 20 seconds to type a response to whatever utterance you've sent their way. If I can predict what you should respond and instead of having to type that sentence, you just click on it, it takes, it goes from 20 seconds to roughly a second. And then the next logical question is that sounds wonderful for how many of my agent responses can you actually predict the right thing where they're going to go from clicking something that takes a second versus typing something that takes 20 seconds. And today in our most mature customers, roughly 80% of everything an agent does is click on those suggestions that the system's generating. So it's, I mean,

AI assessment note: “instead of having to type that sentence, you just click on it”

Q As I think about the AI for customer service, I always wonder how one measures the results, because as you said, the, you know, several times you have loved you three legal tools. So I guess it's agent satisfaction, but ultimately what matters is our satisfaction as end users. Like, how do you make sense of it?

A Yeah, I think that there's really two primary drivers of value, and depending on the company, they'll think about them somewhat differently. One is cost. What is my There's a little cost for having an interaction or serving a customer, however defined. The other one is customer satisfaction, however defined NPS, CSAT, whatever. And those two variables have always been a trade-off. And I'll, I'll, a simple example is pick your favorite cable company. I can't name them now because a bunch of them are customers. So I gotta be very diplomatic with the statements that I make. And they've been trying to pinch Every penny imaginable out of a multi-billion dollar call center operation. And as a result of that, they've made that cost optimization decision at the expense of customer experience, meaning some of these companies are famously bad at treating you. On the other hand, a company like Zappos is famous for being quite delightful at treating you. You can call Zappos today and say, I'm going to Vegas tomorrow. I actually have bought no shares from you, but I'm going to Vegas tomorrow. And I was wondering if you have a favorite bar and they'll actually help you. So obviously that is very delightful. It is extremely expensive. So most companies had to figure out, am I going to be cable company X or am I going to be Zappos and where in that trade of do I want to be? I think the power o…

AI assessment note: “there's really two primary drivers of value... One is cost... The other one is customer satisfaction”

Q Are all those customers cloud or do a lot of them I read somewhere that you have a concept of agent model as opposed to language model. What does that be?

A Well, if you think of what an agent does, they really do two things. They communicate, they're having a conversation with a customer, then they do things based on whatever that conversation, you might be having a conversation about moving to a new address, and then the agent needs to go and update that address on some CRM or whatever system. So they have a verbal or a language workflow. Which large language models can do very good jobs at predicting language and things of that nature, but they really can't do very much on the action space, especially in that action space, not API driven, but it's literally a UI workflow that an agent's doing. So we began training models that are essentially embedding all the workflows, the actions that agents are taking so that we can make predictions of respond this and go take this other three steps in that system over there. And that's That's essentially what we mean by Asian models because language alone doesn't solve the problem.

AI assessment note: “That's essentially what we mean by Asian models because language alone doesn't solve the problem.”

Q Doar, legendary VC at Kleiner Perkins, uh, famous for many things, including leading the series at, at, at Google. So first of all, especially for those first two, how did it come about? I mean, 10 years ago, I, I assume you were not that, you know, many years after college or school. Like how does a young entrepreneur manage to convince those Industry legends to come on their board.

A Well, one, a gigantic amount of good fortune and luck, and two, when presented with that good fortune and luck, that's to that purpose. What are we doing and why does it matter and how are we going to ensure that we're successful at it? And I think we were from a early stage capable of articulating two things. One, the immediacy of this problem of how enterprises interact with their customers. Is arguably the largest problem in enterprise technology. From a data perspective, there's no money being spent in call centers and there's money spent in cybersecurity storage and databases combined. So it's just a gigantic amount of money. It's, it's fundamentally unsexy, right? So it's kind of like, really? Yeah, it's gigantic. There's no call center workers and truck drivers in the United States. So very large. The second thing, which helped with someone like Doris, pretty technically sophisticated and Not necessarily interested in, in, in just some discrete problem, no matter how big it is, is the idea that building AI products that can be deployed to a large number of people, and by virtue of how you build them and who uses them, those products can be improved in somewhat autonomous ways, ends up being a really interesting long-term vision for us, where we're starting with this call center problem, but with what we're doing is building products that automate and augment a given set …

AI assessment note: “one, a gigantic amount of good fortune and luck, and two”

Q So let's dive into the product and maybe starting at a high level from a design philosophy perspective, you all do agent assistance as opposed to agent automation. So it's sort of a co-pilot metaphor. Is that the right way to think?

A We actually do both. When we started, our product vision was Pretty simple and straightforward. There are some interactions that you can automate, try to automate them as best you can, and whatever you cannot, and that interaction ends up on the lap of a human agent, try to make that agent as productive as possible. Because if you make that agent two heads more productive, it's the same as having automated half of those interactions that way. And by the way, while you're augmenting an agent by suggesting what that agent should be responding or what they should be doing. You start to collect a very interesting data set of the supervised usage of whatever model predictions you're making by the agent using or not using those suggestions. So our approach is automate as much as you can, augment the rest, and the more you augment, the more you can actually automate in a bottoms-up way, not in a rules-based way, which is traditionally how things have been automated in this space. So from a product perspective, we started with that vision and we built it first into an end-to-end messaging platform. We have the observation that when you look at how we use this little things, it's primarily for asynchronous messaging. When it's time to interacting with a company, either those channels seven years ago did not exist or they were pretty bad and adoption was less than 10%. So we said, why do…

AI assessment note: “We actually do both. When we started, our product vision was Pretty simple”

Q right. Switching to go-to-market. So you mentioned you work with some of the world's largest enterprises and I saw JetBlue, American Airlines, Arts & Young, and a lot of others. For the entrepreneurs out there and, you know, salespeople, go-to-market people out there, any lessons learned selling to those giant customers and those long sales cycles? Like as a, as a young startup, how do you make sure you don't

A Well, the first one you touched on, which is you introduced me as saying you were on stealth mode for a very long time. That's a very, very stupid idea. Do not be on stealth mode. There's very seldom good reasons for doing it. So I strongly advise against it because reality is if you're selling to the global 2000 or whatever, odds are there's, I don't know, two, three, five people at each of those companies. Who's a potential champion or a relevant influencer in that pursuit. So there was a universe of, I don't know, three to 10,000 people that better know who you are and what you do and why they should take a conversation from you. And if you're in stealth, you're making, unless you have some magical, violent moment of people talking about you by virtue of being stealth, which I think is what we were trying to do in 2015 and 16, which worked for a very limited amount of time. Reality is, at a minimum, go-to-market is as important as everything you do on the technology side, and it might be in many cases more important.

AI assessment note: “Do not be on stealth mode. There's very seldom good reasons for doing it.”

Q So you mentioned earlier that, uh, you built a, uh, research first organization for AI. Any lessons you can learn about how you Do that? Like, especially today in a context where I tell it is so hard to get, how do you recruit them? How do you convince them? How do you retain them?

A Great question, because it's such a relatively small universe of people that are in such high demand that, that it makes a question quite relevant for those who try to do it. And by the way, there's another value, which is some of these people in large tech have athlete type compensation in some cases. So I think it starts with purpose, right? I've had conversations with people That we're considering, say, a fresh PhD graduate from a top program somewhere that's considering, do I go to Google Brain or do I come join your startup? And something that I've many, many times said is Google Brain is going to pay you more. You're probably going to be surrounded by excellent people and you're going to do interesting work. What my problem would be with that choice is whether you go there or whether you don't go there, nothing will happen to Google. Absolutely nothing. And it doesn't mean that you don't go there and you write the next transformer paper, whatever. But reality is your impact is extraordinarily small despite amazing work. Whereas in any startup, our startup, an individual can have a singular impact on the trajectory of the company. So there's some people who want the comfort and all the good things that come with being in a large technology companies and others that like the purpose and mission and the impact that they can have, I think, end up gravitating to, uh, our place…

AI assessment note: “So I think it starts with purpose, right?”

Q Maybe walk us through the state of the market from a technology vendor standpoint, pre-AZAP, if you're one of those gigantic companies with lots of customers, lots of agents, what do you currently use today?

A Well, what's really interesting, I'll give a little bit of context. It is a fascinating problem where you can think of the problem as a three-legged stool. One leg is the company enterprise. Another leg is the customer that has to interact with that enterprise. And another leg is the agent who works for the enterprise. And in this three-legged stool, all three legs are fundamentally broken. Companies that spend billions and billions of dollars really dislike that economic reality. And they're generally spending it for the privilege of that second pillar of the customer, hating their guts when it's time to interact with them. And then agents have an average attrition rate of about a hundred percent per year, depending on the year. I think in the U.S., most estimates have agent counts about three to four million agents. That's more than there are truck drivers and now imagine them at trading every year and the operational nightmare that it means for these companies. So back to the question, the technology, the most interesting thing about the technology, one is fragmented, B, most of the legacy incumbents are overwhelmingly underwhelming technology companies. I mean, the best and brightest engineers have historically not gone and work at Nice Systems or Genesis and I'm naming names By the way, some of these companies have built very successful and large companies, but from an inn…

AI assessment note: “the best and brightest engineers have historically not gone and work at Nice Systems or Genesis”

Q And then you have auto-assist, auto-summary, auto-transcribe, what do all of those?

A So transcribe is speech recognition, and it's primarily speech recognition for the contact center, which has very, very different acoustic properties than general purpose speech recognition like you can buy from Google or Microsoft or Amazon. A lot of those models initially were trained on data from the assistant devices, and these are devices that sit in Your living room and the acoustic properties of our living room are such that they're not so noisy or complex. The contact center is quite tricky because there's two somewhat random variables. On one hand, you have the agent that is oftentimes a very loud and noisy environment. Then you have the customer, which at any event point can be in any number, can be driving home and with highway noise, can be walking down Fifth Avenue. So, solving for that specific problem makes the problem a little bit trickier. And why we decided to build our own rather than use third-party ASR.

AI assessment note: “So transcribe is speech recognition, and it's primarily speech recognition for the contact center”

Q So in particular, you've been doing this since, which was two years after the resurrection of deep learning or the acceleration of deep learning after a, so you've been at this for a few years and in particular before Transformers and then before GBD-II. So what did you build then, and how did you think about the evolution and bringing us to the best of what's going to come out?

A So one interesting thing about ASAP is early on when we came up with this vision of automate and augment and do that seamlessly, and as you augment, leverage that data to increase your automation, we realized quickly this wasn't a product vision that you can grab components from different pieces, put them together, and build a product. There were a bunch of Questions, primarily in fundamental research questions in natural language processing, some in theoretical machine learning, that we needed to sort of move the needle forward if we wanted to make this product vision come to life. And that's why very early on, we started building an AI research organization and we were very fortunate in the people that were able to attract and the critical mass that we built in our organization. So to ask you a question, because we had that team from. 2015 to effectively now. the vast majority of the quote-unquote under the hood AI capabilities that we have in our products have been built internally. Because we have this research organization end up doing fairly low-level work where in early 2017 most of our models were trained on what's called a single recurrent unit, an SRU, which is a neural network architecture that we developed in-house and then we published and at some point it was sort of competing with transformers and Then we hired the guy who ran all NLP research at Google, who the …

AI assessment note: “most of our models were trained on what's called a single recurrent unit, an SRU”

Q And there's a natural tension when you have a bunch of very smart AI researchers between research and supplied research, and people want to write papers and open source, but you need them to build the stuff that you can sell. How do you manage that? And how do you measure the output of a research organization?

A Yeah, I can confidently answer because I have scar tissue on that topic. And I think it starts with, you have to be Extremely upfront about what you want from people. If you want to be a academic lab in industry, by all means, go for it. If you don't want to publish papers and you want to be building products, be upfront about it. So I think just being clear about what you want people for is a fundamental requirement. The second one is, and this is something that, that I think all companies generally fall into the trap of The, the kind of the hierarchy of what team am I in or what my role is or what my title is. And the more that you can suppress it and, and, and kind of remind people we're here, there's no such thing as individual success. There's no world in which I'm successful on ASAP and Andy's not successful on ASAP. And by the same token, if he's successful, it means we're adding a lot of customers and I'm also more successful. So the recognition of this is a, uh, it's a team effort and a team sport, and, and we all play different roles on that team. Uh, I think is, is one of the best pieces of advice I was given by the CEO of a very, very large technology company was, you're gonna come up with your version of this culture thing, and you're gonna articulate it, and you're gonna get in front of your company, and then you're gonna say it, and hopefully you were well prepar…

AI assessment note: “you have to be Extremely upfront about what you want from people.”

Q What does a go-to-market organization look like today? Are you sort of a classic sales, marketing, customer success, see anything there?

A So we have Frank Slootman from Snowflake on our board. So customer success is a touchy feeling with someone who wrote a book and there's a chapter that starts with every time I join a company, I fire the entire customer success department. So we renamed it essentially. Uh, no, I'm joking, but, uh, the, we were as someone who did, I mean, it's, it's really nice. So today ASAP is about 350 people. I would say 80 plus percent of our organization has research, engineering, and product, or all technology roles. So the go-to-market side of the house is one that we started building much more recently than the historically predominant technology side of the house. Andy there runs our financial services. So we've recruited people like him. I have long tenures at companies like EMC and all this classic Enterprise powerhouse at selling and, and they come with pay and understanding of what it takes to win deals with large organizations. In many cases, you can get that plus relationships. That's a good thing. But reality is, I think every startup will go through a period of maturity in which they go from predominantly founder led business development type sales motions To actually the imperative of having to build a, a sales machine, which is a machine that produces without the founder perhaps being involved in every sales pursuit. And I think that's the aspiration that all of us should hav…

AI assessment note: “go-to-market side of the house is one that we started building much more recently”