The Exchanges

Q And you just mentioned open source, and I think you've said somewhere that, um, 80% of the AI work may rely on open models. Um, what's your sense of the reality in the enterprise today of the state of preparedness and adoption of, uh, open source AI models?

A Today, the, for most customer use cases, Uh, uh, the companies are using the closed, closed models at the moment, like, you know, for the ones that at least, you know, that we are familiar with, you know, as we go and work with large enterprises, and we think about the core business processes that they want to automate with AI. Um, right now, the closed, closed models are the ones that tend to dominate, and I think it's the, it's for the right reason. Like, as you are like in this early stages of journey, and you're trying to build An application where you're going to actually use AI as a key component. This is not the time for you to optimize and figure out, like, what, what's the best model, uh, or, like, what's the most cost effective model? Because, well, like, you know, you're building an application and nobody uses it, so it doesn't matter how much it costs. At first, you know, go and use the best, the state of the art model, which, you know, today are still the close, close, close, uh, close domain models. And so, so, so for that reason, you're seeing that largely people are starting with these larger models, you know, from OpenAI or, or Enthropic or Google. Um, but as things reach scale, let's say you have an application that now has millions of users and hundreds of millions of daily interactions. Now, cost becomes a real factor, and that's when open source models star…

AI assessment note: “for most customer use cases, Uh, uh, the companies are using the closed, closed models”

Q uh, as an industry participant and an observer, uh, what do you make of all of this? And, and where do you think we're heading in terms of, uh, what the AI model world is going to look like in a few years? Are we going to have You know, a bunch of models. Are we going to have a small number of like very powerful model and model companies?

A I think all indications are that there's going to be a very large number of models that are going to be available to us, uh, in the industry. Uh, it's going to be both close, close, uh, models, uh, from companies like OpenAI and Anthropic and Google. Um, but also there's going to be a lot of open domain models, um, Models that are like, you know, for example, Lama, and then a lot of derivatives as well, um, that are going to be available to us. So that's, that, that is already happening. It's just going to, you know, there's going to be more and more of that in the future. Um, what it means for us is like, we, you know, we, we will be in this world where it's going to be hard to actually always keep up with, um, the model technology, uh, Like for, for all of us, I think it will become increasingly more a deeper, uh, infrastructure level thing that then software developers actually make sense of and figure out for, for a given application, for a given use case, what's the right model technology to use. And hopefully for enterprises, for customers, the, LLMs, you know, become a technology that they don't really think about. Uh, it's just, you know, something in the background.

AI assessment note: “there's going to be a very large number of models that are going to be available”

Q wrote off, uh, eight billion, uh, a few months, uh, later. And, you know, Verity, like that, that whole generation. So it sort of feels like it was an industry that, that came and then went away in the early Uh, tense. So what was the, uh, what was the, the, the vision and the insight then that, that the industry was ready for reinvention of it when you started?

A I would say it was not so much a vision or a realization that this was the right time to solve a problem. It was just frustration, you know, personal frustration that I had, you know, which forced me to, uh, start this company. Um, finding, finding information at work has become increasingly, um, difficult over the years because, well, number one, there's so much knowledge that we have in our companies these days. Uh, we live in a data-driven world. Number two, that information has increasingly gotten fragmented as we went through this, um, SaaS revolution. Every business has ended up with hundreds of these, uh, cloud-based applications because buying applications is so simple now. So, you know, businesses will end up with hundreds or sometimes even thousand-plus applications, and all of your company's data and knowledge is spread across all of those systems. And, and my company before we started, um, Clean was one of those companies. You know, we were built in the modern SaaS world. We had tons and tons of information and data with 300 different applications, and nobody in our company could ever find anything.

AI assessment note: “It was just frustration, you know, personal frustration that I had”

Q had automated, uh, fifty million plus agent actions. That was through the, the apps. Is that what you're Referring to? Okay. And then in terms of design of the agentic platform, you went very horizontal, uh, basically enabling anyone to, to build their own agents with, with Glean. Uh, walk us through the thinking you could have gone very vertical per function. Uh, how did you think about, uh, this?

A Since, since we started as a search product, um, where our objective was to help people find You know, any piece of information that they're looking for inside the company, we built this horizontal platform. We built integrations to hundreds of enterprise applications. Um, and we look at data and information inside each one of those systems, uh, so that when somebody's looking for something, we got able to immediately, immediately point them to the right pieces of information. So we built the platform before knowing that it would be used to build, you know, AI agents. But now that we have this platform, which is connected to your Salesforce and your SharePoint and Workday and Google Drive, And all these different systems. If you think about, you know, agents fundamentally, um, agents are working on some of their enterprise information, and applying AI to perform some business logic, and then maybe saving the work, the resulting work that the agent did, again, back into your enterprise applications. So we felt that, like, the agent, in general, they all tend to have these common elements, and, and the Glean platform actually solves for all of those You know, because, um, like now whether you're building an agent for a HR team or finance or legal, uh, all of those systems that those teams use are already connected to clean. So that allows us to actually, um, help our customers bu…

AI assessment note: “reason why we started with the horizontal approach was just timing”

Q different way than the prior ones. So, uh, what does that mean for a company like Lean? Does that mean you need to just, uh, Stop what you're doing, look at the new model, evaluate it, and figure out how to integrate it so that you, you don't end up delivering a, um, you know, a, uh, sort of an unsettling user experience with, like, very different kind of behaviors?

A Well, that's the reality of an AI company today is you have to fundamentally learn how to work in a, in a unstable environment. Uh, this technology is moving so fast, and so, so yeah, so you have to do that. Like, you know, as new models come, you don't have the luxury to not look at them. You have to look at them. You have to see the new capabilities. You have to actually build the right evaluation frameworks to quickly see, like, you know, as a new model comes in, like, we need to have a way within, like, you know, 30 minutes to know how well it's going to do on our product, right? I mean, and you can do those things. You can actually build the right, you know, evaluation frameworks, things like that. Now, the way we, we, um, ship these models to our customers is, one is, you know, we just give them the choice. So as they build agents, um, In Glean, um, or, you know, when they ask questions in Glean, the user can manually choose what agent they want to actually use. Oh, sorry, what LM they actually want to use to solve that particular question or, or that task. As you're building these complex multi-step agents, you can, you know, for each step, you can actually choose what's the best, you know, model to use. Um, so part, so part of it is just like, you know, working with advanced users who can make that selection for themselves. But, but, but that's the minority of the peopl…

AI assessment note: “so yeah, so you have to do that. Like, you know, as new models come”

Q does your enterprise search work under the hood, like between the, you know, the AI models and then other, uh, kind of like heuristics that you've built to get great results? Because I, I should preview this by, by, you know, uh, all, all accounts or like anybody that's used Glean search that the product is incredible, like in terms of quality of results. Um, so how does that work?

A Like maybe first let's understand the overall architecture of the product. So the first thing that you do that you have to do to build a search product is you have to get hold of the information that you want to actually search over, which means in an enterprise, you have to build integrations with, um, all the systems that a business uses. So, so that's, so that's the first part of our core tech stack. You know, we build these connectors. And deep integrations into products like Confluence, Jira, Workday, Google Drive, and Slack, and so on and so forth. Um, now once you have the data, now you have to actually solve a whole bunch of problems. Number one, um, enterprise search is fundamentally different from web search in the sense that the, the information inside your business is protected. Not everybody can see every document inside the company, unlike on the web, where we can all see all the web pages that are out there. So when you build a search product, you have to understand an individual. You have to understand permissioning and, you know, of content inside your company for any document that is out there or for any message that's been exchanged in Slack. You need to know who are the people who have the rights to see that information. And And that is, that has to become part, part of your core search stack. You have to, uh, build, build a secure search experience where kn…

AI assessment note: “maybe first let's understand the overall architecture of the product.”

Q And do you think that, uh, we are there or very close from, uh, a technology perspective, or, uh, do you think there's something missing, um, for the full realization of the vision, whether that's open industry protocols, MCP style kind of stuff? Well, what do you think?

A We are in very, very early stages of this journey. I mean, if you think about agents, like, Theoretically, you know, uh, an AI agent is promising you to do any piece of work. Like, you know, in fact, like that's how when we talk about even our product, um, Glean, um, Like, the way we describe Glean to our customers is that, you know, you know, you can come to Glean, and you can ask any question, um, you know, to it, or you can actually give it any task, and Glean will use all of the world's knowledge, as well as all of your internal company's data and knowledge, to answer those questions or complete those tasks for you. Now, when I say that, like, I'm actually making this assertion that I can do everything in the world. And, and that's, that's not, that's not the reality. Like, you know, today, Um, agents are still, um, quite basic, I would say. Uh, they all still need, you know, a significant amount of supervision. Um, you need to, um, so, like, I would, I would think of agents as more being, well, I think, you know, I have a piece of task. I still own it as a human, but an agent can potentially do it for me. But it'll do its work. I'm going to still go and review it. Uh, we, we barely see any application where, uh, customers are running agents in a fully, uh, unattended, uh, unsupervised, you know, setting. So, so early, but the, but the impact is actually very clear. Um, the…

AI assessment note: “We are in very, very early stages of this journey.”

Q Right. You have a concept, a concept of, um, expert as well, right? So authority is partly derived by the source.

A Exactly. Yeah. You have to look into those factors. You have to look into engagement. Typically, if there's a document that is viewed a lot by people, you know, it's probably, it's probably for a good reason. Like it's probably a high quality document. It's probably still up to date. So you have to observe. You have to actually look into the enterprise. You have to look at how people are interacting with knowledge. Inside the company and, and use that information because, you know, ultimately a good search product learned from humans and, and human behavior. And so that's, that's, so that's what we did. Like, and these are all the different, you know, pieces of sort of challenges that you have to solve. Um, and then language models, you know, play a big role in terms of, Like doing that matching. Like language models are good to, like on any given question, they're able to, especially if you customize and tune, you know, fine tune embeddings, you know, for your, for your own enterprise, it's a pretty effective way of taking any, any question, like, you know, bringing the right information back, but then use all those traditional search techniques to figure out like, you know, which one, which information is the most recent one, which one is the most high quality one, things like that. So that, so that's sort of like, you know, roughly how the search stack works.

AI assessment note: “Exactly. Yeah. You have to look into those factors. You have to look into engagement.”

Q And still on the architecture front, uh, now talking about agents specifically, uh, you could have chosen different, uh, kind of, um, agent archetypes. It could have been computer use. It could have been tool use. It could have been, uh, specialized agents, and you chose tool use. Can you walk us through the reasoning here?

A Yeah. So the, I think those are all actually by the way, ah, techniques that you to use all of them in reality. Like I think we, we don't have computer use support in our, in our Asian platform. That doesn't mean that we're not chosen and that probably means we haven't gotten to it. Um, uh, and we, we, we prioritize based on customer demand. So right now, when you look at the kind of agents that people like that our enterprise customers want to build, most of them tend to be, um, like have this pattern, which I mentioned before, that they want that agent to work on certain amount of business information. Um, they want AI to sort of like do some work on it, and then they ultimately want to go and, you know, you know, save the work, you know, of that agent somewhere. Uh, within your enterprise applications. So in, in this model, um, you know, you have to use, um, to actually fetch the right data, you actually use, you know, the tool-based architecture. Uh, you build individual tools to fetch information live from different systems. You also build tools to, to actually save information back into those systems. And so that's, that's the natural architecture that, you know, actually, in fact, everybody uses that. Like everybody, You know, like any agent framework that you will see out there will have the support to actually use external tools.

AI assessment note: “we prioritize based on customer demand”

Q So as we get close to the end of the time we have, you mentioned that there's a lot of agents, a lot of companies in the space. How do you think about Moat at Glean and how you build an increasingly defensible business over time?

A You know, this is a question that, you know, our employees have been asking us, you know, quite a bit as, as the market heats up, as more competition comes our way. Um, my belief, you know, with, with any startup is you don't think about what the mode is. You just have to work hard on solving user problems. So we work with our customers. They have so many demands, you know, for us, so many things that they want. And I think just working hard and, You know, solving those problems one at a time ultimately allows you to build a very robust tech stack, you know, and, and the faster you build, you know, the, the, like, you know, the bigger, you know, bigger that, you know, technology stack becomes, and, and I think that's, that's sort of what is ultimately the, the moat is, is the hard work over the years. Um, in enterprises, you know, it's sort of hard to explore. It's hard to actually, um, you know, have, like, Network effects like, you know, the kind that you have with eBay. Like, you know, enterprise software doesn't work that way. So, so like, you know, like we fundamentally just believe in that, like, you know, add, you know, do, build a good product, move fast, and, and, and, and that's how you stay ahead of, you know, competition.

AI assessment note: “what is ultimately the, the moat is, is the hard work over the years”

Q reportedly over, well over now, a hundred million AR. You know, meanwhile, you have companies that like show up sort of out of nowhere and raise at staggering valuations, uh, you know, ten billion valuation for one, thirty-two billion dollar valuation for the other, before they even have a, uh, a product publicly market. I, uh, you know, how does somebody like you think about that, that state of things?

A Well, the more the number of LLM companies that are out there, the better it is for, for the industry. So, so we actually like seeing new names. We like seeing new models. Um, and, and in fact, you know, we benefit from them today in a big way. Um, there are, there are so many, uh, so many model choices that we have today compared to last year. Uh, models that are of comparable quality, models that are all getting better at certain things. Um, you know, for example, uh, today for a lot of coding use cases, people tend to use Claude a lot more. It works better for reasoning. You know, therefore they're, they're using GPT and, and, you know, so you, you're seeing these, um, models like, you know, get specialized to specific skills. And so for, for application companies like ours, uh, where we're trying to use this innovation And solve real business problems. Um, the more, uh, innovation that we see, the better it is, uh, for us. Um, and not just from a point of view of that, we have more technology to use, but also, um, it creates a competition and it ensures that, you know, we can actually get, um, uh, the LLM advances in a more cost effective manner that we can then, you know, then we can actually bring our products, you know, uh, you know, in a more cost efficient manner to our customers.

AI assessment note: “the more the number of LLM companies that are out there, the better it is”

Q What's, what's why now? Of, uh, agents from your perspective, why do they suddenly become such an important topic? And, uh, and also your sense of, um, you know, reality of what can be done today versus, uh, hype of what people think they might be able to do in the near term.

A So, so ultimately, like, you know, an agent is automating a business process, um, which was before, um, you know, managed by a human. Like, you know, and so that's sort of like, that's the, like, I'm talking in the context of, uh, agents in, in businesses. And of course there are agents, you know, that you can be using in your, in your personal lives as well. But for, for, for a moment, let's, let's stick to, you know, business agents. Um, so these are agents that are actually taking a specific business process and transforming them using and automating them using AI. So like, take some examples. Um, you know, you could actually, You know, in, in your legal team, um, you are, there's a process of like when you get third party contracts, you have to redline them, make sure they're conformed to your company's guidelines. Um, so for that, you know, today, you know, you have somebody in the legal team, you know, a lawyer who's actually reading these long documents and, and they're going to redline it and, you know, pick, pick all the, you know, all the places in the document where, uh, you need to actually change Change, change it to sort of, you know, change the guidelines to conform to like, you know, your, your company's, uh, uh, policies. And so this process, uh, it, it requires a human. It requires them to actually go through a long, you know, document. They read it. They, the…

AI assessment note: “these are agents that are actually taking a specific business process and transforming them”