The Exchanges

Q So Glean, I read somewhere, one should think of it as build.com, but with a brain. Is that the idea? So meaning that you build both the sort of core workflow infrastructure of build.com, but also add AI.

A Yeah. We took a first principles approach to say, listen, if we're collecting all this data, what else can we be doing? So in addition to standard, what's called like AP automation, which is bill comes in, invoice gets extracted. And then you can sync it to your accounting system and get it approved and get it paid. We were like, all right, well, how about we do vendor intake? So if a team wants to bring on a new vendor, let's have a flow for that and, and, and support approvals for that. If you want to set up a budget for your vendors, let's set up monthly budgets. So when the bill comes in, we're comparing it to a budget, not just to last month, but we can get an alert that day that you were over budget. And every bill is like, you can conduct a variance analysis to any prior bill. To see what changed. So there's a lot of functionality that, again, took like a first principles approach to say, how do we just be much more strategic in terms of managing and optimizing our vendor relationships and spend?

AI assessment note: “Yeah. We took a first principles approach to say, listen, if we're collecting all this data”

Q Yes. So how did you get started? What was the first thing that you built and when did you start the company?

A Yeah, so I had the idea in late 2019, we had our, like, pre-seed round in early 2020, like, right as COVID was kind of happening. We started building just like models to ingest documents. So from like a, an AI perspective, what we need to do is when we receive a document, like determine what type of document is it? Is it an invoice? Is it a receipt? Is it a billing statement? We can gather intelligence from those types of documents. If it's a contract, maybe it's like an NDA, maybe it's a, it's like a remittance slip. That's not something that Is going to get processed. So we have to determine what type of document it is first. Once it's a document, like we know it's a document that we can analyze, then who is the canonical vendor? So we have to build models and this is all done with like NLP models, like back in the day before like LLM existed. But like who, you know, who's the canonical vendor? We had to differentiate between Google ads versus like Google workspace versus Google cloud and like all the different taxonomy that exists with that. At the vendor level, then, you know, extracting all the elements off of a page. And it could be invoice date, due date, all the various fields. But the thing that made us very unique is we're extracting all the line items too. So what are you purchasing? For what dates? What's the unit price? What's the quantity ordering? And, you know, …

AI assessment note: “I had the idea in late 2019, we had our, like, pre-seed round”

Q And so you build those models initially, and then you added some LLM, commercial LLMs. I mean, I, I guess, how did you evolve the, your machine learning stack as this whole generic AI popped up?

A Yeah, initially we started with using like OCR vendors on the extraction piece. A lot of the mapping models were, were models that we developed in house. And then over time, We saw that the predictive value of the OCR vendors were no longer adding like the information value we can get versus like an ensemble model of like proprietary models were not worth the cost of maintaining those relationships. So it became like a hundred percent proprietary again, like an NLP ensemble model. But over the course of the last 12 to 18 months, we've done a hundred percent shift to LLM modeling for the complete stack. And we're using Vertex and OpenAI. And we're starting to experiment with Claude.

AI assessment note: “we've done a hundred percent shift to LLM modeling for the complete stack.”

Q Obviously for the world of finance, you know, bills that need to be paid and all the things like that, that's pretty exact. Like you'd better sort of get it right. How far can you go into automation? What is the role of the human as a reviewer or otherwise?

A Yeah. So I'm still learning from my team on some of this stuff. One of the drawbacks of using LLMs is like, we're not receiving confidence scores. On the extractions. So like one thing that we've created is we have to, we're pulling from multiple LLMs and like, that's like a substitute or a proxy for like confidence score. When we have our proxy for confidence score is kind of lower. We will send it to a human team for review and validation. And also anytime we see if like, it's a, like a vendor we haven't seen before. So we haven't mapped that, that vendor from a canonical perspective. It's going to go for a human review just for validation at least.

AI assessment note: “We will send it to a human team for review and validation.”

Q build products based on generative AI to solve these problems. How do you, as a former CFO who doesn't come from a technical machine learning AI background, like what have you Learned in terms of like how you work with these technologies, how you interface with technical people on your team, sort of this marriage of like industry expertise and technical expertise. How do you navigate that as a leader?

A First off, like I'm very transparent with my team about what I don't know. Right. And, uh, I asked them to teach me. So I had a teaching the other day where we were Discussing the merits of potentially bringing the modeling in house. And someone was very opinionated on it. And I was like, I need to get up to speed here. Like, please tell me, I want to hear both sides of the argument here because I'm fascinated by this. And he just appreciated this. Like I didn't come with an opinion and I asked him like, please, like you're the expert here. Like I need to get more up to speed here. So I, I think that's, that's, you just need to be realistic with like what your capabilities are and I had a one-on-one the other day where one of the, the lead data engineers kind of making a lot of progress here. He's kind of siloed. And I was like, at least once a month, let's like promote your brand, like what you're doing to the rest of the organization. Cause I know everyone's interested in this stuff. So let's create a newsletter and send it to everyone. And he, he was like, oh my God, do you think they're so like interested in all the, like the metrics and like, no, no, no, no. Just like show some examples of like some of the cool work you're doing. But he, his eyes like lit up. He's like, oh, this is cool. So yeah, give people the opportunity at your company to talk about this is really like…

AI assessment note: “I'm very transparent with my team about what I don't know.”

Q Since you mentioned customers, do you want to go into a couple of success stories?

A Yeah, well, we just did a case study with a company called League Apps, so it's like fresh in mind. One of his suggestions, or we call them gleans, so we took the, you know, the verb to glean insights, and we made a noun out of it in our product, so you get your, like, your glean list. So they had a Salesforce negotiation coming up, and one of our gleans was basically pointing out that they were overpaying for, I forgot what the specific service was for, for Salesforce. But then in this case study, he reported back that in the contract negotiation, and we, we provide benchmarking data. That's part of the service that we offer because we can compare your pricing to what our other customers are paying. And we know like that we can compare the same number of seats, et cetera. So we gave him benchmarking data and he reported back that they saved 20 K versus their prior contract. So that's just fresh in mind. And then he also said like, he estimates that Glean, like the value is like two to three percent of non-payroll spend. In terms of like how the team collaborates and like how we help save across the board. But, um, the best part of my job is when I get an email from a CEO, not even a CFO, but like a CEO who's become a, like a power user of Glean. And they say, you've helped us change our spend culture. Like that's the problem I helped to solve. I wanted to solve when starting G…

AI assessment note: “we just did a case study with a company called League Apps”