The Exchanges

Q Yeah. How, how, how big a need is that? Because I, I, um, I talked to some people who are thinking about building tools in that domain, but I don't know if people want it.

A I mean, I think bigger companies like, um, just bigger companies like banks, consulting firms, like they all want this requirement, right? The way they're using Lama index is not, uh, with this, obviously, because I don't think we have support for like access control or author or that type of stuff like on a hood, because we're more just like an orchestration framework. Um, and so the way they do it, they, they build these initial apps is more kind of like, Prototype, like, let's kind of, yeah, like, you know, use some publicly available data that's not super sensitive. Let's like, you know, assume that every user is going to be able to have access to the same amount of knowledge, those types of things. Um, I think users have asked for it, but I don't think that's like a P zero. Like, I think the P zero is more on, like, can we get this thing working before we expand this to, like, more users within the org.

AI assessment note: “I think users have asked for it, but I don't think that's like a P zero.”

Q Yeah. Yeah. Nice job. Um, and obviously I've had the pleasure of chatting and working with, uh, a little bit with both of you. Um, what would you say those, those like your top, like one or two values are when, when thinking about that or the culture of the company and that kind of stuff?

A Yeah. Well, I think in terms of, um, the culture of the company, it's, it's really like, uh, I mean, there's a few things I can name off the top of my head. Uh, one is just like, uh, passion, integrity. I think that's very important for us. We want to be honest. We don't want to like, obviously like copy code or, or kind of like, you know, just like, you know, not give attribution, those types of things and, and just like be true to ourselves. I think we're all very like down to earth, like humble people, but obviously I think just willingness to just like Own stuff and dive right in. And I think grit comes with that. I think in the end, like this is a very fast moving space and we want to just like be one of the, you know, like dominant forces and helping to provide like production quality Allen applications.

AI assessment note: “one is just like, uh, passion, integrity. I think that's very important”

Q Awesome. Um, yeah, let's jump into lightning round. Uh, so we have two question, acceleration, exploration, and then one final tag away. The acceleration question is, what's something that already happened in AI that you thought would take much longer to get here?

A I think just the ability of LLMs to generate believable outputs, um, and, and both, uh, for text and also for images. And I think just, um, the, the whole reason I started hacking around with LLMs, honestly, I felt like I got into it pretty late. Actually, I got into it like early, Because GPT-free had been out for a while, like, just the fact that, um, there was this engine that was capable, like, reasoning, and no one was really, like, tapping into it, um, and then the fact that, uh, you know, I used to work in image generation for a while, like, I, I did GANs and stuff back in the day, and that was, like, pretty hard to train. You would generate these, like, 32 by 32, uh, images, and then now taking a look at some of the stuff by, like, Dolly and, and, you know, Mid Journey and those things, so it's, it's just, it's, it's very good, yeah.

AI assessment note: “the ability of LLMs to generate believable outputs, um, and, and both, uh, for text and also for images”

Q Um, which I, I like that original vision. Um, your, your messaging around about then was also that you're creating optimized data structures. Um, how, how, like, what's the sort of journey to that, and like, how does that contrast with Lama Index today?

A Yeah. So, okay, maybe I can tell a little bit about like the beginning intuitions. Um, I think when I first started, this really wasn't supposed to be something that was like a toolkit that people use. It was more just like a system. Um, and the way I wanted to think about the system was more a thought exercise of how language models with their reasoning capabilities, if you just treat them as like brains can organize information and then traverse it. So I didn't want to think about embeddings, right? To me, embeddings just felt like it was just an external thing that was like, Well, it was just external to try and actually tap into the capabilities of language models themselves, right? I really wanted to see, you know, just as, like, a human brain could, like, synthesize stuff, could we create some sort of, like, structure where the, there's this, like, neural CPU, if you will, can, like, organize a bunch of information, you know, auto summarize a bunch of stuff, uh, and then also traverse the structure that I created. That was the inspiration for this initial, like, tree index. Uh, it didn't actually, like, like, to be honest, and I think I said this in the first tweet, it didn't actually work super well, right? Like,

AI assessment note: “maybe I can tell a little bit about like the beginning intuitions.”

Q note on Quora. A lot of people, including myself, like kind of wrote off Quora as like one of the web one point oh, like sort of question answer forums. But, uh, now I think it's becoming a, seeing a resurgence obviously due to Poe. Um, and obviously Adam and D'Angelo has Always been a leading tech figure, but what do you think is, like, kind of underrated about Quora?

A I really like the mission of Quora when I, when I joined. Um, in fact, um, I think when, um, I interned there, like, in 2015, and I joined full-time in 2017, one is, like, they had, and, and they have, like, a very talented engineering team, um, and, and just, like, really, really smart people, and the other part is the whole mission of the company is to just, like, spread knowledge, and to educate people, um, right, and, and to me, that really resonated. I really like the idea of just, like, Education and democratizing the flow of information. And if you imagine, like, um, kind of back then, it was like, okay, you have Google, which is, like, for search, but then you have Quora, which is just, like, user-generated, like, grassroots-type content. And I really like that concept because it's just, like, okay, there's certain types of information that aren't accessible to people, but you can make it accessible by just, like, surfacing it. And so, actually, I don't know if, like, most people know that about, like, Quora, like, and, and if they've used the product, whether through, like, SEO, right, or, or kind of, like, actively. But that really was what drew me to it.

AI assessment note: “the other part is the whole mission of the company is to just, like, spread knowledge”

Q SEC Insights as one of kind of like your demos, and that's like a great example of, hey, I don't want to embed all the historical documents because a lot of them are outdated, and I don't want them to be in the context. What's that problem space like? How much of it are you gonna also help with and versus how much you expect others to take care of?

A Yeah, I'm happy to talk about SCC Insights in just a bit. I think more broadly about the, like, overall retrieval space, we're very interested in it, because a lot of these are very practical problems that people have asked us. Um, so the idea of outdated data, I think, um, how do you, like, deprecate or time-weight data, um, and do that in a reliable manner, I guess, so you don't just, like, kind of set some parameter, and all of a sudden that affects your, all your retrieval arguments, is pretty important, because, uh, people have started bringing that up. Like, I have a bunch of duplicate documents, things get out of date, how do I list on set documents? Um, and then ranking, right? Yeah, so I think this space is not new. Um, I think, uh, like, rather than inventing, like, new retriever techniques for the sake of, like, just inventing better ranking, um, we want to take existing ranking techniques and kind of, like, package it in a way that's, like, intuitive and easy for people to understand. That said, I think there are interesting and new retrieval techniques that, uh, are kind of in place that can be done, um, with, when you tie it into some downstream rack system. I mean, like, the reason for this is just, like, if you think about how, um, like, the idea of, like, chunking text, right? Like, that really, that just really wasn't a thing, um, or at least for this specific…

AI assessment note: “we want to take existing ranking techniques and kind of, like, package it”