The Exchanges

Q um, to help, uh, people who may listen to this, uh, make sense of where Lama Index fits in that emerging, uh, LLM or generative AI infrastructure stack, uh, people may have, uh, heard, heard of names like Langchain or Fixie or Dust or are those Uh, competitors. Are they partners? Uh, is that all, um, sort of still, um, you know, moving pieces that everybody's trying to figure out?

A Yeah, to some extent, it's all moving pieces. Um, I think there are definitely overlaps with certain frameworks, but there's also, uh, key differences. And so, you know, let's talk about, for instance, like blind train, blind train is a great application framework for you to just like, uh, get us out of building blocks for a lot of different components, for instance, from like LL modules to prompts to some basic like retrieval and and vector database abstractions to, uh, like also agent frameworks. Um, we are like almost from the beginning have been very focused around the data. And so just like Um, you know, really what we think about is how do you get your data in the right format so that you can use it with the outline? And then also, how do you get the outline to effectively query your data? Um, and so there are some overlaps between that and link training, but we're like very hyper-focused on developing deep tech around that and making that really good. Um, and so not just doing, again, the basic naive, uh, retrieval augmented generation stack of, you know, the tech, splitting your data into some form, dumping it into a vector database and a single question and doing top hit retrieval. We offer that, but also a rich set of advanced functionality to do, um, additional, um, capabilities, you know, uh, uh, combining out on some top of your data. And so, uh, we've almost inten…

AI assessment note: “there are definitely overlaps with certain frameworks, but there's also, uh, key differences.”

Q While we are on that topic of ingestion, so what is the right format? What does that mean?

A Yeah, so I could, uh, probably spend an hour talking about this, but at a high level, um, the, the trick is, you know, there is a, I can talk about the, the kind of, like, naive thing that everybody does these days, and then the kind of, like, considerations that you might need to think about if you're trying to build something more production quality. So just a quick overview of retrieval augmented generation of the concept. Um, you ingest some source documents, um, Let's say they're a set of PDFs. Now the current stack is basically, uh, you take these documents and then you split it up into a bunch of text chunks, and then you store the text chunks in like a vector database. And these days, you know, there's a bunch of different vector databases from Pinecone to Chroma to WeVA to a bunch of others. Uh, and these act as like a storage layer, right? And so then when you actually, uh, say you're building a question answering system, you ask a question, what's going to happen is it's you're first going to do retrieval from your storage layer. You're going to hit the query interface of the vector database and do embedding based retrieval from the vector database. Cause that's what it's specialized at. Then after you get a set of retrieve context, then you put these contexts into the language model, uh, to actually synthesize an answer. So that's the overall stack of retrievable au…

AI assessment note: “you take these documents and then you split it up into a bunch of text chunks”