The Exchanges

Q And the step that's upstream from you guys, and I don't think that you do that because you focus on the storage part, but the conversion, how is that typically done? Um, are there, like, names that people should know in terms of models and companies that do the conversion?

A Yeah, exactly. So there are language models. I'm sure many of you, all of you, hopefully, have used ChatGPT and you're pretty familiar with what that is. Um, there are also something called embedding models. And the output of embedding model is not a paragraph of text that goes into a chatbot. The output of an embedding model is this list of numbers. So an embedding model is still a machine learning model. It's still trained. Uh, OpenAI has one called Ada-II that just got today, 75% cheaper. Um, there are also closed source embedding models from Cohere, Um, Google Palm has one, um, and then there is a plethora of open source embedding models as well that many of the people, many people in the community find to be very good actually, so.

AI assessment note: “OpenAI has one called Ada-II... closed source embedding models from Cohere, Google Palm”

Q Um, all right. So let, let's jump into it. Uh, what is Chroma first at a high level? And then we'll, we'll get into some definitions as you, um, suggested, uh, run embeddings maybe.

A Yeah. Yeah. Um, obviously what Chrome is today, what Chrome is going to be in the fullness of time, um, all of these things are interesting. I'll talk about today, because that's the most honest, um, they might land the best. Um, so Chroma is an open source vector database. Um, our goal is to give developers the tools to create programmable memory for language models. Um, language models are obviously incredibly powerful, incredibly useful, Um, but difficult to, to tame. Um, you know, the Shoggoth meme comes to mind, for those of you that are, uh, terminally online, like myself. Um, the, you know, green monster with all the tentacles and eyes all over it, and, um, you know, how do we tame this beast? How do we tame this beast to make it useful and reliable? Um, and Chroma's belief is that, uh, programmable memory, so developers being able to set terministically Hey, language model, this is the knowledge you should know about, this is the knowledge you should use, these are the tools you should know about, these are the tools you should use, how to use them, um, uh, poses a path to, again, bringing language models into every use case, being able to depend on them, rely on them, um, I won't say make them aligned, um, but maybe that too, so.

AI assessment note: “Chroma is an open source vector database. Um, our goal is to give developers”

Q Okay. So what does, uh, Chroma do today? So obviously the core mission, as we just, uh, discussed, uh, store and process, um, Uh, embeddings. Uh, but this, there's a list of features in terms of like what, what, what you do now, maybe what you will do soon. What are some of those key aspects?

A Yeah. I think our focus is around developer experience. So all of the work that we've done thus far is trying to meet developers early as possible in their life cycle, um, and making the technology simple, understandable. You know, you go to the website, Even the design of the website is meant to be, like, calm, and you can, I promise you can learn this. It's not that hard. It's really not hard. Um, so, um, You know, the kinds of features that we're excited about, um, so the things that we're working on broadly, um, we're working on a distributed version of Chroma. So in the same way that Elastic, uh, for those of you that are familiar with TextSearch, Elastic picked up Lucene, made it developer-friendly, they made it distributed, Chroma picks up some of these ANN algorithms, go to a detail what that means. Uh, we made it developer-friendly and now we're making it distributed, again, all in open source. That's on the scale vector. And then the other vector is making it usable and understandable. Um, you know, if you look at, uh, conventional database, relational database, Postgres, MySQL, et cetera, it's basically two D data. You can put it on a screen. You can immediately understand what's going on. It's extremely intuitive. It's two D data. It's a table. You've all used Excel. Um, this data is not two D. It might be hundreds or thousands of dimensions. Um, and, you know, we t…

AI assessment note: “we're working on a distributed version of Chroma”

Q Do you want to just double click on that?

A Yeah, there's maybe, maybe some nuance here, which is interesting. Um, so hallucinations are, you ask a Landry model to do something, it kind of ignores you, and it just makes something up. There's a famous case recently where a lawyer was caught using it in a brief to a judge, and the judge rightfully shamed him. Um, so the solution to hallucinations is to give the language model the information you want it to have. Um, and again, through this vector search, vector retrieval process that we've been talking about tonight. Um, so it's a grounding task, and literally the prompt construction that you typically see here is, you know, High language model. Please do not make anything up. If you don't know, say, I don't know. Only consider the following documents. One, two, three. Now answer the following question. Question. Uh, and that's the, the prompt that's being built on the back end, and how the, those, again, those documents from vector search are being inputted, and then how the prompt from the user is getting slotted in, and then that's, yeah, that's, that's, that's the, that's the state of the art today. So, there's lots of stuff around steering language models at the embedding layer itself, and not using text, um, but this is not yet exposed To, um, to at least closed source models, so.

AI assessment note: “the solution to hallucinations is to give the language model the information you want”

Q And talking about go to market a little bit, what are some of the early learnings and lessons in terms of the range of use cases that people use Chroma for?

A Yeah. I think, I mean, the most common use case is what you think. It's your, it's the chat, your data use case. Um, which again, you've seen a bunch of demos already tonight. Um, You know, on one level, you're sort of, I think it's fair to trivialize that and say, this is the newspapers on the internet moment. Um, you know, web, web, one point, oh, the internet came. What are we going to do? Let's put newspapers on the internet. You know, it's just sort of like pattern matching. We already know how to do this. Now let's do it in this way. Um, and you know, I think that the secondary observation is that's not wrong. All newspapers are on the internet now and, uh, you know, arguably are much more successful because of the internet. Um, So I think that use case, even just the Chat Your Data use case, truly will go to the ends of the earth. Um, I think the more, the new use cases you're starting to see emerge, um, are the, is the ability to give, uh, LLM software agents their own memory. Um, especially in the context of, like, multi-agent interactive things. So you've seen things like ABAGI or AutoGPT, Um, there's recently a project called Voyager that used Chroma on the back end for, you know, the vector storage and retrieval. Um, you know, that, I think, is really interesting. It is, is native to AI. Like, you couldn't do that before AI. Um, so it's truly native to AI. Um, and t…

AI assessment note: “the most common use case is what you think. It's your, it's the chat, your data”

Q Uh, one last question from me, because then I want to open up to folks drew over there as a microphone, which hopefully works. Um, so zooming way back out, uh, where do you think we are in a year or two from now in generative AI? Is that just deployed everywhere? Like, what's your vision of the kind of a short-term future?

A The long-term future. So, maybe three to five years from now. Two years is this, like, dead zone of prediction, right? Like, you can predict the next week, and you can predict the future, but, like, don't try to predict two or three years out, because you're always gonna be wrong. Um, so the farther future is, uh, intelligence too cheap to meter. Intelligence built into every product, every service that we use. Um, you know, intelligence too cheap to meter, hopefully leading to a lot more flourishing for all of humanity, um, alongside clean energy too cheap to meter. Um, so that's the big vision. I think, you know, where will we be three years from now? Um, I certainly think probably most enterprises, organizations, companies on earth will have brought language models Into the company, probably in pretty meaningful ways. Um, at minimum, the customer service department, the sales department, ops, back end, legal, um, and hopefully also the core product experience that they develop to, they deliver to users, um, to their customers. So that seems like a safe bet. So I'm willing to make that bet.

AI assessment note: “most enterprises, organizations, companies on earth will have brought language models Into the company”

Q What are the various, um, axes for two, for one to make a selection?

A Sure. Yeah. Yeah, I think there's a, again, nobody knows the future. Certainly not me. Um, there's an interesting open question, which is what will the primary workloads look like? Try to avoid using overly database jargon. Um, what will the workloads look like? Will it be more transactional, meaning it's sort of a classic database, you're constantly querying it and updating it, deleting stuff. Will it be more analytical, where you want to, like, slurp in the whole database and then do some analytics process over the whole thing? Um, or will it be both? Will actually both be important? And, um, we, we fall in the latter camp. So the, the acronym here, again, for the database geeks would be HTAP. Um, hybrid transactional analytical processing. Um, and we think that both certainly transactional has to be the case because it is a online database. It's gonna sit in the loop of applications. Again, you've already seen demos of this happening tonight. Um, but also to make this technology useful for developers, you do want to slurp in the whole database and analyze it in a bunch of different ways. And so that's an analytical process. And so, um, again, to not go too deep on database jargon here, but we think that both are really important. And I think that, again, without Getting into some details. Um, again, that's unique. We have a unique perspective on that.

AI assessment note: “what will the primary workloads look like? Will it be more transactional... Will it be more analytical”