The Exchanges

Q you know, for anybody that spends time in the MLOps world, uh, in the last couple of years before vector databases became the thing that everybody's talking about, like the, the, the concept of storing stuff that would be, uh, fed into machine learning models. People heard the term maybe feature store. Um, is that completely different? Is that part of the same family? How would you think about it?

A Uh, yes. No, I'm just kidding. Uh, yeah, uh, no, it's a completely different thing, right? Uh, feature stores are used mainly either for, uh, marshaling data into training or using real-time features for objects that change very frequently. Right? So, like, I, you know, if I'm a user and I just clicked on something, maybe you want to incorporate that last feature that just happened a second ago in my, my next classification, in my next action or something. Uh, vector databases are completely different. It's really about This long-term memory about these, you know, billions of embeddings of documents or images and so on and making them available to large language models or these other, you know, multimodal models and so on where they, uh, yeah, they operate more like, you know, parts of your brain than, than like, uh, you know, a feature store.

AI assessment note: “no, it's a completely different thing, right?”

Q Um, maybe that's related to the, the, the cows question, but w one really important Part of vector databases is because they are the long-term memory, as you were saying, um, they are a very important tool against that problem of hallucination that people talk about a lot. Is that, is that the right way to think about it?

A A hundred percent. In fact, I was looking at experiments today from one of our teams, and we literally measure reduction in hallucination as one of the core Um, uh, metrics that, that, that we try to drive. Uh, so a hundred percent, like, uh, and, and by the way, I mean, this is, it's, it's almost like, uh, obvious, like if I, if I, if I ask you a question for which you don't know the answer, right, but I compel you to say something anyway, You know, you're gonna make something up, right? But if I give you the right context, you can answer a lot more accurately. And that's exactly what's happening. And so the question is, can we retrieve the right context out of which an informative answer can come out, right? And so, you know, it's, it's, uh, in some sense, it's obvious that that should happen. It's harder to actually get the whole thing to cascade correctly and to work measurably well.

AI assessment note: “A hundred percent. In fact, I was looking at experiments today”

Q Great. I have so many other questions, but, uh, let me pick one or two, and then, um, I'll open up to people. The hands are already raising. Um, um, let's see. You know, look, I'm a VC. I'm always interested in that part as well. Let's talk about go to market. Like, how are you building the company? How are you finding customers? How are you selling to them?

A Um, so, uh, we started Pinecone, or I started Pinecone Uh, as a managed service, uh, because I'm a really big believer in kind of the buyer driven journey, right? I think in the, in the, in the modern world, like nobody wants to like, if you want to use some technology, the last thing you want to do is like send an email and wait for three days. Like, that sounds very unnatural. You want to use it immediately, right? You want to, you know, and if it's great, you want to just start using it and forget, like, be happy and move on. Uh, so we, we really built the whole company about the self-serve journey, right? Today you can start with Pinecone. Now be like, run some example in five minutes, build like a demo app in like two hours, integrate with some like kind of, uh, Like a micro POC for your team in a day, and like in a week be in production, and never talk to us, ah, once if you choose not to, right? Some companies today, very large, larger companies tend to also want to talk to us, and so we are now building a sales team that, that is, ah, happy and willing to engage and help and so on, but it's more about Assisting in the journey if, if that's wanted or kind of standing, like not being in the way when we're not wanted.

AI assessment note: “we really built the whole company about the self-serve journey”

Q There must be like thousands of people now, right?

A What? Several thousands now. Yeah. But, uh, you know, I'm saying this, I'm kind of recounting the whole story because throughout that time, vector search and vector representations and embeddings and that kind of basic concept kept getting more, you know, gradually more and more traction. All the time, more and more people kind of understood what it does and what, what's happening with it. Um, And then, uh, at some point, like, uh, the BERT embeddings, kind of, the, the transformer models kicked in, and then there was a big spike in understanding. And then, that's it. In 2019, in some sense, I felt like that was, uh, the right time. I felt like it was gonna get us about two years to kind of build the right kind of infrastructure correctly. Um, and we kind of timed it correctly, and so this is what happened. We didn't, by the way, foresee any of this ChatGPT thing happening. We knew it was, it would keep growing, but that, I think, completely took everybody by surprise, including us.

AI assessment note: “What? Several thousands now. Yeah.”

Q Okay, fantastic. Um, maybe one last concept, and thank you for explaining all of this so clearly. Semantic search is something that, again, people hear a bunch. What does that mean?

A So semantic just means by meaning, uh, and search means search. Uh, and, uh, You know, traditional search is, is, is keyword based. Uh, not because keyword search is that great, but just because we have great mechanism to run it at scale, and it's very efficient. Keyword search is a pretty old technique. In fact, uh, books had, uh, inverted indexes in the backs of, of them. They're even called an index. Uh, they've had them for a while. In fact, you had indexes in books before we had print. So that's, that's a pretty old, uh, well, before we had print in the West, the Chinese were printing well before the Gutenberg Bibles were even thought of. So, uh, but anyway, it's, it's like early 1200, at least it's a, you know, uh, so keyword search existed for a while. Great technique works great for Google, works great in other places. Doesn't work when you want to search something by meaning, right? When you, uh, When you remember a conversation that you had, I don't know if you've ever searched your inbox for an email you know for a fact you read, right? And the only way to do it is try to, like, hack the search system to figure out what word was there that I didn't use in another email. Like, which I, like, backwards engineer the crappy search. Uh, with semantic search, the whole idea is that You know what it means. You can, you should be able to search by the meaning in free text, a…

AI assessment note: “semantic search, the whole idea is that You know what it means. You can, you should be able to search by the meaning”

Q So, you know, when people talk about, like, how do you do this whole thing? Vector database is like the term that everybody comes back to. Um, What does a stack look like, I guess, you know, based on, uh, what you see customers do this, you know, this vector database, this link chain, there's like different models, like how does that all fit? What does that look like now?

A Yeah. So there is a, uh, I mean, it depends what you call like, uh, so I'll, I'll focus very narrowly on the large language model stack. Of course, they're the model themselves. Uh, there are vector databases. There are libraries that connect them in all sorts of different ways, like Lama Index and, you know, Langchain and others. Um, um, there are, um, you know, I, I think there are a lot of, you know, connective services, but, um, I think we're gonna see an emergence of agents in some form or another. I think that's still very new. So agents are these, uh, recursive, uh, uh, pieces of software. I don't even know how to define them in that. It's like, it's sort of like this, like life hack. On large language models. It's like, how do you use these, uh, large language models enable, like, let them use tools like, you know, search or different APIs and so on, and then use their own answers to somehow feedback into like an input and try to, you know, try to accomplish very complicated tasks, right? So it's not like a one-time, you know, uh, You know, it's like a one-time prompt. Like, you know, if you want to, you know, if you want to get in shape, right? I mean, that's a plan. You need to fit into a schedule. You have to find a gym. You have to whatever, you know, there's like a sequence of things you need to do, and you can research every one of them separately, and you might t…

AI assessment note: “I'll focus very narrowly on the large language model stack. Of course, they're the model”

Q So let's say I have, um, Video data, audio data, data, text data. So if I understand correctly, I need to translate this into numbers to fit it into a machine learning model, an AI model. How do I do that? How do I take, like, video and translate that into numbers?

A So, um, let's, I mean, we can talk about the mechanics of how to do it, which is maybe the less interesting thing, because, you know, The less technically inclined of you don't care and the more technically inclined of you can figure it out. The, the, uh, if you zoom out though, the interesting part of it is that I think there is a, there is a significant shift in the way that people think about these, uh, building these smart AI systems. Um, and, uh, They don't try to fit the data into the model as much as give the model access to the right data when it's doing the inference, right? And so, um, you know, I, I, uh, you know, in the same, I gave this example several times, so, but, but, uh, like if you, if you study medicine, right, you already go into medical school knowing English, And then you study, like, a body of knowledge. You study medicine. You don't study medical English. Like, you read a book, and you, you know, you then know medicine, and then you speak about it, because you know how to speak, right? You don't, you know, become a doctor by just doing the rounds for 10 years, and then you know enough, you know how to sound like a doctor. So, uh, you know, That's the answer in some ways. You don't fit it. You, you allow the model to access external memory and use it correctly, and you give it the right data. So if you wanted to be a doctor, you give it medical knowledg…

AI assessment note: “we can talk about the mechanics of how to do it, which is maybe the less interesting thing”