The Exchanges

Q that very few people talked about that seemed like a solved problem. This idea of transforming your data into a format that vector database Databases can understand. Um, that went from what felt like a solved problem to like a very hot space in the kind of LLM stack, including a race to the bottom in terms of price. What do you think that is and what is happening there?

A Oh, that, that, that's very simple. It's the, um, we're now in this transition from people who were trying it out, building POCs, prototypes, you know, the, the, the usual stuff. And now they want to go into production. And the thing to bear in mind is that, um, um, if you have an embedding of, um, 768 dimensions versus 1536, um, that's like, um, uh, that you need double the memory to, uh, to store the letter, right? And if you do a POC with like a hundred, uh, or a thousand, or a million data objects, that's fine. If you now scale that up to, Hundreds of millions or billions, it gets expensive quickly. So a lot of work has been happening, one, in retrieval speed, because again, if you want to index 10,000 documents, you can wait. If you need to index ten billion, you need speed. Speed becomes of the essence. That is one thing. The second thing is work in the models themselves. So can we lower the, the size of dimensions that we need to store? And the third thing that sits a little bit in the overlap between the database and the models itself, And it has to do with compression algorithms, uh, binary representations, the whole shebang, just to make it as easy as possible to run the stuff in production.

AI assessment note: “we're now in this transition from people who were trying it out, building POCs”

Q What are some other features that you would want to highlight that make Weviets special?

A Oh, thank you for giving me. So, so the, um, uh, so the first thing is the, um, uh, is the, the first is the deployment model. So it's the, um, Because we live in, like, twenty-twenty-four now, we've learned from all these amazing existing infrastructure companies, um, that, like, the generation before us, like, what the most optimal way is how people want to use database. So, you can use Weaviate serverless, BYOC, to the marketplaces, embedded, open source in Docker, open source in Kubernetes, you name it. However you want to use it, you can use it. That's one. The second thing is that you can store the complete data object. So, it's very common to what you're used to from existing databases. But now it has, like, it's really focusing on this, what we call AI native, um, stack first. But the third one, and that's by far the most important, is that we've built the database to help you build AI native applications. So what we mean with that is, like, if you're building something and you want to sprinkle some machine learning stuff over your, over your application, it's great. I mean, you can use your VVA too, but you know, you can do it in other ways too. But if you say, no, I'm building something that it has, like, AI at the core, We've, we give you all the tools and all the infrastructure to build it out of the box, to get you up and running in, like, in minutes. And that is t…

AI assessment note: “the first is the deployment model. So it's the, um, Because we live in”

Q Yes. How about security? It seems to be something that, uh, you guys spend a lot of time on.

A Well, so, I mean, security is a, is a, so the, the short answer is yes, but the more elaborate answer is no, we hate security. Exactly. Yeah. Yeah. No, but the, um, that would be funny though. No, but the, so what starts to happen is that people start to move into production and then you get just very, everything that you would expect from running in production, like, hey, oh, can we actually separate tenants? What kind of security features do you have, and so on, backups, and so on, and so forth. So more and more basic things that one would expect from a core piece of infrastructure. Again, this is really led through a community of users and customers. They just say, you know, great, we did a We want to move to production, but we really need this one thing, and then great, and then we add it to the roadmap, community upvotes it, and that's how we build it.

AI assessment note: “the short answer is yes, but the more elaborate answer is no”

Q So you're, you're very much an open source company, um, It was an interesting question for me to hear how people think about how much do you put in the open source versus how much you put in the commercial product?

A So open source is a jobs to be done problem. So it's a, it's, um, um, the database itself, you're not selling The database. What you're selling are the services around the database. So we, we create proprietary software that is like our serverless offering, those are BYAC things, that has to do with monitoring, with the whole shebang, right? And around that, with a graphical user interface, we create apps, that kind of stuff. Uh, so the core database is, is, is open source, and it's a way for people to tinker around what's happening. Important to know, open source also builds trust. With customers. It's not a black box. You can just be super transparent about what you're doing and how you're doing it. So that's the role of open source, but the job to be done, the value to be captured, Um, uh, does not per se sit in the open source technology itself. It sits in the layers, uh, around that. Somebody today forwarded me a, an, an Harvard, uh, an Harvard article, um, it's called like, um, the value that open source is creating or something. I'm not sure if you've seen that. It was like two weeks ago, and, um, they, the article was super interesting. The article is about, and I'm saying this from the top of my head so I can, the numbers can be off a little bit, but, um, they said like, um, How much the economy is driving on open source? And they also made a, um, uh, calculate that if…

AI assessment note: “the core database is, is, is open source”

Q So that's RAG as an emerging architecture for generative AI. Um, let's get more specifically into what you guys do at Weaviet. So in particular, uh, hybrid search seems to be something that, um, you have spent a lot of time working on. What is it and what are the benefits?

A Yeah. So, um, so what's important to know is that, um, uh, Weaviate is open source. And if your database is open source, you get something beautiful, which is called a community. And the community starts to work with your database, and they tell you what doesn't work. And, um, uh, that can be on one hand, that can be just related to operational issues, but it's especially interesting from use case issues. And one of the things that people started to, you know, to tell us, right, there were actually two things that people started to tell us was this, like, one, they said, vector search is amazing, but sometimes it's not enough. And, and in a bit I'll, I'll share, I'll share with you what that is. And the second thing that they were, that they said, like, These vector embeddings, you know, we need to work with these models. It's like, not everybody knows how to do that, or to how to efficiently do that. I mean, they're like a lot of developers, like the people in the room here, they're like very smart, they know how to do that, but there's not everybody knows that, right? And, or people want to just speed up, you know, their development. And so we learned these two things, and, um, so back to the first point, so if you have a sentence, a query that says, for example, let's say that you have a data set with, um, um, uh, customer support tickets, and you say, like, how was customer…

AI assessment note: “vector search is amazing, but sometimes it's not enough.”

Q Talking about, uh, customers and go to market for a second, what are some, like, fun examples of, um, what you've seen people build with, with some, you know, interesting use cases?

A So the, um, so the use cases are still pretty much, um, um, um, um, I mean, we, we see some, Um, uh, some clustering right around e-commerce and those kind of things, but it's still very much all over the place. I think, so, um, Let me give you two examples of things that I'm very proud of, right? So, so just that comes to mind. One are just tools that I use myself, right? It's kind of nice that if you see, like, a blog from Stack Overflow saying that they use WebAto, oh, thank you, I use that, right? That's one thing that I'm proud of. And the second thing that I'm proud of is if you are part, a core part of technology, um, uh, of, um, uh, new applications that people are, I mean, I came in, Today, and this gentleman came up to me and said, like, hey, you know, we're using Weavis, a core part of our stack. I said, great, that's wonderful, right? Um, um, we saw it with, I don't know, um, Instabase, for example, right? So they, how they rewrote some core infrastructure with Weavis. So those kind of things is something that I'm extremely proud of. That's just, yeah, that's just really cool to see, and it's like a, um, but, so it's a combination of tools that I use myself that I'm very proud of, or just, That it's just such a dream coming true that, that you're helping people to be successful with the businesses and applications that they are building. That's just, that's, that's …

AI assessment note: “we saw it with, I don't know, um, Instabase, for example, right?”

Q RAG as This kind of problem, um, the solution to the problem, like hallucination and all those things. A lot of people talk about it almost like a fait accompli, like it's, uh, you know, something that everybody has agreed is, is working. What's your assessment? Like, how, how, um, is it reworking? What are the issues? Uh, and how do you evaluate if it works in the first place?

A So, I mean, so it's a, it's a first step. So, so what's very important for people to know, if you run a database company, then the big question that you always ask is like, and, and people like yourself, what you guys always ask, so what's the unique use case, right? And then if in the beginning, yeah, we can do stuff with search and recommendations, look how cool it is, but yeah, we can see what that's, what's the unique use case. So the moment, um, uh, so reg is quite, ah, I mean, I mean, old in the sense of how things are young and old in, in, in, in the space of, of AI. So it's, it's an, it's a relatively older concept. For those interested, you can actually, if you go into the, uh, the Hugging Face library, you can see first iterations of it. It's very, very interesting. But the, um, um, so when, uh, JetGPT came to the scene, a lot of people were like, we want to do this with our own data. And people are like, hey, it's great to use vector database for this because the input queries are often not fully working for keyword matching. So this was like a beautiful use case for the, um, uh, for the vector database. And it's not always right because it's like they're, they're the pure vector search. So vector search alone is often not enough, right? So that's like stuff like hybrid service, but we might get to that. Um, but not only that, also, RAG is just the first step in new …

AI assessment note: “RAG is just the first step in new things we can do with vector databases.”

Q How should people think about the criteria to evaluate one vector database versus the other? And you can name names, which would be fun, or not.

A So the, um, um, no, so, so the first thing is this. So the, um, uh, for a long time we were waiting, like, when, so, okay, no, no, let me rephrase. So the vector embedding is just a data type. Um, so it's just, it's just an array of floating point numbers. I mean, you can, you can store that in an old Oracle database, right? So, but the thing where it becomes interesting is the index. So the index and how you store and retrieve the information, you can do the memory for speech, you can do it on disk, uh, if you want to optimize for storage and those kind of things. So the, the, what we try to start to see now is that basically every database under the sun supports vector embeddings. That's a great thing. That's like for, for me as a vector database, That's like a champagne problem, right? Because that means that people see the value in using vector embeddings. So now, if you look at the vector databases space, it's like you get to these purpose-built databases, right? That are really good at dealing with these embeddings first. So maybe if you're building something small or trying something out, you know, you might want to store them somewhere else. That's fine, right? Because it became a universal data type. Now, and now you can make a distinction between that you're saying like, okay, do we want to, is, do we just want to care about these vector embeddings, how we store them,…

AI assessment note: “do we just want to store a couple of embeddings? Or do we need to have help”