Huber: Steering LLMs at embedding layer is not exposed in closed-source models
Jeff Huber · Why Vector Databases Are Exploding: Chroma Co-Founder Jeff Huber on Building AI-Native Infra · Jun 28, 2023 · at 19:55
Jeff Huber, co-founder of vector database startup Chroma, discusses mechanisms for guiding AI models beyond standard text prompt construction.
0:00 / 0:11exact quote · 11.5s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“So, there's lots of stuff around steering language models at the embedding layer itself, and not using text but this is not yet exposed To at least closed source models, so.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Jeff Huber
Huber: Current SOTA LLMs Lack Reliability for Multi-Agent Workflows
“Now, of course, for those of you that have actually played with technology, I think it's questionable whether the current state of the art Language models, embedding models, et cetera, will give you the reliability you want from, ah, you know, agents working t…”
Huber: Chroma is committed to remaining fully open source
“Chroma will always, we are committed to building the ubiquitous open source standard.”
Huber: Vector databases must support both transactional and analytical workloads
“And we think that both certainly transactional has to be the case because it is a online database. It's gonna sit in the loop of applications. Again, you've already seen demos of this happening tonight. But also to make this technology useful for developers, y…”
Huber: Asking whether vector databases replace classic databases is dumb
“People, there's this, like, you know, big question about, oh, are vector databases gonna replace classic databases? Are these competitive in some way? And I think it's just kind of a dumb question.”
Embedding search and analytics enable developers to improve model reliability
“By looking at embeddings doing embedding search, doing analytics over embedding space you could give developers you know, at the minimum of a divining rod, if not a compass, to be able to improve their models and get to the level of reliability they want to ha…”
Huber: Programmable memory enables reliable LLMs across all use cases
“Chroma's belief is that programmable memory, so developers being able to set terministically Hey, language model, this is the knowledge you should know about, this is the knowledge you should use, these are the tools you should know about, these are the tools …”