Liu: Security and access control are not P0 for enterprise apps
Jerry Liu · RAG is a hack - with Jerry Liu of LlamaIndex · Oct 12, 2023 · at 34:32
LlamaIndex founder Jerry Liu discusses enterprise adoption patterns and whether authorization and access control are immediate priorities for companies deploying LLMs.
0:00 / 0:08exact quote · 8.7s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“I think users have asked for it, but I don't think that's like a P zero. Like, I think the P zero is more on, like, can we get this thing working before we expand this to, like, more users within the org.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Jerry Liu
Liu: RAG is fundamentally just an algorithmic prompt-stuffing hack
“RAG is basically just a hack, but it turns out it's a very good hack because what is RAG? RAG is you keep the model fixed, and you just figure out a good way to, like, stuff stuff into the prompt of the language model. Everything that we're doing nowadays in t…”
Liu: RAG will remain essential for managing LLM cost-performance trade-offs
“There's always going to be some curve regardless of like the performance of the best performing models of like cost versus performance. And so what RAG does is it does provide extra data points along that access because you kind of control the amount of contex…”
Liu: Developers will eventually fine-tune new factual knowledge into LLMs
“That's one of those things where I think long-term, you definitely can. I think some people say you can't. I disagree. I think you definitely can. Just right now, I haven't gotten into work yet.”
Liu: Improving vector store lookup algorithms offers low marginal gains
“I don't think the delta on, like, improving the vector store, like, embedding lookup algorithm is that high. I think this stuff has been mostly solved or at least there's just a lot of other stuff you can do to try to improve the overall performance.”
Liu: The final state for personalized AI memory will not be RAG
“I think a lot of people have thoughts about that, but, like, for what it's worth, I don't think the final state will be ragged. I think it will be some, like, fancy algorithm or architecture where you, like, bake it into, like, the architecture of the model it…”