Prediction certainty 3/5 debate potential 3/5

Liu: The final state for personalized AI memory will not be RAG

Jerry Liu · RAG is a hack - with Jerry Liu of LlamaIndex · Oct 12, 2023 · at 1:10:29

Jerry Liu, creator of LlamaIndex, discusses the future of personalized memory in AI assistants during a lightning round on Latent Space.

0:00 / 0:11exact quote · 11.1s

▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark

“I think a lot of people have thoughts about that, but, like, for what it's worth, I don't think the final state will be ragged. I think it will be some, like, fancy algorithm or architecture where you, like, bake it into, like, the architecture of the model itself.”

quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →

The claim, plainly

Jerry Liu predicts that the final state for personalized AI memory will not be RAG, but an architecture baked directly into the model itself.

a quote alone can be ambiguous, so this restates it with the references and context of the conversation around it resolved. AI-written; the tape below the player is the source.

How this resolves

Verification couldn't pin this to a public evidence class, so it stays on the record with its date, unassessed rather than guessed.

no date stated; next review by 2028-10-12

Mentioned

Jerry Liu LlamaIndex Retrieval-Augmented Generation

⚑ Spot an error? Report it

More from Jerry Liu

Insight

Liu: RAG is fundamentally just an algorithmic prompt-stuffing hack

“RAG is basically just a hack, but it turns out it's a very good hack because what is RAG? RAG is you keep the model fixed, and you just figure out a good way to, like, stuff stuff into the prompt of the language model. Everything that we're doing nowadays in t…”

Jerry Liu Oct 12, 2023 ▶ 24:45 RAG is a hack - with Jerry Liu of LlamaIndex

Insight

Liu: RAG will remain essential for managing LLM cost-performance trade-offs

“There's always going to be some curve regardless of like the performance of the best performing models of like cost versus performance. And so what RAG does is it does provide extra data points along that access because you kind of control the amount of contex…”

Jerry Liu Oct 12, 2023 ▶ 15:01 RAG is a hack - with Jerry Liu of LlamaIndex

Prediction Open · timeframe Oct 2028

Liu: Developers will eventually fine-tune new factual knowledge into LLMs

“That's one of those things where I think long-term, you definitely can. I think some people say you can't. I disagree. I think you definitely can. Just right now, I haven't gotten into work yet.”

Jerry Liu Oct 12, 2023 ▶ 29:53 RAG is a hack - with Jerry Liu of LlamaIndex

Insight

Liu: Improving vector store lookup algorithms offers low marginal gains

“I don't think the delta on, like, improving the vector store, like, embedding lookup algorithm is that high. I think this stuff has been mostly solved or at least there's just a lot of other stuff you can do to try to improve the overall performance.”

Jerry Liu Oct 12, 2023 ▶ 1:01:34 RAG is a hack - with Jerry Liu of LlamaIndex

Insight

Liu: Avoid pre-GPT-4 models for tasks requiring complex reasoning

“Like, I'm one of the first to say, like, you know, you shouldn't use anything pre-GPT-IV for anything that requires, like, complex reasoning because it's just going to gonna be unreliable. Okay, disregarding stuff like fine-tuning.”

Jerry Liu Oct 12, 2023 ▶ 9:28 RAG is a hack - with Jerry Liu of LlamaIndex

Opinion

Liu: Security and access control are not P0 for enterprise apps

“I think users have asked for it, but I don't think that's like a P zero. Like, I think the P zero is more on, like, can we get this thing working before we expand this to, like, more users within the org.”

Jerry Liu Oct 12, 2023 ▶ 34:32 RAG is a hack - with Jerry Liu of LlamaIndex

Liu: The final state for personalized AI memory will not be RAG

More from Jerry Liu

Turn any episode into a week of clips.