The Ledger
Every statement that passed quotation and attribution checks. Mix any filter with any other: certainty 1/5, debate potential 5/5, or both at once.
why aren't all 13 resolved? a statement only gets an assessment when the public record can support or contradict it. opinions and what-ifs never can, and 1 checkable ones are still open, waiting for their date. predictions held up or didn't; assertions are supported or contradicted. on every card: ▮▮▮▮▮ certainty · ▮▮▮▮▮ debate potential. speakers are clickable
Liu: RAG is fundamentally just an algorithmic prompt-stuffing hack
“RAG is basically just a hack, but it turns out it's a very good hack because what is RAG? RAG is you keep the model fixed, and you just figure out a good way to, like, stuff stuff into the prompt of the language model. Everything that we're doing nowadays in t…”
Liu: RAG will remain essential for managing LLM cost-performance trade-offs
“There's always going to be some curve regardless of like the performance of the best performing models of like cost versus performance. And so what RAG does is it does provide extra data points along that access because you kind of control the amount of contex…”
Liu: Developers will eventually fine-tune new factual knowledge into LLMs
“That's one of those things where I think long-term, you definitely can. I think some people say you can't. I disagree. I think you definitely can. Just right now, I haven't gotten into work yet.”
Liu: Improving vector store lookup algorithms offers low marginal gains
“I don't think the delta on, like, improving the vector store, like, embedding lookup algorithm is that high. I think this stuff has been mostly solved or at least there's just a lot of other stuff you can do to try to improve the overall performance.”
Liu: The final state for personalized AI memory will not be RAG
“I think a lot of people have thoughts about that, but, like, for what it's worth, I don't think the final state will be ragged. I think it will be some, like, fancy algorithm or architecture where you, like, bake it into, like, the architecture of the model it…”
Liu: Avoid pre-GPT-4 models for tasks requiring complex reasoning
“Like, I'm one of the first to say, like, you know, you shouldn't use anything pre-GPT-IV for anything that requires, like, complex reasoning because it's just going to gonna be unreliable. Okay, disregarding stuff like fine-tuning.”
Liu: Security and access control are not P0 for enterprise apps
“I think users have asked for it, but I don't think that's like a P zero. Like, I think the P zero is more on, like, can we get this thing working before we expand this to, like, more users within the org.”
Liu: Fine-tuned LLMs will become the primary scalable evaluation solution
“No, but these models will get better and you'll probably fine tune a model to be a better judge. I think that's probably what's going to happen. So I'm like reasonably bullish on this because I don't think there's really a good alternative beyond you just huma…”
Liu: Chain-of-thought query decomposition produces superior retrieval results
“Another example here is actually LLM based reasoning, like LLM based chain of thought reasoning. You can take a question, break it down into smaller components and use that to actually send to your retrieval system. And that gives you better results since kind…”
Liu: LlamaIndex is de-emphasizing direct revenue generation in 2023
“I think where our revenue focus this year is kind of is less emphasized. Like, it's more just about, like, can we build some managed offering that, like, provides complimentary value to what the open source library provides.”
Liu: Developers should build RAG from scratch before using framework abstractions
“Building, like, RAG from scratch. I mean, I think everybody should do it, I think. Like, I would check out the guide if you guys haven't already, and I think it's in our docs, but instead of just using you know, either the kind of, like the retriever query eng…”
Liu: LlamaIndex closed its Greylock seed round in under a week
“What we really wanted to do was because for us, like, time was of the essence, like, we wanted to ship very quickly and still kind of build mindshare in this space, we just kept the fundraising process very efficient. I think we basically did it in, like, a we…”