Hallucination

“But I do think hallucinations can be also very helpful for AI when you want it to explore novel kinds of proteins.”

“If the POC is very, very short is that those kind of products don't scale in production and then things starts to break and it's extremely, extremely hard to fix those mistakes because at some point, if you use a very simple architecture of one prompt, then th…”

“Yes, it's an abstraction, but it's a very useful abstraction. It helps us to build up confidence that a particular model output is actually valid. Whether you did not just straight up hallucinate at something, because yes, ultimately what matters is binding af…”

“So, so hallucination at least the intuition that people have is that it can come, for example, from SFT, and it can come from this, like, pursuing pipeline, but if you have good reinforcement in pipeline, that shouldn't happen too often.”

“There are companies which I looked at, which said, well, AI hallucinates. So we are going to go do this. And my thinking is, by the time you go do this, AI will stop hallucinating.”

“We've seen that models internally have some awareness of like uncertainty or some sort of like user pleasing behavior that leads to hallucinatory behavior.”

“And that is very, very powerful because I think this is one of the Ways you can mitigate not totally mitigate, but let's say reduce hallucinations because then the LM suddenly doesn't have to remember everything anymore.”

“There was these things called hallucinations. It's still with us to some extent, but dramatically less than two years ago.”

“The way to, like, get them to behave the way we want them to behave is just with context engineering and injecting the right knowledge for the right kind of tasks they're doing. And that's incredibly effective, right? That reduces hallucinations, that makes th…”

“The current systems we have, the current technology we have, if you put the right context, it is the right answer. And you can decrease a lot hallucination percentages just by creating the right context.”

“LLMs are non-deterministic, right? Now we can't just Put in a command and get something back and accept it. We really need to evaluate it. So, you know, are we seeing hallucinations? What's the reliability? Does it meet like the style that we would typically w…”

“Saying, I don't know, or solving, you know, hallucinations is, ah, intrinsically requires reinforcement learning in many ways.”

“When the models are able to take step by step, they actually can like pause before blurting out an answer is kind of what I, it feels like with a lot of the previous models or hallucinations.”

“Initially, there will be hallucinations, but if you design it from a systems perspective, you will have the control loops in there to kind of, you know, prevent them, identify them, prevent them, and [406] Yash Hemeraj: Work on mechanisms to fix it.”

“Hallucination is like a feature and not a bug, especially in creative industry, because if you're generating a let's say a banner for a newsletter, you want it to be creative, quirky.”

“People talk about hallucinations as this bad thing, but in a lot of ways, for like, kind of this speech-to-speech interaction model, where you're talking to an AI, Hallucination is a feature.”

“I think agentic comes after you solve the hallucination problem because otherwise you've got these long chains where you can introduce hallucinations. It'll kind of work, but it'll work much better after.”

“So by that, by this kind of mechanism, you're, like, systematically able to actually mathematically prove that you got the right answer coming out of this and completely eliminate hallucinations for that area, right? It doesn't mean that we've eliminated hallu…”

“Like in LLMs, hallucination is a problem. So the fact that you're making shit up is a bad idea. But if you have to make shit up to convince someone, LLM is great and perfect.”

“The amazing positive impacts of these generative models and their flaws are very intertwined. What affords their ability to be creative and responsive and aligned with the person using them is also the basis for hallucination.”

“Fine tuning in many cases doesn't work because you need a lot of data to see the results and there are still hallucinations even after fine tuning.”

“And I think hallucinations are gonna, the rate is gonna go down as the models get better, but it's never gonna get to the point where it's a hundred percent perfect.”

“The hallucination problem, that's in some ways a good thing for startups because a lot of the existing products out there have zero tolerance for anything that's going to have a risk of producing something wrong.”

“When we talk about hallucination, even human hallucinate, right? Hallucination is like, okay, you don't know the answer, but you try to like relate it to something else and try to go and answer something. And which simply means that, okay, it's not a precise a…”

“What like every enterprise, the way that they're going to go and adopt the AI is like, okay, here is this, you just, I want you to just play within this boundary. Here is the information that I'm providing you. If you can get the answer, then answer it. If not…”

“And our approach to it is, we're not, quote unquote, solving hallucination. We're solving Unintended hallucination, which would be something like, what's the distance between point A and point B? Or what's one plus one? Like, that should always be two.”

“And so finding the way of harnessing that as a feature rather than a bug, I think is going to be challenging, especially when one of the perceived shortcomings of your product is actually going to be, is like the core of what makes it function. I don't know th…”

“And we also wanted to figure out the issue of hallucinations, which is always an extremely hard problem. But I do think that with this method of tree enforcement learning with human feedback, maybe that is all we need if we push this hard enough.”

“And there is a ton of work that needs to happen here and maybe introducing browsing so you can get fresh information and you can cite information and solve hallucinations. I don't think that's impossible. I think that's achievable.”

“We figured that, I don't know, problem of hallucinations out before anyone else.”

“So I think in some cases it is a feature. If you want to use a language model for creative writing and if you want it to be really, really creative, then you probably want it to hallucinate. So in a way it's a spectrum of groundedness and hallucination where i…”

“You'll hear this term, of course, hallucination, which is kind of when it starts to make things up. And of course, another term for hallucination is just simply creativity.”

“You're going to have models that check the accuracy of other models, and that will help us bring down hallucination to zero.”

“It may actually be easier with small models than with big ones to reduce hallucination from them, but it, you know, I think it's still an open question”

“So I think there's pre-training, which is predicting the next word and developing a world model, so to speak. And with those capabilities, then you can, you still have to say don't hallucinate, but it will be much easier to control that model if it has a notio…”

“If you look at a completely pure case of a transformer model trained on a bunch of data, It doesn't have any mechanisms for truth. Now, except the sort of accidental contingency, and there are inherent reasons why these systems hallucinate, and maybe I can, in…”