Fine Tuning

“People hear, you know, open model and immediately want to train their own model, and I get it. I get why. I was actually the same way. It sounds really cool, but I feel like that's like an advanced move. The practical move, the beginner move, where you should …”

“We might see a future where, like, when you do a fine tune, and you want to, like, change the base model layer, it only costs, like, maybe a few hundred dollars, maybe a few dozen dollars to change it.”

“I think a common misconception that people have is, you know, fine tuning is, is a way to like customize it for that customer. In fact, most of the fine tuning we do is like customizing it for our use case, like the customer service use case.”

“I'm sort of teaching the AI my own procedures. And again, that doesn't happen through fine tuning. That, that happens like in context, because if you were to fine tune on that, you would have to reverse it every single time. You know, you change your procedure…”

“You can't fine tune your way to success, right? Major capabilities emerge from training a base model made accurate and useful during fine tuning.”

“What I'm seeing more and more is there's a lot more people going into the no fine tuning camp than a couple of years ago for very high value enterprise use cases.”

“Just changing how the model behaves, you can do most, most of that via prompting nowadays, and in terms of capabilities, the models are very good out of the box.”

“MOEs are challenging to fine tune. I don't know if we've talked about that in the past, but MOEs in general are like an extremely good architecture. They work great for inference. But when people fine tune them, they struggle a bit. Like they are not as easy t…”

“It would actually really slow us down to have a model that was fine tuned on our tools because we'd have to retrain it and cut a new model every time we did that.”

“The more focused your use case is, the smaller you can make the model through fine-tuning or through just distillation in an even smaller architecture.”

“If there is a core capability that you actually care about, that capability should be part of the foundation and not a fine-tuned artifact.”

“When it comes to like control and design of models, you know, we tried steering with our first API and realized that it still fell short of black box techniques like prompting or fine tuning.”

“I do expect the part of the software in those deployment to increase. So the amount of the way customization occurs today with fine tuning, reinforcement learning, this kind of things, this is going to be abstracted away from the enterprise buyer because it's …”

“I can offer anecdotally that a single negative example goes a long way.”

“I would say for 90% of use cases where you aren't forced to a smaller model, then it's still not a good ROI, and you probably shouldn't invest in it today.”

“The dollar cost, I would say, is basically never a factor. It's just so much less than the time, the amount you're spending this engineer to do the work that it's not, I mean, it's, you know, each of these runs is between five and a couple of hundred dollars.”

“I think it's gonna open up a lot of this democratization of accessing these models and building incredible things. Cause you don't even need to do a lot to get the most sophisticated outcomes. Increasingly. I don't think you need to do a lot of this heavy duty…”

“I think a lot of people have abandoned fine tuning and said, actually, I'm just going to do better context management.”

“Not out of the box, but with a lot of that fine tuning and that the training, et cetera, you are able to kind of close the gap and even have better performance on metrics.”

“50% of developers according to surveys are now fine tuning.”

“It is possible to build a model, but I think it's also possible to fine tune a model which will get you 90% of the way there.”

“Updating the weights of the model is not a very good idea because you cannot really deterministically control that. You can fine tune, but what you're going to get the other end, you know, again, you don't really control.”

“One common misconception about fine tuning is a lot of people think that you can inject new knowledge into a model using fine tuning. And that is not true.”

“You talk to people at DeepMind And they don't even fine tune anymore. Because they have such, you know, what's called big context window”

“And I think we're going back to fine tuning where we realize these models are really cosplay. It's better to use just a small model. We try to specialize it. So I think it's a little bit of a cycle and we're going to start to see like more of fine tuning and l…”

“Taking those models and trying to fine tune them It's just, it's not as effective as building it yourself and you have much fewer levers to pull than if you actually have access to the data and you can change the data that goes into that process.”

“For it to be a business, you need to align with the problem, not the technology. And I think that Automatic optimization is a really great business problem to solve. And I think if you're too fixated on fine tuning as the solution to that problem, then you're …”

“There's a lot of cases now, especially with large context models, where in context learning just beats fine tuning.”

“I will say in my own experience with customers as of the recording date today, which is September or something, yeah, very few of our customers are currently fine-tuning models. And I think a very, very small fraction of them are running fine-tuned models in p…”

“Almost if not all of our customers have moved off of fine-tuned models onto instruction-tuned models and are seeing really good performance.”

“It's actually a lot easier to get started than a lot of people expect. I think they might need Tens of thousands of examples, but even a hundred really high quality ones or a thousand is enough to get going.”

“RAG based systems are so Are so common here, and so, and probably will remain, even with the advent of fine-tuning availability, they're going to remain a useful paradigm.”

“Sorry, it wasn't the end of fine-tuning, but more that we should treat it as a continuum, and we should have much higher expectations of how much you can do with an already trained model. You can really add a lot of behavior to it. You can change its behavior.…”

“People are building too many vanity foundation models rather than taking better advantage of fine-tuning”

“Something over the next 12 months people will be spending time thinking about is how to, like, where to use RAG, where to use fine-tuning, where to use KV cache storage, you know, and how to use state.”

“I think the next level of like value and impact is definitely going to be fine tuning to specific voice.”

“Fine tuning in many cases doesn't work because you need a lot of data to see the results and there are still hallucinations even after fine tuning.”

“So we fine tune on the proprietary data of a particular company, and we can see 10 to 20% improvement on top of the domain specific in fine tuning as well.”

“And like, at the end of the day, it's just all meta-learning, right? Like, all we want is, like, the best meta learning workflow or meta learning setup possible to be able to adapt the model to do anything.”

“Like the fine tuning aspect that has been like the go-to solution is probably a little too low level from What we should be doing.”

“We see people experimenting with it. I think the only real place where they're doing it is when they've reached like really critical scale which I still don't think is that many applications to date.”

“That is something that is, works better than fine-tuning, for example, because if you fine-tune, then you're still dealing with potential hallucination Fair enough, with RAC that's possible too, but it's like, it's less it's less risky.”

“I think where is exactly that, where I think is having more legs is when these companies need to customize it to private data sets. So you have the open, general, big foundation model, but then you have to tune it up to specific data sets that, for example, a …”

“Combining all those techniques all together, right? So we'll give you essentially another boost, right? So that kind of one thing that we learn on the technical side.”

“I think you're still going to want to do RAG anyways. Like, even if you have fine tuned models in the mix, RAG is still sort of this, like, last mile data or context.”

“And if you fine-tune a medium-sized-ish model, sometimes it loses the ability to do effective in context learning, because I think the intuition is, it's devoting more, more of its parameter space to, kind of, like, memorizing the training set so it can do bet…”

“You don't have to fine tune your model. It feels very intuitive. We have this great data set. We own it. It's our data. So we need to do something useful with it. So we need to fine tune our own language model. And so the companies who are offering that servic…”

“Unless, unless you're fine tuning for on device use, I think fine tuning current existing models, especially the smaller ones is a useless waste of time, right?”

“Fine tuning is the technology that got from a research project in 2020 called GPT-III and turned that into ChatGPT, a billion dollar app, right?”

“To me, the right way to do this is to fine, fine-tune language models, is to actually throw away the idea of fine-tuning. There's no such thing. There's only continued pre-training.”

“RAG is like such a inefficient hack, really, isn't it? It's like, You know, segment up my data in some somewhat arbitrary way, embed it, ask questions about that, you know, hope that my embedding, you know, model embeds questions in the same embedding space as…”

“That's one of those things where I think long-term, you definitely can. I think some people say you can't. I disagree. I think you definitely can. Just right now, I haven't gotten into work yet.”

“At the end of the day, these models are like next token predictors, which is, you know, like they kind of look at like what they've predicted until now, and then, you know, figure out like what the next token they're on is. And from that, like, even with fine-…”

“Fine tuning an AI model is quite a difficult, complex process. It's a little bit tricky but it can kind of 10 X, five X your results in terms of the quality outputs.”

“If you have three to five examples, let's say, then I would prompt it. If you have maybe 10 or 50 examples, it would either be prompt tuning or fine tuning. I think generally in that realm, prompt tuning and fine tuning perform similarly, and I would prefer pr…”

“So I think there's a strong incentive to make that happen. So the folks who are providing large language models, they want to make their approaches as easy to use as a possible. And so anything that can go into prompting, it seems to me that people will try to…”

“Even the data stuff, which is probably the most like compelling, or at least the moat, the most moat like thing. We're going to hear that. They go, yeah, yeah, that there's a moat there. Like even that I think is dubious long-term. If there's a ton of value, t…”