Fine Tuned Model
topic on 3 shows · 7 statements across 6 episodes
Latent Space 
Invest Like the Best 
the MAD Podcast
7 statements about Fine Tuned Model, every show
Small specialized pre-trained models can match capabilities of larger fine-tuned models
“When you think of the fact that by doing specialized pre-training, you can train a smaller model, which is as capable as a much larger model when fine-tuned.”
Zhang: Deploying smaller fine-tuned models in AI agents improves performance and latency
“You know how your agent is structured, you know the places where you need models to run, and you can take, you know, smaller fine-tuned models, and that improves the entire system, both in terms of performance, but also latency, and so on.”
Martin: Cognition's Devin uses a fine-tuned model for context summarization
“Devin uses a fine-tuned model for doing summarization within the context of coding.”
Dohmke: Most company codebases are too small for meaningful fine-tuning
“If you look at the individual repository or set of repositories, that code base isn't actually big enough to have a meaningfully fine tuned model.”
Dohmke: Context-aware AI agents with tool calls outperform fine-tuned models
“So this iterative process that the agent does with the help of the tool calls in all the context makes it so much more powerful than a fine-tuned model could ever be.”
Godement: Developers will rely on continuous, automated fine-tuning within years
“The vision we have is, fast forward a couple of years, I think, like, most developers will essentially, like, have an automated, continuous, fine-tuned model. The more, like, you use the model, the more data you pass to the mobile provider, like, the model is …”