GPT 4o Mini

“So, and the, this growing collection of open models includes some of the best models that are already on par or surpass, let's say, Plot Sonnet or even GPT-A for Mini.”

“GPD for a mini can't find, ah, cannot evaluate the hard outputs that's 3.7 might be doing correctly, right?”

“I haven't done the math on it, but it's probably cheaper to use that than to host, self-host any of the open source models.”

“4.1 mini is actually quite significantly better than four o mini and not that far away from the old four o.”

“The original, I think the original, I don't know, what was it, GPT 3.5 or something, was like a hundred x the cost of GPT-Foro Mini today. In, in the API. So a couple years, you've gone down two orders of magnitude in in cost. For much more intelligence.”

“And so we looked into all the various models we had and then we picked, at the moment, it's gbd-for-o and gbd-for-o-mini, Which we basically did a fine tune to really optimize for our use case, and then basically shipping that in the app as Ray one and Ray one…”

“And so we find like the open AI models function calling wise for our use case. So for the best ones and like, yeah, basically verifying the others. We had a beta group and testing different models. So basically from all providers and yeah, find basically the b…”

“The reason it was a hundred dollars, if I ran the optimizer with GPT-Foro mini as the LLMs, it would be 10 dollars. But we use GPT four. Oh, just because I think we did this at a time where many hadn't come out yet.”

“I think this is actually a very common pattern that we're seeing a lot of the interesting products built with AI. You use different kinds of models. So, yes, four O mini is for PDF extraction, and then O one for the reasoning, because it's actually very hard t…”

“OpenAI itself has now enabled distillation internal to its own API. So you can use O-one, you can use even GPT-IV or IV-O to distill it down into a much cheaper model that's internal to them, like GPT-IV, IV-O-Mini. And that's sort of their, you know, lock-in …”

“Yeah, I sat in the distillation session just now, and they showed how they distilled from four to four mini, and it was like only like a two percent hit in the performance, and 15 X cheaper.”

“If I call a GP for a mini 10 times and I do a number of drafts or summaries, and then I have four, oh, judge the summaries that actually is net savings and like a good enough savings then running four, oh, on everything, which given the hundreds and thousands …”

“Actually, the new response format is only available on two models. It's Foro Mini and the new Foro. So the old Foro doesn't have the new response format. However, for function calling, we were able to enable it for all models that support function calling, and…”

“Oh yeah, it's already in production, and for like, 90% of tasks that we're running, it's good enough, so that was a quick switch. And within a day, we can know very quickly, like, that, yes, this is, like, good enough, and we have the right evals.”