Galileo

“In the, on the original leaderboard, we also have flashlight, which is, nobody talks about, I think, even now, but I think it's got a very decent score, which is, like, in the top five models. It's extremely cheap model, like, it's so dirt cheap that it makes …”

“Two years back where we were using our own similar LLMS judge methodology, and we had certain issues and there were some correlation issues at the time, but last year it got pretty strong and this year I feel it's so strong. LLMS judges is so strong that you c…”

“Then another surprise for me was that the reasoning models were not performing well enough. They had certain kind of limitation when we probed into it, like, why are they scoring less overall? They were like the O-one, the O-four, O-three, they, When not perfo…”

“When we released the leaderboard and just in a week that launched 3.7, And that went straight up, and nobody has beaten it so far.”

“Another, of course, the other surprise was that all the Lama models were not performing well on our benchmark. 3.3 and even the Lama four all were really performing extremely poor.”

“Then we saw with V-one that the scores are saturating. We see that the best score has already reached .95.”

“The leaderboard, if you are using different judges with different models, it can, there can be heavy shakeup of the leaderboard also.”