AI2

“Yeah, it was like Whisper, and I don't remember the details, and I had asked, and it's like, I don't think they actually logged the audio. I was like, oh, this would be super cool for, like, other types of multimodal, but I don't think it was in the terms of t…”

“The model didn't work on clocks and then the lead was really on clocks and no models work on clocks. So they're like, we've got to make it work on clocks. One of the interesting things is that it doesn't work on dials, even though it works on clocks.”

“They got about a million captions for 700,000 images, which I'm like, okay, that's kind of expensive.”

“This is just, like, straight base model, no real instruction tuning. There's literally, like, no chat template for multi-turn. It just concatenates the messages together and, like, there's, like, go, look, good luck.”

“The things that this model is good at are things that all the foundation companies, like they're just going to take our data and train on it.”

“We see a lot of these even in our own work of like, you know, as we iterate in the various version of Olmo it's not just like every time we collect from scratch all the data. No, the first step is like, okay, what are the cool data sources and datasets people …”

“We got a tip from a industry lab member to do this a few months early. So we got a head start”

“Like Olmo-Thirty-Tube is if you squint like original GPT-IV level and fully open.”

“If you're trying to have impact in AI right now, it's as an academic, you have to like level up out of papers to artifacts, which is models, datasets, evals. Datasets and evals are easier for people to have impact on.”

“On, like, core evals for our Suite of models from, I think, eight, seven D and four or five B is based on llama at the time. It's like it matches or beats meta on these core valves.”

“It's out of 18 currently. And so we've got an openness index page, but essentially these are points. You get points for being more open across these different categories and the maximum you can achieve is 18. So AI two with their extremely open OMO three, 32 B…”