world models

“Having deeper models of how, let's see, what are the underlying structures of different domains, how they're related or not related, I think will be an important ingredient for models actually being able to make novel contributions.”

“I think it's pretty clear that as these models get bigger, they have a world model, and that world model becomes better with scale. So they are implicitly developing a world model, and I don't think it's something that you need to explicitly model.”

“It's just a matter of like data scale and like the underlying fundamental architectures. But like, I don't think it's going to change that much. We're just like, we're going to put thousand X more data, thousand X more compute, and we'll get like the best phys…”

“I think this is a matter of scaling data and bettering model. I don't think there's anything fundamental that separates these two.”

“I think sequence to sequence is actually in world models, I think we are going to see algorithm or architecture beyond sequence to sequence.”

“Our world models have mouse sensitivity, which is something that, like, gamers absolutely want, right? So you can have these, like, very rapid movements, which you couldn't do in any other world model.”

“I think that there will just be labs coming at this problem from both sides. And everyone ends up in roughly the same place, and the same place will be whatever people think is cool.”

“However, my understanding is they're currently not interactive, which in my opinion is like the whole point of world models, right? It's environments. They're great environments. And I think from a business perspective, I think they picked a really important p…”

“What world models do is they actually have to understand the full range of possibilities and outcomes from the current states and based on the action that you take generates the next states, right? So the next frame. And so it is a much more sort of complex pr…”

“I think text prediction is just one of the actions that is going to come out of these, you know, these policies and world models. I think speech and text generation will just be one of the actions that, that can be a part of that.”

“I think one challenge that people have seen is that, like, world models perhaps designed for one specific use case might not generalize to others.”

“My, like, take on world models is that, like, we have not yet defined, like, what a world model is.”

“I don't think about world models that often. I think because world models are just not really well defined in the first place.”

“In cosmos, we basically, we had all the pieces sitting around. We working on world models, we working on data analysis agent, working on literature agent. And then we're working on, you know, we built a platform for scientific agents. So we had things that can…”

“Your Git repo is like a distillation of all of the work that people put into the PRs, into the commits, and so I think there's a nice analogy between a Git repo and what a world model is.”

“With language models, when you get features, you still have to do auto interpret and things like that to actually get an understanding of what this concept is. But in image and video and world, it's like extremely easy to grok what the concept is because you c…”

“But it's a lot easier to change with code, as opposed to a model that is Learned primarily on data of real world and virtual worlds that are, I guess, like for example, junior, like there's actually trained on a lot of real world data and a lot of virtual gami…”

“I'll maybe start with where we see the platform in three years, which is like, okay, the users would tell us what they want to achieve. The end goal could be, hey, I just, I want to make something to teach my kids the value of humility. Or it could be, hey, I …”

“Depending on your end goal and purpose, the values should differ. So in the context of games, then the most direct way of measuring is how much time are people actually spending in this world that you create? And if your goal is to say, for example, in the con…”

“You only actually have a world model if you can predict, given some action is taken, what is going to change in the world because of that, and in particular that becomes hard over longer time scales, so if you're simply, you know, trying to predict the next vi…”

“I think it's a mixture. I mean, yeah. I mean, there's clearly a visual component of this and it's not that You know, everything can be text, because of course you want to give a visual look, but there's also a massive amount of giving the overall picture of th…”

“I think if you're hoping to do real world deploys and you're purely relying on a world model approach, you probably won't get to something that works before you go bankrupt.”

“A full world model must also be recursive, meaning that the participant in the world model must also be aware that they have a world model. Which is like this whole recursive thing down the line. But yes, and that the world model can be wrong, and that they ne…”

“So word model is like real time, interactive, long horizon videos.”

“World models, don't get me wrong, are cool and everything, but you and I both know that that is a layer of abstraction that is sometimes not usefully precise enough.”

“Unless I'm misunderstanding something, I don't think that they could entirely converge in principle because if an AI is generating exactly the same world that it exists in, The signal to noise ratio is, is non-existent. Like part of what makes us so flexible i…”

“That immediately distinguishes us from other so-called world models, whether it's video models, vision models, they all assume during training and inference, it's a fixed resolution.”