diffusion models

“I think there's a couple companies who make, Tens of millions of dollars of revenue from, yeah, from LLMs or diffusion models for porn”

“The idea of, like, there should be a piece of the generative pipeline which is, like, thinking about the answer and coming up with a sketch of what the answer looks like before you start out putting tokens. That's where it kind of feels like diffusion ought to…”

“So if this Delta T here is very big, you see it goes, like, far from XT to X minus Delta T, then the error that It can have is very big and that can put it on a different trajectory. So you get the wrong trajectory.”

“They found in this paper that it's actually inefficient, that you end up with a lot of wasted a way wasted backwards process backwards diffusion steps that you don't need and you can cut out a whole bunch of it just by using a cosine schedule.”

“The locations that you're learning are not basically on the same in the, in this latent space. They're not in the same trajectory As each other, right? So they like and this causes a lot of inefficiency, and that's sort of the whole point to this, the, these …”

“With the regular diffusion models that we all know and love, they have to iterate multiple times generally to generate a good image. Whereas The, these consistency models are designed so they can generate a good image with only one pass through the network.”

“And then this probability flow ODE is sort of a deterministic version that looks at what is the maximum likelihood path If I started at that trajectory, right?”

“The challenge is that, yeah, the training objective is quite different because you are training based on denoising as opposed to next token prediction. Diffusion models are not causal and that is also kind of problematic. I mean, it's a big advantage of diffus…”

“Well, you can use architectures. I think that at least the shapes you, that, that can be leveraged. So you don't have to reinvent and necessarily completely different neural network architectures can, a lot of the data can be used. Like I think perhaps there a…”

“Diffusion models. Not surprisingly, they work pretty well at the infilling where you really need to be able to use context to the left and to the right.”

“On the inference side, what we're seeing is that diffusion models are much more efficient. We're actually able to Pareto dominate autoregressive models. If you think about the typical trade-off between throughput versus latency, which you kind of like cannot, …”

“If it happens, it's gonna be driven by efficiency. Like we're all constrained by essentially power. And if you have, I mean, at the end of the day, it's all an inference game, right? Okay. Training is expensive, but then the thing that matters is being able to…”

“I'm pretty optimistic about a future where diffusion models Can become the dominant solution. I've seen it happen before with GANs a few years ago, so I wouldn't be surprised if that's the case also here.”

“Left to right reasoning for code doesn't actually really make sense, because in code, we don't, like, we might sometimes write code left to right, but after you write code, you go up and down and figure out, hey, is this variable set? Did I do this? There are …”

“Diffusion models today are, I would say, 80 to 90% of the quality at one-tenth the cost and latency.”

“It turns out that the mathematics that we use for diffusion models, but even for reinforcement learning, for Schrodinger bridges, for MCMC sampling, has the same mathematics as this theory, this physical theory of non-equilibrium Systems.”

“For all the same reasons that those models were really tricky to train for images. Mode collapse was the most famous sort of problem. They didn't work very well for proteins or protein ligand systems. And we sort of had to wait for the right primitive to get c…”