The Exchanges

Q actually using transformers for at least the mercury models. Yeah. Um, and basically does it have to be trained end to end, or can you just take any existing open model backbone and, uh, uh, and, you know, do this process? Like, do you have to initialize from zero basically, or can you just, can you lean, can you sort of leverage off of existing, uh, other, other language models?

A There have been, uh, attempts in the literature, um, where people have tried basically using, um, existing, uh, open source models and then kind of like fine tune them. The challenge is that, yeah, the training objective is quite different because you are training, uh, based on denoising as opposed to next token prediction. Diffusion models are not causal and that is also kind of problematic. Uh, I mean, it's a big advantage of diffusion models that you don't have to be causal. You can actually look at the whole context to the left and to the right. As you decide, kind of like the edits that you want to do, which is giving, you know, it's a, it's a very powerful thing if you think about how you generate objects, but that makes it quite different, quite difficult to adapt kind of like models that have been trained with, with causal masking to, to this, to this, to this, to this new task. And so there have been attempts in the literature, annealing the, the, the masks, uh, the attention mask, like there have been various, Tricks and attempts at doing that, but it's not straightforward.

AI assessment note: “makes it quite difficult to adapt kind of like models that have been trained”

Q And to wrap on maybe the pre-training phase, is there also post-training in the same way that people do with normal LLMs? Well, not normal, but you know what I mean? Like instruction following and things like that. Does that work the same as those models or is it different in diffusion?

A So our, uh, pipelines, yeah, we have something similar in the sense that we also use, uh, we can do basically SFT and we can also do, we do reinforcement learning, RLHF. We do reinforcement learning from German feedback. Again, some of the algorithms have to change. For example, we have a DPO algorithm specialized for diffusion language models. I was one of the original authors of the DPO paper that was originally designed for LLMs, autoregressive LLMs. I worked on a paper extending DPO for diffusion models, more continuous diffusion models, like image generation, video generation. And that's also been quite impactful Like if you look at some of the papers that have been released on some of the leading open source image generative models based on diffusion, you can see that the DPO piece using human feedback to really align the models to human preferences is really important if you want to improve quality. And we've extended a variant of DPO to diffusion language models. So we're actually also able to use preference data to, to fine tune the models. And, uh, but yeah, basically we have pipelines that are compatible with the Existing SFT and RLHF data sets. And we, we do that with customers. Like we often work with customers that have their own internal data sets. They want us to fine tune the models on their proprietary data, and we can easily do that. It's all pretty easy once…

AI assessment note: “we have something similar in the sense that we also use, uh, we can do basically SFT”

Q cognizant of the Serving requirements between autoregressive and diffusion. A lot of people rely on the batching characteristics in, in production. You would not have that, right? Are you aiming to, like, uh, you know, are you aiming to be, to abstract that problem away from people? Or are you just going to release the models open and open ways and then people will deal with it however they want?

A So we don't have a plan at the moment to release models or to open source any model. And in fact, it's a little bit tricky even to, even if we wanted to release, let's say a small, maybe less capable version of the models, we'd have to release the inference code, which is also proprietary. Unlike an autoregressive model where the inference code is kind of like straightforward, there's not that many ways you can do inference. In an autoregressive model, you can tweak a little bit the sampling, but it's, it's pretty straightforward. In a diffusion language model, there is a lot more knobs and a lot more innovation that can happen on the inference side. And so it's very hard for us to open source a model because we'd have to release the inference code. And by the way, we have our own inference engine. So just like you would normally serve an LLM using a VLLM or SGLang or a Tensor or TLLM, we have built our own inference engine. And so we are supporting production traffic already with our own inference engine. We support continuous batch and quantization. We support a lot of the features. That are supported in these serving engines, caching, like prefix caching, like a lot of that stuff is already handled by our own serving engine, but yeah, it gets a little bit tricky if we wanted to, to open source because a lot of the IP is around on the inference side as well.

AI assessment note: “we don't have a plan at the moment to release models or to open source”

Q Not, not like, you know, is there a why now? Obviously you have some secret sauce, but like, I'm just trying to figure out like, is there anything fundamental?

A I mean, it depends how you define fundamental. I think there was some math that developed in terms of like thinking about, I like to think of diffusion models in terms of like learning score functions and, and kind of like gradients basically of probability density functions, which are not really well defined in the discrete world, but there are kind of like mathematical analog goes objects that you can kind of train using Denoising, like objective, like score matching, like objective. So there was some pretty fundamental math that had to be developed. Then there's also a lot of engineering and, and, and tricks that do to build on top. And then I would also add the fact that, you know, once somebody shows that it's possible, you know, suddenly other people are much more incentivized to, to get their own thing to work. And, and I think everybody progresses more rapidly once there is kind of like a proof of concept that, that something is possible. And so, yeah, we've seen, you know, that the, the Google guys also have a, have a model. So it's exciting to see that the, the field is really starting to, to pick up speed.

AI assessment note: “So there was some pretty fundamental math that had to be developed.”

Q something that people should not be using the fusion models for, or do you, this is just kind of In your mind, it's like a superior, just swap everything over to these. And then as we scaled them, it's funny because you're seeing image generation go towards autoregressive models, and then you're seeing the text models go to diffusion. How do you kind of see the model use case match?

A So the API is the same. It's like text in, text out. So in principle, any use cases that is supported by an autoregressive model is also supported by a diffusion language model. In terms of capabilities, yeah, as I mentioned, like we're not Frontier level. And so there's certainly use cases where you probably wouldn't want to use a diffusion language model, or at least not the current, uh, existing diffusion language models. And so, yeah, I think at the moment it's more a function of kind of like what's the intelligence level, how much do you care about latency or cost? So there's definitely a bunch of use cases where people, where diffusion language models are already competitive. They're already the best solution for those use cases. Uh, there are others that are still yet to come, and I think that will require, uh, scaling up and making further improvements to, to the, to the training and the data and various kinds of things. I'm pretty optimistic about a future where, um, diffusion models Can become the dominant solution. I've seen it happen before with GANs a few years ago, so I wouldn't be surprised if that's the case also here. Um, I really like this idea of using context to the left and to the right. The fact that you have error correction that is built in. So you think about an autoregressive model. Once you output something, you can never take it back. And so if you w…

AI assessment note: “there's certainly use cases where you probably wouldn't want to use a diffusion language model”

Q engineer, I think that's a win for us doing this, right? Okay. So I want to move on a little bit in terms of like we, we, let's say we have production and high quality LLMs. You made an interesting controversial statement earlier that you said most frontier models or most language models will use diffusion in the future. I don't think that's consensus. Uh, could you just elaborate why?

A I think that there is a chance. I mean, I, I don't know if that's gonna happen, but I feel that is, there is a world where that could happen. Um, and, and I think it could be driven. If it happens, it's gonna be driven by efficiency. Like we're all constrained by essentially power. Uh, and, uh, and, uh, if you have, I mean, at the end of the day, it's all an inference game, right? Okay. Training is expensive, but then the thing that matters is being able to serve these models cheaply and without consuming more energy that we have access to. And you only have so many gigawatts of Power and so many data centers, and there is a growing demand for tokens that you need to be able to serve to users and all kinds of apps that are being built. And so there is a real need to find a solution that can give us maybe a 10 X improvement on the, on the efficiency side. And there's a really good chance that diffusion models are the thing. So if indeed it's possible to match the quality of frontier models in terms of quality, if you have a win on the, on the inference side, that is going to be the solution that will be Uh, that will be deployed.

AI assessment note: “If it happens, it's gonna be driven by efficiency. Like we're all constrained”