The Exchanges

Q I want to come back to impact because I think the ramifications here are, are, um, really huge. But, uh, if we just go and, like, think about first problem design, you, I think, looked at 52 problems. Why that many? And, like, how do you specify a target? I'm picturing, like, bind to epitope X, but I'm sure there are other requirements you'd want to have as drug designers.

A It's a great question, Sarah. So in the CHI-II paper, we look at over 50 targets. Most of the existing, uh, papers in this area of doing AI for, for drug discovery are usually looking at like one, two or three targets. But again, it was important for us if we were seeing this as an engineering problem to make sure that this is going to be generalizable. It's like, imagine you had a new LLM paper and you said, oh, I solved like one problem in the USIMO, uh, contest, like really, really cool. It's like, yeah, you need a real benchmark and you need to actually have that benchmark at scale. You need to have enough Problems to convince yourself that the system is working. So that's why whenever we do these experiments, you know, sometimes we'll, we'll try one or two targets just to make sure there's not like a huge bug and, you know, make sure not everything fails. But, you know, even if everything fails in one or two, you know, the hit rate's 50%, you could have just gotten unlucky. So that's one of the reasons why we decided to do a big benchmark here, really convince ourselves things are working. The way we selected the 50 problems, the biology people would laugh at this and engineering people would love it. We actually just went to the vendor catalogs to see what was in stock. Cause we wanted to turn around this experiment quickly. We ordered all of these designs at the same tim…

AI assessment note: “it was important for us if we were seeing this as an engineering problem”

Q We have a broad audience for no priors that ranges from like Business people to engineers, machine learning researchers, some scientists in other fields. Like, what intuition can you give listeners for how the model works under the hood? Like, especially for anybody who might start with, um, some familiarity with, like, structure prediction models.

A Yeah, well, structure prediction is really a key part in making these models work, and it's actually the first thing we did when we started the company is we sprinted to build a state-of-the-art structure prediction engine. We actually open-sourced the first version of that. It's called Chi-One, and again, like, Scientists around the world are using that now, but structure prediction basically gives you an atomic level microscope, and it allows you to see where atoms are placed in three D space. So once you can do that and you have this microscope, then the next question is, well, can we start moving those atoms around, right? We can now start to make changes in a sequence, and then we can see the ramifications of those changes in three D space. So the actual design model, you can think of it as a, you prompt it with some information, like here's a target. Uh, that we want to go and, and, and design, uh, an antibody against. And then the model will, will try to place again, these items in three D space in order to satisfy that constraint. Like we tell the model, here's the target and I want you to make a molecule that, you know, binds to that location. And then model will go in and generate, uh, both a, a sequence and a structure that, uh, that kind of fits into that. So that's like the high level intuition for this.

AI assessment note: “the actual design model, you can think of it as a, you prompt it”

Q What has the reception been like so far? What is the biggest objection? Because this is a, you know, significant challenge to the ideas of high throughput screening or even like the workflow that, um, you know, even innovative pharma and biotechs have today.

A Yeah, it's a great question. You know, usually when these kinds of, uh, papers come out, again, people have, Try to, to do this many times. The critique is, is often, you know, does this really work? You know, you, you show this on maybe COVID, for example, is this going to work for a case where we have less training data? Are the molecules going to be high quality? Do we really, you know, kind of believe the data? So I think the approach we did, like benchmarking this at scale has really helped a lot with that reception. Like I think people really appreciated that approach, which has been great. Some of the questions people have is, okay, like I can already discover drugs. So Uh, you know, so now I have AI that can do it a lot faster, but does that actually change the kinds of molecules I can work on? And it goes back to what we just discussed before. I think there are other folks that are responding to that saying like, no, like the, the transformation here is how about those projects that didn't work for you, uh, or, or where you're really struggling today. Now you've got another tool in the toolkit and you kind of have to use this tool now, or, or you might be left behind. So I think that it's been really interesting to see the, the community kind of digesting this. Of course, a lot of the AI folks are, are really excited, right? Like we're getting artificial antibodies, uh…

AI assessment note: “The critique is, is often, you know, does this really work?”

Q That actually begs a question I was, um, going to ask you is, like, if you are an antibody engineer or a biologist today, what advice, you know, given, let's say they believe you about how much is going to change and these, like, CAD for biology, like, software suite that is, um, coming into an existence, like, what should they learn, be good at, like, go study?

A Well, number one, get access to try two. Number two, you know, figure out how to, to get your prompts right and, uh, and actually take full advantage of it. And then I think number three, you know, start dreaming about, uh, the new possibilities. You know, it's interesting. Uh, we've talked to a lot of, uh, antibody engineers since, since starting the company and we've been alluding sometimes to, you know, what we're doing here, you know, uh, sometimes you do the market research question. You ask, you know, suppose you have like a one percent success for an designing antibodies, like what would you use that for? Uh, the conversations are changing now that first of all, it's not one percent, it's 10%, and like people see that it's working. I think that creativity is really being unlocked, even, even ourselves, right? I think when people are thinking about the answer to that question, there's always some big doubt in your mind. It's like, ah, it's a hypothetical question. You know, your neurons are not activating in the same way of doing something with it. It was the same thing with LLN. It's like, imagine asking someone five, 10 years ago, oh, you know, if we could predict the next word in a sentence perfectly, like, what would you do with that? It's actually very hard to imagine until you start playing with the models, even, even our team internally, you know, now, uh, even wit…

AI assessment note: “Well, number one, get access to try two. Number two, you know, figure out”

Q Is there an important future for, like, large-scale wet lab screening? Does it just become a data collection exercise to fill out the distribution for CHI models? Are there areas where you will, you think we'll need that in 10 years, 20?

A Yeah, I think if you just take the models and then you sample more, you probably will get a better result. So we tested only 20 molecules per, per target of the paper, up to 20 molecules. You know, if you were to do 10 times that, a hundred times that, orders of magnitude more, you probably just Get into spaces, uh, with better, better molecules. So, you know, the machine learning model is probabilistic. It's like using ChatGPT. If you, if you're trying to solve a math problem and then you look at the top one response, or if you look at the top 10,000 responses, you're going to get a better result. If you look at the top 10,000, you can't really do that with a product experience on ChatGPT. I'm not going to look through 10,000 math responses. I won't even know which one is correct. The cool thing with the lab actually is we actually could just test all 10,000 of those in the lab. So I don't know if you have to. Uh, but that's definitely something that is, I think, going to be tested out with these models, and I think the future of high throughput screening and how they kind of interact with the models, I think the question is still open, but I, I expect that, uh, you know, people will be creative and, and we'll find ways to actually take the best of AI and marry that with the best of biology, uh, to kind of push the balance forward.

AI assessment note: “The cool thing with the lab actually is we actually could just test all 10,000”

Q Yeah, that's completely wild from a precision of prediction perspective. Uh, you know, if we analogize to LLMs, you know, you have learned grammar, syntax, semantics, capabilities that emerge in the model that you can measure. Is there anything that would be analogous in terms of emergent vocabulary or concepts that you think CHI-II has?

A Yeah, I think this whole point about the atomic level microscope is actually that point, right? There is something really, I don't know, I think deep, we still don't fully understand it about like why these models work. Again, we didn't even know this was possible. Obviously we tried it, so we thought that there was a chance. And I think it just tells you something about, you know, maybe the signature of how proteins interact with one another is really embedded in the data, right? And we're generalizing to a new setting. So it's not like the model has seen You know, specific binders against the target, and then we're just trying to do some in-domain generalization and walk through that space. That's actually quite an impactful application as well, and that's already being done through the biotech industry. Our team published work on that years ago already, but I think this really new frontier about generalizing to new space, it tells us that, again, like the model is learning something really fundamental about how the molecules interact with one another. Again, it's able to generalize to problems that look very different in terms of how we would Actually organize it in the biology. I think the whole rules about, you know, what do we think about like a, a protein family being different? These targets that we tested on are, are again, a biologist, they are very quote unquote diss…

AI assessment note: “the model is learning something really fundamental about how the molecules interact with one another”