The Exchanges

Q So I've, I've heard you, um, in other interviews say something very interesting. So you're, you're as deep in the space as it gets, um, but I've heard you, uh, basically offer words of caution, uh, to people, especially enterprises that want to deploy LLMs. Um, can you explain why?

A So it's really hard to tell at, at a glance, How useful something's going to be when you go to the, you know, the chat GPT UI and you play with it, because it's good, right? And like, I use it all the time, and it is great, and you're like, ok, you can, you can, from there, you're like, ok, I can, like, take this immediately and apply it to my company. No, you can't. Right? Like, it's a big gap between a UI where you can chat and even an API to a production-ready system, because you need to have auditability. You need to be able to trace what happened. Like, there's, your data's private. I mean, all of this stuff is going to open AI, right? So there's so many things that go into this. Um, first is the viability. Like, how hard it is to actually put this stuff into production. But second, like, the model lies a lot, right? And unless your product is, like, a creative product where, like, lying is actually a feature, meaning, like, if it hallucinates something, it's a better image, right? Then that's cool. But, like, you don't usually want your models lying. In those cases, you need to be very careful. So even from a research world, it's really unclear as researchers, how do we keep it from doing those things? It's a lot of like ad hoc rules and things you have to do to it. So like, we haven't really figured this out in research. So I wouldn't go all in on this unless you can hav…

AI assessment note: “big gap between a UI where you can chat and even an API to a production-ready system”

Q Um, there is a little bit of, um, a new political economy that is getting formed around, uh, generative AI. You, you mentioned LeadLama as an open source project. Where, where do you think open source falls, uh, you know, compared to the open AIs of the world, and why does open source matter?

A So I think all, all roads lead to open source at the end of the day. No matter what you do or how much money you have, you cannot compete with the world's resources put together to do something. So, uh, some companies will have an edge for a bit, yes, but open source will catch up, and it'll catch up very quickly. It, it will always be open source. Um, AI came from open source, came from academia. Like, trace it back to the very beginnings. You know, some of the early stuff that Jan was doing was always open source, and in fact, I think that's why FAIR is probably one of the best open source labs today, because of that DNA. So, it, it will always come back to open source. Open source, the role it plays, for me, it's about giving back to the community, it's about, um, sharing knowledge. If you've been in other sciences, like neuroscience, where nothing's open source, progress is super slow. Like, you want to get a data set, you can't do it. You ask a lab and they're like, no, but I grew this monkey for three years. I need to publish on it. And it's like, okay, well, it'll take a long time. AI was exponential. The reason why we're here is because it was open source. So turning your back on the community today, you can do it. It's your business and you, you know, you do what you do for, for profits. It's fine. But, um, you know, like none of this wouldn't exist if Google hadn't pu…

AI assessment note: “all roads lead to open source at the end of the day.”

Q Do you want maybe to go into what Llama does for anybody that may not be familiar?

A Yeah, so Llama is an, it's a language model, right? So it's like an open source alternative to like a ChatGPT, for example. So, uh, you can grab it from open source, and then you can use it like you would with ChatGPT, and there's many technical ways of doing that. Now, when that open source repo was released by Facebook, um, they only gave you the code to do inference, meaning to predict with it, and the model weights were also not, like, given out. You actually sign up for them. Now, the code on the repo is GPL, meaning if you do anything with that code, you have to open source that stuff. That's what the GPL license does, which means it's kind of not, and they did it because they want to keep it mostly for academics, so it means it's not usable for enterprises. So we took, um, the Lama, like, paper and implemented from scratch completely in Apache two, and we open sourced it. So it's fully, fully usable for enterprises, but we also gave you the training code, not just inference, and we also gave you fine tuning methods as well. So they're all in there. Now the repo is, has lightning in it, so it's very simple. There's not a lot of boiler plates. It's very readable. It's like one or two files for most things you want to do. It's not hundreds of files, and we want to keep it super minimal. Now the weights are not there. Obviously you have to grab your own weights, but we will …

AI assessment note: “Llama is an, it's a language model, right? So it's like an open source alternative”

Q Great. And what are some examples of what people do with the framework?

A Yeah, so who's heard of stability, AI, and stable diffusion? That was trained using Lightning, right? Thousands of GPUs and AWS. Um, you can go to GitHub repo and see that today. Um, Open Fold also trained with Lightning. There's, uh, NVIDIA just announced all these Nemo services. Those are all powered by Lightning. Um, there's over 10,000 companies across the world who use Lightning to train, deploy. Facebook, a lot of it is powered today by Lightning. Uh, you have other major companies which I won't talk about here because I don't know if I've gotten permission. Um, all the way from banks to self-driving car companies to tech. Probably if a company's using AI today, there's a team internally who's at least using lightning for something. Um, so we work with a lot of companies on how do, how do we help them standardize more of their code across the org, and you know, if you're training LLMs or using APIs, do other things, we can, we can help you do a lot of that.

AI assessment note: “who's heard of stability, AI, and stable diffusion? That was trained using Lightning”

Q Uh, great. Last question from me, and then I'm gonna open it up to, uh, you all. Uh, maybe zooming back out, uh, what's next for the company? What, what does Lightning AI look like in three years from now?

A Yeah, so I think we're, we're focused right now is on these, like, really high-performing models, and, you know, we're in that growth stage of the company, so we're helping a lot of enterprises revamp their current tool sets into this kind of new generation. What we found is most companies have some sort of platform that they build internally, and that's not scaling, and so they're bringing us in to basically say, okay, you guys know what you're doing here, Let, like, help us figure out how to, like, scale AI across the work. Whether you're using other tools, we're complementary to all the tools, right? So the platform that we have is one where you can use all the things that you, you're used to already. Um, so the company will be more of that. I really think about it as, like, if, if models are the rockets that everyone's gonna be going after, we're like NASA. We help you build them, we help you launch them, we help you, or maybe SpaceX is better, I guess? I don't know. So, so that's where we are, right? So we're really agnostic to this. Um, We will provide certain high performance models because, like, I don't know, like Ferrari, we believe that we can build some of the best models around, and we will give them to you, and you just do what you're gonna do with them, right? Um, so it's more of that. It's more of that, um, platform experience, and yeah, I mean, we'll be doing a…

AI assessment note: “Lightning is, like, the standard platform for you to do anything with AI”

Q Do you have a rough prediction on when that problem might be solved? Because, you know, I think we're all experiencing this crazy moment right now where everything feels exponential and compounding, and the problems that seem to be, you know, intractable recently suddenly are solved. Do you think the hallucination problem is something that's going to stay, or is it going away?

A So it's, it's funny because if you, if you were like, uh, doing a re a PhD or you were in research, you could, you would, we've been feeling this for many years already. Like there was like a hundred papers a week and you're like, ah, how can I possibly read all the papers? You couldn't, right? Today, it's kind of like, okay, the world realized that this is a thing. So everyone's feeling the same way now, but in the research, we've been kind of feeling this for a long time. So I think it's still going to be, I don't know, like five years, probably like, I don't think it's going to be like one day suddenly it's solved. I think it's going to unlock industries sequentially. So like, year one, maybe we can do this industry, and then this one. But like, healthcare and finance are like the last ones that we'll have, that we'll actually be able to do there.

AI assessment note: “I think it's still going to be, I don't know, like five years”

Q And are you planning, or maybe you are ready, uh, to offer it as a service, or is that open source and for people to do whatever they want with it?

A Yeah, I think probably unlike most other companies that are out there today, like, we actually don't really care about models. Like, uh, you know, our, our business is to help enterprises build and adopt AI, not to power APIs. So it's actually, our, our incentives are actually truly aligned with the community, which is like, we just want to give you the fastest models because we know that we can build things like this and, um, and you're gonna need to run them, right? So we can help you run them and do all that stuff, or you can do it on your own. That's fine. Um, but no, we're not gonna be offering like services and APIs and all of that. Um, and really that's so that we can have the same incentives as the open source community.

AI assessment note: “no, we're not gonna be offering like services and APIs and all of that.”

Q There's an emerging stack around, uh, AI in general and generative AI in particular in the enterprise that, um, you know, you guys are very much a part of. What else do you think is important? And I'm going into, you know, vector databases and link chain and all the things. Where, where, how does it all fit?

A Yeah, I mean, good question. There's a lot of noise. I, I think there's a lot of interesting projects, and, like, there's probably, it's early research, so people are tinkering and trying things, but I would think about it that way. I think it's a lot of research. Like, what's gonna last from there? I'm not sure. Like, probably one percent of the things. Uh, but, like, it's, I, I find it fascinating to explore and to, like, see, oh, cool, like, this is interesting, like, okay, so you can chain together a bunch of prompts and things happen. That's great. How far can that take you? I don't know, but it's, I think it's a research question, so like, if people, if, if we were back in academia, no one would be criticizing these things because it's research, and you should be trying things out. But I think because there's like, VC money involved, and all these other things now, it gets a little weird. So I think as long as you can maybe separate both, I think we're okay. Um, but there is a lot of noise in the market. We tend to be agnostic to most of that, because, I mean, we've been doing this for a long time, and we kind of know what we want to be doing, and we know exactly where we need to be. But, We definitely look at, see what's happening, and if there's, like, a real trend that might last, like, we will adjust the tooling. In fact, Fabric is exactly that. Like, there was a long…

AI assessment note: “what's gonna last from there? I'm not sure. Like, probably one percent”