The Exchanges

Q What are people trying to do? Can you give us a flavor of some of like the biggest use cases you see in the enterprise?

A It's super broad. Uh, so it spans pretty much every vertical. I mean, the common things are like Q&A. So speaking to a corpus of documents, for instance, if you're a manufacturing company, you might want to build a Q&A bot for your engineers or your workers who are on the assembly line, uh, and plug in All of the, the manuals of the different tools and diagnostic manuals for common errors in parts, and then let the user chat to that instead of having to open up a thousand page book and try to find what they need. Similarly, Q and A bots for the average enterprise worker. So plugging in your IT FAQ, your HR docs, all the things about your company, and having a centralized chat interface onto the knowledge of your organization so that they can get their questions answered. Those are some of the common ones. Beyond that, there are kind of specific functions that we power. Um, a good example might be for a healthcare company. They have these longitudinal health records. Of patients. And that consists of every interaction that that patient has with the, the healthcare system from visits to a pharmacy, to the different labs or tests that they're getting, uh, to doctors visits, and it can spend decades. And so it's a huge, huge record of someone's medical history. And typically what happens is that patient will call in and they'll ring up the receptionist and be like, my knee hurts. I…

AI assessment note: “the common things are like Q&A. So speaking to a corpus of documents”

Q And in terms of like your North Star of how you organize, um, the team and invest, uh, you obviously come from a research background yourself. Like how much do you think, um, you know, cohere success is dependent on core models versus other, you know, platform and go to market support investments you make?

A It's all of the above. Like the models are the foundation. And if you're building on a foundation that, um, Doesn't meet the customer's needs, then there's no hope. And so the models are crucial and, um, it's like the heart of the company, but in the enterprise world, things like customer support, reliability, security, these are all key. And so we've heavily invested on both sides. We're not just a modeling organization. We're a modeling and go to market organization. Um, And increasingly, product is becoming a priority for Cohere, and so figuring out ways to shorten time to value for our customers. Um, yeah, over the past, like, 18 months, Since the enterprise world sort of woke up to the technology, we've watched, we've watched folks build with our models, seeing what they're trying to accomplish, seeing the common mistakes that they make. That's been helpful. It's been sometimes frustrating, right? Watching the same mistake again and again. But we think there's a huge opportunity to be able to help enterprises avoid those mistakes and implement things right the first time. And so that's really where we're pushing towards.

AI assessment note: “It's all of the above. Like the models are the foundation.”

Q Do you think in like a traditional hype cycle for enterprise technologies, probably for most technologies, but in particular enterprise, um, uh, you know, there's this trough of disillusionment concept of people get very excited about something and ends up being harder to apply or more expensive than they thought. Do we see that in AI?

A I'm sure we see some of it for sure. Um, But I think honestly, like the core technology is still improving at a steady clip and new applications are getting unlocked every few months. So I, I don't think we're in that trough of disillusionment yet. Yeah. It feels like we're super early. It feels like we're really, really early. And if you look at the market, this technology just unlocks an entire new set of things that you can build. You just fundamentally couldn't build them before, and now you can. And so there's a resurfacing of technology, products, systems that's underway. Even if we didn't train a single new language model, like, okay, all the data centers blow up. We can't improve the LLM. We only have what we have today. There's a half decade of work to go integrate this into the economy, to build all these things, to build the, you know, uh, RFP, insurance RFP response Bought to build the healthcare record summarizer. Like there's a half decade of just resurfacing to go do. So there's a lot of work ahead of us. I think we're kind of past that point. There was a question of, oh, is there too much hype? Is this technology actually going to be useful? But it's in the hands of a hundred million people now, hundreds of millions of people now. It's in production. There's very clear value. The project is now Putting it to work and delivering it, uh, to the world.

AI assessment note: “I don't think we're in that trough of disillusionment yet.”

Q enterprise customer will invest in pre-training is, I think, a bit more controversial. I believe some of the lab leaders would say, like, nobody should be touching this, and it doesn't make any sense for people from a scale of compute and data, data curation effort required, and just sort of the talent required to do pre-training in any sort of competitive way. Like, how would you react to that?

A I think if you're building like a, if you're a big enterprise and you're sitting on a ton of data, like hundreds of billions of tokens of data, um, pre-training is a real lever that you're able to pull. I think for most like SMBs and certainly startups, it makes no sense that you should not be pre-training a model. Um, but if you're a large enterprise, I think it's, it should be a serious consideration. The question is how much pre-training? It's not like you have to start from scratch and do a, you know, fifty million dollar training run, but you can do a fraction, you could do a five million dollar training run. That's what we've seen succeed. These sort of continuation pre-training efforts. Um, so yeah, that, that's one of the offerings that we have, but of course we don't jump straight into that. You don't need to Spend massively if you don't want to, and usually Uh, the enterprise buying cycle or, or technology adoption cycle is quite slow. And so you have time to move back into it. I would say it's totally at the customer's discretion. Um, but to the folks who say that no one should be pre-training.

AI assessment note: “pre-training is a real lever that you're able to pull”

Q Maybe if we go to, um, projection and we'll hit on a few things that you've mentioned as well, um, where are we in scaling laws? Like how much capability improvement do you expect over the next few years?

A We're, we're pretty far along, I would say. Like we're starting to enter into a sort of flat part of the curve, um, and we're certainly past the point where if you just interact with a model, You can know how smart it is. Like the, the vibe checks, they're losing utility. And so instead, what you need to do is you need to get experts to measure within very specific domains like physics, math, uh, chemistry, biology, um, You need to get experts to actually assess the quality of these models because the average person can't tell the difference at this stage between generations. Yes, like there's still much more to go do, uh, but those gains are going to be felt in very specialized areas and have impacts on more researchy, um, more researchy domains. I think for enterprises and the general sorts of tasks that they want to automate or tools that they want to build, The technology is already good enough or close enough that a, a little bit of customization will get them there.

AI assessment note: “we're starting to enter into a sort of flat part of the curve”

Q What do you think is the driver in that difference of opinion?

A I don't know. I, I think maybe I'm A little bit more in the weeds of the practical frustrations of the technology, where it breaks, where it's slow, where it, we start to see things plateau or slow down, um, And perhaps others are more, maybe they're more optimistic. Maybe, maybe they see, um, they see a curve increasing and they just think it goes on forever. Like that will just continue arbitrarily, which I, I disagree with. I think there's, there's friction points. Like there is genuinely friction that enters in. Like maybe even if in theory, you know, like a neural net is a universal approximator, it can learn anything to universally approximate. You would need to build a neural net the size of the universe. So like there's some fundamental barriers to reaching limits that people extrapolate out to that I think will, um, bound the practically realizable, um, forms of this technology.

AI assessment note: “I think maybe I'm A little bit more in the weeds of the practical frustrations”