The Exchanges

Q next. Voice is likely next. Voice is a natural way to communicate that we are accustomed to, but nobody seems to have cracked it yet. Uh, do you think the medium of interacting with Tech will move from hardware, which does feel unnatural to hold a thing in your hand and like type into it with your fingers. How does that transition from that kind of hardware to voice happen?

A Yeah, the, you know, like as we think about the future, a hundred percent Voice will be a big interface and big way of how we interact with the technology around us. That's the thing about 11 Labs mission. It's to transform how we interact with technology around, um, holistically and whether that's, you know, calling into customer support, whether that's how you will learn, uh, at school and how you'll deliver education all the way through to how you interact with the robots or devices around you in the future. Um, and to your last part, like, I think the most exciting part of that is like, could you have the technology kind of fold into the background, the phone goes back into the pocket, and you kind of immerse yourself in the, in the, in the, in the world around you. Um, so I think there's like, to make this possible, there's at least three things that need to happen. One is of course the foundational technology and research. It needs to get to the level where, where you feel like you're interacting with A voice at the level of, of, of a human. It's, it's, you can interact, you can interrupt it. It has the right emotions, intonation, and it's quick. Um, it has the high intelligence. And I think the current model is, is good for set of, um, set of use cases. It's not yet like a full human level. Um, so that's the kind of the first component that needs to change on the, on lik…

AI assessment note: “to make this possible, there's at least three things that need to happen”

Q to build the same thing themselves. They seem to have done that in a couple of places. Um, when I talk to friends of mine in the agentic layer of AI or people who have built great applications on top of a model, uh, they all seem to be scared of what happens to these large language models when they come after the end product. Are you worried about that?

A You know, like, in a, in, in our case, we are, of course, continuously looking at, like, OpenAI, Google, incredible teams, and probably, like, as we think about the space, the, the, the biggest potentials of, uh, of, of, of effectively competing with 11 Labs. As you think about the company, and I only started it in, like, the approach from the beginning was doing both the research and the product. So doing the foundational model work, For, for audio and then building product around it. And we think both are important, um, and like would define the company where we today, of course, don't depend on open AI for, for, for voice and, and we can stay ahead and build models that are both great in English, great internationally, great across India languages, great across European languages. And, um, and that's both for generating speech with text to speech, understanding speech with speech to text. And, um, and that's important. I think like over the next two, three years, there's still so much you can do in voice. And like we spoke about how you make it more fluid and interactive. I think that still can be cracked in a better way. So of course worries us whether we continue being ahead, but it's also a good motivation. We have my, my co-founder is one of the smartest people who've been able to assemble an incredible research team and, um, and they keep crushing it. So I think they wi…

AI assessment note: “So of course worries us whether we continue being ahead, but it's also a good motivation.”

Q When you smile, when you say a certain thing in English, isn't it hard to take that emotion into Hindi, for example?

A The, um, it's, um, you know, it's, it's not easy. A hundred percent. Um, and then there's like kind of two, like there's the present and the future and the current way you would still translate it. You would of course dab it. I think you can preserve the emotion, but of course my movement of my lips, when I smile, well that depending on the language structure, um, it might be in a different part of the sentence. So like maybe an easy example is German. From English to German. German usually will have none at the end. So like, yes, that will exactly happen. You will like smile at a different part than, than the conversation. You can do some tricks to, to try to, to, to, to, um, to still smoothen it out. Um, but yes, there will be some discrepancy. So that's, that's in prison can still work in the future, you know, and, and there are some limitations of the current delivery softwares. You'll likely do lip animation to, or lip reanimation to move the lips Tiny bit to apply to where, where you would smile in that part of the sentence. So I think that will happen a hundred percent. I hope this happens this year.

AI assessment note: “it's not easy. A hundred percent.”

Q And it's like, you know, like, let's take the example of a car company, say BMW. What, what do I build that they will pay me enough for that? I can have a company invoice.

A You would start probably with something that's top of mind and something that is easy to deploy. So here you would start with customer experience side. So just being able to help as people call in, how do they get the best service, understanding everything about BMW in this case, how they understand about how the car works. You, you spend a lot of time on bringing that into the equation. So that's kind of the first layer. Then In car experience is like pretty open-ended, but how do you create an interactive radio inside of the car that people can, can interact with, um, across all the time? And here too, you need to figure out what type of languages my BMW might be interested in. How do you create the right voices to make sure the users in those regions are excited and happy? Um, what type of experience will BMW actually need? So there's a lot of actually deployment work that would come in. And then as you build all those steps, you need to figure out, okay, if you are to build like a big company, not only how I build it once for BMW, but how I abstracted a way to, to be able to replicate that into the other car companies. Um, So you need to capture some of those knowledge elements back into, into, into other, into other car parts. Automotive industry is a good example where I like haven't seen as much yet because it's, it's hard with some of the on-device requirements. Like so…

AI assessment note: “You would start probably with something that's top of mind... customer experience side”

Q Do you think that has something to do with?

A You know, I have an example where it did work, which is like, ah, an interesting one. Uh, the, the, um, in Ukraine they have a super app, which I think is brilliant. Um, uh, uh, also we have a pleasure of working with, with their government, uh, and where they are trying to create the first agent in the government, but the, the, the, they have a DIA app. It's effectively a citizen support app where you can ask about the, the benefits programs, employment programs, the, the, the different, different, the, um, things that are happening in India. Um, And that's how it started. So like effectively an app that every citizen can have, uh, of course the passport information, um, visa information, et cetera, that you can travel and idea yourself with. But then they have multiple, um, ministries. And the way they operate. Um, so they have multiple ministries where you have tech resources embedded in those ministries. So foreign affairs will have a few engineers working there. Internal affairs will have that. Education and ministry will have that. And now those people, Try to figure out how to automate part of, of, of, of the government work, and then, then bring it back to the digital transformation ministry, um, which is responsible for the citizen support app. And now what has started happening is that started effectively adding modules across the entire government. So now you can hav…

AI assessment note: “You know, I have an example where it did work”

Q Could a tiny company be built just around this, just around changing how the lip moves when something is being dubbed?

A It could. There's, there's actually a good set of companies that are, like, crashing it and going very quickly. You know, of course I think this is an entry point, not the end point. They will likely try to, to add more of that functionality, but, but, uh, but a hundred percent, you know, it's like, and the, the, of course now we speak about the static part, but the moment you start moving into the real time part, those tiny companies now can be huge companies. And that's kind of like, you know, we spoke about the creative side. If you think about the, The, the agent side, and I think there are some use cases here for, for, for, for both the lip animation and even for, for, for your, for you. Um, uh, uh, I have a, I have one idea for you. Um, so, you know, we spoke about the creative side. On the agent side, effectively what we do is, is, is allow you to create a voice experience, bring any knowledge that, that, that you want inside of that experience. Um, and, um, and then deploy that in, in, in different settings. Um, While monitoring, evaluating how it performs. I don't know if you're familiar with Masterclass.

AI assessment note: “It could. There's, there's actually a good set of companies that are”