The Exchanges

Q and reinforce on what works. You know, when someone pushes something, that's pretty good, uh, indication that you did a good job and you should do more of that. Uh, all these other, uh, types of disciplines are more open-ended. So is it just gonna take a little longer? And maybe that's why code was first, but is it gonna take a little longer? How do you see that evolving?

A That's a very valid point, right? One of the ways we can make, let's say, models and agents, uh, Improve, ah, how they perform tasks in a certain domain is by obviously having a very clear reward function, let's say, on, on which, and enough cases on which we train the model or the agents. And, ah, ah, with code, let's say we have created the ability to have a very clear signal, ah, when somebody uses the product. But also, in my opinion, the other thing that happened with code, in addition to actually, let's say, training the models on code, because there's a lot of code available, ah, we built products. We built IDs that essentially users started using, even if the answer was not perfect. And that started generating a lot more data and how users actually respond to these things, right? When do they accept the change and when they do not accept the change? And that itself is very valuable data and, uh, you know, creates a data flywheel, but then we can go back and improve the way we generate code. I do think that paradigm is applicable to other domains. So you have to probably start with something that is useful, that let's say humans are on the driver's seat, engineers in our case, and, uh, As long as it's, it's a better way of working, it's, it's, it's faster than not without that tool, then you can also create this data flywheel where we get essentially, um, uh, uh, some so…

AI assessment note: “I do think that paradigm is applicable to other domains.”

Q doing. Uh, but I imagine that if one agent breaks, uh, the whole thing goes down. Um, so can you talk a little bit about like how you get them? I mean, trying to get a chap out to do one thing you want is sometimes a challenge. So how are you getting multi-agent, uh, processes and workflows to work? Because it seems to me like a very difficult problem.

A Yes. So first of all, I mean, maybe to use an example, right? Let's say you have two agents that they have to collaborate. Maybe one agent goes and understands documents, and maybe there's another agent that goes and takes actions into a code base, right? To be able to perform a task. So if you have a situation with these two agents, and oftentimes you have more than two, maybe you have five or 10, you have to figure out the plan you want to execute, right? How do you coordinate the work across them? And then they have to essentially do the work and communicate back to each other What they learned, right? So to do that reliably, I think you have to have many layers of, let's say, uh, guardrails, if you wish, right? Not guardrails in the sense of not letting the, the, the models of the agents do something, something wrong, but oftentimes what we do is we have an agent do some work, another agent review that work, and provide feedback, and have the first agent iterate, right? And then now when it expands across multiple agents, maybe They all do the work. They produce an outcome. In our case, let's say an agent goes and does, uh, investigates an incident and produces, like, an analysis of what happened and how to fix it. Then we have another agent go review that work and force the first agent to go back and, you know, redo the work if it finds, like, some, some, uh, hole in its r…

AI assessment note: “we have an agent do some work, another agent review that work, and provide feedback”

Q auditing and the fixing, uh, with technology itself. Um, so then what does that, I mean, what do you think that, that, uh, if this works the way you anticipate, where do you think that leaves the end state of engineering skills? Um, you know, do you think that, that engineers will risk, uh, you know, having some of those skills atrophy if the AI does a good enough job?

A Yeah. Like with every technological evolution, obviously we get a lot of leverage as humans to produce, you know, outcomes a lot faster, right? We're not resisting using machines for other domains. I think the same applies to, to, to software and, you know, software over the last 50 years moved, uh, you know, from very low level, let's say coding for the machine itself to like different layers of abstraction with operating systems and high level languages. I think AI is just another abstraction. And I don't think the answer to this is, ah, or the concern is, are humans, are engineers gonna atrophy the sort of skills to produce code or run code? I think the real answer is that we should produce agents that do both parts very well. They should produce code very quickly, but they should also be able to run, maintain, improve, troubleshoot code the same way. And engineers should be now, should be, will be very quickly, in my opinion, operating at the high level of abstraction, Where they won't have to worry. Also, a lot about these low-level, kind of, specific bespoke things of the tool that you have at your disposal, like query languages, and, you know, how exactly should I call this API or this CLI to get to an answer. Simply, this is going to be done, all this heavy lifting and stressful work is going to be done by AI, and we're going to be operating at the level above. And I th…

AI assessment note: “I think AI is just another abstraction... I don't think it is a risk”

Q are, you're an AI solution effectively that will, um, I mean, it doesn't have to be AI generated code, but all those things about keeping, uh, that the product running, um, you have actively built a product that can go ahead and it doesn't investigate the code base and then look for errors. And then it doesn't, is it, does it have authorization then to go out and fix it?

A Very good question. Because the other thing that happens when you deploy, let's say code, and you run it in your production system, as we call it, There is a lot of sensitivity, both around the data, but also in terms of, like, something going wrong. Like, you know, humans can make mistakes, of course, and they can cause, you know, challenges and outages, and, you know, they can take down a service. But AI, potentially, can do the same. And in my view, the way, actually, we do this is we do it by making a, you know, building AI that focuses a lot on trust, trust for software engineers, or operators in any domain, for that matter. So that the way I think about it is almost as self-driving cars. So we don't let, essentially, self-driving cars on the street unless they have proven, right, with data that they can drive better than human drivers. And, you know, there are also different levels of automation or in driving, right? I think the same thing is going to happen with AI in many domains, but definitely in this one, because initially most people allow the AI to go do the work, do the investigation, report back the findings, and what is the solution? And then a human has to, a human engineer has to decide. Maybe in the next step, the AI is going to be able to take some of these actions on its own, as long as they, they, they, they are, let's say, not too risky or they're reversi…

AI assessment note: “initially most people allow the AI to go do the work... human engineer has to decide”

Q bit about, it's interesting from your perspective, you're working with engineers, so these people generally Tend to appreciate what technology can do. What is the, uh, interest in adopting this stuff been like? And has there been any sort of, oh, if we use this, you know, AI, um, system that's going to make sure that we stay up, if there's a problem, then what are we going to do?

A This is a valid question and concern. Now, I do think maybe engineers are the earliest adopters you can find, so that generally helped also in getting AI to be widely adopted for coding, and now the subsequent steps of software software engineers, like we see ourselves with the adoption of Resolve AI. But there's still sometimes two things happening also, right? There is some natural resistance to change. It's difficult for, for, for humans to change their habits very, very quickly. And it takes, you know, different people, different personalities sometimes are more resistant to change. So that is definitely, uh, an issue. And I do think there is a concern about, you know, people's jobs sometimes. Now, my perspective on this is the following. Especially in software engineering, where some of the most highly paid professionals in the world, And I think by us producing more technology, I, I don't think the end state here is going to look something like where we have fewer software engineers, but also this is not what the, the, the, the, this is not the optimization formula in my opinion, right? It almost doesn't matter if we have like all these highly, more highly paid people or less highly paid people. I think the, the, what we should be optimizing for is can we produce technology a lot faster, but in the end benefits the entire world, right? Like I said, by solving harder probl…

AI assessment note: “I do think there is a concern about, you know, people's jobs sometimes.”

Q errors when they come up, uh, and then push it to a person to be like, Hey, I think this is probably what it is. And then what do they like press a button and then it gets fixed? Or so, so I'm curious to hear that. And then also, um, uh, yeah, how, how far just like the AI is like, I will go in and do it myself.

A I think that we, let's say, there are two aspects to this question, in a way. There is the capability, like the reasoning ability of AI to be, let's say, on par with software engineers, or better for that matter, to allow it. And then there is the whole, ah, compliance, you know, framework that you have to have in place on when do you let an AI do all that work, right? And what happens when something goes wrong? And I think we need to figure out both. But I'll answer more on the capability, ah, kind of, ah, ah, ah, let's say, ah, perspective. It is very hard for us to, to possibly imagine the future when we're still on an exponential curve, because that's very unusual for how we think and how things improve. I do believe that, you know, we are probably a year away from AI becoming the driver of software, the same way, let's say, agents are the primary producer of code today. I think we're going to move to the same place in a year from now, where humans are going to be operating at a high level of abstraction. Still overseeing that AI, making most of the final decisions, but I think probably in two to three years, we're going to be at the place where AI is going to be making most of these decisions, and humans will be delegating, let's say, however, maybe decision frameworks or tasks to the AI. So I give it, I guess, I don't know, level five maybe is difficult to predict, but le…

AI assessment note: “probably in two to three years, we're going to be at the place”