The Exchanges

Q How did that, like, can we tell that story, like, so how did this happen, right? Like, so, Uh, you, you were inspired by, by Grok.

A Yeah, so, I guess, like, you know, the backstory from my point of view is I had used CodeSearch and Grok while at Google, um, but I, I didn't actually know that it was connected to you, Steve. Like, I knew, I knew you from your blog posts, which were always, like, excellent, kind of, like, inside, very thoughtful takes on, uh, from an engineer's perspective on, on some of the challenges facing, like, tech companies and, you know, tech culture and that sort of thing. Um, But my first introduction to you within the context of, like, code intelligence and code understanding was I watched a talk that you gave, I think, at Stanford about Grok when you were first building it, and that was very eye-opening. And I was like, oh, like, that guy, like, the guy who, you know, writes the extremely thoughtful, ranty, like, blog posts also built that system. Um, and so that's, that's how, that's how I knew, you know, you were kind of involved in that. And then it was kind of like, uh, you know, we always, Kind of like wanted to hire you, uh, but never knew quite how to, uh, approach you or, you know, get that, get that conversation started.

AI assessment note: “I had used CodeSearch and Grok while at Google... watched a talk that you gave”

Q Um, so would you basically say that they're tackling the agent's problem that you don't want to tackle?

A No, I would say at a high level, we are after, uh, maybe like the same high level problem, which is like, hey, I want some code written, I want to develop some software, uh, and can, can an automated system go build that software, uh, for me? Um, I think the, the approaches might be different. Um, so, I think the analogy in my mind is, I think about like the AI chess players, right? Like, is, like, Coding in some sense is, I mean, it's similar and dissimilar to chess. Uh, I think one question I ask is like, do you think producing code is, is more difficult than playing chess or less difficult than playing chess? More? I think more, right? And, and if you look at, like, the, the best AHS players, like, yes, you can use an LLM to play chess. Like, people have showed demos where it's like, oh, like, yeah, uh, GPT-IV is actually a pretty decent, like, chess move suggester, right? Um, but you would never build, like, a best-in-class, uh, chess player off of GPT-IV, uh, alone, right? Like, the way that people design, uh, chess players is that you have, kind of, like, a search space, uh, and then you have, Uh, a way to explore that search space efficiently. There's a bunch of search algorithms, essentially, where you were doing tree search in various ways, and, uh, you can have heuristic functions which might be powered by an LLM, right? Like, you might use an LLM to generate proposal…

AI assessment note: “No, I would say at a high level, we are after, uh, maybe like the same”

Q that software developer that is not no longer doing these routine boilerplate type tasks because they're just enhanced by tools like yours. And so you, you mentioned, uh, you open code graph. I mean, that, that is a kind of, Uh, DSL maybe. And, um, because we're releasing this, uh, as you, as you go GA, um, you hope to, uh, for other people to, to take advantage of that?

A Oh yeah, I would say, so OpenCodeGraph is not a DSL, it's more of a protocol. It's basically like, hey, if you want to make, uh, your system, whether it's, you know, chat or logging or whatever, accessible to, um, an AI developer tool like Kodi, um, here's kind of like the, the, the schema. Uh, by which you can provide that context and offer hints. Um, so I would, you know, comparisons like LSP obviously did this for, uh, kind of like standard code intelligence. It's kind of like a lingua franco for providing find references and code definition. There's kind of like analogs to that. There might be also analogs to, uh, kind of the original OpenAI, kind of like plugins, uh, API, where it's like, hey, you know, there's, there's all this like context out there that might be useful for, uh, an LM based system to consume. Uh, and so, at a high level, what we're trying to do is, uh, define a, a common language, uh, for context providers to provide context to other tools in the software development lifecycle.

AI assessment note: “OpenCodeGraph is not a DSL, it's more of a protocol.”

Q your, your AI stack? Um, what do you, like, obviously you build a lot proprietary in-house, um, but, like, what approaches, uh, you know, like, so prompt, prompt engineering, do you have a prompt engineering management tool? You know, what, what approach is there? Do you, do you do, um, pre-processing, pre-processing orchestration, like, do you use Airflow? Do you use something else? Like, you know, that kind of stuff.

A Yeah, ours is very, like, duct taped together at the moment. Um, so in, in terms of stack, uh, I mean, it's essentially, uh, Go and TypeScript, uh, and now Rust. Um, there's the, the knowledge graph, the code knowledge graph that we built, which is using indexers, uh, many of which are open source, um, that speak the skip protocol. Uh, and, uh, we have the code search backend, um, You know, traditionally we've supported regular expression search and, uh, uh, string literal search with like a trigram index, and we're also building more like fuzzy search on top of that now, uh, kind of like natural language or keyword based search on top of that. Um, and we use a variety of open source and proprietary models. We try to be like pluggable with respect to different models so we can easily kind of like swap, swap the latest model in and out, uh, as they come online.

AI assessment note: “in terms of stack, uh, I mean, it's essentially, uh, Go and TypeScript”

Q It's gonna be GA by the time this releases, so, congrats. Uh, congrats to your future self for launching this in two weeks. Um, can you give a quick overview of just what Kodi is? I think everybody understands that it's a AI coding agent, but a lot of companies say they have an AI coding agent, so, uh, yeah, what does Kodi do? How do people interface with it?

A Yeah, so, basically, you know, like, how is it different from the, like, several dozen other AI coding agents that exist in the market now? Um, I think our take, when we thought about building, uh, a coding assistant that would do things like code generation and question answering about your code base, I think we came at it from the perspective of, you know, we've spent the past decade building the world's best code understanding engine for human developers, right? So, like, it's kind of your, your, uh, guide as a human dev if you want to go and dive into a large, complex, uh, code base. And so, Our intuition was that a lot of the context that we're providing to human developers would also be useful, uh, context for AI developers to consume. And so in terms of the feature set, uh, Kodi is very similar to a lot of other assistants. It does inline autocompletion. It does code-based aware chat. Uh, it does specific commands that automate, you know, tasks that you might rather not want to do, like generating unit tests or, uh, adding detailed documentation. Um, But we think the, the, the core differentiator is, is really the quality of the context, uh, which is hard to kind of describe succinctly. It's, it's a bit like saying, you know, what's the difference between Google and AltaVista? Um, there's not like a quick checkbox list of features that you can rattle off, but it really j…

AI assessment note: “It does inline autocompletion. It does code-based aware chat. Uh, it does specific commands”

Q Um, What's kind of the, the future of Sourcegraph in the engineering org?

A Yeah, so that's a really interesting question. Um, and I think it's, it sort of gets at this, like, issue which is, I think, uh, basically like every AI, uh, DevTools creator or producer these days, I think us included, um, we're kind of like focusing on the wrong problem in a way. Um, because like the, the, the real problem of modern software development, I think is, is not how quickly can you write more lines of code. It's really about managing the emergent complexity, uh, of code bases as they evolve, uh, and grow, and how to get, how to make, uh, like efficient development tractable again, because, uh, the bulk of your time becomes more about understanding how the system works, uh, and how the pieces fit together currently so that you can update it in a way, uh, that gets you your added functionality, uh, Um, doesn't break anything and doesn't introduce a lot of additional complexity that will slow you down in the future. Um, and if anything, like, the Interloop developer tools that are all about, like, generating lines of code, uh, yes, they help you get your feature, uh, done faster, they, they generate a boilerplate for you, but they might make this problem of, like, managing large complex code bases, uh, more challenging, just because now you'll, instead of having, like, You know, a pistol, you'll have a machine gun in terms of, like, being able to write, write code, an…

AI assessment note: “real problem of modern software development... is really about managing the emergent complexity”