Everything Beyang Liu said on any show that made the record, most notable first. Each card names its show and opens the statement there.
Beyang Liu: Keyword Search Solves Over 90% of RAG Retrieval
“I think if you talk to a lot of RAG practitioners you'll find that the kind of, like, dirty secret is that keyword search can probably get you more than 90% of the way there.”
Beyang Liu: Simpler baseline systems often match or beat fancy AI models
“Doing the simple thing, it establishes a baseline. Like oftentimes you'll find that like the doing the fancier thing is often sexier. And it's certainly these days it's like trendier, right? Cause you can kind of claim the mantle of like, ah, you know, I made …”
Beyang Liu: RAG remains necessary for context even with fine-tuned models
“I think you're still going to want to do RAG anyways. Like, even if you have fine tuned models in the mix, RAG is still sort of this, like, last mile data or context.”
Liu: Single Developers Will Build Apps That Currently Require Large Teams
“If your goal is to build software as it exists today, then yes, I think in the future, a single human will be able to build applications that today require large numbers of people to coordinate.”
Liu: Software Teams Will Remain Large as Scope and Ambition Expand
“And so I still think we will have large teams of software developers in the future. They will just You know, each individual will be responsible for far more feature scope than they are today, and the system as a whole will be more sophisticated and more power…”
Beyang Liu: CS fundamentals will grow in importance in AI era
“My, maybe a bit contrarian hot take here is that CS Fundamentals if anything, are gonna grow in importance. You know, the stuff you learn in the coding bootcamp, maybe that gets, you know, automated away, but the fundamentals of knowing, you know, which data s…”
Liu: Cody matches GitHub Copilot completion acceptance rates using open-source StarCoder
“Like today, Cody uses StarCoder for inline completions, and with the benefit of the context that we provide, we actually show, like, comparable completion acceptance rate metrics. It's kind of like the standard metric that folks use to evaluate inline completi…”
Liu: Pure transformer models are insufficient to support autonomous AI agents
“We're actually a little bit, I think, more bearish than the average, you know, AI hypefluencer out there on the feasibility of agents with purely kind of like transformer-based models.”
Liu: Reliable AI coding workflows require search-based algorithmic backbones
“The way that we get to this, like, more reliable, multi-step workflows that can do things beyond, you know, generate unit test is, is, it's really gonna be, like, a search-based approach, where, where you use an LLM as, kind of, like, an advisor or a proposal …”
Liu: Open-source AI models are currently state-of-the-art for code completion
“Yeah, I mean, for completions, open source is, is state of the art right now.”
Liu: Reliable single-step generation is a strict prerequisite for true AI agents
“If you want to get to the point where you can actually be truly agentic or like multi-step automated a necessary part of that is like the single step has to be robust and reliable.”
Beyang Liu: Statistical learning and convex optimization will re-emerge in AI
“I'm still waiting for the statistical learning and maybe some of the convex optimization stuff to reemerge. I wouldn't count it entirely out yet. I feel like the pendulum always swings back the other way. It's swung away from statistical learning and convex op…”
Liu: Coding agent chains suffer compounding failure from sub-90% step fidelity
“The fidelity of today's systems far less than 90% for each step. So I think this is the issue that everyone building agents in that way is, is encountering is like, you know, you have compounding failure.”
Liu: Fine-tuning medium models can harm their in-context learning ability
“And if you fine-tune a medium-sized-ish model, sometimes it loses the ability to do effective in context learning, because I think the intuition is, it's devoting more, more of its parameter space to, kind of, like, memorizing the training set so it can do bet…”
Liu: AI coding assistants must pull context beyond Git repositories to succeed
“And I don't think the AI developer will be any different. It will need to pull context from all these different sources.”
Liu: Post-hype reality check will expose the limits of AI search techniques
“And I don't know, I think in the next year or two, maybe as like the, as we get past like the peak AI hype, we'll start to see the gap emerge or become more obvious to more people about like how, how, how many of like the newfangled techniques actually work in…”
Liu: Codebase complexity is software's bottleneck, not code generation speed
“The real problem of modern software development, I think is, is not how quickly can you write more lines of code. It's really about managing the emergent complexity of code bases as they evolve and grow, and how to get, how to make like efficient development t…”
Liu: Developers spend most time reading existing code, not writing new features
“I think we both realized that we're spending a lot of our time in the process of reading and understanding the existing code rather than building new features. Cause all that is a prerequisite for Being able to build quickly and efficiently.”
Beyang Liu: Parsing Semantic Code Entities Is Essential for Code Search
“We found it that it's absolutely essential if you're searching over code to parse things. And so you can extract specific functions and methods and classes along with the corresponding doc string and treat those as separate entities. In your system rather than…”
Liu: Data quality and structuring pipelines outweigh model architecture
“Your model architecture can only go so far if your data is, is garbage. So you really need a high quality data pipeline, and that means not only having, you know, in our domain, high quality code that can serve as the underlying data to use, but also a way to …”
Liu: All production code AI tools today are developer inner-loop assistants
“And so if you look at the landscape of code AI today, the systems that are actually in production and in use they're all inner loop tools. So anything that is, you know, in your editor doing inline completions or chat, that's kind of assisting you in the proce…”
Liu: AI engineering tools will automate tedious maintenance before complex features
“There will be kind of like a range of issues that can be supported in terms of complexity, right? Like there's certain like bugs and issues that, you know, in whole are kind of a form of toil. Like no one wants to do them because it's kind of like busy work, e…”
Liu: Sourcegraph Cody uses StarCoder 7B, matching larger models with context
“The primary model that Cody uses for inline completions right now is StarCoder seven billion. And with the benefit of context that actually matches the performance of you know, larger proprietary models.”
Liu: The Core Problem of Software Development Is Coordination and Visibility
“And I think the core problem of software development is, is one of coordination and visibility.”