Dec 17, 2023 · 1h 33m · latent-space
The "Normsky" architecture for AI coding agents — with Beyang Liu + Steve Yegge of SourceGraph
Beyang Liu · 46m spoken Steve Yegge · 20m spoken Shawn Wang · 11m spoken Alessio Fanelli · 5m spoken
0:00 / 0:00
▶ Watch on YouTube →
gold bands on the timeline = statements, start to end. Hover to read, click to jump. CC turns on captions
Sourcegraph's Beyang Liu and Steve Yegge discuss the General Availability of Cody and explain the 'Normsky' architecture, which synthesizes deterministic code graph indexing with statistical LLMs to master enterprise-scale code intelligence.
How this conversation actually went
Every chapter scored 0–10 on four independent dynamics. Hover any point for the reasoning behind the score. The hosts hold 20.3% of the talking time here. How this is scored →
The hosts as informed peer 4.8 Guest teaching 3.4 Guest disagreement 1.7 The hosts pushing back 1.5
speaking balance: gold is the hosts, purple is the guest (3 minute bins)
Sharpest disagreement ▶ 1:01:25 Beyang Liu rejecting the pure transformer scaling premise
Beyang forcefully reframes Swyx's scaling argument by demanding proof that pure transformer scaling can beat tree search algorithms in chess or coding.
Hardest push from the hosts ▶ 59:36 Swyx pushing back on agent skepticism with compute scaling lawsSwyx refuses the guests' dismissive stance on autonomous agents, detailing compute scaling calculations up to 2030 to challenge their near-term limits.
Biggest teaching moment ▶ 49:55 Beyang Liu detailing LSP versus SCIP and Kythe trade-offsBeyang delivers a comprehensive breakdown of why LSP's range-based abstraction fails to capture symbolic code relationships and why custom knowledge graphs are essential.
The host holds their own ▶ 59:36 Swyx calculating human-year compute scaling projectionsSwyx demonstrates deep domain expertise by citing George Hotz's petaflop calculations and Sam Altman's exponential model generation projections.
the scores for every segment, with the reasoning behind each
| Chapter | Topic | The hosts as informed peer | Guest teaching | Guest disagreement | The hosts pushing back | Why |
|---|---|---|---|---|---|---|
| Introductions and Origins of Sourcegraph and Grok | 4 | 2 | 1 | 1 | The conversation starts with warm, collegial rapport. Swyx demonstrates strong familiarity with Steve's famous engineering essays, Google/Amazon tenure, and Grab's engineering culture. | |
| Introducing Cody and the Power of RAG Context | 6 | 3 | 1 | 1 | Alessio demonstrates sharp domain knowledge by quoting specific benchmark discrepancies between Copilot and Cody regarding package.json start commands. Beyang and Steve elaborate on how RAG serves as an expert consultant compared to fine-tuning. | |
| Code Search as a Recommendation Engine and Context Ranking | 6 | 2 | 1 | 1 | Swyx connects RAG ranking to classical recommendation systems and highlights context attention curves like lost-in-the-middle phenomena. Steve and Beyang validate this architectural view. | |
| Inline UX and Skepticism Toward Autonomous LLM Agents | 4 | 4 | 3 | 2 | Beyang and Steve strongly reject the AI hype around fully autonomous transformer agents, calling multi-hop LLM agents a boondoggle. Beyang articulates why deterministic search algorithms like A-star are necessary. | |
| The 'Normsky' Architecture: Merging Norvig and Chomsky | 3 | 5 | 2 | 1 | Beyang and Steve educate the hosts on the historic Chomsky versus Norvig AI debates and unveil Sourcegraph's hybrid 'NORMSKI' architectural framing. Steve shares firsthand perspective from 1990s Google on scaling Chomskyan systems. | |
| Data Pre-processing Moats and the Future of DSLs | 5 | 3 | 1 | 1 | Alessio asks an informed question regarding whether LLM code generation will eliminate the need for human-oriented DSLs. Steve and Beyang clarify that data pre-processing and compiler AST parsing represent their true competitive moat. | |
| Graph Protocols: LSP, Kythe, SCIP, and the BFG Indexer | 4 | 6 | 2 | 1 | Beyang and Steve deliver an in-depth schooling on graph indexing protocols, contrasting LSP's range-based limitations with Kythe's and SCIP's semantic graphs. They explain how their BFG engine eliminates type errors. | |
| Transformer Scaling Limits versus Algorithmic Search | 7 | 4 | 4 | 6 | Swyx directly challenges the guests' bearishness on agents by presenting compute scaling curves extrapolated to 2030. Beyang forcefully pushes back with a chess algorithm comparison, questioning whether pure transformer scaling ever replaces tree search. | |
| Sourcegraph's Production AI Stack and Database Architecture | 5 | 3 | 1 | 1 | Swyx probes into Sourcegraph's production stack, and Steve reveals they use Postgres and flat files rather than a dedicated graph database. Swyx demonstrates industry familiarity with Fireworks AI leadership. | |
| AI Tooling Wishlists, Synthetic Data, and OpenAI Dynamics | 5 | 2 | 1 | 1 | Swyx analyzes Replit's bounty data moat and compares it to OpenAI dependencies. The group discusses contingency plans during the OpenAI CEO firing weekend. | |
| Managing Codebase Complexity and Enterprise Productivity | 5 | 4 | 1 | 1 | Beyang explains how AI code generators exacerbate codebase complexity and why engineering managers need codebase-level understanding over raw code generation. Swyx shares a personal user story navigating a complex Twitter scraping repo via Cody Web. | |
| Lightning Round and Future Outlook | 4 | 3 | 2 | 1 | In the lightning round, the group explores multimodal coding workflows. Steve forcefully warns engineers who refuse to use coding assistants that they need to start planning another career. |
Statements from this episode (20)
Yegge: Grab succeeded by copying Amazon's playbook and executing with laser focus
“I mean, they had their success because they are laser focused. They copied Amazon. I mean, they're just they're executing really, really, really well.”
Yegge: Super apps failed in India due to bandwidth and hardware limits
“They didn't work in India either, and it was primarily because of bandwidth reasons, and smaller phones.”
Liu: No AI coding tool uses one model for completion and chat
“Everyone uses a range of model. No, like, no one uses the same model for, like, inline completion versus, like, chat because the latency requirements were.”
Liu: Cody matches GitHub Copilot completion acceptance rates using open-source StarCoder
“Like today, Cody uses StarCoder for inline completions, and with the benefit of the context that we provide, we actually show, like, comparable completion acceptance rate metrics. It's kind of like the standard metric that folks use to evaluate inline completi…”
Yegge: AI coding will fragment into many specialized, fine-tuned models
“And that, that fragmentation of models actually, we expected to continue and proliferate, right? Because we are fundamentally, we're a recommender engine right now. We're recommending code to the LLM. We're saying, may I interest you in this code right here so…”
Liu: Long-context recall depends directly on needle-in-haystack training loss
“The skill with which models are able to take advantage of context is always going to be dependent on how that factors into the impact on the training loss, right? So like, If you want long context window models to work well, then you have to have a ton of data…”
Liu: Pure transformer models are insufficient to support autonomous AI agents
“We're actually a little bit, I think, more bearish than the average, you know, AI hypefluencer out there on the feasibility of agents with purely kind of like transformer-based models.”
Yegge: Sourcegraph avoids autonomous AI agents until someone builds one that works
“We're not going in the agent direction, right? I mean, I'll believe in agents when somebody shows me one that works.”
Liu: Reliable AI coding workflows require search-based algorithmic backbones
“The way that we get to this, like, more reliable, multi-step workflows that can do things beyond, you know, generate unit test is, is, it's really gonna be, like, a search-based approach, where, where you use an LLM as, kind of, like, an advisor or a proposal …”
Liu: AI coding assistants must pull context beyond Git repositories to succeed
“And I don't think the AI developer will be any different. It will need to pull context from all these different sources.”
Yegge: Developers will spend far less time learning DSLs like regular expressions
“And I think you're going to see a lot less of people having to slave away learning these things. They just have to know the broad capabilities and then the LLM will take care of the rest.”
Liu: Deterministic graph context eliminates common AI code completion type errors
“Turns out that works really well. Like, you can eliminate a lot of type errors that, that AI coding tools make just by pulling in that context.”
Yegge: PostgreSQL matches dedicated graph databases on most graph workloads
“There was some joint study between IBM and some other That basically showed that Postgres was performing as well as most of the graph databases for most graph workloads.”
Liu: Post-hype reality check will expose the limits of AI search techniques
“And I don't know, I think in the next year or two, maybe as like the, as we get past like the peak AI hype, we'll start to see the gap emerge or become more obvious to more people about like how, how, how many of like the newfangled techniques actually work in…”
Liu: Open-source AI models are currently state-of-the-art for code completion
“Yeah, I mean, for completions, open source is, is state of the art right now.”
Liu: Synthetic data and task-specific fine-tuning provide alpha for code automation
“I feel like most models today, they still use, like, combination of, like, the stack and the pile as, like their training corpus but you can only stretch that so far. At some point, we need more data and I don't know. I think there's still more alpha in, like,…”
Liu: Codebase complexity is software's bottleneck, not code generation speed
“The real problem of modern software development, I think is, is not how quickly can you write more lines of code. It's really about managing the emergent complexity of code bases as they evolve and grow, and how to get, how to make like efficient development t…”
Yegge: Current AI coding assistant form factors are not the steady state
“The form factor that coding assistants have today is probably not the steady state that we're seeing, you know, long term. I mean, you'll, you'll always have completions and you'll always have chat and commands and so on, but I think we're going to discover a …”
Liu: Reliable single-step generation is a strict prerequisite for true AI agents
“If you want to get to the point where you can actually be truly agentic or like multi-step automated a necessary part of that is like the single step has to be robust and reliable.”
Yegge: Software engineers ignoring AI coding assistants risk career obsolescence
“If you're one of those engineers, man, you better start like, you know, planning another career. Okay. Because this stuff is in the future and it's honestly, it takes some effort to actually make coding assistants work today, right? You have to, you know, just…”