Aug 3, 2023 · 43m · no-priors
No Priors Ep. 26 | With Weights & Biases CEO Lukas Biewald
0:00 / 0:00
▶ Watch on YouTube →
gold bands on the timeline = statements, start to end. Hover to read, click to jump. CC turns on captions
In this episode of No Priors, hosts Sarah Guo and Elad Gil interview Weights & Biases co-founder and CEO Lukas Biewald to discuss the evolution of machine learning infrastructure, developer-first software design, and the emerging realities of enterprise LLM adoption.
How this conversation actually went
Every chapter scored 0–10 on four independent dynamics. Hover any point for the reasoning behind the score. The hosts hold 23.2% of the talking time here. How this is scored →
The hosts as informed peer 5.1 Guest teaching 3.0 Guest disagreement 1.8 The hosts pushing back 0.8
speaking balance: gold is the hosts, purple is the guest (3 minute bins)
Sharpest disagreement ▶ 22:40 Calling out the disconnect in LLM tooling hype
Lukas forcefully dismisses the common narrative that LLMs are already widely deployed, stating that there are likely more funded LLM tooling startups than companies with LLMs actually deployed in production.
Hardest push from the hosts ▶ 22:50 Sarah interrupts to clarify production definitionSarah directly interrupts Lukas to test whether his definition of 'production LLMs' is restricted to self-hosted and fine-tuned models rather than general API usage.
Biggest teaching moment ▶ 23:04 Exposing the reality of enterprise adoption lagLukas educates the hosts on the grounded reality of the market, pointing out that despite constant enterprise buzz, almost none have production deployments and the current tooling TAM remains very small.
The host holds their own ▶ 37:08 Sarah's analysis of open source failure at the application layerSarah articulates an insightful technical framework detailing why open source works for deep infrastructure but fails for complex workflow applications due to missing telemetry loops and contributor incentives.
the scores for every segment, with the reasoning behind each
| Chapter | Topic | The hosts as informed peer | Guest teaching | Guest disagreement | The hosts pushing back | Why |
|---|---|---|---|---|---|---|
| Studying at Stanford Under Daphne Koller | 4 | 2 | 1 | 1 | Sarah sets the context of studying under Daphne Koller and mentions recent discussions around probabilistic graphs. Lukas reflects candidly on Koller's rigorous teaching and how early machine learning and Bayes nets largely failed to deliver at the time. | |
| From Academic NLP to Yahoo and Data Insights | 4 | 3 | 1 | 0 | Sarah and Lukas discuss his early academic work on word sense disambiguation before shifting to Yahoo search ranking. Lukas explains the realization that model algorithms mattered less than iterative training data quality, highlighting the flaw of waterfall requirements. | |
| The Dolores Labs Journey and Market Cycles | 6 | 2 | 1 | 1 | Elad demonstrates substantial historical context by recalling Lukas's early Dolores Labs days, Mechanical Turk's limitations, and Travis Kalanick's Hackpad meetups. Lukas details the eight-year lull in ML growth before autonomous vehicles took off, candidly admitting Scale AI beat them. | |
| Genesis and Developer Philosophy of Weights & Biases | 5 | 3 | 2 | 1 | Elad quotes Lukas's founding thesis on developer tooling for ML, prompting Lukas to share how deep learning forced him to update his views. Lukas humorously contrasts DevOps complexity like Docker and Git LFS with the practical needs of ML researchers who prefer simple, reliable tooling. | |
| The LLM Shift and Launching Prompts | 5 | 3 | 2 | 0 | Sarah asks how Weights and Biases adapted its roadmap to the sudden rise of LLMs. Lukas explains realizing that LLMs posed an existential threat to traditional classification tasks, leading the company to pivot resources quickly to their Prompts product. | |
| Enterprise LLM Adoption and Model Selection | 6 | 6 | 4 | 3 | Elad brings up the open source versus proprietary model debate. Lukas pushes back against venture optimism, clarifying that almost no enterprises actually have LLMs running in production yet and that current tooling TAM is quite small, which Elad contextualizes through enterprise sales cycles. | |
| Machine Learning in Pharma and Drug Discovery | 6 | 3 | 1 | 1 | Lukas identifies pharma as an under-the-radar ML boom area due to massive hiring for in silico drug testing. Both Sarah and Elad contribute domain expertise on biotech commercialization lag and fund economics. | |
| Broad Horizontal Adoption from Gaming to Agriculture | 4 | 2 | 1 | 0 | Sarah inquires about the broad customer base of Weights and Biases. Lukas describes widespread adoption across gaming and agtech, highlighting smart spraying systems by John Deere, and notes that tooling is horizontal across ML teams. | |
| Developer-First Strategy Versus Traditional MLOps | 4 | 4 | 3 | 0 | Sarah asks about developer versus enterprise adoption strategies. Lukas criticizes traditional enterprise MLOps teams who raised venture capital but build Kubernetes-heavy tools disconnected from what actual ML developers and researchers want. | |
| Open Source Strategy and Telemetry Advantages | 7 | 3 | 2 | 2 | Elad asks about closed versus open source strategies, and Lukas explains how closed-source telemetry allows continuous UX improvement. Sarah delivers a sophisticated breakdown of why open source fails at the application UI layer compared to deep infrastructure. | |
| Founder Insights and Effective Customer Discovery | 5 | 2 | 2 | 0 | Elad asks for second-time founder lessons and advice for AI entrepreneurs. Lukas emphasizes ruthlessly prioritizing long-term value over short-term quarterly ARR targets and remaining brutally honest during customer discovery. |
Statements from this episode (22)
Biewald: Bayesian Networks Never Worked Well For Many Applications
“Daphne was actually really obsessed at the time with a thing called base nets, which you don't hear about too much anymore, because I don't think they ever really you know, worked for many applications. I hope I'm not offending anyone, but that's my understand…”
Biewald: Data-Driven ML Outperforms Linguistic Strategies In NLP
“My general sense is that these sort of like linguistic oriented strategies really don't work that well. It's kind of like by feeding more data in and sort of like Working on outcomes, you can figure these things out much better.”
Biewald: Why I Turned Down Google To Join Yahoo
“I actually turned down an offer from Google because they didn't tell me what I would be working on to go to Yahoo because they were like, okay, you can work on, you know, search rank ranking in different languages.”
Biewald: Yahoo Search Success Depended Entirely On Local Training Data Quality
“The model that I'm building is like the same for each country. It's the training data though is different. So some countries would take the training data collection process really seriously and they'd get a great model. And some would just like really half-ass…”
Biewald: Scale AI Completely Ate Our Lunch In Self-Driving Data Labeling
“And then it was a funny experience cause like scale came along and totally ate our lunch on the, in the self-driving market, which is a market like I knew and loved.”
Gil: Machine learning progress is discontinuous and bumpy, not linear
“And I think a lot of people basically view ML as this sort of continuity and everything has always been kind of rising in a, Sort of almost linear way. And in reality, it's this very bumpy set of discontinuities in terms of the set of technologies and markets …”
Biewald: I Interned Briefly At OpenAI To Learn Modern Deep Learning
“I actually like interned briefly at OpenAI where I was just like, look, I will just do whatever, you know, work you want.”
Biewald: Research code inevitably bleeds into production at every company
“I think companies have this idea that like the researchers are just going to like throw the thing over the fence and then it's going to be in production. But it doesn't really work actually. Like, I think that's a bad pattern that people sort of like imagine t…”
Biewald: Most Modern LLMs Were Built Using Weights And Biases
“And we actually helped most of the LLMs out there were built using weights and biases.”
Biewald: 95% Of ML Developers Are Using OpenAI's GPT Models
“Like I see like 95% of the people out there, you know, using GPT for these ML tasks.”
Biewald: There Are More Funded LLM Tools Than Production LLM Apps
“Very, very few people have LLMs in production. Like there's probably more companies that have raised money as like LLM tools than companies that have LLMs in production, which is like insane. It's just like an insanely saturated tools market with very few peop…”
Gil: Big Enterprises Need 1-2 Years To Deploy LLMs Into Production
“And the big enterprises are going to take another year or two because it's, they're just in their planning cycle still around this stuff. They just started really thinking about it and how to incorporate it and what to use it for. And then they're going to hav…”
Biewald: No ML-Developed Drug Has Ever Completed Clinical Trials
“So no drug developed by ML has gone through clinical trials.”
Gil: One Top Biotech VC Shipped Zero Drugs In 20 Years
“There are certain venture funds that have done incredibly well financially in pharma, where there's one in particular I can think of that never shipped a drug. Until the COVID era, and they were in business for 20 years. And they made all this money, and they …”
Biewald: John Deere Has Deployed ML-Powered Targeted Weed Sprayers
“Like, you know, we worked with John Deere for years back from a figure eight days to, you know, weights and biases and they're, they've deployed sprayers that only target the weeds in, in fields. It's deployed.”
Biewald: Top-Down Enterprise Software Startups Cannot Transition to Product-Led Growth
“The early companies had to sell to executives, which I totally understand. Like that's what crowdflower had to do. And the problem there is you kind of get stuck in these like multimillion dollar deals and like, you just can't get out of that. Like you can't s…”
Biewald: Nvidia Chips Forced ML Researchers To Become Software Developers
“When NVIDIA came along and these chips worked for deep learning, it just, like, broke the entire stack. Like, it was, like, a first time in my career where I'm, like, running into, like, linker errors. I'm like, what the fuck is a linker error? Like, I vaguely…”
Biewald: DevOps-Founded MLOps Startups Struggle to Connect with Ordinary Developers
“Every MLOps team then realizes they could raise like a shitload of funding, you know? And so like you got like every, every major company, their MLOps team like went off and like raised money to like make a new product in the market, which I think from an inve…”
Biewald: Closed-Source Telemetry Gives W&B A Massive UX Advantage
“There's been a major pro, which is that all our competitors are open source. And what that means is that they don't get to see how users actually use their software. And so I think our software is a lot more ergonomic because we have like metrics on what peopl…”
Guo: Almost No Open-Source Application-Layer Software Products Have Ever Succeeded
“There are a, you know, zero to marginal number of open source applications. That have actually succeeded.”
Biewald: Chasing Short-Term ARR Hurts A Startup's Next Quarter Growth
“I think everybody, like, chases, every entrepreneur chases, like, short-term, like, ARR numbers, like, in quarter, but then it, like, hurts your growth rate the next quarter. It's like, it would actually be better always to, like, push out deals, but, like, no…”