Everything Lukas Biewald said on any show that made the record, most notable first. Each card names its show and opens the statement there.
Biewald: Most enterprises have not deployed LLMs into production yet
“I think that LLMs in particular, we talk to a lot of the people and we don't see a ton of people getting them into production yet. And I think it's funny, like VCs are always surprised, like when we tell them that I think that I don't know. I'm bullish on LMS,…”
Biewald estimates 99% of Global 2000 use ML for core operations
“I bet 99% of the global 2000 is using machine learning for something that they actually really care about.”
Biewald: PyTorch beat TensorFlow through developer empathy, not eager execution
“I don't think they really, I think people tell this, the story of sort of the silver bullet. Of like you know, the eager execution model. But I think the reality is they just built a product with so much more empathy.”
Biewald: 95% Of ML Developers Are Using OpenAI's GPT Models
“Like I see like 95% of the people out there, you know, using GPT for these ML tasks.”
Biewald: There Are More Funded LLM Tools Than Production LLM Apps
“Very, very few people have LLMs in production. Like there's probably more companies that have raised money as like LLM tools than companies that have LLMs in production, which is like insane. It's just like an insanely saturated tools market with very few peop…”
Biewald: Almost all major LLMs were trained using Weights & Biases
“I think all of the major LLMs out there, almost all were trained using weights and biases.”
Biewald: Simple operational errors cause more model failures than data drift
“People talk a lot about data drift in the industry. And that's this idea that like, you know, like language changes over time and you want to know that it's changing and sort of like have your model you know, notice that and update it. But I guess like what I …”
Biewald: OpenAI is a W&B customer with a small number of production models
“OpenAI has been, like, a longtime customer. I mean, I consider them, like, extraordinarily sophisticated, and they have a pretty small number of models in, in production, so.”
Biewald: Bayesian Networks Never Worked Well For Many Applications
“Daphne was actually really obsessed at the time with a thing called base nets, which you don't hear about too much anymore, because I don't think they ever really you know, worked for many applications. I hope I'm not offending anyone, but that's my understand…”
Biewald: Most Modern LLMs Were Built Using Weights And Biases
“And we actually helped most of the LLMs out there were built using weights and biases.”
Biewald: Top-Down Enterprise Software Startups Cannot Transition to Product-Led Growth
“The early companies had to sell to executives, which I totally understand. Like that's what crowdflower had to do. And the problem there is you kind of get stuck in these like multimillion dollar deals and like, you just can't get out of that. Like you can't s…”
Biewald: DevOps-Founded MLOps Startups Struggle to Connect with Ordinary Developers
“Every MLOps team then realizes they could raise like a shitload of funding, you know? And so like you got like every, every major company, their MLOps team like went off and like raised money to like make a new product in the market, which I think from an inve…”
Biewald: Closed-Source Telemetry Gives W&B A Massive UX Advantage
“There's been a major pro, which is that all our competitors are open source. And what that means is that they don't get to see how users actually use their software. And so I think our software is a lot more ergonomic because we have like metrics on what peopl…”
Biewald: Chasing Short-Term ARR Hurts A Startup's Next Quarter Growth
“I think everybody, like, chases, every entrepreneur chases, like, short-term, like, ARR numbers, like, in quarter, but then it, like, hurts your growth rate the next quarter. It's like, it would actually be better always to, like, push out deals, but, like, no…”
Biewald: Founders Are Lying About Their Volume Of Customer Discovery Meetings
“I think people are all lying to each other about how much like actual kind of customer meetings they're doing.”
Increasing dataset accuracy from 90% to 95% repeatedly halves error rates
“So this is my same data set that I published online, but if you take it from 90% to 95%, you actually have the error rate, and then going up to a hundred percent, you have it again, right?”
Biewald: Data labeling software requires a top-down sales model
“Data labeling, I think really wants to be a top down sale”
Biewald: LLM API developers are less mathematically specialized than traditional ML engineers
“Even the people Working with a lot of these APIs you know, are, like, less huge math nerds than, you know, some of the people that have been training, you know, models for a long time.”
Biewald: Developers building with LLMs focus heavily on qualitative anecdotes over metrics
“In the LL world, like the anecdote is something people really pay attention to.”
Biewald: Traditional ML methods like boosted trees remain a large share of W&B usage
“A lot of people still running, you know, boosted trees or, you know, random forests inside of weights and biases. So we, you know, it's like actually huge. We should probably do a block, but it's still a big fraction of our You know, of our user base.”
Biewald: Pharma is investing far more in deep learning than realized
“I think pharma is investing way more in deep learning than people realize.”
Biewald: Technical buyers don't want sales dinners, they just want facts
“Nobody wants to golf or anything. I mean, that's for sure, right? Like, I mean, there's people like a super aggressive salesperson. Some of my salespeople are really competitive. I am actually really competitive myself, but they kind of like suppress it in a w…”
Biewald: Data-Driven ML Outperforms Linguistic Strategies In NLP
“My general sense is that these sort of like linguistic oriented strategies really don't work that well. It's kind of like by feeding more data in and sort of like Working on outcomes, you can figure these things out much better.”
Biewald: Scale AI Completely Ate Our Lunch In Self-Driving Data Labeling
“And then it was a funny experience cause like scale came along and totally ate our lunch on the, in the self-driving market, which is a market like I knew and loved.”