Lukas Biewald is the co-founder and CEO of Weights & Biases. He is discussing the true breadth of enterprise machine learning adoption across major corporations.
Assertion Supported
Biewald: Most enterprises have not deployed LLMs into production yet
“I think that LLMs in particular, we talk to a lot of the people and we don't see a ton of people getting them into production yet. And I think it's funny, like VCs are always surprised, like when we tell them that I think that I don't know. I'm bullish on LMS,…”
Insight
Biewald: PyTorch beat TensorFlow through developer empathy, not eager execution
“I don't think they really, I think people tell this, the story of sort of the silver bullet. Of like you know, the eager execution model. But I think the reality is they just built a product with so much more empathy.”
Assertion Not checkable as stated
Biewald: Almost all major LLMs were trained using Weights & Biases
“I think all of the major LLMs out there, almost all were trained using weights and biases.”
Insight
Biewald: Simple operational errors cause more model failures than data drift
“People talk a lot about data drift in the industry. And that's this idea that like, you know, like language changes over time and you want to know that it's changing and sort of like have your model you know, notice that and update it. But I guess like what I …”
Assertion Not checkable as stated
Biewald: OpenAI is a W&B customer with a small number of production models
“OpenAI has been, like, a longtime customer. I mean, I consider them, like, extraordinarily sophisticated, and they have a pretty small number of models in, in production, so.”
Assertion Not checkable as stated
Increasing dataset accuracy from 90% to 95% repeatedly halves error rates
“So this is my same data set that I published online, but if you take it from 90% to 95%, you actually have the error rate, and then going up to a hundred percent, you have it again, right?”