Clark: Generative AI platform leaders are traditional machine learning veterans
Scott Clark · Building AI Systems You Can Trust · May 23, 2025 · at 7:55
Scott Clark, CEO of Distributional, discusses buyer personas and infrastructure continuity across machine learning generations with a16z partner Matt Bornstein.
0:00 / 0:09exact quote · 10.0s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“A lot of the people who are now in charge of building Gen AI platforms or productionizing these massive use cases are the same people who built those original machine learning systems.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Scott Clark
Clark: AI testing should use many weak estimators to detect behavioral differences
“Instead of trying to come up with a small number of strong estimators for performance, where we want to be able to conclusively say A is better than B, Instead, what we want is a large number of potentially weak estimators to be able to determine whether or no…”
Clark: Untuned deep learning models perform worse than tuned simple algorithms
“An untuned, sophisticated system will underperform a tuned simple system.”
Clark: System trust, not performance, limits enterprise AI value
“The thing that's holding back people getting value from these AI systems is not performance. It's not about squeezing out that last half a percent from some eval function or some performance metric. It's about being able to confidently trust these systems.”
Clark: High-level LLM evaluations mask undesired AI system behaviors
“We're seeing people do the exact same thing again today with LLMs, where they're focusing on these high-level metrics, these end outputs, these performance evals, and that ends up masking all of these potentially undesired behaviors within the system itself.”
Clark: Evaluating end-to-end AI performance hides upstream system failures
“And what I think a lot of firms are running into right now is if you're only looking at that last step, if you're only looking at the system's performance as a whole, it can be very difficult to understand when, where, and why behaviors are shifting within thi…”