Clark: Google pays $1 million for talent with deep learning intuition
Scott Clark · a16z Podcast | AI, from 'Toy' Problems to Practical Application · Jan 2, 2019 · at 12:50
Scott Clark, co-founder and CEO of SigOpt, discusses hyperparameter tuning and deep learning compensation during a panel on machine learning optimization.
0:00 / 0:06exact quote · 6.2s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“This is why Google will pay like a million dollars for someone with 10 years of deep learning experiences is that intuition that's built up.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Scott Clark
Clark: AI testing should use many weak estimators to detect behavioral differences
“Instead of trying to come up with a small number of strong estimators for performance, where we want to be able to conclusively say A is better than B, Instead, what we want is a large number of potentially weak estimators to be able to determine whether or no…”
Clark: Untuned deep learning models perform worse than tuned simple algorithms
“An untuned, sophisticated system will underperform a tuned simple system.”
Clark: System trust, not performance, limits enterprise AI value
“The thing that's holding back people getting value from these AI systems is not performance. It's not about squeezing out that last half a percent from some eval function or some performance metric. It's about being able to confidently trust these systems.”
Clark: High-level LLM evaluations mask undesired AI system behaviors
“We're seeing people do the exact same thing again today with LLMs, where they're focusing on these high-level metrics, these end outputs, these performance evals, and that ends up masking all of these potentially undesired behaviors within the system itself.”
Clark: Evaluating end-to-end AI performance hides upstream system failures
“And what I think a lot of firms are running into right now is if you're only looking at that last step, if you're only looking at the system's performance as a whole, it can be very difficult to understand when, where, and why behaviors are shifting within thi…”