Brady: Centralized AI gateways restrict Elicit's model and prompt experimentation
James Brady · How To Hire AI Engineers (ft. James Brady and Adam Wiggins of Elicit) · Jun 21, 2024 · at 26:45
James Brady, head of engineering at Elicit, discusses whether centralized corporate AI gateways make sense for agile AI-native startups.
0:00 / 0:29exact quote · 29.7s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“For illicit where really the secret source of the real secret source is which models we're using, how we're using them, how we're combining them, how we're thinking about the user problem, how we're thinking about all these pieces coming together. You really need to have all of the affordances available to you to be able to experiment with things and iterate rapidly. And Generally speaking, whenever you put these kind of layers of abstraction and control and generalization in there, that, that gets in the way. So, so for us, it would not work.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from James Brady
Brady: Prompt Engineering Won't Be a Durable Differentiating Skill
“I don't think that prompt engineering is going to be a kind of durable differential skill that people will hold. I do think that the way that you set up the ML problem to kind of ask the right questions, if you see what I mean, rather than the specific phrasin…”
Brady: Elicit uses checked exceptions in Python to handle edge cases
“We use checked exceptions inside our Python code base, which means that we can use the type system to make sure we are handling, properly handling, all of the various things that could be going wrong, all the different exceptions that could be getting raised. …”
Brady: Elicit does not run an ML-focused interview for AI engineers
“We don't have an ML-focused interview for the AI engineer role at all, actually.”
Brady: Fault-tolerant engineering discipline and ML curiosity are in natural tension
“I think the fault first mindset and the ML curiosity attitude could be somewhat in tension, right? Because for example, the stereotypical, stereotypical version of someone that is great at building fault tolerant systems has probably been doing it for a decade…”
Brady: Early-twenties engineers today show strikingly higher capability and maturity
“The maturity and capabilities and just kind of general put togetherness of people at that age now is strikingly different to where I was then.”
Brady: Effective AI engineers need SWE fundamentals, ML curiosity, and a fault-first mindset
“The three things that we say are most important for a highly effective AI engineer first of all, conventional software engineering skills, which is kind of a given, but definitely worth mentioning. The second thing is a curiosity and enthusiasm for machine lea…”