Jun 21, 2024 · 1h 8m · latent-space
How To Hire AI Engineers (ft. James Brady and Adam Wiggins of Elicit)
0:00 / 0:00
▶ Watch on YouTube →
gold bands on the timeline = statements, start to end. Hover to read, click to jump. CC turns on captions
Elicit engineering leaders James Brady and Adam Wiggins break down the core competencies, architectural principles, and hiring strategies required to build effective AI engineering teams. They share actionable frameworks for managing model non-determinism, designing probabilistic systems, and replacing standard interview puzzles with practical work simulations.
How this conversation actually went
Every chapter scored 0–10 on four independent dynamics. Hover any point for the reasoning behind the score. How this is scored →
The hosts as informed peer 5.0 Guest teaching 4.1 Guest disagreement 1.2 The hosts pushing back 1.9
speaking balance: gold is the hosts, purple is the guest (3 minute bins)
Sharpest disagreement ▶ 21:24 James Brady rejects universal fallback rules
James directly pushes back against generalized assumptions regarding model fallbacks, emphasizing the maintenance overhead of stale prompts and insisting decisions must be evaluated task by task.
Hardest push from the hosts ▶ 20:53 Host challenges model shadowing and fallback feasibilityThe host refuses the common premise that multi-model failover is standard practice, arguing that disparate prompt formatting and maintenance costs make cross-provider shadowing unrealistic.
Biggest teaching moment ▶ 44:20 James Brady explains relinquishing control in ML-first designJames educates the host by detailing his painful transition from deterministic symbolic enforcement to trusting probabilistic model outputs with lightweight regex parsing.
The host holds their own ▶ 27:50 Host confronts Adam Wiggins with his Heroku backgroundThe host uses his deep technical knowledge of Adam's career at Heroku to challenge his cautionary view on creating early platform abstractions for AI development.
the scores for every segment, with the reasoning behind each
| Chapter | Topic | The hosts as informed peer | Guest teaching | Guest disagreement | The hosts pushing back | Why |
|---|---|---|---|---|---|---|
| Introductions and Engineering Backgrounds of the Guests | 4 | 2 | 1 | 1 | The host establishes rapport with both guests, citing their previous collaborative history and framing AI engineering's breakdown as 90% software engineering. The guests recount their transitions into AI tooling and science literature synthesis in an agreeable, narrative tone. | |
| Defining AI Engineering and Its Three Core Pillars | 4 | 5 | 1 | 1 | James Brady lays out Elicit's three pillars of AI engineering, specifically elaborating on the fault-first mindset required to tame volatile LLM latency and outputs. Adam Wiggins educates the host by drawing a parallel between AI application development and distributed systems chaos engineering. | |
| Engineering Resilient AI Architectures and Defensive Interviewing | 5 | 4 | 1 | 2 | The host presses for concrete architectural tools rather than broad distributed systems buzzwords, prompting James to explain checked exceptions and open API typing across Python and TypeScript. The host also queries how defensive coding can be tested without breaking traditional happy-path interview structures. | |
| Navigating Trade-offs in Retries, Fallbacks, and Multi-Model Routing | 6 | 4 | 2 | 4 | The host challenges the practicality of model shadowing and fallbacks, pointing out that prompt variations and distinct model behaviors make seamless failovers rare in practice. James agrees on prompt maintenance friction but outlines Elicit's selective fallback methodology. | |
| Enterprise AI Gateways versus Agile Startup Abstractions | 6 | 5 | 2 | 3 | The host prompts a discussion on enterprise AI gateways and playfully challenges Adam Wiggins on his skepticism of early abstractions given his background founding Heroku. Wiggins explains the distinction between mature design patterns like Rails and the evolving 'wild west' of LLMs. | |
| Architectural Inversion and Continuous Model Capability Evaluation | 6 | 4 | 1 | 2 | The host articulates his framework of architectural inversion ('LLM at the core' vs 'code at the core') and queries how engineers assess model capabilities when technical reports lack detail. James and Adam detail Elicit's internal Slack culture and continuous evaluation suites. | |
| Embracing an ML-First Mindset and Relinquishing Deterministic Control | 5 | 6 | 1 | 1 | James Brady recounts having to unlearn 15 years of deterministic software engineering instincts in favor of Andreas Stuhlmüller's ML-first mindset. He illustrates this paradigm shift with an anecdote about generating inline citations using relaxed prompts and regex post-processing rather than rigid symbolic constraints. | |
| Balancing Fault-Tolerant Skepticism with Creative Exploration | 4 | 4 | 1 | 1 | The discussion explores the intrinsic tension between the skeptical fault-tolerance of seasoned principal engineers and the creative, open-ended optimism of junior builders. Both guests emphasize the necessity of synthesizing these dual mindsets in AI engineering teams. | |
| Effective Sourcing Strategies for AI Engineering Talent | 5 | 4 | 1 | 3 | The guests outline outbound and inbound talent sourcing strategies, including targeted job boards like 80,000 Hours. The host probes on the condition of Effective Altruism post-FTX/SBF, prompting James to reflect candidly on the sobering fallout while affirming ongoing safety-oriented hiring channels. | |
| Work Simulation Interviews, Final Takeaways, and Concluding Remarks | 5 | 3 | 1 | 1 | James advocates for replacing disconnected LeetCode interviews with realistic work-simulation loops. The host and guests close by reflecting on the coining of the 'AI Engineer' category and the practical labor-market dynamics driving its adoption. |
Statements from this episode (15)
Brady: Effective AI engineers need SWE fundamentals, ML curiosity, and a fault-first mindset
“The three things that we say are most important for a highly effective AI engineer first of all, conventional software engineering skills, which is kind of a given, but definitely worth mentioning. The second thing is a curiosity and enthusiasm for machine lea…”
Brady: Elicit routinely sees 10x p90 latency variation when prompting LLMs
“We do often normally, in fact, see a 10 X variation in P-ninety latency over the course of half an hour. When we're prompting these models, which is way higher than if you're working with a, you know, a more, more kind of conventional conventionally backed API…”
Brady: ML apps require distributed systems engineering skills from day one
“The kind of person that is deep in the guts of some kind of distributed systems, really high, high scale backend kind of a problem would probably naturally have these kinds of skills, but you'll find them on, on day one if you're building a, you know, an ML po…”
Brady: Elicit uses checked exceptions in Python to handle edge cases
“We use checked exceptions inside our Python code base, which means that we can use the type system to make sure we are handling, properly handling, all of the various things that could be going wrong, all the different exceptions that could be getting raised. …”
Brady: Elicit generates TypeScript types from Python OpenAPI specs instead of GraphQL
“We don't use GraphQL. So we've got the types defined in Python. That's the source of truth. And we go from the open API spec and there's a tool that you can use to generate types dynamically, like TypeScript types from those opening API definitions.”
Brady: Multi-model fallbacks require maintaining distinct prompts per LLM provider
“You definitely need to have a different prompt if you want to stay within a few percentage points degradation, like I said before. And that certainly comes at a cost.”
Brady: Centralized AI gateways restrict Elicit's model and prompt experimentation
“For illicit where really the secret source of the real secret source is which models we're using, how we're using them, how we're combining them, how we're thinking about the user problem, how we're thinking about all these pieces coming together. You really n…”
Wiggins: It is too early for standardized abstractions in AI engineering
“My sense is, yeah, still the wild west. That's what makes it so exciting and feels kind of too early for too much in the way of standardized abstractions. Not that it's not interesting to try, but you know, you can't necessarily get there in the same way rails…”
Brady: Prompt Engineering Won't Be a Durable Differentiating Skill
“I don't think that prompt engineering is going to be a kind of durable differential skill that people will hold. I do think that the way that you set up the ML problem to kind of ask the right questions, if you see what I mean, rather than the specific phrasin…”
Brady: Handling LLM Non-Determinism and Latency at Scale Is Unsolved
“There isn't some kind of industry-wide accepted way of handling that at massive scale. There are definitely patterns and anti-patterns and tools and whatnot, but it's not like this is a solved problem. So I would expect that it's not going to go down easily as…”
Brady: Elicit does not run an ML-focused interview for AI engineers
“We don't have an ML-focused interview for the AI engineer role at all, actually.”
Brady: Fault-tolerant engineering discipline and ML curiosity are in natural tension
“I think the fault first mindset and the ML curiosity attitude could be somewhat in tension, right? Because for example, the stereotypical, stereotypical version of someone that is great at building fault tolerant systems has probably been doing it for a decade…”
Brady: Elicit hires through 80,000 Hours and aligns with EA movement
“We're definitely affiliate affiliated with the safety, effective altruists kind of movement. We've gone to a few EA globals and have hired people effectively through the 80,000 hours list as well.”
Brady: Early-twenties engineers today show strikingly higher capability and maturity
“The maturity and capabilities and just kind of general put togetherness of people at that age now is strikingly different to where I was then.”
Brady: Elicit structures interviews as real work simulations, rejecting arbitrary questions
“I really have a strong dislike and distaste for interview questions, which are arbitrary and kind of strip away all the context from what it really is to do the work. We try to make the interview process that's illicit A simulation of working together.”