James Brady
1 appearance on the record.
computed by AI from the episodes · how this works → · full disclaimer →
2 not checkable as stated how the 2 claims stand · each chip opens the sources
1 prediction · 1 assertion · 2 opinions · 5 insights · 5 disclosures · every statement was checked. The prediction and assertion are the 2 claims: statements the public record can support or contradict. 0 are resolved, and 2 name no date, number or outcome precise enough to check. Everything else (opinions, insights, what ifs, disclosures) can never be settled by the record, so it carries no assessment.
Everything James Brady said on Latent Space that made the record, most notable first. Filter by type, assessment or year in the ledger →
Brady: Prompt Engineering Won't Be a Durable Differentiating Skill
“I don't think that prompt engineering is going to be a kind of durable differential skill that people will hold. I do think that the way that you set up the ML problem to kind of ask the right questions, if you see what I mean, rather than the specific phrasin…”
Brady: Elicit uses checked exceptions in Python to handle edge cases
“We use checked exceptions inside our Python code base, which means that we can use the type system to make sure we are handling, properly handling, all of the various things that could be going wrong, all the different exceptions that could be getting raised. …”
Brady: Centralized AI gateways restrict Elicit's model and prompt experimentation
“For illicit where really the secret source of the real secret source is which models we're using, how we're using them, how we're combining them, how we're thinking about the user problem, how we're thinking about all these pieces coming together. You really n…”
Brady: Elicit does not run an ML-focused interview for AI engineers
“We don't have an ML-focused interview for the AI engineer role at all, actually.”
Brady: Fault-tolerant engineering discipline and ML curiosity are in natural tension
“I think the fault first mindset and the ML curiosity attitude could be somewhat in tension, right? Because for example, the stereotypical, stereotypical version of someone that is great at building fault tolerant systems has probably been doing it for a decade…”
Brady: Early-twenties engineers today show strikingly higher capability and maturity
“The maturity and capabilities and just kind of general put togetherness of people at that age now is strikingly different to where I was then.”
Brady: Effective AI engineers need SWE fundamentals, ML curiosity, and a fault-first mindset
“The three things that we say are most important for a highly effective AI engineer first of all, conventional software engineering skills, which is kind of a given, but definitely worth mentioning. The second thing is a curiosity and enthusiasm for machine lea…”
Brady: ML apps require distributed systems engineering skills from day one
“The kind of person that is deep in the guts of some kind of distributed systems, really high, high scale backend kind of a problem would probably naturally have these kinds of skills, but you'll find them on, on day one if you're building a, you know, an ML po…”
Brady: Multi-model fallbacks require maintaining distinct prompts per LLM provider
“You definitely need to have a different prompt if you want to stay within a few percentage points degradation, like I said before. And that certainly comes at a cost.”
Brady: Handling LLM Non-Determinism and Latency at Scale Is Unsolved
“There isn't some kind of industry-wide accepted way of handling that at massive scale. There are definitely patterns and anti-patterns and tools and whatnot, but it's not like this is a solved problem. So I would expect that it's not going to go down easily as…”
Brady: Elicit hires through 80,000 Hours and aligns with EA movement
“We're definitely affiliate affiliated with the safety, effective altruists kind of movement. We've gone to a few EA globals and have hired people effectively through the 80,000 hours list as well.”
Brady: Elicit structures interviews as real work simulations, rejecting arbitrary questions
“I really have a strong dislike and distaste for interview questions, which are arbitrary and kind of strip away all the context from what it really is to do the work. We try to make the interview process that's illicit A simulation of working together.”
Brady: Elicit routinely sees 10x p90 latency variation when prompting LLMs
“We do often normally, in fact, see a 10 X variation in P-ninety latency over the course of half an hour. When we're prompting these models, which is way higher than if you're working with a, you know, a more, more kind of conventional conventionally backed API…”
Brady: Elicit generates TypeScript types from Python OpenAPI specs instead of GraphQL
“We don't use GraphQL. So we've got the types defined in Python. That's the source of truth. And we go from the open API spec and there's a tool that you can use to generate types dynamically, like TypeScript types from those opening API definitions.”
Appearances (1)
| Episode | Date | Speaking time |
|---|---|---|
| How To Hire AI Engineers (ft. James Brady and Adam Wiggins of Elicit) | Jun 21, 2024 | 31m |