Andreas Stuhlmüller
Co-founder & CEO, Elicit · 1 appearance on the record.
computed by AI from the episodes · how this works → · full disclaimer →
founderexecutivescientist@stuhlmueller ↗stuhlmueller.org ↗
Andreas Stuhlmüller is the co-founder and CEO of Elicit, an AI research assistant platform for scientific literature reviews and reasoning. He earned a Ph.D. in cognitive science from MIT, conducted postdoctoral research at Stanford, and co-created the probabilistic programming language WebPPL.
18statements → 4claims → 0claims resolved → 3.39/5average certainty → 2.11/5average debate potential → 4said about them ↓
4 not checkable as stated how the 4 claims stand · each chip opens the sources
2 predictions · 2 assertions · 2 opinions · 9 insights · 3 disclosures · every statement was checked. The predictions and assertions are the 4 claims: statements the public record can support or contradict. 0 are resolved, and 4 name no date, number or outcome precise enough to check. Everything else (opinions, insights, what ifs, disclosures) can never be settled by the record, so it carries no assessment.
Everything Andreas Stuhlmüller said on Latent Space that made the record, most notable first. Filter by type, assessment or year in the ledger →
Stuhlmüller: AI requires deeper world models to make novel scientific discoveries
“Having deeper models of how, let's see, what are the underlying structures of different domains, how they're related or not related, I think will be an important ingredient for models actually being able to make novel contributions.”
Stuhlmüller: Academia cannot build great software tools due to paper timelines
“It's really hard to actually build interesting tools as an academic. You can't really hire great engineers. Everything is kind of on a paper to paper timeline.”
Stuhlmüller: Generalist AI research platforms will be a winner-take-all market
“So I think there will be, at least within research, I think there will be, like, one best platform, more or less for this type of generalist research. I think there may still be, like, some particular tools, like, for genomics, like, particular types of module…”
Stuhlmüller: Trading inference compute for answer accuracy is undervalued in AI
“Being able to invest more or less compute into getting more or less accurate answers is, I think, one of the core things we care about, and that I think is currently undervalued in the AI space.”
Stuhlmüller: Pure long-context LLMs are significantly harder to debug than RAG
“In one sense, I think you're right that the throw everything into the context window thing is easier to maintain because you just can swap out a model. In another sense, it's, if things go wrong, it's harder to debug, where, like, if you know, here's the proce…”
Stuhlmüller: Elicit builds scaffolding rather than training foundation models
“The way we are building Elicit is not let's train a foundation model to do more stuff. It's like let's build a scaffolding such that we can deploy powerful models to good ends.”
Stuhlmüller: Seed VCs urged Elicit to build legal AI over research
“We did encounter, I guess talking to VCs for our seed round. A lot of VCs were like, you know, researchers, they don't have any money. Why don't you build a legal assistant?”
Stuhlmüller: Claude Haiku offers an optimal balance of cost and accuracy
“Specifically, I think Cloud Haiku is like a good point on the kind of Pareto frontier, so I think it's like, it's not the, it's neither the cheapest model nor is it the most accurate, most high quality model, but it's just like a really good trade-off between …”
Stuhlmüller: Closed-source models consume most of Elicit's compute budget
“I'd say, like, in terms of number of careers, it's maybe similar. In terms of, like, cost and compute, I think the closed models make, make up more of the budget, since the main cases where you want to use closed models are cases where they're just smarter, wh…”
Stuhlmüller: Notebooks enable debugging and scaling workflows far better than chat
“But the important difference in our minds is with notebooks you can define a process. So in, in data science you can go like, here's like my data analysis process that takes in a CSV and then does some Extraction, and then generates a figure at the end, and yo…”
Stuhlmüller: AI primitives should be semantic tasks, not granular chain-of-thought
“I think chain of thought is maybe still like kind of one level lower on the abstraction hierarchy than we would think of notebooks. I think we'll probably want to think about more semantic pieces, like a building block is more like A paper search, or an extrac…”
Stuhlmüller: Separate evaluator models yield better uncertainty estimates than self-evaluation
“I think in some cases we also use the different models for the uncertainty estimates. Yes, then, for the question answering. So, one model would say, here's my chain of thought, here's my answer, and then a different type of model. Let's say the first model is…”
Stuhlmüller: List-wise re-ranking outperforms per-item scoring in search
“In the past, I think a lot of ranking was kind of per item ranking where you would score each individual item, maybe using increasingly expensive scoring methods, and then rank based on the scores, but I think list-wise re-ranking where you have a model that c…”
Stuhlmüller: Agentic search ideally balances parametric memory with retrieved context documents
“I think probably the ideal thing looks a bit more like agent control where the model can issue a query that then is intended to surface documents that substantiate its hunch. So I would, that's maybe a reasonable middle ground between model just telling you an…”
Stuhlmüller: In 10-20 years, today's scientific methods will look incredibly unsystematic
“Probably, yeah, I, I'd guess in like, 1020 years, we'll look back and it will be incredible how unsystematic science was back in the day.”
Stuhlmüller: AI orchestration resembles software engineering far more than ML research
“I think a lot of this looks more like traditional software engineering than it does look like machine learning research, and I think the people who are, like, really good at building good abstractions building applications that can kind of survive even if some…”
Stuhlmüller: Elicit continues to use T5-based models
“We do also use, like, T-Five-based models, even, even now but started, yeah, started with GPT-II.”
Stuhlmüller: Elicit co-founders wrote a 50-page mutual evaluation document before starting
“We also did a pretty lengthy mutual evaluation process where we had a Google Doc where we had all kinds of questions for each other, and I think it ended up being around 50 pages or so of, like, various, like, questions and back and forth.”
The other half of the tape: Andreas Stuhlmüller's own voice is left out of every number here. Other people bring the name up 4 times in 1 episode on Latent Space. every mention, with the transcript →
Who brings them up most James Brady 1
Every mention by year
tap a year for its mentions
2024 4 mentions in 1 episode
Appearances (1)
| Episode | Date | Speaking time |
|---|---|---|
| Supervise the Process of AI Research — with Jungwon Byun and Andreas Stuhlmüller of Elicit | Apr 11, 2024 | 21m |