Zelikman: Language models can be trained to simulate students for test design
Eric Zelikman · No Priors Ep. 135 | With Humans& Founder Eric Zelikman · Oct 9, 2025 · at 22:54
Humans& founder Eric Zelikman describes his doctoral research at Stanford on simulating human behavior using language models.
0:00 / 0:16exact quote · 16.8s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“Like, even back in my PhD, I think one of my, I guess, less well-known works was actually about, we showed that you can train language models to simulate different kinds of students. For tests. Yeah, yeah. And by simulating students, you can actually design better tests for those students.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Eric Zelikman
Zelikman: Most AI labs treat humans as mere intermediates to full automation
“And maybe this is a strong statement, but I say for most labs, like the human is kind of, you know, the intermediate until you have like this fully automated, like, you know, system. And so spending a lot of time optimizing things for being really good at unde…”
Zelikman: AI field underinvests in memory due to task-centric training regimes
“I would say that memory is definitely like a feature that has been under, under-invested in by the field. But I would say that it is kind of difficult to invest in memory in this very, like, task-centric regime. Because if you have, like, A bunch of these, lik…”
Zelikman: Training reasoning models on just positive examples causes a plateau
“So if you only train on like the positive examples, then you end up in this kind of like potential minimum where there's just no more data that it can actually solve.”
Zelikman: Frontier AI models solve questions that stump actual PhD researchers
“Some of the HLE questions that these models are able to solve are genuinely things that are, like, non-trivial for, like, actual, like, PhD researchers.”
Zelikman: Current frontier AI models fundamentally lack emotional intelligence
“One of the core things is that they're not smart Like, emotionally, or, like, they're not smart on the level of, like, actually understanding kind of what people care about, or kind of, like, how to actually, like, help people accomplish the things that they c…”