LLM Architecture
topic on 3 shows · 3 statements across 3 episodes
Latent Space 
No Priors 
the MAD Podcast
3 statements about LLM Architecture, every show
Edunov: LLM architectures are boring and fundamentally unchanged since 2017
“And honestly, LLM architectures are relatively boring. I don't know, probably alienate half of your audience. But it's like, it's a transforming layer in the end, like paper was published in 2017, and you go to any LLM lab today, you will see very, very simila…”
Future LLMs will prioritize architectural efficiency over larger model sizes
“I wouldn't expect bigger architectures. I would expect a more efficient architectures tweaks getting, The same modeling performance for less compute”
Wickramasekara: Science is uniquely structured to benefit from LLM architectures
“I go back to that sort of returns to intelligence piece where I think science is a problem that has some shape to it that really benefits from the LLM architecture. Like you just think about the corpus of scientific literature as like this vast pool of unstruc…”