Ethan He
AI Researcher & Engineer. On 1 show, 2 appearances. The Shows tab opens the full record on each.
scientistengineeracademic@EthanHe_42 ↗LinkedIn ↗yihui-he.github.io ↗
Ethan He previously worked on NVIDIA's Cosmos world model platform and Megatron-LM MoE architectures before joining xAI to help build Grok Imagine. He has authored research on model compression and currently focuses on LLM context architecture and video agents.
1shows
2appearances
56statements
8resolved
7supported
1contradicted
88%fully supported
2said about them ↓ Everything Ethan He said on any show that made the record, most notable first. Each card names its show and opens the statement there.
He: Distillation works because teacher models are simpler than the internet
“I guess the, from the modeling perspective, the strong model, the teacher model is trying to model The image and videos of entire internet. And that distribution is extremely complex. As a step distilled model is just trying to learn from the teacher. The teac…”
He: Grok Imagine 0.9 was first large-scale joint audio-video model deployed
“So Grok Imagine, there were .9, I believe it's is a first first audio video trends model deployed at a large scale.”
Ethan He defines world models as real-time, interactive, long-horizon video
“So word model is like real time, interactive, long horizon videos.”
Ethan He predicts world models will culminate in real-time neural computers
“I think the final state will be, for example, like a video version of Playbook where you can interact with a neural computer. You move your mouse and you click on the generative interface. And it will reply to you through, through pixels generally in real time…”
He: Manual reference video conditioning is a workaround, not true long context
“It doesn't need to have a very long context, but it's, I feel like it's an intermediate solution. It's cheating. Yeah, the model should Be able to like selectively know, like where, where should I draw references?”
Ethan He: RLMs and video models will dynamically pull context like humans
“But humans' contacts can, like, attention can work because we can dynamically pull in contacts from different places. The same mechanism I think it's going to happen for RLMs and video models.”
He: AI first-principles planning calculates the theoretical minimum days to ship
“If you think about some limitation, for example, the current data, like how, how fast can we acquire the videos? And if you think about training the models, like what's the iteration speed? For training a model end-to-end and how, how would adding more GPUs ac…”
He: Elon Musk is very hands-on and works closely with xAI teams
“He also worked very closely with people like people imagine online, like he, he's very hands-on.”
Ethan He: AI watermarking will remain vulnerable to reverse-engineering
“As a limitation is like the technology is, as a paper, Was out there and people can reverse engineer that how to get rid of it. And it's, I think even as it advance, it's still, still possible to reverse engineer it.”
Ethan He: Video Agents Will Transition to Fully Automated Video Production
“So in, in Asian, in Gorky Imagine agent mode, you can still go in there and do, do stuff by yourself. Gradually, as the model capability increase, it will be able to do everything fully automated.”
Ethan He: Language models prompt AI models better than humans
“Most of the people were actually not very good at prompting. Actually, language models have a better sense of how to prompt AI models. AI models know AI models better.”
He: Video Agents Are Inherently Costlier Due to Iterative Multi-Sample Generation
“I think the enterprise will have much more budget for video models because the agents are inherently more expensive than the other video models themselves because they do this iterative process. They generate many, many variations.”
Ethan He left xAI because changing corporate priorities limited LLM research
“For me there's a lot of research you want to do that you cannot do at, as a company. And also like the priorities and objective, the, for company typically can change very fast. It is, it's also the same for XAI. So, so now it's kind of like the time to, there…”
Ethan He: LLMs will soon become context-aware and manage context
“I think one thing pretty, pretty interesting. I think might be happening soon is the language models will be like context aware and manage its own context.”
Ethan He: External heuristic engineering gets absorbed into models
“From our experience, the heuristic engineering also have the models get absorbed into the models themselves.”
Ethan He: Megatron MoE was first to train trillion-parameter MoEs at 40% MFU
“The Megatron MOEs was the first It was the first framework open source to be able to train these MOEs at very large scales, like a hundred billion parameters to even trillion parameters efficiently at like 40% MFU.”
He: Token dropping works in MoE pre-training; dropless excels in fine-tuning
“A lot of pre-training experiments show that Token dropping is very efficient, and it doesn't impact performance, but in some of the, like, the downstream fine-tuning, people realize drop-less is better.”
Ethan He: Hugging Face's sequential GEMM loop for Mixtral is inefficient
“Let's also look at the implementation of Mixtro eight by seven on Hagen-Phys transformer. You will soon notice the, in the expert operation there, You would iterate over all of the experts and compute each of the gem operations one by one. We found that this i…”
Ethan He: MoE experts do not cleanly specialize into semantic domains
“Unfortunately, people didn't find, like, a significant interpretability inside these experts. Say, one expert focus on math, the other focus on literature. I think the problem is that neural network hidden states are already very entangled. So, when hidden sta…”
He: Upcycling a 15B model on 1T tokens yielded 4% MMLU gain
“On other scaling experiments, we tried on 15 B models upcycling and applied on one trillion tokens and achieved roughly about five percent improvement in terms of the validation loss and four percent improvement on MMLU.”
Ethan He: 64 experts is the sweet spot for MoE upcycling
“We found, 64 experts is kind of like the sweet spot. If you increase the number of experts beyond 64, it provides diminishing return.”
Ethan He: NVIDIA spent about a year building the Cosmos model
“One thing I say, like, thanks to my experience at NVIDIA, because first time when we were building Cosmos together, we built it for about a year.”
Ethan He: NVIDIA Cosmos required labelers to describe videos for blind reconstruction
“So that's in the protocol of Cosmos labeling. We required the objective we gave to the labelers was that you have to describe the video as detailed as possible, such that a blind person hears a blob of text, can reconstruct what the video is like from their he…”
The other half of the tape: Ethan He's own voice is left out of every number here. Other people bring the name up 2 times in 1 episode across the shows. every mention, with the transcript →
Who brings them up most Shawn Wang 2
Every mention by year
tap a year for its mentions
Latent Space 2
One line per show, most statements first. The link opens Ethan's full record on that show: the calibration, argument clarity, speaking style and every statement made there.
| Show | Role there | Eps | Statements | Record | |
|---|---|---|---|---|---|
| LATENT SPACELEDGER | AI Researcher & Engineer | 2 | 56 | 88% 7/8 | full record on Latent Space → |