Pixels
topic on 5 shows · 5 statements across 5 episodes · said 3 times in 1 episodes since 2024
Big Technology 3 
Latent Space 
No Priors 
the a16z Podcast 
TBPN
Mentions by year, every show
tap a year for its mentions
Big Technology 3
2024 3 mentions in 1 episode
every mention on every show, scene by scene, with the transcript →
5 statements about Pixels, every show
Ethan He: Training transformers directly on raw image pixels is impossible
“If you're trying, if you can technically, theoretically train image or video models on pure pixels, but the problem is that the, it's a lot of tokens. So like one image, like it's a thousand by a thousand is like one million tokens, one million pixels. It's im…”
Johnson: Pixels offer a more lossless world representation than tokenized text
“And then like you actually lose something if you translate to this like purely tokenized representations that we use in LLMs, right? Like you lose the font, you lose the line breaks, you lose sort of the two D arrangement on the page. And for a lot of cases, f…”
Ohanian: Pixel constraints force better gameplay and weirder indie game design
“And what I love about the constraint Is it similar to a great tweet, right? You're limited on characters. You got to make each one count. You're limited on pixels. You make each count and it forces you to think more about gameplay and do weirder interesting st…”
Fei-Fei Li: Language data contains more implicit human labeling than pixels
“Yes, philosophically, that's a really important question. But that actually is more true in language than pixels.”
Doshi: Pixels carry far higher information density than language
“So we know that like pixels have an enormous Amount of high information density compared to language and language is just really between me and you. It's like a compressed way that you and I can converse with each other at a higher, at somewhat of a higher ban…”