StarZero All shows Try the podcast clipper →

context length

also referred to as: context lengths

4 statements across 4 episodes · 1 bullish · 1 bearish · 4 people on the record · first statement Aug 31, 2023 by Eugene Cheah · across every show →

Everything said about context length, oldest first

Aug 31, 2023 neutral
Insight
RWKV Performance Degrades on Context Lengths Beyond Its Training Data
“Well, it will, as a neural network, it will happily keep going on for infinite context, man. It will just keep generating. does it do well? That's the answer is no, because if you didn't train it to handle that situation, and that's actually a child rule. So,…”
Eugene Cheah Aug 31, 2023 ▶ 1:13:03 RWKV: Reinventing RNNs for the Transformer Era
Feb 12, 2026 neutral
Assertion Supported
Dean: Single needle-in-a-haystack benchmark is saturated up to 128k context lengths
“As you say that needed single needle in a haystack Benchmark is really saturated for at least context lengths up to one 28 K or something.”
Jeff Dean Feb 12, 2026 ▶ 13:24 The AI Frontier: from Gemini 3 Deep Think distilling to Flash — Jeff Dean
Apr 18, 2026 bearish
Prediction Open · timeframe Apr 2031
Swix: Frontier model context windows will not reach billion-token scales
“We took, you know, three years to go from 100 K context to one million context in every frontier model, but we're not going to a billion, you're not going to a trillion with context graphs, context lengths.”
Shawn Wang Apr 18, 2026 ▶ 25:12 ⚡️ How to turn Documents into Knowledge: Graphs in Modern AI — Emil Eifrem, CEO Neo4J
Jun 1, 2026 positive
Insight
Ethan He: Temporal video compression cuts context length up to 4x
“The difference is if you compress the temporal dimension, you get a much higher compression rate. Because there is temporal redundancy between frames, because this frame and the last frame, likely they are mostly similar. So there's only some small difference.…”
Ethan He Jun 1, 2026 ▶ 22:09 Inside xAI: Building Grok Imagine in 3 Months, Videogen vs World Models, and Video Agents— Ethan He
Made with StarZero

Turn any episode into a week of clips.

This entire site, over 200 episodes transcribed, diarized, checked and made playable, runs on the StarZero media pipeline. Drop in your own episode and the podcast clipper finds the moments worth sharing, cuts them, captions them, and reframes them for every feed.

Try the podcast clipper Learn more