Prediction Not checkable as stated
Algorithmic unhobblers could soon expand context windows to 100 million tokens
“I wouldn't be surprised if we do see the ability to, like, break through to, like, ten million, twenty million, a hundred million context through the, an unhobbler showing up.”
Insight
LLMs favor CLI tools over APIs due to massive pre-training data volumes
“I think that in pre-training, there's just an enormous amount of command line data. Like even let's ignore, let's like, let's ignore RL. Like you're doing no harness post training. Just the amount of like CLI versus API documentation for just like navigating t…”
Prediction Open · timeframe Dec 2026
AI agents will achieve 24-hour self-consistent autonomous runtimes by late 2026
“We will see before the end of the year an agent that is capable of running for longer than 24 hours with like self consistency the entire time.”
Insight
NVIDIA uses 'Speed of Light' framework to define minimum viable goals
“SOL is a term that NVIDIA is used to sort of like instigate a compelling event. You say, this is done. How do we get there? What is the minimum, as much as necessary, as little as possible thing that it takes for us to get exactly here? And it helps you just b…”
Assertion Supported
Jensen Huang prioritizes strategic investments in 'zero billion dollar markets'
“Jensen, He says, we're completely happy investing in zero billion dollar markets. We don't care if this creates revenue. It's important for us to know about this market. We think it will be important in the future. It can be zero billion dollars for a while.”
Assertion Supported
NVIDIA announces Rubin CPX as a dedicated prefill-specific hardware accelerator
“And like with our future generations, generations of hardware, we actually announced like with Rubin, this new accelerator that is pre-fill specific. It's called Rubin CPX.”
Insight
Local document pre-fill with global sequence decode solves transformer quadratic scaling
“If pre-fill becomes local and decode is, is still global, you solve that pre-fill quadratic scaling problem because you have a bunch of like small chunks that you pre-fill independently.”
Assertion Supported
NVIDIA chip design begins three to five years before market release
“The design process starts like- Exactly. ...three to five years before the chip gets to the market.”
Assertion Supported
Scaling beyond eight NVIDIA H100 GPUs requires inter-GPU communication over InfiniBand
“The maximum NVLink domain, domain for H 100 for most DGX H 100 Is eight GPUs, right? So if you scaled up past that, You're gonna have to figure out ways to handle the fact that now for the GPUs to communicate, you have to do it over InfiniBand, which is still …”
Insight
LLM prefill remains compute-bound while decoding phases are strictly memory-bound
“So prefill typically, and this changes as model architecture changes, prefill is right now compute bound. Most of the time. If the sequence is sufficiently long, it's compute bound on the decode side because you're doing a full pass over all the weights and th…”
Assertion Supported
NVIDIA Dynamo dynamically sizes and schedules Kubernetes prefill and decode workers
“Dynamo has a set of components that A, tell you how to scale. It tells you how many pre-fill workers and decoded workers it thinks you should have. And also provides a scheduling API for Kubernetes that allows you to actually represent and affect this scheduli…”
Assertion Supported
DeepSeek 128k context fits in 8GB KV cache versus Llama's 80GB
“For context like the total I think the total context length of DeepSeq is a 128,000 tokens, or it might be 256,000 with rope extension. That entire context, I think it's a 128,000, fits into eight gigabytes. And previously context, like I think the Lama four o…”
Assertion Not checkable as stated
NVIDIA's build.nvidia.com was internally the company's largest inference deployment
“At one point, there's a website called build.nvd.com, and also for us, inference.nvd.com, that is, allows people to try models. It gives an API service, you can call the model with like a REST API, and, you know, you get a response. I ran the model site for th…”
