Jan 29, 2026 · 1h 8m · mad
State of LLMs 2026: RLVR, GRPO, Inference Scaling — Sebastian Raschka
0:00 / 0:00
▶ Watch on YouTube →
gold bands on the timeline = statements, start to end. Hover to read, click to jump. CC turns on captions
In this episode of The MAD Podcast, AI researcher Sebastian Raschka joins Matt Turck to discuss the state of large language models in 2026. They cover the shift from pre-training to post-training methods like RLVR and GRPO, inference-time scaling, Transformer optimizations, benchmark saturation, and enterprise AI strategies.
How this conversation actually went
Every chapter scored 0–10 on four independent dynamics. Hover any point for the reasoning behind the score. Matt holds 11.2% of the talking time here. How this is scored →
Matt as informed peer 4.3 Guest teaching 4.9 Guest disagreement 1.3 Matt pushing back 1.3
speaking balance: gold is Matt, purple is the guest (3 minute bins)
Sharpest disagreement ▶ 30:46 Guest pushes back on RL complexity premise
When the host implies RLVR is finicky and complex to scale, the guest directly counters that it is not super complicated, noting he implemented GRPO in a 39-page notebook and that it is ten times cheaper than pre-training.
Hardest push from Matt ▶ 17:26 Host challenges guest on pre-training viabilityThe host explicitly interrupts the guest's focus on post-training to press whether he truly believes pre-training is dead or if there is still room for progress.
Biggest teaching moment ▶ 22:00 Guest explains GRPO model memory reductionsThe guest breaks down the exact technical difference between PPO and GRPO, explaining how replacing separate reward and value models reduces memory overhead from three model copies down to one.
Matt holds his own ▶ 50:11 Host synthesizes private data moats in enterprise LLMsThe host demonstrates deep domain insight by tying together generalist model parity, private data moats, and the strategic rationale for enterprise in-house training.
the scores for every segment, with the reasoning behind each
| Chapter | Topic | Matt as informed peer | Guest teaching | Guest disagreement | Matt pushing back | Why |
|---|---|---|---|---|---|---|
| Evaluating the Transformer Architecture and Modern Alternatives | 5 | 4 | 1 | 1 | The host opens with an informed question about whether the transformer architecture's days are numbered given developments in SSMs and diffusion models. The guest explains that transformers remain state-of-the-art while modern variants mostly focus on cost efficiency. | |
| World Models and Internal State Prediction for Code Generation | 4 | 5 | 1 | 1 | The host asks targeted questions about world models and small recursive models. The guest educates the host on internal state prediction for code LLMs and how recursive models target specific logic benchmarks like ARC. | |
| Text Diffusion Models versus Auto-Regressive Transformers | 5 | 5 | 1 | 1 | The host demonstrates strong knowledge by bringing up DeepMind's Gemini Diffusion announcement. The guest explains the architectural differences between sequential auto-regressive generation and parallel diffusion denoising. | |
| LLM Architectural Optimizations, MoE Adoption, and Pre-Training Limits | 5 | 5 | 2 | 2 | The host pushes the guest on whether architecture progress is just superficial tuning and checks if pre-training is dead. The guest re-frames pre-training as 'boring' rather than dead and highlights MoE adoption trends. | |
| Post-Training Evolution: RLVR, GRPO, and Reasoning Unlocking | 5 | 6 | 1 | 1 | The host cites the guest's technical blog post to prompt an explanation of RLVR and GRPO. The guest provides an in-depth breakdown of how GRPO eliminates memory overhead by getting rid of separate reward and value models. | |
| Process Reward Models (PRMs) and Verifiable Rewards Beyond Code | 5 | 5 | 1 | 1 | The host prompts the guest regarding why Process Reward Models (PRMs) haven't been successful yet. The guest outlines reward hacking risks in PRMs and explains multi-model grader setups. | |
| Cost and Complexity of Scaling RLVR vs. Pre-Training | 4 | 6 | 2 | 2 | The host suggests RL scaling is notoriously finicky and complex based on the multi-model architecture. The guest counters that it is not overly complicated, citing his own 39-page Jupyter notebook implementation and cost differences. | |
| Meta-Lessons on AI Progress and Infrastructure Specialization | 4 | 4 | 1 | 1 | The host asks about the meta-lessons from 2025 AI progress. The guest explains that progress is an aggregation of small engineering micro-optimizations across specialized sub-teams rather than a single magic bullet. | |
| "Bench-Maxing", Leaderboard Gaming, and Evaluation Reality | 4 | 5 | 2 | 2 | The host brings up 'bench-maxing' and asks if corporate economic motives drive leaderboard gaming. The guest gives a nuanced breakdown of how evaluation style bias distorts leaderboards. | |
| Inference-Time Scaling and Recursive Prompt Chunking | 4 | 5 | 1 | 1 | The host steers the discussion toward non-post-training factors like inference scaling. The guest explains parallel sampling, vote aggregation, and prompt chunking. | |
| System Engineering, Tool Calling, and Local Execution Security | 3 | 5 | 1 | 1 | The host listens as the guest explains how tool calling and prompt wrappers create a gap between raw open-weight models and cloud LLM platforms. | |
| Generalist Model Parity and In-House Enterprise LLM Training | 5 | 5 | 2 | 2 | The host links generalist model parity with private enterprise data as a moat, asking if companies are resuming in-house training. The guest clarifies that large enterprises are indeed building data-center scale models in-house. | |
| Evaluating Continual Learning and Iterative Model Updates | 4 | 5 | 1 | 1 | The host brings up NeurIPS buzz around continual learning and asks about the guest's skeptical timeline. The guest explains why catastrophic forgetting and hosted API paradigms delay implementation. | |
| Sebastian Raschka's Code-First Research and Writing Methodology | 3 | 4 | 1 | 1 | The host asks the guest about his research methodology, book authoring, and use of LLMs. The guest describes his code-first approach of implementing architectures from scratch to verify paper claims. |
Statements from this episode (25)
Transformer remains state of the art for LLM performance
“I would say right now, yes, because it's still the state of the art. So there is nothing really better in terms of state of the art performance, getting better quality results.”
DeepSeek's architecture is still built on a GPT-2 scaffold
“And you can actually, in fact, Take a GPT one or two model and with a few, I mean, few lines of code almost, you can transform it into the latest let's say deep seek version, 3.2 architecture. It's not like a big leap. It's still the same as scaffold.”
Predicting internal states might push state-of-the-art for code LLMs
“And I do think that is something that is maybe more expensive to do, but it is also something that might push the state of the art a little bit.”
Hierarchical Reasoning Model matches larger LLMs on ARC using Transformer architecture
“Hierarchical reasoning model, it became like popular because it performed relatively well on that benchmark compared to very expensive models like Gemini, Chachupiti, and so forth. And it is a transformer architecture.”
Enterprises should transition from generalist LLMs to cheaper task-specific models
“If you have a business problem, you are maybe manufacturing something, maybe you can start with a generalist model, but then once you know exactly what the task is and you want to hone in on it, maybe it makes sense to replace that expensive thing by something…”
Text diffusion models will not replace autoregressive Transformers at state-of-the-art
“So it is a interesting direction to go into these diffusion, diffusion models as alternative to the auto regressive transformers, but it is not I would say the replacement at the state of the art.”
Major AI firm will launch a frontier text diffusion model in 2026
“I think one company will launch a big diffusion model this year.”
Future LLMs will prioritize architectural efficiency over larger model sizes
“I wouldn't expect bigger architectures. I would expect a more efficient architectures tweaks getting, The same modeling performance for less compute”
Pre-training is no longer where the low-hanging AI gains lie
“Pre-training is not dead, but pre-training is boring. So it's not where the low hanging fruit is anymore.”
50 RLVR steps boosted Qwen 3 MATH-500 score from 15% to 50%
“I took the Quinn three model as part of my book, the reasoning from scratch book. And I trained it just for 50 steps with RLVR, and it goes from 15%, so one five percent accuracy on math 500 to 50% on math 500.”
RLVR unlocks pre-training knowledge rather than teaching LLMs new math
“The knowledge is already there in the pre-training, and this just unlocks it. It's just like a step that maybe shows the model how to use its own knowledge, basically.”
Process Reward Models will eventually become standard in LLM post-training
“I think it is promising and we will see it working at some point. I think it's just like right now it's still Tricky to make it work, but I am quite sure we'll see it as part of the standard repertoire at some point.”
Bigger LLM gains will come from multi-model process refinement, not scaling
“That's where you make the bigger gains rather than scaling the model size. I think that's one of those things where you will see more progress coming from.”
DeepSeek-R1 cost roughly $300,000 to train, 10x cheaper than DeepSeek-V3
“Deep seek version three, they had like a five million dollar price tag on that given the, I think two dollars per GPU, they assume whether that's a correct assumption or not it's a different question, but if you compare it relative to the cost of R one, I thin…”
Original GRPO algorithm is flaky but stabilizes with practical engineering tricks
“Vanilla GRP or the original algorithm, it is pretty flaky. Like where it is, you have to babysit it. Over the course of the year, many people had these tips and tricks where some people were saying, remove the KL divergence term. Like if you just drop it for m…”
RLVR and GRPO will stabilize into canonical algorithms similar to AdamW
“It's similar to, you know, optimizers with Adam. So Adam is, I mean, right now there's Adam W. There was SGD and all the other RMS prop and how they were called. And they kind of all converge to Adam W by adding more and more tricks. And I think that's the sam…”
The AI industry is rapidly running out of challenging evaluation benchmarks
“The only thing is we are running out of is really benchmarks. So the improvement on benchmarks, it's kind of like harder to measure.”
AI evaluation will shift from single-shot answers to multi-step agentic execution
“Maybe it's not the one shot problem anymore where it's not really answering knowledge question. That's not really solving math problems in, in one iteration of the benchmark. It is maybe more like the agentic cycle, like where you have like a more like a objec…”
Current AI model leaderboards reward response style over actual factual correctness
“It rewards the style more, more than the correctness because there is no correctness check.”
Tool calling reduces LLM hallucinations by outsourcing memory retrieval tasks
“And that is very, very powerful because I think this is one of the
Ways you can mitigate not totally mitigate, but let's say reduce hallucinations because then the LM suddenly doesn't have to remember everything anymore.”
GPT OSS benchmarks demonstrate 1.2x capability jump when tool calling is enabled
“And also you can actually go to the GPT OSS release block, and they did have benchmarks to show how the performance on the benchmarks is with the same model with tool called enabled and disabled. And you can actually see there is, I mean, it's not like two tim…”
Leading generalist models like ChatGPT, Gemini, and Claude show functional parity
“Like if you use or compare ChatGPT, Gemini Claude, Grock. I think they are all pretty much on the same level. Like, and I think that's because they're trying to do everything. Like the generalist models for a general person to do a lot of things. I mean, Claud…”
Large enterprises are secretly hiring teams to train ChatGPT-scale LLMs in-house
“I know for a fact that big companies are training now LLMs in-house. Really, like, big companies who have the financial means to train chat to be like model are hiring people who train LLMs.”
Self-improving AI and continual learning agents will not be feasible in 2026
“If you have an LLM that self improves or like an agent that does something fails and learns, I don't think anything like that is feasible this year.”
Raschka does not use LLMs to write his books or blog posts
“For blog writing on book writing, not so much because honestly, I, for fun, I tried it out. It's just, it generates okay text, but it's, I don't know, it does not I can ask it to generate text like me, but it's almost like, then I don't like it and I end up ed…”