Jul 2, 2025 · 1h 18m · latent-space
Information Theory for Language Models: Jack Morris
0:00 / 0:00
▶ Watch on YouTube →
gold bands on the timeline = statements, start to end. Hover to read, click to jump. CC turns on captions
Cornell Tech PhD researcher Jack Morris joins swyx to unpack the information-theoretic foundations of language models, exploring vector embedding privacy, empirical parameter storage limits, and how academic researchers can drive high-impact discoveries amidst rapid industry scaling.
How this conversation actually went
Every chapter scored 0–10 on four independent dynamics. Hover any point for the reasoning behind the score. How this is scored →
The hosts as informed peer 6.1 Guest teaching 4.3 Guest disagreement 2.1 The hosts pushing back 2.6
speaking balance: gold is the hosts, purple is the guest (3 minute bins)
Sharpest disagreement ▶ 1:10:35 Asserting transformers were not strictly necessary
Jack takes a provocative devil's advocate stance, arguing that web-scale pre-training on sophisticated RNNs could have produced ChatGPT without needing transformers.
Hardest push from the hosts ▶ 1:12:18 Challenging the dataset-only thesis with efficiency multipliersswyx challenges Jack's pure dataset thesis by pointing out that algorithmic and optimizer improvements act as major multipliers equivalent to orders of magnitude more training data.
Biggest teaching moment ▶ 15:25 Defining extractable V-information under computation limitsJack provides a clear theoretical distinction showing why two files with identical Shannon bit-entropy differ vastly in usable, extractable information.
The host holds their own ▶ 12:29 Detailed analysis of Modular Mojo and hardware compilationswyx demonstrates deep technical and industry insight by breaking down Chris Lattner's Mojo compiler approach to CUDA replacement and fast kernel experimentation.
the scores for every segment, with the reasoning behind each
| Chapter | Topic | The hosts as informed peer | Guest teaching | Guest disagreement | The hosts pushing back | Why |
|---|---|---|---|---|---|---|
| Navigating AI Research Meta and Scaling to 8B Models | 5 | 4 | 2 | 2 | swyx demonstrates solid familiarity with current AI market dynamics and PhD startup valuations. Jack explains the emergence gap between 100M BERT-scale models and 8B parameter models in academic research. | |
| Systems Engineering, GPU Training, and Modern Toolchains | 8 | 2 | 2 | 3 | swyx shares detailed industry advice regarding HPC resources, PyTorch, DeepSpeed, and Modular Mojo. Jack clarifies his public stance on learning CUDA versus higher-level execution frameworks like vLLM and SGLang. | |
| Information Theory Foundations and Usable Information in LLMs | 7 | 6 | 1 | 2 | Jack introduces V-information and the idea of measuring usable information under computational constraints. swyx engages deeply by comparing this to Kolmogorov complexity and Shannon information limits. | |
| Inverting Embeddings and Vector Database Privacy Risks | 6 | 5 | 2 | 2 | Jack details his research on inverting text embeddings and its privacy implications for commercial vector databases. swyx navigates the slides and connects the findings to real-world context leakage and security attacks. | |
| Universal Geometry of Embeddings and the Platonic Hypothesis | 5 | 4 | 2 | 2 | Jack connects embedding geometry to the Platonic Representation Hypothesis and discusses model capacity plateaus. swyx raises questions about context length degradation and mathematical matrix bounds. | |
| Cognitive Core Concept and Emergence of Reasoning | 7 | 3 | 2 | 3 | swyx introduces Karpathy's concept of the cognitive core and contrasts human biological efficiency with dense LLMs. Jack agrees in spirit but notes the technical difficulty of separating reasoning from factual memorization. | |
| Multimodal Adapters and Theoretical Limits of Small Models | 6 | 4 | 2 | 2 | Jack explains how CycleGAN inspired unaligned latent space mapping across distinct model architectures. swyx applies this insight to modular multimodal adapters like Gemma 3n. | |
| Quantifying Model Capacity and Memorization Bounds | 6 | 5 | 2 | 4 | Jack presents his finding that 32-bit parameters only store roughly 3.6 to 3.9 bits of memorized information. swyx questions whether optimizing memorization capacity conflicts with the primary goal of generalization. | |
| Approximating Training Data Directly from Model Weights | 5 | 5 | 1 | 2 | Jack explains his method for approximating proprietary fine-tuning data from the weight delta between base and instruct checkpoints. swyx highlights the novelty of using synthetic checkpoints for data reconstruction. | |
| Kuhnian Paradigm Shifts and Dataset-Driven AI Progress | 6 | 5 | 5 | 4 | Jack presents a contrarian Kuhnian thesis arguing that AI progress is almost entirely dataset-driven rather than architectural. swyx offers pushback, arguing that compute and optimization efficiency act as substantial data multipliers. |
Statements from this episode (23)
Jack Morris: Most AI research was previously open, but is now closed
“Most stuff was open. Now most stuff is not open.”
Jack Morris: Fundamental AI science shifted to companies due to academic compute limits
“That's when I think things really started to change in terms of the types of questions you wanted to ask can't always be answered with academic resources. So a lot of the like fundamental kind of like boundary pushing and AI science moved into companies.”
Morris: Re-implementing new paradigm shifts quickly is the best AI grad strategy
“Honestly, if I were to give advice to a younger grad student, I think the way to do it would be literally just like sit and wait until the next kind of paradigm shift and then just immediately start working as fast as you can to like re-implement it. Like, I d…”
Swyx: Stanford RL students founded pre-product startup with $500M valuation
“I just saw this morning that one of the recent Stanford grad students who worked on RL, they just started their company and they're worth five hundred million. It's like absolutely bonkers right now. It's just like no product, just three dudes, you know, sitti…”
Morris: Two years of academic AI research on small models was inconsequential
“There was like kind of two years where everyone in academia was working on like smaller models and none of it really mattered.”
Morris: Top AI graduate programs do not teach multi-node distributed training
“Oh, to be clear, they don't teach you anything, like anything, like if you see a paper coming out from even, you know, Stanford, they're probably the best school in AI if you had to choose. And it's not like they're learning how to do like multi-node distribut…”
Morris: Deep understanding of GPU architecture makes engineers exceptionally hireable
“That said, if you do it, you're, you've gotta be one of the most hireable people in the world. Like if you like, Really deeply understand the architecture of the new GPUs coming out and how to control it. You're in a very small handful of people and like every…”
Morris: vLLM and SGLang are here to stay and will grow more complex
“I also think, ah, VLLM and SGLang seem, like, really good and important and here to stay. Like, they'll probably just get larger and more complex to accommodate future systems”
Morris: Information in AI should be measured under computational constraints
“There's this theoretical framework proposed in this paper, which is a theory of usable information under computational constraints from 20, 20. It really doesn't have that much press. They're not aren't as many citations as you would think, but I think it's a …”
Morris: Machine learning lacks a fundamental unit of deep learning information
“I don't think we know what a bit is yet in terms of like deep learning models.”
Morris: New embedding inversion model exactly recovers 90% of source text
“Like we ended up building a system that can do this quite well, like taking an embedding and I think our highlight number is like at a certain length, like a long sentence length, we can get 90% of the text back exactly.”
Morris: Embedding inversion requires access to and repeated queries of the encoder
“Like none of the vector to text stuff works unless you have this assumption of like knowing the encoder and also being able to make a lot of queries to it.”
Morris: Language models hit a hard memorization plateau regardless of dataset scaling
“Like, no matter how you scale the training size, you hit this like perfect, perfect ish plateau in auto memorization, which we call the model capacity.”
Morris: No Evidence We Can Build Pure Reasoning Models Without World Knowledge
“I don't think we have a lot of evidence that we can build a system like this that like is really, really good at reasoning, but really dumb about the world. Like, I don't know if we have the tools.”
Morris: CycleGAN mapping aligns disparate model embeddings without paired data
“We took it and we applied it to model embeddings where instead of zebras and horses, we have like BERT embeddings and GPT embeddings, or like two completely different models with different architectures. So I think these are GTR, which is a T five based retrie…”
Morris: Small models should be defined as runnable on a single GPU
“I think that we should establish the definition of small model as being a model that a grad student can inference at reasonable time on a single GPU. Which is probably like seven B maybe. I don't think 27 is small under any reasonable.”
Morris: 32-bit transformer models store only 3.6 to 3.9 bits per parameter
“Transformers that are trained in 32 bit precision, we approximate can store about 3.6 bits of information to maybe 3.9 bits somewhere in there per parameter.”
Morris: LLaMA architecture will likely store more information per parameter than GPT
“Maybe even if we tested this with LALAMA architecture, like, there's sort of like a GPT++ architecture, like, I would guess that can store better data just because the kind of numerical flow is a little bit better, the nonlinearities are maybe, like, A little …”
Morris: Open model creators do not use differential privacy or anonymization
“I would be extremely surprised if they do any type of like private training. Like there are these mechanisms for doing like differentially private language model training, or even just anonymization in the pre-training pipeline. I bet they don't do any of that…”
Morris: Weight deltas can reconstruct a competitor's proprietary fine-tuning dataset
“There's some tricks to it, but it's basically just like gradient based selection based on this weight difference. And it seems to be okay. Like it can get us pretty good training data. So I guess if you actually wanted to use this, it would be like your compet…”
Morris: AI paradigm shifts are driven by novel datasets, not architectures
“I think, like, all of the things that I would consider paradigm shifts in the Kuhnian sense came from a new technique, but trained on new data, and I think the new data is super, super important”
Morris: ChatGPT could likely have been built using RNNs instead of Transformers
“And I think like, we honestly probably could have gotten this with RNNs. I know like the scaling laws paper shows that RNNs have worse curves for scaling, but probably people would have been like, I bet you could have built chat GPT with a very sophisticated R…”