Everything Liam Fedus said on any show that made the record, most notable first. Each card names its show and opens the statement there.
Fedus: Physical sciences and engineering will follow machine learning scaling laws
“And I think the physical sciences, physical engineering, Will have a very similar property where we establish these scaling properties and Bring that mindset.”
Fedus: Intelligence is not scalar; AI shows extreme, non-intuitive spikiness
“I think one fallacy is thinking about intelligence as a scalar. We've consistently seen these systems have a very odd spikiness, and it's actually possible to architect a system that is world-class on some math domain, but then you could do some perturbations …”
Fedus: Self-improving software AI will not automatically generalize to domains like biology
“So rolling forward that software engineering self-improvement, I think you're going to have a system that can write complete repositories, identify bugs, refactor code, But it doesn't suddenly understand biology. Right? It's just like there's a domain gap ther…”
Fedus: Recursive self-improvement is happening in software and entering AI research
“So, In that domain, I think it's happening now-ish. And I think we'll see the same thing too for AI research. That's a slower outer loop because now the experiment isn't just checking some unit tests passing, but it's checking what was the scaling property? Di…”
Fedus: Pre-training on domain data outperforms retrieval-augmented generation
“However, as we've seen with things like ChatGPT and other things, when you pre-train on the data, when you actually encode the knowledge into the weights, it's not just a retrieval system, you have a richer, deeper understanding of the material.”
Fedus: John Schulman steered OpenAI toward a general chatbot over narrow tools
“And we're all spitballing ideas, like writing bot coding bot. You know, very natural at the time. Some of our least interesting ideas were a meeting bot, so it would just sit in a Google Meet, take notes, and then send out, like, to-dos after. But John Schulma…”
Fedus: AI must connect to the physical world to accelerate science
“The opinion that I and others held as periodic was, you're not going to see the same kind of acceleration in science and technology unless you start connecting these things to the physical world.”
Fedus: Training materials AI on academic literature fails to find ground truth
“One of the engineers on our team was looking at a reported material property and It was just sort of extracted values from literature, and it was really interesting to see the reported value spanned many orders of magnitude. And so you train an ML system on th…”
Fedus: AI Matter Synthesis Will Profoundly Impact Semiconductors, Aerospace, and Energy
“As you're pointing out, you're going from systems that aren't just writing essays, not just writing software, but to literally generating matter. And I think it has pretty profound implications to semiconductors, airspace, energy.”
Fedus: Physics and chemistry demonstrate scaling laws similar to AI
“On the material science side, we're seeing scaling laws within physics, within chemistry both with respect to simulations, with respect to experiment, and it's like the same kind of principles at play and ML.”
Fedus: Physics provides ideal verifiable reward functions for AI
“Physics is very verifiable. It's a great reward function, fairly fast iteration loop. You have simulators for large classes of physical systems.”
Fedus: Periodic Labs uses physical experiments as RL reward functions
“And what we're doing, and by having the lab, is we create a physically grounded reward function. That becomes the basis on which we're optimizing against. And so, If a simulator has some deficiencies or some issues, we always error correct, because for us, the…”
Fedus: Early ChatGPT was mathematically weak due to friendliness rewards
“The reward functions that we were using originally couldn't determine whether you were mathematically correct or not. So early versions of Chachapiti were mathematically not particularly strong, and it sort of results from the reward function. What did you opt…”
Fedus: AI physics requires generating new experimental data, not web scrapes
“The technology that we think is necessary to do it has really just emerged in the last couple of years, and this data Isn't like on a Reddit forum or something like you need to actually go produce experimental data, simulation data.”
Fedus: Post-Higgs experimental bottlenecks drove physicists to transition into AI
“After the discovery of the Higgs I think a lot of high energy physicists were sort of looking for what's next. Ultimately it becomes bottlenecked on the new apparatus for, you know, pushing the next energy frontier. And I think a lot of physicists were looking…”
Fedus: Reasoning and coding agents connect software AI to physical domains
“And I think those were foundational technologies necessary To then connect these systems to the physical world. Like it was just not impossible, not possible with like the AI technology of.”
Fedus: Quantum AI models do not generalize across abstraction levels to fluids
“But like if you produce a system that has modeled quantum mechanical objects really accurately, it's not really helping much on like, you know, fluid dynamics or, you know, like another kind of like level of abstraction.”
Fedus: Periodic Labs launches as a software intelligence layer, not discovery model
“We're thinking about us ourselves as an intelligence layer for these companies. So you can think about system of record control plane for different Experiments and getting to solutions. But like you're saying, there is a very interesting aspect of some breakth…”
Fedus: Compute costs outweigh physical infrastructure in materials AI labs
“What's interesting is just the compute costs relative to physical infrastructure is actually surprising where, you know, so much money is spent on the compute that the physical infrastructure sometimes is actually lower, but, you know, has very large lead time…”
Fedus: Hybrid automated lab systems already produce large amounts of reliable data
“Already the reliability of the sort of like hybrid systems is sufficient to produce Huge amounts of reliable data”
Fedus: Periodic Labs uses off-the-shelf robotics instead of building custom hardware
“Right now we're using almost like more like off the shelf robotics. It's like very simple, very commoditized not doing like a huge amount of innovation on, on that front.”
Fedus: AI science requires real-world experimental feedback loops
“Ultimately science is driven against experiment in the real world. And so that's what we're doing with periodic labs. We're taking these precursor technologies and we're saying, okay, if you care about advancing science, we need to have experiment in the loop.”
Fedus: Unpublished negative scientific results are uniquely valuable for AI
“Then another point is, it's very uncommon to publish negative results. All of the results are basically positive, and a valid negative result is very valuable.”
Fedus: Periodic Labs will build AI co-pilots for space and defense
“Basically co-pilots for engineers, researchers in advanced industries. So maybe perhaps just being in Silicon Valley, we, you know, we really think about like computer oriented work. Everything is digital. Everything is bits, but there's so many industries. Li…”