The Ledger
Every statement that passed quotation and attribution checks. Mix any filter with any other: certainty 1/5, debate potential 5/5, or both at once.
why aren't all 15 resolved? a statement only gets an assessment when the public record can support or contradict it. opinions and what-ifs never can, and 0 checkable ones are still open, waiting for their date. predictions held up or didn't; assertions are supported or contradicted. on every card: ▮▮▮▮▮ certainty · ▮▮▮▮▮ debate potential. speakers are clickable
Chip Huyen argues post-training is what differentiates frontier AI models
“So, so I do think that post-training is what makes this, like, really big lab models are, like, different.”
Evaluation is the single biggest bottleneck holding back enterprise AI adoption
“So, so I do think that evaluation is the biggest bottleneck for AI adoptions, because unless, like, if we can, like, if we can, like, develop a more reliable way to evaluate the application, that application is not going to get adopted. Like, or maybe, maybe i…”
Chip Huyen argues human-generated plans are poor training data for AI agents
“When we ask humans to generate like what they consider the best plan for an actions, for a task, it's actually like not quite the best plan for AI, because what is what is easy or efficient for humans is not the same as easy and efficient for AI, right?”
Huyen: Legacy systems prevent US companies from matching Chinese online learning
“So a lot of American, American internet companies are like a lot older than the average, like the new, like Chinese internet company. So it means it's like American internet companies have legacy systems that you from like, 20 years ago. And just have to build…”
Chip Huyen: AI becomes harder to evaluate as intelligence increases
“As a more intelligent AI becomes like the harder it is to evaluate it.”
Chip Huyen: Lack of labeled data requirements makes language modeling uniquely scalable
“You don't need to curate, like, labels, like, reference data, so that, that you can use a train models that make language modeling, like, so, so much easier to scale than other types of tasks.”
Chip Huyen: Machine learning projects should start with problems, not models
“So so the main idea is you go backward from the problems. So I think a lot of approach machine is saying it's like, you start with the solutions and it tries to have like five problems when machine can I be applied. So, and it's like, so tend to be like, oh, H…”
Chip Huyen: MLOps startups ignore real-time learning for low-hanging fruit
“We don't have tools for it yet. And I see very, very, very little tools focusing on it because most people are like targeting on focusing on low hanging fruit.”
Chip Huyen: AI engineering requires developers to have stronger product sense
“It requires engineers to have a much, much better product sense.”
Code generation leads generative AI adoption because it is easily evaluated
“For JDIF AI, like, one of the most common JDIF AI use cases today is coding. Okay, so there are many reasons why coding is popular, and I said, like, one of the reasons is that it's very, it's a lot easier to evaluate coding than, like, other, because, like, h…”
Chip Huyen: AI evaluation metrics must be derived backward from business use cases
“For applications it's really, really important to understand the use cases well, so they can design like the set of metrics and then you can walk backward from that and map it to like the model metrics.”
Chip Huyen says prompt engineering should treat each prompt as an experiment
“I do think that it can be very systematic. You know, you should need to make it very systematic. So if you consider like each prompt is experiment, it should be watching like versions of prompt. It should be able to systematically track your progress with diff…”
Open-source AI developers lack usage visibility to ensure model safety
“Open source model developers try their best to make the models, like, safe as well, right? But they also have less visibilities into how the open source models are being used. So which gives them like less information for them to like make the model safe.”
Chip Huyen says sampling improves AI performance without model retraining
“It's a cheap way to improve some model, like, so, so, so, so application's performance without to retreat, you're going to have to retreat in the model.”
Defensive prompt engineering grows critical as AI executes complex tasks
“So I do think that's a topic is getting increasingly important, especially as AI is being like first it is being used for more like high stack tasks, right? And a more complex task. And the second is, is like, it's now AI has increasing access to more tools an…”