The Ledger, every show
Every statement that passed quotation and attribution checks, across all 44 shows. Pick shows below, then mix any filter with any other.
shows 
every show 44 of 44
Albrecht: Benchmark performance differences vanish once ambiguous questions are cleaned
“The main takeaway from any of the, like, actual performance is like, once you fix these ambiguous examples, a lot of these benchmarks are really saturated. Like, I think it's important to look at like, you know, like when you're talking about performance on NL…”
Albrecht: AgentBench paper's appendix examples are actually incorrect solutions
“Like we were looking at the agent bench paper, I think just last week for our paper club. And one of the things that we noticed is that actually like both of the examples in the appendix that are given as like traces where it got it right. This is actually not…”
Albrecht: Imbue has sufficient compute to train state-of-the-art sized AI models
“We have enough compute to be able to train models that are as large as the largest models have been trained today to date.”
Albrecht: Dynamic data mix schedules during LLM training yield negligible gains
“We did some experiments and we've actually talked to a bunch of researchers who were doing work here as well and looking at kind of their experiments on this. And we were originally pretty hopeful because it sounds like something that should work and make sens…”
Albrecht: Imbue will hire fewer recruiting coordinators as internal agents handle scheduling
“Now, you know, I think probably within the next year, we'll probably, you know, not be hiring as many recruiting coordinators because, oh, we're going to do some of the scheduling with the agent that we've built, right?”
Albrecht: Imbue's GPU cluster failure rate is well below industry 3% benchmark
“The number that we've heard from other people is like they're having about three percent. I don't think we're experiencing failure rates that are that high. I think ours is actually quite a bit lower than that, probably because we've taken the time to like dig…”
Albrecht: Narrow AI agents for email triage will work by late 2024
“Yeah, I think a year from now we're going to start to see some of these use cases actually work that today you could you can write these like we have the capabilities you can make some kind of agent to triage your email or to do scheduling or many of these wor…”
Albrecht: Users will create bespoke agents via natural language within five years
“And I think five years from now we're going to have something where it's not just you know, okay we have a scheduling bot we have this other thing but we really have these more general more robust systems where Each of us can individually say, like, I want a t…”
Albrecht: AI agents will dramatically improve codebase quality across the industry
“I think there'll just be a huge flourishing of much higher quality, better software as a result, not just more software, but just taking the existing software and making it so much better, which will make it so much nicer and more fun to interact with as progr…”
Albrecht: 4K GPU clusters require 3-tier networking versus standard 1K 2-tier setups
“The normal, the like vanilla setup or, you know, these large clusters as vanilla as it can be is what's normally like a 127 node cluster. So closer to like 10, 24 GPUs instead of 4000. Here we have a larger cluster. As you start to get into the larger clusters…”