The Ledger
Every statement that passed quotation and attribution checks. Mix any filter with any other: certainty 1/5, debate potential 5/5, or both at once.
why aren't all 9 resolved? a statement only gets an assessment when the public record can support or contradict it. opinions and what-ifs never can, and 0 checkable ones are still open, waiting for their date. predictions held up or didn't; assertions are supported or contradicted. on every card: ▮▮▮▮▮ certainty · ▮▮▮▮▮ debate potential. speakers are clickable
Corbitt: GRPO is likely a dead end due to parallel rollout constraints
“The big downside, the huge downside of GRPO, and I think actually the reason why GRPO actually is likely to be a dead end, and we probably will not be continue using it indefinitely. The fact that you need to have these parallel rollouts in order to train on i…”
Corbitt: Prompt optimization methods like JEPA failed OpenPipe's agent benchmarks
“It didn't work on the problems we tried it on. It just didn't. It got like a minor boost over the sort of like more naive prompt we had and was just like, it was like, okay, Just kind of like our naive prompt with our model gets maybe like 50% on this benchmar…”
Corbitt: OpenPipe Beat Frontier Models Using a Qwen 32B Judge
“One of the results we published was we used Quen 2.5 14 B as the model we're training, and as the judge we used Quen 2.5 32 B, which is, like, Not, I mean, it's fine, but it's like not a, it's much worse than any frontier model. Right. And even with that combi…”
Corbitt: Fine-tuning offers poor ROI for 90% of unconstrained use cases
“I would say for 90% of use cases where you aren't forced to a smaller model, then it's still not a good ROI, and you probably shouldn't invest in it today.”
Corbitt: GPU Cloud Fine-Tuning Offerings Failed Due to Poor Usability
“I did not see the competition ever really materialize from the Neo clouds, from the GPU providers. Everybody had an offering in fine tuning. When we were talking to customers, nobody used them because they just were really hard to use.”
Corbitt: Building RL training environments is currently a services-heavy business
“It seems to me like that definitely is a services heavy business at the moment as it, as it's presently constituted.”
Corbitt: Realistic sandbox environments almost universally do not exist in enterprises
“When we talk to enterprises almost universally, that's like not something that really exists. So there are some startups, like there's some companies we've talked to that do have it and we can just like use that, but it's a very, very small number that, that a…”
Corbitt: OpenPipe Hit $1M ARR Within Eight Months of Launch
“And so we got our first three customers after launching probably within a month, and we were doing significant revenue. Over the next six months, we actually got to a million in ARR over about a eight month period following that launch.”
Corbitt: Weights & Biases founders drove CoreWeave's OpenPipe acquisition
“So that was driven by actually mostly the weights and biases founding team. Lucas and Sean, particularly. So they, had recently been acquired by CoreWeave and CoreWeave was looking to continue growing up the stack. And so, yeah, they approached me and were lik…”