The Ledger
Every statement that passed quotation and attribution checks. Mix any filter with any other: certainty 1/5, debate potential 5/5, or both at once.
why aren't all 5 resolved? a statement only gets an assessment when the public record can support or contradict it. opinions and what-ifs never can, and 0 checkable ones are still open, waiting for their date. predictions held up or didn't; assertions are supported or contradicted. on every card: ▮▮▮▮▮ certainty · ▮▮▮▮▮ debate potential. speakers are clickable
Younis: Scale AI is fundamentally a data labeling services company
“You know, scale was, is more of a services company, data labeling company, fundamentally.”
Lambert: Scale AI has historically struggled to retain technical ML talent
“I think they've historically had trouble keeping, like, technical ML talent, but they've started a new research lab, so that should help.”
Lambert: Annotator disagreement data in RLHF is not used for anything
“I don't really think this disagreement data is used for anything, but it's good to know, like, what the distribution of prompts is, who's doing it, how many samples you have, controlling the workforce.”
Mohan: Scale AI increased revenue 10x with each major pivot
“Effectively, it starts out as, like, you know, human task interface, then an AV labeling company, then a cataloging company, then Now a generative AI labeling company, and every time the revenue of the company kind of goes up by a factor of 10, even though the…”
Conover: Repeatable AI annotation pipelines require unsexy people management
“It's one thing to do like a single monolithic push to create a,
Training data set like that, or an evaluation corpus, but I think it's another to have a repeatable process, and a lot of that, I think, realistically is pretty unsexy, like, people management wor…”