The Ledger
Every statement that passed quotation and attribution checks. Mix any filter with any other: certainty 1/5, debate potential 5/5, or both at once.
why aren't all 8 resolved? a statement only gets an assessment when the public record can support or contradict it. opinions and what-ifs never can, and 0 checkable ones are still open, waiting for their date. predictions held up or didn't; assertions are supported or contradicted. on every card: ▮▮▮▮▮ certainty · ▮▮▮▮▮ debate potential. speakers are clickable
Attackers can poison the LAION dataset simply by purchasing expired domains
“Here's this new dataset. It is being distributed in such a way that anyone in the world can buy domains that let you then inject arbitrary images in the dataset.”
Carlini extracted production models from Google and OpenAI with legal permission
“We ran the attack that let us, yeah, stole several of OpenAI's models. With their permission... We notified everyone who was vulnerable to this attack. Some Google models were vulnerable. Some open AM models were vulnerable. There were one or two other people …”
Prompting ChatGPT to repeat a word indefinitely leaks verbatim training data
“One of my co-authors, Milad was working on some other random experiments, and he figured out that if you prompt ChatGPT to repeat a word forever, then it will repeat the word many, many, many times in a row, and then like explode and like just start doing rand…”
Carlini: ChatGPT emitted verbatim 50+ word sequences from internet training data
“And what I can say is that the output of the model was a verbatim, at least 50 word in a row match. To some other document that appeared on the internet previously.”
Carlini published a paper proving C's printf function is Turing-complete
“A while ago as part of a research paper, I was able to show that in C, if you call into print def. It's Turing complete, like printf, you know, like which, like, you know, you can print numbers or whatever, right?”
Stanford researchers detect benchmark contamination by testing evaluation question ordering
“There's a paper by Tatsu at a Stanford. Where they check if the order that the specific questions happen to be in matters. And if the answer is yes, then you probably trained on it because the order of the questions is arbitrary and shouldn't matter.”