Community Notes

“Community notes, like the project that I was mentioning Keith, you know, started, we intentionally structured that like a startup. Like it was literally like we made a seed bet on Keith and his team. And we were like, you don't like, Don't worry about the OKRs…”

“I always felt really bullish on that being the future of how essentially how like content is moderated on Twitter, just cause it was very clear that the way we were handling content moderation Among many other flaws just wasn't scalable”

“If you said I think like back in 2020, before we started building anything here, whether this could work at all, I think a room of ML engineers would say, oh, you have to keep it closed source. You know, people are going to be manipulating this all the time. Y…”

“We actually you know, specifically model that topic as well as some other topics. So we can like estimate people's opinion on, on that particular debate.”

“Recently, for example, we just released an update to what we call the community notes bat signal or the ability to request a community note. So we, anyone on X can say, hey, I think this post needs a community note, and now they can even add a source explainin…”

“And over, ah, let's say the last year, 20, 24, we had something like 95,000 notes that were seen about thirty billion times. That's more than double the prior year. Prior year was something like 37 K notes seen fourteen billion times.”

“One distinction that I would make, which may maybe can come off as nitpicky, but I think is important is community notes adds additional context. It's not fact checking necessarily. So So there are cases where the post could be true, but maybe it's just mislea…”

“This is, I think, two years before any acquisition stuff happened, Elon is just a Twitter user building rockets and electric cars and other cool stuff, and stumbles on this thing that depicts the prototype that we've been testing, and he writes back, yeah, def…”

“There was, you know, original research you know, before, before birdwatch even started or community notes even started from external researchers showing that, you know, crowdsource fact checkers can do, you know, lay people can do about as well as Fact checker…”

“Although we don't actually use the fact that a post was noted in the core ranking algorithm”

“Vitalik Peteran had a blog post where he, you know, talks about his explorations, you know, making sure the algorithm really does what it says it does.”

“So every post is eligible for notes. It's, and that was again, another really important principle. It's like, It, we shouldn't exempt Elon. We shouldn't exempt government figures. We should, like everyone, even advertisers can get notes. So any posts on the pl…”

“So, so pulling the context up, you know, so that you have zero clicks that you need to make and keeping it specific is so important.”

“Initially, some people inside the company were suggesting like, Hey, why don't you have journalists or, you know, some select group be the first participants, but Very specifically, we're like, no, that's like, we're trying to move away from the idea of curate…”

“Whenever we update the code, there's, or update the scoring system, there's an update in GitHub when the data is published daily, so you can download it”

“You know, an interesting thing about that is no one ever asked us or brought up or seemed to care about anything related to cost savings in this process. And I think that's like a pro like an assumption people have outside the company that like, this must've b…”

“If there's a problem with a note, it's so bad you want to do something about it. It's a problem with the system. Like we need to redesign the system to be showing good notes.”

“Jack has made fun of it, calling it like the ultimate Facebook name or something like that, but the most boring Facebook name, which is funny because they're now, you know, launching community notes.”

“Our view is it's actually, we want to have all of humanity here, because if we have all of humanity, we can, we then have the data to understand what notes will be helpful to actual humanity. You know, we can better model that better or better understand and b…”

“It's also a pretty cool example of something you can do on the internet in the social media world that was difficult in kind of like a print or standard news world where you would see maybe a correction, like the next day in a corner of a paper, but it was har…”

“If you write one that people, you know, people who normally disagree, find not helpful, you actually will ultimately lose your ability to write and have to earn it back.”

“Authors become 80% more likely to decrease, or sorry, to delete their post after they get noted.”

“The actual first algorithm that I put into production was very focused on anti manipulation. It was this kind of page rank variant but it didn't solve the problem of, you know, bias basically. So if there are some, if there are more users on one side, the, a p…”

“Multiple different external research groups have both found consistently that There's like a 50 or 60% drop in total reposts, which is just nuts after a node is applied.”

“When we first launched US wide, this was like in two, a note appeared on a white house tweet. And the White House deleted the tweet and reissued an updated statement.”

“Inevitably there's going to be a time gap between when a post goes live and when people figure out what's going on and when they get the note out there. And so we send those notifications to try to close that gap.”

“There's something like 950,000 contributors around the world. That's, you know, nearing a million people making this happen.”

“When we built the first mock-ups, we had just, these were just, like, pictures of, you know, depicting what community notes might look like. We showed those to people across the political spectrum. We saw, like, hey, people really like these, whether they're o…”

“The whole OKR determination and planning process took longer than it would take us to pick a goal and then execute it on it and finish it.”

“There is a very long running doc that has had to be chopped and purged because it was breaking Google Docs and Chrome at various points in time. It's sort of like a note taking doc. It's really where we coordinate what we're doing.”

“The fact that the note applies is directly attached to the post and visible by anyone who sees the post helps, you know, cross the, those filter bubbles and can kind of I think for some people it's the first time they've actually seen counter arguments to, you…”

“Community note shows you people really can agree on quite a lot, even on super controversial topics related to politics and everything. Like there's a lot of agreement. That's why notes work.”

“We did not assign anyone to this project. Like people reached out to join or they applied to join the job. You know, I and the team interviewed every single person that joined the team and we're like, we want that person on the team. They want to be on the tea…”

“First of all, there is no, we don't have a button that will change the status of the note or of a note. So if a note is showing because the people have rated it, And found it helpful. It is going to show. Like, we can't change that.”

“The biggest change there happened in X. The biggest changes prior to that were just the decision to put this out there and you know, and, Have it be operating at, you know, in public at first us wide scale. But yeah, then the big, the bigger switches came in t…”

“People are actually more willing to cross partisan boundaries when they are anonymous or pseudonymous than when they are under their real name.”

“I think the key thing really that we do is we actually look for agreement from people who have disagreed in the past. And what we see is when people actually have that sort of surprising agreement, that's what makes the notes so neutral and accurate and well w…”

“We probably show about eight percent of notes that get proposed. I think that's, it's been between, let's say, seven percent and 10% or 11%, something like that over time.”

“In the first few days of the conflict, the median time from a post going live to a note showing up was five hours, which is like crazy fast. Typical fact checking is like two to four. It's really common to see it take two to four days. These notes were showing…”

“Super Notes is probably the first very substantial potential change in like the algorithm of the way it works that was coming, kind of coming from the outside and plausibly could be part of the core.”

“Nothing could handle the scale of the problem. Nothing could handle the speed. And a lot of people just didn't trust the existing approaches. Like this, the existing approaches were either fact checkers or internal trust and safety teams making decisions about…”

“We saw more like 30 to 40% engagement rate drops for likes and reposts and an AB test we ran. When comparing showing a post with or without a note”

“For a note to be proposed, the person proposing has to have earned the ability to write notes. So there is that, that aspect where you have to, like, earn in to be able to do this. And the way you earn that ability is, is through your ratings by demonstrating …”

“In the first, I think it was like first three days or something of that conflict. We had 500 notes covering all sorts of different you know, like out of context imagery”

“If we need, need something or have a question on something, I talked to Elon and it was, you know, it was like that from the beginning. It's like that now.”

“Oh, like, 500 gigs. It'll take like a day if you don't do anything special to speed it up.”

“Actually, there have been external studies, you know, run by people totally independent of us who have found that if you take a community note or a post with or without a community note, that actually people's agreement with the core claims in the post does ch…”