Cloudflare

“OpenAI, 250 mentions of a site relative to one direct traffic sent to the website. Anthropic, 6000. I mean, 6000 crawls. 6006 thousand crawls to one.”

“Now for Google, it's 18 pages crawled, one visit. OpenAI, it's 1500 pages crawled, One visit. Anthropic, it's, was, it's now gone up from 6000 to 60,000 pages crawled. One visit.”

“What we've done is we've actually tried to identify with real narrow precision. What are those pages that are on the web that have some indication that they are going to be monetized. And generally that is look at, is it behind a paywall or is it, does it have…”

“I think in both cases though, you're actually costing the content creator something. There is literally, they're paying for that traffic. They're paying for that load that the crawlers are Pulling off of it.”

“Starting really 18 months ago is that those companies said, hey, I know we hired you in order to stop the Chinese hackers, but we have this new threat that's there... And it was only when we actually saw the data and we saw how AI companies were taking content…”

“There's actually an original one of the protocols set out a four or two response and that response says payment required. And so we're actually tapping into that exact original specification to say when a robot tries to access, access a page, Where there's an …”

“80% of the AI companies are our customers of ours, and so, so we project them as well.”

“And so what we've proposed, and we're working with the IETF as well as regulators is extensions to robots.txt to give it that granularity.”

“And then just today on July first, we announced that we are hard blocking the AI crawlers unless they will actually compensate Content creators for the content that they're creating.”

“And we have very reliable data at Cloudflare going back 10 years, looking just at Google and the metric that has stayed very consistent over time is how much Google crawls the web. They've actually crawled at a very consistent rate over the last 10 years.”

“At the end of the day, who wins in the AI risk is whoever has the access to the most data. And I just think that the good guys are always going to have access to a lot more data than the bad guys.”

“What we're seeing though, is that for the first time in history, searches across the major search engines, Google in particular, are actually on the decline.”

“Medium is a Cloudflare customer and a happy one, and we do use Cloudflare to block basically any crawler that isn't crawling in a way that drives traffic back to us, we're blocking right now.”

“So we've done a bit of work with a number of our partners, including, you know, partners like Cloudflare that do a really, really good job at the CDN level. To analyze bots and bot traffic and what to make available in a merchant environment, like, you know, p…”

“And what we have said at cloudflare is that by mid September, we're gonna treat Google the same way on. If you have an, you know, ads, if you're a business that's ad supported, subscription supported, then we're going to restrict Google from being able to acce…”

“In May of 2026, you won't believe it, but we now see that automated traffic is past human traffic for the first time in the history of the internet.”

“Visa, largest payments network in the world, handles at peak during the holidays, somewhere around 20,000 financial transactions per second. Cloudflare handles about half a billion internet transactions per second.”

“We think that somewhere between one and 10% of those will day one be monetizable through agents with a with a micropayments transaction, which means that if you're going to build something like this, you've got to build a payments infrastructure that can day o…”

“I think at the same time, you've got sort of agents that are operating on their own. That's probably the smallest of the categories right now. I think that that will grow and become a pretty large category, but right now that's sort of Agent to agent kind of c…”

“And if we just assume the current trends, what's, what's blows my mind is that if you just fast forward five years from now we estimate now that automated traffic online will be a thousand times what human traffic is.”

“The majority of companies that are out there, the majority of content that's created They actually want that content in AI, like Cloudflare, like our knowledge base is a way of us sharing how to use Cloudflare really well. And we want to make sure that every A…”

“If every one of those agents ran in a traditional container, like, you know, which is not even a VM, not even its own server, but just a container, right? You would need more than 40 times the current CPU production in an, in a calendar year in order to suppor…”

“The internet, European internet came this close to completely collapsing in COVID because in COVID over a period of two weeks, Internet traffic doubled, and it especially doubled for video streaming, which was, ah, which was a very high bandwidth.”

“I do think that we're going to move more and more away from kind of an app store model. Because that's not going to be the interface where we reconnect with it.”