Assertion Supported AI assessment confidence: 95% certainty 4/5 debate potential 3/5

Smith: The llms.txt protocol does not work

Ethan Smith · #2285 AEO: How to make ChatGPT promote you · Nov 11, 2025 · at 22:20

Ethan Smith explains why the proposed llms.txt standard for optimizing content ingestion by AI models is an unadopted myth.

0:00 / 0:09exact quote · 9.5s

720p mp4 · rendered on demand · StarZero watermark

“So the intention of LMS.txt is to get more traffic and to send information about, it's not to block, it's to send more information to the LMS so that you get more traffic, but it doesn't work.”

quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →

The claim, plainly

The llms.txt proposal is not an officially supported or recognized protocol used by major LLM providers (such as OpenAI or Google) to ingest structured website information.

a quote alone can be ambiguous, so this restates it with the references and context of the conversation around it resolved. AI-written; the tape below the player is the source.

Assessment

Official technical documentation and public statements from major LLM providers (including Google, OpenAI, and Anthropic) confirm that the llms.txt proposal is an informal community standard rather than an officially recognized or supported protocol for web crawling and ingestion. Representatives from Google explicitly confirmed in 2025 that Google does not support or parse llms.txt files, while official documentation for crawlers operated by OpenAI (GPTBot) and Anthropic (ClaudeBot) relies on standard robots.txt and HTML extraction pipelines without official technical recognition or parsing support for llms.txt.

Google Search Relations representatives (John Mueller and Gary Illyes) confirmed in mid-2025 that Google does not support or parse llms.txt for crawling, search indexing, or AI features.
Official crawler documentation from OpenAI (GPTBot, ChatGPT-User) and Anthropic (ClaudeBot) contains no technical recognition or support for llms.txt as an ingestion standard.
Empirical server log analyses across hundreds of thousands of domains in late 2025 and 2026 demonstrated that major AI web crawlers do not routinely request or rely on llms.txt files.

Checked against

[1] llmstxt.org [2] cruelx.com [3] aeochecker.ai [4] thesocialtarget.com [5] limy.ai [6] michaellivs.com [7] medium.com [8] seroundtable.com

does OpenAI or Google support llms txt llms txt proposal Jeremy Howard standard support OpenAI Google Anthropic llms txt OpenAI Google crawler documentation support

assessed as of 2026-09-03

This assessment is an AI reading of a public statement, made against the cited references with the confidence shown. It can be wrong. The quote, the tape and the references are all on this page; if something is off, use report an error below. full disclaimer →

Mentioned

LLMs llms.txt

⚑ Spot an error? Report it

More from Ethan Smith

Opinion

Smith: AEO software tools are defenseless commodities

“Who's unique and special? And the answer is no one. If, because it's impossible to be unique and special, And the reason it's impossible to be unique and special is because these are not hard to build. These are commodity. These are easy to build relatively co…”

Ethan Smith Nov 11, 2025 ▶ 34:19 #2285 AEO: How to make ChatGPT promote you

Insight

Smith: Startups should ignore SEO until Series A or B

“I think it's hard for early stage companies, which is why I mentioned when a startup asked me what they should do in SEO, I say, don't work on it and wait until you're larger. Wait until you're, you know, series A or later, maybe series B and later, because it…”

Ethan Smith Nov 11, 2025 ▶ 17:29 #2285 AEO: How to make ChatGPT promote you

Assertion Supported

Smith: Over 90% of ChatGPT users still use Google

“Similar web I have data showing that the overlap Basically everyone who uses ChatGPT uses Google. Nearly all of them. I think it's like 90, 90% plus or something. So people are not using it instead of search.”

Ethan Smith Nov 11, 2025 ▶ 18:42 #2285 AEO: How to make ChatGPT promote you

Assertion Supported

Smith: TechRadar Is Among Most Cited LLM Domains Despite Affiliate Disclosures

“TechRadar is a good example. TechRadar shows up A huge amount. It's one of the most cited domains in LMs. And you'll see a snippet at the top saying we're an affiliate site and it does really well. There's no evidence that a paid for mention does worse than a …”

Ethan Smith Nov 11, 2025 ▶ 26:31 #2285 AEO: How to make ChatGPT promote you

Assertion Supported

Smith: Half of newly published internet content is AI-generated

“We just posted an original piece of original research showing that half of the content published on the internet is generated by AI.”

Ethan Smith Nov 11, 2025 ▶ 42:14 #2285 AEO: How to make ChatGPT promote you

Assertion Not checkable as stated

Ethan Smith: 8% of Webflow's signups come from LLMs

“Eight percent of their signups come from LMs.”

Ethan Smith Nov 11, 2025 ▶ 1:41 #2285 AEO: How to make ChatGPT promote you

Smith: The llms.txt protocol does not work

More from Ethan Smith

Turn any episode into a week of clips.