McCloy: ChatGPT filters out clickbait metadata in favor of descriptive information
Robert McCloy · AI is Eating Search · Jul 23, 2025 · at 18:06
Robert McCloy, co-founder of Scrunch.ai, explains how AI search engines re-rank search results based on page metadata before content summarization.
0:00 / 0:32exact quote · 32.9s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“The metadata has been super optimized for human click through, right? So think about like clickbait, think about things that are like, you know, like Creating urgency, especially true in e-commerce, right around sales and things like that. But they're not very informational about what actually is on the page. And then what we see is like, regardless of search ranking, often Like, a system like ChatTVT chooses not to use those, right? So it will re-rank them out of the consideration set, and then it'll just kind of go on to the next best result that has a more descriptive metadata saying, hey, this page actually has something that's relevant to your question on it.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Robert McCloy
McCloy: AI is fundamentally replacing web browsing, not just search
“Thinking about what's going on in AI as really replacing search is the wrong take, or maybe not that interesting. It's really more like AI is replacing web browsing. I think that's actually like a more meaningful, like more fundamental shift.”
McCloy: Gray-Hat SEO Tactics Currently Work in AI Search
“In terms of like some of the more gray area SEO tactics like that, you know, Penguin was addressing with Google back in the day. I mean, I think the reality is a lot of those things do work today. Like I can't tell you they don't work, but I would say as like,…”
McCloy: AI chat queries represent higher commercial intent than Google informational searches
“In chat search people, people don't really do that. People just say, solve the problem for me. What I think is really interesting about that is, like, it's actually the highest intent. Like, it's the most valuable thing you could be doing. Because you're going…”
McCloy: ChatGPT Search Bans for Prompt Injection Are Coming
“I think it works until it stops working.
Right.
And I would say like, there's not a lot of stories of people getting banned for like Chatsby D search so far, but it's coming.”
McCloy: ChatGPT Does Not Index or Retrieve llms.txt by Default
“I knew that there's debate about this, but I'd say the evidence is like ChatTriPT is not indexing and it's not retrieving content from LMS.txt by default.”
McCloy: Users dislike on-site website chatbot popups
“Nobody wants to use a chatbot on your website. I don't want to use chatbots on people's websites. Like if things pop up on a website when I go to visit it, I just like, I fly into a visceral rage. I'm definitely closing the window if not closing the tab.”