John Mueller has stated that LLMs.txt, a proposed file to guide AI crawlers, is not being used by major AI services and offers no real benefit.
Comparing it to the outdated keywords meta tag, Mueller confirmed that AI bots aren’t even checking for the file, making it ineffective for SEO or content control.
What Is LLMs.txt and Why It’s Not Working
LLMs.txt was introduced as a way to make it easier for LLMs to extract the main content of a website.
It’s not designed to block bots like robots.txt, but rather to provide a stripped-down version of site content—free of ads, navigation, and other non-essential elements.
The problem? No one is listening.
Mueller confirmed that companies like Google, OpenAI, and Anthropic haven’t adopted the standard. AI bots don’t even request the file, making it effectively invisible to the services it was meant to help.
Community Feedback Confirms: No Impact, No Use
A Reddit user who had implemented LLMs.txt on their blog noticed zero activity in their logs from AI crawlers.
Simone De Palma, the user behind the post, later shared similar observations on LinkedIn. Others joined in, saying bots weren’t accessing the file, and some reported the same results across thousands of domains.
One domain host with over 20,000 websites said only obscure bots like BuiltWith accessed the LLMs.txt file. None of the mainstream AI agents touched it.
Mueller’s Verdict: Like the Keywords Meta Tag
Mueller compared LLMs.txt to the keywords meta tag, a feature that once let site owners declare what their pages were about. Search engines stopped using that tag years ago because it was easy to manipulate and unreliable.
He raised a concern that is, if bots already download full web pages and structured data, why should they rely on a separate file? Worse, LLMs.txt could be abused by showing AI bots one version of content and everyone else another—essentially a backdoor for cloaking.
Another Issue: Bad User Experience
De Palma also pointed out a user experience flaw. If AI systems cite LLMs.txt files instead of actual web pages, users might end up on bare text files instead of properly formatted articles. These markdown files don’t support navigation or links back to the source content.
This undermines the usefulness of citations and creates a dead end for users trying to learn more.
What Site Owners Should Focus On Instead
LLMs.txt doesn’t offer practical benefits today. Here’s where to spend your time instead:
Structured data: Use schema markup to clearly define your content.
Robots.txt: Continue using this to manage bot access to your site.
Readable, useful content: Write for people, not for AI.
Monitor logs: Keep an eye on what bots are actually crawling.
Stay updated: Track developments from AI companies and search engines.
Until LLMs.txt gains support—and that’s a big “if”—there’s no incentive to implement it.
Key Takeaways
- LLMs.txt is being ignored by all major AI services.
- It doesn’t block or manage bots, unlike robots.txt.
- It creates opportunities for abuse, such as cloaking.
- It could damage the user experience if cited directly by AI.
- Experts recommend focusing elsewhere, like structured data and content quality.
Dileep Thekkethil
AuthorDileep Thekkethil is the Director of Marketing at Stan Ventures and an SEMRush certified SEO expert. With over a decade of experience in digital marketing, Dileep has played a pivotal role in helping global brands and agencies enhance their online visibility. His work has been featured in leading industry platforms such as MarketingProfs, Search Engine Roundtable, and CMSWire, and his expert insights have been cited in Google Videos. Known for turning complex SEO strategies into actionable solutions, Dileep continues to be a trusted authority in the SEO community, sharing knowledge that drives meaningful results.