Does llms.txt Work? What the 2026 Data Shows

Last updated: 21 October 2026

Does llms.txt work? The short answer is no, not in any way that moves the needle yet: a study of 94,614 cited URLs across ChatGPT, Claude, Gemini, Grok and Perplexity found llms.txt pages accounted for just 0.001% of citations, according to ALLMO.ai.

Key Takeaways

• Only one llms.txt citation turned up in a study of 94,614 cited URLs and 11,867 AI responses, according to ALLMO.ai.
• 97% of llms.txt files received zero crawler requests in a given month, based on analysis of 137,000+ domains by Ahrefs (2026).
• Google's own Search Advocate, John Mueller, has stated "no AI system currently uses llms.txt", directly contradicting vendor claims that it drives AI visibility.
• Adoption is rising fast regardless — from 1.04% to 5.61% of top 10,000 domains in 12 months — according to Casey Burridge's analysis of the HTTP Archive BigQuery dataset (2026), meaning uptake and usage are moving in opposite directions.
• Aether AI's measured baseline found UK brands named in just 3.3% of 700 live AI engine scans (July–August 2026), underlining how much of AI visibility depends on factors other than a text file.

What is llms.txt and What is it Supposed to Do?

llms.txt is a proposed plain-text file, hosted at a website's root (example.co.uk/llms.txt), that lists a site's most important pages in a simplified, markdown-style format intended to help large language models (LLMs — the AI systems behind ChatGPT, Claude and Gemini) understand a site's content more efficiently. It was proposed in September 2026 by developer Jeremy Howard as a kind of curated index, similar in spirit to an XML sitemap but written for machines that read natural language rather than crawl HTML.

The theory is straightforward: AI crawlers have limited context windows and limited patience for parsing cluttered HTML, so a clean, hand-picked summary should, in principle, make it easier for a model to find and cite the right pages. That's the pitch. Whether any AI company actually acts on it is a separate question — and it's the one that matters.

Which AI Companies Actually Read and Use llms.txt Files?

None of the major AI companies have confirmed they use llms.txt for crawling, training or retrieval, based on both public statements and server-log evidence. OpenAI, Anthropic, Google and Perplexity have not published documentation listing llms.txt as a recognised input to their systems.

Google's John Mueller has been the most direct voice on this. He has said plainly: "FWIW no AI system currently uses llms.txt," and added, "To my knowledge, none of the AI services have declared they're utilizing LLMs.TXT (and examining server logs confirms they aren't checking for it)." He also described it as "not done for search."

This isn't guesswork on his part — server-log studies back it up. Of the vast number of AI bot events analysed, only a tiny handful of requests targeted /llms.txt directly. That's not a rounding error; it's close to statistical noise.

Is There Evidence llms.txt Improves AI Visibility or Citations?

The available evidence points the opposite direction: llms.txt shows no measurable link to AI citation performance. SE Ranking (2026) built an XGBoost predictive model to test whether llms.txt presence correlated with AI citation frequency — and found that removing llms.txt as a variable actually improved the model's accuracy, suggesting the file was adding noise rather than signal.

Real-world traffic data tells the same story. OtterlyAI (2026) ran a 90-day experiment and found only 0.1% of AI bot traffic — 84 visits out of 62,100+ — ever touched the /llms.txt file, performing three times worse than the site's average content page.

Perhaps the most telling figure: among the 50 most AI-cited domains in SE Ranking's dataset (2026), only one had an llms.txt file at all. If llms.txt were driving citations, the most-cited sites would be the first to show it. They aren't.

"Ask three things: does it query the engines the way real buyers do, does it show you the sources behind every answer, and can it act on what it finds rather than just chart the decline? Dashboards that only observe are grief counselling with a subscription." — Lauren Dawkins, Head of Content, Aether AI

That's the standard worth applying to any GEO tactic, llms.txt included: does it show up in what the engines actually cite, not just in what a vendor claims it should do.

How is llms.txt Different From robots.txt and XML Sitemaps?

llms.txt, robots.txt, and XML sitemaps serve three distinct and non-overlapping purposes, despite surface similarities in naming and format. robots.txt is a decades-old standard, formalised by the Robots Exclusion Protocol and now an official IETF (Internet Engineering Task Force) standard (RFC 9309, 2022), that tells crawlers which parts of a site they may or may not access. XML sitemaps, backed by Google, Bing and other search engines through the sitemaps.org protocol, list every indexable URL so search engines can discover content efficiently.

llms.txt has no equivalent backing. No search engine or AI lab has adopted it as a recognised standard, and it carries no enforcement mechanism — a crawler can simply ignore it, and the evidence suggests most do.

The practical difference: skipping robots.txt or a sitemap has measurable consequences for crawl behaviour. Skipping llms.txt, based on current evidence, has essentially none.

How Much Does it Cost or Take to Implement llms.txt?

Creating a basic llms.txt file typically takes a few hours for a small site, since the format is simple markdown — a title, a short summary, and a bullet-pointed list of key page links. There's no licensing cost, no hosting fee beyond what a site already pays, and no specialist tooling required to generate a minimal version.

The cost consideration isn't the file itself — it's the opportunity cost. Given that Ahrefs (2026) found 97% of these files go completely unread by crawlers, the hours spent curating and maintaining an llms.txt file (keeping link lists current as pages change, rewriting summaries, testing formatting) are hours not spent on tactics with demonstrated impact: structured data markup, clear FAQ content, and genuinely crawlable, well-linked HTML that both Google and AI engines can already parse without a special file.

For a UK business weighing GEO priorities against a content calendar and finite marketing hours, that trade-off matters more than the file's near-zero build cost.

What Are the Common Mistakes Companies Make With llms.txt?

The most common mistake is treating llms.txt as a replacement for genuine technical SEO and content quality, rather than as the unproven, experimental format it currently is. Teams sometimes build an llms.txt file and then neglect the things that are known to matter — clean HTML structure, fast page load times, proper heading hierarchy, and schema markup.

Other recurring errors include:

• Assuming adoption equals effectiveness. llms.txt adoption among the top 10,000 domains grew from 1.04% to 5.61% in a year, according to Casey Burridge's HTTP Archive analysis (2026) — but growth in adoption says nothing about whether the file is read, let alone whether it changes citation outcomes.
• Publishing the file without validating it. Many implementations return a 200 HTTP status (the file "exists") but fail basic markdown formatting, making them unparseable even by a system that did check.
• Letting the file go stale. Link lists that reference deleted or moved pages create dead ends precisely where a business wanted to look authoritative.
• Skipping robots.txt and sitemap maintenance in favour of llms.txt. Only about 1 in 10 websites (10.13%) use llms.txt at all, based on nearly 300,000 domains studied by SE Ranking (2026) — meanwhile robots.txt and sitemaps remain near-universal and demonstrably functional.

Is There an Official Standards Body Endorsing llms.txt?

No official standards body — not the IETF, not W3C, not any major search or AI engine — has endorsed llms.txt as a recognised protocol. It originated as an independent proposal from developer Jeremy Howard in September 2026, published informally rather than through a standards process like the one that produced RFC 9309 for robots.txt.

Google has been explicit that it isn't participating. John Mueller's comment that llms.txt is "not done for search" reflects Google's position that its crawlers, including those behind AI Overviews, don't check for or rely on the file. Neither OpenAI, Anthropic nor Perplexity have published technical documentation listing llms.txt as an input to ChatGPT, Claude or Perplexity's answer engine respectively.

This matters for UK businesses evaluating where to spend limited GEO budget: a format with no standards body, no vendor commitment, and a near-zero crawl rate carries a fundamentally different risk profile to something like schema.org structured data, which is jointly backed by Google, Microsoft, Yahoo and Yandex.

Is it Worth Implementing llms.txt Now, or Should UK Businesses Wait?

For most UK businesses, llms.txt is a low-priority, low-risk addition — worth a few hours if time allows, but not worth delaying higher-impact GEO work for. The honest comparison is cost versus evidence: implementation is cheap, but the data consistently shows near-zero crawler interest and no measurable citation benefit.

Aether AI's own measured baseline illustrates where the real leverage sits. Across 700 live AI engine scans of four UK brands' target keywords in July–August 2026, the brands were named in just 23 answers (3.3%), with the longest-established brand (in security services) reaching 6.3% across its 270 scans, as published at aether-ai.co.uk/insights/ai-visibility-baseline-2026. That's a visibility gap driven by content depth, entity clarity and citation-worthy structure — not by the absence of a root-level text file.

The better question isn't "llms.txt or nothing" — it's where llms.txt ranks against alternatives known to influence AI answers: clear, well-structured articles with named sources; FAQ sections written in direct-answer format; schema markup; and a Google Search Console presence that confirms what's actually being indexed. If those are already in place, adding llms.txt costs little. If they aren't, build those first.

Your llms.txt decision checklist

• Check whether your top competitors already have an llms.txt file and whether it correlates with their AI citation frequency (it likely won't).
• Review your server logs for any AI bot requests to /llms.txt before assuming it's worth maintaining.
• Prioritise schema.org structured data and clean HTML hierarchy ahead of llms.txt — both have confirmed engine support.
• Audit your existing content for direct-answer formatting, since that has stronger evidence of AI citation value.
• If you do build llms.txt, validate the markdown formatting properly rather than just confirming a 200 status.
• Keep link lists current if you maintain one — a stale llms.txt undermines the authority it's meant to project.
• Re-run a free AI-visibility check periodically to see whether your actual citation rate is moving, regardless of llms.txt status.
• Treat llms.txt adoption figures as a trend to monitor, not a reason to act immediately.

FAQ

Does llms.txt work for improving AI search visibility?

No — the available evidence shows no measurable link between having an llms.txt file and appearing in AI citations. A study of 94,614 cited URLs across five major AI engines found just one llms.txt citation, according to ALLMO.ai.

Do ChatGPT, Claude and Gemini actually read llms.txt files?

There's no confirmed evidence they do. Of the vast number of AI bot events analysed, only a tiny handful of requests targeted /llms.txt directly, and Google's John Mueller has stated outright that "no AI system currently uses llms.txt."

Is llms.txt the same as robots.txt?

No. robots.txt is an IETF-standardised protocol (RFC 9309) that controls crawler access and is respected by virtually all major bots, while llms.txt is an unstandardised, voluntary proposal with no confirmed uptake by any AI company.

How many websites currently have an llms.txt file?

Around 1 in 10 websites (10.13%) use llms.txt, based on analysis of nearly 300,000 domains by SE Ranking (2026), though adoption among top 10,000 domains has grown to 5.61% as of June 2026 according to Casey Burridge's analysis (2026).

Can llms.txt harm my AI visibility or SEO?

There's no evidence it actively harms visibility, but it can create a false sense of progress. Time spent maintaining an llms.txt file is time not spent on tactics like structured data and direct-answer content formatting, which have stronger links to AI citation outcomes.

What did Google say about llms.txt?

Google's Search Advocate John Mueller has said plainly that no AI system currently uses llms.txt and that it is "not done for search," adding that server log analysis confirms AI services "aren't checking for it."

Should a UK business implement llms.txt in 2026?

It's a low-cost, low-priority option rather than an urgent one. Businesses are better served prioritising schema markup, clear content structure and Google Search Console hygiene before spending time on llms.txt, given adoption keeps growing (per Casey Burridge, 2026) while citation evidence remains essentially flat.

Building AI visibility that's actually measured, not assumed

Questions like "does llms.txt work" come up constantly because businesses want a quick technical fix for AI visibility — and the honest answer is that the fix usually isn't technical at all, it's content and structure. Aether AI's platform is built around that distinction: it tracks citation performance across six AI engines (ChatGPT, Perplexity, Google AI Overviews, Claude, Gemini and Copilot) so a business can see what's actually driving mentions, rather than guessing based on a file no major AI system reads.

Aether AI's own measured baseline — 700 live AI engine scans of four UK brands' target keywords across July–August 2026, with brands named in 3.3% of answers overall and up to 6.3% for the most established brand — reflects exactly this kind of evidence-based tracking rather than assumption.

If you want to see where your own site currently stands before investing time in speculative fixes, Aether AI's free AI-visibility audit at /audit shows your current citation rate across major engines, and the pricing page outlines the self-serve plans for ongoing tracking and content generation.

Related Reading

• Best Content Format for AI Citation (UK Guide 2026)
• Do AI Crawlers Execute JavaScript? The 2026 UK Answer
• Do AI Crawlers Render JavaScript? 2026 UK Guide