An llms.txt text file at the root of a website, at the center of a technical debate between Google's teams

The essentials in 30 seconds

  • On May 20, 2026, Search Engine Journal documented an internal contradiction at Google over the llms.txt file.
  • The Google Search team advises against the file: its AI optimization guide explicitly says to skip it.
  • Lighthouse 13.3, Chrome's audit tool, instead added it to its checks via an "Agentic Browsing" category.
  • John Mueller compared llms.txt to the now-obsolete meta keywords tag.
  • For SMBs: don't spend energy on this file. Invest it in clear, citable HTML content.

On May 20, 2026, the specialist outlet Search Engine Journal documented a contradiction at the very heart of Google: the Google Search team advises against the llms.txt file, while Lighthouse — the audit tool built into the Chrome browser — has just added it to its checks in version 13.3. Two teams at the same company, two opposite messages about a file that many site owners are still wondering whether they should create.

The llms.txt is a proposed standard: a Markdown-formatted file, placed at the root of a website, meant to give large language models a structured summary of a site's key content. The idea looks appealing on paper. But in 2026, knowing whether to adopt it depends, literally, on which Google team you ask.

What the Google Search team actually says

The Search team's position is clear, and it is negative. Google's official guide on AI features and your website explicitly places llms.txt on the list of tactics to skip, alongside content chunking and AI-specific rewriting.

John Mueller, a long-time Google Search spokesperson, was even more direct: he compared llms.txt to the meta keywords tag — that 2000s relic no search engine has used in years — and called building separate Markdown pages for bots a poor idea. At a Search Central Live event in Asia-Pacific, Gary Illyes and Amir Taboul confirmed the same line: Google is not pursuing the llms.txt standard. One episode captures the confusion well: on December 3, 2025, an llms.txt file briefly appeared in Google's own developer documentation — before being removed the same day.

Why Lighthouse tells a different story

This is where the consistency cracks. Lighthouse 13.3 now ships an experimental category called "Agentic Browsing" that checks, among other things, for the presence of an llms.txt file. Its documentation describes the file as "a machine-readable summary of a website's content, specifically designed for LLMs and AI agents," and notes that without it, "agents may spend more time crawling the site to understand its structure."

The contradiction is only apparent once you unpack it: the two teams are not talking about the same use case. The Search team is talking about visibility in Google Search, AI Overviews and AI Mode. The Chrome team is talking about agents browsing inside a web browser and carrying out tasks for a user. These are two distinct scenarios — but for an SMB owner who reads "Google recommends" on one side and "Google advises against" on the other, the result is the same: confusion.

The trap to avoid. The danger is not the llms.txt file itself: it is harmless. The danger is the time you spend creating it, keeping it up to date, and duplicating your content in Markdown — time taken away from what actually decides your AI visibility.

What this changes for your visibility

First takeaway: do not make llms.txt a priority in your GEO strategy. No major search engine has confirmed using it for ranking or citation. To appear in AI Mode, AI Overviews or the answers of ChatGPT and Perplexity, it is your public HTML pages that get read, analyzed and cited — not a side file.

Second takeaway: the real guideline has not changed. As we explained in our analysis of Google's official guide for AI search visibility, the company keeps repeating that optimizing for AI remains fundamentally SEO: useful content, clear structure, sourced data, schema.org markup. The llms.txt is a distraction from that foundation. If you want to audit what truly matters, start from our breakdown of GEO versus SEO priorities rather than from a file Google Search ignores.

0Search engines that confirmed using llms.txt for ranking
13.3Lighthouse version that now audits the file
2Google teams holding opposite positions

3 priorities to apply now

1

Don't create (or keep prioritizing) an llms.txt

If you don't have an llms.txt file, don't rush to create one "just in case." If you have one, leave it — it does no harm — but devote no resources to keeping it current or duplicating your content in Markdown. The priority is elsewhere.

2

Make your HTML pages directly citable

A clear question in the heading, a direct answer in the first paragraph, figures with their source, up-to-date schema.org markup. That is the public content AI reads and cites. It is also the core of the SEO priorities we covered after Google I/O 2026.

3

Check AI bot access to your site

The real technical issue is not llms.txt, it is your robots.txt and your server headers: can AI agents and crawlers (GPTBot, ClaudeBot, Google-Extended) reach your pages? A page accidentally blocked will never be cited — no side file will change that.

Is your site citable by AI?

Cicéro runs a GEO + SEO diagnosis of your site and tells you exactly which pages are readable and citable by Google AI Mode, ChatGPT and Perplexity — without selling you a useless file.

What this article does not cover

This analysis covers the llms.txt debate as it stands on May 20, 2026. It does not settle the question of autonomous browsing agents: if those agents become widespread and end up valuing the file, the picture may shift — we will update the article accordingly. We provide no llms.txt adoption figures: there is no reliable public measure at this stage. Finally, this article does not detail robots.txt configuration against AI crawlers, a topic that deserves its own guide. For broader context on how search is evolving, see our coverage of Google I/O 2026 and the search overhaul.

The Cicéro take

The llms.txt is the tree that hides the forest. While part of the industry debates a file Google Search refuses to use, the essentials remain unchanged: AI cites the pages it understands, not the ones with a neatly filed side document. Our advice is blunt — ignore the noise, perfect the content that users and machines actually read.

Sources

Frequently asked questions

What is the llms.txt file?
llms.txt is a Markdown-formatted file, proposed as a standard, placed at the root of a website. It is meant to give large language models and AI agents a structured, machine-readable summary of a site's key content. It is a community proposal, not an official standard adopted by search engines.
Do I need an llms.txt file to be visible in Google AI Mode?
No. The Google Search team explicitly states that llms.txt is not needed for Google's generative AI features. John Mueller compared it to the now-obsolete meta keywords tag. To appear in AI Mode and AI Overviews, what matters is public, structured, high-quality HTML content.
Why does Lighthouse check for the llms.txt file then?
Lighthouse 13.3, the audit tool built into Chrome, added an experimental Agentic Browsing category that checks for the presence of an llms.txt file. That audit targets agents browsing inside a web browser, a different use case from Google Search ranking. The two Google teams are not pursuing the same goal.
Can the llms.txt file hurt my SEO?
The file itself does not penalize a site. The real risk is elsewhere: creating and maintaining parallel Markdown pages diverts time and resources from what actually matters, namely clear, sourced, structured public HTML content. That effort is better invested in the quality of the pages users and AI actually see.
Alexis Dollé, founder of Cicéro
Alexis Dollé
CEO & Founder

Growth and SEO content strategist, I founded Cicéro to help businesses build lasting organic visibility — on Google and in AI-generated answers alike. Every piece of content we produce is designed to convert, not just to exist.

LinkedIn