llms.txt explained: format, examples and whether it helps

That distinction matters for AI answers. Whether Google AI Overviews, ChatGPT search or Perplexity can cite you still depends on whether your pages have been crawled and indexed in their current version, and an llms.txt file doesn't change that.

What is llms.txt?

llms.txt is a community proposal for a Markdown file that tells large language models what a website is and where its best content lives. Jeremy Howard published the proposal at llmstxt.org on September 3, 2024, and the site shows a revised version 2 dated August 10, 2026.

The reasoning is practical: a model's context window is limited, and a typical HTML page is mostly markup. An llms.txt file hands the model a clean, hand-picked index instead.

The proposal also suggests a companion habit: serving a clean Markdown version of important pages at the same URL, with .md appended or the extension replaced (for example, /pricing.md alongside /pricing). Version 2 also recommends a rel="alternate" type="text/markdown" link from each page to its Markdown copy. Tools that follow your llms.txt links can then read text instead of HTML.

Three things llms.txt is not:

• It's not an access control. It doesn't allow or block any crawler. That's still robots.txt's job.
• It's not a URL inventory. It's a curated shortlist, not a sitemap of every page.
• It's not an adopted search standard. As of October 2026, no search engine or AI company has said it uses llms.txt to choose sources.

The format: title, summary, sections and links

The llms.txt format is plain Markdown in a fixed order: an H1 title, an optional blockquote summary, optional notes, then optional H2 sections that list links. According to llmstxt.org, the parts are:

1. H1 with the project or site name. This is "the only required section."
2. A blockquote with a short summary of the project, containing the key information needed to understand the rest of the file.
3. Zero or more Markdown sections of any type except headings, such as paragraphs or bullet lists, with more detail about the site and how to use the linked files.
4. Zero or more H2 sections containing file lists. Each entry is a Markdown link (the page name in square brackets, then its URL in parentheses), optionally followed by a colon and a short note.

One H2 name has a special meaning. A section called Optional holds secondary links that a tool can skip when it needs a shorter context.

The file usually lives at /llms.txt on your domain; version 2 of the proposal also allows files at subpaths, each covering the pages under it. Some tools also look for a longer llms-full.txt that inlines full page content. That's a community convention, not part of the proposal, so treat it as optional.

An example llms.txt for a SaaS site

A good SaaS llms.txt reads like the briefing you'd give a new account manager: one clear description, then the handful of pages that answer the questions prospects ask. Here's an example for a fictional product analytics company on example.com:

The file, line by line (each line exactly as it appears in llms.txt):

A few choices in the example are deliberate:

• The summary is factual. No slogans, and it matches the about page and Organization schema, because consistent descriptions help AI systems understand an entity.
• Links point to answers. Pricing, security, quick start and comparisons are what people ask assistants about.
• Notes are one line. Each tells the model what it will find before it fetches the page.
• Long pages go under Optional, so tools short on context can skip them.

If you don't serve .md versions, link to the normal HTML URLs.

llms.txt vs robots.txt vs sitemap.xml

llms.txt, robots.txt and sitemap.xml do three different jobs: llms.txt summarizes, robots.txt controls access, and sitemap.xml lists URLs for discovery. A site that cares about search and AI answers should treat robots.txt and the sitemap as essential and llms.txt as optional.

The practical consequence: if robots.txt blocks OAI-SearchBot, no llms.txt file will get you into ChatGPT search. OpenAI says sites opted out of OAI-SearchBot "will not be shown in ChatGPT search answers" (OpenAI crawler docs). Fix access and discovery first.

Who reads llms.txt today, and who doesn't

Google Search ignores llms.txt, and as of October 2026 none of the major AI search engines documents using it to choose sources. Its real audience today is tools a person deliberately points at your site.

Google is explicit. Its guide to optimizing for generative AI features (last updated July 2026) says, "You don't need to create new machine readable files, AI text files, markup, or Markdown to appear in Google Search." On such files: "Doing so will neither harm nor help your site's visibility or rankings in Google Search, as Google Search ignores them." That covers AI Overviews and AI Mode. The same goes for the Google-Extended robots.txt token, which people sometimes confuse with an AI opt-in: Google says it "does not impact a site's inclusion in Google Search" (Google's common crawlers), so blocking it won't take you out of AI Overviews and allowing it won't get you in.

OpenAI, Perplexity and Anthropic document how their crawlers work, including OAI-SearchBot, GPTBot, ChatGPT-User, PerplexityBot, Perplexity-User, ClaudeBot, Claude-SearchBot and Claude-User (our AI crawlers guide covers each one). None of that crawler documentation says the company reads other sites' llms.txt files to choose sources (OpenAI, Perplexity, Anthropic). Their search products retrieve from indexes built by crawling ordinary pages. To understand that pipeline, read how AI search chooses sources.

Where llms.txt does get used is closer to developer workflows. OpenAI, Anthropic, Google (for the Gemini API) and Perplexity all publish llms.txt files for their own developer documentation. Documentation platforms generate it, and a developer can paste a site's llms.txt URL into a coding assistant or agent so the tool starts from a clean map instead of crawling blind. If your product has an API or docs that people build against, that's a real benefit.

If what you actually want is for AI assistants to act on your behalf rather than read about you, a file is the wrong tool. The Model Context Protocol (MCP) exists for that. For example, FreshlyCited's API and MCP server let an assistant run visibility checks and pushes directly (the MCP server is available on the Agency Pro and Agency Scale plans).

Should you add one? Costs and benefits

Adding an llms.txt file is worth it when the cost is an hour and the file stays accurate. It isn't worth it when you expect it to win citations. The trade-off looks like this:

Benefits

• Agents and assistants that a user points at your site get an accurate summary and the right links.
• Writing the summary forces you to settle on one factual, one-line description of your company. That consistency helps entity understanding everywhere, not only in llms.txt.
• On docs and API sites, .md versions plus an llms.txt index make your content easier for coding tools to use correctly.

Costs and risks

• Drift. A stale llms.txt that quotes last year's pricing or plan names gives AI tools the wrong facts. Generate the file from your CMS, or put it on your release checklist.
• Opportunity cost. Time spent here isn't spent on crawl access, indexing and better pages.
• No measurable search upside. Google says Search ignores the file, and we haven't seen any engine document a citation effect.

A sensible rule: add one if you run a developer-facing product or docs site, or anywhere it takes under an hour and you can keep it current. Skip it if it would delay fixing pages that aren't indexed.

How to generate and validate an llms.txt file

You can write an llms.txt file by hand in about an hour, or let your CMS or docs platform generate it. Either way, check it before and after you publish. A simple process:

1. Pick the pages. Choose the 10 to 30 URLs that answer the questions prospects and users ask most: product, pricing, security, docs, comparisons. Leave out thin and outdated pages.
2. Write the H1 and summary. Use your company or product name as the H1. Write a factual two-sentence blockquote that matches your about page.
3. Group the links under H2s. Give each link a one-line note, and put long or secondary material under ## Optional.
4. Generate it if you can. The llmstxt.org site lists tools that generate llms.txt files, including Mintlify, GitBook, Yoast SEO, AIOSEO and Wix, plus plugins for VitePress and Docusaurus. On a custom stack, render the file from the same CMS fields you use for meta descriptions so it updates when pages change. That's how we serve ours.
5. Publish at /llms.txt and make sure the URL returns HTTP 200 as plain text.

There's no official validator; llmstxt.org doesn't name one. Run these checks instead:

• The file starts with exactly one H1, and the blockquote follows it.
• Every link resolves with a 200, isn't redirected to a login, and isn't blocked by robots.txt or marked noindex.
• Every list entry is a Markdown link with an optional note, under an H2.
• The facts in the summary match the live pages.
• Test it with an agent. Give an AI assistant only your llms.txt URL and ask it the five questions customers ask most. This is the test llmstxt.org itself suggests. Wrong or vague answers mean the summary or the link set needs work.

What matters more for AI citations

What matters more than llms.txt is everything that decides whether an engine can find, trust and quote your pages: crawl access, indexing, current content and clear, specific passages. Google puts it bluntly: "optimizing for generative AI search is optimizing for the search experience, and thus still SEO" (Google).

In order of impact:

1. Let the right crawlers in. Googlebot and Bingbot for search, OAI-SearchBot for ChatGPT search, PerplexityBot for Perplexity. Check your CDN and firewall too, not only robots.txt.
2. Get pages indexed. Google says a page "must be indexed and eligible to be shown in Google Search with a snippet" to appear as a supporting link in AI Overviews or AI Mode (Google Search Central). An unindexed page can't be cited, however good its llms.txt entry.
3. Keep the indexed version current. AI answers cite what's been crawled, and they lean toward pages that are kept current. If you updated a page but the engine hasn't re-crawled it, the old version is the one it can quote. This is the step FreshlyCited handles for Google: it checks which of your pages Google has indexed, pushes only the ones it hasn't, and re-checks, so good updates become eligible for AI Overviews and AI Mode sooner.
4. Write quotable passages. Self-contained answers, specific numbers with sources and direct definitions. In the original GEO study, adding quotations and statistics raised visibility in generative answers by roughly 30 to 41% in lab tests.
5. Earn mentions elsewhere. In an Ahrefs study of 75,000 brands (2025), brand web mentions correlated more strongly with AI Overview visibility than backlinks did (Ahrefs).

None of these steps guarantees a citation. Each engine picks its own sources. They are, however, the steps engines document, and llms.txt isn't one of them. For the content side, see our guide to LLM SEO. For the full picture, start with the AI search optimization guide.

If you've just published or updated the pages your llms.txt points to, confirm Google has actually indexed them. A visibility check shows which pages are seen by Google and which are invisible, and you can push the invisible ones in one step. Check your own links free.