Do the big AI assistants actually read llms.txt today?

As of 2026, none of the major assistants has publicly committed to llms.txt as a ranking or retrieval input, and support is inconsistent. Treat it as a cheap, forward-looking convention rather than a guaranteed pipeline. The load-bearing work is still crawlable content, structured data, and explicit crawler permissions.

Is llms.txt the same as robots.txt?

No — they do opposite jobs. robots.txt tells crawlers what they may not access. llms.txt is an invitation and a map: it points assistants at the cleanest version of your most important content. You want both files, and you want them to agree with each other.

Do I need llms-full.txt as well?

It depends. llms-full.txt concatenates the full text of your key pages into one file so an assistant can ingest everything at once. It is genuinely useful for documentation and reference sites and overkill for a small marketing site. I ship both here because when they are generated from source, the second file is nearly free.

Where exactly does the file go?

The web root: yourdomain.com/llms.txt, same place as robots.txt. If your platform will not let you serve a file at the root — some hosted builders do not — you route it through an edge worker or reverse proxy so it resolves at the right path.

What Is llms.txt (and Do You Need One)?

llms.txt is a plain-text file you place at the root of your domain — yourdomain.com/llms.txt — that hands AI assistants a clean, curated map of your site in Markdown, instead of making them reconstruct it from your HTML. Think of it as a table of contents written for a machine that reads.

It is one of the more hyped ideas in "AI SEO" right now, which means it is worth being precise about what it is, what it is not, and whether you actually need one. Here is the practitioner version.

Where it came from

llms.txt was proposed in 2024 by Jeremy Howard of Answer.AI as a convention — a shared idea, not a rule handed down by OpenAI or Google. The premise: a modern web page is mostly navigation, scripts, and layout, and an assistant working under a limited context window wastes most of it parsing chrome to find the few paragraphs that matter. A curated Markdown index fixes that by pointing straight at the signal.

What actually goes in it

The format is intentionally simple: an H1 with your site or project name, an optional one-line summary, then Markdown sections of links — each link ideally pointing at a clean Markdown version of the page, with a short description of what it is.

# Your Company

> One sentence on what you do and who for.

## Core pages

- [Services](https://yourco.com/services.md): what you offer, and pricing
- [About](https://yourco.com/about.md): who you are, and proof

## Reference

- [FAQ](https://yourco.com/faq.md): the questions buyers actually ask

A companion file, llms-full.txt, takes it further: the full text of your key pages concatenated into one document, so a model can pull everything in a single fetch. Useful for documentation-heavy sites; overkill for a small brochure site.

A real example (this site’s)

Here is the top of the llms.txt this site actually serves — generated at build time from the same content that renders the pages, so it never drifts:

# sukisong.com — LLM index

Suki Song builds AI-discoverable websites — found by Google and cited by ChatGPT, Claude, and Perplexity (SEO + GEO as one system).
…

## Machine-readable pages
…
- [About](https://sukisong.com/about.md): A builder, a writer, a solo traveler. Currently thinking about how engineering teams ship at AI pace.

It is short on purpose. A good llms.txt is a curated menu, not a data dump.

When it helps — and when it doesn’t

It helps most when you have real depth an assistant benefits from navigating: documentation, a product with many features, a knowledge base, a services business with several distinct offers. Paired with per-page Markdown twins, it gives models a fast, clean path to your best content.

It helps least — bordering on pointless — if your pages are not crawlable in the first place, or if you have three pages total. A file that points at content a crawler is already blocked from reading does nothing. llms.txt is a convenience layer on top of a discoverable site, not a substitute for one.

How to write a good one

Curate, don’t dump. Link your most important pages, not every URL. If it reads like a sitemap, you have missed the point.
Point at Markdown, not HTML. Serve a .md version of each linked page and link that; the whole value is giving the model clean text.
Write descriptions for a reader. One useful sentence per link — what the page is and who it is for — not keyword soup.
Generate it, don’t hand-maintain it. If you can build it from the same source as your pages, it can never fall out of sync. A stale index is worse than none.
Put it at the web root, next to robots.txt, and make sure your robots.txt actually lets the AI crawlers in.

Common mistakes

Treating it as a ranking switch. It is not a confirmed input to any assistant’s ranking or retrieval today. Ship it as a low-cost, forward-looking convention.
Linking HTML pages with no Markdown twin — you send the model right back into the noise you were trying to avoid.
Writing it once and letting it rot as the site changes.
Shipping an llms.txt while your robots.txt still blocks GPTBot, ClaudeBot, or PerplexityBot — an invitation on one door and a lock on the other.
Dumping your entire content library into llms-full.txt when a curated index would have served the model better.

llms.txt is one layer of a larger stack. For how it sits alongside Markdown twins, a resolvable Schema.org graph, and crawler permissions — with the real snippets — read the full teardown of this site’s stack.

I generate llms.txt (and its twins) from one source on every site I build, so it stays honest and current. If you want it on your site without the redesign, that is Get Found.

What Is llms.txt — And Does Your Website Actually Need One? (2026)