- llms.txt is a proposed standard, a curated markdown file at your site root that tells AI crawlers what your site is about
- No major AI company has publicly confirmed its crawlers read the file or use it to pick sources
- Creating one takes about 10 minutes and carries zero risk, so do it, then move on
- It doesn't replace being cited where AI already looks, Semrush found 40.1% of LLM citations point to Reddit
llms.txt is a proposed standard: a plain markdown file at your site root that gives AI systems a curated summary of what your site is, what it does, and which pages matter most. Think robots.txt, except instead of telling crawlers what to skip, it tells language models what to read.
Should you bother? Yes, with clear eyes. Creating one takes about 10 minutes and can't hurt you. But no major AI company, not OpenAI, not Google, not Anthropic, not Perplexity, has publicly confirmed its crawlers use the file to shape answers. Cheap insurance, not a growth lever.
This guide covers the format, how to generate yours in minutes, and the real state of adoption in 2026. No hype in either direction.
What llms.txt is and where it came from
The standard was proposed in September 2024 by Jeremy Howard, co-founder of Answer.AI, and the spec lives at llmstxt.org. The problem it targets is real: your website is written for humans and for Google. It's full of navigation, scripts, pop-ups, and layout markup that language models have to burn context parsing before they get to the substance.
llms.txt fixes that by handing them a clean map. It's not a replacement for the files you already have, it's a third thing:
- robots.txt tells crawlers what they may not touch
- sitemap.xml lists every URL, with no opinion about which ones matter
- llms.txt is a curated, human-written summary: here's what we do, here are the pages worth reading, here's why
The proposal also defines an optional companion, llms-full.txt, which inlines the full content of your key pages into one giant markdown document so a model can ingest everything in a single fetch.
One nuance worth naming: this is a proposed standard. Nobody governs it, no engine is required to respect it, and the spec could evolve. That's exactly how robots.txt and sitemaps started too. It's also why the honest answer to "does it work" is more complicated than the LinkedIn posts suggest.
The format: one markdown file, four parts
An llms.txt file is ordinary markdown with a fixed structure. Here's a minimal, complete example:
# Acme Analytics
> Acme Analytics is privacy-first web analytics for indie SaaS
> founders. No cookies, one-line install, GDPR-compliant by default.
## Product
- [Features](https://acme.com/features): What's included, with screenshots
- [Pricing](https://acme.com/pricing): All plans and what each unlocks
## Docs
- [Quickstart](https://acme.com/docs/quickstart): Install and first report
## Optional
- [Blog](https://acme.com/blog): Guides on privacy-first analytics
Four parts, in order:
- An H1 with your site or product name. The only strictly required element.
- A blockquote summary. One short paragraph: what you are, who you're for, why you're different. This is the text a model is most likely to lift verbatim, so write it like positioning copy, not a mission statement.
- H2 sections with link lists. Each link gets a one-line description. Group by theme: product, docs, comparisons.
- An "Optional" section. A signal that these links can be skipped when the model is short on context.
That's the whole spec. If you can write a README, you can write an llms.txt.
How to create yours in 10 minutes
The file is simple enough to write by hand, and you know your positioning better than any scraper does. If you want the structure handled for you, our free llms.txt generator builds a spec-compliant file in your browser from a short form: fill it in, copy or download, done. Either way, here's the process:
- Write the blockquote first. One paragraph, plain language, no adjectives you can't defend. This is your elevator pitch to every AI system that reads the file.
- Pick 5 to 15 pages, not 500. Pricing, top features, docs quickstart, honest comparison pages. Curation is the entire point; a dump of every URL is just a worse sitemap.
- Give every link a one-line description. Say what a model will find there, not marketing copy.
- Save it as
llms.txtat your domain root, so it resolves atyourdomain.com/llms.txt. It's a static text file, any hosting can serve it. - Update it when your positioning or pricing pages change. A stale llms.txt that contradicts your live site is worse than none.
The mistakes we see most: fluffy summaries ("we empower teams to unlock growth" tells a model nothing), exhaustive link lists, and files created once in 2025 and never touched again.
Who actually reads llms.txt in 2026
Here's the part most guides skip. Adoption has two sides, and they look very different.
Publishing is real. A growing number of sites, especially developer tools and documentation, ship an llms.txt. Docs platforms like Mintlify generate the file automatically for the sites they host. Some AI companies even publish one for their own docs, Anthropic included. That tells you they like the idea for their content. It doesn't tell you their crawlers request yours.
Reading is unproven. No major AI provider has publicly documented that its crawler fetches llms.txt or that the file influences retrieval, ranking, or citations. Google's John Mueller has publicly compared it to the old keywords meta tag: a signal sites volunteer, that engines never committed to using. Server-log reports from site owners are mixed, some see AI user agents fetch the file, others never do. We haven't run our own crawler-log study, so we won't pretend otherwise.
As of 2026, no major AI company has publicly confirmed that its crawlers read llms.txt or use it to select sources. The upside is real but unproven. The cost is 10 minutes.
That's the whole risk profile. Weigh it accordingly.
The honest verdict: do it, then move on
Ship an llms.txt. Ten minutes, zero downside, and it guarantees your positioning exists somewhere in a clean, unambiguous, machine-readable format. If engines start honoring it, you're early. If they never do, you lost nothing. Generate yours free here if you'd rather not start from a blank file.
Just don't confuse it with a GEO strategy. AI answers are assembled from sources the engines already read and trust, and your website is only one of them. Semrush's analysis of LLM citations found 40.1% point to Reddit, the most-cited domain ahead of Wikipedia (Semrush, 2025). Our own 1,000-search study found Perplexity cited Reddit in 44.7% of sourced shopping answers and made it the very first source in 23.5% of them, while an engine running on a web index without Reddit access cited it 0 times in 500 answers.
An llms.txt polishes how machines read your site. It does nothing about what other people say about you in the places AI actually quotes. So the priority order looks like this:
- Be present in the sources AI cites, Reddit threads, comparison posts, community discussions. That's the core of generative engine optimization.
- Structure your own site answer-first, clear questions as headings, pricing stated plainly, comparison tables.
- Add llms.txt as the finishing touch. Last, because it's the cheapest and the least proven.
Then measure instead of guessing: run your brand through our AI visibility checker, or follow the manual process in how to check your AI visibility to see which sources engines actually cite for your category.
FAQ
Does llms.txt actually work?
Nobody can honestly say yes yet. The file is easy to publish and some sites see AI crawlers fetch it, but no major AI provider has confirmed it reads llms.txt or uses it to pick sources. Treat it as a 10-minute, zero-risk hedge, and put your real effort into the sources engines demonstrably cite, like Reddit at 40.1% of LLM citations (Semrush).
Is llms.txt the same as robots.txt?
No, they're close to opposites. robots.txt is a blocklist: it tells crawlers which paths they may not access. llms.txt is an invitation: a curated markdown summary of your most important pages, written so a language model can understand your site without parsing your HTML. You can, and should, have both.
Where do I put the llms.txt file?
At your domain root, so it resolves at yourdomain.com/llms.txt, the same location pattern as robots.txt. It's a plain text file, so any host can serve it: write it in any text editor, upload it like any static asset, and check that the URL returns the raw markdown.
What is llms-full.txt?
An optional companion file from the same proposal. Where llms.txt is a curated index of links, llms-full.txt inlines the complete content of your key pages into a single markdown document, so a model can ingest everything in one fetch. It's most useful for documentation sites; for a typical marketing site, the standard llms.txt is enough.
Paul-Marie is the founder of Readyt, the Reddit growth platform for SaaS. He has generated 16K€+ in pre-sales in 2 months using nothing but Reddit, and now helps founders turn Reddit threads into their #1 acquisition channel.
