Guide
llms.txt explained
llms.txt is a root-level plain-text file that helps AI systems understand what your site is, which pages matter, and where to find deeper documentation.
What llms.txt is
Think of llms.txt as an orientation memo for language models and AI crawlers. It usually lives at /llms.txt and lists your product summary, important URLs, and optional pointers to longer material such as/llms-full.txt.
OpenCrawl publishes its own files at /llms.txt and/llms-full.txt, and audits customer sites for the same discovery signals.
What llms.txt is not
- Not a substitute for robots.txt crawl policy
- Not a ranking guarantee for ChatGPT, Claude, Perplexity, or Google AI
- Not a replacement for a sitemap, metadata, or readable page content
- Not useful if it links to pages that do not exist
How it fits with other signals
| Signal | Role |
|---|---|
| robots.txt | Says which crawlers may fetch which paths |
| sitemap.xml | Lists canonical public URLs |
| llms.txt | Explains the site to AI systems in plain language |
| Meta + JSON-LD | Describes each page for search and rich results |
Practical checklist
- Write a concise product summary without marketing hype
- Link only to real, indexable public pages
- Add a longer
llms-full.txtwhen the short file is not enough - Keep claims aligned with pricing, features, and legal pages
- Re-check after major product changes
FAQ
Does llms.txt guarantee AI citations?
No. llms.txt is a discovery and orientation file for AI systems. Useful sites still need crawlable content, clear metadata, sitemaps, and relevant public pages.
Where should llms.txt live?
Publish it at the site root as /llms.txt. Optionally add /llms-full.txt for a longer summary, and reference both from robots.txt or your documentation.