Guide for AI crawlers, content paths, and GEO
llms.txt is a community proposal for a Markdown file that points language models to useful pages. It is not a Google Search ranking requirement and does not control crawler access.
Plan the page cluster
An llms.txt file can help tools that support the proposal find your most useful public pages. Keep its links current, and use robots.txt or access controls to restrict crawling.
What to review
- Suggested file sections
- Priority page list
- Docs and policy links
- Crawler notes
- Content freshness checks
- GEO integration
Where to start
- List the public pages AI systems should understand first.
- Write a concise llms.txt file with URLs and notes.
- Keep the file aligned with your sitemap and content updates.
Who this helps
- Developers supporting marketing teams
- SEO teams documenting AI-friendly pages
- Product teams exposing docs and policy pages
Questions about this approach
It is a proposed Markdown file at your website root that lists useful pages and gives language models concise context about your site.
No. Google says llms.txt is not used by Google Search. Keep your pages crawlable, accurate and useful; treat the file as optional context for tools that choose to support it.
The common pattern is the site root, similar to robots.txt, so it can be found at /llms.txt.
Include important URLs, short descriptions, documentation links, product facts, and pages that explain the brand.
Use robots.txt and crawler controls for blocking. llms.txt is more about guidance and context.
Update it when product pages, documentation, policy pages, or major content hubs change.
Put the guide to work