What is llms.txt?
llms.txt is a plain-text file that summarises a website for AI agents and links to its key pages. The format is still an open proposal, so publishing it is optional and support varies by tool.[1][7]
The file uses Markdown, a plain-text format for headings and links. You can open a published file directly in your browser by adding /llms.txt to the domain.[1]
What does an llms.txt file contain?
The proposal starts with the website or project name as the main heading. You can add a short description, group links by topic and describe what each page covers.[1]
This shortened extract from Shopify's public llms.txt shows how the file groups links to help pages under headings.[13]
# Shopify
## Sell Internationally
- [International settings](https://help.shopify.com/en/manual/international)
## Need help?
- [Shopify Help Center](https://help.shopify.com/en/)Does your website need llms.txt?
Give an AI assistant your website address and ask it a question visitors already ask. If it reaches the right page and gives the correct answer, you can skip the file for that task.
If the assistant can read the page when you give it the link but struggles to find it from the homepage, try a short index. If the destination page leaves out part of the answer, add that information to the page first.
Does llms.txt improve Google Search visibility?
Google says Search ignores llms.txt when determining visibility and rankings, including in its generative AI features.[2]
To qualify as a supporting link in AI Overviews or AI Mode, a page must be indexed and eligible to appear with a search snippet, according to Google.[5]
Which tools actually use llms.txt?
Context7 processes the file when you submit its URL. Whether an assistant finds the file on its own depends on how it looks for information.[6]
Lighthouse flags server errors when it tries to fetch llms.txt. If the server returns a 404, Lighthouse marks the audit as "Not Applicable" because the file is optional.[7]
How llms.txt differs from a sitemap and robots.txt
| File | Purpose |
|---|---|
| llms.txt | Points AI tools to selected pages and describes what they cover.[1] |
| robots.txt | Sets crawl rules for bots that honour them.[9] |
| XML sitemap | Lists the URLs you want search engines to discover.[10] |
| llms-full.txt | Some publishing platforms use this file to combine the text of several pages into one document.[8] |
Check the documentation for each crawler before setting rules in robots.txt. Protect private pages with access controls.[9]
How to create an llms.txt file
Copy the template into a plain-text editor and replace the site name, headings and links with your own. Start with public pages that answer common questions about your site. Give each link a short description that explains what the page covers. Save the file as llms.txt.[1]
# Website name
> Describe what the site covers.
## Topic
- [Page title](https://example.com/page): Say what this page explains.Publish the file on your site
For a site-wide index, publish the file at /llms.txt on your domain. The original proposal allowed files in subdirectories, and version 2 defines their scope. For example, /docs/llms.txt covers pages under /docs/. When several files cover the same page, the one in the deepest directory on its path takes precedence.[3]
On WordPress, check whether a plugin you already use can publish an llms.txt file. If you have access to your site's files, you can upload your own file to the public folder for your domain using your hosting control panel's file manager or SFTP.[14]
If you deploy a site from a project, add llms.txt to the folder your build publishes as static files and redeploy. If you use a site builder, check whether its settings can generate the file or let you publish a text file at /llms.txt.
Open /llms.txt on your domain and check that it shows the text you published. Follow the links while signed out to make sure the pages are public. If you see your homepage or a 404, check where your platform published the file.
Check that tools can read the pages you list
If a visitor can see the answer in a browser but an assistant misses it, check what that assistant receives. Depending on the tool, it may read the HTML, inspect a screenshot or work from a description of the page's controls.[4]
Put the answer in visible text and keep structured data consistent with it. If a condition matters, such as when a policy takes effect, include it next to the detail it qualifies.[5][11]
How to test whether the file helps
Ask the same question in two fresh conversations with the same assistant. Give it your website address in both and the llms.txt link in the second. Keep the settings and website content the same so you can see what changes when you provide the file.
Check which pages the assistant opened and compare the answer with what those pages say. If your logs show a crawler request, use the provider's published guidance to verify its identity before drawing a conclusion.[12]
| If you see this | What it tells you |
|---|---|
| The tool opened the index after you supplied the link. | It could fetch the index in that run. |
| The tool found the index when you gave it only the website address. | You observed automatic discovery in that run. |
| The tool opened the right pages and gave a correct answer. | It found the information it needed for that question. |
Repeat the comparison before relying on the result. Keep the file if it helps the tool reach the right pages and you can maintain the links. Measure search visibility or referral traffic separately if either is your goal.
Keep the index current
Update the index when a linked page moves or its description stops matching the content. If your publishing platform generates the file, check it after changing how pages are organised.
Sources
- Jeremy Howard, "The /llms.txt file, v2". Originally published 3 September 2024. Revised on 10 August 2026.
- Google Search Central, "Optimizing your website for generative AI features on Google Search". See the section on llms.txt and other special markup. Page last updated 10 July 2026.
- llms-txt, "Changes". Version 2, August 2026.
- Kasper Kulikowski and Omkar More, "Build agent-friendly websites", web.dev. Last updated 1 April 2026.
- Google Search Central, "AI features and your website".
- Context7, "Add an llms.txt file". API documentation for explicit submission and processing.
- Chrome for Developers, "llms.txt", Lighthouse agentic-browsing audit. Last updated 5 May 2026.
- Mintlify, "llms.txt". Documentation for its generated indexes and combined exports.
- IETF, RFC 9309, "Robots Exclusion Protocol". September 2022. See sections 1 and 3 for the distinction between crawl rules and access authorisation.
- Google Search Central, "Learn about sitemaps".
- Google Search Central, "General structured data guidelines".
- Google Crawling Infrastructure, "Verify requests from Google crawlers and fetchers".
- Shopify, public llms.txt file
- WordPress Advanced Administration Handbook, "Updating WordPress using FTP". See the guidance on using a file manager or SFTP.


