llms.txt is a plain text file, written in Markdown, that usually sits at the root of a website and gives AI tools a short, curated summary of the site with links to its most useful pages. It was proposed in September 2024 as a way to help AI agents find the right content without wading through every page. It is not an official standard, and the major AI search tools have not said they use it, so treat it as a cheap extra, not a ranking lever.
Picture a large office building. robots.txt is the sign on the door that says which floors visitors may enter. A sitemap is the lobby directory that lists every room. llms.txt is the one-page cheat sheet a good receptionist hands a new visitor: here is who we are, and here are the few rooms you probably want.
That cheat sheet is one small part of being easy for AI tools to read. The rest of that work, from crawler access to the pages AI tools quote, is what our generative engine optimization services handle.
llms.txt is a proposal published at llmstxt.org by Jeremy Howard. It suggests that a website add a Markdown file at /llms.txt that gives large language models background on the site and a list of links to pages worth reading. Markdown is a simple way to format plain text with symbols, such as a # for a heading and a dash for a list item.
The proposal is aimed at AI agents doing a task for a person, such as a coding assistant looking up how a software library works, or a chat assistant reading about a product to answer a question. Its author writes that llms.txt was expected to be useful mainly when an AI tool answers a question, not when a model is trained.
Three things are worth knowing up front:
It is published as an informal overview with a public GitHub repository for discussion. No standards body has adopted it.
The proposal was revised in August 2026. More on that below.
The proposal itself says that is where llms.txt files are used most, because coding agents follow them to find API references and tutorials.
Web pages are built for people. The proposal points out that a typical page wraps its main content in menus, ads and scripts, and that turning that back into clean text is hard and imprecise. AI tools can also only hold a limited amount of text at once, so they cannot read a whole website in one go.
llms.txt tries to fix both problems with one small file. It hands an AI tool a short summary and a list of the pages that matter, ideally pointing to clean Markdown copies of those pages. The detail stays behind the links, and the tool fetches only what it needs.
The proposal sets out a specific order. Only the first item is required:
One H1 line with the name of the site or project, such as “# Your Business Name”.
A blockquote, a line that starts with “>”, holding a short description of what the site is about.
Plain paragraphs or lists with any extra context an AI tool needs to understand the links below. No headings in this part.
Each section starts with an H2 heading, such as “## Services”, followed by a list. Each list item is a Markdown link, then a colon and a short note about the page.
By convention, a section titled “Optional” holds secondary links an AI tool can skip when it is short on room.

The proposal also gives a few writing tips: use short, clear language, write a brief note for every link, avoid unexplained jargon, and test the file by asking an AI tool questions about your business with only the llms.txt file as its starting point.
Most published examples come from software companies. Here is a plain example for a service business. Every name and address in it is made up.
# Example Plumbing Co.
> Example Plumbing Co. repairs and installs residential plumbing. This file lists our main service and policy pages.
## Services
- [Water heater repair](https://example.com/water-heater-repair/): repair and replacement for tank and tankless heaters
- [Drain cleaning](https://example.com/drain-cleaning/): how we clear kitchen, bath and main line drains
## Policies
- [Service area](https://example.com/service-area/): the areas we cover and how scheduling works
- [Warranty](https://example.com/warranty/): what our work warranty covers
## Optional
- [About us](https://example.com/about/): our team and historyA good file for a business is short. It points to the pages that answer the questions customers ask most, written so each note makes sense on its own. It does not try to list every page on the site. That is the sitemap’s job.
They do three different jobs, and llms.txt replaces neither of the others:
It sets which crawlers may visit which parts of your site. The proposal is clear that llms.txt has a different purpose and is read on demand, when an AI tool needs information to help a person.
It covers every page you want search engines to find. The proposal notes that a sitemap usually holds far more than an AI tool needs and will not point to clean Markdown versions of pages.
It is a short, hand-picked guide to the pages that matter most, and it can also link to useful pages on other sites.

The practical takeaway: if a crawler is blocked in robots.txt or by your firewall, an llms.txt file will not open the door. Our GEO audit checklist starts with those access checks for exactly that reason.
Not as far as their own documentation says. Here is what the companies themselves publish:
says, on its page about AI features in Search, “You don’t need to create new machine readable files, AI text files, or markup to appear in these features.”
describe their crawlers and how to allow or block them in robots.txt. None of the crawler pages we read says the crawler reads llms.txt or uses it to choose sources.
for their own developer documentation. OpenAI, Anthropic and Google’s Gemini API docs all serve one. That tells you the format is useful for helping coding tools read documentation. It does not tell you their search products use your file to decide who to cite.
Lighthouse, the site testing tool in Chrome’s developer tools, now includes an llms.txt check in its agentic browsing audits. It flags a server error when it tries to fetch the file. If the file simply does not exist, the check is marked not applicable, and the Chrome docs say providing the file is optional for now.

So one part of Google tests for the file while Google Search says it is not needed for its AI features. Both can be true. Lighthouse is checking whether AI agents browsing your site will have an easier time. Google Search is telling you what it takes to appear in its results.
For what each AI tool does document, see our guides on how to rank in ChatGPT, how to rank in AI Overviews and Perplexity SEO, and the wider view in answer engine optimization.
There is no published evidence from Google, OpenAI, Anthropic or Perplexity that an llms.txt file improves rankings, citations or traffic. Google Search says outright that it is not needed for its AI features.
That does not make it worthless. It costs very little to publish, it can help AI agents that do read it find your best pages, and it forces a useful exercise: deciding which pages on your site best explain your business. If you struggle to pick a short list, that is a sign your site needs clearer pages, and fixing that helps every search tool.
What it will not do is make up for weak basics. Pages that crawlers cannot reach, text that only appears after scripts run, or thin answers will hold you back with or without the file.
The file goes at the root of your site, so it loads at an address like https://example.com/llms.txt. The proposal also allows a file inside a folder, such as /docs/llms.txt, that covers only the pages in that folder. When more than one file applies, AI tools are meant to use the most specific one.
Ways to create one:
Write it in any plain text editor, save it as llms.txt in lowercase, and upload it to your site’s root folder.
The proposal lists WordPress plugins Yoast SEO and AIOSEO as tools that generate an llms.txt file, and Wix says it creates one for every Wix site. Documentation platforms like Mintlify and GitBook serve one automatically.
Online generators build a first draft from your sitemap. Treat that draft as a starting point. A generator cannot tell which pages matter most to your customers, and it will not write good notes for each link.
After it is live, open the address in your browser to confirm it loads as plain text, then run Lighthouse to make sure the server returns it without an error.
The proposal was revised in August 2026. The changes page on llmstxt.org lists what is new:
Pages can now point to their llms.txt file and to a Markdown copy of themselves with standard link tags, either in the page code or in a response header set by the server.
A clean Markdown copy of a page can add .md to the full address or replace the file ending with .md.
A file in a folder covers the pages under that folder, and the most specific file wins.
AI tools are expected to read or search llms.txt, then follow the links they need. The “Optional” section stays as a convention, but it no longer has a special technical meaning.
For a typical business website, none of this changes the basic advice. A short root file with good links is still the whole job. The link tags and Markdown copies matter most for large documentation sites.
The proposal names a few good fits beyond software docs: a business setting out its structure and policies, a personal site answering questions about someone’s work history, or a school sharing course information. In our read, the file earns its keep on:
that coding tools and AI agents visit to look things up.
where a short guide to the best ones saves an AI agent real work.
such as returns, warranties or service terms, that customers often ask AI tools about.
For a small site with a handful of pages, the file is still cheap to add. Just expect the bigger gains to come from the pages themselves.
No AI search tool has said it ranks sites higher for having one, so anyone selling it that way is overselling.
The file is public and anyone can read it. Lines written only to steer AI tools toward recommending you add nothing a real visitor or a careful AI tool would value.
A file that copies your sitemap defeats the purpose. Pick the pages that matter.
It cannot. Access is controlled by robots.txt and your firewall.
A file that links to old pages or leaves out your newest ones points AI tools in the wrong direction. Update it when you add or retire key pages.
No one can guarantee that an AI tool will cite your business, with or without an llms.txt file.
If you can upload a file to your site, you can add an llms.txt file yourself. It is one of the easier technical jobs in AI search work. Getting help makes sense when the file is part of a bigger question: whether AI crawlers can reach your site at all, which pages deserve a spot in the file, and whether those pages answer buyer questions clearly enough to be quoted.
Generative Digital is run by one person and works with businesses across the US. Our AI SEO agency page explains how we handle SEO and AI search as a single program, with llms.txt as one small piece of it.
No. It is an optional proposal, not a requirement from any search engine or AI company. Google Search says you do not need new AI text files to appear in its AI features, and Chrome’s Lighthouse marks a missing file as not applicable.
It works as a guide for AI tools that choose to read it, mostly coding tools reading software documentation. No major AI search tool has said it uses llms.txt to pick the sources it cites, so do not expect it to change your AI search results on its own.
No. llms.txt only points AI tools toward content. To allow or block a crawler, use your robots.txt file and check that your firewall settings match.
llms-full.txt is a single file that holds the full text of a site’s key pages, offered by some documentation platforms such as Mintlify. It is not part of the proposal at llmstxt.org, so treat it as an optional extra for documentation sites.
Add /llms.txt to the end of the site’s home page address, such as https://example.com/llms.txt, and open it in your browser. If the site has one, it loads as plain text. If not, you will usually see the site’s error page.
Not sure whether AI tools can reach your site, let alone read an llms.txt file? Our free digital audit reviews how search engines and AI tools see your site right now and what to fix first.
Get startedWorking with businesses across the US
© 2026 Generative Digital LLC