AI Strategy
llms.txt Explained: Should You Add One to Your Website?
Updated Muhammad Taha6 min read
An llms.txt file is a Markdown file at your site root (yoursite.com/llms.txt) that gives language models a short summary and a list of your best pages. Jeremy Howard of Answer.AI proposed it in 2024. It is a proposal, not a standard, and major search engines have not confirmed they use it.
Key takeaways
- llms.txt is a convention, not an official web standard.
- Google representatives have said Google's AI features do not use it. Other providers have not made firm public commitments.
- Where it does help: developers and AI tools that fetch your docs on demand, such as coding assistants, can read it directly.
- It takes about ten minutes to write and costs nothing to maintain if you keep it short.
- Do not put anything private in it. It is public.
What is in an llms.txt file
The proposal asks for Markdown with a simple layout: a title, a one-paragraph summary in a quote block, then sections of links, each with a short description. An optional second file, llms-full.txt, contains longer text.
# Example Co
> Example Co builds accounting software for dental clinics in Pakistan. We offer setup, data migration and staff training.
## Services
- [Clinic accounting setup](https://www.example.com/services/accounting): What is included, timeline and who it suits.
- [Data migration](https://www.example.com/services/migration): How we move records from spreadsheets or old software.
## Guides
- [How much does clinic software cost?](https://www.example.com/blog/clinic-software-cost): Typical price ranges and what drives them.
## About
- [About us](https://www.example.com/about): Who we are, where we work and how to contact us.Does it work?
Be honest about this: there is no public proof that publishing llms.txt raises your chances of being cited by ChatGPT, Gemini, Perplexity or Claude. Search engines already crawl and index your pages, and they have sitemaps for discovery, so a second summary file adds little for them. Google has said it does not use the file.
It has real, if narrow, use in a few places:
- Developer tools. AI coding assistants and documentation tools can be pointed at an llms.txt to read a product's docs quickly. Many developer-focused companies publish one for this reason.
- Agent workflows. If someone builds an AI agent that needs to understand your site, a clean index file saves it work.
- A forcing function for you. Writing the file makes you decide which ten pages best explain your business. That clarity is useful on its own.
llms.txt vs robots.txt vs sitemap.xml
| File | Purpose | Status |
|---|---|---|
| robots.txt | Tells crawlers which paths they may or may not fetch | Established, widely honoured by reputable crawlers |
| sitemap.xml | Lists URLs you want crawled and when they changed | Established, supported by all major search engines |
| llms.txt | Gives language models a readable summary and a curated link list | Proposal, uneven and unconfirmed support |
llms.txt does not control access. It cannot block or allow anything. To manage which AI crawlers may visit, you need robots.txt. See our robots.txt guide for AI crawlers.
How to write a good one
- Write the summary like you would say it to a new customer. What you do, for whom, where.
- List 10 to 30 links, not 300. Pick pages that explain your business best: services, pricing approach, key guides, About, contact.
- Add a one-line description per link that says what the reader learns, not a keyword.
- Use plain text and Markdown only. No scripts, no HTML.
- Keep facts identical to your website and schema markup. Mismatches hurt trust.
- Review quarterly, or whenever services or pricing change.
Where to put it
At the root: https://yoursite.com/llms.txt. In Next.js, drop the file in the public folder. In WordPress, upload to the site root through your host's file manager or with a plugin. Confirm it loads in a browser and returns plain text. Growmerz publishes one at growmerz.com/llms.txt, and a longer version at llms-full.txt.
The verdict
Publish one if you have a documentation-heavy site or want a tidy summary for tools that fetch pages. Skip the anxiety if you do not. The work that moves results is the same as before: clear answers, accurate structured data, a consistent identity and mentions on trusted sites. Our AEO and GEO overview covers the full picture.
Common llms.txt mistakes
- Wrong filename. The convention is
llms.txt, with an s. On growmerz.com we redirect/llm.txtand/llm-full.txtto the correct names because the singular form is a common slip. - HTML instead of Markdown. The file should be plain text. A rendered page or a login redirect defeats the purpose.
- Listing every page. Three hundred links is a sitemap. Keep it to the pages that explain your business best.
- Facts that differ from the website. If llms.txt says one founding year and your About page another, you have created a contradiction.
- Treating it as access control. It cannot block or allow anything.
Test that it works
Fetch the file the way a tool would, and confirm you get Markdown text and a 200 status, not a redirect to your home page:
curl -i https://www.example.com/llms.txtCheck that the content type is plain text or Markdown, that links are absolute URLs, and that every linked page returns 200. Then check your server logs after a few weeks to see whether anything requests the file. Do not assume it is being read just because it exists.
What we publish
Our own file at growmerz.com/llms.txt summarises Growmerz in a short paragraph and links to our services, key guides and About page. The longer llms-full.txt holds fuller text for tools that want everything in one fetch. We keep both in step with the website and update them when services change. We publish them because they cost little, and we are clear in this guide that we cannot show they raise AI visibility.
A sensible way to decide
| Situation | Recommendation |
|---|---|
| Documentation-heavy product or developer tool | Publish llms.txt and consider llms-full.txt |
| Service business with a small site | Optional, ten minutes of work, low priority |
| Site with weak basic SEO | Fix that first: clear pages, structured data, access for crawlers |
Related guides
- robots.txt for AI Crawlers: GPTBot, ClaudeBot, PerplexityBot and Google-Extended
- Google AI Overviews: How to Get Your Content Cited
- AI Agency vs Freelancer vs In-House Team: Which Should You Choose?
- Entity SEO for AI: How Search and AI Decide Who Your Business Is
- Service: AEO and GEO: Get Found by AI Search
Frequently asked questions
Is llms.txt an official standard?
No. It is a community proposal. There is no governing body that requires or enforces it.
Will llms.txt improve my Google ranking?
No evidence supports that, and Google has said it does not use the file.
Can llms.txt block AI from using my content?
No. It only provides information. Use robots.txt and, where needed, server rules to control access.
Should the file list every page?
No. Curate it. A long list defeats the purpose, which is to point to the pages that explain your business best.
What is llms-full.txt?
An optional companion file with the full text of key pages in one document, for tools that want everything in a single fetch.
Where did the llms.txt idea come from?
Jeremy Howard of Answer.AI proposed it in 2024, and the specification and examples are published at llmstxt.org.
Sources and further reading
Primary documentation and research behind this guide.
If you want help deciding whether an llms.txt file is worth it for your site, we can review that with the rest of your AI visibility basics. Book a free audit or see our AEO and GEO service.