What Is llms.txt and Does Your Business Website Need One?
By Prashant Kumar Pandey, Founder, Digital Prashant Published 20 Nov 2026 · 7 min read
Quick answer
llms.txt is a proposed Markdown file at a site's root that gives AI tools a short summary and links to key pages. Google Search ignores it, so it won't help or hurt rankings, and few AI crawlers request it. It's worth adding for software and documentation sites; most local businesses should fix their core pages first.
"Are we losing AI traffic because our site has no llms.txt file?" Business owners ask us this more and more. It's a fair question. Plugins now offer to generate one, SEO tools flag it as missing, and plenty of posts call it essential. So what is llms.txt, who actually reads it, and should a business website bother?
Short answer: it's a proposed standard, it's easy to add, and Google Search ignores it. For most small business sites, it's a "nice to have" at best, and there are better uses of your time.
What is llms.txt?
llms.txt is a plain text file, written in Markdown, that sits at the root of a website (yoursite.com/llms.txt). It gives AI tools a short, clean summary of what the site is about and links to the pages that matter most.
It was proposed by Jeremy Howard of Answer.AI on 3 September 2024. The proposal's own description is "a proposal to standardise on using an /llms.txt file to provide information to help agents use a website." The idea: web pages are full of menus, scripts and ads, and AI models work better with a short, tidy summary that points to the useful content.
The format is simple:
An H1 with the name of the site or project. This is the only required part.
A short summary in a blockquote.
Optional paragraphs with more detail.
H2 sections with lists of links, each with the page name, its URL and a short note.
An optional section literally called "Optional", for links an AI can skip when it needs a shorter context.
The proposal also suggests offering clean Markdown versions of pages at the same URL with .md added.
For example, a dental clinic's file would start with the H1 "Sharma Dental Clinic", then a one-line summary such as "Family dental clinic offering check-ups, implants and braces, with online booking." Below that, a "Services" section would link the implants page and the braces page, each with a short note ("cost, process, recovery"), and an "Optional" section could link the blog.
Does Google use llms.txt?
No. Google has been unusually direct about this. In its guidance on optimizing for generative AI features in Search, Google lists llms.txt among the things you don't need. In June 2026 Google added a note saying it's "completely fine" to create and maintain llms.txt files for other services, but that doing so "won't harm (nor help) your visibility or rankings in Google Search, as Google Search ignores them." The update also says Google Search itself doesn't use them.
That covers AI Overviews and AI Mode too, since they're part of Google Search. Google's position on those features hasn't changed: there are no special files or markup required to appear in them. We covered Google's approach in more detail in our post on AEO and GEO for AI answers.
There's one twist. According to Search Engine Journal, Chrome's Lighthouse 13.3 added an experimental "Agentic Browsing" category that includes an llms.txt audit. That audit is separate from Lighthouse's SEO checks and doesn't affect rankings or AI citations. So if a report flags "llms.txt missing", don't treat it as an SEO problem.
Who actually reads llms.txt?
This is where the hype and the data part ways.
Adoption among tech sites is real. Ahrefs looked at 137,000 domains in its analytics data and found 28% publish an llms.txt file. Ahrefs notes the sample leans toward tech-savvy sites, so adoption across the wider web is likely lower.
Requests are rare. Of the roughly 38,000 domains with a valid file, Ahrefs found 97% received zero requests for it in May 2026.
No major AI provider has committed to it. Ahrefs' assessment, updated June 2026: "no major LLM provider currently supports llms.txt." Some AI companies publish llms.txt files for their own developer documentation, which is a different thing from their crawlers reading yours.
Where llms.txt can help today is narrower: developer docs, software products and APIs, where coding assistants and AI agents may be pointed at the file directly by a user.
llms.txt vs robots.txt: what's the difference?
People often mix these up, so here's a side-by-side.
robots.txt
llms.txt
Purpose
Tells crawlers which URLs they may or may not crawl
Gives AI tools a summary and a list of key pages
Status
Long-established, widely respected standard
Proposed standard (2024), voluntary
Can it block AI crawlers?
Yes, by naming each crawler's user agent
No, it doesn't control access
Used by Google Search?
Yes
No, Google Search ignores it
Format
Plain text rules (User-agent, Disallow)
Markdown headings, summary and links
Effort
Minutes, but mistakes can block your whole site
About an hour
If your goal is to stop AI companies from crawling your content, llms.txt won't do it. Use robots.txt with the crawler names each AI company publishes, and check that you don't block Googlebot by mistake.
Should your business website have an llms.txt file?
Our take, by type of site:
Local service businesses (clinics, contractors, agencies, shops): optional. It won't hurt, but it won't move your Google rankings either. Spend the time on your service pages and Google Business Profile first.
E-commerce stores: low priority. Product feeds, structured data and clean category pages matter far more.
SaaS, developer tools and documentation sites: worth doing. Users and coding assistants may read it, and a good one makes your docs easier to use with AI.
Publishers and content-heavy sites: optional. It can be a tidy index, but it's no substitute for strong internal linking.
If you decide to add one, keep it honest and short, link only your most useful pages, and update it when those pages change. A stale llms.txt is worse than none.
How to create an llms.txt file in 5 steps
List your 5–15 most important pages: core services, pricing, contact, key guides.
Write a one-line summary of what your business does and who it serves.
Build the file with an H1 (your name), a blockquote summary and H2 sections of links, each with a short note.
Upload it to your root folder so it loads at yoursite.com/llms.txt. Some SEO plugins can generate one, but check what they list.
Check it in a browser and add a reminder to review it whenever you add or remove a key page.
What helps AI search visibility instead?
If the real goal is to show up in AI answers, these matter more than any file:
Clear answers on your pages. Put a short, direct answer near the top of each important page, then the detail.
Accurate business facts everywhere. Same name, address, phone and services on your site, Google Business Profile and directories.
Pages Google can crawl and index. Check the Page indexing report in Search Console.
Original, experience-based content. It's what earns links, mentions and trust over time.
Measurement. Track how often AI tools actually send you visitors before investing more. Our guide on how to measure AI search traffic shows how to do it in GA4 and Search Console.
Treating llms.txt as a ranking factor. Google Search ignores it.
Using it to block AI crawlers. It can't; use robots.txt.
Paying for "llms.txt optimization" packages. It's a simple text file you can write yourself.
Linking every page. The point is a short, curated list.
Editing robots.txt in a hurry. One wrong line can block Googlebot from your whole site.
Forgetting to update it. Broken or outdated links send AI tools to the wrong place.
Next steps
If you're a local business, put llms.txt at the bottom of your list and fix your key pages first. If you run a software or docs site, add one this week. Not sure where your site stands on AI search? Book a free 30-minute strategy call and we'll look at it with you.
llms.txt is a Markdown text file placed at the root of a website to give AI tools a short summary of the site and links to its most important pages. Jeremy Howard of Answer.AI proposed it in September 2024. It's a voluntary, proposed standard, not an official web standard, and it doesn't control which pages AI crawlers can access.
Does Google use llms.txt?
No. Google says Google Search ignores llms.txt files, so adding one won't help or harm your visibility or rankings, including in AI Overviews and AI Mode. Google also lists llms.txt among the things you don't need for generative AI features in Search. You can still create one for other tools, but don't expect any Google ranking benefit.
Should a small business add an llms.txt file?
It's optional for most small businesses. It takes about an hour, does no harm and may help a few AI tools, but it won't change Google rankings. Most local businesses get more from clear service pages, accurate Google Business Profile details and indexable content. Software and documentation sites benefit more, because AI coding tools may read the file.
Is llms.txt the same as robots.txt?
No. robots.txt tells crawlers which URLs they may or may not crawl, and Google Search follows it. llms.txt only offers a summary and a list of key pages for AI tools; it can't allow or block anything. If you want to stop AI companies from crawling your site, use robots.txt with the crawler names those companies publish.
What helps AI search visibility instead of llms.txt?
Good SEO basics. Write clear, direct answers near the top of important pages, keep business details consistent across your site and listings, make sure Google can crawl and index your pages, and publish original content based on real experience. Then measure AI referrals in GA4 and Search Console so you know whether the work is paying off.
Prashant Kumar Pandey
Founder of Digital Prashant and a performance marketer since 2018. He has generated 10,000+ leads with Meta Ads and managed ad budgets up to ₹40 lakh a month. View profile.