Picture this: your team spends hours crafting an llms.txt file, formats it perfectly, and drops it at your domain root. You expect AI assistants to follow that roadmap, and cite your best pages. The reality is blunt: Ahrefs measured a 97 percent chance that an llms.txt file receives zero reads, a statistic that should change how you budget time and engineering effort.
The term llms.txt read rate crops up in every GEO checklist. If your motivation is AI citations and referral traffic, the raw numbers say the file is low impact right now. Use the signal, but do not bet your program on it. Instead, move effort toward content architecture, canonical assets, and monitoring that actually influence whether AI assistants find and recommend your brand.
Key takeaways
- Ahrefs found a 97 percent probability that an llms.txt file gets no reads, so treat the file as experimental and low priority.
- Publish an llms.txt only if your CMS does it for you, and keep maintenance minimal unless logs show meaningful reads.
- Prioritize canonical assets, structured machine readable pages, and a monitoring plan to drive AI citations and referrals.
- Use server logs, bot traces, and AEO trackers to decide whether to scale llms.txt work or reassign resources.
Know where your brand stands in AI search
Track how ChatGPT, Perplexity and Gemini mention and recommend your brand compared to competitors, and get actionable steps to improve your AI visibility.
Book a demo →What is the current llms.txt read rate?
Answer: The observed llms.txt read rate is near zero for most sites, with Ahrefs finding 97 percent of files received no hits in their sample.
Ahrefs analyzed 137,210 domains and reported that the overwhelming majority of llms.txt files attracted no readers, a finding that directly measures the llms.txt read rate on real sites. That same research also showed that 28 percent of their sample published an llms.txt file, so adoption outpaced actual consumption. The study is public and details the methodology and practical advice on how to proceed if you test the file yourself https://ahrefs.com/blog/llmstxt-study. A separate piece from Ahrefs summarizes adoption and context, including the 28 percent presence figure https://ahrefs.com/blog/what-is-llms-txt.
What does a near zero read rate mean for you? It means llms.txt is currently advisory and not a reliable distribution channel. Multiple industry monitors report negligible crawler interest. For example, SEMrush logged zero visits from major AI bots to their llms.txt during a multiweek observation window, which supports the claim that popular AI crawlers do not treat the file as authoritative today https://www.semrush.com/blog/llms-txt. Still, a low read rate does not make llms.txt worthless. If your user base includes developer tooling, coding agents, or you operate extensive docs, an llms.txt can help when agents are explicitly routed to it.
Should I prioritize llms.txt given the llms.txt read rate?
Answer: No, not as a primary investment. Prioritize higher impact work and treat llms.txt as a low cost experiment or a CMS default.
Costs are low for publishing an llms.txt. Many site builders already generate one automatically, and platforms like Wix and Framer are starting to adopt support, so the marginal effort can be near zero. Ahrefs explicitly recommends keeping the effort minimal and checking your logs before investing more https://ahrefs.com/blog/llmstxt-study. If your CMS will host it for free, include it and move on. If it requires engineering time, delay that work until you have evidence of returns.
How to decide where llms.txt sits in your roadmap. First, check server logs and bot analytics for reads. Second, ask whether your audience includes developer or coding agents that rely on machine readable docs. Third, consider the opportunity cost. If llms.txt maintenance would delay creating canonical assets, structured data, or a content inventory, reallocate to those items first. Tools to validate assumptions include server access logs, Cloudflare or CDN logs, and bot detection query strings. If you detect agent traffic or increased reads from relevant endpoints, move llms.txt up the priority list and automate regeneration paired with your documentation pipeline.
Focus on content architecture, not file flags
Answer: Structural signals like canonical pages, clean markdown endpoints, and schema markup matter more to AI retrieval than a single root file.
AI systems and RAG pipelines prioritize concise, high quality sources when answering queries. If you control a canonical asset that answers a common prompt, that page will be a candidate for retrieval regardless of whether an llms.txt exists. Practical steps start with inventory and canonicalization. Run a crawl with Screaming Frog, Sitebulb, or Ahrefs to map duplicate content and canonical conflicts. Consolidate topic clusters into a single canonical asset per intent, and apply canonical tags and consistent URL structure.
Structured data and machine friendly formats
Implement JSON LD for product details, FAQPage for common questions, and schema for events and articles. Provide Markdown or text exports for longform documentation so retrieval systems can fetch clean content without heavy HTML noise. Where possible, expose a small set of prioritized markdown pages at predictable URLs, for example /docs/topic.md or /guides/topic.md, which aligns with recommendations from the llms.txt proposal to link to LLM friendly content.
Tradeoffs and timeline
This work takes more time than dropping a single file, but it also increases the probability that retrieval systems find a quality source. Expect canonicalization and schema rollout to require cross functional work between product, content, and engineering teams and to show measurable improvements in weeks to months, not hours. Prioritize the top 20 pages driving leads, then expand.
How to build canonical assets and machine readable pages for AI
Answer: Identify high intent queries, build single source canonical pages, and publish machine readable versions for agents to fetch on demand.
Step 1: Audit intent and traffic. Use query logs, GA4, and your CRO experiments to list the top 20 user intents that drive conversions or brand recognition. Step 2: Create one canonical page per intent with a short summary, clear metadata, and a single canonical URL. Keep summaries under 300 words for quick retrieval, but link to the full resource for depth.
Step 3: Publish machine readable versions
Export each canonical asset as Markdown and host it at a predictable path that an agent can fetch. The llmstxt.org proposal and community examples show linking to markdown pages from a small index file. This pattern helps retrieval systems that prefer clean text or markdown. If you cannot serve public markdown, provide a well structured HTML with minimal boilerplate and clear headings so RAG systems can chunk text reliably.
Step 4: Add schema and internal signals
Mark up the canonical page with article, product, or FAQ schema as appropriate. Add internal linking from hub pages and use prominent CTAs to reinforce the page role. Track crawl frequency and referral hits to those canonical assets. If you find repeated citation behavior from an assistant, raise the page priority and maintain an llms.txt entry that points to it, but only after you see evidence that agents fetch such pointers.
Monitoring, measurement, and when llms.txt matters
Answer: Use server logs, bot identification, and AEO monitoring to measure reads, then scale llms.txt work only when you see agent traffic.
Start with server logs. Search for requests to /llms.txt and to your machine readable assets. Identify user agents and request patterns. Many reports show negligible reads, but if you detect targeted hits from known agent user agents, that is a green light to invest more. Ahrefs recommends checking your own logs first, because the base rate of zero readership is so high that most teams will find no reads at all https://ahrefs.com/blog/llmstxt-study.
Metrics to track
- Reads of /llms.txt per week, segmented by user agent.
- Fetches of machine readable pages listed in llms.txt, and downstream referral traffic.
- Share of AI citations, measured by AEO trackers or manual prompt sampling.
- Conversion rate for traffic referred by AI assistants compared with organic search.
Set a threshold for action. If your /llms.txt sees consistent, recognizable agent hits and those hits link to content that drives traffic or conversions, automate updates and integrate llms.txt building into your docs pipeline. Otherwise, keep llms.txt as a lightweight safety net and focus budget on canonical assets and monitoring that produce measurable results.
FAQ
What is the llms.txt read rate right now?
The llms.txt read rate is effectively near zero for most domains. Ahrefs studied 137,210 domains and reported a 97 percent chance that an llms.txt file received zero reads. That finding means agents are not routinely fetching the file across a broad sample https://ahrefs.com/blog/llmstxt-study.
Does llms.txt improve my chances of being cited by ChatGPT or Perplexity?
Not reliably. Major AI providers have not committed to treating llms.txt as a ranking signal and independent log audits, such as those reported by SEMrush, show little to no bot activity on llms.txt files. Focus on content quality, canonical pages, and machine readable formats first.
When should I publish llms.txt?
Publish it if your CMS does it automatically, or if you serve extensive documentation and developer audiences. Otherwise, publish a minimal file and revisit only if your logs show agent reads. Treat it as a low cost experiment, not a core strategy.
How do I test whether agents read my llms.txt?
Check server and CDN logs for requests to /llms.txt and to the markdown pages referenced from it. Filter by user agent and source IP ranges linked to known AI providers. If you see repeated, identifiable fetches, that is evidence agents are using the file.
How often should I update llms.txt and related assets?
Update your llms.txt and linked machine readable assets on major content releases or quarterly. If you detect agent traffic, move to an automated update pipeline tied to your docs build so the file stays current.
Practical next step: run a quick log check for requests to /llms.txt and to your top 20 canonical pages, and allocate any remaining time to cleaning up canonical assets and adding schema. If you want ongoing visibility into how AI assistants mention and recommend your brand, check our guide on tracking brand presence across AI assistants /blog/track-brand-presence-ai-assistants and read about using GEO as a strategy /blog/how-generative-engine-optimization-geo-can-transform-your-br.
If you want automated monitoring and alerts for AI citations, platforms that track AI assistant mentions can help; idou.ai tracks how assistants reference your brand versus competitors and surfaces opportunities to improve AI driven traffic and leads.





