AI agents — ChatGPT, Claude, Perplexity, and the new class of autonomous MCP clients — don't browse like people. They look for machine-readable signals to understand and use your site. If those signals are missing, agents literally can't see you.
We audited 143 major websites and found only 12% publish an llms.txt. Here's the checklist to not be in that 88%.
1. Add llms.txt
The single highest-leverage file. It tells LLMs, in plain markdown, what your site is and how to use it.
/llms.txt).Minimum: a one-line description + links to your key pages.
# Your Brand We help small businesses with AI automation. - [Services](https://yoursite.com/services) - [Pricing](https://yoursite.com/pricing) - [Docs](https://yoursite.com/docs)
2. Add agents.txt and .well-known/agents.json
Describe what agents are allowed to do on your site — crawl, transact, call APIs. agents.json is the structured version.
3. Publish robots.txt and sitemap.xml
Still foundational. Allow AI crawlers (GPTBot, ClaudeBot, PerplexityBot) explicitly, and give them a sitemap.
4. Add an x402 manifest (optional but powerful)
If you want AI agents to be able to pay you directly, publish a .well-known/x402 manifest describing your payable endpoints. It's the emerging standard for machine-to-machine payments.
5. Verify with a free check
Before and after, run your URL through a checker:
Or see how 143 top sites score on the AI Visibility Index.
We'll audit your site and deliver a concrete fix plan — or generate the files for you.
Weekly AI Visibility Report (€19/mo) · One-time audit
Deterministic checks, no AI used in scoring. See the live index for methodology.