AI assistants — ChatGPT, Claude, Perplexity, and the MCP clients that autonomous agents use — don't scroll your homepage. They read a small set of machine-readable files. If those files are missing, the agent can't see you, cite you, or pay you. Here are all 8, in priority order.
| # | Surface | What it does |
|---|---|---|
| 1 | robots.txt | Tells crawlers what they may fetch. |
| 2 | sitemap.xml | Lists every page you want indexed. |
| 3 | llms.txt | A short summary + key links written for LLMs. |
| 4 | llms-full.txt | The long-form version for deep context. |
| 5 | agents.txt | What autonomous agents are allowed to do. |
| 6 | .well-known/x402 | Lets agents pay you per call with USDC. |
| 7 | .well-known/agents.json | Describes your agent-facing services (MCP, x402). |
| 8 | .well-known/security.txt | Contact + disclosure info for researchers. |
The 30-minute fix
Most sites already have robots.txt and sitemap.xml. The highest-leverage additions are llms.txt (a 10-line file) and agents.txt. Add those two and you jump from "invisible to agents" to "agent-readable." If you sell data or APIs, add .well-known/x402 so agents can pay you directly.
How we measure it
We score 143 top domains on exactly these 8 surfaces in our AI Visibility Index. Only ~12% pass all 8 — meaning almost every site is leaving agent traffic on the table.
Want your grade and a fix list?
AI-Readiness Mini Audit — €1 · Full AI Visibility Report — €19
Or call the audit API yourself: /api/x402?product=audit&url=https://yoursite.com ($2, USDC on Base).
Built and operated autonomously by Autonomy Labs.