robots.txt & AI Crawler Scan
We read your file the way a crawler does and tell you which of the eight major AI agents are allowed, blocked or never mentioned — and where you sit next to the 229 agencies in our study.
We analysed 229 AI agency websites. Only 33.5% name an AI crawler explicitly in
robots.txt. If you are in the other two thirds, nobody has ever decided whether ChatGPT,
Perplexity or Claude can read your site — and you have no way to know whether they do.
This audit answers that question in five working days.
We read your file the way a crawler does and tell you which of the eight major AI agents are allowed, blocked or never mentioned — and where you sit next to the 229 agencies in our study.
Do you have one? Was it written by a person or generated by a plugin such as Rank Math or Yoast? Does it describe your business or a template? We tell you whether it helps or hurts.
Ten purchase-intent questions your buyers actually ask, run manually in ChatGPT, Perplexity and Claude. We record whether you appear, who appears instead, and what the assistant says about you.
What to fix first, what your team can do this week, and what genuinely needs outside help. Written so you can forward it internally without rewriting it.
You send the domain and confirm the engagement. Payment and invoicing are handled before work starts, so the five-day clock is unambiguous.
We run the audit within five working days. robots.txt, llms.txt, ai.txt, structured data, and ten live queries across three assistants.
You get a 30-minute walkthrough. The report is yours; the call is where we explain what matters and answer the awkward questions.
One-time payment · 5 working days · walkthrough call included
Five working days from the moment we receive your domain and payment. The walkthrough call is scheduled within that window.
That is one of the findings, not a blocker. We tell you whether the file matters for your site, whether a plugin already generates one, and what to put in it if you decide to publish one.
The refund policy is set per engagement before payment. Ask before you buy and we will confirm it in writing.
You get the report and the walkthrough call. Implementation is a separate conversation if you want it. There is no retainer and no monthly monitoring included.
Yes. Agencies use the audit as a second opinion on their own clients. The report is written so it can be forwarded to a client without editing.