# AI Search Audit — Documentation

> See which AI crawlers your site allows and what its served HTML lets them read. A free, deterministic technical-readiness audit with evidence for every finding.

Canonical site: https://www.aisearchaudit.pro

HTML version: https://www.aisearchaudit.pro/docs

## Product facts

- Checks: 31 deterministic, rule-based checks (10 scored, 17 advisory, 4 informational).
- Score: a technical-readiness score across 6 equally weighted capabilities. It is null ("assessment incomplete") when a required observation is missing.
- Crawlers tracked: 17 AI crawlers and fetchers, reported by purpose (search, training, user-triggered).
- Pages per scan: the homepage plus up to four more, ranked from the sitemap and homepage links, plus robots.txt, sitemap.xml and llms.txt.
- No LLM is involved in any finding or score.
- AI Visibility (citation check): AI Visibility is in beta and not yet open to accounts.
- Limits: A 100 does not measure access from AI vendors’ own IP addresses, content that only appears after JavaScript runs, citations, rankings or traffic.
- The product does not guarantee AI rankings, citations or visibility.

## The 6 scored capabilities

- **HTTPS page access** (CR-001, CR-002, SC-001): The page was served as HTML over HTTPS, without a redirect loop. Establishes that this scanner could read it; not that every crawler can.
- **Indexable and quotable** (CR-005, CR-009): No noindex directive and no snippet control that would keep the page out of search results or AI answers that quote it. Assumes the audited page is meant to be public.
- **Usable page title** (MD-001): The page declares a non-empty title. Length and wording are advice and are not scored.
- **Canonical declarations are usable** (MD-003): Every declared canonical resolves to an HTTP(S) URL and they agree. Not applicable when none is declared — declaring one is optional.
- **Answer-engine crawlers not refused** (CR-004, CR-008): robots.txt allows the crawlers AI assistants use to find and cite pages, and the homepage did not refuse their user agents. Access from the vendors’ own IP addresses is not measured.
- **Content present without JavaScript** (CE-005): The served HTML is not an empty application shell. Crawlers that do not run JavaScript read only what is served.

## AI crawlers tracked (17)

| Token | Vendor | Purpose |
| --- | --- | --- |
| GPTBot | OpenAI | model training |
| OAI-SearchBot | OpenAI | search / citations |
| ChatGPT-User | OpenAI | user-triggered fetch |
| ClaudeBot | Anthropic | model training |
| Claude-SearchBot | Anthropic | search / citations |
| Claude-User | Anthropic | user-triggered fetch |
| anthropic-ai | Anthropic | model training |
| Google-Extended | Google | model training |
| Googlebot | Google | search / citations |
| PerplexityBot | Perplexity | search / citations |
| Perplexity-User | Perplexity | user-triggered fetch |
| bingbot | Microsoft | search / citations |
| Applebot | Apple | search / citations |
| Applebot-Extended | Apple | model training |
| CCBot | Common Crawl | model training |
| meta-externalagent | Meta | model training |
| Bytespider | ByteDance | model training |

Blocking a training crawler is an editorial choice and is not scored. Blocking a search crawler removes the site from that assistant's answers and is scored.

## Rate limits

- Anonymous: 3 scans per day.
- Free account: 5 scans per day, 1 project.
- Pro: 40 scans per hour.
- Any single target domain: 60 scans per hour across all callers.

There is no public API or CI integration today.