AI crawler report — October 2026
31 day(s) of data in this period, read directly from this site's own access log. Complete period.
Summary
- 391,893 total requests, of which 111,879 were recognised crawler requests (28.5%).
- 11 distinct crawlers identified. Largest: ClaudeBot (21,203 requests).
Crawlers by engine
| Crawler | Requests | Share of crawler traffic |
|---|---|---|
| ClaudeBot | 21,203 | 19.0% |
| 12,907 | 11.5% | |
| meta-externalagent | 11,211 | 10.0% |
| Bing | 5,134 | 4.6% |
| PerplexityBot | 4,568 | 4.1% |
| Amazonbot | 3,423 | 3.1% |
| GPTBot | 1,752 | 1.6% |
| Applebot | 1,537 | 1.4% |
| CCBot | 1,244 | 1.1% |
| OAI-SearchBot | 1,190 | 1.1% |
| Bytespider | 455 | 0.4% |
Most-crawled pages
| Path | Requests |
|---|---|
| / | 8,753 |
| /data/live.json | 2,129 |
| /article/ | 1,726 |
| /tools/ | 1,693 |
| /disclaimer/ | 1,691 |
| /case/ | 1,688 |
| /manifest.json | 1,402 |
| /robots.txt | 602 |
| /sitemap.xml | 456 |
| /tools/schema-generator/ | 314 |
| /guide/orphan-page-identification-and-repair.html | 283 |
| /guide/dead-link-detection-and-repair.html | 265 |
Method, and what this is not
- Source: our own nginx access log, aggregated per day, then summed for this period. No third-party analytics, no cookies, no client script.
- Crawlers are identified by user agent string. This can over-count: some automated traffic claims a crawler identity. We treat these numbers as indicative, not audited.
- One site only. This is not a benchmark, index, or projection. It is a dated, first-hand record — useful precisely because it is small, local and verifiable.
- Data file for this period: /reports/data/2026-10.json
See also: live crawler view · all reports