Live view digest — 7 November 2026
/ took 223 crawler requests in the current window. That is 35% below its recent daily average.
Most-crawled pages
| Path | Requests |
|---|---|
| / | 223 |
| /sitemap.xml | 51 |
| /tools/ | 44 |
| /article/ | 44 |
| /case/ | 43 |
| /article/why-crawled-does-not-mean-indexed-the-pipeline.html | 42 |
| /robots.txt | 30 |
| /data/live.json | 25 |
| /sitemap-index.xml | 23 |
| /article/robotstxt-for-ai-crawlers-token-guide.html | 18 |
Change since the last full day
How to read this
- Why the window is 24 hours and 7 days. The 24-hour figure answers 'what is happening now'; the seven-day figure shows whether that is normal for this site. A single day can be distorted by one large crawl, so the trend matters more than the snapshot.
What this is not
- One site only. This is a dated record of one small site, not a benchmark, an index or a projection.
- Crawler identity is claimed, not proven. User agents can be spoofed, so unrecognised traffic stays unverified rather than being assigned to a named engine.
- A crawl is not a citation. Being fetched is a precondition for being mentioned, never evidence of it.
- Every figure is derived from this server's own access log. No third-party analytics are used, so there is no second set of numbers.
Do the same on your own site
- AI Crawler Log Analyzer — find which of your own pages are fetched most
- AI Crawler Radar — see what those crawlers can actually read there
Source: this server's own nginx access log, aggregated every five minutes. Snapshot 07 November 2026 13:00 UTC. Verify any figure against /live/ or /data/live.json.
