Live view digest — 17 November 2026
/ took 157 crawler requests in the current window. That is 54% below its recent daily average.
Most-crawled pages
| Path | Requests |
|---|---|
| / | 157 |
| /data/live.json | 23 |
| /tools/ | 12 |
| /robots.txt | 10 |
| /article/ | 9 |
| /article/robotstxt-for-ai-crawlers-what-each-token-means.html | 9 |
| /live/ | 8 |
| /stats/ | 7 |
| /article/robotstxt-content-signal-llmstxt-make-them-agree.html | 7 |
| /article/experience-vs-expertise-what-small-sites-can-prove.html | 7 |
Change since the last full day
How to read this
- How fresh these numbers are. The log is processed every five minutes, so the figures on the live page are never more than a few minutes old. If the page and the raw file ever disagree, the raw file is right.
What this is not
- One site only. This is a dated record of one small site, not a benchmark, an index or a projection.
- Crawler identity is claimed, not proven. User agents can be spoofed, so unrecognised traffic stays unverified rather than being assigned to a named engine.
- A crawl is not a citation. Being fetched is a precondition for being mentioned, never evidence of it.
- Every figure is derived from this server's own access log. No third-party analytics are used, so there is no second set of numbers.
Do the same on your own site
- AI Crawler Log Analyzer — find which of your own pages are fetched most
- AI Crawler Radar — see what those crawlers can actually read there
Source: this server's own nginx access log, aggregated every five minutes. Snapshot 17 November 2026 13:00 UTC. Verify any figure against /live/ or /data/live.json.
