Live view digest — 17 November 2026

/ took 157 crawler requests in the current window. That is 54% below its recent daily average.

Snapshot 17 November 2026 13:00 UTC · read from this server’s own nginx log · live view · all digests

Most-crawled pages

/157/data/live.json23/tools/12/robots.txt10/article/9/article/robotstxt-for-ai-crawlers-what-each-token-means.html9/live/8/stats/7
Pages by crawler requests, current 24-hour window. Source: this server's own nginx access log, aggregated every five minutes. Snapshot 17 November 2026 13:00 UTC. Verify any figure against /live/ or /data/live.json.
PathRequests
/157
/data/live.json23
/tools/12
/robots.txt10
/article/9
/article/robotstxt-for-ai-crawlers-what-each-token-means.html9
/live/8
/stats/7
/article/robotstxt-content-signal-llmstxt-make-them-agree.html7
/article/experience-vs-expertise-what-small-sites-can-prove.html7

Change since the last full day

previouslatestRequests 24h8,9196,793Recognised crawlers2,845286
The last completed day (left) against the current 24-hour window (right). Columns are drawn to the same scale. Source: this server's own nginx access log, aggregated every five minutes. Snapshot 17 November 2026 13:00 UTC. Verify any figure against /live/ or /data/live.json.

How to read this

  • How fresh these numbers are. The log is processed every five minutes, so the figures on the live page are never more than a few minutes old. If the page and the raw file ever disagree, the raw file is right.

What this is not

  • One site only. This is a dated record of one small site, not a benchmark, an index or a projection.
  • Crawler identity is claimed, not proven. User agents can be spoofed, so unrecognised traffic stays unverified rather than being assigned to a named engine.
  • A crawl is not a citation. Being fetched is a precondition for being mentioned, never evidence of it.
  • Every figure is derived from this server's own access log. No third-party analytics are used, so there is no second set of numbers.

Do the same on your own site

Source: this server's own nginx access log, aggregated every five minutes. Snapshot 17 November 2026 13:00 UTC. Verify any figure against /live/ or /data/live.json.