Live view digest — 7 November 2026

/ took 223 crawler requests in the current window. That is 35% below its recent daily average.

Snapshot 07 November 2026 13:00 UTC · read from this server’s own nginx log · live view · all digests

Most-crawled pages

/223/sitemap.xml51/tools/44/article/44/case/43/article/why-crawled-does-not-mean-indexed-the-pipeline.html42/robots.txt30/data/live.json25
Pages by crawler requests, current 24-hour window. Source: this server's own nginx access log, aggregated every five minutes. Snapshot 07 November 2026 13:00 UTC. Verify any figure against /live/ or /data/live.json.
PathRequests
/223
/sitemap.xml51
/tools/44
/article/44
/case/43
/article/why-crawled-does-not-mean-indexed-the-pipeline.html42
/robots.txt30
/data/live.json25
/sitemap-index.xml23
/article/robotstxt-for-ai-crawlers-token-guide.html18

Change since the last full day

previouslatestRequests 24h12,3989,149Recognised crawlers3,3112,873
The last completed day (left) against the current 24-hour window (right). Columns are drawn to the same scale. Source: this server's own nginx access log, aggregated every five minutes. Snapshot 07 November 2026 13:00 UTC. Verify any figure against /live/ or /data/live.json.

How to read this

  • Why the window is 24 hours and 7 days. The 24-hour figure answers 'what is happening now'; the seven-day figure shows whether that is normal for this site. A single day can be distorted by one large crawl, so the trend matters more than the snapshot.

What this is not

  • One site only. This is a dated record of one small site, not a benchmark, an index or a projection.
  • Crawler identity is claimed, not proven. User agents can be spoofed, so unrecognised traffic stays unverified rather than being assigned to a named engine.
  • A crawl is not a citation. Being fetched is a precondition for being mentioned, never evidence of it.
  • Every figure is derived from this server's own access log. No third-party analytics are used, so there is no second set of numbers.

Do the same on your own site

Source: this server's own nginx access log, aggregated every five minutes. Snapshot 07 November 2026 13:00 UTC. Verify any figure against /live/ or /data/live.json.