The 402 Return of Payment Required: What HTTP Status Codes Tell Analytics About Bot Traffic
HTTP status codes carry more signal than most analytics tools use. Here's how server-side status parsing helps separate humans from bots without tracking anyone.
Most web analytics decisions happen client-side, after a page has loaded and JavaScript has run. That model quietly assumes every request that reaches your server is worth counting. It isn't. A large share of raw traffic never renders anything, never belongs to a human, and never should reach your dashboards. The clearest early signal for filtering it is something you already generate on every request: the HTTP status code.
Status codes are a first-class analytics signal
The HTTP semantics specification (RFC 9110) defines status codes as a three-digit integer describing the result of a request, grouped into five classes: 1xx informational, 2xx successful, 3xx redirection, 4xx client error, and 5xx server error. Because Monoid runs on the Cloudflare edge, we see the status code the origin (or the edge itself) returns before any client script executes. That ordering matters. A page view recorded by a browser beacon can only exist if the document was actually served with a 2xx and parsed. But the request stream includes far more than successful documents.
Consider what a typical origin returns across a day: successful HTML documents, 304 Not Modified responses to conditional requests, 301/308 redirects from canonicalisation, 404s for probing scanners, and 5xx bursts during incidents. Counting only successfully rendered documents and ignoring the rest throws away context that explains anomalies in your traffic curve.
Where bots hide in the status distribution
Automated clients behave differently from browsers, and status codes expose the difference cheaply. Vulnerability scanners generate dense clusters of 404 and 403 responses as they probe for admin panels and known CMS paths. Aggressive crawlers ignore 429 Too Many Requests — defined in RFC 6585 as the response for rate limiting — and keep hammering, producing a signature no human session creates. Naive scrapers frequently follow redirect chains that a real browser would short-circuit via bfcache or history.
The 402 Payment Required code is instructive here. RFC 9110 explicitly notes it is reserved for future use and has no standardised semantics, yet it has recently seen renewed attention for metered API and agent access. If your origin starts emitting 402s, that traffic is almost certainly programmatic, not a person browsing. A status-aware pipeline can tag it accordingly instead of silently inflating a metric.
Why client-side-only analytics gets this wrong
A JavaScript-only tool literally cannot observe most of this. If a request returns 403, no analytics script runs, so the event is invisible — but the load on your infrastructure is real, and the intent behind it (reconnaissance, credential stuffing, scraping) is often the thing you actually want to know about. Conversely, tools that count server log lines without discriminating by status over-report, treating every 301 hop and every 404 probe as a "hit".
The useful middle ground is edge-level status classification feeding a privacy-first counting model:
- 2xx text/html with a real navigation is a candidate page view.
- 3xx should be attributed to the destination, not counted twice.
- 4xx is diagnostic, not audience — surface it separately.
- 5xx belongs on a reliability panel, correlated with any drop in successful views.
Doing this without tracking anyone
None of this requires identifying a visitor. Status code, method, response class, and coarse timing are properties of the request-response exchange, not of a person. Monoid never sets cookies, never reads localStorage, and never fingerprints devices. We do not need a stable identifier to know that a burst of 429s at 03:00 UTC came from automation rather than an audience — the status distribution says so on its own. This is data minimisation working in our favour: the least personal signal is also one of the most honest for traffic quality.
There is a compliance benefit too. Filtering non-human traffic at the edge means fewer spurious events enter the analytics store at all, keeping recorded data tighter to its stated purpose. Under the storage-limitation and purpose-limitation principles of the GDPR (Articles 5(1)(c) and 5(1)(e)), not collecting junk is strictly better than collecting and later discarding it.
Practical takeaways
If you build or evaluate analytics, ask three questions. Does it see the status code, or only rendered documents? Does it separate reliability signals (5xx) from audience signals (2xx)? And does it achieve bot filtering without an identifier? A pipeline that answers yes to all three gives you cleaner numbers and a smaller privacy surface at the same time — which, for once, are the same decision.
Comments
Loading comments…