Security guide

Agent Spoofing: Verify Bots Before You Trust Them

A claimed crawler or AI agent identity is a starting point for verification, not an authorization decision.

What is agent spoofing?

Agent spoofing is a malicious bot that impersonates an AI agent, a search engine crawler, a price-comparison tool, or any other trusted client in order to bypass the security of an online retail site.

By gaining unrestricted access, a bot may scrape content, probe an application, consume inventory, or distort traffic data. The risk is not unique to AI agents: any identity inferred from a request header alone can be forged.

The cost to retailers

Treat bot classification as a risk signal. A verified crawler may be legitimate, but it still should not receive access to an authenticated, administrative, or transactional route.

The technical reality

Use layers of evidence rather than one signal:

The User-Agent header, a text string indicating the client's name and version, is simply a self-declaration. Any malicious bot can present itself as a known AI agent with a single line of code.
The IP range can support verification when its owner publishes and maintains a documented range or reverse-DNS process. It is not proof by itself, and generic cloud ranges are not a bot identity.
Behavior and route context help distinguish normal crawling from scraping, credential abuse, and inventory or checkout automation.
Cryptographic verification can prove that a request was signed by a key the sender controls when the sender publishes a complete, verifiable scheme.

The emerging response: cryptographic signatures

HTTP Message Signatures (RFC 9421) standardize a way to sign selected HTTP message components. They do not define a universal bot identity, public-key directory, or authorization policy.

When a provider supports signed requests, validate its documented key-discovery and trust model, signed components, expiry, replay protection, and allowed routes. Do not treat a valid signature as an unrestricted allow rule.

Audit Bot Management and WAF policy by route. Start in observe mode, verify claimed Google crawlers using Google's documented process, then apply a graduated response: allow authorized access, throttle suspicious behavior, or block high-confidence abuse. Record the evidence, owner, review date, and rollback condition for every exception.

Authoritative references

Verify bots without blocking legitimate access

Talk to our team about Bot Management, WAF configuration, and verifying signed agents across your retail estate.

Get Started