---
title: "Agent Spoofing: Verify Bots Before You Trust Them"
description: "Learn how to verify AI agents and crawlers with layered controls, signed requests, IP validation, and route-specific bot policies."
canonical_url: https://optimi.com/en/guides/agent-spoofing
md_url: https://optimi.com/en/guides/agent-spoofing.md
last_updated: 2026-09-04
---

# Agent Spoofing: Verify Bots Before You Trust Them

A claimed crawler or AI agent identity is a starting point for verification, not an authorization decision.

## What is agent spoofing?

Agent spoofing is a malicious bot that impersonates an AI agent, a search engine crawler, a price-comparison tool, or any other trusted client in order to bypass the security of an online retail site.

By gaining unrestricted access, a bot may scrape content, probe an application, consume inventory, or distort traffic data. The risk is not unique to AI agents: any identity inferred from a request header alone can be forged.

> **The cost to retailers**
>
> Treat bot classification as a risk signal. A verified crawler may be legitimate, but it still should not receive access to an authenticated, administrative, or transactional route.

## The technical reality

Use layers of evidence rather than one signal:

- **The User-Agent header**, a text string indicating the client's name and version, is simply a self-declaration. Any malicious bot can present itself as a known AI agent with a single line of code.
- **The IP range** can support verification when its owner publishes and maintains a documented range or reverse-DNS process. It is not proof by itself, and generic cloud ranges are not a bot identity.
- **Behavior and route context** help distinguish normal crawling from scraping, credential abuse, and inventory or checkout automation.
- **Cryptographic verification** can prove that a request was signed by a key the sender controls when the sender publishes a complete, verifiable scheme.

## The emerging response: cryptographic signatures

**HTTP Message Signatures (RFC 9421)** standardize a way to sign selected HTTP message components. They do not define a universal bot identity, public-key directory, or authorization policy.

When a provider supports signed requests, validate its documented key-discovery and trust model, signed components, expiry, replay protection, and allowed routes. Do not treat a valid signature as an unrestricted allow rule.

Audit Bot Management and WAF policy by route. Start in observe mode, verify claimed Google crawlers using [Google's documented process](https://developers.google.com/search/docs/crawling-indexing/verifying-googlebot), then apply a graduated response: allow authorized access, throttle suspicious behavior, or block high-confidence abuse. Record the evidence, owner, review date, and rollback condition for every exception.

**A claimed agent identity requires route-specific verification**

![A decision flow where a claimed agent first encounters route policy. Routes that do not permit automation are denied. Permitted routes require documented verification of published network identity or cryptographic signatures, plus behavior review; verified agents receive a scoped allow and budget, while uncertain or abusive requests are observed, throttled, or blocked.](/diagrams/agent-spoofing/claimed-agent-verification.svg)

*A User-Agent claim is not authorization. Verification establishes a bounded identity signal, then route policy and behavior determine the response.*

## Authoritative references

- [OWASP Bot Management and Anti-Automation Cheat Sheet](https://cheatsheetseries.owasp.org/cheatsheets/Bot_Management_and_Anti-Automation_Cheat_Sheet.html)
- [RFC 9421: HTTP Message Signatures](https://www.rfc-editor.org/rfc/rfc9421.html)
- [Cloudflare: Web Bot Auth](https://developers.cloudflare.com/bots/reference/bot-verification/web-bot-auth/)

[Get Started](/en/contact): Verify bots without blocking legitimate access — Talk to our team about Bot Management, WAF configuration, and verifying signed agents across your retail estate.