Open Bot Conduct Standard (OBCS) v1.0

A behavioral standard for web bots and AI agents.

Identity tells you WHO a bot is. Conduct tells you HOW it behaves.

This standard defines the HOW.

Purpose

As AI agents and web bots become the majority of web traffic, websites need more than cryptographic identity (see Cloudflare Web Bot Auth). They need verifiable proof of good behavior. OBCS defines 10 measurable criteria that separate well-behaved bots from harmful ones.

The 10 Conduct Criteria

1. IDENTIFY

The bot MUST send a descriptive User-Agent header containing: bot name, version, operator name, and a URL with documentation about the bot.

Example: BotName/1.2 (operated by AcmeCorp; https://acme.com/bot-info)

Measurement: User-Agent header is checked for completeness.

2. DECLARE

The operator MUST publish a machine-readable declaration at a public URL stating:

Format: JSON at the URL specified in the User-Agent.

Measurement: Declaration URL is fetched and validated for required fields.

3. OBEY

The bot MUST respect robots.txt directives and HTML meta tags (noindex, nofollow, noarchive).

Measurement: The test environment sets specific robots.txt rules and verifies compliance.

4. THROTTLE

The bot MUST NOT exceed reasonable request rates. Default ceiling: 1 request per second per domain unless the site specifies otherwise via Crawl-delay or custom headers.

Measurement: Request frequency is logged and evaluated against thresholds.

5. DISTRIBUTE

The bot MUST distribute its requests over time, avoiding burst patterns that stress servers. No more than 60% of daily requests should occur within any single hour.

Measurement: Request timestamp distribution is analyzed for burst patterns.

6. PROTECT

The bot MUST NOT collect, store, or transmit Personally Identifiable Information (PII) unless explicitly authorized by the data subject or required by law.

Measurement: The test environment plants synthetic PII (emails, phone numbers, names). The bot is evaluated on whether it captures and stores this data.

7. RETAIN

The operator MUST declare data retention policies and honor them. Collected data must have a defined expiry.

Measurement: Verified via the declaration document (Criterion 2).

8. RESPOND

The operator MUST provide a functional contact endpoint (email or URL) that responds to complaints within 72 hours.

Measurement: A test message is sent to the declared contact endpoint. Response time is recorded.

9. RESPECT

The bot MUST honor opt-out requests. When a website sends a X-Bot-Optout: true header or adds the bot to a deny list, the bot must cease crawling that domain within 24 hours.

Measurement: The test environment sends opt-out signals and verifies cessation of crawling.

10. REPORT

The operator MUST publish periodic transparency reports (at minimum quarterly) documenting: domains crawled, data volume collected, incidents, and opt-out requests honored.

Measurement: Report URL is checked for existence, recency, and required content.

Scoring

Each criterion is scored 0-10. Total score: 0-100.

|-------|--------|---------|

Certification

Bots scoring 70+ receive a verifiable certificate containing:

Certificates are verifiable via API:

GET https://botconduct.org/api/verify/{certificate-id}

Relationship to Existing Standards

License

This standard is published under Creative Commons Attribution 4.0 International (CC BY 4.0). Anyone may implement, reference, or build upon it.

Open Bot Conduct Standard v1.0 — Published 2026

https://botconduct.org

botconduct.org — hello@botconduct.org