Add idempotency_key auto-generation and IdempotencyConflictError

[bokelley]

Tracking issue for the client-side ergonomics of AdCP #2308 — idempotency_key is now required on every mutating request in the spec.

Client-side changes

1. Auto-generate idempotency_key when caller omits it

Every mutating tool call should generate a UUID v4 if the caller didn't pass one:

# Caller doesn't think about it — SDK handles it
result = client.create_media_buy(account=..., brand=..., start_time=..., end_time=...)

# Caller passes their own (e.g., restored from DB) — SDK respects it
result = client.create_media_buy(idempotency_key=stored_key, ...)

The list of mutating tasks is in the spec — 26 total across media_buy, creative, signals, brand, governance, accounts, SI.

2. Retry-safety: reuse the same key

When the SDK retries internally (timeout, 5xx, network error), it MUST reuse the same key across retries. Re-generating a key on retry defeats the whole point.

3. Surface the key for external correlation

The key should be observable on both the sent request and the returned response so callers can log it, store it alongside their own objects, or correlate across retries:

result = client.create_media_buy(...)
print(result.idempotency_key, result.media_buy_id)

4. Typed IdempotencyConflictError

Map the new IDEMPOTENCY_CONFLICT error code to a dedicated exception class:

from adcp_client import IdempotencyConflictError

try:
    client.create_media_buy(idempotency_key=reused_key, ...different_payload)
except IdempotencyConflictError:
    # Caller reused a key by mistake — use a fresh one
    ...

5. Optional: use_idempotency_key(key) context manager for bring-your-own-key

For buyers who persist keys across process restarts (e.g., store key in their DB alongside a campaign row):

with client.use_idempotency_key(campaign.idempotency_key):
    client.create_media_buy(...)

Scope notes

• SI send_message needs the key too but lives in session scope — SDK should auto-generate per turn.
• si_terminate_session is naturally idempotent via session_id — SDK can continue to omit idempotency_key.

Acceptance

• All mutating calls auto-generate idempotency_key when not provided
• Internal retries reuse the same key
• idempotency_key surfaced on the sent request for logging
• IdempotencyConflictError exported as a typed exception
• Docs/README updated with the "retry safety for free" story
• Integration test against the idempotency compliance storyboard

Related: adcontextprotocol/adcp#2308, adcontextprotocol/adcp#2315

[bokelley]

Additional requirements surfaced during spec review (adcp#2315):

Spec additions that flow into the SDK

1. Canonical form is now RFC 8785 JCS. The seller compares canonical-JSON hashes (excluding idempotency_key and context) to determine equivalence. The SDK MUST freeze the exact serialized bytes on first attempt and resend byte-identical payloads on retry — Python dict iteration order is stable but serializer float precision and default separators aren't universally stable. Cache the serialized body before the first send rather than re-serializing on each retry.

2. Responses now carry replayed: true/false on the protocol envelope. Surface it on the SDK's response object:

   res = client.create_media_buy(...)
   if res.replayed:
       pass  # cached — skip double-counting

3. Read adcp.idempotency.replay_ttl_seconds from get_adcp_capabilities. Expose for BYOK callers:

   caps = client.get_adcp_capabilities()
   ttl = caps.adcp.idempotency.replay_ttl_seconds if caps.adcp.idempotency else 86400

Fixes to original issue scope

4. si_send_message — per logical turn, NOT per HTTP attempt. Re-sending the same turn after a network error MUST reuse the key. Tie key lifetime to the SI turn object, not the transport attempt.

5. Distinct exception for missing/invalid key.

   • InvalidRequestError (or MissingIdempotencyKeyError) — seller rejected because key was absent or malformed.
   • IdempotencyConflictError — key reused with different canonical-form payload.

6. Keys are security-sensitive — don't log by default. A live key is a retry-pattern oracle within its TTL. SDK logging SHOULD prefix-truncate (first 8 chars) or omit entirely. Opt-in full logging only via explicit flag.

[bokelley]

Spec round 2 additions for client-side:

1. Expose IdempotencyExpiredError alongside IdempotencyConflictError. Buyers handle them differently — expired means "check my DB and either accept the prior result or retry with a fresh key" vs conflict means "I reused a key by mistake."

2. Client MUST NOT fall back to an assumed TTL when the seller omits adcp.idempotency.replay_ttl_seconds from capabilities — raise a clear exception. A seller without the declaration is non-compliant and unsafe for retry-sensitive operations.

3. Canonical form exclude list: idempotency_key, context, governance_context, push_notification_config.authentication.credentials. Other fields MUST be byte-identical on retry.

[bokelley]

Consolidated acceptance criteria (post spec round 3)

Spec changes since this issue was filed materially expand the client-side obligations. The PR #2315 revision is ready to merge; before coding, treat this list as the updated acceptance criteria:

1. Two typed exceptions, not one. IdempotencyConflictError AND IdempotencyExpiredError — distinct recovery paths.

• IdempotencyConflictError → "you reused a key with a different payload." Recovery: resend the original payload (cache hit) OR mint a fresh key (new intent).
• IdempotencyExpiredError → "key was seen but TTL expired." Recovery: MUST do a natural-key check (e.g., get_media_buys by context.internal_campaign_id) BEFORE minting a fresh key — otherwise you double-create. Minting a new key in that state is the failure mode IDEMPOTENCY_EXPIRED exists to catch.

2. Freeze-bytes-on-first-send. This is bigger than a UUID generator. Spec now pins payload equivalence to RFC 8785 JCS. Re-serializing a Pydantic model on each retry risks JCS drift on non-deterministic field ordering or number formatting, which the seller will reject as IDEMPOTENCY_CONFLICT. The SDK MUST serialize the request once on first send and cache the canonical bytes alongside the key. Retries resend verbatim. This is a transport-layer change, not a per-method one.

3. Fail closed on missing capability. Spec makes adcp.idempotency.replay_ttl_seconds REQUIRED on get_adcp_capabilities — no default. Open design questions worth a design note before coding:

• Do we block the first mutating call on a capabilities fetch, or fetch lazily and fail the call if capabilities haven't arrived?
• Cache capabilities per-agent with what TTL? (The capability is stable; can be cached for hours.)
• Hard error vs. warn? Recommendation: hard error on mutating calls (spec says "MUST NOT assume a default"); warn once on read-only ops with a link to the relevant spec section. Read-only calls are safe against non-compliant sellers.

4. Client-side key format validation for BYOK. Schema enforces ^[A-Za-z0-9_.:-]{16,255}$. When a caller uses use_idempotency_key(k), validate locally and raise before the network round-trip — faster feedback than waiting for INVALID_REQUEST.

5. Key logging hygiene. Spec explicitly calls idempotency_key a retry-pattern oracle within its TTL and mandates prefix-only logging (first 8 chars). Default request logger needs a redactor (or just exclude the field entirely from debug output). Opt-in full logging only via explicit flag.

6. Expose replayed on the result object. Spec adds replayed: boolean on the protocol envelope, injected at response time by the seller's idempotency layer. The Python result object MUST surface result.replayed alongside result.idempotency_key. The primary use case per the spec is agent side-effect suppression — agentic callers need to see replayed before re-emitting notifications, re-inserting LLM memory, or re-calling downstream tools on a retry.

7. Async task caching behavior (new). For async tasks, the seller caches the submitted response (with task_id). The cache entry is immutable — even after the task completes/fails/is canceled, a replay returns the originally-cached submitted response with replayed: true. The buyer polls task_id for current state, exactly as on the first call. SDK polling/webhook logic should assume this: a retry during an in-flight async operation returns the same task_id, not the terminal state.

Related

• Spec PR: adcp#2315
• Security deep dive: docs/building/implementation/security.mdx#idempotency
• Canonical JCS: RFC 8785, reference impl candidates listed in the spec
• Compliance storyboard: static/compliance/source/universal/idempotency.yaml