feat(canonical-formats): public API + v2→v1 projection (#741, half 1 of 2)#841
Conversation
…alidator (#741) First half of the canonical-formats projection layer (issue #741) — public API surface, v2 → v1 projection algorithm, and ``format_options[]`` closed-set validator. The v1 → v2 reverse projection, ``pixel_tracker`` bidirectional contract, and 14 reference fixtures land in a follow-up. ## Public API surface on ``adcp.types`` (29 new exports) Discriminator + projection ref: ``CanonicalFormatKind``, ``ProductFormatDeclaration``, ``ProductFormatSellerPreference``, ``CanonicalProjectionReference``, ``CanonicalAssetSource``, ``CanonicalSlotOverride``. Plus 13 canonical format classes, pixel tracker asset trio, and the 7 registry types. ## Hand-rolled ``ProductFormatDeclaration`` The upstream schema is a discriminated ``oneOf`` over 13 ``format_kind`` values, each binding ``params`` to a canonical-specific schema. ``datamodel-code-generator`` flattens this to a single class carrying only the shared properties — ``format_kind`` and ``params`` disappear entirely, and ``extra='ignore'`` silently drops them on construction. ``src/adcp/types/canonical_decl.py`` adds a hand-rolled class with the full wire shape (discriminator + open ``params`` dict + ``extra='allow'``) and rewires the public alias. The codegen output is preserved as ``_GeneratedProductFormatDeclaration``. ## New module ``adcp.canonical_formats`` - ``project_declaration_to_v1`` / ``project_product_to_v1`` walk the resolution order from ``registries/v1-canonical-mapping.json`` and return projected ``format_ids[]`` plus any SDK-source advisories (``FORMAT_DECLARATION_V1_LOSSY_MULTI_SIZE``, ``FORMAT_DECLARATION_V1_AMBIGUOUS``). - ``validate_format_kind_in_options`` / ``find_declaration_by_kind`` are the seller-side closed-set guards with ``capability_id`` disambiguation. - ``load_default_registry`` loads the bundled mapping, cached; with ``glob_match`` and ``structural_match`` matchers. Registry-based inversion (v2 → v1 via structural match) is explicitly NOT implemented because the registry forbids it: "SDKs MUST NOT synthesize a v1 ``format_id`` from the registry by inverting structural matches." Step 4 of the resolution order surfaces ``FORMAT_DECLARATION_V1_AMBIGUOUS`` instead; the seller's path is to author ``v1_format_ref``. All emitted ``Error`` entries carry ``source: "sdk"`` and ``sdk_id: "adcontextprotocol-adcp-python@<version>"`` per ``core/error.json#sdk_id``. ## Tests 51 new tests across four files covering every branch of the resolution order (parametrised across all 13 canonical kinds + the ``v1_translatable`` table), closed-set validator edge cases, registry loader caching + matchers, and the public-API surface. ## Layering ``canonical_decl.py`` and ``canonical_formats/advisory.py`` are added to ``tests/test_import_layering.py``'s ``ALLOWED_FILES`` — both legitimately import from ``generated_poc`` (hand-rolled override + wire-shape ``Error`` construction respectively), same role as ``aliases.py`` / ``capabilities.py`` / ``_forward_compat.py``. ## Verified - ``ruff check src/`` ✓ - ``mypy src/adcp/`` ✓ (892 source files) - ``pytest tests/`` ✓ (5063 passed, 38 skipped, 1 xfailed) Refs: #741
There was a problem hiding this comment.
LGTM. Follow-ups noted below. Half 1 of 2 lands additively on a clean architectural seam — hand-rolled ProductFormatDeclaration for the discriminated oneOf, registry consulted on v1→v2 only, advisory-only on errors[] for AMBIGUOUS / LOSSY_MULTI_SIZE.
Things I checked
V1_TRANSLATABLEtable atsrc/adcp/canonical_formats/projection.py:54-68matches the fourdefault: falseoverrides inschemas/cache/3.1.0-beta.3/formats/canonical/{agent_placement,sponsored_placement,responsive_creative,image_carousel}.jsonexactly.native_in_feedcorrectly stays True.test_v1_translatable_table_matches_kind_enumpins coverage against the enum.- Resolution-order algorithm at
projection.py:111-201matches the 4-step contract inregistries/v1-canonical-mapping.json("Direction of truth (normative)"). Step 3 silence forv1_translatable=Falsecanonicals is correct — surfacing AMBIGUOUS there would spam the wire. - Registry-based v2→v1 inversion is explicitly NOT implemented (
projection.py:28-31); matches the registry'sMUST NOT synthesize v1 format_id from registry by inverting structural matchesclause. FORMAT_DECLARATION_V1_AMBIGUOUSandFORMAT_DECLARATION_V1_LOSSY_MULTI_SIZEboth present inschemas/cache/3.1.0-beta.3/enums/error-code.json;source=sdk+sdk_id="adcontextprotocol-adcp-python@<version>"(advisory.py:43,77-86) matchescore/error.json.- Hand-rolled
ProductFormatDeclarationatsrc/adcp/types/canonical_decl.py:59-169carries all 11 properties from the upstream schema.extra='allow'correct._GeneratedProductFormatDeclarationpreserved as escape hatch. - Layering:
canonical_decl.pyandcanonical_formats/advisory.pyadded toALLOWED_FILES(tests/test_import_layering.py:55,63) — same architectural role asaliases.py/capabilities.py. No new violations. - Conventional commit
feat(canonical-formats):is correct — additive onadcp.types(29 names) and a newadcp.canonical_formatsmodule. No!needed, no public symbol removed or narrowed. code-reviewer: no blockers, two Major items below.ad-tech-protocol-expert: sound-with-caveats — algorithm + advisory codes verified against upstream schemas; caveats land as Follow-ups below.
Follow-ups (non-blocking — file as issues)
paramsisrequiredon the wire but defaults toNoneon the class.src/adcp/types/canonical_decl.py:91-99. Upstreamcore/product-format-declaration.jsonmarksparamsrequired alongsideformat_kind. A declaration constructed withoutparamsround-trips locally (extra='allow') but emits JSON a spec-conformant counterparty MUST reject. Either makeparamsnon-optional or add amodel_validatorthat fails closed. Half-2-territory if you want to land it alongside the divergent-narrowing check.allOfconstraints not enforced on the Pythonic class.format_kind='custom'MUST carryformat_shape+format_schema;canonical_formats_only=trueis mutually exclusive withv1_format_ref. Neither is enforced —extra='allow'lets contradictions through. Same root cause as theparamsFollow-up: hand-rolled class skipped the schema'sallOfblock. Reasonable for half 1; flag for the second PR.format_options: Iterable[ProductFormatDeclaration]exhausts single-pass generators.src/adcp/canonical_formats/format_options.py:61,85.validate_format_kind_in_options(...); find_declaration_by_kind(...)paired against a generator silently returnsNonefrom the second call because the validator drained the iterator. Narrow toSequence[ProductFormatDeclaration]or snapshot vialist(...)at entry._versions_overlapsilently accepts<=and silently drops</>.src/adcp/canonical_formats/registry.py:131-153. The docstring at L106-115 advertises>=,4.x, exact only. A registry entry inadvertently authored as"<4.0"matches nothing without an error. Either reject unknown prefixes loudly or document the full accepted set.FORMAT_DECLARATION_DIVERGENTnarrowing check. Resolution-order step 1 inregistries/v1-canonical-mapping.jsonmandates a narrows check betweenparamsand the referenced v1requirements. Acknowledged by the PR body as half-2-territory — flagged so it doesn't get lost.- Docs drift on new public surface.
README.mdfeatures section,AGENTS.md:209-243import quick reference,llms.txt:36-46module index, and theCLAUDE.md"Import Architecture" narrative all missadcp.canonical_formatsand the two new ALLOWED_FILES entries.skills/adcp-media-buy/SKILL.md:60andskills/adcp-creative/SKILL.md:58point at the spec doc but don't name the SDK helpers sellers actually call.MIGRATION_v5_to_v6.md:120-142recipe lacksfrom adcp import …/from adcp.types import Errorso a copy-paste getsNameError.
Minor nits (non-blocking)
_GeneratedProductFormatDeclarationin__all__.src/adcp/types/canonical_decl.py:174. Underscore-prefixed names in__all__is unusual; either drop from__all__and let consumers import the dunder directly, or rename to a non-underscore alias.detailsannotation symmetry.src/adcp/canonical_formats/projection.py:176. Mirror the explicitdetails: dict[str, Any] = {...}annotation from L145 to defuse mypy inference on the subsequentdetails["product_id"] = product_idassignment.V1_TRANSLATABLEis a hand-maintained mirror. A drift test that loads each canonical's schema and asserts thev1_translatabledefault matches the table would catch upstream additions before they default to True and start emitting AMBIGUOUS on structurally-v1-unreachable canonicals.
Safe to merge once CI completes (storyboard runners + Python 3.11/3.12/3.13 still in progress).
Addresses code-reviewer, ad-tech-protocol, adtech-product, and security
reviews of the canonical-formats projection layer. Material correctness,
security, and adopter-ergonomics fixes; no functional removals.
## Schema-conformance fixes (NORMATIVE)
- ``ProductFormatDeclaration.params`` is now required (schema:
``required: ["format_kind", "params"]``). Previously optional —
wire-invalid declarations could construct.
- ``canonical_formats_only=True`` + ``v1_format_ref[]`` are rejected at
construction (schema ``allOf.not`` clause). Previously the model
accepted the combination and projection silently discarded the refs.
- Projection step 1 no longer fires on ``format_kind=custom`` alone.
``custom`` declarations MAY carry seller-asserted ``v1_format_ref[]``
and project via step 2; only ``canonical_formats_only=True``
triggers the unconditional silent skip.
## Security fixes
- Credential-shaped key guard on ``ProductFormatDeclaration`` — same
suffix list and rationale as the dispatcher's ``ctx_metadata`` gate.
``params`` and model extras are walked recursively; sellers cannot
stuff credentials onto a declaration where they would round-trip
into buyer responses + the idempotency replay cache.
- ``load_default_registry()`` returns a fresh deep copy of the cached
parsed registry — multi-tenant SDK processes can no longer poison
each other's projection view by mutating ``mappings``.
- Seller-controlled identifiers (``product_id``) capped to 128 chars
before echoing into advisory ``details`` — defends against
log-injection through multi-hop ``errors[]``.
## Error-handling fixes
- Malformed registry bundles raise ``RegistryLoadError`` with
contextual ``ADCP_VERSION`` + failure mode instead of propagating
raw exceptions.
- ``_versions_overlap`` accepts ``<``, ``>``, ``!=``, ``==``
(previously only ``<=`` / ``>=``); raises on unrecognised operator
prefixes (``~``, ``^``) rather than silently never matching.
- ``SDK_ID`` resolved lazily via module ``__getattr__``; reads the
actual distribution name from
``importlib.metadata.metadata("adcp")["Name"]`` rather than
hardcoding a prefix that could drift from the PyPI distribution.
## Adopter-ergonomics additions
- ``ProductFormatDeclaration.params_as(CanonicalFormatImage)`` —
validates the open ``params`` dict against the typed canonical class.
- ``find_declaration_by_v1_format_id(format_id, format_options)`` —
seller-side helper for v1-inbound ``create_media_buy`` requests
against a product publishing v2 ``format_options[]``.
- ``FormatKindNotInClosedSetError.to_wire_error()`` — turns the
closed-set exception into the wire-correct ``UNSUPPORTED_FEATURE``
``Error`` with ``details.rejected_value`` + sorted/dedup'd
``accepted_values``. Replaces the 14-line MIGRATION recipe.
## Layering cleanup
- ``Recovery`` and ``Source`` re-exported via ``adcp.types``;
``canonical_formats/advisory.py`` removed from
``ALLOWED_FILES``. ``canonical_decl.py`` stays on the allowlist
(hand-rolled override of a generated class).
## Public-API restoration
- ``NotificationConfig``, ``WholesaleFeedEvent``, ``WholesaleFeedWebhook``
re-exports restored to ``adcp.types`` and ``adcp`` (silently dropped
by ruff's auto-fix during the original PR pass).
## Tests
5093 passed locally. ``tests/test_canonical_formats_declaration.py``
(new) covers the new ``ProductFormatDeclaration`` invariants. Existing
test files extended with custom flow-through, multi-tenant registry
isolation, ``RegistryLoadError`` wrapping, version-operator DSL +
fail-loud, ``to_wire_error`` shape, ``find_declaration_by_v1_format_id``
matching, seller-controlled identifier truncation.
Refs: #741
…atus # Conflicts: # src/adcp/__init__.py
There was a problem hiding this comment.
LGTM. The resolution-order implementation lines up with the registry's "Direction of truth" contract — refusing to invert structural matches and routing canonical-formats projection through seller-asserted v1_format_ref[] is the right shape for a v2 → v1 layer. Follow-ups noted below.
Things I checked
- Resolution order (steps 1–4) at
src/adcp/canonical_formats/projection.py:115-207. Step 1 short-circuits before refs check; step 2 emits + computes the multi-size advisory; step 3 silently skips non-translatable canonicals; step 4 emits AMBIGUOUS. Order matches the registry normative section. V1_TRANSLATABLEtable atprojection.py:58-72. Cross-checked againstschemas/cache/3.1.0-beta.3/formats/canonical/*.json— the four explicitfalseoverrides (agent_placement,sponsored_placement,image_carousel,responsive_creative) pluscustom=Falsematch the per-canonical schemas.tests/test_canonical_formats_projection.py:172-177enforces full enum coverage so a new upstream kind without a table entry trips CI rather than silently defaulting to True.- Hand-rolled
ProductFormatDeclarationatsrc/adcp/types/canonical_decl.py:122-236. Carries all 9 shared properties plus discriminator + openparams. TheallOf.notmutual-exclusion clause is enforced in_check_mutual_exclusion.paramsis required, matching the schema'srequired: ["format_kind", "params"].extra='allow'for forward-compat is safe because the upstream object doesn't lockadditionalProperties: false. - Credential-shaped key guard at
canonical_decl.py:76-116, 238-264. Suffix list and case-insensitive.lower()matching are byte-identical toadcp.decisioning.dispatch._CREDENTIAL_SHAPED_KEY_SUFFIXES. Walksparamsrecursively + extras via__pydantic_extra__+ nestedBaseModelviamodel_dump(mode='python'). This is the right shape — same fail-closed rationale as the dispatcher'sctx_metadatagate. - Registry multi-tenant isolation at
registry.py:119-132._load_registry_uncopied()islru_cache(maxsize=1);load_default_registry()returnsmodel_copy(deep=True). Tenant A mutating the projection registry cannot leak into tenant B's view. - Advisory wire shape at
advisory.py:113-122.source=sdkmatches the upstreamsourceenum["producer","sdk"].sdk_idformat<dist>@<version>matchescore/error.json. Both new error codes (FORMAT_DECLARATION_V1_AMBIGUOUS,FORMAT_DECLARATION_V1_LOSSY_MULTI_SIZE) are inenums/error-code.json. - Type layering at
tests/test_import_layering.py:32-56.canonical_decl.pycorrectly added toALLOWED_FILESwith the architectural rationale;canonical_formats/advisory.pywas removed from the allowlist after the follow-up commit re-exportedRecovery/Sourceviaadcp.types. No new layering violations. - Public-API audit. 29 net new exports under
adcp.types, all additive.feat:without!is the right semver signal.MIGRATION_v5_to_v6.mddocuments the additions and the hand-roll rationale. - Test plan. All 6 checked items in the PR body match actual artifacts (5063 passed, 51 new tests across 4 files, public-API snapshot regenerated, layering test passes).
Follow-ups (non-blocking — file as issues)
-
agent_urlcanonicalization infind_declaration_by_v1_format_id(src/adcp/canonical_formats/format_options.py:175-181).schemas/cache/3.1.0-beta.3/core/format-id.json:11is normative: callers MUST canonicalizeagent_urlbefore comparison. The currentstr(ref.agent_url) == target_urldoes Pydantic-AnyUrltrailing-slash normalization but not RFC 3986 §6 host-casefolding or default-port stripping. A seller publishing"https://Creative.AdContextProtocol.org"will silently miss-match a buyer's"https://creative.adcontextprotocol.org"and the SDK returnsNone→ wrongfulUNSUPPORTED_FEATURE.src/adcp/signing/brand_jwks.py:_canonicalize_urlalready exists in this repo — wire it through. Flagged by bothcode-reviewerandad-tech-protocol-expertas the only documented MUST divergence in the diff. -
Operator-facing log/ANSI injection via
_echo_identifier(src/adcp/canonical_formats/advisory.py:65-71). The 128-char cap is the right defense against unbounded growth into the replay cache; it doesn't strip newlines, CRs, or ANSI escapes. A seller publishing aproduct_idcontaining\nor\x1b[escape sequences gets that string round-tripped intoerrors[].details.product_id→ forged log lines / terminal manipulation in operator tooling that emits one advisory per line.security-reviewerMedium. Scrub control chars + ANSI ESC before the length cap. -
sdk_iddiffers between installed and dev (advisory.py:55-62). Installed: reads_pkg_metadata("adcp")["Name"]→"adcp@<version>"(perpyproject.tomlname = "adcp"). Fallback:"adcontextprotocol-adcp-python@0.0.0-dev". Two different attribution strings for the same SDK violate thecore/error.jsondedup-by-sdk_idcontract. Pick one canonical form; the docstring's example (adcontextprotocol-adcp-python@1.2.0) suggests hardcoding the long form and using_pkg_versiononly. -
_versions_overlapfail-loud on~/^(registry.py:205-210) is stricter than the registry DSL's forward-compat posture.canonical-format-kind.jsondocuments open-enum semantics for unknown values; the operator parser should mirror that — log-and-skip the constraint as no-match rather than raising. Latent until half 2 wires the v1 → v2 reverse path that actually consumes the registry, but file it now so the regression doesn't ship under load. -
Partial-half public exposure of
glob_match/structural_match/load_default_registry— these are public but unused on the v2 → v1 path until half 2 lands. Consider an@experimentalmarker in the docstrings so adopters don't build against an API that may shift.
Minor nits (non-blocking)
- Duplicate
NotificationConfigin__all__atsrc/adcp/types/__init__.py:1198,1200. Harmless at runtime; tripsset(__all__) == list-leninvariants tooling sometimes asserts. _walk_for_credential_keyshas no cycle guard atsrc/adcp/types/canonical_decl.py:94-116. Not wire-reachable (JSON has no cycles, Pydantic rejects them on validation) but worth a depth bound orseenset if a future caller can hand-build cyclic dicts.SDK_IDexposed via module__getattr__incanonical_formats/__init__.py:71-82doesn't appear indir(). No test pullsSDK_IDthrough the documented public path —tests/test_canonical_formats_projection.py:18reaches intoadvisory.py. Add one test that importsfrom adcp.canonical_formats import SDK_ID._echo_identifierlisted inadvisory.py's__all__(line 130) but it's a private symbol (leading underscore). Either drop the underscore or drop it from__all__.
Approved. The expert-review follow-up commit c58d4031 already closed the load-bearing issues from the prior pass (params required, mutual-exclusion validator, deep-copy on registry load, lazy SDK_ID, identifier truncation, fail-loud version DSL). The remaining follow-ups don't block — they sharpen edges that show up under multi-hop traffic, and half 2 is the natural place to bundle the canonicalization + sdk_id alignment work.
Argus APPROVED #841 and noted five non-blocking follow-ups + four nits. This commit addresses the ones that sharpen wire correctness or close real attack surfaces. ## agent_url canonicalisation in v1 inbound lookup ``find_declaration_by_v1_format_id`` now canonicalises both buyer and seller ``agent_url`` per RFC 3986 §6 before comparing: scheme lowercased, host lowercased, default port (443/80) stripped. Without this a seller publishing ``https://Creative.AdContextProtocol.org`` would silently miss-match a buyer's ``https://creative.adcontextprotocol.org`` and the SDK would return a wrongful ``UNSUPPORTED_FEATURE``. Argus flagged this as the only documented MUST divergence — ``core/format-id.json:11`` is normative. ## ANSI / control-char scrub in advisory identifier echo ``_echo_identifier`` now escapes every C0 control char (incl. ``\\n`` ``\\r`` ``\\x1b``), the C1 range, and Unicode LS/PS line separators to a literal ``\\u<hex>`` form before applying the 128-char cap. A seller publishing ``product_id`` with ``\\n`` or ``\\x1b[`` previously round-tripped that string into ``errors[].details.product_id`` and could forge log lines or manipulate ANSI-aware operator tooling. ## sdk_id alignment between installed + dev Hardcodes ``_SDK_DIST_NAME = "adcontextprotocol-adcp-python"`` (the spec example form). Installed wheels publish the dist name as plain ``adcp``; the prior code emitted ``adcp@<version>`` from a wheel install vs ``adcontextprotocol-adcp-python@0.0.0-dev`` from a checkout, breaking the multi-hop ``(code, field, sdk_id)`` dedup contract in ``core/error.json`` for the same SDK. ## Experimental markers + nits - Added ``.. note:: Experimental`` to ``glob_match``, ``structural_match``, ``load_default_registry``. - Removed duplicate ``NotificationConfig`` in ``adcp.types/__all__``. - Removed ``_echo_identifier`` from ``advisory.py``'s ``__all__``. - Two new tests pull ``SDK_ID`` through the documented public path (``from adcp.canonical_formats import SDK_ID``). ## Tests 5117 passed locally. New coverage: agent_url host-casefolding + default-port stripping; newline / ANSI / Unicode line-separator scrubbing; ``SDK_ID`` public-import + canonical-prefix invariants. ## Deferred per Argus's own suggestion - ``_versions_overlap`` fail-loud on ``~`` / ``^`` — latent until #741 half 2 actually consumes the registry on the inbound path. - ``_walk_for_credential_keys`` cycle guard — not wire-reachable. Refs: #741, #841
There was a problem hiding this comment.
Approving. Follow-ups noted below. Hand-rolls ProductFormatDeclaration correctly per upstream's discriminated oneOf (codegen flattens it and drops format_kind/params), mirrors the existing dispatcher credential gate verbatim, and implements the registry's "Direction of truth" resolution order step-for-step.
Things I checked
- 13
CanonicalFormatKindenum values all accounted for inV1_TRANSLATABLEatsrc/adcp/canonical_formats/projection.py:58-72(image=T, html5=T, display_tag=T, image_carousel=F, video_hosted=T, video_vast=T, audio_hosted=T, audio_daast=T, sponsored_placement=F, native_in_feed=T, responsive_creative=F, agent_placement=F, custom=F). Cross-checked againstschemas/cache/3.1.0-beta.3/formats/canonical/*.json. _CREDENTIAL_SHAPED_KEY_SUFFIXESatcanonical_decl.py:76-85mirrors the dispatcher's list atdecisioning/dispatch.py:422-431exactly. Same threat model (openparamsdict +extra='allow'extras round-tripping throughformat_options[]and the idempotency replay cache), same fail-closed behavior, same suffix set.- Mutual-exclusion check at
canonical_decl.py:219-236faithfully implements the schema'sallOf.notclause (product-format-declaration.json:118-138);paramsrequired at L132-145 matchesrequired: ["format_kind", "params"](L6-9). model_copy(deep=True)atregistry.py:139— multi-tenant isolation via Pydantic v2 deep-copy.lru_cacheon the parsed registry, fresh copy per caller.- 29 new public exports are purely additive vs
origin/main— none renamed, none removed.feat:is the correct semver signal. tests/test_import_layering.py:55allowlist scopes correctly tocanonical_decl.pyonly;advisory.py/projection.py/registry.py/format_options.pyall import viaadcp.types.code-reviewer-deep: clean layering.- Resolution order at
projection.py:140-207matches registry "Direction of truth" steps 1-4 exactly.ad-tech-protocol-expert: sound-with-caveats; closed-set rejection shape (UNSUPPORTED_FEATURE+details.rejected_value+details.accepted_values) is correct pererror.json:93-96.
Follow-ups (non-blocking — file as issues)
_canonicalize_agent_urlcovers RFC 3986 §6 but not §5.2.4 path dot-segment normalization.format_options.py:36-63lowercases scheme + host and strips default port — that's two of the four bullets the spec mandates.manifest.json:990andbundled/content-standards/calibrate-content-request.json:304both spell out "normalize path dot-segments" as part of the documented/docs/reference/url-canonicalizationrule. Failure mode is a false-negative onfind_declaration_by_v1_format_id→ wrongfulUNSUPPORTED_FEATURE. Pipeparts.paththroughposixpath.normpath(or hand-rolledremove_dot_segments) and add a test pairhttps://x.example/a/./b/../c↔https://x.example/a/c. Percent-encoding casing (RFC 3986 §6.2.2.1) worth aligning while in there.- Credential-suffix list is mirrored from the dispatcher and inherits its gaps. Both lists miss
access_key,private_key,authorization,passwd,signature.security-reviewer-deepflagged this at High. Right fix is one follow-up that updates BOTHcanonical_decl.py:76-85anddecisioning/dispatch.py:422-431in lockstep so the gates don't drift — not a PR-841-only patch. _echo_identifierscrub misses BOM + bidi overrides + zero-width chars.advisory.py:96-104covers C0/C1/LS/PS but not U+FEFF, U+202A-U+202E, U+2066-U+2069, U+200B-U+200D, U+2060. Same threat model as the existing scrub (operator-tooling log injection through seller-publishedproduct_id); add to the codepoint guard list._versions_overlapfail-loud asymmetry.registry.py:228-230silentlycontinues on unparseable RHS (">=foo") while the bare-operator typo path at L219-224 fails loud. Latent until #741 half 2 consumes the registry on the inbound path — Argus's prior pass already deferred this and the reasoning still holds.FORMAT_DECLARATION_DIVERGENTnarrows-check.product-format-declaration.json:59puts the v2-side check between declarationparamsand referenced v1requirementsin-scope for v2→v1. PR description already lists it as deferred to #741 half 2; confirm it lands there.params_asaudit.canonical_decl.py:266-282invokescanonical_type.model_validate(self.params)on seller-controlled data. Codegen models don't typically emit custom validators, but_ergonomic.py/_forward_compat.pypatch some types withBeforeValidatorcoercion — grep those for side-effecting validators before 3.1 GA.
Minor nits (non-blocking)
- Duplicate
WholesaleFeedEvent/WholesaleFeedWebhookinadcp.types.__all__.src/adcp/types/__init__.py:1201-1202and:1236-1237. The duplication shows up in the snapshot attests/fixtures/public_api_snapshot.json:1043-1046. The de-dup pass at c58d403 caught theNotificationConfigduplicate but missed this pair — interesting how snapshot dupes hide right next to the entries you're staring at. Drop the new pair, regenerate snapshot. canonical_formats/__init__.pymodule docstring drift. Public-surface block at L17-32 listsproject_*/validate_*/find_*/load_default_registry/SdkAdvisorybut__all__(L85-101) also exportsRegistryLoadError,V1_TRANSLATABLE,V2ToV1Projection,glob_match,structural_match,find_declaration_by_v1_format_id,make_sdk_advisory. Add them.CanonicalFormatHtml5Bannerdoesn't match the discriminator value. The wire value ishtml5and the canonical schema ishtml5.json.ad-tech-protocol-expert: not wire-divergent (class names aren't on the wire) but worth aligning toCanonicalFormatHtml5before 3.1 GA — public-rename is cheap pre-GA, expensive after._canonicalize_agent_urlexception scoping.format_options.py:51-63wrapsurlsplitintry/except ValueError, butparts.portat L59 is what actually raises on bad authority likehttps://x:abc. Unreachable today viaFormatId.agent_url: AnyUrlupstream validation, but moveport = parts.portinside thetryto keep the function's "non-throwing on malformed input" docstring honest.
Safe to merge.
Three findings from the inline code-quality bot review were blocking the admin merge per the repo's "all comments must be resolved" rule. ## Empty except in registry packaged-bundle lookup ``_read_registry_json`` swallows ``ModuleNotFoundError``, ``FileNotFoundError``, ``OSError`` from the packaged ``_schemas/`` lookup so dev installs can fall through to the schema-cache walk-up. Added an explanatory comment documenting that the ``pass`` is intentional and what the fallback path is, so static analysis (and future readers) know the bare ``except`` isn't hiding a bug. ## SDK_ID listed in __all__ but not module-level Replaced the lazy ``__getattr__`` dispatch with a plain module-level ``SDK_ID = _resolve_sdk_id()`` assignment in ``canonical_formats/advisory.py``. The lazy form was over-engineered — ``_resolve_sdk_id`` is already ``@lru_cache``'d, and the ``importlib.metadata`` lookup runs at module-import time anyway in practice (no setuptools-scm late-resolution path actually flows through this module). The eager assignment is identical in cost once and satisfies CodeQL's explicit-export check. ``canonical_formats/__init__.py`` now imports ``SDK_ID`` from ``advisory`` like any normal name; the ``__getattr__`` shim is removed. ## Tests 89 canonical-formats tests pass. The existing public-import test (``test_sdk_id_reachable_via_public_package_path``) still exercises the documented public path; just via a real module-level binding now. Refs: #741, #841
There was a problem hiding this comment.
Approving. The CodeQL-bot cleanup at ca09d4b1 swaps the lazy __getattr__ SDK_ID dispatch for an eager SDK_ID: str = _resolve_sdk_id() at src/adcp/canonical_formats/advisory.py:110 — lru_cache(maxsize=1)'d, no setuptools-scm late-resolution path flowing through this module, no import-chain risk. Strictly safer than the __getattr__ shim, and CodeQL's __all__ introspection check goes quiet.
Things I checked
ca09d4b1eagerSDK_IDis total._pkg_version("adcp")runs once at module-import;PackageNotFoundError → "0.0.0-dev"fallback covers dev installs.advisory.pyonly importsError/Recovery/Sourcefromadcp.types, which is fully loaded by the timeadcp.canonical_formatsinitializes — no circular risk.security-reviewer: re-entrancy risk on the prior lazy form is now gone.- Resolution order at
src/adcp/canonical_formats/projection.py:115-207matches the 4-step "Direction of truth (normative)" section inregistries/v1-canonical-mapping.jsonexactly.ad-tech-protocol-expert: SOUND. V1_TRANSLATABLEtable atprojection.py:58-72verified againstschemas/cache/3.1.0-beta.3/formats/canonical/*.json— four explicitFalseoverrides (agent_placement,sponsored_placement,responsive_creative,image_carousel) pluscustom=False; remaining 8 inheritTruefrom_base.json. Enum-coverage test pins it against drift.- Hand-rolled
ProductFormatDeclarationatsrc/adcp/types/canonical_decl.py:122-264carries all 9 schema properties plus discriminator + openparams.allOf.notmutual-exclusion enforced at_check_mutual_exclusion. Credential-shaped-key gate at:238-264mirrors the dispatcher byte-for-byte. - Credential-suffix list parity:
canonical_decl.py:76-85↔decisioning/dispatch.py:422-431are identical 8-suffix tuples.security-reviewer: CLEAN. - Multi-tenant registry isolation at
registry.py:91-145:_load_registry_uncopiedlru_cache'd;load_default_registryreturnsmodel_copy(deep=True). Tenant A mutation cannot reach tenant B's view. sdk_idformat atadvisory.py:47,71:adcontextprotocol-adcp-python@<version>matchescore/error.json:117spec example verbatim.- Closed-set rejection at
format_options.py:87-114:UNSUPPORTED_FEATURE+details.rejected_value+details.accepted_valuesis the wire-correct shape pererror.json:95. _echo_identifiercontrol-char scrub atadvisory.py:74-107: covers C0 + DEL/C1 + U+2028/U+2029. Length cap fires post-escape (so escape expansion stays bounded).- Public-API audit: 29 net-new exports, purely additive.
feat(canonical-formats):without!is correct — no removals, no required→optional flips, no enum-value removals. - Layering:
canonical_decl.pyadded totests/test_import_layering.py:55ALLOWED_FILESwith documented rationale.canonical_formats/*modules import viaadcp.types(verified). code-reviewer-deep: SOUND-WITH-CAVEATS, no blockers.ad-tech-protocol-expert-deep: SOUND-WITH-CAVEATS.security-reviewer-deep: LOW, no must-fix.
Follow-ups (non-blocking — file as issues)
- Test docstring drift from
ca09d4b1.tests/test_canonical_formats_public_api.py:75-80still claimsSDK_ID"is resolved lazily via module__getattr__." The shim is gone; the assertion still passes, but the comment lies about the path under test. Update or drop the second sentence. allOf[0]not enforced onProductFormatDeclaration.schemas/cache/3.1.0-beta.3/core/product-format-declaration.json:66-117requiresformat_kind=customto carryformat_shape+format_schemaAND (canonical_formats_only=trueORv1_format_ref), and non-custom kinds MUST NOT carryformat_shape/format_schema. The hand-rolled class accepts wire-invalid combinations. Same family as the existingallOf.notvalidator atcanonical_decl.py:219-236— bundle with #741 part 2.- Path dot-segment normalization in
_canonicalize_agent_url(format_options.py:36-63). RFC 3986 §5.2.4 —manifest.json:990enumerates it as part of the canonicalization rule. Failure mode is over-strict identity matching on URLs with/./or/../segments; v1 catalog URLs in practice don't carry these, so latent. - Bidi / zero-width / BOM scrub in
_echo_identifier(advisory.py:96-107). C0/C1/LS/PS covered; U+FEFF, U+202A-U+202E, U+2066-U+2069, U+200B-U+200D, U+2060 are not. Operator tooling that renders advisories in a terminal/UI (Slack, Sentry, log viewers) honors bidi/ZW for visual spoofing.security-reviewer: LOW — length cap still bounds blast radius and no credential exfil path. _walk_for_credential_keyscycle guard (canonical_decl.py:94-116). Not wire-reachable (JSON has no cycles); reachable only via SDK-internal programmatic constructor. LatentRecursionError. ~3 lines ofseen: set[int]closes it. Argus-prior pass already deferred this.- Carry-forward from prior Argus passes:
_versions_overlapfail-loud on~/^is latent until #741 part 2 wires the v1 → v2 inbound consumer;FORMAT_DECLARATION_DIVERGENTnarrowing check betweenparamsand referenced v1requirementsis on the half-2 list.
Minor nits (non-blocking)
WholesaleFeedEvent/WholesaleFeedWebhookstill duplicated inadcp.types.__all__atsrc/adcp/types/__init__.py:1201-1202and:1236-1237. The duplication is baked into the snapshot attests/fixtures/public_api_snapshot.json:1043-1046. The de-dup pass atc58d4031and the snapshot regeneration at202474f4both saw these and walked past — notable how a snapshot dupe right next to the entries you're staring at hides better than a hidden one.tests/test_import_layering.py:2-7module docstring still names "aliases.py, _ergonomic.py, _generated.py, and the publicadcp.types/__init__.py" —ALLOWED_FILESis now 7 entries deep. Refresh the docstring when_versions_overlapdoes.
Safe to merge.
Summary
First half of #741 — canonical-formats projection layer. Lands the public-API surface, the v2 → v1 projection algorithm, and the
format_options[]closed-set validator. Reverse v1 → v2 projection,pixel_trackerbidirectional contract, and the 14 reference fixtures + round-trip tests follow in a second PR.Public API on
adcp.types(29 new names)CanonicalFormatKind,ProductFormatDeclaration,ProductFormatSellerPreference,CanonicalProjectionReference,CanonicalAssetSource,CanonicalSlotOverrideCanonicalFormatImage,CanonicalFormatHtml5Banner,CanonicalFormatDisplayTag,CanonicalFormatImageCarousel,CanonicalFormatHostedVideo,CanonicalFormatVastVideo,CanonicalFormatHostedAudio,CanonicalFormatDaastAudio,CanonicalFormatNativeInFeed,CanonicalFormatResponsiveCreative,CanonicalFormatAgentPlacement,CanonicalFormatSponsoredPlacement(+CanonicalFormatBase,CanonicalCompositionModel)PixelTrackerAsset,PixelTrackerEvent,PixelTrackerMethodV1V2CanonicalFormatMappingRegistry,V1CanonicalMapping,V1CanonicalGlobPattern,V1CanonicalStructuralPattern,V1CanonicalStructural,V1CanonicalV2Projection,V1CanonicalDimensionsHand-rolled
ProductFormatDeclarationThe codegen path is broken for this class: the upstream schema is a discriminated
oneOfover 13format_kindvalues, each bindingparamsto a canonical-specific schema.datamodel-code-generatorflattens it to a single class carrying only the shared properties —format_kindandparamsdisappear entirely, andextra='ignore'silently drops them on construction.src/adcp/types/canonical_decl.pyhand-rolls the wire-faithful class (discriminator + openparams: dict[str, Any]+extra='allow'). The codegen output stays available as_GeneratedProductFormatDeclaration.New module
adcp.canonical_formatsv2 → v1 resolution order (per
registries/v1-canonical-mapping.json)canonical_formats_only=Trueorformat_kind='custom'→ no v1 emit, no advisory.v1_format_ref[]set → emit refs; emitFORMAT_DECLARATION_V1_LOSSY_MULTI_SIZEwhenlen(v1_format_ref) < len(params.sizes).v1_format_ref[]absent, canonical withv1_translatable=False(agent_placement/sponsored_placement/responsive_creative/image_carousel) → silent.v1_format_ref[]absent, canonical withv1_translatable=True→ emitFORMAT_DECLARATION_V1_AMBIGUOUS.Registry-based inversion is explicitly NOT implemented per the registry's "Direction of truth" rule: "SDKs MUST NOT synthesize a v1
format_idfrom the registry by inverting structural matches." The seller's path on AMBIGUOUS is to authorv1_format_ref.All advisories carry
source: "sdk"+sdk_id: "adcontextprotocol-adcp-python@<version>".Closed-set
format_options[]validatorSeller-side pre-call guard for
create_media_buy.FormatKindNotInClosedSetErrorexposesformat_kind+accepted_kindson the exception for directerror.details.accepted_valuesround-tripping.find_declaration_by_kinddisambiguates withcapability_idwhen the closed set carries multiple declarations of the same kind.What's NOT in this PR (lands in #741 half 2)
project_v1_to_v2helper)pixel_trackerbidirectional downgrade/upgrade contract (PIXEL_TRACKER_LOSSY_DOWNGRADE/PIXEL_TRACKER_UPGRADE_INFERRED)static/examples/products/canonical/upstream + round-trip testsFORMAT_DECLARATION_DIVERGENTnarrowing check between v2paramsand referenced v1requirementsLayering
canonical_decl.pyandcanonical_formats/advisory.pyare added totests/test_import_layering.py'sALLOWED_FILES— both legitimately import fromgenerated_poc(hand-rolled override + wire-shapeErrorconstruction respectively), same architectural role asaliases.py/capabilities.py/_forward_compat.py.Test plan
ruff check src/— all checks passedmypy src/adcp/— 892 files, no issuespytest tests/— 5063 passed, 38 skipped, 1 xfailedRefs: #741