Skip to content

Surveyor's two-valued disclosure field conflates a human-interactive PR with a missing marker #117

Description

@devantler

🤖 Generated by the Agentic Engineer

Evidence

portfolio-surveyor.agent.md reports a maintainer-login PR's ownership hint as a two-valued field:

disclosure=<yes|no>

derived from whether "the body leads with the deployment's AI-disclosure prefix" (line ~127). That is a prefix-only match on one party's marker, so it answers a different question than the one the orchestrator asks.

A consuming deployment measured this over 75 open maintainer-login PRs (devantler-tech/monorepo#2762). The field was 100% precise when it said yes (26/26 agent-authored), and carried no information when it said no — 49 of 75 rows (65%), spanning three classes that demand opposite handling:

actual class count correct action
human-interactive (the deployment's other marker) 5 HANDS-OFF
agent-authored, marker not at position zero 7 safe to drive
no marker at all 37 unknown; resolve from the creation record

Downstream, an orchestrator acted on that conflation and ran gh pr update-branch on two of the maintainer's own PRs — moving their heads, re-running CI and staling his reviews.

Why the prefix framing is the defect

Position is not the discriminator, measured in both directions: one interactive PR carried its marker at the start of the body, another as a trailing line, and an agent disclosure placed under a repo PR template's heading is at neither end. Any single-anchor check misses one of those shapes.

Proposed change (generic)

Emit which marker was found, over the whole body:

disclosure=<agent|human-tool|none>
  • human-tool — the body contains the deployment's human-tool marker anywhere ⇒ HANDS-OFF
  • agent — no human-tool marker, and the deployment's agent-disclosure fragment appears anywhere
  • none — neither; genuinely unknown, and not a synonym for either

When both appear, the human-tool value wins: mistaking a maintainer's PR for the agent's licenses an unrequested mutation of his work, while the reverse merely parks a PR a later run picks up.

The concrete literals stay deployment-specific and resolve from the consumer contract, exactly as today — this changes only the shape of the field and the whole-body matching rule.

Consumer status

The deployment-local overlay is fixed in devantler-tech/monorepo#2767. This issue is the generic half, filed rather than bundled because the upstream wording is intentionally marker-agnostic.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

Projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions