orb(reputation): submitter_stats counts webhook passes, not submissions — a rival's review comments push any contributor into the permanent burst-hold state

[JSONbored]

Problem

submitter_stats.submissions does not count submissions — it counts webhook passes — and it has no
per-PR idempotency key, so any third party can inflate an arbitrary contributor's counter.

recordSubmissionOutcome (src/review/submitter-reputation.ts:265-279) blindly increments:

submissions = submissions + 1, <col> = <col> + 1

There is no pull_number in the key. It is invoked from the public-surface block of
handlePullRequestWebhookEvent on every pass, and reputationOutcomeFromTerminalState returns
"manual" for a still-open PR whenever gate.conclusion === "failure" || "action_required".

That block runs for opened|reopened|synchronize|ready_for_review|edited|closed and for
pull_request_review, pull_request_review_comment, and pull_request_review_thread
(src/queue/processors.ts:7353-7370 — handlePullRequestWebhookEvent is entered for any payload
carrying pull_request).

The rule that consumes it is stats.submissions >= 8 && stats.merged < 1
(src/review/reputation-wire.ts:47-48, :61).

Two triggers

Self-inflicted. A contributor's first PR is held. Each body edit or push re-gates it and adds a
submissions + manual. After 8 passes they are burst-flagged on that repo — permanently, since
submitter_stats is all-time with no decay and the only documented exit is a merge they can no longer
easily get.

Adversarial. Any account can post review comments on a rival's held PR. Each comment re-gates the
PR and increments the rival's counter. Eight comments is a denial-of-review weapon against an arbitrary
contributor, keyed on a counter a stranger drives.

Live evidence

#9015's own live dump records dale053 with 12 real PRs on JSONbored/loopover but 21 recorded
submissions. The drift is already present in production data, and #9015 treated the counter as
authoritative when reasoning about the burst rule.

Impact

Post-#9041 a burst-flagged author's PRs get an ai_review_inconclusive hold rather than a skip — which
is the correct fail-closed direction, and it converts this into a targeted stall weapon. The contributor
cannot clear it, and no surface explains why.

Dedup

Distinct from #9015 (closed). That issue diagnosed the burst rule's fail-open direction and cited
this counter as ground truth ("21 subs"). It never asked whether submissions counts submissions.
#9015's fix does not touch recordSubmissionOutcome.

Requirements

• Key the outcome record on (project, submitter, pull_number, outcome) so it is idempotent per PR.
• Stop recording "manual" for non-terminal PRs, or record it in a separate non-cumulative column — a
  still-open PR is not a submission outcome.
• Do not re-gate reputation from pull_request_review* events at all; a third party's comment is not a
  submission signal.
• Window and decay submitter_stats, so a burst state is recoverable without a merge.
• Recompute the existing rows from the live pull_requests ledger and record how many contributors were
  wrongly flagged.

Test Coverage Requirements

99%+ patch coverage, branch-counted. Regression tests: N re-gates of one PR yield exactly one submission;
a review comment by a third party yields zero.

Links & Resources

• src/review/submitter-reputation.ts ~265-279; src/queue/processors.ts ~7353-7370, ~8133-8145, ~6862-6876;
  src/review/reputation-wire.ts ~47-48, ~59-66
• orb(security): reputation burst-downgrade fails OPEN — suspected abusers get LESS review (LIVE in prod) #9015 (the rule), fix(review): reputation-based AI-review skip must fail closed #9041 (the fail-closed fix), orb(miner): reputation throttle is a permanent, unrecoverable self-ban — cadenceFactor is computed and read by nothing #9062 (miner-side reputation self-ban)

maintainer-only — anti-abuse and contributor-fairness authority.

[linear-code]

JSO-6306