feat(review): persist a login-keyed predict-gate-vs-live-gate calibration ledger - #4577
Merged
Merged
Conversation
…tion ledger Neither the MCP predict_gate tool nor contributor_gate_history persists whether a contributor's self-reported predict-gate verdict matched the REAL gate decision their PR eventually received -- the one calibration signal the review stack itself uniquely owns, since it terminates both the self-review call and the live gate for the same login/repo. predicted_gate_calibration_ledger pairs the most recent predict_gate_calls row (#4516) for a (login, project) against the real decision at the same call sites as recordContributorGateDecision, writing one immutable row per (login, project, pr, commit) -- ON CONFLICT DO NOTHING, never DO UPDATE, so a webhook replay can never overwrite an already-recorded pairing. Write-only and server-side only: nothing reads it yet (mirrors contributor_gate_history's own precedent), and no MCP tool or other contributor-reachable surface can write to or read from it, preserving its value as anti-farming-resistant ground truth for a future #2349 consumer. Depends on #4516 (predicted_gate_calls) -- built stacked on that branch per the issue's own explicit note that the two may be built together; will be rebased onto main once that PR merges. Fixes #4517
… main) origin/main independently landed three files at 0134 and two at 0135 (from already-merged PRs #4549, #4558, #4563) since this branch's last rebase. Rather than touch already-merged migration files here, take the next genuinely free number for this branch's own new migration; the 0134/0135 collision among already-merged files is a separate main-red issue handled in its own PR.
…PRs) Three already-merged PRs independently claimed 0134 (#4549's review_targets_cadence_idx, #4558's predicted_gate_calls, and #4563's pr_last_backlog_convergence_regated_at). CI on this branch inherits main's full migrations/ directory regardless of which PR fixes it, so this can't be deferred to a separate PR without also blocking this one. Renumber the two newer files (keeping #4563's oldest 0134 file in place); this branch's own new migration was already moved to 0138 to stay clear of both this collision and #4563's separate 0135 collision.
Contributor
|
Superagent didn't find any vulnerabilities or security issues in this PR. |
JSONbored
added a commit
that referenced
this pull request
Jul 10, 2026
…ay collision on main) main currently has three DIFFERENT already-merged files at 0134 (#4549/#4558/#4563) -- tracked separately as its own fix (PR #4577, not yet merged). Since that fix already claims through 0138, take 0139 here to avoid colliding with it once it lands; this branch's own db:migrations:check will stay red until #4577 merges (unrelated to this branch's own changes), and will need one more rebase afterward.
loopover-orb
Bot
added
the
gittensor:feature
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #4577 +/- ##
=======================================
Coverage ? 94.12%
=======================================
Files ? 430
Lines ? 38155
Branches ? 13912
=======================================
Hits ? 35913
Misses ? 1585
Partials ? 657
🚀 New features to boost your workflow:
|
Contributor
|
Warning 🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨 ⏸️ Gittensory review result - manual review recommendedReview updated: 2026-07-10 05:12:43 UTC
⏸️ Suggested Action - Manual Review
Review summary Nits — 7 non-blocking
Linked issue satisfactionAddressed Review context
Contributor next steps
Signal definitions
🟩 Safe / merged · 🟦 Advisory · 🟨 Held for review · 🟥 Blocked / closed 💰 Earn for open-source contributions like this. Gittensor lets GitHub contributors earn for the work they already do — register to start earning →. Checked by Gittensory, a quiet PR intelligence layer for OSS maintainers.
|
JSONbored
added a commit
that referenced
this pull request
Jul 10, 2026
…ay collision on main) main currently has three DIFFERENT already-merged files at 0134 (#4549/#4558/#4563) -- tracked separately as its own fix (PR #4577, not yet merged). Since that fix already claims through 0138, take 0139 here to avoid colliding with it once it lands; this branch's own db:migrations:check will stay red until #4577 merges (unrelated to this branch's own changes), and will need one more rebase afterward.
JSONbored
added a commit
that referenced
this pull request
Jul 10, 2026
…-wide for confirmed miners (#4546) * fix(review): make submitter-reputation burst/AI-spend defense install-wide for confirmed miners getSubmitterReputation's burst/low-sample thresholds are scoped WHERE project = ? AND submitter = ? -- a single repo. A fleet identity spreading a handful of gate-passing-but-low-value PRs across dozens of repos in one self-hosted install never accumulates enough same-repo sample density to read as "burst" or "low" anywhere, so the reputation defense never fires for it and every one of its submissions burns full paid AI-review spend indefinitely. - New getSubmitterReputationAcrossInstall in submitter-reputation.ts: the identical quality-weighted signal derivation, scoped by review_targets.installation_id instead of project (that column already existed, migrations/0050; this adds the supporting index). - New getEffectiveSubmitterReputation in reputation-wire.ts: the per-repo signal, additionally widened to the install-wide view for a CONFIRMED official Gittensor miner -- but only when the per-repo signal alone doesn't already justify caution, so an ordinary contributor or an already-flagged submitter pays no extra lookup. - Extracted the miner-identity check (shared with #4512) into src/gittensor/miner-detection-cache.ts so both this module and unlinked-issue-guardrail.ts can use it without a circular import through processors.ts. - Wired into both call sites: shouldSkipAiForReputation (the main AI-spend gate) and the vision-review reputation check. Fixes #4513 * fix: renumber migration 0130 -> 0131 (0130 was claimed by #4538, already merged to main) * fix(test): account for getEffectiveSubmitterReputation's miner-identity check in visual-vision tests runVisualVisionForAdvisory now resolves reputation via getEffectiveSubmitterReputation (#4513), which checks confirmed-official-miner identity (a fetch to api.gittensor.io/miners) whenever the submitter's per-repo signal is neutral -- a real, intentional behavior change this test file's existing "no network calls at all" assertions didn't account for. Stub that one identity check to resolve cleanly and assert precisely that no OTHER (BYOK/vision-spend) network call happens, rather than asserting zero fetch calls outright. * fix(db): renumber migration 0131 -> 0133 (0131/0132 claimed by merged PRs) origin/main has since merged #4545 (0131_screenshot_table_gate_matrix.sql) and #4554 (0132_impact_map_query_cache.sql, itself a collision fix); rebase onto current main and take the next free number. * fix(db): renumber migration 0133 -> 0134 (0133 claimed by merged PR #4556) Rebase onto current main and take the next free number now that migrations/0133_screenshot_table_gate_skill_link.sql (from #4556) occupies the number this branch previously took. * fix(db): renumber migration 0134 -> 0139 (0134 has a pre-existing 3-way collision on main) main currently has three DIFFERENT already-merged files at 0134 (#4549/#4558/#4563) -- tracked separately as its own fix (PR #4577, not yet merged). Since that fix already claims through 0138, take 0139 here to avoid colliding with it once it lands; this branch's own db:migrations:check will stay red until #4577 merges (unrelated to this branch's own changes), and will need one more rebase afterward. * test(review): close 2 coverage gaps surfaced by the #4514 rebase-merge getEffectiveSubmitterReputation's getRepository().catch() needed a real read-failure test (added); its isConfirmedOfficialMiner().catch() is unreachable (that function already catches every internal failure point itself) and getSubmitterReputationAcrossInstall's results ?? [] fallback is the same "D1 always populates results" case already v8-ignored elsewhere in this codebase -- both marked accordingly. * fix(review): isolate #4507's reputation-single-read invariant from cross-test miner-cache pollution Several earlier tests in this file cache "contributor" as a confirmed official Gittensor miner (5-min TTL) in the shared per-file D1 instance. That collided with #4513's new install-wide reputation widening, which correctly detects the stale cache and adds a 4th reputation-scan D1 read for a submitter this test never intended to be a miner. Use a submitter login unique to this test instead.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Neither the MCP
predict_gatetool norcontributor_gate_historypersists whether a contributor's self-reported predict-gate verdict matched the REAL gate decision their PR eventually received — the one calibration signal the review stack itself uniquely owns, since it terminates both the self-review call and the live gate for the same login/repo.src/review/predicted-gate-calibration-ledger.ts(new, migration0138): pairs the most recentpredicted_gate_callsrow (feat(review): record MCP predicted-gate verdicts to review_audit and measure predicted-vs-live gate agreement #4516) for a(login, project)against the real gate decision, writing ONE immutable row per(login, project, pr, commit)at the same call sites asrecordContributorGateDecisioninsrc/queue/processors.ts.ON CONFLICT DO NOTHING(neverDO UPDATE) — a webhook replay at the same commit is a no-op, never a silent overwrite of the originally-recorded pairing. This is the tamper-resistance property the issue asks for ("not exposed back to the miner as a writable or self-reportable value").contributor_gate_history's (migration 0126) own "write-only, nothing reads yet" precedent — no MCP tool or other contributor-reachable surface can write to or read from this table. The eventual maintainer: personalized gate-prediction tuning per contributor/miner history #2349 consumer is explicit future work.fix(db): resolve migration collision at 0134 on main): three already-merged PRs (feat(review): add a submission-cadence/inter-arrival-time signal to flag machine-paced submitters #4549, feat(review): record MCP predicted-gate verdicts and measure predicted-vs-live gate agreement #4558, fix(review): harden backlog-convergence-sweep against duplicate work #4563) independently claimed migration0134, and fix(review): harden backlog-convergence-sweep against duplicate work #4563 separately claimed0135too. CI on this branch inheritsmain's fullmigrations/directory regardless of which PR fixes it, so this couldn't be deferred to a separate PR without also blocking this one — renumbers the two newer already-merged files, kept as its own clearly-separated commit.Fixes #4517
Test plan
npx tsc --noEmitcleannpm run db:migrations:check/db:schema-drift:checkclean (contiguous 0001-0138)predicted-gate-calibration-ledger.test.ts, 100% coverage): agreement/disagreement recorded correctly, cold-start (no prior prediction) records nothing, correlation-window boundary, cross-repo isolation for the same login, non-binary decision/predicted-action skip, missing-login skip, immutability (a replay at the same commit — even with a different decision — never changes the original row; a new commit gets its own row), flag gating, fail-safe on read/write errorsgit diff origin/main --statthat this branch's diff contains only its own intended files (caught and fixed a stale-merge-base issue from an unrelated PR along the way)