feat(review): personalize predicted-gate readiness by contributor calibration history (#2349) - #5069
Merged
JSONbored merged 1 commit intoJul 11, 2026
Conversation
…ibration history (#2349) Extends buildPredictedGateVerdict to read a login's own predicted_gate_calibration_ledger track record (predicted-vs-real agreement) and nudge the returned readinessScore within a fixed +/-10-point clamp -- a strong track record tightens confidence, a weak one is more conservative. Cold start (never-seen or <5 samples) is byte-identical to today. Applied strictly downstream of evaluateGateCheck's blockers/conclusion/warnings, so personalization structurally cannot flip a hard blocker off or add/remove a finding -- it only ever touches the one numeric field. The raw calibration numbers are never echoed back in the verdict, wired into both real call sites (MCP predict_gate/explain_gate_disposition and the /v1/local/branch-analysis route). Closes #2349
Contributor
|
Superagent didn't find any vulnerabilities or security issues in this PR. |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #5069 +/- ##
=======================================
Coverage 94.13% 94.13%
=======================================
Files 465 466 +1
Lines 39554 39572 +18
Branches 14433 14436 +3
=======================================
+ Hits 37235 37253 +18
Misses 1664 1664
Partials 655 655
Flags with carried forward coverage won't be shown. Click here to find out more.
🚀 New features to boost your workflow:
|
This was referenced Jul 11, 2026
Closed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
buildPredictedGateVerdictwith an optionalcontributorCalibrationinput that nudges the returnedreadinessScorebased on a login's own historical predict-vs-real agreement (predicted_gate_calibration_ledger, written since feat(review): persist a login-keyed predict-gate-vs-live-gate disagreement ledger as the review stack's own calibration ground truth #4517) — a strong track record tightens confidence (score up), a weak one is more conservative (score down), clamped toMAX_READINESS_ADJUSTMENT(10 points) either direction.computeContributorCalibration(src/review/predicted-gate-calibration-ledger.ts) to aggregate one login's full ledger history into{sampleSize, agreementRate}— read-side counterpart to the existing writer, completing that module's own documented future work. Not flag-gated (unlike the writer): it just reads whatever rows already exist.predict_gate/explain_gate_disposition(src/mcp/server.ts) andPOST /v1/local/branch-analysis(src/api/routes.ts).MIN_CALIBRATION_SAMPLES= 5 pairings) is byte-identical to today — no penalty, no bonus.Safety boundary
Personalization is applied in
buildPredictedGateVerdictstrictly afterevaluateGateCheckhas already finalizedconclusion/blockers/warnings— the calibration adjustment function receives and returns only the numericreadinessScore, so it is structurally incapable of flipping a hard blocker off or suppressing a finding, not just clamped by convention. Verified directly: a test seeds the strongest possible personalization signal (10,000 samples, 100% agreement) against a PR with a real missing-linked-issue blocker and asserts the blocker andfailureconclusion are untouched.The raw calibration numbers (
sampleSize,agreementRate) are never echoed back in the verdict — only their clamped, already-public downstream effect onreadinessScoreis. Verified with explicit "never echoed back" tests at the pure-verdict level and at both real wire call sites, per the issue's own design-note deliverable (mirrorssrc/signals/redaction.ts's "no raw per-actor trust signal, ever public" precedent).Deliverables (from #2349)
No schema changes —
predicted_gate_calibration_ledger(migration 0138) already has everything needed; this PR is purely a reader + a pure adjustment function + wiring.Test plan
packages/gittensory-enginebuilds clean (tsc --incremental false) and its fullnode:testsuite passesnpm run typecheckcleantest/unit/predicted-gate.test.ts— 22 new tests: pureapplyContributorCalibrationunit tests (cold-start x3, strong/weak, clamp boundaries incl. malformed input, null-baseline) +buildPredictedGateVerdictintegration tests (byte-identical when omitted, score shifts, blockers/conclusion/warnings untouched, hard-blocker safety, no raw-number leak)test/unit/predicted-gate-calibration-ledger.test.ts— 7 new tests forcomputeContributorCalibration(cold start, aggregation, per-login scoping, cross-repo aggregation, missing login, unflagged read, fail-safe)test/unit/mcp-predict-gate.test.ts+test/unit/routes-remediation-plan.test.ts— end-to-end wiring tests through both real call sites--coverage.include, not whole-file):contributor-calibration.ts100/100/100/100,predicted-gate-calibration-ledger.ts100/100/100/100,predicted-gate.ts's two remaining branch partials are pre-existing/unrelated to this diff (confirmed viagit diffhunk boundaries), new lines inserver.ts/routes.tsconfirmed hit via raw coverage JSONCloses #2349