Part of #4496. P8 — medium severity, high confidence, but zero current blast radius (feature flag off by default today) — fix before enabling.
Context
The reconcile-open-prs job type does real, per-repo paginated GitHub REST work every 10 minutes (isReconciliationWindow = minute % 10 === 0, src/index.ts:114, on the */2 * * * * cron) but is excluded from BOTH rate-limit admission and job-key dedup:
- It's missing from
GITHUB_BUDGET_BACKGROUND_TYPES (src/selfhost/queue-common.ts:86-98), so it never yields to shouldWaitForGitHubRateLimit at dequeue time — unlike every other real GitHub-REST-calling background job in this file.
src/index.ts:183 pushes it unconditionally with NO !sweepThrottledUntil check — contrast the very next block (index.ts:165-169) which DOES gate backfill-registered-repos this way.
jobCoalesceKey (queue-common.ts:882-1023) has no case "reconcile-open-prs", so it falls through to return null — a null key means pg-queue.ts's enqueue-time coalesce/supersede path is skipped entirely and a fresh duplicate row is inserted every tick (pg-queue.ts:911-942), even while a prior scan is still processing.
src/selfhost/maintenance-admission.ts's own header comment (lines 5-7) asserts every MAINTENANCE_JOB_TYPES member "already yield[s] to an EXHAUSTED GitHub REST budget (shouldWaitForGitHubRateLimit)" — reconcile-open-prs IS in MAINTENANCE_JOB_TYPES (line 62) but only gets that module's local-queue-depth/host-load admission, never the GitHub-budget one this comment claims — a real, confirmable contradiction between the module's own doc comment and reality.
Meanwhile runOpenPrReconciliation (src/review/pr-reconciliation.ts:94-124) sequentially loops every watched repo, and reconcileOpenPullRequests (src/github/backfill.ts:3254-3274) makes up to RECONCILE_OPEN_PRS_MAX_PAGES = 10 sequential GET /pulls?state=open&per_page=100 calls per repo, then sequentially awaits catchUpMissingPullRequest for every missing PR found — real, potentially large GitHub REST volume with none of the protections its sibling scheduled jobs have.
This cannot fire in production today — isPrReconciliationEnabled defaults OFF (GITTENSORY_PR_RECONCILIATION unset ⇒ false) — but the moment an operator enables this flag, the dual gap (no GitHub-budget admission + no job-key dedup, stacked) becomes real.
Requirements
- Add
"reconcile-open-prs" to GITHUB_BUDGET_BACKGROUND_TYPES in queue-common.ts (or give it its own admission check at enqueue time mirroring backfill-registered-repos's !sweepThrottledUntil guard).
- Add a
case "reconcile-open-prs": return type; to jobCoalesceKey so a second 10-minute tick coalesces into an already-pending/processing scan instead of creating a duplicate row.
- Correct
maintenance-admission.ts's header comment to accurately describe which job types get GitHub-budget admission vs. only local-load admission.
- Invariant + regression tests (non-negotiable): an invariant test asserting
reconcile-open-prs correctly yields when shouldWaitForGitHubRateLimit reports an exhausted budget (mirroring the equivalent test for backfill-registered-repos); a regression test asserting a second reconcile-open-prs enqueue while a prior one is pending/processing coalesces rather than duplicates; a test enabling the flag in a test harness and confirming both protections actually engage end-to-end (since this path has zero test coverage of the gap today, per the audit).
Deliverables
Expected outcome
reconcile-open-prs gets the same GitHub-budget admission and job-key dedup every comparable scheduled GitHub-calling job already has, so enabling GITTENSORY_PR_RECONCILIATION in the future doesn't reintroduce the exact rate-limit/duplication incident class fixed elsewhere tonight.
References
src/selfhost/queue-common.ts:86-98, 882-1023 (the missing-from-both-sets job type + the coalesce switch)
src/index.ts:114, 165-169, 183 (the cron cadence + the sibling's guard this job lacks)
src/selfhost/maintenance-admission.ts:5-7, 40-66 (the incorrect header comment + the actual membership)
src/review/pr-reconciliation.ts:94-124 (the sequential per-repo reconciliation loop)
src/github/backfill.ts:3230-3274 (the paginated per-repo GitHub calls)
src/selfhost/pg-queue.ts:911-942 (the pending-only coalesce mechanism this job never reaches)
Effort
S
Part of #4496. P8 — medium severity, high confidence, but zero current blast radius (feature flag off by default today) — fix before enabling.
Context
The
reconcile-open-prsjob type does real, per-repo paginated GitHub REST work every 10 minutes (isReconciliationWindow = minute % 10 === 0,src/index.ts:114, on the*/2 * * * *cron) but is excluded from BOTH rate-limit admission and job-key dedup:GITHUB_BUDGET_BACKGROUND_TYPES(src/selfhost/queue-common.ts:86-98), so it never yields toshouldWaitForGitHubRateLimitat dequeue time — unlike every other real GitHub-REST-calling background job in this file.src/index.ts:183pushes it unconditionally with NO!sweepThrottledUntilcheck — contrast the very next block (index.ts:165-169) which DOES gatebackfill-registered-reposthis way.jobCoalesceKey(queue-common.ts:882-1023) has nocase "reconcile-open-prs", so it falls through toreturn null— a null key meanspg-queue.ts's enqueue-time coalesce/supersede path is skipped entirely and a fresh duplicate row is inserted every tick (pg-queue.ts:911-942), even while a prior scan is still processing.src/selfhost/maintenance-admission.ts's own header comment (lines 5-7) asserts everyMAINTENANCE_JOB_TYPESmember "already yield[s] to an EXHAUSTED GitHub REST budget (shouldWaitForGitHubRateLimit)" —reconcile-open-prsIS inMAINTENANCE_JOB_TYPES(line 62) but only gets that module's local-queue-depth/host-load admission, never the GitHub-budget one this comment claims — a real, confirmable contradiction between the module's own doc comment and reality.Meanwhile
runOpenPrReconciliation(src/review/pr-reconciliation.ts:94-124) sequentially loops every watched repo, andreconcileOpenPullRequests(src/github/backfill.ts:3254-3274) makes up toRECONCILE_OPEN_PRS_MAX_PAGES = 10sequentialGET /pulls?state=open&per_page=100calls per repo, then sequentially awaitscatchUpMissingPullRequestfor every missing PR found — real, potentially large GitHub REST volume with none of the protections its sibling scheduled jobs have.This cannot fire in production today —
isPrReconciliationEnableddefaults OFF (GITTENSORY_PR_RECONCILIATIONunset ⇒ false) — but the moment an operator enables this flag, the dual gap (no GitHub-budget admission + no job-key dedup, stacked) becomes real.Requirements
"reconcile-open-prs"toGITHUB_BUDGET_BACKGROUND_TYPESinqueue-common.ts(or give it its own admission check at enqueue time mirroringbackfill-registered-repos's!sweepThrottledUntilguard).case "reconcile-open-prs": return type;tojobCoalesceKeyso a second 10-minute tick coalesces into an already-pending/processing scan instead of creating a duplicate row.maintenance-admission.ts's header comment to accurately describe which job types get GitHub-budget admission vs. only local-load admission.reconcile-open-prscorrectly yields whenshouldWaitForGitHubRateLimitreports an exhausted budget (mirroring the equivalent test forbackfill-registered-repos); a regression test asserting a secondreconcile-open-prsenqueue while a prior one is pending/processing coalesces rather than duplicates; a test enabling the flag in a test harness and confirming both protections actually engage end-to-end (since this path has zero test coverage of the gap today, per the audit).Deliverables
reconcile-open-prsadded toGITHUB_BUDGET_BACKGROUND_TYPES(or equivalent enqueue-time guard)jobCoalesceKeycase added for dedupmaintenance-admission.tsheader comment correctedExpected outcome
reconcile-open-prsgets the same GitHub-budget admission and job-key dedup every comparable scheduled GitHub-calling job already has, so enablingGITTENSORY_PR_RECONCILIATIONin the future doesn't reintroduce the exact rate-limit/duplication incident class fixed elsewhere tonight.References
src/selfhost/queue-common.ts:86-98, 882-1023(the missing-from-both-sets job type + the coalesce switch)src/index.ts:114, 165-169, 183(the cron cadence + the sibling's guard this job lacks)src/selfhost/maintenance-admission.ts:5-7, 40-66(the incorrect header comment + the actual membership)src/review/pr-reconciliation.ts:94-124(the sequential per-repo reconciliation loop)src/github/backfill.ts:3230-3274(the paginated per-repo GitHub calls)src/selfhost/pg-queue.ts:911-942(the pending-only coalesce mechanism this job never reaches)Effort
S