Skip to content

fix(selfhost): make the Postgres connection pool size operator-tunable - #2583

Merged
JSONbored merged 1 commit into
mainfrom
fix/selfhost-postgres-pool-size
Jul 2, 2026
Merged

fix(selfhost): make the Postgres connection pool size operator-tunable#2583
JSONbored merged 1 commit into
mainfrom
fix/selfhost-postgres-pool-size

Conversation

@JSONbored

Copy link
Copy Markdown
Owner

Summary

Closes #2536.

The self-host Postgres backend (src/server.ts's buildPostgresBackend) constructed its pg.Pool with no max set, so it silently ran on pg's own hardcoded default of 10 connections shared across every HTTP handler and every queue worker's DB traffic in that instance — including jobs that fan out several concurrent writes (e.g. hydrateMergedPullRequestFiles). On a small/idle instance that's fine; at real volume (many registered repos, higher QUEUE_CONCURRENCY), a fixed pool of 10 becomes an app-side bottleneck well before Postgres's own max_connections or the existing GittensoryPostgresConnectionPressure alert would ever fire, and there was no way for an operator to raise it without a code change.

What changed

  • Added resolvePostgresPoolMax() to src/selfhost/queue-common.ts, reading a new PGPOOL_MAX env var via the module's existing parsePositiveIntEnv helper (min 1, fallback 10 — the same default pg.Pool used implicitly before).
  • Wired it into src/server.ts's buildPostgresBackend: new pg.Pool({ connectionString: url, max: resolvePostgresPoolMax() }).
  • Documented PGPOOL_MAX in .env.example and in the self-host backup/scaling docs (docs.self-hosting-backup-scaling.tsx), explaining the distinction between this per-instance app pool and PgBouncer's pooling, and pointing operators at the GittensoryPostgresConnectionPressure alert as the signal to watch when raising it.

The pool-sizing logic was extracted into queue-common.ts rather than left inline in server.ts because server.ts runs main() at module top level (unsafe to import in a test) and is explicitly excluded from Codecov's patch-coverage gate for exactly that reason — queue-common.ts is the already-proven-testable home for this kind of self-host config accessor.

Correctness notes

  • Default behavior is unchanged: PGPOOL_MAX unset still resolves to 10, matching pg's own prior implicit default.
  • An invalid override (non-numeric, zero, negative) falls back to the default with a logged warning, matching every other parsePositiveIntEnv-based knob in this module.

Validation

  • npm run typecheck
  • npx vitest run test/unit/selfhost-queue-common.test.ts
  • npm run test:coverage (unsharded, full suite)
  • npm run db:migrations:check
  • npm audit --audit-level=moderate
  • git diff --check

Scope

  • Change is narrow and limited to the stated problem
  • No secrets, wallets, hotkeys, trust scores, or reward values added anywhere
  • No edits to site/, CNAME, **/lovable/**, or CHANGELOG.md

Safety

  • No auth/session/CORS surface touched
  • No new external inputs; PGPOOL_MAX is an operator-controlled env var validated the same way as existing pool/queue knobs

The self-host Postgres connection pool is created with no `max` option, so
it silently uses the pg driver's hardcoded default of 10 connections. That
single pool is shared by every HTTP handler AND every queue worker's own
database operations, including jobs that intentionally fan out several
concurrent writes (e.g. hydrateMergedPullRequestFiles). A realistic burst --
a handful of concurrent job workers each running a fan-out step, plus normal
webhook traffic -- can plausibly want more concurrent connections than the
pool provides, well before Postgres's own connection ceiling or the existing
GittensoryPostgresConnectionPressure alert would trip. There was no
environment variable to raise this without a code change.

- Add PGPOOL_MAX, wired into the pool construction in src/server.ts via a
  small extracted, directly-testable resolvePostgresPoolMax() in
  queue-common.ts (server.ts itself has no test infrastructure -- it boots a
  real server at module load and is Codecov-ignored for exactly that reason
  -- so the resolution logic needed to live somewhere side-effect-free to be
  unit-tested at all).
- Default (10) matches the pg driver's own prior implicit default exactly,
  so this is a pure opt-in tuning knob with no behavior change when unset.
- Documented in .env.example and the self-hosting backup/scaling doc,
  including the distinction from PgBouncer (which pools connections between
  instances and Postgres; this pools connections within one instance) and
  what to watch (GittensoryPostgresConnectionPressure) before raising it
  further.

Validation: full local gate green; resolvePostgresPoolMax() is fully unit
tested (default, override, invalid-value fallback). src/server.ts itself
carries no Codecov patch-coverage obligation (already ignored in
codecov.yml, alongside pg-adapter.ts/pg-queue.ts, as self-host process-entry
code validated by the Docker build+boot smoke test instead).
@dosubot dosubot Bot added the size:M label Jul 2, 2026
@loopover-orb

loopover-orb Bot commented Jul 2, 2026

Copy link
Copy Markdown
Contributor

Tip

🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩

✅ Gittensory review result - approve/merge recommended

Review updated: 2026-07-02 10:40:00 UTC

5 files · 1 AI reviewer · no blockers · readiness 98/100 · CI green · clean

✅ Suggested Action - Approve/Merge

  • safe to merge

Review summary
This change makes the self-host Postgres pool size explicit and operator-tunable while preserving the prior effective default of 10. The implementation is small and wired through the production pool construction path in `src/server.ts`, with focused unit coverage for unset, valid, and invalid env values. The documentation explains the distinction between the app-side pool and the database/PgBouncer ceiling clearly enough for operators to use it safely.

Nits — 5 non-blocking
  • nit: `test/unit/selfhost-queue-common.test.ts:1189` leaves the `console.warn` spy active for any future tests appended after this block; restore it in the test or add `vi.restoreAllMocks()` in the describe-level cleanup.
  • nit: `src/selfhost/queue-common.ts:568` carries a long operational comment with issue-specific wording; consider trimming it to the contract and leaving the detailed tuning guidance in docs.
  • In `test/unit/selfhost-queue-common.test.ts:1173`, extend the cleanup to restore mocks so the new invalid-env test remains isolated as the file grows: `afterEach(() => { delete process.env.PGPOOL_MAX; vi.restoreAllMocks(); });`.
  • In `src/selfhost/queue-common.ts:568`, keep the helper comment focused on why the default is explicit and where the pool is used, and rely on `apps/gittensory-ui/src/routes/docs.self-hosting-backup-scaling.tsx:88` for the longer operator guidance.
  • PR author also opened the linked issue — Link an issue that was opened by a different contributor, or provide a rationale for why this self-authored issue represents genuine discovery work.
Signal Result Evidence
Code review ✅ No blockers 1 reviewer
Linked issue ✅ Linked #2536
Related work ✅ No active overlap found No same-issue or scoped active PR overlap found.
Change scope ✅ 20/20 Low review scope from cached public metadata (size label size:M; 1 linked issue).
Validation posture ✅ 25/25 PR body includes validation/test evidence.
Contributor workload ✅ 10/10 Author activity: 65 registered-repo PR(s), 55 merged, 553 issue(s).
Contributor context ✅ Confirmed Gittensor contributor JSONbored; Gittensor profile; 65 PR(s), 553 issue(s).
Gate result ✅ Passing No configured blocker found.
Review context
  • Author: JSONbored
  • Role context: owner (maintainer lane)
  • Public audience mode: oss maintainer
  • Lane context: Repository registration is not available in the local Gittensory cache.
  • Public profile languages: not available
  • Official Gittensor activity: 65 PR(s), 553 issue(s).
  • PR-specific overlap: none found.
Contributor next steps
  • Treat this as maintainer-lane context rather than normal contributor-lane activity.
  • No action.
Signal definitions
  • Related work = same linked issue, overlapping active PRs, or title/path similarity.
  • Change scope = cached public metadata such as size labels, draft state, and review-burden hints.
  • Validation posture = whether the PR provides enough public validation/test evidence for maintainer review.
  • Contributor workload = public contributor activity and cleanup pressure, not a repo-wide quality failure.
  • Contributor context = public GitHub/Gittensor identity context; non-Gittensor status is not a blocker.

🟩 Safe / merged · 🟦 Advisory · 🟨 Held for review · 🟥 Blocked / closed


💰 Earn for open-source contributions like this. Gittensor lets GitHub contributors earn for the work they already do — register to start earning →.

Checked by Gittensory, a quiet PR intelligence layer for OSS maintainers.

  • Re-run Gittensory review

@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Jul 2, 2026

Copy link
Copy Markdown

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Preview URL Updated (UTC)
✅ Deployment successful!
View logs
gittensory-ui d6b5902 Commit Preview URL

Branch Preview URL
Jul 02 2026, 10:20 AM

@codecov

codecov Bot commented Jul 2, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 95.98%. Comparing base (492c2f6) to head (d6b5902).
⚠️ Report is 23 commits behind head on main.
✅ All tests successful. No failed tests found.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #2583      +/-   ##
==========================================
+ Coverage   95.97%   95.98%   +0.01%     
==========================================
  Files         226      229       +3     
  Lines       25533    25811     +278     
  Branches     9293     9389      +96     
==========================================
+ Hits        24505    24775     +270     
- Misses        417      425       +8     
  Partials      611      611              
Files with missing lines Coverage Δ
src/selfhost/queue-common.ts 93.15% <100.00%> (+0.02%) ⬆️

... and 2 files with indirect coverage changes

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@JSONbored
JSONbored merged commit 90896fe into main Jul 2, 2026
15 checks passed
@JSONbored
JSONbored deleted the fix/selfhost-postgres-pool-size branch July 2, 2026 10:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

gittensor:bug Gittensor-scored bug fix — scores a 0.05x multiplier.

Development

Successfully merging this pull request may close these issues.

fix(selfhost): make the Postgres connection pool size operator-tunable

1 participant