Skip to content

fix(selfhost): make the Postgres connection pool size operator-tunable #2536

Description

@JSONbored

Parent: #1936

Problem

The self-host Postgres connection pool is created with no max option, so it silently uses the driver's default of 10 connections (src/server.ts). This single pool is shared by every HTTP request handler AND every queue worker's database operations, including fan-out jobs that intentionally run several database writes concurrently as part of one job. A realistic burst — a handful of concurrent job workers each running a fan-out step, plus normal HTTP/webhook traffic — can plausibly want more concurrent connections than the pool provides, well before Postgres's own connection ceiling or the existing Postgres-side alerting would trip. There is currently no environment variable to raise this without a code change.

Requirements

  • Expose the pool's max connection count as a documented environment variable with a sensible default (matching or close to the current implicit default, so this is a pure opt-in tuning knob, not a behavior change by default).
  • Document the tradeoff (higher pool size vs. Postgres's own max_connections, and interaction with the optional pgbouncer profile) in the self-hosting docs.
  • Add a note/warning path if the configured value would exceed a sane fraction of Postgres's max_connections, if that's cheaply detectable at boot.

Deliverables

  • A new env var (e.g. PGPOOL_MAX) wired into the pool construction in src/server.ts, with a fallback to the current default when unset.
  • .env.example entry with guidance on when/how to raise it.
  • A test covering that the env var is actually respected by the constructed pool.

Acceptance criteria

  • Setting the env var changes the pool size; leaving it unset preserves current behavior exactly.
  • Documentation clearly explains when an operator should raise this (registering many repos, high job concurrency) and what to weigh against.

Expected outcome

An operator running this at a larger scale than the reference deployment can tune database concurrency without patching the source.

Metadata

Metadata

Assignees

Labels

maintainer-onlyOwner-only work — yields no Gittensor points.roadmapOn the Wave-2 agent-layer roadmap board (project 9)

Projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions