feat(webapp): per-client database pool metrics that survive the driver adapter

[ericallam]

What

Follow-up to #4539. The driver-adapter work is inert until a client flips to the pg driver adapter, but the moment one does, our database observability degrades: the OTel metrics pipeline reads pool stats from Prisma's $metrics, which is owned by the Rust engine's quaint pool. Under the adapter, pg.Pool owns the pool, so those gauges read zero. The pipeline also only ever scraped a single client (the control-plane writer singleton).

This PR makes database metrics driver-agnostic and per-client:

• Every configured client registers a metrics source: control-plane writer/replica, run-ops writer/replica, legacy writer/replica. Previously only the control-plane writer singleton was scraped.
• Each OTel instrument is observed per client with db_client and db_driver (quaint | pg-adapter) attributes. db_client uses our canonical datasource-role labels (control-plane-writer, control-plane-replica, run-ops-writer, run-ops-replica, legacy-run-ops-writer, legacy-run-ops-replica) — the same strings used for the db.datasource span attribute, so a metric and a trace point at the same pool.
• Pool figures come from the authoritative source per driver:
  • pg-adapter: pg.Pool (totalCount/idleCount/waitingCount, plus cumulative opened/closed from connect/remove events).
  • quaint: the Rust engine's $metrics pool gauges/counters, exactly as before.
• Query counters and duration histograms still come from $metrics for both drivers (the Rust engine executes queries in both cases).
• New db.pool.connections.waiting gauge (pg.Pool exposes this; quaint reports 0).
• Stops exporting Prisma metrics from the Prometheus /metrics route. Pool observability now lives entirely in the OTel pipeline, per driver, per client.

Why

So we can flip any client (including the control-plane writer, the primary desync-fix target) to the driver adapter without losing pool visibility. Existing dashboards keyed on the same metric names keep working; they gain a per-client dimension.

Testing

Unit (apps/webapp/app/utils/databaseMetrics.server.test.ts): the pure normalizer — quaint reads pool from $metrics; adapter reads pool from pg.Pool and keeps engine query metrics; busy never goes negative; graceful zeroing when $metrics is unavailable (adapter still reports live pool figures).

Live smoke test against a prod-shaped local stack: three physically-distinct Postgres DBs (control-plane, run-ops, legacy) behind dual PgBouncers, split mode on, with a mix of adapter and quaint clients. Reading the actual emitted OTel metrics, every pool shows up as its own series:

db.pool.connections.total{db_client="control-plane-writer",  db_driver="pg-adapter"} = 1
db.pool.connections.total{db_client="control-plane-replica", db_driver="quaint"}     = 1
db.pool.connections.total{db_client="run-ops-writer",        db_driver="pg-adapter"} = 1
db.pool.connections.total{db_client="run-ops-replica",       db_driver="quaint"}     = 1
db.pool.connections.total{db_client="legacy-run-ops-writer", db_driver="quaint"}     = 1
db.pool.connections.total{db_client="legacy-run-ops-replica",db_driver="quaint"}     = 1
db.client.queries.total{db_client="control-plane-writer",db_driver="pg-adapter"} = incrementing
db.client.queries.duration.count{db_client="control-plane-writer",db_driver="pg-adapter"} = incrementing

Confirms: metrics are attributed per pool with the correct driver; adapter pools' figures come from pg.Pool; and query counters/duration histograms keep incrementing under the pg adapter. Also verified /metrics (Prometheus) now returns zero prisma_* series while still serving the app's own metrics.

pnpm run typecheck --filter webapp passes.

Notes

• /metrics (Prometheus) no longer includes prisma_* series. Anything scraping that endpoint for Prisma metrics should read the equivalent db.* metrics from the OTel exporter instead.
• PgBouncer + ?schema= gotcha (separate from this PR, worth flagging for rollout): since feat(webapp,database): opt-in per-client Prisma driver adapters #4539 parses ?schema= from the DSN and passes { schema } to the adapter, node-postgres sends search_path as a startup parameter. A transaction-mode PgBouncer rejects that with FATAL: unsupported startup parameter: search_path. Our prod control-plane DSNs use the default public schema with no ?schema= param, so this is latent, but any client we flip to the adapter must not carry ?schema= in its DSN (or the pooler needs ignore_startup_parameters = search_path).

[changeset-bot]

⚠️ No Changeset found

Latest commit: 2ec585b

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

[github-actions]

Observability map

As of 2ec585b.

18/100 over 413 measured of 429 entry points (base 18, no change)

What this PR changed
No entry point this PR touches changed its score.

FIX FIRST

• /api/v1/projects/:projectRef/envvars (sensitive) - auth-boundary, request-context
• /auth/sso (sensitive) - auth-boundary, request-context
• /_app/orgs/:organizationSlug/settings/team (sensitive) - error-classification, auth-scope, request-context

AUDIT 3 of 50 sensitive mutations record an actor. 47 without one.
CONTEXT 11 of 413 entry points name a tenant on a failure path. 325 appear only here, 39 of them sensitive, in the JSON rather than the fix list.

What the score is made of

CHECKS
  error-classification  168 applicable,  94 pass,   0 sole, global without it 10
  auth-boundary          62 applicable,  57 pass,   0 sole, global without it 15
  auth-scope             19 applicable,  17 pass,   0 sole, global without it 18
  request-context       413 applicable,  11 pass, 223 sole, global without it 64
  audit-trail            50 applicable,   3 pass,   0 sole, not in the score

The score and findings here are report-only and never gate the merge. Separately, a required test suite keeps this tool's symbol and route lists in sync with the code they name, and can fail a pull request that renames or removes a symbol they reference, or that adds the first route with a segment they anticipate. Each failure names the list to edit. The rules and their reasons: internal-packages/observability-map/README.md.

[coderabbitai]

Walkthrough

The change adds shared database metric registration and normalization. Standard Prisma and driver-adapter clients register pool, connection, query, and histogram sources. The tracer collects metrics for every registered client and records client, driver, pool, and waiting-connection observations. The metrics route now serves metricsRegister without separate Prisma metrics collection. Tests cover normalization and fallback behavior.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Title check	✅ Passed	The title clearly summarizes the main change: per-client database pool metrics that support driver adapters.
Description check	✅ Passed	The description is detailed, on topic, and includes testing results, but it omits several template sections such as the checklist and screenshots.

✨ Finishing Touches 💡 2

📝 Generate docstrings 💡

• Create stacked PR
• Commit on current branch

🛠️ Fix failing CI checks 💡

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch feat/db-pool-metrics-per-client

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[devin-ai-integration]

Devin Review found 2 new potential issues.

[devin-ai-integration]

🟡 Database metrics report zeros instead of being skipped when a client's stats read fails

Query and pool figures for a database connection are reported as zero (json = undefined at apps/webapp/app/utils/databaseMetrics.server.ts:139-142, then zero-filled by normalizeDatabaseMetrics) whenever reading that connection's stats fails, so dashboards show phantom drops to zero and cumulative totals appear to reset.
Impact: Monitoring can briefly show a connection pool as empty and query totals as restarting from zero, producing false alerts and misleading rate spikes on the next successful read.

Cumulative counters observed as 0 on a failed $metrics read (behavior change from previous early-return)

Previously the batch observable callback read Prisma metrics once and, on any failure, returned without observing anything — a failed scrape simply produced no data points. Now collectDatabaseClientMetrics swallows the per-client failure (apps/webapp/app/utils/databaseMetrics.server.ts:136-142) and normalizeDatabaseMetrics zero-fills every $metrics-derived value (apps/webapp/app/utils/databaseMetrics.server.ts:81-88, 104-110, 118-123). The callback then unconditionally observes those zeros for the monotonic instruments db.client.queries.total, db.datasource.queries.total, and (for quaint clients) db.pool.connections.opened.total / closed.total (apps/webapp/app/v3/tracer.server.ts:524-527).

For cumulative counters, a value of 0 followed by the real value is interpreted downstream as a counter reset, yielding an increase equal to the entire lifetime total. For quaint clients the pool gauges (open, busy, idle) also read 0, which looks like a dead/empty pool.

A safer shape is to signal "stats unavailable" from the collector (e.g. leave the engine-derived fields undefined) and skip res.observe for those instruments, while still observing the live pg.Pool figures for adapter clients.

Prompt for agents

In apps/webapp/app/utils/databaseMetrics.server.ts, collectDatabaseClientMetrics swallows a failed `source.client.$metrics.json()` and passes `undefined` to normalizeDatabaseMetrics, which zero-fills every engine-derived value (query counters, query gauges, and for quaint clients the pool gauges/counters). The batch observable callback in apps/webapp/app/v3/tracer.server.ts then observes those zeros for cumulative counters (db.client.queries.total, db.datasource.queries.total, db.pool.connections.opened.total/closed.total) and gauges. The previous implementation returned early on failure and observed nothing, so a failed read produced no data points instead of a spurious zero. Consider propagating an 'engine metrics unavailable' signal from the collector (e.g. an `engineMetricsAvailable: boolean` field or leaving the engine-derived fields undefined) so the tracer callback skips observing engine-derived instruments for that client, while still observing live pg.Pool figures for adapter clients. Update the unit tests in databaseMetrics.server.test.ts accordingly.

---

Was this helpful? React with 👍 or 👎 to provide feedback.

[devin-ai-integration]

🔍 Removing prisma metrics from /metrics also removes the only Prometheus-side pool visibility

The Prometheus route no longer emits any prisma_* series; pool visibility now exists only through the OTLP metric exporter. Note this is conditional: when INTERNAL_OTEL_METRIC_EXPORTER_ENABLED === "0", setupMetrics() returns the no-op global meter and never calls configurePrismaMetrics (apps/webapp/app/v3/tracer.server.ts:389-392), so such deployments (including self-hosters) lose database pool observability entirely rather than merely relocating it. No in-repo dashboards reference prisma_*, so nothing in this repo breaks.

---

Was this helpful? React with 👍 or 👎 to provide feedback.