Skip to content

fix(selfhost): stop leaking Postgres credentials via verify-backup.sh argv - #2512

Merged
JSONbored merged 4 commits into
mainfrom
fix/verify-backup-credential-leak
Jul 2, 2026
Merged

fix(selfhost): stop leaking Postgres credentials via verify-backup.sh argv#2512
JSONbored merged 4 commits into
mainfrom
fix/verify-backup-credential-leak

Conversation

@JSONbored

Copy link
Copy Markdown
Owner

Summary

  • scripts/backup.sh (PR fix(selfhost): hide Postgres backup credentials #2459) had the same class of vulnerability this script still carries: db_identity(), the scratch pg_restore --dbname call, and the post-restore sanity psql call all passed a full postgres(ql):// URL — potentially including a password, via userinfo or the libpq password=... query-string form — directly as a process argument, exposing it via ps//proc/PID/cmdline to any other user on the same host.
  • Ported the same sanitization approach from backup.sh (see that file for the full URI-parsing rationale): strip only the password — from userinfo, restricted to the authority component so a literal @/: in the query string is never mistaken for credentials, or from a password= query parameter — and hand back everything else (host, port, dbname, every other query parameter) untouched as the connection argument, with the password supplied out-of-band via a temporary, 600-permission, wildcarded PGPASSFILE.
  • Unlike backup.sh, this script may connect to two different URLs in the same run (the live source and a scratch database) via db_identity(), so the shared logic here (pg_connect_arg) takes the URL as an argument instead of reading a single script-global $PG_DB, tracks every passfile it creates in a list for cleanup, and always unsets PGPASSFILE before checking the current URL for a password — otherwise a previous call's password could leak into a connection for a URL that doesn't have one of its own (e.g. a passwordless live source checked between two scratch-database connections).
  • Not extracted into a shared sourced helper file: docker-compose.yml's backup service bind-mounts backup.sh and verify-backup.sh as individual files at the container root, not the whole scripts/ directory, so a shared file would need its own new mount entry kept in sync by hand — more deployment coupling than the ~90 duplicated lines it would save.

Scope

  • The PR title follows type(scope): short summary Conventional Commit format.
  • This PR is focused: one script + its test file, no product code changes.
  • This follows CONTRIBUTING.md.
  • No issue is linked — this is a security follow-up to fix(selfhost): hide Postgres backup credentials #2459's fix, found via direct code audit (the same vulnerability class, in a sibling script), not a pre-filed bug.

Validation

  • git diff --check
  • npm run actionlint
  • npm run typecheck
  • npm run test:coverage — full unsharded run green on Node 22.23.1 (matching CI's pinned .nvmrc): 320 passed / 2 skipped, 6067 tests passed, 0 failures.
  • npm run test:workers
  • npm run build:mcp
  • npm run test:mcp-pack
  • npm run ui:openapi:check
  • npm run ui:lint
  • npm run ui:typecheck
  • npm run ui:test
  • npm run ui:build
  • npm audit --audit-level=moderate
  • Every existing test that mocks psql by exact URL match was updated: the real script now calls psql/pg_restore with a SANITIZED (password-free) URL, not the raw one, so fakePsql's identity map is now keyed on the sanitized form via a new sanitizedUrl() test helper mirroring the shell logic exactly.
  • Added a test exercising the full scratch-restore flow (4 psql/pg_restore calls plus the initial structural pg_restore --list) with a password supplied via the query-string form on one URL and no password on the other: asserts no password ever appears in any captured argv, and the passwordless URL's connection never inherits a PGPASSFILE left over from the other URL's.
  • Verified against every URL form backup.sh's own fix was tested against (userinfo password, query-string-only host, + in password, fake @/: in a query value, combined real-password-plus-fake-query-userinfo, query-string password= in first/middle/last position and before a fragment, percent-encoded query password, the negative case where a value merely contains the substring "password") — confirmed via a standalone shell harness before wiring into the vitest suite.
  • Reverting the fix reproduces a real r.status !== 0 failure against the sanitized-URL-keyed mocks, confirming the new tests actually exercise the behavior change rather than passing vacuously.

If any required check was skipped, explain why:

  • N/A — everything ran.

Safety

  • No secrets, wallet details, hotkeys, coldkeys, user PATs, private keys, raw trust scores, private rankings, or private maintainer evidence are exposed.
  • Public GitHub text stays sanitized, low-noise, and does not imply compensation guarantees or optimization tactics.
  • N/A — no auth, cookie, CORS, GitHub App, Cloudflare, or session changes (this is a self-host operator script, not the review engine).
  • N/A — no API/OpenAPI/MCP behavior change.
  • N/A — no UI code changes.
  • N/A — no visible UI change.
  • N/A — no docs/changelog changes needed.

Notes

… argv

scripts/backup.sh (a prior fix) had the same class of vulnerability this
script still carried: db_identity(), the scratch pg_restore --dbname call,
and the post-restore sanity psql call all passed a full postgres(ql):// URL
-- potentially including a password, via userinfo OR the libpq
`password=...` query-string form -- directly as a process argument,
exposing it via `ps`/`/proc/PID/cmdline` to any other user on the same host.

Ported the same sanitization approach from backup.sh (see that file for the
full URI-parsing rationale): strip only the password -- from userinfo,
restricted to the authority component so a literal '@'/':' in the query
string is never mistaken for credentials, or from a `password=` query
parameter -- and hand back everything else (host, port, dbname, every other
query parameter) untouched as the connection argument, with the password
supplied out-of-band via a temporary, 600-permission, wildcarded PGPASSFILE.

Unlike backup.sh, this script may connect to TWO different URLs in the same
run (the live source and a scratch database) via db_identity(), so the
shared logic here (pg_connect_arg) takes the URL as an argument instead of
reading a single script-global $PG_DB, tracks every passfile it creates in a
list for cleanup (rather than backup.sh's single $PGPASSFILE_CREATED), and
always unsets PGPASSFILE before checking the current URL for a password --
otherwise a PREVIOUS call's password could leak into a connection for a URL
that doesn't have one of its own (e.g. a passwordless live source checked
between two scratch-database connections).

Not extracted into a shared sourced helper file: docker-compose.yml's
`backup` service bind-mounts backup.sh and verify-backup.sh as individual
files at the container root (./scripts/backup.sh:/backup.sh:ro, similarly for
verify-backup.sh), not the whole scripts/ directory, so a shared file would
need its own new mount entry kept in sync by hand -- more deployment
coupling than the ~90 duplicated lines it would save.

Updated every existing test that mocks psql by exact URL match: the real
script now calls psql/pg_restore with a SANITIZED (password-free) URL, not
the raw one, so fakePsql's identity map must be keyed on the sanitized form
-- added a sanitizedUrl() test helper mirroring the shell logic exactly (a
test that could still pass keyed on the raw URL would not actually verify
the credential never reaches argv). Added a new test exercising the full
scratch-restore flow (4 psql/pg_restore calls plus the initial structural
pg_restore --list) with a password supplied via the query-string form on one
URL and no password on the other, asserting: no password ever appears in any
captured argv, and the passwordless URL's connection never inherits a
PGPASSFILE left over from the other URL's.
@loopover-orb

loopover-orb Bot commented Jul 2, 2026

Copy link
Copy Markdown
Contributor

Warning

🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨

⏸️ Gittensory review result - manual review recommended

Review updated: 2026-07-02 07:35:45 UTC

2 files · 1 AI reviewer · no blockers · readiness 95/100 · CI pending · blocked

⏸️ Suggested Action - Manual Review

  • Touches a guarded path — held for manual review

Review summary
The change routes every scratch/live Postgres connection through a sanitizer that removes userinfo and query-string passwords from argv, supplies non-empty decoded passwords through temporary pgpass files, clears stale PGPASSFILE between URLs, and cleans the files on exit. The visible implementation covers the flagged percent-encoded query key case and repeated password parameters, and the updated tests exercise the multi-connection scratch-restore flow rather than only a synthetic helper path. I do not see a reachable correctness or security break in the provided diff.

Nits — 5 non-blocking
  • nit: scripts/verify-backup.sh:128 matches decoded query keys case-sensitively; if libpq treats connection option names case-insensitively, document that assumption or normalize before comparing.
  • nit: test/unit/selfhost-verify-backup-script.test.ts:87 duplicates the shell sanitizer in TypeScript, which is useful for assertions but can drift; add a small direct capture assertion for a mixed-case or invalid-percent key if you keep expanding sanitizer behavior.
  • scripts/verify-backup.sh:128: consider comparing `url_decode "$pg_key_raw"` after a lowercase transform only if libpq option names are case-insensitive; otherwise add a comment stating lowercase `password` is the supported URI keyword spelling.
  • test/unit/selfhost-verify-backup-script.test.ts:87: keep the mirrored sanitizer narrowly scoped and add a test fixture whenever the shell parser gains a new parsing rule, because this helper can otherwise encode the same mistake as production.
  • Touches a guarded path — held for manual review — A maintainer must review and merge this change.
Signal Result Evidence
Code review ✅ No blockers 1 reviewer
Linked issue ✅ No-issue rationale PR body explains why no issue is linked.
Related work ✅ No active overlap found No same-issue or scoped active PR overlap found.
Change scope ✅ 20/20 Low review scope from cached public metadata (size label size:L; no linked issue context).
Validation posture ✅ 25/25 PR body includes validation/test evidence.
Contributor workload ✅ 10/10 Author activity: 65 registered-repo PR(s), 55 merged, 548 issue(s).
Contributor context ✅ Confirmed Gittensor contributor JSONbored; Gittensor profile; 65 PR(s), 548 issue(s).
Gate result ⚠️ Not blocking Advisory; not blocking this PR.
Review context
  • Author: JSONbored
  • Role context: owner (maintainer lane)
  • Public audience mode: oss maintainer
  • Lane context: Repository registration is not available in the local Gittensory cache.
  • Public profile languages: Python, TypeScript, JavaScript, Ruby, Go, Kotlin, MDX, Shell
  • Official Gittensor activity: 65 PR(s), 548 issue(s).
  • PR-specific overlap: none found.
Contributor next steps
  • Treat this as maintainer-lane context rather than normal contributor-lane activity.
  • Triage stale or unlinked PRs.
  • No action.
  • Link the issue being solved, or explicitly explain why this is a no-issue PR.
Signal definitions
  • Related work = same linked issue, overlapping active PRs, or title/path similarity.
  • Change scope = cached public metadata such as size labels, draft state, and review-burden hints.
  • Validation posture = whether the PR provides enough public validation/test evidence for maintainer review.
  • Contributor workload = public contributor activity and cleanup pressure, not a repo-wide quality failure.
  • Contributor context = public GitHub/Gittensor identity context; non-Gittensor status is not a blocker.

🟩 Safe / merged · 🟦 Advisory · 🟨 Held for review · 🟥 Blocked / closed


💰 Earn for open-source contributions like this. Gittensor lets GitHub contributors earn for the work they already do — register to start earning →.

Checked by Gittensory, a quiet PR intelligence layer for OSS maintainers.

  • Re-run Gittensory review

@loopover-orb loopover-orb Bot added gittensor gittensor:bug Gittensor-scored bug fix — scores a 0.05x multiplier. labels Jul 2, 2026
…word

The AI review on #2459 (scripts/backup.sh) found that the query-string
password-stripping there only removed the FIRST `password=` occurrence, so a
malformed URL repeating the key (not rejected by libpq's own parser) left a
second occurrence sitting in argv, still a leaked credential. This script's
pg_connect_arg has the identical query-string-password logic, so it carries
the same gap -- ported the identical fix: loop the extraction until no
`password=` remains rather than stripping once. Each iteration overwrites
pg_password_value, so the LAST occurrence is what ends up in the PGPASSFILE;
which one libpq itself would use for a duplicate key is unspecified, but
every occurrence is a credential either way, so none may reach argv.

Also added a full-scratch-restore-flow test for the userinfo-password form
(user:password@host) -- the existing multi-connection flow test only proved
the query-string form end to end, per a non-blocking nit from the same
review round asking for both forms to be exercised through the complete
flow, not just the isolated single-URL cases already covered.

Verified against the duplicate-key case and every prior regression case
(still passing). Reverting just the loop fix reproduces the exact residual
leak.
JSONbored added 2 commits July 2, 2026 00:06
… name

The AI review on the sibling PR #2519 (scripts/backup.sh) found that the
query-string password-stripping there only matched the LITERAL string
"password=", so a percent-encoded key name like `pass%77ord=secret` (%77
decodes to 'w', and libpq percent-decodes query key names before matching
them against connection keywords) still leaked a real credential into argv.
This script's pg_connect_arg has the identical logic, so it carries the
same gap -- ported the identical fix: walk each '&'-separated query pair
individually, decode only the key half of each, compare the decoded key
against "password", and rebuild the query from every pair whose decoded key
isn't a match, in original order, with values left percent-encoded exactly
as given.

Added a matching regression test through the full scratch-restore flow.

Verified against the exact encoded-key case and every prior regression case
(still passing). Reverting just this change reproduces the exact leak.
…hecks

db_identity() is invoked via command substitution ($(db_identity ...)),
which forks a subshell -- calling pg_connect_arg from inside its body
meant the PG_PASSFILES cleanup-list append never propagated back to the
parent, orphaning a real, credential-bearing 600-permission passfile on
disk for every identity check that needed one.
@JSONbored
JSONbored merged commit 82c46da into main Jul 2, 2026
7 checks passed
@JSONbored
JSONbored deleted the fix/verify-backup-credential-leak branch July 2, 2026 07:41
@github-project-automation github-project-automation Bot moved this from Todo to Done in gittensory - v1 roadmap Jul 2, 2026
@codecov

codecov Bot commented Jul 2, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 95.95%. Comparing base (7e0acf4) to head (2013ee2).
⚠️ Report is 35 commits behind head on main.
✅ All tests successful. No failed tests found.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #2512   +/-   ##
=======================================
  Coverage   95.94%   95.95%           
=======================================
  Files         226      226           
  Lines       25369    25411   +42     
  Branches     9227     9242   +15     
=======================================
+ Hits        24341    24383   +42     
  Misses        417      417           
  Partials      611      611           
🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

gittensor:bug Gittensor-scored bug fix — scores a 0.05x multiplier.

Projects

No open projects
Status: Done

Development

Successfully merging this pull request may close these issues.

1 participant