Skip to content

DEPLOYMENT.md accuracy audit never flags real env vars that are undocumented (one-directional only) #6601

Description

@JSONbored

Context

packages/loopover-miner/lib/deployment-docs-audit.js (#5180, wired into CI via scripts/check-miner-deployment-docs.mjs and npm run test:miner-deployment-docs-audit) exists to keep packages/loopover-miner/DEPLOYMENT.md accurate: it parses the doc's claimed env vars, file-path links, and loopover-miner <subcommand> mentions, and its exported auditDeploymentDocs(claims, reality) fails CI when a DOCUMENTED claim no longer matches reality (an env var with no real read, a linked path that no longer exists, a subcommand not registered in the CLI).

This is only half the contract its own header comment describes ("assert every ... env var ... it documents still exists"): the audit never checks the reverse direction — a real, currently-read LOOPOVER_MINER_* env var that is simply missing from DEPLOYMENT.md entirely. scripts/check-miner-deployment-docs.mjs's buildLiveMinerDeploymentReality already scans packages/loopover-miner/lib/**/*.js, packages/loopover-miner/bin/**/*.js, and packages/loopover-engine/src/miner/**/*.ts for every LOOPOVER_MINER_*/MINER_* token via scanEnvVarTokens (the same helper used to parse the doc's own claims) and stores the result as envReads — but only exposes it through a boolean hasEnvRead(name) membership test, never as an enumerable set the reverse direction could diff against.

DEPLOYMENT.md (around its "Config, state & data" section) already documents ONE family of env vars generically — "each store ... also honors its own LOOPOVER_MINER_<NAME>_DB path override — e.g. LOOPOVER_MINER_PORTFOLIO_QUEUE_DB" — rather than enumerating every individual LOOPOVER_MINER_*_DB var by name. That generic sentence is a deliberate documentation choice and must stay valid, so any new reverse-direction check must NOT flag individual LOOPOVER_MINER_*_DB tokens as missing.

Outside that *_DB family, confirmed empirically against the current tree, the following real, currently-read env vars have ZERO mentions anywhere in DEPLOYMENT.md and no generic sentence covers them either:

  • LOOPOVER_MINER_AMS_COLLECTOR_TOKEN, LOOPOVER_MINER_AMS_COLLECTOR_URL (packages/loopover-miner/lib/orb-export.js's network-send wiring)
  • LOOPOVER_MINER_AMS_POLICY_PATH (packages/loopover-miner/lib/ams-policy.js)
  • LOOPOVER_MINER_CHAT_ACTIONS (packages/loopover-miner/lib/chat-action-dispatch.js's CHAT_ACTION_DISPATCH_FLAG)
  • LOOPOVER_MINER_LEDGER_RETENTION_DAYS, LOOPOVER_MINER_LEDGER_RETENTION_MAX_ROWS (packages/loopover-miner/lib/store-maintenance.js's retention opt-ins)
  • LOOPOVER_MINER_LOG_LEVEL (packages/loopover-miner/lib/logger.js)
  • LOOPOVER_MINER_NO_UPDATE_CHECK (packages/loopover-miner/lib/update-check.js)
  • LOOPOVER_MINER_REPO_CLONE_DIR (packages/loopover-miner/lib/repo-clone.js)
  • LOOPOVER_MINER_SENTRY_DSN, LOOPOVER_MINER_SENTRY_ENVIRONMENT (packages/loopover-miner/lib/sentry.js)
  • LOOPOVER_MINER_VERSION (packages/loopover-miner/lib/version.js's release-id override)
  • LOOPOVER_MINER_WORKTREE_DIR (packages/loopover-miner/lib/worktree-allocator.js's resolveWorktreeBaseDir)

The shorter bare MINER_* alias namespace is also NOT a reliable signal for a reverse check on its own: it matches non-env-var exported identifiers already in the tree (e.g. MINER_PR_OUTCOME_EVENT, MINER_EVENTS_TOTAL, MINER_GOAL_SPEC_FILENAMES, MINER_PACKAGE_VERSION are event-type/metric/filename constants, not environment variables), so including it in the reverse check would produce false-positive CI failures.

Requirements

  • packages/loopover-miner/lib/deployment-docs-audit.js's auditDeploymentDocs(claims, reality) MUST gain a new check: every token in reality's real env-var-read set that (a) starts with the literal prefix LOOPOVER_MINER_, (b) does NOT end with the literal suffix _DB, and (c) is NOT present in claims.envVars MUST produce a failure naming that exact env var (e.g. env var "LOOPOVER_MINER_LOG_LEVEL" is read under packages/loopover-miner/** but is not documented in DEPLOYMENT.md).
  • Tokens ending in _DB MUST be excluded from this reverse check — DEPLOYMENT.md documents that whole family generically via its existing LOOPOVER_MINER_<NAME>_DB pattern sentence, and this issue must not force exhaustive per-store enumeration of that already-covered family.
  • Tokens matching only the shorter bare MINER_ alias prefix (i.e. not also starting with LOOPOVER_MINER_) MUST NOT be included in the reverse (undocumented-in-reality) direction — the existing forward direction (a documented MINER_* claim must have a real read) is unaffected and stays exactly as it is today.
  • reality (the second parameter to auditDeploymentDocs) MUST gain a new field exposing the real env-var-read set as an iterable of strings (not just the existing boolean hasEnvRead(name) membership test), since the reverse direction needs to enumerate reality's own vars rather than test one name at a time.
  • scripts/check-miner-deployment-docs.mjs's buildLiveMinerDeploymentReality MUST populate that new field from the envReads set it already builds via scanEnvVarTokens, with no new source-scanning logic — reuse the existing scan, just expose it.
  • The existing forward-direction checks (documented-env-var-without-a-real-read, dead file-path links, undocumented-but-real subcommands) MUST be unchanged in behavior.
  • packages/loopover-miner/DEPLOYMENT.md MUST be updated in the same change to document every env var listed in the Context bullet list above, so the PR that ships this check also leaves the doc passing it.

Deliverables

  • auditDeploymentDocs in packages/loopover-miner/lib/deployment-docs-audit.js implementing the new reverse-direction check, scoped to LOOPOVER_MINER_-prefixed, non-_DB-suffixed tokens.
  • scripts/check-miner-deployment-docs.mjs's buildLiveMinerDeploymentReality updated to supply the new enumerable env-var-read field.
  • packages/loopover-miner/DEPLOYMENT.md updated to document the thirteen env vars listed in Context above.
  • Tests added to test/unit/miner-deployment-docs-audit.test.ts covering: an undocumented non-_DB LOOPOVER_MINER_* real read is flagged by name; a documented one is not flagged; an undocumented LOOPOVER_MINER_*_DB token is NOT flagged (the generic-pattern exemption); an undocumented bare MINER_* token (no LOOPOVER_MINER_ prefix) is NOT flagged.
  • A test added to test/unit/check-miner-deployment-docs.test.ts asserting the live-built reality object's new field is populated and the full audit passes against the real DEPLOYMENT.md after this change.

Test Coverage Requirements

packages/loopover-miner/lib/deployment-docs-audit.js is under packages/**, covered by this repo's 99%+ Codecov patch gate — every new branch in auditDeploymentDocs (flagged vs. not-flagged; _DB-suffix exemption; LOOPOVER_MINER_ vs. bare MINER_ prefix) must be exercised by the tests above. scripts/check-miner-deployment-docs.mjs lives outside packages/** at the repo root and is covered by its own existing test file (test/unit/check-miner-deployment-docs.test.ts), not the package's patch gate — but its changed lines still need a passing assertion in that file per the deliverable above.

Expected Outcome

The miner's DEPLOYMENT.md accuracy audit catches drift in both directions: a documented env var/path/subcommand that no longer exists in reality (already caught today), and a real, currently-read, non-_DB LOOPOVER_MINER_* env var that operators would have no way to discover from the docs (not caught today). DEPLOYMENT.md documents every such env var the package actually reads.

Links & Resources

Metadata

Metadata

Assignees

No one assigned

    Labels

    gittensor:bugGittensor-scored bug fix — scores a 0.05x multiplier.help wantedExtra attention is needed

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions