Skip to content

fix(scripts): extend the packaged-secret detector (forbidden-content.mjs) with the repo's other known-precise secret formats #7433

Description

@JSONbored

Context

scripts/forbidden-content.mjs exports FORBIDDEN_CONTENT, whose own header comment calls it the
"[s]ingle source of truth for the miner package's secret-shape detector." It's consumed by two places:
(1) scripts/check-miner-package.mjs and scripts/check-mcp-package.mjs — both run in CI
(.github/workflows/ci.yml) and at publish time (.github/workflows/publish-miner.yml) to reject any
packed @loopover/miner/@loopover/mcp npm file containing secret-like content before it ships; (2)
test/unit/miner-mcp-contract.test.ts, which asserts no MCP tool response ever contains a match.

Today FORBIDDEN_CONTENT only matches: PEM-style private-key headers, github_pat_*, gh[pousr]_*,
gts_[0-9a-f]{64}, and a generic [A-Z0-9_]*(TOKEN|SECRET|PRIVATE_KEY)= assignment shape.

The same repo already has a materially larger, individually-verified-precise set of concrete secret-
format patterns living one file away, in src/review/secret-patterns.ts's SECRET_PATTERNS (the subset
in HARD_SECRET_KINDS is explicitly documented there as safe for an unconditional hard block — "near-
zero false-positive risk"). None of the following, all already implemented and tested there, are matched
by forbidden-content.mjs today: AWS access keys (AKIA[0-9A-Z]{16}), Slack tokens
(xox[baprs]-...), Google API keys (AIza...), GitLab tokens (glpat-...), npm tokens (npm_...),
Stripe live/restricted keys (sk_live_/rk_live_), SendGrid keys, Hugging Face tokens (hf_...),
Voyage/Firecrawl keys, a watermark-anchored OpenAI key pattern, and a watermark-anchored Anthropic key
pattern. Concretely: an AWS key, Slack bot token, or an OpenAI/Anthropic API key accidentally baked into
a packed miner/mcp npm file, or echoed in an MCP tool response, would NOT be caught by this specific
"single source of truth" detector today, even though this exact repo already ships a tested, precise
regex for every one of those formats.

Requirements

  • Extend FORBIDDEN_CONTENT in scripts/forbidden-content.mjs to also match all 12 of: aws_access_key,
    slack_token, google_api_key, gitlab_token, npm_token, stripe_secret_key, sendgrid_key,
    huggingface_token, voyage_api_key, firecrawl_api_key, openai_api_key, anthropic_api_key — using
    the exact same regex bodies already defined for each in src/review/secret-patterns.ts's
    SECRET_PATTERNS array. Do not invent new patterns or alter the 4 already-covered shapes
    (private-key block, github_pat_, gh[pousr]_, gts_, generic assignment).
  • First check whether scripts/** can import directly from src/review/secret-patterns.ts (composing
    FORBIDDEN_CONTENT from SECRET_PATTERNS.filter(p => HARD_SECRET_KINDS.has(p.name)) at load time) —
    scripts/check-engine-parity.ts already imports across a similar scripts/src//packages/
    boundary, so this may be straightforward. If a real ESM/build-tooling obstacle rules that out, keep
    FORBIDDEN_CONTENT a single literal RegExp and hand-copy in the missing patterns' exact bodies
    instead — either approach is acceptable, but state which one the PR took and why.
  • Do NOT add jwt, seed_or_mnemonic, or bittensor_key from that same file to the hard-block set.
    seed_or_mnemonic/bittensor_key are documented in secret-patterns.ts as weak, false-positive-prone
    heuristics deliberately excluded from HARD_SECRET_KINDS (a coldkey:/hotkey = line or the word
    "mnemonic" in ordinary Bittensor-related docs is not a leaked credential). jwt is out of scope for
    this issue — note in the PR description that it was considered and deliberately left out, don't
    silently add it and don't silently omit the note either.
  • The existing call sites (check-miner-package.mjs, check-mcp-package.mjs,
    miner-mcp-contract.test.ts) must need zero changes — this is purely a widening of what the
    shared constant matches, not a signature/API change.

Deliverables

  • FORBIDDEN_CONTENT (or its replacement composition) matches all 12 additional concrete secret
    formats listed above, in addition to the 4 it already covers
  • A short code comment in forbidden-content.mjs recording that jwt/seed_or_mnemonic/
    bittensor_key were considered and deliberately excluded, and why

Test Coverage Requirements

99%+ Codecov patch target (scripts/** itself is excluded from the Codecov coverage.include/ignore
list per codecov.yml, but the new/changed behavior is still exercised by real assertions — see below;
this repo's convention per test/unit/forbidden-content.test.ts's existing header is to pin the shared
constant's behavior directly). Add a case to test/unit/forbidden-content.test.ts asserting
FORBIDDEN_CONTENT.test(...) is true for one representative fixture string per newly-added format —
assembled from fragments the way this repo's own test/unit/secret-patterns.test.ts already does for its
AWS_EXAMPLE_KEY-style fixtures, so no real-shaped literal ever ships inside the test file itself — and
unchanged (still exactly as before) for the 4 pre-existing formats. Also re-run (or extend)
test/unit/miner-mcp-contract.test.ts's existing "rejects secret-like content" assertions against one of
the newly-added shapes to confirm the widening actually reaches that consumer.

Expected Outcome

The pre-publish npm-package secret check (check-miner-package.mjs/check-mcp-package.mjs) and the
MCP-tool-response leak assertion (miner-mcp-contract.test.ts) both actually catch an AWS/Slack/Google/
GitLab/npm/Stripe/SendGrid/HuggingFace/Voyage/Firecrawl/OpenAI/Anthropic-shaped credential — matching what
the file's own header comment ("single source of truth") already claims, but the code didn't deliver
until now.

Links & Resources

  • scripts/forbidden-content.mjs
  • src/review/secret-patterns.ts (SECRET_PATTERNS, HARD_SECRET_KINDS)
  • scripts/check-miner-package.mjs, scripts/check-mcp-package.mjs
  • test/unit/miner-mcp-contract.test.ts (line ~41), test/unit/forbidden-content.test.ts
  • test/unit/secret-patterns.test.ts (fixture-assembly convention to reuse)

Metadata

Metadata

Assignees

No one assigned

    Labels

    gittensor:bugGittensor-scored bug fix — scores a 0.05x multiplier.help wantedExtra attention is needed

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions