orb(privacy): private repo names and PR numbers ship to the vendor's PostHog and to unauthenticated /metrics, with no tenant opt-out — and the scrubber corrupts the repo label while doing it

[JSONbored]

Problem

Private repository identifiers leave the box on two egress paths that the codebase's own anonymisation
machinery was built to prevent.

1. PostHog: private repo full names, PR numbers and head SHAs, with no opt-out

initPostHog resolves const apiKey = processEnvString(env, "POSTHOG_API_KEY") ?? processEnvString(env, "LOOPOVER_CENTRAL_POSTHOG_KEY") (src/selfhost/posthog.ts:143), and
control-plane/src/container-driver.ts:104 sets the central key on every tenant container start.

Every capture path runs through operationalProperties, whose allowlist OPERATIONAL_TAG_KEYS includes
repo, repository, owner, pull, pullNumber, pr, head_sha in the clear.
capturePostHogAiGeneration is called on the success path too (src/selfhost/ai.ts:1374, :1389),
passing context: { repo: options.repoFullName, pullNumber: options.pullNumber }.
scrubRecord's shouldRedactKey("repo") is false, so before_send leaves it untouched.

The neighbouring installationIdHash / hashedInstallationContext machinery exists precisely to avoid
identifying a tenant — and is defeated by the repo key sitting in the same properties bag.

processEnvString uses nonBlank, so setting POSTHOG_API_KEY="" does not disable it: it falls
through to the central key. There is no coded or documented way for a hosted tenant to turn this off.
Contrast src/selfhost/orb-collector.ts, which HMACs repo/pr with a per-instance secret the collector
never holds, and honours ORB_AIR_GAP — the exact discipline missing here.

2. /metrics: raw private repo names, on by default for every self-host

publicLabelsForMetric strips the repo label from PRIVATE_REPO_LABEL_METRICS
(loopover_gate_decisions_total, loopover_reviews_published_total, loopover_ops_anomaly_total) —
but only while self-host mode is off (src/selfhost/metrics.ts:188-223):

if (selfHostedMetricsMode || !PRIVATE_REPO_LABEL_METRICS.has(name)) return labels;

src/server.ts:355 calls setSelfHostedMetricsMode(true) unconditionally at boot. So on every
self-hosted instance those three metrics carry the raw repo="owner/name", and /metrics is served at
src/server.ts:975 before any auth check. The module's own comment concedes the risk — "metrics with
labels derived from private queue internals stay redacted because /metrics may be exposed by an
operator's reverse proxy before application/session authentication" — and then exempts exactly the
metrics that name repos. ALWAYS_REDACT_REPO_LABEL_METRICS proves the pseudonym mechanism
(redacted-N) already exists.

Latent sibling in the same file: observe() (metrics.ts:303-304) builds its series key without calling
publicLabelsForMetric, and renderMetrics re-renders h.labels raw. No histogram carries a repo
label today, so the first observe(..., { repo }) silently bypasses redaction — including the
ALWAYS_REDACT set.

3. Bonus correctness bug: the scrubber corrupts the label it groups by

scrubString runs PUBLIC_UNSAFE_SCRUB — \b(reward|score|wallet|hotkey|…)\w*\b — over every string
value, and repo reaches it via scrubStringField (src/selfhost/redaction-scrub.ts:38, :148). / is
a non-word char, so \b holds at the segment start. Reviewing ossf/scorecard emits
repo = "ossf/private context" on every event. Same for any repo whose name segment starts with
score*, reward*, wallet*, ranking*, or cohort* — and two such repos in one org collapse into a
single series.

Impact

The private-repo inventory and per-repo review activity of every hosted customer is continuously
exported to the vendor's analytics project, and to any reader of an exposed /metrics. Meanwhile the one
place the repo label is preserved correctly (OTel) disagrees with the one place it is corrupted
(PostHog), so the two telemetry systems name repos differently.

Requirements

• HMAC/hash repo and pullNumber in operationalProperties when the key came from
  LOOPOVER_CENTRAL_POSTHOG_KEY — reuse hmacAnonymize + orb:anon_secret, matching orb-collector.ts.
• Honour ORB_AIR_GAP=true and an explicit POSTHOG_DISABLED in initPostHog, and make an empty
  POSTHOG_API_KEY mean off rather than fall through to central.
• Make self-host mode use pseudonymised repo labels rather than raw ones, or gate the exemption on an
  explicit LOOPOVER_METRICS_REPO_LABELS=raw. Route observe() through publicLabelsForMetric.
• In scrubStringField, skip the vocabulary scrubbers (PUBLIC_UNSAFE_SCRUB, PRIVATE_TEXT) for the
  structured identifier keys in OPERATIONAL_TAG_KEYS, keeping the credential-shape scrubbers
  (SECRET_VALUE, JWT_VALUE, QUERY_SECRET_VALUE) unconditional.
• Document, for hosted tenants, exactly what leaves their instance.

Test Coverage Requirements

99%+ patch coverage, branch-counted. Both arms of central-vs-local key, both arms of the metrics mode,
and a regression test asserting ossf/scorecard survives scrubbing intact.

Links & Resources

• src/selfhost/posthog.ts ~143; src/selfhost/ai.ts ~1374, ~1389; src/selfhost/redaction-scrub.ts ~38, ~44-72, ~148;
  src/selfhost/metrics.ts ~188-223, ~303-304; src/server.ts ~355, ~975;
  control-plane/src/container-driver.ts ~104; src/selfhost/orb-collector.ts (the model to follow)
• orb(security): ingest endpoints fail OPEN when their token is unset, and /metrics is unauthenticated #9046 (/metrics unauthenticated — this is what is in the response, distinct), Hosted tenant containers never actually consume the central PostHog key #7876 injects #8626 (hosted tenants
  never consumed the central key — wiring; this is what the events contain once they do)

maintainer-only — tenant confidentiality.

[linear-code]

JSO-6317