ops: a post-merge workflow can stay red for days with no signal — generalise #9951's escalation

[JSONbored]

A post-merge workflow can stay red indefinitely with nobody noticing

selfhost.yml is push-to-main only. It caught migration 0209's SQLite-only AUTOINCREMENT (#10138) correctly, on the very first push — the "applies every migration" test against real Postgres failed exactly as designed. Then it stayed red across five consecutive runs while PRs kept merging, and the breakage was only discovered by deploying the resulting image to production and watching the Orb crash-loop.

Nothing was wrong with the test. The gap is that a post-merge failure blocks nothing and pages no one, so a red run is indistinguishable from a red run somebody is already handling.

This is a known class, already solved once

#9951 hit the identical thing with the publish workflows — MCP and miner publishes failed on every single main commit for as far back as the run history went (a one-line missing build step), and its own comment says why it went unnoticed:

nobody noticed because the only signal was a ::warning:: nobody reads and a red check that looks like release noise

It added consecutive-failure escalation as ~30 lines of inline bash inside mcp-release-please.yml. Correct, and now needed in a second place.

Fix

Move the mechanism into scripts/escalate-workflow-outage.ts and have both workflows call it, rather than growing a second copy:

• Consecutive failures at the head of the run history, threshold 3. One red run is a flake; a deterministic failure fails identically every time. Below threshold it stays silent deliberately — an alert that fires on every transient red is how an alert gets muted.
• Once per outage. An open tracking issue is reused rather than a new one filed per commit.
• Never fails its caller. The workflow it runs in has already failed; turning "couldn't read the run history" into a second red is noise on top of the real problem.

selfhost.yml gets an escalate-persistent-failure job gated on failure() (never always() — a success must file nothing) with issues: write, which is the one permission build-boot deliberately lacks.

The arithmetic worth pinning

leadingNonSuccessCount has one case that is easy to get exactly backwards: a window with no success anywhere. indexOf("success") returns -1 there, and -1 treated as a count reports "no failures" for the single worst state — a workflow that has never once succeeded in its recorded history. That is precisely the shape #9951 found and the shape selfhost.yml was in. Unit-tested, along with cancelled/timed_out/null counting as non-successes so a cancelled run cannot silently reset the streak.

Behaviour change for the existing caller: the tracking issue title becomes workflow-generic (workflow outage: X has failed on consecutive runs) rather than publish-specific. No open issue currently uses the old title, so no reuse is broken.

[linear-code]

JSO-7098