epic(rate-limit): inbound DoS vector, a permanently-stuck limiter, and outbound backoffs being cleared by the wrong bucket

[JSONbored]

Summary

Tracking epic for rate-limiting defects from the 2026-07-27 adversarial audit — inbound (protecting this service) and outbound (respecting GitHub's limits). Both directions have a defect whose worst case is webhook intake stopping for an installation, which is why they are grouped.

Sub-issues

Issue	Direction	Worst case
#9469 — webhook bucket keyed on unverified installation.id	Inbound	Unauthenticated DoS: 10 req/min silences a target installation
#9493 — INCR+EXPIRE not atomic	Inbound	A bucket denies forever until manual key deletion
#9494 — App-JWT rows, Retry-After truncation, GraphQL crowding	Outbound	Parked jobs wake into 403s every 30 min; secondary blocks extended

Known background noise (not a defect — do not chase)

rate_limit.denied audit events run 3k–30k/day from anonymous on strict routes; that is scraper traffic and is expected. It is written 1:1 with denials and pruned by audit retention, so it does not amplify. #9469 asks for a metric that distinguishes a targeted attempt from this baseline, precisely because the baseline is loud enough to hide one today.

Verified solid (recorded so it is not re-audited)

The audit specifically probed and cleared a lot of this stack. Inbound: trusted-peer-only header trust with loopback gating (#9044), rejection of spoofable XFF on untrusted topologies, session-keyed buckets for token-bearing routes, the device-flow carve-out (#6792), the bounded 1 MB body peek, and fail-open scoped to classification rather than authorization (#9223/#9225). Outbound: scoped exact+fallback SQL admission with the fallback-cannot-override-exact rule, deterministic jitter on the self-host path, peer deferral on confirmed exhaustion, foreground-liveness release, per-installation background concurrency caps, scheduled-job phase spreading (#1948), cold-cache single-flight, JWT eviction-on-401 with double eviction (#2453), broker stale-token grace, the cross-tenant mint guard (#9152), and the careful use of primary/secondary discrimination to keep rate-limit 403s out of permission classification.

The limiter is also not more expensive than the requests it blocks: one audit INSERT plus SHA-256 identity hashing per denial.

Acceptance criteria

• All three sub-issues closed.
• An unauthenticated third party cannot affect another installation's webhook intake.
• No limiter state can become permanently stuck without self-healing.
• A real exhaustion backoff is never cleared by an observation from a different bucket.

[linear-code]

JSO-6559

[JSONbored]

All child issues are closed and merged. Closing the epic.