Problem
Counterfactual sub-epic phase 4 (capstone). With the harness proven, close the loop the same way #8138 did for thresholds: a PR that bumps a reviewer prompt version gets an advisory comparison — old prompt vs new — replayed against the fixture set, surfaced on the PR, never blocking.
Requirements
⚠️ Required pattern. Mirror the #8138/#8139 advisory posture end-to-end: path-filtered trigger on the prompt-version constants, update-in-place PR comment via the shared renderer, persisted calibration.counterfactual_backtest_run events feeding the SAME track-record aggregation, fail-open on every operational error (a broken replay must never red a PR — the #8140-era advisory-CI lesson).
- Spend guard: the check runs only when the deployment has an explicit counterfactual budget flag set; otherwise it posts a 'skipped: no replay budget configured' notice.
- Sampling per the design contract keeps per-PR cost bounded and deterministic per head SHA.
Deliverables
Links & Resources
The sub-epic (parent), the harness issue (blocked-by), #8138/#8139 (the posture), scripts/backtest-logic-check-core.ts
Boundaries
Advisory forever until a separate, explicit authority decision (the #8105 pattern). Budget-gated by default-off flag.
maintainer-only — CI + spend wiring.
Problem
Counterfactual sub-epic phase 4 (capstone). With the harness proven, close the loop the same way #8138 did for thresholds: a PR that bumps a reviewer prompt version gets an advisory comparison — old prompt vs new — replayed against the fixture set, surfaced on the PR, never blocking.
Requirements
Deliverables
Links & Resources
The sub-epic (parent), the harness issue (blocked-by), #8138/#8139 (the posture), scripts/backtest-logic-check-core.ts
Boundaries
Advisory forever until a separate, explicit authority decision (the #8105 pattern). Budget-gated by default-off flag.
maintainer-only — CI + spend wiring.