Infrastructure: Qualify and require native Storybook CI coverage #4349

Goal

Qualify and promote the native Storybook pipelines delivered by #4256 into reliable required desktop coverage. Parent: #4275.

Status - 2026-10-01

Open; all three pipelines exist and run native apps. macOS, Windows Fabric, and Win32 Paper explicitly build/verify native helpers, execute the owned stories-and-tests lifecycle, and retain diagnostics.

All three Storybook jobs passed in run 36927750317. The macOS log and retained Windows/Win32 reports each record 29 authored tests: 7 passed, 22 skipped. Current main catalogs contain 168 macOS, 158 Windows, and 152 Win32 stories. Hosted Windows/Win32 disable keyboard, physical click, and wheel; hosted macOS disables physical click.

Current main rules require only the PR status. The aggregate's needs excludes all three Storybook jobs and bundle-size; a green aggregate therefore does not make those checks required.

Remaining scope and acceptance criteria

• Provision authoritative, owned interactive Windows/Win32 and managed macOS lanes with the capabilities their required tests actually need. Locked sessions, denied authority, helper failures, app crashes, and assertion failures have distinguishable results.
• Publish an auditable executed cohort, not just discovered tests or render counts; integrate the regression-parity and accessibility follow-ups.
• Reconcile endpoint-specific catalog identities and the actual WDIO story/test/source digests. Full catalogs intentionally differ. portablePlanDigest currently hashes only legacy static plans (none in the component catalog), so matching that digest does not prove callback equivalence.
• Retain versioned machine-readable run, JUnit, event, process-ownership, helper identity, native log, tree, and available screenshot evidence for the active WDIO path. Failure upload must not mask the original error.
• Define bounded retry policy and expiring quarantine records with owner, reason/issue, and expiry. Quarantined/skipped cases do not count toward exit coverage.
• Name runner-capacity, escalation, artifact-retention/redaction/access, and fork-PR safety owners and document their operating policy.
• Measure at least 100 representative executions per endpoint over 14 days, with at least 99% infrastructure success, zero leaked app/helper processes, zero unreleased input, and actionable metadata for every infrastructure failure.
• Prevent all-skipped selected cohorts from being reported as successful completed coverage; preserve truthful mixed pass/skip reports.
• Promote only qualified capabilities through the real required-status path, and prove a relevant failing native job blocks merging.
• Preserve authenticated readiness, exact app/helper leases, loopback-only listeners, source verification, serial physical input, and cleanup/cancellation contracts.
• Document adding/filtering/reproducing native tests and diagnosing failures. Pixel differences remain advisory, not an implicit merge gate.

Evidence and dependencies

• Native PR jobs and aggregate
• Runner authority and promotion guidance
• Executable-test and manifest contract
• Current manifest digest construction
• Delivered milestone: Infrastructure: Add Storybook end-to-end test pipelines #4256. Coordinate Infrastructure: Restore Windows Storybook focus-regression parity #4347 (regression parity), Infrastructure: Complete portable Storybook accessibility coverage and reporting #4343/Infrastructure: Fix native accessibility-state projection gaps on macOS and Win32 #4346 (accessibility), Infrastructure: Complete Desktop Driver production release qualification #4345 (release), Infrastructure: Qualify Windows/Win32 capture fidelity and provenance #4348 (capture), Infrastructure: Restore Storybook stories excluded by native desktop crashes #4344 (catalog), and Infrastructure: Reject all-skipped WDIO runs as completed coverage #4342 (all-skipped reporting). Share reliability evidence with driver release qualification rather than claiming two independent windows.