Repository navigation
Infrastructure: Add Storybook end-to-end test pipelines #4256
Description
Goal
Deliver native end-to-end Storybook pull-request pipelines for macOS, Windows Fabric, and Win32 Paper through one supported workflow. Parent: #4275.
Status - 2026-10-01
Completed pipeline-delivery milestone. Production qualification, operations, reliability, and required promotion remain open in #4349. The all-skipped reporting bug discovered during reconciliation is #4342.
Delivered
- All three desktop PR jobs build/verify/diagnose source-built native helpers and pin subsequent resolution to verified artifacts.
- The shared
stories-and-testslifecycle traverses the native catalog and runs component-authored tests against the exact owned native app, not a fake fallback. - Named, typed WDIO callbacks execute with correlated readiness, source/manifest identity, fresh per-case preview/session/worker, and ownership-safe cleanup.
- Native build logs, doctor reports, run artifacts, and command diagnostics are uploaded with
if: always(). - The authored cohort extends beyond Button: current discovery is 29 cases across 17 stories and ten component/primitive families per endpoint.
- Hosted input restrictions are explicit. Windows/Win32 disable keyboard, physical click, and wheel; macOS disables physical click.
Delivery evidence: PRs #4294, #4306, #4313, #4314, #4320, and #4321. All three Storybook jobs passed in run 36927750317, each with 7 authored passes / 22 capability/state skips.
What closure does not mean
The pipelines are not qualified required interactive gates. Main currently requires only PR, whose aggregate excludes Storybook and bundle-size jobs. Matching portablePlanDigest also does not prove WDIO equivalence: it hashes legacy static plans, and the component catalog now uses callbacks.
The original later-stage acceptance criteria are preserved in #4349: authoritative environments, callback/catalog identity, complete active-run reporting, retry/quarantine policy, named artifact/runner owners, a measured reliability window, restored regression/accessibility cohorts, and required-status promotion.
Bugs and related follow-ups
- Infrastructure: Reject all-skipped WDIO runs as completed coverage #4342: a selected all-skipped WDIO run currently reports
passed; retained Windows/Win32 Checkbox reports reproduce this. - Infrastructure: Restore Storybook stories excluded by native desktop crashes #4344: native host crashes exclude Accordion/Callout on Fabric and Accordion/ListItem on Paper.
- Infrastructure: Restore Windows Storybook focus-regression parity #4347 / Infrastructure: Complete portable Storybook accessibility coverage and reporting #4343: complete original focus-regression parity and accessibility coverage.
- Infrastructure: Complete Desktop Driver production release qualification #4345 / Infrastructure: Qualify Windows/Win32 capture fidelity and provenance #4348: driver release and qualified capture/provenance.
- Infrastructure: Add native Storybook visual comparison reports for Windows, Win32, and macOS #4328: native baseline/candidate/diff reports remain unimplemented. Pixel differences stay advisory.
Activity
[-]Add Storybook end-to-end test pipelines[/-][+]Infrastructure: Add Storybook end-to-end test pipelines[/+]JasonVMo commentedon Sep 3, 2026
Status update - 2026-09-03
PRs #4294 and #4306 completed the pipeline foundation and native transition:
stories-and-testslifecycle;native helper before Storybook starts;
fake target;
retained; and
The jobs intentionally mask non-authoritative hosted-runner capabilities.
Windows/Win32 disable keyboard, physical click, and wheel, while hosted macOS
disables physical click. Explicit skips are truthful but do not qualify those
interactions.
Remaining: reach the documented story/component cohort, run required
capabilities on authoritative interactive runners, reconcile manifest/digest
evidence across endpoint-specific catalogs, measure flake rates, define retry
and quarantine behavior, assign artifact retention/redaction and runner owners,
add locked/non-interactive session policy, restore the focus-crash cohort from
#4254, and promote only reliable jobs to required status.
JasonVMo commentedon Oct 1, 2026
All three native Storybook E2E pipelines are delivered and current jobs execute real native tests. Closing pipeline delivery; authoritative-runner qualification, operations, reliability, reporting, and required promotion remain #4349. The reproduced all-skipped reporting defect is #4342, and catalog crashes are #4344.