JS screenshot suite: ~25% flaky hard-stall at ButtonTheme (worker↔host Document-wrapper degradation)

[shai-almog]

Summary

After the host-ref canvas-leak fix (#5143) the JavaScript screenshot suite reliably reaches 94/94 matched in ~75% of CI runs, but ~25% of runs hard-stall at ButtonThemeScreenshotTest (the first DualAppearanceBaseTest theme test, ~suite index 85) and time out (exit 5, SUITE:FINISHED never emitted).

Symptom

The worker goes completely silent immediately after CN1SS:INFO:suite starting test=ButtonThemeScreenshotTest — no further console output, not even other scheduler tasks. Because the cooperative scheduler is single-threaded, total silence means a synchronous loop with no yield (a parked host-call would let other tasks keep logging). The in-worker per-test watchdog (awaitTestCompletion) therefore can't fire, so the whole run times out regardless of CN1SS_ALLOWED_MISSING.

Root cause (diagnosis)

The stall is inside DualAppearanceBaseTest.installModernThemeIfRequested() → Resources.open(in) parsing the modern .res, where in comes from the JS port's getBundledAssetAsDataURL asset-read bridge. Deep in the suite the worker↔host bridge intermittently returns a degraded receiver — the chartDocStaleness family where getDocument() / createElement returns null (or a coerced Number), re-wrapping the Document and wiping its cached __class. A degraded asset stream then wedges Resources.open in a parse loop.

This is the same degradation behind the historical chart-tail issues (partially addressed in 08b1248, 5dce6a2) and is orthogonal to the canvas leak fixed in #5143.

Attempts that BACKFIRED (don't repeat)

• Pre-warming the modern .res at suite start (read+cache once, before pressure): made it worse — the suite-start Resources.open did host-bridge round-trips that wiped the Document class early, turning the one late ButtonTheme wedge into early animation_grid_failed=NullPointerException (hung at index 11/21/25). Reverted.
• Trimming the per-test settle (1500→700ms): perturbed animation-grid capture timing → same early grid NPE. Reverted.

Real lever

Fix the worker↔host Document/canvas wrapper degradation itself (in browser_bridge.js / parparvm_runtime.js — wrapJsObject class-wipe on null re-resolve, and/or the host bridge returning a Number/degenerate for getDocument/getContext under load). This has resisted multiple prior partial fixes and needs a dedicated investigation.

Follow-up to #5143.

[shai-almog]

Deep-dive findings (paused here) — #5145 is multi-mechanism

Investigated via host+worker id-tracing across several CI runs (the wedge reproduces ~50% of runs). Summary so the next session can resume cold.

It is NOT host-computes-wrong; it is the callback channel + a release regression

• Host logs are captured (startParparVmApp, DIAG:INIT present in browser.log).
• NUMBER_LEAK = 0 (host never computes a number for getContext/getDocument) and receiverRehydrated = 0 (host resolves the right object) — yet the worker resumes those object-reads with a number or a thrown error.

Three distinct mechanisms, all in a late-suite dense dimension-read burst (host-call ids ~1180-1219, values 375/667/187/222/1 = canvas width/height):

1. getDocument (object return) via=value → receives a number. A response cross: the object-read resumes with a nearby getWidth/getHeight value. No id collision (HOSTREQ_DUP=0) and no real-id double-post (HOSTCB_DUP only ever fires for id=0, the shared no-response/fire-and-forget bucket — harmless).
2. getWidth/getContext via=throw "Missing host receiver". The canvas resolveHostRefs to null on the host. Strong lead: likely a regression from fix(JS port): release dead canvas host-refs to stop late-suite screenshot bridge thrash #5143's host-ref release — releaseHostRefs deletes a canvas from hostRefById once the worker's wrappers are GC'd, but a canvas can still be reachable via a raw __jsValue host-ref marker (the @JSBody-unwrap hole that fix(JS port): release dead canvas host-refs to stop late-suite screenshot bridge thrash #5143 guarded against for non-canvas refs only). Worker then uses the released canvas → null receiver → throw.
3. Layer-3 emit/capture stall — SCREENSHOT_START pngLen=... logged then total worker silence. __cn1_capture_canvas_png__ (a host native invoked directly via invokeHostNative, NOT the JSO-bridge path) appears to never respond. This is what actually wedges the suite — and it is untouched by the JSO-bridge retry.

Retry mitigation (PR #5146, branch fix-5145-document-wrapper-degradation)

Re-issuing degraded idempotent reads (getters + getContext/getBundledAssetAsDataURL/getImageData/measureText), incl. thrown transient errors, provably recovers (1)+(2) — JSO_RETRY ... recovered=yes 8/8 in a wedge run. But the suite still wedges via (3), so the retry alone does not close #5145. PR #5146 also carries diagnostic-only instrumentation (HOSTCB/HOSTREQ/JSO_RETRY) that must be stripped before any merge.

Recommended next steps

• (2): make fix(JS port): release dead canvas host-refs to stop late-suite screenshot bridge thrash #5143's release safe — free the canvas backing store but keep the hostRefById entry (so a raw-marker re-reference resolves instead of throwing), or only release detached canvases never re-referenced. Add resolve-after-release detection to confirm.
• (1): pin the callback response-cross — needs worker-observable tracing of pendingHostCalls/resolveHostCall id↔value at resume time (the worker nextHostCallId++ is monotonic, so suspect the resume/enqueue path or an async-handler ordering interaction, not an obvious id dup).
• (3): instrument __cn1_capture_canvas_png__ host handler entry/exit to find why it never responds late in the suite (likely toDataURL/toBlob on a degraded/detached canvas, or the rAF attempt loop never settling).

Repro / tooling

• parparDiag=1 (harness default) enables host diag(); parparBridgeDiag=1 enables per-op HOST jsoBridge diags.
• Wedge signature: WebSocket server summary: 66 screenshot(s) written + Timed out waiting for CN1SS:SUITE:FINISHED (exit 5).
• Playwright CANNOT attach CDP to the worker — worker introspection needs raw CDP / puppeteer or in-VM logging.