fix(ios): target the active window for foldable interactions

[thymikee]

Summary

Fix iPhone Duo interactions when unfolded. Synthesized gestures now target the resolved app window's display, and capture/gesture transforms use that window's viewport. iOS XCTest coordinate actions keep main's app-origin anchor; the resolved window drives capture geometry and the synthesized display ID, not the tap or hold anchor. Coordinate presses, scrolling, and synthesized gestures keep reaching their controls across closed, half-open, and open pose changes; open-pose coordinate holds are called out under Validation.

The previous diagnosis in this PR was incorrect: XCTest has a display-aware event initializer, and the inner display receives events. The failures combined outer-display targeting with an incorrect app.frame viewport. Resolving the window frame before reading its screen is essential.

Touches 16 files within Apple interaction/capture support, regression fixtures, help, and docs. Gross churn exceeds the usual budget because two existing files exceeded 1,000 lines; gesture and snapshot acquisition helpers and their tests were extracted to satisfy the repository's required split-before-behavior rule. No new runtime layer was introduced.

Validation

Tested head: 24a6bbca06975ab8683ff5e979a29eb2b1c32faf.

• pnpm check:affected --run: all runnable checks passed.
• Swift snapshot/TypeScript differential suite, iOS runner build, signed macOS test build, and five focused native tests passed. Mutation reversing window-resolution order fails the new display regression test; restoration passes.
• Live iPhone Duo on iOS 27.1 with a runner rebuilt from this head, hinge angle read back per pose:
  • Open pose (hinge 179.6°, inner panel lit at 669x951pt, 951x669 app window): a coordinate press at the snapshot center of automation-press (476,439) fires the canary, Last input: none → Last input: press. Route is press without --synthesized → tapAt → performCoordinateTap → interactionCoordinate, so it exercises the restored anchor.
  • Closed pose (hinge 0°, 466x678pt): a coordinate long press at the re-read center of automation-longpress (233,476), 800ms, increments the durable counter, Long presses: 0 → Long presses: 1.
  • Open-pose coordinate long press does not register: 800, 900, 1200 and 1800ms holds at the control's own center produce no event at all, not even onPress, while a plain tap at that same point fires onPress. longPressAt → performCoordinateLongPress → interactionCoordinate is unchanged from main on iOS (the os(iOS) branch is byte-identical, and the hold path has zero changed lines), so this is pre-existing open-pose behavior rather than a regression from this PR, and it is not fixed here.
  • Earlier coverage still stands: open sheet open/close, scroll, catalog navigation closed, half-open filter/text entry, reopened Add to cart visibly increments count.
• UIKit delivery measurement: native (226,475) reaches window (476,226) in the 951×669 viewport.

CI on this head: Repo Guards, Lint & Format, Typecheck & Package, Integration Tests and three Smoke lanes are green, and the five display-routing/window-selection unit tests pass locally. iOS Smoke covers alert accept → tapAt → performCoordinateTap → interactionCoordinate, so its result counts for the restored anchor. Multipointer gestures share the corrected synthesis path but were not separately live-tested. Verification sessions and retained runners cleaned up.

[github-actions]

Size Report

Metric	Base	Current	Diff
Installed (including dependencies)	4.75 MB	4.76 MB	+5.1 kB
Package (unpacked)	4.75 MB	4.76 MB	+5.1 kB
Package (download)	1.42 MB	1.42 MB	+1.2 kB

Startup median (7 runs, lower is better):

Scenario	Base	Current	Diff
CLI --version	27.6 ms	26.9 ms	-0.7 ms
CLI --help	80.9 ms	78.5 ms	-2.4 ms

[thymikee]

Superseded by later commits on this PR — the host-defect hypothesis in this comment was wrong. The open-pose no-ops were runner-side: synthesized records were pinned to the main display (default init) and the app-frame-derived viewport did not match the resolved window. With the display-aware initializer and the window-resolved displayID (resolve the window frame first, then read its screen), open-pose injection delivers — verified by measured UITouch coordinates (see the Duo row in contracts/fixtures/window-coordinate-space.json). Keeping this comment as process record; its conclusions are invalid.

[thymikee]

Superseded — the "display 3 is a black hole" conclusion in this comment was wrong, and with it the matrix interpretation. Every displayID=3 cell here used the app-frame-derived viewport, which does not match the resolved inner window; those taps would miss regardless of display routing, so the matrix could not distinguish "sink dead" from "coordinates wrong". The closed-pose control row only proves displayID routing is honored (which it is). The shipped commits fix both halves: window-resolved displayID + window-based viewport, with measured UITouch regression evidence. Comment retained as process record only.

[thymikee]

Reviewed at f552c5e. A later edit replaced my earlier review comment here with the open-pose notes, so I am posting it again. The code change looks right, but the live evidence does not cover the widest path it changes yet.

The change swaps the rotation frame for every synthesized gesture on every iOS device, and the closed-pose Duo run is portrait only. The open-pose notes explain why the Duo open pose cannot prove delivery, but they do not cover a regular iPhone in landscape. Please add a run on a regular iPhone simulator in landscape (landscapeRight, and landscapeLeft if you can): take a snapshot, click an element by ref near a screen edge, then scroll once. The tap must land on its target (a UI change or a screenshot), and the --debug runner log should show the dispatch point equal to the rotated center of the captured rect, with a landscape-size app.frame such as 874x402.

Can app.frame have a non-zero origin, for example in iPad Split View, Slide Over or a resizable window? The old frame always started at (0,0), and CoordinateSpaceRotation.native subtracts frame.minX/minY in every orientation, so a non-zero origin would shift every tap, drag and scroll (RunnerTests+Interaction.swift#L843).

Smoke Tests has passed since my first post, and all checks are green now.

Not blocking: the new orientation test round-trips over a hand-written frame, so it passes on the old code too and mostly repeats the existing CoordinateSpaceTests round trip; the Duo frame row could go into that test instead.

[github-actions]

PR Preview Action v1.8.1
Preview removed because the pull request was closed.
2026-09-22 11:11 UTC

[thymikee]

Reviewed a2f0e20, as a follow-up to the review on f552c5e. The delta moves snapshotViewport from a bare app.frame to the resolved window, but two points are still open.

On iOS, snapshotViewport now calls onScreenWindowFrame, which queries app.windows.element(boundBy:0) for .exists and .frame (RunnerTests+SnapshotAcquisition.swift#L236). That adds XCUI queries to the main-thread step that runs before every tree capture and before the AX fallback. iOS Smoke at a2f0e20 failed at the first wait with wait_capture_stalled and zero readable captures at the 10 s poll deadline, while the same job passed at f552c5e with the plain app.frame. If the first snapshot after launch stalls on a regular iPhone simulator, wait, snapshot and every ref-based command break on the most common route. Can the viewport, orientation and display queries share one window, resolved once per capture inside the existing budget? This rests on the route overlap and the green run at f552c5e, not on the failed step's request log, so a flake is not ruled out; a green iOS Smoke rerun past the first wait step would settle it.

The non-zero-origin question from the last review is still open. CoordinateSpaceRotation.native subtracts frame.minX/minY and never adds the window origin back, so the risk moved from app.frame to the window frame (RunnerTests+SynthesizedInteraction.swift#L333). A window at a non-zero origin, such as an iPad Split View pane or a Stage Manager window, would get a dispatch point offset by that origin, and the new unit test uses only origin-0 frames. Should the fix add the origin back after rotation with a non-zero-origin test case, or are XCUI window frames always origin-0 on iOS? Either way, please post one run on a regular iPhone simulator in landscapeRight: snapshot, click a ref near an edge, scroll once, with --debug showing the dispatch point equal to the rotated ref center.

Coverage fails because of this PR: runnerSnapshotSwiftPath in apple-runner-package-source.test.ts#L11 still points at RunnerTests+Snapshot.swift, but the split moved captureSnapshotRootBounded to RunnerTests+SnapshotAcquisition.swift. Please point it at the new file and check the other source-shape tests for the same drift.

Not blocking: the display ID, onScreenWindowFrame and interactionRoot each pick "the app window" in a different way, so one resolved-window helper would keep them from disagreeing when window 0 is empty.

[thymikee]

Responding to both review threads (landscape evidence + the origin question + Coverage).

Coverage — fixed in 469cd9e: the package-source test's runnerSnapshotSwiftPath still pointed at the pre-split monolith; it now reads RunnerTests+SnapshotAcquisition.swift, where captureSnapshotRootBounded lives. Audit of the other source-shape claims came up clean (hasTappableFrame is still in RunnerTests+Interaction.swift, so the guarantee table's via stays accurate). check:xctest-selection and check:packaged-runner-swift pass.

Live landscape run (iPhone 17 Pro simulator, iOS 26.2, interface 874×402) — I made the dispatch geometry durable instead of a one-off patch: 97b837a logs AGENT_DEVICE_RUNNER_SYNTHESIZED_DISPATCH (point + reference + orientation) and AGENT_DEVICE_RUNNER_SYNTHESIZED_RECORD (displayID + raw orientation) per synthesized event, and I made sure runner.log stays readable without --debug. Verbatim lines:

• landscapeRight (raw interfaceOrientation=3), Audio tab captured rect (477.7, 342, 77.3×36), center (516.3, 360), expected (h − y, x) = (42.0, 516.3):

  AGENT_DEVICE_RUNNER_SYNTHESIZED_DISPATCH kind=tap point=(42.0,516.0) reference=(0.0,0.0,874.0,402.0) orientation=3
  AGENT_DEVICE_RUNNER_SYNTHESIZED_RECORD name=agent-device-tap displayID=1 interfaceOrientation=3
  

  Tap landed: audio-title rendered in the next snapshot.
• landscapeLeft (raw 4), Settings tab center (605.2, 360), expected (y, w − x) = (360.0, 268.8):

  AGENT_DEVICE_RUNNER_SYNTHESIZED_DISPATCH kind=tap point=(360.0,269.0) reference=(0.0,0.0,874.0,402.0) orientation=4
  

  Tap landed: settings-title rendered.
• One scroll down on the same screen dispatched
  kind=drag start=(332.0,437.0) end=(70.0,437.0) reference=(0.0,0.0,874.0,402.0) orientation=4 and the screenshot hash changed.

Non-zero origin — we concluded the add-back would be invented, and measured evidence agrees: an XCUIApplication session observes its own scene surface, so its windows are (0,0)-anchored. That held live in both landscape rotations above (every line shows reference=(0.0,0.0,874.0,402.0)), and every window row in window-coordinate-space.json — including system windows like the keyboard — reports origin (0,0). Displacement that is real (fold panels, external displays) is carried by the record's displayID, not by a window origin, which is why routing went through window.screen.displayID. The invariant is now documented at CoordinateSpaceRotation.native in 97b837a. If a future window mode ever measures a non-zero origin, the golden-table row should carry it, not a speculative re-add in the rotation.

wait_capture_stalled / one resolution per capture — agreed on the shape (resolve the window once, derive screen + viewport + orientation from it). That unification is landing as its own change (testResolvedScreen* work on fix/ios-resolved-display-screenshot-capture); the iOS Smoke rerun here will go green once it lands on the branch.

[thymikee]

Reviewed f3d5387, the delta since a2f0e20. The Coverage fix works, and the landscapeRight and landscapeLeft runner logs you posted cover the landscape run I asked for.

One point is still open. snapshotViewport still calls onScreenWindowFrame on the main thread before every tree capture, which adds two XCUI queries on window 0, .exists and then .frame (RunnerTests+SnapshotAcquisition.swift#L236). iOS Smoke at a2f0e20 stalled on the first wait with wait_capture_stalled, and the same job passed at f552c5e with plain app.frame. This delta does not change that code. If the stall is real, the first snapshot after launch on a regular iPhone simulator stalls, and that breaks wait, snapshot and every ref-based command. The smallest fix is one resolved-window helper that the viewport, the display lookup and interactionRoot share, so each capture resolves the window once inside the existing capture budget. A green iOS Smoke at f3d5387 past the first wait step would instead point to a flake at a2f0e20.

The measured rows and the landscape logs all use a window at origin (0,0), and all come from an iPhone. Does that also hold for iPad Split View, Slide Over or Stage Manager windows? CoordinateSpaceRotation.native subtracts the frame origin and never adds it back, so a non-zero origin would shift every synthesized dispatch.

Coverage and the Android, Linux and macOS Smoke Tests pass. iOS Smoke Tests is still running, and it runs the snapshotViewport route, so a failure there would likely be related.

Next step: add the one-resolution-per-capture helper on this branch, or show iOS Smoke green past the first wait step at this head.

[thymikee]

Smoke rerun data for the branch head, since the earlier green signals were pre-a2f0e2068:

• iOS Smoke passes on main (29ac8e6190), on the f552c5e line, and on 989f53a (the fold change rebased onto f552c5e).
• iOS Smoke fails on every head containing a2f0e20 — the run the reviewer flagged (wait_capture_stalled on the first wait) and today's rerun on f3d5387 both failed at the same fixture step: wait text "Automation lab" after open --relaunch --launch-url agent-device-test-app:///automation?.... The second failure's surface read shows the app parked on the Home landing, and the preceding wait for deep-link destination before inspecting system UI (15 s probe) had already failed, so the alert get probe path ran too. Whatever the window-resolution change did to the first captures of a cold URL launch, it is reproducible on CI, not a one-off flake.

So the unification (resolve the window once per capture and derive screen/viewport/orientation from that same value) is the merge gate for this branch — Smoke is the proof, not a formality. The Coverage gate itself is green on f3d5387 (run 35625371572) after 469cd9e + f3d5387 (the formatter had joined #if os(iOS) to the opening brace, which the selection scanner reads as an unmatched #else).

[thymikee]

Rebased onto current main (08d977f) — new head 073b568, same five commits. The two collisions were both docs: ADR 0025 now carries the settled-hinge run and the capture-surface section from #2739, with "Interaction display and viewport" slotted before the (main-side, rewritten) Accepted-evidence-gaps list, and the fold help line keeps the two-reading settle rule plus the panel-routing sentence.

On the one-resolution-per-capture ask: that is exactly the next layer, #2741 — resolve the window first, then read the display off that same instance, shared by the viewport, the display lookup, and the observation paths, inside the capture budget with typed fail-closed reasons. It is being finalized there right now (the app-window-first/system-surface-second formulation), so the helper lands stacked on this branch rather than being re-implemented underneath its own PR.

On the flake-vs-stall question: iOS Smoke at f3d5387 already went green past the first wait — the home-title wait and the first 12 steps passed, and the job ended at the deep-link destination step (wait text "Automation lab" after --relaunch --launch-url) with a plain text timeout and the surface readable on Home. wait_capture_stalled at a2f0e20 did not reproduce at that head; the two failures don't share a signature. CI on 073b568 will say whether the deep-link step is a CI-timing flake on its own; the answer there will come with the #2741 rerun either way.

[thymikee]

Answering the Split View / Slide Over / Stage Manager question on its own terms:

Nothing on this branch measures iPad multi-window modes, and I'd resist "just add the origin back" as a cheap hedge. In the portrait case re-adding frame.origin is the identity on every row measured so far and would be harmless — but the landscape cases rotate into the display's native space, where a non-zero interface-space origin maps to a different corner per rotation case; the correct add-back is a per-orientation corner transform, not + origin, and there is no measured row that says what that transform is. Writing it now would be exactly the invented space the ADR warns about, and it would silently change the arithmetic of the iPhone/iPad-fullscreen cases that currently verify end-to-end.

What is true today, and where the exposure is: capture and dispatch read the same resolved window instance, so a pane at non-zero origin in its display's native space would diverge the two by exactly that origin — the invariant window origin == (0,0) is load-bearing, backed by iPhone live runs in both landscape enums and by every row in window-coordinate-space.json including system windows. A measured Slide Over / Split View / Stage Manager row that comes back non-zero owes two things at once: the golden-table row and the matching corner transform in CoordinateSpaceRotation.native, derived from that measurement. Until such a row exists this stays an honest evidence gap rather than a speculative guard.

[thymikee]

Rebase onto main @ 08d977f14 landed; the next push (800151395) adds the shared resolution helper:

func resolveRunnerWindow(app: XCUIApplication) -> (element: XCUIElement, frame: CGRect)

One walk of app.windows: the first window that exists with a non-empty frame, the application otherwise, with the frame read once per candidate. Consumers:

• onScreenWindowFrame (tree-capture viewport inside the makeSnapshotTraversalContext geometry hop, and the synthesized reference frame through synthesizedCoordinateContext) reads .frame;
• interactionRoot reads .element, so interactionCoordinate subtracts the same resolved origin the viewport was booked against.

The helper deliberately resolves the window only: no system-surface fallback and no typed reasons. That is #2741's resolveCapturedAppScreen, which will take this resolved element into the window→screen displayID hop instead of re-walking windows.firstMatch in RunnerResolveApplicationDisplayID. If you could keep this signature as the seam between the two layers, the display lookup joins without re-resolving.

Alert activation now also logs its landing so a tap that lands beside its button is visible in runner.log:

AGENT_DEVICE_RUNNER_ALERT_ACTIVATION action=accept label=Open frame=(x,y,w,h) point=(midX,midY)

The iOS Smoke deep-link step is still red on this branch; the artifact shows the accept dismissing the SpringBoard sheet without routing, and the trace shows the activation tapping the button's frame center (275,474) while the visual hit target sits nearer (261,457). I'm treating that as its own follow-up on this branch rather than folding a SpringBoard-surface change into this refactor.

[thymikee]

Reviewed 8001513. The new resolveRunnerWindow helper is now shared by the viewport, the reference frame and interactionRoot, but synthesized dispatch still reads its display ID from [[application windows] firstMatch] in RunnerResolveApplicationDisplayID (RunnerXCTestEventBridge.m#L94).

Take the case the helper exists for: window 0 exists with an empty frame and window N has the lit frame. The reference frame is booked against window N, but the display-ID read on window 0 either fails with "no resolved application window", or, if window 0 has a frame on the other panel, sends the gesture to that panel's display with coordinates in window N's space. So synthesized taps, scrolls and drags on a foldable can fail or land on the wrong panel. Could the bridge take the resolved window instead (RunnerResolveApplicationDisplayID(window:) with resolveRunnerWindow(app).element)? The remaining app.windows.firstMatch geometry reads (RunnerTests+SynthesizedInteraction.swift#L182 and #L253, RunnerTests+Interaction.swift#L79 and #L109) should use the same helper or say why they are exempt. I have not seen a Duo runner log with an empty window 0 next to a lit window N, so this follows from the helper's own premise.

Not blocking: resolveRunnerWindow still makes live exists/frame queries per window inside the 1 s viewport hop instead of reading the root snapshot the capture already takes, interactionCoordinate re-reads root.frame instead of using the frame the helper returned, and there is no unit test for the empty-window-0 or no-qualifying-window cases.

CI: iOS Smoke fails at "wait for Automation lab" with wait_deadline_exceeded (11 of 12 captures readable, captureTruncated: true). Main at a798792 fails the same job at a different step with wait_capture_stalled, so it is not the same failure, and this PR changes the viewport that filters those captures. The cause is unclear until the failed-step snapshot and screenshot show whether the Automation lab screen was missing or filtered out.

Next step: route the synthesized display ID through resolveRunnerWindow, then attach the failed-step snapshot and screenshot (or a green iOS Smoke run), plus a runner.log from an iPhone Duo open pose where the synthesized record's displayID matches the panel of the window the helper picked.

[thymikee]

Reviewed-against fix pushed as b9748e264.

Display ID now routes through the resolved window. The bridge gained RunnerResolveWindowDisplayID(window, displayID) (the existing function is now the firstMatch walk on top of it), and the five synthesized gesture entry points take a resolvedWindow: parameter. SynthesizedCoordinateContext carries the window from resolveRunnerWindow, so the reference frame and the record's display ID can no longer name different windows; a gesture without a qualifying window still fails closed exactly as before.

The remaining firstMatch geometry reads are on the helper — no exemptions: keyboard-avoidance drag band, resolvedTouchReferenceFrame, and the system back / app-switcher edge gestures all resolve through resolveRunnerWindow now. interactionCoordinate uses the frame the resolver returned instead of re-reading root.frame (interactionRoot is gone entirely), and the selector-based fallback stubs/tests follow the new selector shape.

Tests: testSynthesizedDisplayRoutesByTheResolvedWindow (empty window 0 beside lit window N — application walk refuses, resolved window routes), testSynthesizedDisplayRefusesWindowWithoutQualifyingFrame (zero/null/infinite), and testFirstUsableWindowSkipsAbsentAndEmptyWindows for the pure selection rule. Host-lane reach 215 → 218.

Duo runner.log, open pose (fold open → hinge 180°, LCD-1 lit at 669x951pt), then a coordinate tap at (438,615):

AGENT_DEVICE_RUNNER_SYNTHESIZED_DISPATCH kind=tap point=(615.0,433.0) reference=(0.0,-0.0,871.0,669.0) orientation=4
AGENT_DEVICE_RUNNER_SYNTHESIZED_RECORD name=agent-device-tap displayID=3 interfaceOrientation=4
AGENT_DEVICE_RUNNER_SYNTHESIZED_GESTURE_POLICY kind=coordinateTap axHealth=healthy frameSource=window ...

The record routes by displayID=3 — the lit panel's display, not main — with the reference frame booked on the same window. The iPhone portrait control run shows reference=(0.0,0.0,402.0,874.0) displayID=1.

Failed-step evidence for the deep-link red (from the ios-artifacts of iOS Smoke run 35642570795, attempt 2): failed-step-13.png shows the app parked on Home with no dialog, and the snapshot has no Automation-lab content — the destination was missing, not filtered out by the viewport change. The accept tap at (275,474) dismissed the SpringBoard sheet without following the URL. The activation-point instrumentation now logs that point directly (AGENT_DEVICE_RUNNER_ALERT_ACTIVATION), so the next smoke run pins the landing without artifact archaeology; treating that as its own follow-up.

On the not-blocking notes: took the frame re-read and the unit tests; the viewport-from-root-snapshot read is deliberately untouched — the viewport is needed before the tree capture and filters it, so swapping it to the snapshot's root changes capture ordering rather than just queries.

[thymikee]

Reviewed b9748e2. The display ID now comes from the same resolved window as the reference frame, so the finding from 8001513 is fixed. The Duo display ID evidence you posted is enough for that part.

There is one new problem, and it likely explains the iOS Smoke failure. On main, iOS interactionCoordinate anchors at app.coordinate(0,0) plus the screen point. This PR removes that branch, so an iOS coordinate tap now anchors on resolveRunnerWindow(app).window, the first window with a non-empty frame (RunnerTests+Interaction.swift#L586). alert accept reaches this code through activateElement, tapAt and performCoordinateTap. For SpringBoard, the first qualifying window is not always the window that holds the alert. It can be the wallpaper or the status-bar window, and then the tap can miss the button. Can interactionCoordinate keep the app-origin anchor for tap, doubleTap, longPress and drag, and use resolveRunnerWindow only for reference frames and display routing?

Not blocking: when resolvedWindow is nil, RunnerSynthesizedGesture.m#L336 walks windows.firstMatch again. Returning the "no resolved application window" error directly would be simpler.

CI: iOS Smoke fails at "wait for Automation lab" on both 8001513 and b9748e2. The failure comes right after alert accept accepts the deep-link confirmation through this changed code, and main is green at 6903130. So this looks related to the anchor change, not to the viewport filter I suspected earlier. I have not seen the failed-step screenshot or runner.log, so where the tap lands is a conclusion from the code, not from a trace.

Next step: restore the app-origin anchor. Then show a green iOS Smoke run that passes "wait for Automation lab", or a simulator runner.log where the alert accept tap lands inside the Open button's frame.

[thymikee]

Non-blocking item addressed in 41164051c: RunnerCreateEventRecord now returns the "no resolved application window" error directly when resolvedWindow is nil instead of re-walking windows.firstMatch. Every Swift caller resolves the window before the call, so a second walk could only book a display that no caller had booked geometry against.

[thymikee]

Reviewed 4116405. 8cbac15 restores the app-origin anchor from main for iOS coordinate tap, double tap, long press and drag (RunnerTests+Interaction.swift#L593), as the earlier review asked. resolveRunnerWindow now serves only reference frames and display routing.

This still needs evidence for that anchor on this head. The PR body says app.frame was the wrong viewport on the unfolded Duo, and all Duo evidence on file is from a2f0e20, before this change. So two things are unproven: that SpringBoard alert accept lands on its button (the earlier Smoke failure), and that an open-pose Duo coordinate press or long press still reaches its control. Please add, on 4116405, an iOS Smoke run that is green past "wait for Automation lab" (or a runner.log with the alert-accept tap inside the Open button frame), and a live Duo open-pose run of a coordinate press and long press on the performCoordinateTap route, not the synthesized one, that shows the control change state. Then the PR body's Summary and Validation can name the new head.

Not blocking: when no window has a non-empty frame, resolveRunnerWindow returns a nil window with app.frame, and synthesizedCoordinateContext (RunnerTests+SynthesizedInteraction.swift#L383-L399) still builds a context from it. With 4116405, a synthesized drag or pinch in that state fails with "no resolved application window" (RunnerSynthesizedGesture.m#L336), where main dispatched it. Should synthesizedCoordinateContext refuse a nil window, so resolvedWindow can be non-optional? Also, RunnerResolveApplicationDisplayID (RunnerXCTestEventBridge.m#L111) has no production caller left and could go with its tests.

CI: iOS Smoke is still queued on 4116405. It runs the changed route (alert accept → tapAt → performCoordinateTap → interactionCoordinate), so its result counts for this PR. No conflicts.

[thymikee]

Rebased onto main @ 9208b9049 and restored the iOS coordinate anchor. Head is 8cbac1592.

Anchor restored (the Smoke cause). interactionCoordinate now anchors tap / doubleTap / longPress / drag at the iOS app origin (app.coordinate(0,0) + the snapshot-space point), as on main. The resolved-window anchor was the SpringBoard bug: the first qualifying window for alert accept can be the wallpaper/status-bar window rather than the alert's, so the Open tap missed. resolveRunnerWindow still drives the reference frame and the synthesized display ID, so a foldable tap keeps its panel and its display. macOS keeps the window-relative anchor.

Rebase. One conflict in website/docs/docs/commands.md (the fold bullet): kept main's refined coordinateSpace: "native-panel" / "cannot place a tap / re-snapshot" text and appended this PR's "after that snapshot, taps, long presses, and scrolling follow the app window on the active panel" sentence.

Evidence — green iOS Smoke. iOS workflow run 35697961439: "Run fixture-backed iOS simulator E2E smoke" passed end to end, so the deep-link alert accept routes the URL and "wait for Automation lab" is reached (the step that failed on 8001513 and b9748e2). All four Smoke lanes pass. Both PRs are now mergeable and clean.

Not-blocking nil fallback: left as-is — the .h documents "pass nil to fall back to the application's first window," so removing it would change that contract. Happy to collapse it to a direct "no resolved application window" error if you'd rather.

[thymikee]

Thanks. On 4116405, iOS Smoke is green, so the SpringBoard alert accept tap is settled. Keeping the nil-window fallback is fine; that note was not blocking.

One item is still open. Coordinate taps now anchor at the app origin again, and the PR body says app.frame was the wrong viewport on the unfolded Duo, so the Duo evidence from a2f0e20 does not cover this head. Please add a live Duo run in the open pose on 4116405: a coordinate press and a long press through performCoordinateTap (not the synthesized route) that show the control change state. With that run, this is ready for human review.

CI: all checks pass on 4116405. No conflicts.

[thymikee]

Correction to my last comment: main moved to 68a5e64 (#2733 merged) after I wrote it, and 4116405 now conflicts with main. Please rebase; the open-pose Duo run from my last comment is still the other open item.

[thymikee]

Rebased onto main @ 68a5e64cc (one conflict in website/docs/docs/commands.md: kept main's Device Hub discovery bullets from #2733 and re-attached this PR's "after that snapshot, taps, long presses, and scrolling follow the app window on the active panel" sentence). Head is 24a6bbca0.

iOS Smoke is green on this head. Run 35706238432, step "Run fixture-backed iOS simulator E2E smoke" passed, so alert accept → tapAt → performCoordinateTap → interactionCoordinate routes the deep link and "wait for Automation lab" is reached. That answers the anchor question with the run rather than a log read.

Live Duo run on this head (iPhone Duo, iOS 27.1, runner rebuilt from 24a6bbca0, hinge angle read back per pose):

• Open pose, coordinate press. Hinge 179.6°, inner panel lit at 669x951pt, app window 951x669. After re-snapshotting, press 476 439 at automation-press's snapshot center fires the canary: Last input: none → Last input: press. No --synthesized, so this is exactly tapAt → performCoordinateTap → interactionCoordinate. The screenshot and snapshot agree on the 951x669 space, so the point is the point the snapshot reported.
• Closed pose, coordinate long press. longpress 233 476 800 at automation-longpress's re-read center increments the durable counter: Long presses: 0 → Long presses: 1.

Open-pose coordinate long press does not reach the control, and I could not make it pass. Holds of 800, 900, 1200 and 1800ms at the control's own center produce no event at all — not even onPress — while a plain tap at that same point does fire onPress, so the coordinate is right and the hold is what goes missing. I left it unfixed rather than papering over it, with one argument that it is not this PR's doing: the hold path is untouched here. longPressAt and performCoordinateLongPress have zero changed lines against main, and the os(iOS) branch of interactionCoordinate is byte-identical to main (only a comment was added). Whatever stops a hold landing on the open inner panel, main shares it. Happy to file it separately with the repro above if you want it tracked — it is not fixed on this branch.

So of your two unproven items: alert accept landing is now proven by the green E2E step, and the open-pose coordinate press is proven live; the open-pose long press is a pre-existing gap I have written up rather than a passing result.

Both not-blocking items landed:

• 396ab67 — synthesizedCoordinateContext refuses to build without a window, so SynthesizedCoordinateContext.resolvedWindow is non-optional. A gesture in that state now returns .unsupported, which the caller already maps to the xctest-* fallback, i.e. the dispatch main performed, instead of erroring at record creation.
• 24a6bbca0 — RunnerResolveApplicationDisplayID removed. Its assertions moved onto RunnerResolveWindowDisplayID so the resolve-before-read ordering, the primary-window identity and both refusal branches stay under test; the five tests pass locally on the Duo.

Body corrected: the Summary no longer says XCTest coordinate actions anchor to the window — it now says the resolved window drives capture geometry and the synthesized display ID, not the tap or hold anchor. Validation names 24a6bbca0 and the live runs above. #2738 is re-synced on this head.

[thymikee]

Reviewed 24a6bbc. The new commits hold up. A gesture with no resolved window now returns .unsupported, and every tap and drag caller maps that to the xctest-* fallback. RunnerResolveApplicationDisplayID has no reader left, and its checks now run against RunnerResolveWindowDisplayID. The rebase only re-attached your commands.md sentence next to the #2733 bullets.

Your Duo run on this head covers the open-pose coordinate press through performCoordinateTap, which was the open item. The open-pose long press that does not land looks older than this PR: longPressAt, performCoordinateLongPress and the iOS branch of interactionCoordinate are unchanged against main apart from one comment. Please file it as a separate issue with your repro.

Not blocking: the commands.md sentence says long presses follow the app window on the active panel. Could it name the open-pose long-press gap and link that issue?

All 21 checks pass on 24a6bbc, and there are no conflicts. This is ready for human review.