Repository navigation
fix: only expand point-click descendants that can hold the point (#147) - #157
Merged
Merged
Conversation
iamnbutler
force-pushed
the
click-remote-hit
branch
from
October 20, 2026 03:35
0c0acc1 to
1befa72
Compare
Reading the children of SwiftUI's lazy lists materializes their rows and grows the layout. The point-click walk read all of System Settings' Spotlight pane, so the app list grew and moved Search Privacy… off the clicked point before its frame was checked, and the click was refused. Why: native/vendor/Peekaboo/Core/PeekabooAutomationKit/Sources/PeekabooAutomationKit/Services/UI/BackgroundInputDriver.swift | The point-click descendant walk must not read children of elements whose frame excludes the point: SwiftUI lists materialize rows when read and grow, moving the target off the point (System Settings > Spotlight, #147)
iamnbutler
force-pushed
the
click-remote-hit
branch
from
October 20, 2026 03:44
1befa72 to
de17075
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #147.
Cause. A background point click hit-tests the point, then walks the hit's descendants breadth-first and presses the first enabled candidate whose frame contains the point. In System Settings > Spotlight, the hit is the content hosting view. The walk read the rows of the "Results from Apps" list, and SwiftUI materializes those rows when they're read, so the list grew. Everything below it moved down, so Search Privacy… was 40–136 pt below the clicked point by the time frames were checked, and the click was refused as finding no pressable element.
Read-only probes confirmed it: three point-pruned walks left the button at y=757, and one full walk moved it to y=797. A pixel screenshot showed the content had really moved.
Fix.
BackgroundInputDriver.descendantsBreadthFirstwalksAutomationElementRepresentingand doesn't expand an element whose non-empty frame excludes the point. Elements without a usable frame are still expanded. Off-point candidates were already rejected, so only the reads change, along with the layout those reads could move.Tests. A new
BackgroundInputDriverPositionalTargetTestscase models the Spotlight pane. All 31 tests in that suite pass withnpm run peekaboo-test -- --package automationkit --filter BackgroundInputDriverPositionalTargetTests.Live check (dev app). After scrolling Spotlight to the bottom, a point click on Search Privacy… from a pixels snapshot completed and opened its sheet, which I closed with Done. On 0.16.0 the same click was refused every time.