Skip to content

fix(ios): keyboard chrome is unclassified in interactive captures, inflating settle diffs and the broad-transition rule #3469

Closed

Description

@thymikee

Problem

--settle on iOS loses keyboard-chrome classification. The settle loop captures with interactiveOnly: true (src/commands/interaction/runtime/stable-capture.ts, captureStableSignalWithinDeadline), and those interactive captures do not contain the Keyboard container node. collectKeyboardChrome (packages/capture-kit/src/snapshot-chrome.ts:77) needs that container, so it classifies nothing. Two effects:

  1. Noisy settle diffs. After a fill or a tap that focuses a field, the settled diff lists every key (+ @e42 [key] "Q", W, E…, about 40 lines) instead of collapsing the keyboard into one line, as withoutSettleChrome intends (settle.ts buildSettleDiff).
  2. Slow fills. The iOS broad-transition confirmation (stableCaptureSignalsHaveBroadReplacement, which fires at less than 50% shared identities) counts about 40 new Key nodes as a screen replacement. Every fill that raises the keyboard then pays the 1.5 s BROAD_TRANSITION_CONFIRMATION_QUIET_MS.

Evidence, from the iPhone 17 simulator running the RN test app's Catalog search fill: interactive capture snapshot -i --json has 82 nodes, keys at depth 2 with type Key, and no Keyboard node. withoutSettleChrome strips 0 of them. A prototype that excluded Key nodes from the replacement comparison took the fill's settle from 1806 to 775 ms (5 runs each, default window) with no late diffs.

Required behavior

  • Classification. Keyboard chrome is classified the same way in interactive captures as in full captures. Fix it where the projection drops the container, or classify from what interactive captures do carry. Do it at the owning projection or classification, not with a settle-local type !== 'Key' filter.
  • Settle diffs. A settle diff after a fill shows the keyboard as a single line, not one line per key.
  • Broad-transition check. It does not count keyboard chrome as replaced content.

Done when

  • Unit tests cover both the diff collapse and the broad-replacement verdict, built from a real interactive capture fixture with the keyboard up.
  • Live check. On an iOS simulator with the RN test app, a fill id=catalog-search kit --settle diff has no per-key lines, and its median waitedMs is about 0.8 s instead of about 1.8 s (5 runs, default window).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions