Repository navigation
ios: text-entry lanes fail intermittently on unrelated branches (unpinned simulator keyboard state) #1874
Description
Activity
- added 12 commits that reference this issue
on Aug 20, 2026 - added a commit that references this issue
on Aug 21, 2026 15 remaining items
- added 6 commits that reference this issue
on Aug 25, 2026 Body rewritten to match what actually landed (#2035,
7db5ad73dd). Summary of the corrections, since the previous text would have sent the next reader down two dead ends:The two symptoms had two unrelated causes, not "one cause" as filed.
smoke:form-inputis root-caused and fixed, and it was never a race:field-name's placeholder was"Ada Lovelace"and every checkout-form suite filled exactly"Ada Lovelace"(same forfield-email/ada@example.com) — twelve fills across eight files. An empty field renders its placeholder as its AX value, so the commit wait refuses before polling rather than claiming a commit it cannot prove. It only surfaced under load because the synthesized-replacement route is gated onxCTestChannelPenalized. That also explains the datapoint this issue found most puzzling —rerun_failed_jobsreproducing identically on32847092583. That was not persisted per-device state; it was the same guard firing on the same input.testBareType…is mitigated, not root-caused: the flat 3s deadline could not tell a throttled pipeline from a wedged one, and now grants time against progress with a 10s ceiling.Both originally filed mechanisms are disproven, and are kept in a "Superseded hypotheses" section with the measurements rather than deleted. Pinning
ConnectHardwareKeyboardis a no-op under headlesssimctl boot(A/B table in the body) — with the reassuring corollary that a headless lane has no hardware keyboard to inherit, so the ambient state this issue is named for is not a variable in these lanes. And neither failing signature reacheswaitForTextEntryReadiness; the readiness fallback was hardened anyway, because that latent sensitivity was real, just not what was firing.What the issue is now open for is stated explicitly, with the discriminator: the
[DEBUG-1874]cadence tells throttle (prefix grows, absorbed) from wedge (prefix freezes, condemned at 3s) from the placeholder guard (no poll lines at all), so the next red lane should be readable from the uploadedrunner.logwithout adding instrumentation.The occurrence table gained the two runs recorded in comments but missing from it, including
32889322172— this issue's own fix branch, which is where the decisive trace came from.I have dispatched
1874-diagnose.ymlonmain(25 iterations × isolated/pair × x86_64/arm64) to try to force a cadence trace post-fix rather than wait for one: run 32983174128. Results to follow.
Generated by Claude Code
Post-fix diagnose dispatch: 100/100 green, and the residual slowness is before the wait
Run 32983174128 on
mainat7db5ad73dd— 25 iterations × isolated/pair × x86_64/arm64.leg verdict min median p90 max arm64 isolated 25 passed, 0 stalled 654 ms 796 ms 1131 ms 14146 ms arm64 pair 25 passed, 0 stalled 628 ms 702 ms 807 ms 1106 ms x86_64 isolated 25 passed, 0 stalled 927 ms 1175 ms 1790 ms 2581 ms x86_64 pair 25 passed, 0 stalled 908 ms 1180 ms 1542 ms 1765 ms durationMsis thetype-allphase — the whole synthesized dispatch, not just the wait.Three things worth recording.
1. The loop no longer reproduces the failure. 100 iterations, zero stalls, on the same harness and hardware class that produced the filed episodes. That is consistent with #2035 absorbing them, and it is not evidence the throttle is gone — see (3).
2. Rosetta costs ~40% at the median (arm64 796/702 ms vs x86_64 1175/1180 ms), independently corroborating the ~30% figure that motivated the
AGENT_DEVICE_XCUITEST_ARCHS=arm64pin in #1941.3. Every slow outlier settled on the FIRST poll. Five iterations exceeded 2 s across the two
isolatedlegs — 14146, 4021, 2581, 2403, 2245 ms — and all five reportpolls=1. The commit wait observed the text on its first look; the seconds went somewhere before it, in the synthesize post or the element resolution ahead of it. That reframes this issue's open question: not "why does delivery throttle mid-burst", but "what blocks before the first observation".A related shape, offered as hypothesis rather than conclusion (n is small): both outlier-carrying legs are
isolated; neitherpairleg has a single outlier, on either arch.pairaddstestBareDelayedTypeFailsWhenTappedInputDisappearsMidCommand, which sorts first and so launches the fixture app before the measured test runs. Cold input/AX path, warmed by the neighbour, would explain both that and theiter=1outlier present in each isolated leg.The dispatch also proved the harness was not collecting the evidence
Zero
[DEBUG-1874]cadence lines and zero per-iteration logs came back — the artifacts are fourstall-summary.txtfiles and nothing else. The harness dumped cadence only for failing iterations, and since #2035 an absorbed episode passes. So a fully green run, containing a 14-second outlier, produced no trace of it.#2059 fixes that: cadence is captured for any iteration slower than 2000 ms (a threshold measured from the 100 healthy iterations above), and the classifier reads xcodebuild's own verdict word instead of counting every non-
passedresult as a stall — which matters now that #2035 gave the looped test anXCTSkipIf.Next step for closing this issue: re-dispatch after #2059 lands. If the outliers reproduce with cadence attached, the
synthesize posted … tookMsline should localize the pre-poll cost, which is theneeds-infothis is labelled for.
Generated by Claude Code
The dispatch answers it: the input path is fast, the accessibility read path is what stalls
Run 33072911858 — 25 iterations × isolated/pair × x86_64/arm64, on the fixed harness (#2059), which unlike the previous dispatch captures cadence for slow passes.
100/100 passed again, 0 skipped, 0 failed, 0 no-result. Distributions:
leg median p90 max arm64 isolated 827 ms 1076 ms 1869 ms arm64 pair 842 ms 896 ms 1085 ms x86_64 isolated 1083 ms 2042 ms 2166 ms x86_64 pair 1407 ms 2427 ms 14334 ms The 14.3 s iteration, with its cadence attached for the first time:
[DEBUG-1874] synthesize posted 17 chars status=0 tookMs=484 [DEBUG-1874] wait start expectedLen=17 route=append [DEBUG-1874] poll t=440ms observedLen=17 expectedPrefixLen=17 [DEBUG-1874] wait outcome=settled elapsedMs=1229 route=appendPosting the 17 characters took 484 ms. The commit was observed on the first poll, and the whole wait was 1229 ms. Yet
type-all— which bracketstypeIntoCurrentTarget— measured 14334 ms.So roughly 12.6 s went to neither the synthesis nor the commit. What is left inside that bracket is
resolveTextEntryElementandeditableTextValue: accessibility round-trips, before a single character is posted.That reframes this issue's premise. It is titled and was investigated as an input-delivery throttle. On this evidence the input pipeline is not throttled at all — 484 ms to post, first-poll commit. What stalls is the AX read path, which independently matches the 2026-08-21 note that "during stall windows everything touching the input system is slow (AX reads ~12s)". Those AX reads are the same ones #1105 tracks.
Caveat, and a defect this dispatch exposed: the per-iteration logs were not in the artifact, so this rests on the 40-line cadence excerpt rather than the full log.
.tmpis a hidden directory andupload-artifactskips those unless told to include them — so every per-iteration log this workflow has ever kept was silently discarded. Fixed in #2059 (include-hidden-files: true); the next dispatch will carry the intervening lines, which would let the 12.6 s be attributed to a specific AX call rather than to the pair of them.Proposed disposition
Both filed symptoms are resolved:
smoke:form-inputroot-caused and fixed (the fixture placeholder collision), and the targeted XCTest mitigated by the progress-aware budget, with 200 consecutive green loop iterations across two dispatches. The remaining slowness is real but is an AX-read cost, not the delivery throttle this issue is named for — and it no longer reddens lanes.I would close this as mitigated, with the throttle premise corrected, and let the AX-read cost be pursued under #1105 where that path is already tracked. #2080 carries the one live correctness defect found along the way.
Generated by Claude Code
- added 4 commits that reference this issue
on Aug 27, 2026 - added a commit that references this issue
on Oct 5, 2026
The targeted iOS XCUITest and the fixture-backed iOS smoke both fail intermittently on branches that touch no Swift and no iOS production code, blocking the most expensive PR gate.
Two symptoms — and, it turned out, two unrelated causes:
RunnerTests.testBareTypeUsesTappedInputWhenSoftwareKeyboardIsHidden—** TEST EXECUTE FAILED **/ exit 65smoke:form-input→ stepfill full name→TEXT_INPUT_COMMIT_NOT_OBSERVED32229709270ci/1824-coverage-fork-death(#1854 — test-setup, docs, vitest configs only)32176591415agent/fix-install-source-network-cause(#1863)32847092583(both attempts)claude/mcp-registry-security-el547d(#2023 — MCP/release only, 16 files)32655577032(attempt 1 only; attempt 2 green on the same SHA)claude/monorepo-modularity-analysis-t1kd3c(#1985 — one TS module renamed, three imports retargeted)32889322172claude/1874-text-entry-commit-budget(#2035, this issue's own fix branch — the run that produced the decisive trace)Root cause
smoke:form-input— resolved, and it was never a raceThe fixture asked
fillfor the one value it cannot verify.field-name's placeholder was"Ada Lovelace", and every checkout-form suite filled exactly"Ada Lovelace";field-emailhadthe same collision with
ada@example.com. Twelve fills across eight files.An empty text field renders its placeholder as its accessibility value, so
element.valueisbyte-identical whether the placeholder is showing or the committed text happens to equal it, and
placeholderValuedoes not disambiguate.awaitSynthesizedReplacementCommitOutcomethereforerefuses such a wait outright (
textMatchesPlaceholder) rather than reporting a commit it cannotprove — the documented contract, see the
TEXT_INPUT_COMMIT_NOT_OBSERVEDparagraph inwebsite/docs/docs/commands.mdand the three-state model in #1924.Caught from run
32889322172,where the trace is unambiguous:
Zero poll lines: the wait never entered its loop.
expectedLen=12is"Ada Lovelace". The 3.6s isthe AX resolution for the placeholder read that precedes the guard, not a deadline.
Why it looked intermittent. The synthesized-replacement route is gated on
xCTestChannelPenalized— it is taken only when the host is loaded. A deterministic refusal on aload-gated route is indistinguishable from a load flake from the outside. It is also what actually
explains the datapoint this issue found most puzzling:
rerun_failed_jobson32847092583reproducing identically. That is not persisted per-device state; it is the same guard firing on
the same input.
Fixed at the fixture (placeholders changed), not at the filled values — frozen
test/replay-compat/scripts/**corpora carry the same fills and must not be edited. The collisionhad also been making the read-back assertions vacuous:
assertJsonContains(name, 'Ada Lovelace')issatisfied by an empty field rendering the placeholder.
testBareType…— mitigated, not root-causedThe commit wait used a flat 3s deadline, which cannot tell a throttled input pipeline
(characters keep landing, slowly) from a wedged one (nothing lands). It condemned both at the
same instant, so a working
typereportedTEXT_INPUT_COMMIT_NOT_OBSERVED.The wait now grants time against progress — the observed value's expected-prefix growing, the same
length-only evidence
logCommitCadencealready emitted — with an absolute 10s ceiling. A pipelinemaking no progress still expires at exactly the 3s the flat deadline used, so nothing that failed
before starts passing merely by waiting longer.
This reduces exposure; it does not explain the throttle. The ~40s episode traced in the
2026-08-21 diagnosis comment below would still exhaust the ceiling and report honestly.
Superseded hypotheses
Both mechanisms originally filed here are disproven. Recording them so they are not re-derived:
1. "Nothing pins the simulator keyboard state." Measured on a dedicated simulator, headless
simctl boot, iOS 26.2, then deleted:ConnectHardwareKeyboardtrue(hardware keyboard "connected")falsecom.apple.iphonesimulatoris Simulator.app's preference domain; with no Simulator.app running,CoreSimulator does not consult it. Pinning it would have added a step, a guard test and a green
assertion proving only that
defaults writeran. The useful half of the finding is the reassuringone: a headless lane has no hardware keyboard to inherit, so the ambient state named in this
issue is not a variable in these lanes at all.
2. "Text-entry readiness depends on that ambient state." Neither failing signature reaches
waitForTextEntryReadiness:testBareTypeUsesTappedInputWhenSoftwareKeyboardIsHiddentypes through the tap witness, andrememberedTextEntryTarget()returns before readiness is ever consulted.smoke:form-input'sfillfails insiderunSynthesizedReplacementRoute, whichshouldUseSynthesizedFirstResponderReplacementgates onhasResolvedElement == false— thepenalized coordinate route builds that target directly in
executeTypeCommand, bypassingfocusTextInputForTextEntryentirely.The readiness fallback was hardened anyway (it now returns only on confirmed focus of the target,
re-arming otherwise), because the latent sensitivity described here was real — it just was not what
was firing.
What shipped
#2035 (
7db5ad73dd):SynthesizedCommitDeadline), stall budget unchanged at 3s, ceiling 10s;testBareTypeUsesTappedInputWhenSoftwareKeyboardIsHidden's keyboard-hidden precondition isXCTSkipIf, notXCTAssertFalse— an environment flip must not read as a product regression;What is still open
Why does the simulator input pipeline throttle? Unchanged by the above. What is different is
that the evidence to answer it is now on the shipped path rather than needing new instrumentation:
the
[DEBUG-1874]per-poll cadence discriminates three shapes, and after #2035 the sameexpected-prefix signal decides the deadline rather than merely being logged.
So the next red lane should be readable from
runner.login the uploadedios-artifactsbundlewithout adding anything.
1874-diagnose.yml(matrixisolated/pair×x86_64/arm64) remainsdispatchable for forcing the question rather than waiting for it.
Related: #1924 (honest reporting, and the three-state placeholder model), #1941 (commit-wait
instrumentation), #1995 (fill's synthesized-replacement verification), #2032 (folded in here),
#2053 (a different Coverage-lane flake seen while working on this, not this issue).