Skip to content

feat(replay): recorded landmark identity for wait, is coverage — read-only step identity (#1349)#1381

Open
thymikee wants to merge 5 commits into
mainfrom
claude/agent-device-1349-design-fd6ed3
Open

feat(replay): recorded landmark identity for wait, is coverage — read-only step identity (#1349)#1381
thymikee wants to merge 5 commits into
mainfrom
claude/agent-device-1349-design-fd6ed3

Conversation

@thymikee

Copy link
Copy Markdown
Member

Closes #1349.

What

Extends ADR 0012's target-v1 identity verification to the eligible read-only steps without changing their execution semantics, and strengthens ADR 0016's destination guard from selector existence to recorded landmark identity. Four commits, sequenced dependency-first:

  1. refactor(replay): shared tree helpers — moves the pure evidence primitives (buildIndexMap/buildAncestryChain/filterIdentitySet) from the daemon into src/replay/target-evidence-tree.ts so the commands-layer wait loop can consume them (commands → replay was already a legal DAG edge); deletes press-retarget.ts's private buildIndexMap duplicate.
  2. feat(replay): wait landmark verification + is coverage — the core.
  3. feat(publication): guard requires verified evidence + the reshuffled-screen false-pass regression.
  4. refactor(replay): dedupe/complexity cleanup shaken out by the fallow gates, plus ADR/help docs.

Design decisions a reviewer should check

  • Same annotation, new phase trait. No new .ad format: what's new is targetIdentityVerification: 'pre-dispatch' | 'post-resolution' on CommandDescriptor, pinning the evidence-carrying set ({click, press, longpress, fill, get, is} → pre-dispatch; wait → post-resolution) with a parity test. An annotated wait therefore structurally cannot enter the generic pre-action path — the issue's key acceptance criterion. Only ADR 0012 path 1 (recorded-unverifiable, which consults no screen) still refuses a wait before polling.
  • Wait verifies in-loop, keep-polling, identity tier only. Success strengthens from "selector matches" to "a selector match carries the recorded identity (local identity + leaf-anchored ancestry prefix)". A same-selector impostor never aborts the wait mid-poll (transient look-alikes during screen transitions are what waits exist to wait through); at the deadline, rejected candidates become a fail-closed identity-mismatch REPLAY_DIVERGENCE (matchCount from the last matching poll), while a never-matched selector stays the plain timeout (action-failure → state-repair routing, unchanged). Positional signals (sibling/viewportOrder) are deliberately not compared — a guard proves the landmark exists, not that it kept its list position, and comparing position would false-fail legitimately reshuffled-but-correct screens.
  • Landmark evidence mode at record time. computeTargetEvidence(..., {mode:'landmark'}) self-checks identity-set membership (mirroring what replay verifies); an identity-empty match (no id, no label after Android replay: non-unique framework id android:id/title used as primary identity → systematic false divergences (FDR 1.0) #1269 demotion) records no annotation — fail-open to today's semantics, and publication then refuses it as a guard with a hint naming a labeled landmark.
  • is (except exists) = the exact get pattern: evidence from its resolved node, generic pre-dispatch verification, assertExpectedResolvedTarget guard threaded through dispatch. Direct-iOS wait/is fast paths are gated during recording and guarded replays (same rule get has had since feat: parse and preserve .ad target-v1 evidence (ADR 0012 migration step 3) #1196; authoring-only cost).
  • Explicitly deferred, with tests proving no annotation/no check: is exists, all read-only find variants (fuzzy-locator resolution has no selector-chain identity token for the classifier), wait text/stable/duration/@ref (wait @ref stays rejected by publication, never converted).
  • ADR 0016 guard: a qualifying destination guard is now a selector wait whose annotation is verification: "verified".

Wire/plumbing

The recorded landmark travels the daemon-only internal.replayLandmarkGuard channel (unspoofable, same as replayTargetGuard); the loop's deadline refusal is reason: wait_landmark_identity_mismatch with a compact {matchCount, observed, observedAncestry} payload, converted by the step loop into the standard bounded divergence (shared shaping with the guard-mismatch path via buildPostDispatchIdentityMismatchResponse). node/preActionNodes are stripped before both session.actions and the wire on every path.

Validation

  • Unit: wait loop semantics (impostor-then-landmark success proving keep-polling; deadline mismatch payload; plain timeout preserved; unannotated unchanged), landmark evidence mode (twins verified, identity-empty omitted, broken-walk unverifiable), trait parity, is record/verify/guard, read-only find + is exists deferral proofs.
  • Daemon: annotated wait dispatches with the guard and no pre-action capture; recorded-unverifiable wait refuses pre-poll with matchCount omitted; loop refusal → identity-mismatch divergence with repairHint: caution; duration-wait annotation inert.
  • Provider (end-to-end): the ADR 0016 reshuffled-screen false-pass regression — record → session save-script (annotation lands in the .ad) → replay against a same-label/different-ancestry tree → REPLAY_DIVERGENCE identity-mismatch with matchCount ≥ 1 (the proof the selector alone would have false-passed).
  • Gates: full vitest suite ×3 across the arc, tsc, oxlint, oxfmt, layering, fallow (complexity/duplication/production-exports), plus an independent adversarial review pass over the whole diff (no findings; two informational observations adjudicated in the ADR text/code comments).

Perf / size

No measurable regression: live unannotated wait polls differ by one small allocation; the annotated-replay identity check is O(tree) per poll, dwarfed by the poll's own snapshot capture. No new startup-parse weight (the commands bundle already reached the identity modules; the new shared module is ~80 dependency-free lines). The one deliberate cost: recording sessions use the snapshot path for wait/is instead of the direct-iOS runner query, which evidence capture requires.

Docs: ADR 0012 decision-3 amendment (#1349) + migration row 11, ADR 0016 amendment, help workflow/session save-script text.

thymikee added 4 commits July 23, 2026 11:17
…c/replay

Move buildIndexMap/buildAncestryChain/filterIdentitySet out of the daemon's
session-target-evidence into the shared replay zone so the commands runtime
(wait's polling loop, #1349) can consume them without importing the daemon;
press-retarget drops its private buildIndexMap duplicate.
…attern coverage for is (#1349)

- New CommandDescriptor trait targetIdentityVerification pins the
  evidence-carrying command set and routes wait to a post-resolution
  phase so an annotated wait never enters the generic pre-dispatch
  verification (an absent landmark is its expected starting condition).
- wait <selector> records landmark-mode target-v1 evidence (existence
  self-check; identity-empty matches record no annotation) and, on
  replay, keeps polling until a selector match carries the recorded
  identity; a deadline with only impostor matches fails closed as an
  identity-mismatch REPLAY_DIVERGENCE, a recorded-unverifiable
  annotation refuses before polling, and a plain timeout stays an
  action-failure divergence.
- is (except exists) joins the get pattern: evidence at record time,
  generic pre-dispatch verification, and the post-resolution guard
  threaded through dispatch; direct-iOS fast paths for wait/is are
  gated during recording and guarded replays.
- Read-only find stays intentionally unannotated (fuzzy-locator
  resolution has no selector-chain identity token), proven by test.
…ark identity (#1349)

A qualifying ADR 0016 guard is now a selector wait whose target-v1
annotation is verified; identity-less or unverifiable guards are refused
with a recovery hint. Adds the reshuffled-screen false-pass regression:
record -> publish -> replay against a same-label/different-ancestry tree
diverges as identity-mismatch (matchCount >= 1 proving the selector
alone would have false-passed).
…m evidence-writer complexity

Shared buildPostDispatchIdentityMismatchResponse behind the guard and
wait-landmark conversions; extracted payload-ceiling helpers from
computeTargetEvidence; identity-refusal conversion split out of
resolveReplayStepResponse. Docs: ADR 0012 decision 3 amendment (#1349),
ADR 0016 guard strengthening, help workflow/save-script text.
@github-actions

github-actions Bot commented Jul 23, 2026

Copy link
Copy Markdown

Size Report

Metric Base Current Diff
JS raw 1.8 MB 1.8 MB +2.2 kB
JS gzip 578.3 kB 579.0 kB +723 B
npm tarball 692.8 kB 693.6 kB +763 B
npm unpacked 2.4 MB 2.4 MB +2.2 kB

Startup median (7 runs, lower is better):

Scenario Base Current Diff
CLI --version 26.7 ms 26.1 ms -0.6 ms
CLI --help 56.4 ms 56.4 ms +0.0 ms

Top changed chunks:

Chunk Raw diff Gzip diff
dist/src/session.js +1.3 kB +439 B
dist/src/context.js -1.0 kB -319 B
dist/src/runtime.js +769 B +318 B
dist/src/selector-runtime.js +463 B +114 B
dist/src/cli-help.js +316 B +110 B

…xplicit, trim ADR-restating docs

The landmark-mode self-check was provably a tautology (the winner is a
member of its own identity set whenever the parent walk is intact), so a
membership scan defended only by a comment is replaced with the explicit
decision: broken walk fails closed, landmark is verified by construction,
action mode keeps decision 3's step-5 self-check. Doc comments that
re-argued the ADR amendment now state behavior and point to it.
@thymikee

thymikee commented Jul 23, 2026

Copy link
Copy Markdown
Member Author

Reviewed exact head 323e3dac; the replay implementation traces cleanly, but the PR is not ready:

  1. P2 — remove unrelated generated Android binaries. The first replay refactor commit also adds android/multitouch-helper/build/classes/** and android/multitouch-helper/build/dex/classes.dex. These derived artifacts are unrelated to Design read-only identity verification without breaking wait polling #1349, and analogous helper build directories are ignored. Remove them from this PR and add the matching ignore rule if this output is routinely generated.

  2. Readiness blocker — required live backend evidence is missing. The mocked/provider reshuffled-tree regression is strong and non-vacuous, but ADR 0016 explicitly requires live iOS and Android runs proving the saved artifact, identity-guarded destination wait, post-save handoff, and subsequent live-session command on real backends. Please attach those runs.

Descriptor phase → replay verifier → daemon-only landmark guard → snapshot-polling wait → bounded divergence conversion otherwise looks correct: impostors keep polling, ordinary timeout remains intact, and guarded recording/replay disables direct iOS fast paths as intended. Exact-head CI is green and merge state is clean. No ready label.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Design read-only identity verification without breaking wait polling

1 participant