Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion src/cli/parser/cli-help.ts
Original file line number Diff line number Diff line change
Expand Up @@ -350,7 +350,7 @@ Validation and evidence:
agent-device screenshot
agent-device press 124 817
agent-device snapshot -i
Startup/CPU/memory/frame first pass: perf metrics --json (bare perf and metrics are aliases). Focused frame/jank health: perf frames --json. Memory-only sample: perf memory sample --json returns compact JSON with bounded top offenders. Heap/memgraph artifact escalation: perf memory snapshot --out heap.artifact; use --kind android-hprof on Android or --kind memgraph on supported Apple simulator/macOS app sessions. Android native profiling: perf cpu profile start|stop|report --kind simpleperf --out <path>; Android native traces: perf trace start|stop --kind perfetto --out <path>. Artifact collectors return compact state/path/size metadata only; raw heap/profile/trace files stay on disk. Treat native perf output as the agent evidence: for example, a Perfetto stop can return state=stopped, outPath=/tmp/app.perfetto-trace, sizeBytes=5392410, and method=adb-shell-perfetto while the 5.3 MB raw trace stays in the artifact. This is better than raw dumps for agents because it is stable, bounded, and keeps large artifacts out of context. heapprofd is deferred until Perfetto plumbing is available. Replay divergence and resume: a failing replay/test step returns REPLAY_DIVERGENCE with a bounded report (screen digest, ranked selector suggestions, resume). Repair app state, then resume with replay --from <n> --plan-digest <sha256> (both from the report's resume field) to continue from the failed step without re-running earlier ones; resume never re-executes skipped steps, so app state is the caller's responsibility, and it is rejected with INVALID_ARGS when the plan digest is stale, --from is out of range, or the skipped range/target touches runtime control flow. The plan digest binds the script, its includes, the effective --platform/--target, and per-action runtime/identity. Native .ad interpolation is late-bound after planning, so changing only its values keeps the digest; Maestro environment substitution occurs during compatibility parsing and can change action inputs, includes, or control expansion, so it can change the digest. --from is replay-only; test rejects it. --update/-u no longer rewrites the script (ADR 0012) — it is a no-op kept for compatibility; every divergence already carries the same ranked suggestions. Agent-supervised repair (heal-by-doing): arm replay <file>.ad --save-script[=<out>] before step 1 (armed once; --from continuations do not need it again). Every divergence carries a repairHint: record-and-heal means press the correct control via a blessed @ref from the divergence's screen.refs, recorded (no --no-record), then continue with replay --from <n+1> --plan-digest <sha256>; state-repair means the script is correct but app state is not, so fix state with --no-record actions, then replay --from <n> --plan-digest <sha256> to re-run the unchanged step; caution means something already matches the recorded selector, so a blind re-press may repeat the mistake; manual means no safe automated repair could be proven. While armed, read-only inspection YOU run to locate the repair target (snapshot -i, get attrs, find, is) is excluded from the healed script by default — no --no-record needed; the script's own authored get/is/find steps are unaffected and stay in the heal. If the step you are repairing is itself a read, pass --record on that one command so it lands in the heal (--record is accepted only on snapshot/get/is and a read-only find; it is mutually exclusive with --no-record). Ending the repair with close --save-script[=<out>] writes only the steps recorded since the arming replay as <out>, defaulting to the <file> sibling <stem>.healed.ad — review its diff before promoting it over the original.
Startup/CPU/memory/frame first pass: perf metrics --json (bare perf and metrics are aliases). Focused frame/jank health: perf frames --json. Memory-only sample: perf memory sample --json returns compact JSON with bounded top offenders. Heap/memgraph artifact escalation: perf memory snapshot --out heap.artifact; use --kind android-hprof on Android or --kind memgraph on supported Apple simulator/macOS app sessions. Android native profiling: perf cpu profile start|stop|report --kind simpleperf --out <path>; Android native traces: perf trace start|stop --kind perfetto --out <path>. Artifact collectors return compact state/path/size metadata only; raw heap/profile/trace files stay on disk. Treat native perf output as the agent evidence: for example, a Perfetto stop can return state=stopped, outPath=/tmp/app.perfetto-trace, sizeBytes=5392410, and method=adb-shell-perfetto while the 5.3 MB raw trace stays in the artifact. This is better than raw dumps for agents because it is stable, bounded, and keeps large artifacts out of context. heapprofd is deferred until Perfetto plumbing is available. Replay divergence and resume: a failing replay/test step returns REPLAY_DIVERGENCE with a bounded report (screen digest, ranked selector suggestions, resume). Repair app state, then resume with replay --from <n> --plan-digest <sha256> (both from the report's resume field) to continue from the failed step without re-running earlier ones; resume never re-executes skipped steps, so app state is the caller's responsibility, and it is rejected with INVALID_ARGS when the plan digest is stale, --from is out of range, or the skipped range/target touches runtime control flow. The plan digest binds the script, its includes, the effective --platform/--target, and per-action runtime/identity. Native .ad interpolation is late-bound after planning, so changing only its values keeps the digest; Maestro environment substitution occurs during compatibility parsing and can change action inputs, includes, or control expansion, so it can change the digest. --from is replay-only; test rejects it. --update/-u no longer rewrites the script (ADR 0012) — it is a no-op kept for compatibility; every divergence already carries the same ranked suggestions. Agent-supervised repair (heal-by-doing): arm replay <file>.ad --save-script[=<out>] before step 1 (armed once; --from continuations do not need it again). Every divergence carries a repairHint: record-and-heal means press the correct control via a blessed @ref from the divergence's screen.refs, recorded (no --no-record), then continue with replay --from <n+1> --plan-digest <sha256>; state-repair means the script is correct but app state is not, so fix state with --no-record actions, then replay --from <n> --plan-digest <sha256> to re-run the unchanged step; caution means something already matches the recorded selector, so a blind re-press may repeat the mistake; manual means no safe automated repair could be proven. (If close --save-script is run before a required resume, the repair is aborted and no script is written). While armed, read-only inspection YOU run to locate the repair target (snapshot -i, get attrs, find, is) is excluded from the healed script by default — no --no-record needed; the script's own authored get/is/find steps are unaffected and stay in the heal. If the step you are repairing is itself a read, pass --record on that one command so it lands in the heal (--record is accepted only on snapshot/get/is and a read-only find; it is mutually exclusive with --no-record). Ending the repair with close --save-script[=<out>] writes only the steps recorded since the arming replay as <out>, defaulting to the <file> sibling <stem>.healed.ad — review its diff before promoting it over the original.
Recording: record start/stop. The default scope is app and expects an active session created by open <app>; this keeps app proof videos tied to the intended app session. Use record start --scope device/system to explicitly request whole-screen capture where the selected backend supports it, such as recordings that intentionally span multiple apps, home screen, settings, or app transitions. Use --max-size to cap the longest edge and --quality medium|high to choose output quality across Android and Apple targets. By default, stop burns touch overlays into the video; use record start --hide-touches for the fastest raw recording. Android record start publishes a durable device manifest. Android adb screenrecord has a 180s platform limit, so longer Android recordings are returned as multiple MP4 chunks while the daemon stays alive; after daemon restart, record stop recovers only manifest-owned chunks and warns when gesture overlays are unavailable. For gesture-heavy iOS simulator proof videos, prefer --hide-touches because overlay timing depends on a stable runner session while gestures are executing. Tracing: trace start ./trace.log, trace stop ./trace.log. Paths are positional.
Stable known flow: batch ./steps.json, not workflow batch.
Inline batch JSON example:
Expand Down
28 changes: 24 additions & 4 deletions src/daemon/handlers/session-close.ts
Original file line number Diff line number Diff line change
Expand Up @@ -89,6 +89,7 @@ function shouldStopAppleRunnerBeforeTargetedClose(session: SessionState): boolea
type RepairCloseOutcome =
| { kind: 'not-armed' }
| { kind: 'committed'; path?: string }
| { kind: 'aborted' }
| { kind: 'failed'; error: AppError };

function commitRepairBeforeClose(
Expand Down Expand Up @@ -117,6 +118,9 @@ function commitRepairBeforeClose(
session.actions.length = actionsBeforeClose;
return { kind: 'failed', error: result.error };
}
if (!session.saveScriptComplete) {
return { kind: 'aborted' };
}
return { kind: 'committed' };
}

Expand Down Expand Up @@ -281,7 +285,7 @@ function repairPlatformCloseIdentity(req: DaemonRequest): string {
}

type RepairClosePreparation =
| { repairArmed: boolean; healedScriptPath?: string }
| { repairArmed: boolean; healedScriptPath?: string; aborted?: boolean }
| { response: DaemonResponse };

async function prepareRepairClose(params: {
Expand Down Expand Up @@ -317,7 +321,10 @@ async function prepareRepairClose(params: {
session.repairPlatformCloseIdentity = undefined;
return {
repairArmed,
...(repairCommit.kind === 'committed' ? { healedScriptPath: repairCommit.path } : {}),
...(repairCommit.kind === 'committed' && repairCommit.path
? { healedScriptPath: repairCommit.path }
: {}),
...(repairCommit.kind === 'aborted' ? { aborted: true } : {}),
};
}

Expand Down Expand Up @@ -508,6 +515,19 @@ export async function handleCloseCommand(params: {
// artifact path so the agent learns the repair published (and where) without
// an extra round-trip.
const savedScript = repair.healedScriptPath ? { savedScript: repair.healedScriptPath } : {};
const text = `Closed: ${session.name}`;
if (repair.aborted && req.flags?.saveScript) {
return {
ok: false,
error: {
code: 'COMMAND_FAILED',
message: 'The repair was aborted and no script was written.',
hint: 'Recovery: replay --from <n> --plan-digest <digest> before closing.',
details: { session: session.name },
},
};
}

if (shutdownResult) {
return {
ok: true,
Expand All @@ -518,15 +538,15 @@ export async function handleCloseCommand(params: {
...savedScript,
...(leaseRelease.providerData ? { provider: leaseRelease.providerData } : {}),
},
`Closed: ${session.name}`,
text,
),
};
}
return {
ok: true,
data: {
session: session.name,
...successText(`Closed: ${session.name}`),
...successText(text),
...savedScript,
...(leaseRelease.providerData ? { provider: leaseRelease.providerData } : {}),
},
Expand Down
127 changes: 127 additions & 0 deletions test/integration/provider-scenarios/replay-repair-abort-close.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,127 @@
import assert from 'node:assert/strict';
import fs from 'node:fs';
import os from 'node:os';
import path from 'node:path';
import { test } from 'vitest';
import { ANDROID_SNAPSHOT_HELPER_FIXTURE_ARTIFACT } from '../../../src/__tests__/test-utils/index.ts';
import type { AndroidAdbProvider } from '../../../src/platforms/android/adb-executor.ts';
import { assertRpcError, assertRpcOk } from './assertions.ts';
import { androidSettingsXml, androidSnapshotHelperOutput } from './android-world.ts';
import { PROVIDER_SCENARIO_ANDROID } from './fixtures.ts';
import { createProviderScenarioHarness } from './harness.ts';

const SEARCH_SELECTOR = 'id=com.android.settings:id/search';

test('Provider-backed integration: repair close paths (abort, plain close, committed)', async () => {
const adbProvider: AndroidAdbProvider = {
snapshotHelperArtifact: ANDROID_SNAPSHOT_HELPER_FIXTURE_ARTIFACT,
exec: async (args) => androidRepairAdbResult(args),
};

const tempRoot = fs.mkdtempSync(path.join(os.tmpdir(), 'agent-device-repair-close-'));
const repairPath = path.join(tempRoot, 'repair.ad');
fs.writeFileSync(
repairPath,
['snapshot -i', 'press label="NoSuchControl"', `is visible ${SEARCH_SELECTOR}`, ''].join('\n'),
);

// Helper to get divergence report
async function armRepair(client: any, selection: any) {
const open = await client.apps.open({ app: 'settings', ...selection });
assert.equal(open.device?.id, PROVIDER_SCENARIO_ANDROID.id);

const divergenceError = await client.replay
.run({ path: repairPath, saveScript: true, ...selection })
.then(
() => null,
(error: unknown) => error as { code?: string; details?: Record<string, unknown> },
);
assert.ok(divergenceError);
return divergenceError.details?.divergence as {
resume: { allowed: boolean; from: number; planDigest: string; repairSessionHeld?: boolean };
};
}

try {
// 1. Aborted repair with close --save-script fails and leaves no artifact
let daemon = await createProviderScenarioHarness({
androidAdbProvider: () => adbProvider,
deviceInventoryProvider: async () => [PROVIDER_SCENARIO_ANDROID],
});
let client = daemon.client();
const selection = { platform: 'android' as const, serial: PROVIDER_SCENARIO_ANDROID.id };

await armRepair(client, selection);

const abortedClose = await daemon.callCommand('close', [], { saveScript: true });
assertRpcError(abortedClose, 'COMMAND_FAILED', /repair was aborted/);
assert.equal(fs.existsSync(path.join(tempRoot, 'repair.healed.ad')), false);
assert.equal(daemon.session(), undefined);
await daemon.close();

// 2. Aborted repair with plain close succeeds and leaves no artifact
daemon = await createProviderScenarioHarness({
androidAdbProvider: () => adbProvider,
deviceInventoryProvider: async () => [PROVIDER_SCENARIO_ANDROID],
});
client = daemon.client();

await armRepair(client, selection);

const plainClose = await daemon.callCommand('close', []);
assertRpcOk(plainClose);
assert.equal(fs.existsSync(path.join(tempRoot, 'repair.healed.ad')), false);
assert.equal(daemon.session(), undefined);
await daemon.close();

// 3. Committed repair with close --save-script succeeds and writes artifact
daemon = await createProviderScenarioHarness({
androidAdbProvider: () => adbProvider,
deviceInventoryProvider: async () => [PROVIDER_SCENARIO_ANDROID],
});
client = daemon.client();

const divergenceReport = await armRepair(client, selection);

const resumed = await client.replay.run({
path: repairPath,
resumeFrom: divergenceReport.resume.from + 1,
resumePlanDigest: divergenceReport.resume.planDigest,
...selection,
});
assert.equal(resumed.replayed, 1);

const committedClose = await daemon.callCommand('close', [], { saveScript: true });
assertRpcOk(committedClose);
assert.equal(fs.existsSync(path.join(tempRoot, 'repair.healed.ad')), true);
assert.equal(daemon.session(), undefined);
await daemon.close();
} finally {
fs.rmSync(tempRoot, { recursive: true, force: true });
}
}, 30_000);

function androidRepairAdbResult(args: string[]): {
stdout: string;
stderr: string;
exitCode: number;
} {
if (args.join(' ') === 'shell getprop sys.boot_completed') {
return { stdout: '1\n', stderr: '', exitCode: 0 };
}
if (args.join(' ') === 'shell dumpsys window windows') {
return {
stdout: 'mCurrentFocus=Window{42 u0 com.android.settings/.Settings}\n',
stderr: '',
exitCode: 0,
};
}
if (args.join(' ').startsWith('shell am instrument ')) {
return {
stdout: androidSnapshotHelperOutput(androidSettingsXml('Display')),
stderr: '',
exitCode: 0,
};
}
return { stdout: '', stderr: '', exitCode: 0 };
}
Loading