Skip to content

Recording touch overlay exports at ~220x480 or all-black whenever touches are drawn #2707

Description

@thymikee

Purpose

record with the default touch overlay produces an unusable export as soon as the recording contains an interaction: the track collapses to roughly 220x480 regardless of the device's native capture size, and on a long recording it comes out entirely black. Touch indicators are therefore unavailable for exactly the recordings that need them, and the caller gets exit code 0 either way.

This is the recording transport, not the display-selection path. The raw capture honors --display correctly since #2703.

Observed

iPhone 17 simulator 0E0FCBDC-88E2-4CB6-A1DA-E148F77743F4 (iOS 27.0, non-rot90 panel), Xcode 27.1 beta, agent-device at 057dfb7c1. Mean luma measured over sampled frames; content changes across frames are marked where they occur.

Overlay Interactions during capture Exported size Result
--hide-touches none 1206x2622 correct
default none 1206x2622 correct
default two tab taps 220x480 bright, content changes present
--hide-touches two tab taps 1206x2622 correct, content changes present
default taps, scroll, 97s 480x220 meanLuma=0.00 every frame

Two things follow from the matrix. The trigger is the overlay drawing touch events, not the interactions themselves — --hide-touches stays full size while recording the same taps. And it is not orientation-dependent: the failing device here has a normal portrait panel.

The overlay does draw something, since the small export tracks the screen. Only the geometry and, in the long case, the frame content are lost.

Required behavior

  • A recording with touches exports at the same track size as the same recording without touches.
  • A recording never exports all-black. If the overlay compositor cannot produce frames, fail the command or drop the overlay and say so in the result, rather than returning a black file with exit 0.
  • The choice to drop the overlay must be reported on the response, not inferred from pixel dimensions by whoever notices later.

Completion conditions

  • Table above re-measured on a non-rot90 target and on the iPhone Duo inner panel, with the overlay rows matching the --hide-touches rows for size and non-blackness.
  • A regression test asserts exported track size equals the raw path's for a scripted tap, since "file exists and is non-zero bytes" would have passed all four broken rows above.
  • docs/adr/0025-foldable-apple-panels.md and the --hide-touches note in docs/agents/device-verification.md currently state that the defect is specific to rot90 inner-panel tracks and that the same path works on non-rotated panels. That claim is wrong and was written before this matrix existed; both need correcting as part of this work.

Dependencies

None. Reproducible on main.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions