Skip to content

fix(agent): preserve recorded cursor visibility - #1075

Merged
EtienneLescot merged 2 commits into
getopenscreen:mainfrom
satwiksharma01:fix/agent-cursor-visibility
Oct 10, 2026
Merged

EtienneLescot merged 2 commits into
getopenscreen:mainfrom
satwiksharma01:fix/agent-cursor-visibility

Conversation

@satwiksharma01

@satwiksharma01 satwiksharma01 commented Oct 9, 2026 •

Copy link
Copy Markdown
Contributor

Summary

getCursorTrack loses the recorded cursor's visibility flag, and compresses a stationary hide/show interval down to its endpoints. For example, a pointer that is visible at 0s, hidden at 0.5s and visible again at 1s is returned as two ordinary visible points. Hidden clicks likewise arrive without a hidden marker, so the AI editor cannot distinguish them from visible activity.

Preserve visibility changes as mandatory compression anchors and emit visible:false on hidden points. Missing visibility in older sidecars continues to mean visible. The tool's returned time-base description and architecture documentation explain this compact contract.

Related issue

Related to #725 for cursor visibility context. This is a separate agent telemetry fix; it does not claim to replace the recording/rendering correction already merged in #731.

Type of change

  • Bug fix

Release impact

  • Patch

Desktop impact

  • Not platform-specific

Screenshots / video

No UI changes; regression tests cover the actual tool JSON payload.

Testing

  • Added regressions for rapid stationary hide/show transitions below the sampling rate, hidden clicks, legacy sidecars and getCursorTrack JSON propagation. Three failed against the original code; all 140 targeted cursor/tool tests pass with the fix.
  • Both app and test TypeScript checks, full lint, docs and renderer/Electron build pass. Lint reports 26 existing warnings.
  • Full suite: 4,308 tests passed, 5 skipped and one unrelated clipboard-button/session-load wait timed out. That file passed all three tests on an isolated rerun; no unrelated source or test changes were made.
  • Independent review found no actionable defect in visibility propagation or compression.

Validated with pinned Node 22.22.1 and npm 10.9.4. AI assistance was used. This changes the agent's observation payload, not capture or rendering behavior.

Summary by CodeRabbit

  • New Features
    • Cursor tracks preserve whether the pointer is visible or hidden, including when stationary or downsampled. Hidden clicks remain marked as hidden.
    • Older recordings without visibility information continue to display the cursor as visible by default.

@coderabbitai

coderabbitai Bot commented Oct 9, 2026 •

Copy link
Copy Markdown
Contributor

Review in Change Stack →

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration
  • Configuration used: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: a4f8ef80-b1c2-47cf-a666-afee65596537

📥 Commits

Reviewing files that changed from the base of the PR and between 9ed89e5 and a3918d8.


📒 Files selected for processing (1)
  • electron/ai-edition/deep-agent/service.ts

Included review availability: This review used your included allowance. Your plan provides up to 8 included reviews per hour; 6 remain after this review.



📝 Walkthrough

Walkthrough

Cursor track samples now support optional visibility. Missing visibility means visible. Track generation retains visibility changes during downsampling and marks hidden output points with visible: false. Tests and tool descriptions cover these semantics.

Changes

Cursor track visibility

Layer / File(s) Summary
Visibility in sampled cursor tracks
src/lib/ai-edition/timeline/cursor-track.ts, src/lib/ai-edition/timeline/cursor-track.test.ts, electron/ai-edition/agent-tools.test.ts, technical-documentation/architecture/ai-agent.md, electron/ai-edition/deep-agent/service.ts
Cursor samples support optional visibility. Visibility changes are retained during downsampling, and hidden output points include visible: false. Tests cover hidden clicks, visibility transitions, and legacy samples. Tool descriptions document hidden-point and default-visible semantics.

Priority: ⬇️ Low

Estimated code review effort: 2 (Simple) | ~12 minutes

Change: Bug fix


Merge Risk: ⚪ Minimal · up to a3918

Cursor visibility changes are retained in track output, and legacy samples remain visible by default. No actionable merge-blocking risk was identified.

Pre-merge checks | Passed 4 | Failed 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 1 functions across 4 files. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check Passed The title clearly and concisely describes the primary change: preserving recorded cursor visibility in the agent output.
Description check Passed The description includes all required sections and provides clear change details, issue context, impact, testing results, and platform information.
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.

  • Fix all pre-merge checks with AI
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create a new PR

  • Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@EtienneLescot EtienneLescot left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed: visibility changes kept as compression anchors, backward compatible with older sidecars. Thanks @satwiksharma01!

@EtienneLescot
EtienneLescot merged commit 91b0fe6 into getopenscreen:main Oct 10, 2026
19 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants