Skip to content

feat(desktop): capture any window with a global shortcut - #8103

Open
Bil0000 wants to merge 12 commits into
pingdotgg:mainfrom
Bil0000:feat/window-capture
Open

feat(desktop): capture any window with a global shortcut#8103
Bil0000 wants to merge 12 commits into
pingdotgg:mainfrom
Bil0000:feat/window-capture

Conversation

@Bil0000

@Bil0000 Bil0000 commented Aug 24, 2026

Copy link
Copy Markdown
Contributor

What Changed

  • Adds an opt-in global window capture shortcut for macOS, Windows, Linux X11, and Linux Wayland.
  • Saves captures to a durable desktop queue, then attaches them to the current draft with app name, window title, and app icon metadata.
  • Adds capture sound, flash, attachment animation, onboarding, command palette access, and a dedicated settings page.
  • Preserves capture metadata through inline and pre-uploaded attachment paths.

Why

Users can add visual context from another app without leaving their current T3 Code workflow or managing screenshot files by hand. The implementation keeps native behavior in one desktop service and reuses the existing settings, IPC, draft, attachment, and timeline paths.

The diff is larger than a typical contribution because the complete cross-platform workflow includes the native boundary, durable handoff, settings, feedback, and every user entry point. It remains scoped to window capture.

UI Changes

This is a new surface, so there is no prior equivalent UI.

First-run onboarding

Window capture onboarding dialog

Window Capture settings

Window Capture settings page

Setup flow

t3-window-capture-settings.mp4

Verification

  • 227 focused tests passed across contracts, desktop capture, settings search, draft persistence, attachment upload, and timeline rendering.
  • Focused lint passed.
  • Contracts, web, desktop, and server type checks passed.
  • Production web, server, and desktop builds passed.
  • Electron smoke test passed with isolated state under Xvfb.
  • Live Linux Electron check confirmed direct mode and successful global shortcut registration and unregister.

Checklist

  • This PR is focused on one user workflow
  • I explained what changed and why
  • I included clear screenshots for the new UI
  • I included a video for the setup interaction

Generated by GPT-5.6-sol in T3 Code using the Codex harness.

Note

Add global window capture shortcut to desktop app

  • Adds DesktopWindowCapture service that registers a global shortcut, hides the app window, captures the active window using platform-specific methods (macOS screencapture CLI, Linux Wayland portal or X11, Windows get-windows), and saves PNG and JSON metadata to disk.
  • Exposes the service to the renderer via IPC handlers (captureWindow, listPendingWindowCaptures, readWindowCapture, acknowledgeWindowCapture).
  • Introduces WindowCaptureCoordinator to drain pending captures, compress them, attach them to the current or new draft thread, and display them in the chat composer with source metadata (app icon, name, window title).
  • Adds a /settings/window-capture route to toggle the feature, record shortcuts, configure feedback (sound, flash, animations), and trigger immediate captures, plus an onboarding dialog for first-time users.
  • Risk: New ClientSettings fields (windowCaptureEnabled, etc.) default to disabled; ChatImageAttachment and UploadChatImageAttachment schemas gain an optional source field that must be preserved by the orchestrator and normalizer.

Macroscope summarized 68fb035.

@coderabbitai

coderabbitai Bot commented Aug 24, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 12f92718-0adc-4a7c-93d4-808439069ba3

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions github-actions Bot added vouch:trusted PR author is trusted by repo permissions or the VOUCHED list. size:XXL 1,000+ changed lines (additions + deletions). labels Aug 24, 2026

@macroscopeapp macroscopeapp Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed the new Effect service (apps/desktop/src/windowCapture/DesktopWindowCapture.ts), its IPC methods, layer wiring, and contract additions. Module shape (namespace imports, Context.Service with inline interface, exported make/layer, dependencies acquired via yield*) follows the conventions; the Effect.runPromiseWith bridge for the Electron globalShortcut callback matches the existing desktop native-callback pattern. Two error-modeling issues below.

Posted via Macroscope — Effect Service Conventions

Comment thread apps/desktop/src/windowCapture/DesktopWindowCapture.ts Outdated
Comment thread apps/desktop/src/windowCapture/DesktopWindowCapture.ts Outdated
Comment thread apps/web/src/components/desktop/WindowCaptureCoordinator.tsx Outdated
Comment thread apps/desktop/src/windowCapture/DesktopWindowCapture.ts Outdated
Comment thread apps/web/src/components/desktop/WindowCaptureCoordinator.tsx Outdated
Comment thread packages/contracts/src/ipc.ts Outdated
Comment thread apps/desktop/src/windowCapture/DesktopWindowCapture.ts Outdated
Comment thread apps/desktop/src/windowCapture/DesktopWindowCapture.ts Outdated

@macroscopeapp macroscopeapp Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

UI consistency review

Two findings, both in the new window-capture UI.

  1. apps/web/src/index.css introduces a global rule keyed on the class trio .animate-in.fade-in.zoom-in-95, which does not exist as a utility in this project. The repo's established pattern (see ComposerStashBadge/prompt-stash-count-enter) is keyframes in index.css plus an animate-[…] utility at the owner. As written, the unlayered global rule wins over Tailwind's layered animate-none, so the call-site motion-reduce:animate-none is inert (reduced motion only works because of the duplicated media query), duration-200 never affects the animation, and any future element combining those class names silently inherits this animation.

  2. WindowCaptureSettings records a shortcut on a Button without the data-keybinding-capture opt-out that app-level shortcut handlers check, so global capture-phase handlers (e.g. sidebar.toggle in AppSidebarLayout) consume the keypress before the recorder sees it.

Details inline.

Posted via Macroscope — UI Consistency

Comment thread apps/web/src/components/chat/ChatComposer.tsx Outdated
Comment thread apps/web/src/index.css Outdated
Comment thread apps/web/src/components/settings/WindowCaptureSettings.tsx Outdated
@macroscopeapp

macroscopeapp Bot commented Aug 24, 2026

Copy link
Copy Markdown
Contributor

Approvability

Verdict: Not approved

Macroscope's review found this PR not approvable — This PR introduces a substantial cross-platform window-capture workflow spanning native permissions, global shortcuts, filesystem persistence, IPC, contracts, and chat attachment state. Its broad runtime and user-facing impact warrants human review beyond the supplied minor UI consistency comment.

You can add or adjust custom eligibility rules. Learn more.

@macroscopeapp macroscopeapp Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Window capture UI review: one new finding on WindowCaptureOnboardingDialog (dialog body bypasses the DialogPanel contract). The three findings from the previous run (index.css .animate-in.fade-in.zoom-in-95 global override, the matching ChatComposer class string, and the missing data-keybinding-capture on the shortcut recorder) are unchanged in this head commit and are not re-posted.

Posted via Macroscope — UI Consistency

Comment thread apps/web/src/components/desktop/WindowCaptureOnboardingDialog.tsx Outdated

@macroscopeapp macroscopeapp Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

UI consistency review of the web-side window capture surfaces. Four findings, all on newly added lines; the three items flagged in the previous run (global .animate-in override, call-site animation ownership, and the missing data-keybinding-capture marker) are resolved in this revision.

Posted via Macroscope — UI Consistency

Comment thread apps/web/src/components/chat/WindowCaptureAttachmentDetails.tsx Outdated
Comment thread apps/web/src/components/desktop/WindowCaptureOnboardingDialog.tsx
Comment thread apps/web/src/components/desktop/WindowCaptureOnboardingDialog.tsx
Comment thread apps/web/src/components/settings/WindowCaptureSettings.tsx Outdated

@macroscopeapp macroscopeapp Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

UI consistency review of the window-capture web surfaces. Two findings on the new settings page; the composer/timeline attachment cards, the onboarding dialog, the keyframe ownership, and the command palette action all follow the existing primitives and conventions.

Posted via Macroscope — UI Consistency

Comment thread apps/web/src/components/settings/WindowCaptureSettings.tsx Outdated
Comment thread apps/web/src/components/settings/WindowCaptureSettings.tsx Outdated

@macroscopeapp macroscopeapp Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed the web UI surface of this feature (composer/timeline capture cards, capture settings page, onboarding dialog, command palette entry, keyframe ownership). The earlier findings on the window-capture-card-enter keyframe ownership, data-keybinding-capture, DialogPanel composition, capability gating of the onboarding dialog, and the disabled state of the sound/flash/animation switches are all resolved. Two remaining items below.

Still open from earlier reviews (not re-posted): the shortcut recorder still preventDefault()s Tab so focus cannot leave the button, and WindowCaptureAttachmentDetails still anchors its title to text-foreground while rendering on bg-background/70 inside the text-message-foreground bubble.

Posted via Macroscope — UI Consistency

Comment thread apps/web/src/components/desktop/WindowCaptureCoordinator.tsx Outdated
Comment thread apps/web/src/components/chat/ChatComposer.tsx Outdated

@macroscopeapp macroscopeapp Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One consistency finding on the new window-capture web code. Previously flagged items (recorder Tab handling, data-keybinding-capture, disabled-state parity for the feedback switches, capability gating on the onboarding dialog, DialogPanel composition, and the named window-capture-card-enter keyframes) all look resolved at this head.

Posted via Macroscope — UI Consistency

Comment thread apps/web/src/components/desktop/WindowCaptureCoordinator.tsx Outdated
Comment thread apps/web/src/components/desktop/WindowCaptureCoordinator.tsx Outdated
Comment thread apps/web/src/components/desktop/WindowCaptureCoordinator.tsx Outdated

@macroscopeapp macroscopeapp Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed the changed web UI surfaces (window-capture settings page, onboarding dialog, composer/timeline attachment cards, command palette entry, index.css keyframe). Earlier rounds' items (shared focus-event constant, DialogPanel composition, data-keybinding-capture, Tab handling, named keyframe animation, uniformly gated switches) all look resolved. Two remaining consistency points on the new settings page.

Posted via Macroscope — UI Consistency

Comment thread apps/web/src/components/settings/WindowCaptureSettings.tsx Outdated
Comment thread apps/web/src/components/settings/WindowCaptureSettings.tsx Outdated

@macroscopeapp macroscopeapp Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two findings on the settings surface introduced/changed here. The composer and timeline capture cards, the extracted SettingsUnavailableGroup, the keyframe ownership, and the onboarding dialog composition all look consistent with the existing primitives now.

Posted via Macroscope — UI Consistency

Comment thread apps/web/src/components/settings/settingsLayout.tsx
Comment thread apps/web/src/components/settings/WindowCaptureSettings.tsx

@macroscopeapp macroscopeapp Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One consistency finding on the new window-capture attachment card; everything else (settings layout group, dialog composition, keyframe ownership, palette action) matches the existing contracts.

Posted via Macroscope — UI Consistency

className="h-full w-full cursor-zoom-in"
className={
image.source?.kind === "window-capture"
? "h-full w-24 shrink-0 cursor-zoom-in overflow-hidden border-r border-border/70"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The capture card shell is composed inline on both surfaces and the thumbnail widths diverge: w-24 here versus w-28 in MessagesTimeline's user row, even though both cards are h-20 and share WindowCaptureAttachmentDetails. Since the image is object-cover, the same capture crops to a different aspect in the composer than in the sent message. The sibling wide attachment cards (ComposerPreviewAnnotationCards and UserMessagePreviewAnnotationCard) deliberately keep the same size-14 thumbnail across those two surfaces for this reason. Suggest settling on one width for both call sites (or extracting the card shell next to the shared details component).

Suggested change
? "h-full w-24 shrink-0 cursor-zoom-in overflow-hidden border-r border-border/70"
? "h-full w-28 shrink-0 cursor-zoom-in overflow-hidden border-r border-border/70"

Posted via Macroscope — UI Consistency

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:XXL 1,000+ changed lines (additions + deletions). vouch:trusted PR author is trusted by repo permissions or the VOUCHED list.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant