feat: select the input source through the host

dist/build.js

@@ -65,6 +65,7 @@ const patches = [
     "peekaboo-web-text-selection.patch",
     "peekaboo-clipboard-image-export.patch",
     "peekaboo-operation-record.patch",
+    "peekaboo-input-source.patch",
 ].map((name) => join(native, "patches", name));
 function spawn(command, env) {
     return spawnSync(command[0], command.slice(1), { encoding: "utf8", env: env ?? process.env });

docs/app.md

@@ -17,7 +17,7 @@ Anything running as the user can use those grants while the app serves, so grant
 
 Identical operations in a row fold into one line with a count. Every desktop change is listed, and so are screen captures and clipboard reads. Listings and snapshot bookkeeping are not. Only the operation and its target are kept, never typed text, clipboard contents or values. The list lives in memory and is gone when the app quits.
 
-**Pause All Input** stops agents from changing anything until you turn it off again. While paused, the app refuses every click, key, typing, scroll, drag, menu command, app or window change, dialog action and clipboard write before it is sent. The refusal tells the agent to ask you to resume. Agents can still observe: listings, the accessibility tree, screen captures and clipboard reads keep working. The menu bar icon becomes a pause sign while input is paused. Only this menu item changes it; no client can pause or resume. Pausing isn't saved, so the app starts resumed. `desktop_input_source` select runs in the client rather than through the app, so pausing does not stop it.
+**Pause All Input** stops agents from changing anything until you turn it off again. While paused, the app refuses every click, key, typing, scroll, drag, menu command, app or window change, dialog action and clipboard write before it is sent. The refusal tells the agent to ask you to resume. Agents can still observe: listings, the accessibility tree, screen captures and clipboard reads keep working. The menu bar icon becomes a pause sign while input is paused. Only this menu item changes it; no client can pause or resume. Pausing isn't saved, so the app starts resumed.
 
 ## Build
 

docs/tools.md

@@ -178,14 +178,16 @@ With `ocr: true`, `desktop_inspect` (in either mode) and `desktop_screen` also r
 
 `desktop_input_source` lists the enabled keyboard input sources that can be selected, as the input menu shows them. Each source has an `id`, `name`, `kind` (`keyboard_layout`, `input_mode` for one mode of an input method such as Hiragana, or `input_method`), its first 8 `languages` and whether it is selected. The result also gives `current` and the `keyboard_layout` that `desktop_key` maps characters through.
 
-With `select` set to a listed `id`, it selects that source with Text Input Sources (`TISSelectInputSource`), with no keys pressed, then reads the current source back for up to a second:
+With `select` set to a listed `id`, the host selects that source with Text Input Sources (`TISSelectInputSource`), with no keys pressed, and the client then reads the current source back for up to a second:
 - **`completed`** means the source reads back as selected, with `before` and `current`.
 - **Already selected:** the result is `completed` with `changed: false`, and nothing is changed.
 - **Not enabled:** the request is refused.
 - **Locked session:** the request is refused.
+- **Input paused** in the app: the request is refused, and nothing is changed.
+- **macOS refuses the selection:** if the source still reads back unchanged, the result is `refused`.
 - **No readback:** the result is `unknown`.
 
-It runs in the client process, which is in the user's GUI session, so it needs no Accessibility or host operation. The switch applies to every app until it is changed again. An input method's new mode may reach the frontmost app only after its next focus change.
+Listing runs in the client process and needs no permission. Selecting goes through the host like other input, so the app's pause and activity list cover it; it needs no Accessibility either. The switch applies to every app until it is changed again. An input method's new mode may reach the frontmost app only after its next focus change.
 
 `desktop_notifications` inspects Notification Center's windows on screen: its banners and alerts, and its panel when open (click the clock with `desktop_menubar_click` to open it). Each notification has an `id`, its app, title, subtitle, body and time where they can be read, its raw `texts` and `description`, and its `actions`: `open`, plus the custom actions Notification Center lists for it by exact name, such as Close, Show, Options or the posting app's own. `desktop_notification_action` takes that `id` and one of those actions. `open` is the notification's default click (on a stack, it expands the group); `close` runs Close; any other name runs that one listed custom action through Accessibility. Only Notification Center's elements list or run custom actions, at most 8 per element, each one line of at most 64 characters, and a name listed twice is never offered. A notification that has gone or changed is refused before input. Afterwards the notifications are read again: the result is `completed` only when the notification is gone, and `unknown` otherwise. System alerts such as permission prompts are dialogs, so `desktop_notifications` lists them under `system_alerts` for `desktop_dialog` and `desktop_dialog_answer` and never acts on them. This is built against the Notification Center structure of macOS 11 to 15.
 

native/patches/README.md

@@ -527,6 +527,12 @@ response, descriptor and semantics with the read; the operation inventory tests
 - **Advisory only.** The report changes no routing or response. Without an observer, the server does no extra work.
 - **No new operation.** The wire count is unchanged.
 
+`peekaboo-input-source.patch` adds a `selectInputSource` operation, so selecting a keyboard input source goes through the Bridge like other input. The embedding app's pause and activity list then cover it, where the client's own `TISSelectInputSource` call would bypass both.
+- **Request.** It carries one input source ID.
+- **Host.** The host selects the source with Text Input Sources on the main actor. Only enabled keyboard sources that can be selected are eligible. An unknown ID is `notFound`, and a nonzero status from TIS is `internalError`.
+- **Result contract.** No permission is required. It is a global, one-unit `dispatchedUnverified` native-framework delivery answered with `ok`. It is service-owned, so it runs in the native desktop lane, and it joins the default allowlists. The client reads the current source back itself.
+- **Tests.** The wire count is now 131, and the operation joins the unconditionally mutating set.
+
 The same patch adds an `operationAdmission` hook, for the embedding app's pause switch. The server asks it for every authorized request, after operation access is validated and before the target window is reported or anything is dispatched. It passes the operation and whether its result contract mutates the desktop. A returned message fails the request as a pre-dispatch `permissionDenied`, so it may not have completed and is recorded as `refused`. Without a hook, nothing changes.
 
 When updating Peekaboo, review the upstream fixes and these patches together. Update the revision

native/patches/peekaboo-input-source.patch

@@ -0,0 +1,225 @@
+diff --git a/Core/PeekabooCore/Sources/PeekabooBridge/InputSourceSelection.swift b/Core/PeekabooCore/Sources/PeekabooBridge/InputSourceSelection.swift
+new file mode 100644
+index 000000000..76ed9a986
+--- /dev/null
++++ b/Core/PeekabooCore/Sources/PeekabooBridge/InputSourceSelection.swift
+@@ -0,0 +1,27 @@
++import Carbon
++import Foundation
++
++enum InputSourceSelection {
++    /// Selects the enabled, selectable keyboard input source with this ID, as the input menu would.
++    /// Text Input Sources are changed on the main thread, where the run loop delivers their changes.
++    @MainActor
++    static func select(_ id: String) throws {
++        let filter = [
++            kTISPropertyInputSourceCategory as String: kTISCategoryKeyboardInputSource as String,
++            kTISPropertyInputSourceIsSelectCapable as String: true,
++            kTISPropertyInputSourceID as String: id,
++        ] as CFDictionary
++        let sources = TISCreateInputSourceList(filter, false)?.takeRetainedValue() as? [TISInputSource] ?? []
++        guard let source = sources.first else {
++            throw PeekabooBridgeErrorEnvelope(
++                code: .notFound,
++                message: "No enabled input source has the ID \(id)")
++        }
++        let status = TISSelectInputSource(source)
++        guard status == noErr else {
++            throw PeekabooBridgeErrorEnvelope(
++                code: .internalError,
++                message: "macOS refused to select the input source (status \(status))")
++        }
++    }
++}
+diff --git a/Core/PeekabooCore/Sources/PeekabooBridge/PeekabooBridgeClient+Interaction.swift b/Core/PeekabooCore/Sources/PeekabooBridge/PeekabooBridgeClient+Interaction.swift
+index 0ce0bf96c..21d7f6a1f 100644
+--- a/Core/PeekabooCore/Sources/PeekabooBridge/PeekabooBridgeClient+Interaction.swift
++++ b/Core/PeekabooCore/Sources/PeekabooBridge/PeekabooBridgeClient+Interaction.swift
+@@ -308,6 +308,17 @@ extension PeekabooBridgeClient {
+         }
+     }
+
++    /// Selects one enabled keyboard input source by ID; the caller reads the current source back.
++    public func selectInputSourceWithOutcome(id: String) async throws -> UIAutomationActionResult<Void> {
++        try await self.actionResult(
++            for: .selectInputSource(PeekabooBridgeSelectInputSourceRequest(id: id)),
++            expectedResponse: "input source")
++        { response in
++            guard case .ok = response else { return nil }
++            return ()
++        }
++    }
++
+     public func waitForElement(target: ClickTarget, timeout: TimeInterval, snapshotId: String?) async throws
+         -> WaitForElementResult
+     {
+diff --git a/Core/PeekabooCore/Sources/PeekabooBridge/PeekabooBridgeModels.swift b/Core/PeekabooCore/Sources/PeekabooBridge/PeekabooBridgeModels.swift
+index c4c149896..d744d9f77 100644
+--- a/Core/PeekabooCore/Sources/PeekabooBridge/PeekabooBridgeModels.swift
++++ b/Core/PeekabooCore/Sources/PeekabooBridge/PeekabooBridgeModels.swift
+@@ -95,6 +95,7 @@ public enum PeekabooBridgeOperation: String, Codable, Sendable, CaseIterable, Ha
+     case drag
+     case moveMouse
+     case systemDefinedKey
++    case selectInputSource
+     case waitForElement
+     // Windows
+     case listWindows
+diff --git a/Core/PeekabooCore/Sources/PeekabooBridge/PeekabooBridgeOperation+Policy.swift b/Core/PeekabooCore/Sources/PeekabooBridge/PeekabooBridgeOperation+Policy.swift
+index 0522ba9a0..430dc5318 100644
+--- a/Core/PeekabooCore/Sources/PeekabooBridge/PeekabooBridgeOperation+Policy.swift
++++ b/Core/PeekabooCore/Sources/PeekabooBridge/PeekabooBridgeOperation+Policy.swift
+@@ -78,6 +78,7 @@ extension PeekabooBridgeOperation {
+         .drag,
+         .moveMouse,
+         .systemDefinedKey,
++        .selectInputSource,
+         .waitForElement,
+         .listWindows,
+         .focusWindow,
+diff --git a/Core/PeekabooCore/Sources/PeekabooBridge/PeekabooBridgeOperationDescriptor.swift b/Core/PeekabooCore/Sources/PeekabooBridge/PeekabooBridgeOperationDescriptor.swift
+index 48d19bf4e..71384d1a1 100644
+--- a/Core/PeekabooCore/Sources/PeekabooBridge/PeekabooBridgeOperationDescriptor.swift
++++ b/Core/PeekabooCore/Sources/PeekabooBridge/PeekabooBridgeOperationDescriptor.swift
+@@ -301,6 +301,12 @@ extension PeekabooBridgeOperationResultSemantics {
+                 completion: .dispatchedUnverified(globalForeground),
+                 targetPolicy: .global,
+                 responseFamilies: [.ok])
++        case .selectInputSource:
++            descriptor(
++                ownership: .service,
++                completion: .dispatchedUnverified(nativeBackground),
++                targetPolicy: .global,
++                responseFamilies: [.ok])
+         case .targetedHotkey:
+             descriptor(
+                 ownership: .service,
+diff --git a/Core/PeekabooCore/Sources/PeekabooBridge/PeekabooBridgeOperationResultSemantics.swift b/Core/PeekabooCore/Sources/PeekabooBridge/PeekabooBridgeOperationResultSemantics.swift
+index fe1593526..535e995a7 100644
+--- a/Core/PeekabooCore/Sources/PeekabooBridge/PeekabooBridgeOperationResultSemantics.swift
++++ b/Core/PeekabooCore/Sources/PeekabooBridge/PeekabooBridgeOperationResultSemantics.swift
+@@ -494,6 +494,7 @@ extension PeekabooBridgeOperationResultSemantics {
+              .drag,
+              .moveMouse,
+              .systemDefinedKey,
++             .selectInputSource,
+              .waitForElement,
+              .listWindows,
+              .listWindowMutationInventory,
+@@ -610,6 +611,7 @@ extension PeekabooBridgeOperationResultSemantics {
+         case .clipboardTextWrite, .clipboardImageWrite, .clipboardFilesWrite, .literalInsert, .agentExecutionTrace:
+             return [.dispatchedUnverified]
+         case .requestPostEventPermission, .browserExecute, .swipe, .drag, .moveMouse, .systemDefinedKey,
++             .selectInputSource,
+              .menuCommand, .clickMenuItem, .clickMenuItemByName, .clickMenuExtra,
+              .clickMenuBarItemNamed, .clickMenuBarItemIndex,
+              .launchDockItem, .rightClickDockItem,
+diff --git a/Core/PeekabooCore/Sources/PeekabooBridge/PeekabooBridgePayloads.swift b/Core/PeekabooCore/Sources/PeekabooBridge/PeekabooBridgePayloads.swift
+index aa7a56df4..06809ce88 100644
+--- a/Core/PeekabooCore/Sources/PeekabooBridge/PeekabooBridgePayloads.swift
++++ b/Core/PeekabooCore/Sources/PeekabooBridge/PeekabooBridgePayloads.swift
+@@ -397,6 +397,15 @@ public struct PeekabooBridgeSystemDefinedKeyRequest: Codable, Sendable {
+     }
+ }
+
++public struct PeekabooBridgeSelectInputSourceRequest: Codable, Sendable {
++    /// The `kTISPropertyInputSourceID` of an enabled keyboard input source that can be selected.
++    public let id: String
++
++    public init(id: String) {
++        self.id = id
++    }
++}
++
+ public struct PeekabooBridgeWaitRequest: Codable, Sendable {
+     public let target: ClickTarget
+     public let timeout: TimeInterval
+diff --git a/Core/PeekabooCore/Sources/PeekabooBridge/PeekabooBridgeRequestResponse.swift b/Core/PeekabooCore/Sources/PeekabooBridge/PeekabooBridgeRequestResponse.swift
+index 3462f5963..a3245fdd3 100644
+--- a/Core/PeekabooCore/Sources/PeekabooBridge/PeekabooBridgeRequestResponse.swift
++++ b/Core/PeekabooCore/Sources/PeekabooBridge/PeekabooBridgeRequestResponse.swift
+@@ -62,6 +62,7 @@ public enum PeekabooBridgeRequest: Codable, Sendable {
+     case drag(PeekabooBridgeDragRequest)
+     case moveMouse(PeekabooBridgeMoveMouseRequest)
+     case systemDefinedKey(PeekabooBridgeSystemDefinedKeyRequest)
++    case selectInputSource(PeekabooBridgeSelectInputSourceRequest)
+     case waitForElement(PeekabooBridgeWaitRequest)
+     case listWindows(PeekabooBridgeWindowTargetRequest)
+     case listWindowMutationInventory(PeekabooBridgeWindowTargetRequest)
+@@ -203,6 +204,7 @@ extension PeekabooBridgeRequest {
+         case .drag: .drag
+         case .moveMouse: .moveMouse
+         case .systemDefinedKey: .systemDefinedKey
++        case .selectInputSource: .selectInputSource
+         case .waitForElement: .waitForElement
+         case .listWindows, .listWindowMutationInventory: .listWindows
+         case .focusWindow: .focusWindow
+diff --git a/Core/PeekabooCore/Sources/PeekabooBridge/PeekabooBridgeServer+Handlers.swift b/Core/PeekabooCore/Sources/PeekabooBridge/PeekabooBridgeServer+Handlers.swift
+index c2dcae48d..2be933e32 100644
+--- a/Core/PeekabooCore/Sources/PeekabooBridge/PeekabooBridgeServer+Handlers.swift
++++ b/Core/PeekabooCore/Sources/PeekabooBridge/PeekabooBridgeServer+Handlers.swift
+@@ -101,7 +101,8 @@ extension PeekabooBridgeServer {
+              .foregroundModifierClick,
+              .setValue, .selectText, .performAction, .scroll, .targetedScroll, .hotkey, .targetedHotkey,
+              .exactWindowTargetedHotkey, .targetedClick,
+-             .exactWindowTargetedClick, .swipe, .drag, .moveMouse, .systemDefinedKey, .waitForElement:
++             .exactWindowTargetedClick, .swipe, .drag, .moveMouse, .systemDefinedKey, .selectInputSource,
++             .waitForElement:
+             return try await self.handleAutomationRequest(request)
+         case .createExactWindowHeldPointerOwner, .beginExactWindowHeldPointer,
+              .releaseExactWindowHeldPointer, .revokeExactWindowHeldPointer,
+@@ -345,6 +346,16 @@ extension PeekabooBridgeServer {
+         case let .systemDefinedKey(payload):
+             try SystemDefinedKeyEvents.post(payload.key)
+             return Self.globalPointerMutationResponse()
++        case let .selectInputSource(payload):
++            try InputSourceSelection.select(payload.id)
++            return .init(
++                response: .ok,
++                mutation: .init(
++                    outcome: .dispatchedUnverified(
++                        delivery: .init(mechanism: .nativeFramework, mode: .background),
++                        evidence: .deliveryAccepted,
++                        unitCount: .one),
++                    target: .global))
+         case let .waitForElement(payload):
+             let result = try await self.services.automation.waitForElement(
+                 target: payload.target,
+diff --git a/Core/PeekabooCore/Tests/PeekabooTests/PeekabooBridgeOperationResultSemanticsTests.swift b/Core/PeekabooCore/Tests/PeekabooTests/PeekabooBridgeOperationResultSemanticsTests.swift
+index 53e0e37fd..02cec2501 100644
+--- a/Core/PeekabooCore/Tests/PeekabooTests/PeekabooBridgeOperationResultSemanticsTests.swift
++++ b/Core/PeekabooCore/Tests/PeekabooTests/PeekabooBridgeOperationResultSemanticsTests.swift
+@@ -299,6 +299,7 @@ struct PeekabooBridgeOperationResultSemanticsTests {
+             .drag,
+             .moveMouse,
+             .systemDefinedKey,
++            .selectInputSource,
+             .focusWindow,
+             .moveWindow,
+             .resizeWindow,
+diff --git a/Core/PeekabooCore/Tests/PeekabooTests/PeekabooBridgeOperationSemanticPlanTests.swift b/Core/PeekabooCore/Tests/PeekabooTests/PeekabooBridgeOperationSemanticPlanTests.swift
+index 939ba0f01..6e0c56664 100644
+--- a/Core/PeekabooCore/Tests/PeekabooTests/PeekabooBridgeOperationSemanticPlanTests.swift
++++ b/Core/PeekabooCore/Tests/PeekabooTests/PeekabooBridgeOperationSemanticPlanTests.swift
+@@ -28,7 +28,7 @@ struct PeekabooBridgeOperationSemanticPlanTests {
+
+         // This count is intentionally reviewed whenever the wire enum grows. The exhaustive
+         // policy switches also make a missing classification a compile error.
+-        #expect(operations.count == 130)
++        #expect(operations.count == 131)
+
+         func partition<Value: Equatable>(_ values: [Value], by keyPath: KeyPath<Semantics.OperationPolicy, Value>) {
+             let groups = values.map { expected in
+diff --git a/Core/PeekabooCore/Tests/PeekabooTests/PeekabooBridgeRequestPlanTests.swift b/Core/PeekabooCore/Tests/PeekabooTests/PeekabooBridgeRequestPlanTests.swift
+index 4c4960007..af5a3c937 100644
+--- a/Core/PeekabooCore/Tests/PeekabooTests/PeekabooBridgeRequestPlanTests.swift
++++ b/Core/PeekabooCore/Tests/PeekabooTests/PeekabooBridgeRequestPlanTests.swift
+@@ -9,7 +9,7 @@ struct PeekabooBridgeRequestPlanTests {
+     @Test
+     func `Every wire operation has one complete static descriptor`() {
+         let operations = PeekabooBridgeOperation.allCases
+-        #expect(operations.count == 130)
++        #expect(operations.count == 131)
+
+         let descriptors = operations.map(Semantics.operationDescriptor(for:))
+         #expect(descriptors.map(\.operation) == operations)

native/sources/client/client.swift

@@ -96,7 +96,7 @@ private enum Client {
 		case "switch-space":
 			return try await nativeSwitchSpace(client, handshake: handshake)
 		case "input-source":
-			return try await nativeInputSource()
+			return try await nativeInputSource(client, handshake: handshake)
 		case "management":
 			return try await nativeManagement(client, handshake: handshake)
 		case "launch":