Skip to content

Wait for local channel storage to close before deleting it #119

Closed

Description

@iamnbutler

Deleting a running local channel can remove its SQLite files while the channel worker is still closing storage. The worker logs disk I/O error and remains alive after the channel disappears from the catalog.

Observed during a real githubnext/desktop-tools#11 dev-runner smoke on main 677c226 plus the native-client environment fix. A gateway-created disposable channel finished a real model run, then was deleted through the normal gateway. delete returned, but its worker was still parented by the source host. Bounded logs showed kill → worker.exit → channel.close → unhandled disk I/O error on October 5 at 19:02:26 UTC. Only that verified disposable worker required termination; no existing channel data was changed.

The local ordering is:

  • apps/host/src/manage.ts: remove awaits kill, then removes lane worktrees and the catalog/store.
  • kill uses the short request helper, which resolves on the reply.
  • apps/host/src/worker.ts acknowledges kill before asynchronously closing the channel. Its connection is deliberately ended only after storage/tool cleanup, but this caller does not await that receipt.
  • SQLite close fails if removal wins the race; the unhandled rejection leaves the worker stuck with exiting=true before it reaches process exit.

Expected: local deletion waits for the worker's close/cleanup receipt before removing storage, with a bounded failure that preserves the store when cleanup is not confirmed. Reuse the connection-closure pattern already used by host shutdown; hosted-channel kill has different semantics and must not wait for the hosted cell to terminate its socket.

Validate through a real gateway-created channel with actual storage: delete an idle live worker, confirm close precedes removal and the worker exits, and preserve the store if shutdown cannot be confirmed. Track under #5; discovered while validating githubnext/desktop-tools#11/#97.

Activity

  1. iamnbutler commented on Oct 5, 2026

    @iamnbutler
    ContributorAuthor

    The focused fix now waits for the local worker's connection-close receipt before removing channel files, with a ten-second timeout that preserves the store. Hosted kill keeps its existing behavior. Review also caught an earlier shutdown window: the worker stopped accepting connections before closing SQLite. The listener now remains reachable through storage/tool cleanup, so the existing shutdown guard can refuse concurrent requests instead of making a still-closing worker look dormant.

    Real source-host validation passed:

    • Five live SQLite channel deletions with committed messages and active watches; every worker exited cleanly.
    • Five pairs of concurrent delete requests; one completed, the competing request failed cleanly, and there were no SQLite or unhandled shutdown errors.
    • Dormant deletion without starting a worker.
    • A deliberately paused disposable worker caused deletion to time out after ten seconds. Catalog, SQLite, and WAL hashes were unchanged. Resuming the worker and retrying completed cleanup.

    All disposable workers and the source host exited; the temporary profile and project were removed. Full type and lint checks passed.

    Separate follow-up scope from source review: gateway connections are cached per client, and a separate client or CLI can attempt worker admission after the old worker has fully closed but before deletion finishes removing its lane/store files. There is no shared per-channel admission barrier. This broader race was not reproduced by these checks and is not introduced by the close-ordering fix; investigate it separately rather than expanding this fix into a lifecycle redesign.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions