Skip to content

feat(miner-concurrency): add git-worktree-per-attempt allocator #4297

Description

@JSONbored

Note the scope split up front: the tracker (#2353, Phase 3) separately lists feat(miner-hands): git-worktree-per-attempt isolation primitive (still open, not yet released as an individually-filed contributor issue). That item defines the PRIMITIVE — create/teardown one git worktree for one coding-agent attempt. This issue is the different, complementary piece: the ALLOCATOR that manages a POOL of worktrees across multiple concurrent fleet attempts (acquire/release, a concurrency cap, and reclaiming orphaned worktrees after a crash) — the piece fleet mode (this Phase's sibling Dockerfile issue) actually needs in order to run more than one attempt at a time without two attempts colliding on the same working tree. Confirmed by search: there is currently zero worktree-related code anywhere in packages/gittensory-miner/ or packages/gittensory-engine/ — this is genuinely new ground, not a rename or extension of something that already exists.

The package's existing local-store convention is the shape to reuse rather than invent a new persistence style: packages/gittensory-miner/lib/claim-ledger.js, lib/run-state.js, and lib/portfolio-queue.js are all plain-JS, node:sqlite-backed local stores under GITTENSORY_MINER_CONFIG_DIR — claim-ledger.js's own header comment calls this out explicitly as the pattern siblings should mirror ("Mirrors the package's existing local-store pattern (run-state.js, portfolio-queue.js, event-ledger.js) — plain JS + node:sqlite", claim-ledger.js:9-10). An allocator needs the same kind of durable bookkeeping — which worktree paths are currently allocated to which attempt IDs — so a crashed/restarted miner process can tell a stale allocation from a live one instead of silently reusing (or permanently leaking) a path.

Deliverables

  • packages/gittensory-miner/lib/worktree-allocator.js (or split pure scheduling logic into gittensory-engine with a thin SQLite-bookkeeping wrapper in the miner package — either split is fine as long as the pure allocation logic is unit-testable without touching a real filesystem).
  • acquire(attemptId, repoFullName) → allocates an unused worktree path (base directory configurable, defaulting under GITTENSORY_MINER_CONFIG_DIR), enforcing a configurable max-concurrency cap.
  • release(attemptId) → frees the path for reuse.
  • Orphan reclamation: on startup, detect worktree paths recorded as allocated whose owning process/attempt is no longer live, and make them available again (or flag them for manual cleanup) rather than leaking them forever.
  • Persist allocations the same way claim-ledger.js/run-state.js do (local SQLite, chmodSync 0o600, busy_timeout pragma for same-file concurrent access) so the bookkeeping survives a restart.
  • Unit tests colocated with the rest of the miner test suite — miner tests live at repo-root test/unit/miner-*.test.ts, not inside the package (see test/unit/miner-claim-ledger.test.ts / test/unit/miner-run-state.test.ts for the existing pattern).

References

  • Tracker 🧩 Wave 2 — Analyze, Plan, Create & Deploy (HELD tracker) #2353, Phase 3 — feat(miner-hands): git-worktree-per-attempt isolation primitive (the sibling primitive item; different scope, not yet filed individually).
  • packages/gittensory-miner/lib/claim-ledger.js:6-10,75-94 — the local-store pattern (plain JS + node:sqlite, 0600-permissioned file under GITTENSORY_MINER_CONFIG_DIR) this allocator's persistence should mirror.
  • packages/gittensory-miner/lib/run-state.js:12-27 — the GITTENSORY_MINER_*_DB / GITTENSORY_MINER_CONFIG_DIR / XDG_CONFIG_HOME path-resolution chain every sibling store reuses.
  • test/unit/miner-claim-ledger.test.ts, test/unit/miner-run-state.test.ts — existing miner test conventions/location to follow.

Metadata

Metadata

Assignees

No one assigned

    Labels

    gittensor:featureGittensor-scored feature linked to a feature issue — scores a 0.25x multiplier.gittensor:priorityMaintainer-selected Gittensor priority — scores a 1.5x multiplier.help wantedExtra attention is needed

    Projects

    Status
    Done

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions