From ef445de704fe7c8fb9013290b4dda2e223d8a53c Mon Sep 17 00:00:00 2001 From: Sutu Sebastian Date: Mon, 25 May 2026 16:46:14 +0300 Subject: [PATCH 1/5] feat(mcp): add trace, explore, and node MCP/HTTP tools Compose call-path and symbol-neighborhood recipes with budget-capped snippets, register tools on MCP/HTTP, and document recipe-first fallbacks. --- docs/architecture.md | 2 + docs/plans/agent-surface-delivery.md | 14 +- src/application/http-server.test.ts | 64 +++ src/application/http-server.ts | 27 ++ src/application/mcp-server.test.ts | 99 +++++ src/application/mcp-server.ts | 45 +++ src/application/mcp-tool-allowlist.ts | 3 + src/application/output-budget.test.ts | 27 ++ src/application/output-budget.ts | 27 ++ src/application/tool-handlers.ts | 135 +++++++ src/application/trace-engine.test.ts | 150 +++++++ src/application/trace-engine.ts | 372 ++++++++++++++++++ templates/agent-content/mcp-instructions.md | 8 +- .../agent-content/skill/10-recipes-context.md | 3 + 14 files changed, 967 insertions(+), 9 deletions(-) create mode 100644 src/application/output-budget.test.ts create mode 100644 src/application/output-budget.ts create mode 100644 src/application/trace-engine.test.ts create mode 100644 src/application/trace-engine.ts diff --git a/docs/architecture.md b/docs/architecture.md index 9e723d0a..2abf3d89 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -135,6 +135,8 @@ A local SQLite database (`.codemap/index.db`) indexes the project tree and store **Affected wiring:** **`src/cli/cmd-affected.ts`** (argv — positional paths / `--stdin` / `--changed-since ` / `--params test_glob|max_depth` + `--json`; bootstrap absorbs `--root`/`--config`) + **`src/application/affected-engine.ts`** (engine — `resolveAffectedChangedPaths` + `executeAffectedTests`; pure recipe composer over bundled `affected-tests` SQL). CLI / MCP / HTTP dispatch the same engine via `tool-handlers.ts`'s `handleAffected` (MCP/HTTP) and `runAffectedCmd` (CLI). Path precedence: explicit paths (CLI positional / MCP `paths` array) → CLI `--stdin` → git vs `changed_since` / `HEAD` (`paths: []` on MCP/HTTP skips git). Result envelope: JSON array of `{test_path, impact_depth, actions?}` — file paths only; CI composes the runner command. **`tryRecordRecipeRun("affected-tests")`** lives at the orchestration layer (`handleAffected` + `runAffectedCmd`), not in the engine — same boundary discipline as `query_recipe` (see [§ `recipe_recency`](#recipe_recency--per-recipe-last-run--run-count-user-data-strict-without-rowid)). Recency records only when at least one changed path was resolved and the recipe SQL ran (empty path sets return `[]` without a recency write). +**Trace / explore / node wiring (MCP + HTTP only):** **`src/application/trace-engine.ts`** (engine — `executeCallPath` / `executeSymbolNeighborhood` recipe composers + `composeTraceResult` / `composeExploreResult` / `composeNodeResult` snippet batching) + **`src/application/output-budget.ts`** (`applySourceCharBudget`, default 15k chars). MCP/HTTP dispatch via `tool-handlers.ts`'s `handleTrace` / `handleExplore` / `handleNode`. **`trace`** → `call-path` recipe + disk snippets per hop; **`explore`** → `symbol-neighborhood` once per `names[]` entry, merged rows + budget-capped snippets; **`node`** → `show` center (`findSymbolsByName` + `buildShowResult`) + depth-1 neighborhood + optional snippets. Recipe twins remain the Moat A fallback (`query_recipe call-path`, `query_recipe symbol-neighborhood`). **`tryRecordRecipeRun`** at orchestration only (`call-path` on trace success; `symbol-neighborhood` on explore/node success). + **Apply wiring:** **`src/cli/cmd-apply.ts`** (argv — `` + `--params` + `--dry-run` + `--yes` + `--json`; bootstrap absorbs `--root`/`--config`) + **`src/application/apply-engine.ts`** (engine — `applyDiffPayload({rows, projectRoot, dryRun})`). Pure transport-agnostic substrate-shaped fix executor: consumes the existing `--format diff-json` row contract from any recipe (`{file_path, line_start, before_pattern, after_pattern}`), validates each row against current disk, and either previews (dry-run) or writes (apply). CLI / MCP / HTTP all dispatch the same engine via `tool-handlers.ts`'s `handleApply`. **Phase 1** (always) resolves the project root via `path.resolve(projectRoot)` once, then for each row: rejects absolute `file_path` inputs and any candidate whose `path.resolve(resolvedRoot, file_path)` lands outside `resolvedRoot` (conflict `path escapes project root` — guards CLI + MCP + HTTP write paths against `../escape.ts`-style traversal); rejects duplicate `(file_path, line_start)` tuples (conflict `duplicate edit on same line` — without this, two phase-1-passing rows targeting the same line would split the run mid-phase-2 because the first replace invalidates the second's substring assertion, leaving Q2 (c) cross-file partial state). Reads each file at most once into `sourceCache`, splits on `/\r?\n/` for conflict reporting, checks `actual.includes(before_pattern)` (substring match — mirrors `buildDiffJson`'s contract; `rename-preview` emits `before_pattern = old_name` as the bare identifier, so whole-line exact match would conflict every time). Conflicts collect five reasons (`file missing` / `line out of range` / `line content drifted` / `path escapes project root` / `duplicate edit on same line`) — Q3 scan-and-collect, not fail-fast. **Phase 2** (gated on `!dryRun && conflicts.length === 0`) re-splits the cached source on raw `"\n"` (preserves CRLF as trailing `\r` per line; rejoining with `"\n"` round-trips losslessly), applies each file's edits in descending line order via `actual.replace(before, after)` with `$`-pre-escape (`replace(/\$/g, "$$$$")` — matches `buildDiffJson`'s GetSubstitution defence so identifiers like `$inject` round-trip safely), writes to a sibling temp path (`.codemap-apply-.tmp`), then `renameSync` into place — POSIX-atomic per file; concurrent readers see either pre-rename or post-rename content, never a torn write. **Q2 (c) all-or-nothing (semantic)**: any phase-1 conflict aborts phase 2 entirely before any file is touched. Phase-2 I/O failures (`writeFileSync` / `renameSync`) are NOT transactional across files — per-file atomicity holds (temp + rename), but a crash on file N leaves files `1..N-1` already renamed with no rollback; cross-file rollback would require pre-write backups + restore-on-throw and is deferred to a future PR. **Q6 gate**: TTY no `--yes` → phase-1 preview + `Proceed? [y/N]` prompt on stderr (default-N, `node:readline/promises`); TTY `--yes` → no prompt; non-TTY (CI / agents / MCP) without `--yes`/`--dry-run` rejected with stderr message. `--dry-run` + `--yes` mutually exclusive (parse-time error). MCP/HTTP transports (`handleApply`) require `yes: true` for the write path — there's no prompt to fall back on; `dry_run + yes` rejected as mutually exclusive. Result envelope (Q5; identical across modes): `{mode: 'dry-run'|'apply', applied: bool, files: [{file_path, rows_applied, warnings?}], conflicts: [{file_path, line_start, before_pattern, actual_at_line, reason}], summary: {files, files_modified, rows, rows_applied, conflicts, files_with_conflicts}}`. `applied: true` only when `mode === 'apply'` AND zero conflicts AND at least one row applied. Q7 idempotency: re-running on already-applied code reports a `line content drifted` conflict with `actual_at_line` showing the post-rename content; the user reads it and re-runs `codemap` to refresh the index → next run produces 0 rows (recipe finds nothing to rename) → vacuous clean apply. **Same-line ambiguity caveat (documented limitation):** `actual.replace(before_pattern, after_pattern)` rewrites only the **first** occurrence on the line. When `before_pattern` appears twice (e.g. `const foo = foo();` with `before = "foo"`) only the leftmost is replaced; the engine still reports `applied: true`. This mirrors `buildDiffJson`'s formatter contract verbatim — recipe authors who hit it normalise their SQL to emit a more specific pattern, or accept it (the formatter's `--format diff` preview shows the same shape). Promotion path: tighten phase-1 to conflict on ambiguity in a future PR if real users complain, but only alongside the formatter so preview and execution stay in lockstep. SARIF / annotations not supported (write action, not findings). TOCTOU: phase-1 reads through `sourceCache`; phase-2 transforms the cached source and writes — the gap between read and rename is a deliberate v1 simplification (apply isn't adversarial). Per Q10, only `cli/cmd-apply.ts` + `application/tool-handlers.ts` (+ the test files) may import `apply-engine.ts` for production execution — re-runnable forbidden-edge query at [§ Boundary verification — apply write path](#boundary-verification--apply-write-path). **Show / snippet wiring:** **`src/cli/cmd-show.ts`** + **`src/cli/cmd-snippet.ts`** — sibling CLI verbs sharing the same parser shape (`` + `--kind` + `--in ` + `--json`) and the pure engine **`src/application/show-engine.ts`** (`findSymbolsByName({db, name, kind?, inPath?})` for the lookup; `readSymbolSource({match, projectRoot, indexedContentHash?})` + `getIndexedContentHash(db, filePath)` for the snippet-side FS read; **`buildShowResult`** + **`buildSnippetResult`** envelope builders — same engine the MCP show/snippet tools call). Both verbs return the same `{matches, disambiguation?}` envelope per plan § 4 uniformity — single match → `{matches: [{...}]}`; multi-match adds `{n, by_kind, files, hint}`. Snippet matches add `source` / `stale` / `missing` fields (additive — no shape divergence). **`--in `** is normalized through `toProjectRelative(projectRoot, p)` (from **`src/application/validate-engine.ts`**) so `--in ./src/cli/`, `--in src/cli`, and `--in src/cli/cmd-show.ts` all resolve identically. Stale-file behavior on `snippet`: `hashContent` (from **`src/hash.ts`** — same primitive `cmd-validate.ts` uses) compares the on-disk content_hash against `files.content_hash`; mismatch sets `stale: true` but the source IS still returned (read tool, no auto-reindex side-effects). MCP tools `show` and `snippet` register parallel to the CLI surface (see [§ MCP wiring](#cli-usage)). diff --git a/docs/plans/agent-surface-delivery.md b/docs/plans/agent-surface-delivery.md index d6476edd..3bf29b40 100644 --- a/docs/plans/agent-surface-delivery.md +++ b/docs/plans/agent-surface-delivery.md @@ -10,11 +10,11 @@ ## Quick resume -| Next action | Detail | -| -------------------- | --------------------------------------------------------------------------- | -| **Review / merge** | [#133](https://github.com/stainless-code/codemap/pull/133) — MCP `affected` | -| **Start next** | **PR 6** — MCP trace tools (`trace` / `explore` / `node`) | -| **Do not start yet** | PR 9 (eval harness) until PR 8 | +| Next action | Detail | +| -------------------- | --------------------------------------------------------------------------------------------------- | +| **Review / merge** | [#133](https://github.com/stainless-code/codemap/pull/133) — MCP `affected` (perf baseline pending) | +| **Start next** | **PR 6** — MCP trace tools (`trace` / `explore` / `node`) — branch `feat/mcp-trace-tools` open | +| **Do not start yet** | PR 9 (eval harness) until PR 8 | Update the table below when a PR merges or a new branch opens. @@ -36,11 +36,11 @@ Merge each PR to `main` directly. No long-lived integration branch (`feat/agent- Max **3 parallel tracks** at once. | PR | Plans | Status | Blocked by | Parallel with | -| ----- | ---------------------------------------------------------------------------------------------------------------------------------------------- | ------- | ---------------------------------------------------------------------------------------------------------------------- | --------------------------------- | +| ----- | ---------------------------------------------------------------------------------------------------------------------------------------------- | ------- | ---------------------------------------------------------------------------------------------------------------------- | --------------------------------- | --- | | **3** | [`index-lock-and-error-log`](./index-lock-and-error-log.md) → [`parse-worker-hardening`](./parse-worker-hardening.md) (stack) | merged | [#129](https://github.com/stainless-code/codemap/pull/129), [#130](https://github.com/stainless-code/codemap/pull/130) | 4, 5 | | **4** | Recipe half of [`mcp-trace-explore-tools`](./mcp-trace-explore-tools.md) (`call-path`, `symbol-neighborhood` SQL + tests) | merged | [#131](https://github.com/stainless-code/codemap/pull/131) | 3, 5 | | **5** | [`affected-tests-recipe`](./affected-tests-recipe.md) (+ Phase 2 MCP `affected` in [#133](https://github.com/stainless-code/codemap/pull/133)) | merged | [#132](https://github.com/stainless-code/codemap/pull/132), [#133](https://github.com/stainless-code/codemap/pull/133) | 3, 4 | -| **6** | MCP half of trace (`trace` / `explore` / `node` tools) + update instructions | planned | PR 1, PR 4 | — | +| **6** | MCP half of trace (`trace` / `explore` / `node` tools) + update instructions | open | `feat/mcp-trace-tools` | PR 1, PR 4 | — | | **7** | [`field-qualified-search`](./field-qualified-search.md) | planned | PR 1 | 4, 5 if `mcp-server.ts` untouched | | **8** | [`agents-init-mcp-wiring`](./agents-init-mcp-wiring.md) | planned | PR 1 | 3–5 | | **9** | [`agent-eval-harness`](./agent-eval-harness.md) | planned | PR 1, PR 8, allowlist | **last P1** | diff --git a/src/application/http-server.test.ts b/src/application/http-server.test.ts index 4a81a873..458507b7 100644 --- a/src/application/http-server.test.ts +++ b/src/application/http-server.test.ts @@ -124,6 +124,7 @@ describe("http-server — health + tools catalog", () => { expect(body.tools.map((t) => t.name)).toContain("query"); expect(body.tools.map((t) => t.name)).toContain("audit"); expect(body.tools.map((t) => t.name)).toContain("affected"); + expect(body.tools.map((t) => t.name)).toContain("trace"); }); it("404 for unknown route", async () => { @@ -405,6 +406,69 @@ describe("http-server — POST /tool/{other tools}", () => { expect(r.json.error).not.toContain("--changed-since"); }); + function seedTraceGraph() { + writeFileSync( + join(benchDir, "src", "trace.ts"), + "export function alpha() {\n return beta();\n}\nexport function beta() {\n return 1;\n}\n", + ); + const db = openDb(); + try { + db.run( + `INSERT INTO files (path, content_hash, size, line_count, language, last_modified, indexed_at) + VALUES ('src/trace.ts', 'ht', 100, 6, 'typescript', 1, 1)`, + ); + db.run( + `INSERT INTO symbols (name, kind, file_path, line_start, line_end, signature, is_exported, parent_name, visibility) + VALUES ('alpha', 'function', 'src/trace.ts', 1, 3, 'alpha()', 1, NULL, 'export'), + ('beta', 'function', 'src/trace.ts', 4, 6, 'beta()', 1, NULL, 'export')`, + ); + db.run( + `INSERT INTO calls (file_path, caller_name, caller_scope, callee_name, line_start, column_start, column_end) + VALUES ('src/trace.ts', 'alpha', 'alpha', 'beta', 2, 0, 0)`, + ); + } finally { + closeDb(db); + } + } + + it("trace returns path and snippets", async () => { + seedTraceGraph(); + serverHandle = await startServer(); + const r = await postTool(serverHandle.port, "trace", { + from: "alpha", + to: "beta", + }); + expect(r.status).toBe(200); + expect(r.json.path).toHaveLength(1); + expect(r.json.snippets.length).toBeGreaterThan(0); + expect(r.json.truncated).toBe(false); + }); + + it("explore merges neighborhoods", async () => { + seedTraceGraph(); + serverHandle = await startServer(); + const r = await postTool(serverHandle.port, "explore", { + names: ["alpha", "beta"], + }); + expect(r.status).toBe(200); + expect(r.json.names).toEqual(["alpha", "beta"]); + expect(r.json.rows.length).toBeGreaterThan(0); + }); + + it("node returns center + neighborhood", async () => { + seedTraceGraph(); + serverHandle = await startServer(); + const r = await postTool(serverHandle.port, "node", { + name: "alpha", + include_snippets: true, + }); + expect(r.status).toBe(200); + expect(r.json.center.matches[0]?.name).toBe("alpha"); + expect( + r.json.neighborhood.some((row: { name: string }) => row.name === "beta"), + ).toBe(true); + }); + it("list_baselines returns array (empty when none saved)", async () => { serverHandle = await startServer(); const r = await postTool(serverHandle.port, "list_baselines", {}); diff --git a/src/application/http-server.ts b/src/application/http-server.ts index 91252e9d..b72b0a65 100644 --- a/src/application/http-server.ts +++ b/src/application/http-server.ts @@ -19,12 +19,16 @@ import { auditArgsSchema, contextArgsSchema, dropBaselineArgsSchema, + exploreArgsSchema, handleApply, handleAudit, handleAffected, handleContext, handleDropBaseline, + handleExplore, handleImpact, + handleNode, + handleTrace, handleListBaselines, handleQuery, handleQueryBatch, @@ -34,6 +38,8 @@ import { handleSnippet, handleValidate, impactArgsSchema, + nodeArgsSchema, + traceArgsSchema, listBaselinesArgsSchema, queryArgsSchema, queryBatchArgsSchema, @@ -94,6 +100,9 @@ const TOOL_NAMES = [ "snippet", "impact", "affected", + "trace", + "explore", + "node", "apply", "save_baseline", "list_baselines", @@ -489,6 +498,24 @@ async function dispatchTool( result = handleAffected(r.value, opts.root); break; } + case "trace": { + const r = validate(traceArgsSchema, args, "trace"); + if (!r.ok) return writeJson(res, 400, { error: r.error }, opts.version); + result = handleTrace(r.value, opts.root); + break; + } + case "explore": { + const r = validate(exploreArgsSchema, args, "explore"); + if (!r.ok) return writeJson(res, 400, { error: r.error }, opts.version); + result = handleExplore(r.value, opts.root); + break; + } + case "node": { + const r = validate(nodeArgsSchema, args, "node"); + if (!r.ok) return writeJson(res, 400, { error: r.error }, opts.version); + result = handleNode(r.value, opts.root); + break; + } case "apply": { const r = validate(applyArgsSchema, args, "apply"); if (!r.ok) return writeJson(res, 400, { error: r.error }, opts.version); diff --git a/src/application/mcp-server.test.ts b/src/application/mcp-server.test.ts index 4deca2dd..0d0c913e 100644 --- a/src/application/mcp-server.test.ts +++ b/src/application/mcp-server.test.ts @@ -1420,3 +1420,102 @@ describe("MCP server — affected tool", () => { } }); }); + +describe("MCP server — trace / explore / node tools", () => { + function seedTraceGraph() { + writeFileSync( + join(benchDir, "src", "trace.ts"), + "export function foo() {\n return bar();\n}\nexport function bar() {\n return 1;\n}\n", + ); + const db = openDb(); + try { + db.run( + `INSERT INTO files (path, content_hash, size, line_count, language, last_modified, indexed_at) + VALUES ('src/trace.ts', 'ht', 100, 6, 'typescript', 1, 1)`, + ); + db.run( + `INSERT INTO symbols (name, kind, file_path, line_start, line_end, signature, is_exported, parent_name, visibility) + VALUES ('foo', 'function', 'src/trace.ts', 1, 3, 'foo()', 1, NULL, 'export'), + ('bar', 'function', 'src/trace.ts', 4, 6, 'bar()', 1, NULL, 'export')`, + ); + db.run( + `INSERT INTO calls (file_path, caller_name, caller_scope, callee_name, line_start, column_start, column_end) + VALUES ('src/trace.ts', 'foo', 'foo', 'bar', 2, 0, 0)`, + ); + } finally { + closeDb(db); + } + } + + it("lists trace, explore, and node in tools/list", async () => { + const { client, server } = await makeClient(); + try { + const tools = await client.listTools(); + const names = tools.tools.map((t) => t.name); + expect(names).toContain("trace"); + expect(names).toContain("explore"); + expect(names).toContain("node"); + } finally { + await server.close(); + } + }); + + it("trace returns path and snippets", async () => { + seedTraceGraph(); + const { client, server } = await makeClient(); + try { + const r = await client.callTool({ + name: "trace", + arguments: { from: "foo", to: "bar" }, + }); + const json = readJson(r) as { + path: { callee_name: string }[]; + snippets: { source?: string }[]; + truncated: boolean; + }; + expect(json.path).toHaveLength(1); + expect(json.path[0]?.callee_name).toBe("bar"); + expect(json.snippets.length).toBeGreaterThan(0); + expect(json.truncated).toBe(false); + } finally { + await server.close(); + } + }); + + it("explore merges neighborhoods for multiple names", async () => { + seedTraceGraph(); + const { client, server } = await makeClient(); + try { + const r = await client.callTool({ + name: "explore", + arguments: { names: ["foo", "bar"] }, + }); + const json = readJson(r) as { names: string[]; rows: unknown[] }; + expect(json.names).toEqual(["foo", "bar"]); + expect(json.rows.length).toBeGreaterThan(0); + } finally { + await server.close(); + } + }); + + it("node returns center + neighborhood", async () => { + seedTraceGraph(); + const { client, server } = await makeClient(); + try { + const r = await client.callTool({ + name: "node", + arguments: { name: "foo", include_snippets: true }, + }); + const json = readJson(r) as { + center: { matches: { name: string }[] }; + neighborhood: { name: string }[]; + snippets: unknown[]; + }; + expect(json.center.matches[0]?.name).toBe("foo"); + expect(json.neighborhood.some((row) => row.name === "bar")).toBe(true); + expect(json.snippets.length).toBeGreaterThan(0); + } finally { + await server.close(); + } + }); +}); diff --git a/src/application/mcp-server.ts b/src/application/mcp-server.ts index d0e0e6b3..d8952b5b 100644 --- a/src/application/mcp-server.ts +++ b/src/application/mcp-server.ts @@ -33,7 +33,13 @@ import { handleAffected, handleContext, handleDropBaseline, + exploreArgsSchema, + handleExplore, handleImpact, + handleNode, + handleTrace, + nodeArgsSchema, + traceArgsSchema, handleListBaselines, handleQuery, handleQueryBatch, @@ -149,6 +155,9 @@ export function createMcpServer(opts: ServerOpts): McpServer { maybeRegister("snippet", () => registerSnippetTool(server, opts)); maybeRegister("impact", () => registerImpactTool(server)); maybeRegister("affected", () => registerAffectedTool(server, opts)); + maybeRegister("trace", () => registerTraceTool(server, opts)); + maybeRegister("explore", () => registerExploreTool(server, opts)); + maybeRegister("node", () => registerNodeTool(server, opts)); maybeRegister("apply", () => registerApplyTool(server, opts)); registerResources(server); logMcpToolAllowlist(allowlistResolved, registered); @@ -312,6 +321,42 @@ function registerImpactTool(server: McpServer): void { ); } +function registerTraceTool(server: McpServer, opts: ServerOpts): void { + server.registerTool( + "trace", + { + description: + "Shortest call path between two symbols plus budget-capped snippets. Composes `call-path` recipe + disk reads. Args: from, to (symbol names), max_depth (optional), via (calls|dependencies|all), budget_chars (default 15000). Returns {from, to, via?, path: [{file_path, caller_name, callee_name, line_start, hop, via}], snippets: [{name, file_path, source, stale, missing, ...}], truncated}. Fall back to `query_recipe` with recipe call-path when unsure.", + inputSchema: traceArgsSchema, + }, + (args) => wrapToolResult(handleTrace(args, opts.root)), + ); +} + +function registerExploreTool(server: McpServer, opts: ServerOpts): void { + server.registerTool( + "explore", + { + description: + "Multi-symbol neighborhood survey with budget-capped snippets. Composes `symbol-neighborhood` (once per name) + disk reads. Args: names (non-empty array), depth (optional hop budget), kind (optional filter), budget_chars (default 15000). Returns {names, rows: [...], snippets: [...], truncated}. Fall back to `query_recipe` with recipe symbol-neighborhood.", + inputSchema: exploreArgsSchema, + }, + (args) => wrapToolResult(handleExplore(args, opts.root)), + ); +} + +function registerNodeTool(server: McpServer, opts: ServerOpts): void { + server.registerTool( + "node", + { + description: + "One-hop symbol survey: `show` center match + depth-1 `symbol-neighborhood` + optional inline snippets. Args: name, kind?, in? (path filter), include_snippets (default false), budget_chars (default 15000 when snippets enabled). Returns {center: {matches, disambiguation?}, neighborhood: [...], snippets: [...], truncated}.", + inputSchema: nodeArgsSchema, + }, + (args) => wrapToolResult(handleNode(args, opts.root)), + ); +} + function registerApplyTool(server: McpServer, opts: ServerOpts): void { server.registerTool( "apply", diff --git a/src/application/mcp-tool-allowlist.ts b/src/application/mcp-tool-allowlist.ts index 4b7af019..e8c310d8 100644 --- a/src/application/mcp-tool-allowlist.ts +++ b/src/application/mcp-tool-allowlist.ts @@ -17,6 +17,9 @@ export const MCP_TOOL_NAMES = [ "snippet", "impact", "affected", + "trace", + "explore", + "node", "apply", ] as const; diff --git a/src/application/output-budget.test.ts b/src/application/output-budget.test.ts new file mode 100644 index 00000000..b6dccb4e --- /dev/null +++ b/src/application/output-budget.test.ts @@ -0,0 +1,27 @@ +import { describe, expect, it } from "bun:test"; + +import { + applySourceCharBudget, + DEFAULT_OUTPUT_CHAR_BUDGET, +} from "./output-budget"; + +describe("applySourceCharBudget", () => { + it("returns all items when under budget", () => { + const items = [{ source: "abc" }, { source: "de" }]; + expect(applySourceCharBudget(items, 10)).toEqual({ + items, + truncated: false, + }); + }); + + it("truncates when cumulative source exceeds budget", () => { + const items = [{ source: "aaaa" }, { source: "bbbb" }, { source: "c" }]; + const r = applySourceCharBudget(items, 6); + expect(r.items).toEqual([{ source: "aaaa" }]); + expect(r.truncated).toBe(true); + }); + + it("defaults budget constant is 15k", () => { + expect(DEFAULT_OUTPUT_CHAR_BUDGET).toBe(15_000); + }); +}); diff --git a/src/application/output-budget.ts b/src/application/output-budget.ts new file mode 100644 index 00000000..c98ecb88 --- /dev/null +++ b/src/application/output-budget.ts @@ -0,0 +1,27 @@ +/** Default char budget for trace/explore/node snippet payloads (plan L.3). */ +export const DEFAULT_OUTPUT_CHAR_BUDGET = 15_000; + +export interface SourceCharBudgetResult< + T extends { source?: string | undefined }, +> { + items: T[]; + truncated: boolean; +} + +/** Keep items in order until cumulative `source` length exceeds `budget`. */ +export function applySourceCharBudget< + T extends { source?: string | undefined }, +>(items: T[], budget: number): SourceCharBudgetResult { + if (budget <= 0) return { items: [], truncated: items.length > 0 }; + let used = 0; + const out: T[] = []; + for (const item of items) { + const len = item.source?.length ?? 0; + if (used + len > budget) { + return { items: out, truncated: true }; + } + out.push(item); + used += len; + } + return { items: out, truncated: false }; +} diff --git a/src/application/tool-handlers.ts b/src/application/tool-handlers.ts index b7f298d2..68e4e4e3 100644 --- a/src/application/tool-handlers.ts +++ b/src/application/tool-handlers.ts @@ -67,6 +67,12 @@ import { buildSnippetResult, findSymbolsByName, } from "./show-engine"; +import { + composeExploreResult, + composeNodeResult, + composeTraceResult, + executeCallPath, +} from "./trace-engine"; import { computeValidateRows, toProjectRelative } from "./validate-engine"; import { isWatchActive } from "./watcher"; @@ -824,6 +830,135 @@ export function handleImpact(args: ImpactArgs): ToolResult { } } +// === trace / explore / node ================================================= + +export const traceArgsSchema = { + from: z.string().min(1, "from must be a non-empty string"), + to: z.string().min(1, "to must be a non-empty string"), + max_depth: z.number().int().nonnegative().optional(), + via: z.enum(["calls", "dependencies", "all"]).optional(), + budget_chars: z.number().int().positive().optional(), +}; + +export interface TraceArgs { + from: string; + to: string; + max_depth?: number; + via?: "calls" | "dependencies" | "all"; + budget_chars?: number; +} + +export function handleTrace(args: TraceArgs, root: string): ToolResult { + try { + const pathResult = executeCallPath({ + root, + from: args.from, + to: args.to, + maxDepth: args.max_depth, + via: args.via, + }); + if (!pathResult.ok) { + return err( + pathResult.error, + pathResult.kind === "internal" ? 500 : undefined, + ); + } + tryRecordRecipeRun("call-path"); + const payload = composeTraceResult({ + root, + from: args.from, + to: args.to, + via: args.via, + path: pathResult.rows, + budgetChars: args.budget_chars, + }); + return ok(payload); + } catch (e) { + return err(e instanceof Error ? e.message : String(e), 500); + } +} + +export const exploreArgsSchema = { + names: z + .array(z.string().min(1)) + .min(1, "names must contain at least one symbol"), + depth: z.number().int().nonnegative().optional(), + kind: z.string().optional(), + budget_chars: z.number().int().positive().optional(), +}; + +export interface ExploreArgs { + names: string[]; + depth?: number; + kind?: string; + budget_chars?: number; +} + +export function handleExplore(args: ExploreArgs, root: string): ToolResult { + try { + const composed = composeExploreResult({ + root, + names: args.names, + depth: args.depth, + kind: args.kind, + budgetChars: args.budget_chars, + }); + if (!composed.ok) { + return err( + composed.error, + composed.kind === "internal" ? 500 : undefined, + ); + } + tryRecordRecipeRun("symbol-neighborhood"); + return ok(composed.result); + } catch (e) { + return err(e instanceof Error ? e.message : String(e), 500); + } +} + +export const nodeArgsSchema = { + name: z.string().min(1, "name must be a non-empty string"), + kind: z.string().optional(), + in: z.string().optional(), + include_snippets: z.boolean().optional(), + budget_chars: z.number().int().positive().optional(), +}; + +export interface NodeArgs { + name: string; + kind?: string; + in?: string; + include_snippets?: boolean; + budget_chars?: number; +} + +export function handleNode(args: NodeArgs, root: string): ToolResult { + try { + const inPath = + args.in !== undefined && args.in.length > 0 + ? toProjectRelative(root, args.in) + : undefined; + const composed = composeNodeResult({ + root, + name: args.name, + kind: args.kind, + inPath, + includeSnippets: args.include_snippets, + budgetChars: args.budget_chars, + }); + if (!composed.ok) { + return err( + composed.error, + composed.kind === "internal" ? 500 : undefined, + ); + } + tryRecordRecipeRun("symbol-neighborhood"); + return ok(composed.result); + } catch (e) { + return err(e instanceof Error ? e.message : String(e), 500); + } +} + // === apply ================================================================== export const applyArgsSchema = { diff --git a/src/application/trace-engine.test.ts b/src/application/trace-engine.test.ts new file mode 100644 index 00000000..d9df6c15 --- /dev/null +++ b/src/application/trace-engine.test.ts @@ -0,0 +1,150 @@ +import { afterEach, beforeEach, describe, expect, it } from "bun:test"; +import { mkdirSync, mkdtempSync, rmSync, writeFileSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; + +import { resolveCodemapConfig } from "../config"; +import { closeDb, createTables, openDb } from "../db"; +import { initCodemap } from "../runtime"; +import { + composeExploreResult, + composeNodeResult, + composeTraceResult, + executeCallPath, + executeSymbolNeighborhood, +} from "./trace-engine"; + +let benchDir: string; + +function seedCallGraph() { + writeFileSync( + join(benchDir, "src", "a.ts"), + "export function foo() {\n return bar();\n}\nexport function bar() {\n return 1;\n}\n", + ); + const db = openDb(); + try { + createTables(db); + db.run( + `INSERT INTO files (path, content_hash, size, line_count, language, last_modified, indexed_at) + VALUES ('src/a.ts', 'h1', 100, 6, 'typescript', 1, 1)`, + ); + db.run( + `INSERT INTO symbols (name, kind, file_path, line_start, line_end, signature, is_exported, parent_name, visibility) + VALUES ('foo', 'function', 'src/a.ts', 1, 3, 'foo()', 1, NULL, 'export'), + ('bar', 'function', 'src/a.ts', 4, 6, 'bar()', 1, NULL, 'export')`, + ); + db.run( + `INSERT INTO calls (file_path, caller_name, caller_scope, callee_name, line_start, column_start, column_end) + VALUES ('src/a.ts', 'foo', 'foo', 'bar', 2, 0, 0)`, + ); + } finally { + closeDb(db); + } +} + +beforeEach(() => { + benchDir = mkdtempSync(join(tmpdir(), "trace-engine-")); + mkdirSync(join(benchDir, "src"), { recursive: true }); + initCodemap(resolveCodemapConfig(benchDir, undefined)); +}); + +afterEach(() => { + rmSync(benchDir, { recursive: true, force: true }); +}); + +describe("executeCallPath", () => { + it("returns hop rows for a connected call graph", () => { + seedCallGraph(); + const r = executeCallPath({ root: benchDir, from: "foo", to: "bar" }); + expect(r.ok).toBe(true); + if (!r.ok) return; + expect(r.rows).toEqual([ + expect.objectContaining({ + file_path: "src/a.ts", + caller_name: "foo", + callee_name: "bar", + line_start: 2, + hop: 1, + via: "calls", + }), + ]); + }); + + it("returns empty path when no route exists", () => { + seedCallGraph(); + const r = executeCallPath({ root: benchDir, from: "bar", to: "foo" }); + expect(r.ok).toBe(true); + if (!r.ok) return; + expect(r.rows).toEqual([]); + }); +}); + +describe("executeSymbolNeighborhood", () => { + it("returns direct callees and callers", () => { + seedCallGraph(); + const r = executeSymbolNeighborhood({ + root: benchDir, + name: "foo", + depth: 1, + }); + expect(r.ok).toBe(true); + if (!r.ok) return; + expect( + r.rows.some((row) => row.name === "bar" && row.edge === "callee"), + ).toBe(true); + }); +}); + +describe("composeTraceResult", () => { + it("attaches snippets for call hops", () => { + seedCallGraph(); + const path = executeCallPath({ root: benchDir, from: "foo", to: "bar" }); + expect(path.ok).toBe(true); + if (!path.ok) return; + const composed = composeTraceResult({ + root: benchDir, + from: "foo", + to: "bar", + path: path.rows, + }); + expect(composed.path).toHaveLength(1); + expect(composed.snippets.length).toBeGreaterThanOrEqual(1); + expect(composed.snippets[0]?.source).toContain("bar"); + expect(composed.truncated).toBe(false); + }); +}); + +describe("composeExploreResult", () => { + it("merges neighborhoods for multiple names", () => { + seedCallGraph(); + const r = composeExploreResult({ root: benchDir, names: ["foo", "bar"] }); + expect(r.ok).toBe(true); + if (!r.ok) return; + expect(r.result.names).toEqual(["foo", "bar"]); + expect(r.result.rows.length).toBeGreaterThan(0); + }); +}); + +describe("composeNodeResult", () => { + it("returns show envelope and one-hop neighborhood", () => { + seedCallGraph(); + const r = composeNodeResult({ root: benchDir, name: "foo" }); + expect(r.ok).toBe(true); + if (!r.ok) return; + expect(r.result.center.matches[0]?.name).toBe("foo"); + expect(r.result.neighborhood.some((row) => row.name === "bar")).toBe(true); + expect(r.result.snippets).toEqual([]); + }); + + it("includes snippets when requested", () => { + seedCallGraph(); + const r = composeNodeResult({ + root: benchDir, + name: "foo", + includeSnippets: true, + }); + expect(r.ok).toBe(true); + if (!r.ok) return; + expect(r.result.snippets.length).toBeGreaterThan(0); + }); +}); diff --git a/src/application/trace-engine.ts b/src/application/trace-engine.ts new file mode 100644 index 00000000..69551e4a --- /dev/null +++ b/src/application/trace-engine.ts @@ -0,0 +1,372 @@ +/** + * Shared composers for MCP/HTTP `trace`, `explore`, and `node` — thin wrappers + * over bundled `call-path` / `symbol-neighborhood` recipes plus `show` / snippet reads. + */ + +import { closeDb, openDb } from "../db"; +import { + applySourceCharBudget, + DEFAULT_OUTPUT_CHAR_BUDGET, +} from "./output-budget"; +import { executeQuery } from "./query-engine"; +import { + getQueryRecipeActions, + getQueryRecipeParams, + getQueryRecipeSql, +} from "./query-recipes"; +import { resolveRecipeParams } from "./recipe-params"; +import { + buildShowResult, + buildSnippetResult, + findSymbolsByName, +} from "./show-engine"; +import type { ShowResult, SnippetMatch, SymbolMatch } from "./show-engine"; + +export type TraceFailureKind = "param" | "query" | "internal"; + +export interface CallPathHop { + file_path: string; + caller_name: string; + callee_name: string; + line_start: number; + hop: number; + via: string; +} + +export interface SymbolNeighborhoodRow { + name: string; + kind: string; + file_path: string; + line_start: number; + line_end: number; + signature: string; + edge: string; + depth: number; + via: string; +} + +function executeBundledRecipe(opts: { + recipeId: string; + root: string; + provided: Record; +}): + | { ok: true; rows: Record[] } + | { ok: false; error: string; kind: TraceFailureKind } { + const declared = getQueryRecipeParams(opts.recipeId); + const resolved = resolveRecipeParams({ + recipeId: opts.recipeId, + declared, + provided: opts.provided, + }); + if (!resolved.ok) { + return { ok: false, error: resolved.error, kind: "param" }; + } + + const sql = getQueryRecipeSql(opts.recipeId); + if (sql === undefined) { + return { + ok: false, + error: `codemap: bundled recipe "${opts.recipeId}" missing`, + kind: "internal", + }; + } + + const payload = executeQuery({ + sql, + bindValues: resolved.values, + root: opts.root, + recipeActions: getQueryRecipeActions(opts.recipeId), + }); + + if ( + payload !== null && + typeof payload === "object" && + !Array.isArray(payload) && + "error" in payload + ) { + return { + ok: false, + error: String((payload as { error: string }).error), + kind: "query", + }; + } + + return { ok: true, rows: payload as Record[] }; +} + +export function executeCallPath(opts: { + root: string; + from: string; + to: string; + maxDepth?: number | undefined; + via?: string | undefined; +}): + | { ok: true; rows: CallPathHop[] } + | { ok: false; error: string; kind: TraceFailureKind } { + const provided: Record = { + from: opts.from, + to: opts.to, + }; + if (opts.maxDepth !== undefined) provided.max_depth = opts.maxDepth; + if (opts.via !== undefined) provided.via = opts.via; + + const result = executeBundledRecipe({ + recipeId: "call-path", + root: opts.root, + provided, + }); + if (!result.ok) return result; + return { ok: true, rows: result.rows as unknown as CallPathHop[] }; +} + +export function executeSymbolNeighborhood(opts: { + root: string; + name: string; + depth?: number | undefined; + kind?: string | undefined; +}): + | { ok: true; rows: SymbolNeighborhoodRow[] } + | { ok: false; error: string; kind: TraceFailureKind } { + const provided: Record = { + name: opts.name, + }; + if (opts.depth !== undefined) provided.depth = opts.depth; + if (opts.kind !== undefined) provided.kind = opts.kind; + + const result = executeBundledRecipe({ + recipeId: "symbol-neighborhood", + root: opts.root, + provided, + }); + if (!result.ok) return result; + return { ok: true, rows: result.rows as unknown as SymbolNeighborhoodRow[] }; +} + +function symbolKey(name: string, filePath: string): string { + return `${name}\0${filePath}`; +} + +function isCallHopSnippetEligible(hop: CallPathHop): boolean { + return hop.via === "calls" && hop.line_start > 0; +} + +function snippetsForSymbolMatches(opts: { + db: ReturnType; + matches: SymbolMatch[]; + projectRoot: string; +}): SnippetMatch[] { + return buildSnippetResult({ + db: opts.db, + matches: opts.matches, + projectRoot: opts.projectRoot, + }).matches; +} + +function lookupSymbolInFile( + db: ReturnType, + name: string, + filePath: string, +): SymbolMatch | undefined { + const matches = findSymbolsByName(db, { name, inPath: filePath }); + return matches[0]; +} + +function snippetsForNeighborhoodRows(opts: { + db: ReturnType; + rows: SymbolNeighborhoodRow[]; + projectRoot: string; +}): SnippetMatch[] { + const seen = new Set(); + const matches: SymbolMatch[] = []; + for (const row of opts.rows) { + const key = symbolKey(row.name, row.file_path); + if (seen.has(key)) continue; + seen.add(key); + matches.push({ + name: row.name, + kind: row.kind, + file_path: row.file_path, + line_start: row.line_start, + line_end: row.line_end, + signature: row.signature, + is_exported: 0, + parent_name: null, + visibility: null, + }); + } + return snippetsForSymbolMatches({ + db: opts.db, + matches, + projectRoot: opts.projectRoot, + }); +} + +export interface TraceComposeResult { + from: string; + to: string; + via?: string | undefined; + path: CallPathHop[]; + snippets: SnippetMatch[]; + truncated: boolean; +} + +export function composeTraceResult(opts: { + root: string; + from: string; + to: string; + via?: string | undefined; + path: CallPathHop[]; + budgetChars?: number | undefined; +}): TraceComposeResult { + const budget = opts.budgetChars ?? DEFAULT_OUTPUT_CHAR_BUDGET; + const db = openDb(); + try { + const seen = new Set(); + const matches: SymbolMatch[] = []; + for (const hop of opts.path) { + if (!isCallHopSnippetEligible(hop)) continue; + for (const name of [hop.caller_name, hop.callee_name]) { + const key = symbolKey(name, hop.file_path); + if (seen.has(key)) continue; + seen.add(key); + const match = lookupSymbolInFile(db, name, hop.file_path); + if (match !== undefined) matches.push(match); + } + } + const allSnippets = snippetsForSymbolMatches({ + db, + matches, + projectRoot: opts.root, + }); + const budgeted = applySourceCharBudget(allSnippets, budget); + return { + from: opts.from, + to: opts.to, + via: opts.via, + path: opts.path, + snippets: budgeted.items, + truncated: budgeted.truncated, + }; + } finally { + closeDb(db, { readonly: true }); + } +} + +export interface ExploreComposeResult { + names: string[]; + rows: SymbolNeighborhoodRow[]; + snippets: SnippetMatch[]; + truncated: boolean; +} + +export function composeExploreResult(opts: { + root: string; + names: string[]; + depth?: number | undefined; + kind?: string | undefined; + budgetChars?: number | undefined; +}): + | { ok: true; result: ExploreComposeResult } + | { ok: false; error: string; kind: TraceFailureKind } { + const merged: SymbolNeighborhoodRow[] = []; + const seenRows = new Set(); + for (const name of opts.names) { + const neighborhood = executeSymbolNeighborhood({ + root: opts.root, + name, + depth: opts.depth, + kind: opts.kind, + }); + if (!neighborhood.ok) return neighborhood; + for (const row of neighborhood.rows) { + const key = `${row.name}\0${row.file_path}\0${row.edge}\0${row.depth}\0${row.via}`; + if (seenRows.has(key)) continue; + seenRows.add(key); + merged.push(row); + } + } + + const budget = opts.budgetChars ?? DEFAULT_OUTPUT_CHAR_BUDGET; + const db = openDb(); + try { + const allSnippets = snippetsForNeighborhoodRows({ + db, + rows: merged, + projectRoot: opts.root, + }); + const budgeted = applySourceCharBudget(allSnippets, budget); + return { + ok: true, + result: { + names: opts.names, + rows: merged, + snippets: budgeted.items, + truncated: budgeted.truncated, + }, + }; + } finally { + closeDb(db, { readonly: true }); + } +} + +export interface NodeComposeResult { + center: ShowResult; + neighborhood: SymbolNeighborhoodRow[]; + snippets: SnippetMatch[]; + truncated: boolean; +} + +export function composeNodeResult(opts: { + root: string; + name: string; + kind?: string | undefined; + inPath?: string | undefined; + includeSnippets?: boolean | undefined; + budgetChars?: number | undefined; +}): + | { ok: true; result: NodeComposeResult } + | { ok: false; error: string; kind: TraceFailureKind } { + const neighborhood = executeSymbolNeighborhood({ + root: opts.root, + name: opts.name, + depth: 1, + kind: opts.kind, + }); + if (!neighborhood.ok) return neighborhood; + + const db = openDb(); + try { + const matches = findSymbolsByName(db, { + name: opts.name, + kind: opts.kind, + inPath: opts.inPath, + }); + const center = buildShowResult(matches); + + let snippets: SnippetMatch[] = []; + let truncated = false; + if (opts.includeSnippets === true) { + const budget = opts.budgetChars ?? DEFAULT_OUTPUT_CHAR_BUDGET; + const allSnippets = snippetsForNeighborhoodRows({ + db, + rows: neighborhood.rows, + projectRoot: opts.root, + }); + const budgeted = applySourceCharBudget(allSnippets, budget); + snippets = budgeted.items; + truncated = budgeted.truncated; + } + + return { + ok: true, + result: { + center, + neighborhood: neighborhood.rows, + snippets, + truncated, + }, + }; + } finally { + closeDb(db, { readonly: true }); + } +} diff --git a/templates/agent-content/mcp-instructions.md b/templates/agent-content/mcp-instructions.md index 5a3341c7..12edd108 100644 --- a/templates/agent-content/mcp-instructions.md +++ b/templates/agent-content/mcp-instructions.md @@ -16,6 +16,9 @@ Operational playbook injected into the MCP initialize handshake. Full schema, re | Kind / pattern lookup | **`query_recipe`** | `find-symbol-by-kind` | | Source at symbol | **`snippet`** | same rows as `show` + disk text | | Blast radius | **`impact`** (`target`, `direction`, `via`, `depth`) | `fan-in` for file hubs; symbol call graph via SQL or `impact` | +| Call path + snippets | **`trace`** (`from`, `to`, `via?`, `max_depth?`, `budget_chars?`) | `call-path` | +| Multi-symbol survey | **`explore`** (`names`, `depth?`, `kind?`, `budget_chars?`) | `symbol-neighborhood` (once per name) | +| One-hop symbol card | **`node`** (`name`, `kind?`, `in?`, `include_snippets?`) | `show` + `symbol-neighborhood` with `depth=1` | | Affected tests | **`affected`** (`paths?`, `changed_since?`, `test_glob?`, `max_depth?`) | `affected-tests` (RS-delimit multiple paths in `query_recipe` params) | | CI / SARIF | **`query_recipe`** + `format: "sarif"` | `deprecated-symbols`, `boundary-violations`, … | | Ad-hoc SQL | **`query`** | — | @@ -27,6 +30,7 @@ Operational playbook injected into the MCP initialize handshake. Full schema, re ## Chains - Rename: `find-symbol-definitions` → `find-symbol-references` (both via **`query_recipe`**). +- Call path: **`trace`** (`from`, `to`) or **`query_recipe`** `call-path`; add snippets via **`trace`** / **`node`** / **`explore`** (budget-capped) or **`snippet`** per row. - Refactor risk: `fan-in` + `refactor-risk-ranking`. - Edit path: **`show`** → **`snippet`**; if `stale: true`, line range may have drifted. @@ -39,6 +43,6 @@ Operational playbook injected into the MCP initialize handshake. Full schema, re ## Recipe ids cited here -`find-symbol-definitions`, `find-symbol-by-kind`, `find-symbol-references`, `fan-in`, `affected-tests`, `deprecated-symbols`, `boundary-violations`, `refactor-risk-ranking`. Others: list via **`codemap://recipes`** before **`query_recipe`**. +`find-symbol-definitions`, `find-symbol-by-kind`, `find-symbol-references`, `fan-in`, `call-path`, `symbol-neighborhood`, `affected-tests`, `deprecated-symbols`, `boundary-violations`, `refactor-risk-ranking`. Others: list via **`codemap://recipes`** before **`query_recipe`**. - + diff --git a/templates/agent-content/skill/10-recipes-context.md b/templates/agent-content/skill/10-recipes-context.md index 4fc756d4..df3433d3 100644 --- a/templates/agent-content/skill/10-recipes-context.md +++ b/templates/agent-content/skill/10-recipes-context.md @@ -47,6 +47,9 @@ Each emitted delta carries its own `base` metadata so mixed-baseline audits are - **`show`** — `{name, kind?, in?}`. Exact symbol lookup → `{matches, disambiguation?}`. Fuzzy lookup belongs in `query` with `LIKE`. - **`snippet`** — same shape as `show` but each match also carries `source` (file text) + `stale` / `missing` flags. No reindex side-effects. - **`impact`** — `{target, direction?, via?, depth?, limit?, summary?}`. Symbol/file blast-radius walker (replaces hand-composed `WITH RECURSIVE`). Auto-resolves symbol vs file target; `via` defaults to every backend compatible with the kind. +- **`trace`** — `{from, to, max_depth?, via?, budget_chars?}`. Shortest call path + budget-capped snippets (`call-path` recipe twin). +- **`explore`** — `{names, depth?, kind?, budget_chars?}`. Multi-name neighborhood survey + snippets (`symbol-neighborhood` per name). +- **`node`** — `{name, kind?, in?, include_snippets?, budget_chars?}`. `show` center + depth-1 neighborhood; optional inline snippets. - **`affected`** — `{paths?, changed_since?, test_glob?, max_depth?}`. Reverse-dependency walk from changed files to test paths (same preprocessor as **`codemap affected`** → **`affected-tests`** recipe). Explicit `paths` (including `paths: []` for empty — skips git) wins over git discovery; omit `paths` for working tree vs `changed_since` (default `HEAD`). When both `paths` and `changed_since` are sent, `paths` wins (mirrors CLI positional + `--changed-since`). - **`apply`** — `{recipe, params?, dry_run?, yes?}`. Executes the diff hunks a recipe row produces (`{file_path, line_start, before_pattern, after_pattern}`). **All-or-nothing**: any conflict aborts before any file is written. Over MCP/HTTP `yes: true` is required for the write path; `dry_run` and `yes` are mutually exclusive. From 4e37820c73852b0ae57e03dd35f7214d3e3476ce Mon Sep 17 00:00:00 2001 From: Sutu Sebastian Date: Mon, 25 May 2026 16:46:46 +0300 Subject: [PATCH 2/5] docs: link PR 6 trace tools in delivery tracker --- docs/plans/agent-surface-delivery.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/plans/agent-surface-delivery.md b/docs/plans/agent-surface-delivery.md index 3bf29b40..37684b86 100644 --- a/docs/plans/agent-surface-delivery.md +++ b/docs/plans/agent-surface-delivery.md @@ -13,7 +13,7 @@ | Next action | Detail | | -------------------- | --------------------------------------------------------------------------------------------------- | | **Review / merge** | [#133](https://github.com/stainless-code/codemap/pull/133) — MCP `affected` (perf baseline pending) | -| **Start next** | **PR 6** — MCP trace tools (`trace` / `explore` / `node`) — branch `feat/mcp-trace-tools` open | +| **Open** | [#134](https://github.com/stainless-code/codemap/pull/134) — MCP trace tools (PR 6) | | **Do not start yet** | PR 9 (eval harness) until PR 8 | Update the table below when a PR merges or a new branch opens. @@ -36,11 +36,11 @@ Merge each PR to `main` directly. No long-lived integration branch (`feat/agent- Max **3 parallel tracks** at once. | PR | Plans | Status | Blocked by | Parallel with | -| ----- | ---------------------------------------------------------------------------------------------------------------------------------------------- | ------- | ---------------------------------------------------------------------------------------------------------------------- | --------------------------------- | --- | +| ----- | ---------------------------------------------------------------------------------------------------------------------------------------------- | ------- | ---------------------------------------------------------------------------------------------------------------------- | --------------------------------- | | **3** | [`index-lock-and-error-log`](./index-lock-and-error-log.md) → [`parse-worker-hardening`](./parse-worker-hardening.md) (stack) | merged | [#129](https://github.com/stainless-code/codemap/pull/129), [#130](https://github.com/stainless-code/codemap/pull/130) | 4, 5 | | **4** | Recipe half of [`mcp-trace-explore-tools`](./mcp-trace-explore-tools.md) (`call-path`, `symbol-neighborhood` SQL + tests) | merged | [#131](https://github.com/stainless-code/codemap/pull/131) | 3, 5 | | **5** | [`affected-tests-recipe`](./affected-tests-recipe.md) (+ Phase 2 MCP `affected` in [#133](https://github.com/stainless-code/codemap/pull/133)) | merged | [#132](https://github.com/stainless-code/codemap/pull/132), [#133](https://github.com/stainless-code/codemap/pull/133) | 3, 4 | -| **6** | MCP half of trace (`trace` / `explore` / `node` tools) + update instructions | open | `feat/mcp-trace-tools` | PR 1, PR 4 | — | +| **6** | MCP half of trace (`trace` / `explore` / `node` tools) + update instructions | open | [#134](https://github.com/stainless-code/codemap/pull/134) | PR 1, PR 4 (#133 merge first) | | **7** | [`field-qualified-search`](./field-qualified-search.md) | planned | PR 1 | 4, 5 if `mcp-server.ts` untouched | | **8** | [`agents-init-mcp-wiring`](./agents-init-mcp-wiring.md) | planned | PR 1 | 3–5 | | **9** | [`agent-eval-harness`](./agent-eval-harness.md) | planned | PR 1, PR 8, allowlist | **last P1** | From 7246bb27802f990631191a165bf193ada29dd163 Mon Sep 17 00:00:00 2001 From: Sutu Sebastian Date: Mon, 25 May 2026 16:49:14 +0300 Subject: [PATCH 3/5] docs: mark #133 merged in delivery tracker --- docs/plans/agent-surface-delivery.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/docs/plans/agent-surface-delivery.md b/docs/plans/agent-surface-delivery.md index 37684b86..d7e29ee5 100644 --- a/docs/plans/agent-surface-delivery.md +++ b/docs/plans/agent-surface-delivery.md @@ -10,11 +10,11 @@ ## Quick resume -| Next action | Detail | -| -------------------- | --------------------------------------------------------------------------------------------------- | -| **Review / merge** | [#133](https://github.com/stainless-code/codemap/pull/133) — MCP `affected` (perf baseline pending) | -| **Open** | [#134](https://github.com/stainless-code/codemap/pull/134) — MCP trace tools (PR 6) | -| **Do not start yet** | PR 9 (eval harness) until PR 8 | +| Next action | Detail | +| -------------------- | ----------------------------------------------------------------------------------- | +| **Review / merge** | [#134](https://github.com/stainless-code/codemap/pull/134) — MCP trace tools (PR 6) | +| **Recently merged** | [#133](https://github.com/stainless-code/codemap/pull/133) — MCP `affected` | +| **Do not start yet** | PR 9 (eval harness) until PR 8 | Update the table below when a PR merges or a new branch opens. @@ -40,7 +40,7 @@ Max **3 parallel tracks** at once. | **3** | [`index-lock-and-error-log`](./index-lock-and-error-log.md) → [`parse-worker-hardening`](./parse-worker-hardening.md) (stack) | merged | [#129](https://github.com/stainless-code/codemap/pull/129), [#130](https://github.com/stainless-code/codemap/pull/130) | 4, 5 | | **4** | Recipe half of [`mcp-trace-explore-tools`](./mcp-trace-explore-tools.md) (`call-path`, `symbol-neighborhood` SQL + tests) | merged | [#131](https://github.com/stainless-code/codemap/pull/131) | 3, 5 | | **5** | [`affected-tests-recipe`](./affected-tests-recipe.md) (+ Phase 2 MCP `affected` in [#133](https://github.com/stainless-code/codemap/pull/133)) | merged | [#132](https://github.com/stainless-code/codemap/pull/132), [#133](https://github.com/stainless-code/codemap/pull/133) | 3, 4 | -| **6** | MCP half of trace (`trace` / `explore` / `node` tools) + update instructions | open | [#134](https://github.com/stainless-code/codemap/pull/134) | PR 1, PR 4 (#133 merge first) | +| **6** | MCP half of trace (`trace` / `explore` / `node` tools) + update instructions | open | [#134](https://github.com/stainless-code/codemap/pull/134) | PR 1, PR 4 | | **7** | [`field-qualified-search`](./field-qualified-search.md) | planned | PR 1 | 4, 5 if `mcp-server.ts` untouched | | **8** | [`agents-init-mcp-wiring`](./agents-init-mcp-wiring.md) | planned | PR 1 | 3–5 | | **9** | [`agent-eval-harness`](./agent-eval-harness.md) | planned | PR 1, PR 8, allowlist | **last P1** | From a436f0707e874e81ec16a9394bb4bc50c4902ff4 Mon Sep 17 00:00:00 2001 From: Sutu Sebastian Date: Mon, 25 May 2026 17:04:23 +0300 Subject: [PATCH 4/5] fix(mcp): harden trace/explore/node from PR review Cross-file snippet lookup, homonym scoping via inPath, explore row cap and dedupe, truncation metadata, recency-after-success, and tests/docs. --- docs/architecture.md | 20 +- docs/plans/mcp-trace-explore-tools.md | 43 ++-- docs/roadmap.md | 2 +- src/application/http-server.test.ts | 67 +++++++ src/application/mcp-server.test.ts | 94 +++++++++ src/application/mcp-server.ts | 6 +- src/application/tool-handlers.ts | 2 +- src/application/trace-engine.test.ts | 138 +++++++++++++ src/application/trace-engine.ts | 186 ++++++++++++++++-- templates/agent-content/mcp-instructions.md | 32 +-- .../agent-content/skill/10-recipes-context.md | 6 +- 11 files changed, 521 insertions(+), 75 deletions(-) diff --git a/docs/architecture.md b/docs/architecture.md index 2abf3d89..fc5acb9d 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -16,13 +16,13 @@ A local SQLite database (`.codemap/index.db`) indexes the project tree and store ## Layering -| Layer | Role | -| -------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| **`cli/`** (`bootstrap`, `main`, `cmd-*`) | Parses argv; **dynamic `import()`** loads only the command chunk (`cmd-index`, `cmd-query`, `cmd-agents`) so `--help` / `version` / `agents init` avoid the indexer. | -| **`api.ts`** | Public programmatic surface: `createCodemap()`, `Codemap` (`query`, `index`), re-exports `runCodemapIndex` for advanced use. | -| **`application/`** | Pure transport-agnostic engines + handlers: `run-index.ts` / `index-engine.ts` (orchestration + indexing); `query-engine.ts` (`executeQuery` / `executeQueryBatch`); `audit-engine.ts` (`runAudit` + `resolveAuditBaselines` + `runAuditFromRef` + `makeWorktreeReindex`); `audit-worktree.ts` (sha-keyed cache + atomic populate); `context-engine.ts` (`buildContextEnvelope`); `validate-engine.ts` (`computeValidateRows` + `toProjectRelative`); `show-engine.ts` (lookup + envelope builders); `impact-engine.ts` (`findImpact` — graph blast-radius walker); `affected-engine.ts` (`resolveAffectedChangedPaths` + `executeAffectedTests` — `affected-tests` recipe composer); `apply-engine.ts` (`applyDiffPayload` — substrate-shaped fix executor over the diff-json row contract); `coverage-engine.ts` (`upsertCoverageRows` core + `ingestIstanbul` / `ingestLcov` / `ingestV8` parsers; schema in [§ Schema → coverage](#schema)); `query-recipes.ts` + `recipes-loader.ts` (recipe registry); `output-formatters.ts` (SARIF + GH annotations + Mermaid `flowchart LR` with bounded-input contract); `watcher.ts` (chokidar-backed debounced reindex; pure helpers + injectable backend); `tool-handlers.ts` + `resource-handlers.ts` (transport-agnostic tool / resource handlers shared by MCP + HTTP); `mcp-server.ts` (MCP transport — stdio); `http-server.ts` (HTTP transport — `node:http`). Engines depend on `db.ts` / `runtime.ts`; **never** on `cli/`. | -| **`adapters/`** | `LanguageAdapter` registry; built-ins call `parser.ts` / `css-parser.ts` / `markers.ts` from `parse-worker-core`. | -| **`runtime.ts` / `config.ts` / `db.ts` / …** | Config, SQLite, resolver, workers. | +| Layer | Role | +| -------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| **`cli/`** (`bootstrap`, `main`, `cmd-*`) | Parses argv; **dynamic `import()`** loads only the command chunk (`cmd-index`, `cmd-query`, `cmd-agents`) so `--help` / `version` / `agents init` avoid the indexer. | +| **`api.ts`** | Public programmatic surface: `createCodemap()`, `Codemap` (`query`, `index`), re-exports `runCodemapIndex` for advanced use. | +| **`application/`** | Pure transport-agnostic engines + handlers: `run-index.ts` / `index-engine.ts` (orchestration + indexing); `query-engine.ts` (`executeQuery` / `executeQueryBatch`); `audit-engine.ts` (`runAudit` + `resolveAuditBaselines` + `runAuditFromRef` + `makeWorktreeReindex`); `audit-worktree.ts` (sha-keyed cache + atomic populate); `context-engine.ts` (`buildContextEnvelope`); `validate-engine.ts` (`computeValidateRows` + `toProjectRelative`); `show-engine.ts` (lookup + envelope builders); `impact-engine.ts` (`findImpact` — graph blast-radius walker); `affected-engine.ts` (`resolveAffectedChangedPaths` + `executeAffectedTests` — `affected-tests` recipe composer); `trace-engine.ts` + `output-budget.ts` (`executeCallPath` / `composeTraceResult` / `composeExploreResult` / `composeNodeResult` — call-path + symbol-neighborhood composers); `apply-engine.ts` (`applyDiffPayload` — substrate-shaped fix executor over the diff-json row contract); `coverage-engine.ts` (`upsertCoverageRows` core + `ingestIstanbul` / `ingestLcov` / `ingestV8` parsers; schema in [§ Schema → coverage](#schema)); `query-recipes.ts` + `recipes-loader.ts` (recipe registry); `output-formatters.ts` (SARIF + GH annotations + Mermaid `flowchart LR` with bounded-input contract); `watcher.ts` (chokidar-backed debounced reindex; pure helpers + injectable backend); `tool-handlers.ts` + `resource-handlers.ts` (transport-agnostic tool / resource handlers shared by MCP + HTTP); `mcp-server.ts` (MCP transport — stdio); `http-server.ts` (HTTP transport — `node:http`). Engines depend on `db.ts` / `runtime.ts`; **never** on `cli/`. | +| **`adapters/`** | `LanguageAdapter` registry; built-ins call `parser.ts` / `css-parser.ts` / `markers.ts` from `parse-worker-core`. | +| **`runtime.ts` / `config.ts` / `db.ts` / …** | Config, SQLite, resolver, workers. | `index.ts` is the package entry: re-exports the public API and runs `cli/main` only when executed as the main module (Node/Bun `codemap` binary). @@ -97,7 +97,7 @@ A local SQLite database (`.codemap/index.db`) indexes the project tree and store | `index.ts` | Package entry — re-exports `api` / `config`, runs CLI when main | | `cli/` | CLI — bootstrap argv, lazy command modules, `query` / `validate` / `context` / `agents init` / index modes | | `api.ts` | Programmatic API — `createCodemap`, `Codemap`, `runCodemapIndex` | -| `application/` | Pure transport-agnostic engines (`run-index`, `index-engine`, `query-engine`, `audit-engine`, `context-engine`, `validate-engine`, `show-engine`, `impact-engine`, `affected-engine`, `apply-engine`, `coverage-engine`, `query-recipes`, `recipes-loader`, `mcp-server`, `http-server`, `watcher`) | +| `application/` | Pure transport-agnostic engines (`run-index`, `index-engine`, `query-engine`, `audit-engine`, `context-engine`, `validate-engine`, `show-engine`, `impact-engine`, `affected-engine`, `trace-engine`, `output-budget`, `apply-engine`, `coverage-engine`, `query-recipes`, `recipes-loader`, `mcp-server`, `http-server`, `watcher`) | | `worker-pool.ts` | Parallel parse workers (Bun / Node) | | `db.ts` | SQLite adapter — schema DDL, typed CRUD, connection management | | `parser.ts` | TS/TSX/JS/JSX extraction via `oxc-parser` — symbols (with JSDoc + generics + return types), type members, imports, exports, components, markers | @@ -135,7 +135,7 @@ A local SQLite database (`.codemap/index.db`) indexes the project tree and store **Affected wiring:** **`src/cli/cmd-affected.ts`** (argv — positional paths / `--stdin` / `--changed-since ` / `--params test_glob|max_depth` + `--json`; bootstrap absorbs `--root`/`--config`) + **`src/application/affected-engine.ts`** (engine — `resolveAffectedChangedPaths` + `executeAffectedTests`; pure recipe composer over bundled `affected-tests` SQL). CLI / MCP / HTTP dispatch the same engine via `tool-handlers.ts`'s `handleAffected` (MCP/HTTP) and `runAffectedCmd` (CLI). Path precedence: explicit paths (CLI positional / MCP `paths` array) → CLI `--stdin` → git vs `changed_since` / `HEAD` (`paths: []` on MCP/HTTP skips git). Result envelope: JSON array of `{test_path, impact_depth, actions?}` — file paths only; CI composes the runner command. **`tryRecordRecipeRun("affected-tests")`** lives at the orchestration layer (`handleAffected` + `runAffectedCmd`), not in the engine — same boundary discipline as `query_recipe` (see [§ `recipe_recency`](#recipe_recency--per-recipe-last-run--run-count-user-data-strict-without-rowid)). Recency records only when at least one changed path was resolved and the recipe SQL ran (empty path sets return `[]` without a recency write). -**Trace / explore / node wiring (MCP + HTTP only):** **`src/application/trace-engine.ts`** (engine — `executeCallPath` / `executeSymbolNeighborhood` recipe composers + `composeTraceResult` / `composeExploreResult` / `composeNodeResult` snippet batching) + **`src/application/output-budget.ts`** (`applySourceCharBudget`, default 15k chars). MCP/HTTP dispatch via `tool-handlers.ts`'s `handleTrace` / `handleExplore` / `handleNode`. **`trace`** → `call-path` recipe + disk snippets per hop; **`explore`** → `symbol-neighborhood` once per `names[]` entry, merged rows + budget-capped snippets; **`node`** → `show` center (`findSymbolsByName` + `buildShowResult`) + depth-1 neighborhood + optional snippets. Recipe twins remain the Moat A fallback (`query_recipe call-path`, `query_recipe symbol-neighborhood`). **`tryRecordRecipeRun`** at orchestration only (`call-path` on trace success; `symbol-neighborhood` on explore/node success). +**Trace / explore / node wiring (MCP + HTTP only):** **`src/application/trace-engine.ts`** (engine — `executeCallPath` / `executeSymbolNeighborhood` recipe composers + `composeTraceResult` / `composeExploreResult` / `composeNodeResult` snippet batching) + **`src/application/output-budget.ts`** (`applySourceCharBudget`, default 15k chars on snippet `source` text). MCP/HTTP dispatch via `tool-handlers.ts`'s `handleTrace` / `handleExplore` / `handleNode`. **`trace`** → `call-path` recipe + disk snippets per hop (cross-file symbol lookup); **`explore`** → deduped `symbol-neighborhood` per `names[]` entry, row cap 500, budget-capped snippets; **`node`** → `show` center + scoped depth-1 neighborhood (filters to center instance when unique) + optional center+neighbor snippets. `truncated` is true when snippet budget and/or explore row cap hit (`truncation.snippets` / `truncation.rows`). Recipe twins remain the Moat A fallback (`query_recipe call-path`, `query_recipe symbol-neighborhood`). **`tryRecordRecipeRun`** at orchestration only (`call-path` on trace success; `symbol-neighborhood` on explore/node success). **Apply wiring:** **`src/cli/cmd-apply.ts`** (argv — `` + `--params` + `--dry-run` + `--yes` + `--json`; bootstrap absorbs `--root`/`--config`) + **`src/application/apply-engine.ts`** (engine — `applyDiffPayload({rows, projectRoot, dryRun})`). Pure transport-agnostic substrate-shaped fix executor: consumes the existing `--format diff-json` row contract from any recipe (`{file_path, line_start, before_pattern, after_pattern}`), validates each row against current disk, and either previews (dry-run) or writes (apply). CLI / MCP / HTTP all dispatch the same engine via `tool-handlers.ts`'s `handleApply`. **Phase 1** (always) resolves the project root via `path.resolve(projectRoot)` once, then for each row: rejects absolute `file_path` inputs and any candidate whose `path.resolve(resolvedRoot, file_path)` lands outside `resolvedRoot` (conflict `path escapes project root` — guards CLI + MCP + HTTP write paths against `../escape.ts`-style traversal); rejects duplicate `(file_path, line_start)` tuples (conflict `duplicate edit on same line` — without this, two phase-1-passing rows targeting the same line would split the run mid-phase-2 because the first replace invalidates the second's substring assertion, leaving Q2 (c) cross-file partial state). Reads each file at most once into `sourceCache`, splits on `/\r?\n/` for conflict reporting, checks `actual.includes(before_pattern)` (substring match — mirrors `buildDiffJson`'s contract; `rename-preview` emits `before_pattern = old_name` as the bare identifier, so whole-line exact match would conflict every time). Conflicts collect five reasons (`file missing` / `line out of range` / `line content drifted` / `path escapes project root` / `duplicate edit on same line`) — Q3 scan-and-collect, not fail-fast. **Phase 2** (gated on `!dryRun && conflicts.length === 0`) re-splits the cached source on raw `"\n"` (preserves CRLF as trailing `\r` per line; rejoining with `"\n"` round-trips losslessly), applies each file's edits in descending line order via `actual.replace(before, after)` with `$`-pre-escape (`replace(/\$/g, "$$$$")` — matches `buildDiffJson`'s GetSubstitution defence so identifiers like `$inject` round-trip safely), writes to a sibling temp path (`.codemap-apply-.tmp`), then `renameSync` into place — POSIX-atomic per file; concurrent readers see either pre-rename or post-rename content, never a torn write. **Q2 (c) all-or-nothing (semantic)**: any phase-1 conflict aborts phase 2 entirely before any file is touched. Phase-2 I/O failures (`writeFileSync` / `renameSync`) are NOT transactional across files — per-file atomicity holds (temp + rename), but a crash on file N leaves files `1..N-1` already renamed with no rollback; cross-file rollback would require pre-write backups + restore-on-throw and is deferred to a future PR. **Q6 gate**: TTY no `--yes` → phase-1 preview + `Proceed? [y/N]` prompt on stderr (default-N, `node:readline/promises`); TTY `--yes` → no prompt; non-TTY (CI / agents / MCP) without `--yes`/`--dry-run` rejected with stderr message. `--dry-run` + `--yes` mutually exclusive (parse-time error). MCP/HTTP transports (`handleApply`) require `yes: true` for the write path — there's no prompt to fall back on; `dry_run + yes` rejected as mutually exclusive. Result envelope (Q5; identical across modes): `{mode: 'dry-run'|'apply', applied: bool, files: [{file_path, rows_applied, warnings?}], conflicts: [{file_path, line_start, before_pattern, actual_at_line, reason}], summary: {files, files_modified, rows, rows_applied, conflicts, files_with_conflicts}}`. `applied: true` only when `mode === 'apply'` AND zero conflicts AND at least one row applied. Q7 idempotency: re-running on already-applied code reports a `line content drifted` conflict with `actual_at_line` showing the post-rename content; the user reads it and re-runs `codemap` to refresh the index → next run produces 0 rows (recipe finds nothing to rename) → vacuous clean apply. **Same-line ambiguity caveat (documented limitation):** `actual.replace(before_pattern, after_pattern)` rewrites only the **first** occurrence on the line. When `before_pattern` appears twice (e.g. `const foo = foo();` with `before = "foo"`) only the leftmost is replaced; the engine still reports `applied: true`. This mirrors `buildDiffJson`'s formatter contract verbatim — recipe authors who hit it normalise their SQL to emit a more specific pattern, or accept it (the formatter's `--format diff` preview shows the same shape). Promotion path: tighten phase-1 to conflict on ambiguity in a future PR if real users complain, but only alongside the formatter so preview and execution stay in lockstep. SARIF / annotations not supported (write action, not findings). TOCTOU: phase-1 reads through `sourceCache`; phase-2 transforms the cached source and writes — the gap between read and rename is a deliberate v1 simplification (apply isn't adversarial). Per Q10, only `cli/cmd-apply.ts` + `application/tool-handlers.ts` (+ the test files) may import `apply-engine.ts` for production execution — re-runnable forbidden-edge query at [§ Boundary verification — apply write path](#boundary-verification--apply-write-path). @@ -619,7 +619,7 @@ Bundled recipes consuming the table — `untested-and-dead`, `files-by-coverage` Tracks `last_run_at` (epoch ms) + `run_count` per recipe id so agent hosts can rank live recipes ahead of historic ones. Surfaces inline on `--recipes-json` and the matching `codemap://recipes` / `codemap://recipes/{id}` MCP resources (live read every call — the resource cache was dropped to avoid freezing recency at first-read for the server-process lifetime). Same lifecycle posture as `query_baselines` / `coverage`: **intentionally absent from `dropAll()`** so `--full` and `SCHEMA_VERSION` rebuilds preserve user-activity history. Local-only — no upload primitive ever ships (resists telemetry-creep PRs by construction). -Three write sites call `tryRecordRecipeRun` (the failure-isolated wrapper around `recordRecipeRun`) from `application/recipe-recency.ts`: `handleQueryRecipe` in `application/tool-handlers.ts` (covers MCP + HTTP for generic recipes), `handleAffected` in the same module (MCP + HTTP for `affected-tests`), and the CLI paths `runQueryCmd` in `cli/cmd-query.ts` + `runAffectedCmd` in `cli/cmd-affected.ts` (each keys success locally — `runQueryCmd`'s finally-block uses a `recipeQuerySucceeded` flag, NOT `process.exitCode`, so `--ci`'s deliberate exit-1-on-findings is recognised as success). Counts only successful runs; recency-write failures are swallowed with a stderr `[recency] write failed: ` warning so they NEVER block the recipe response. The 90-day rolling window is enforced eagerly on the write path (single indexed `DELETE` inside `recordRecipeRun` before the upsert); reads filter at SELECT time (`WHERE last_run_at >= cutoff`) and never mutate the DB so the catalog stays side-effect free for `--recipes-json` and the MCP `codemap://recipes` resources. +Three write sites call `tryRecordRecipeRun` (the failure-isolated wrapper around `recordRecipeRun`) from `application/recipe-recency.ts`: `handleQueryRecipe` in `application/tool-handlers.ts` (covers MCP + HTTP for generic recipes), `handleAffected` in the same module (MCP + HTTP for `affected-tests`), `handleTrace` / `handleExplore` / `handleNode` (MCP + HTTP for bundled `call-path` / `symbol-neighborhood`), and the CLI paths `runQueryCmd` in `cli/cmd-query.ts` + `runAffectedCmd` in `cli/cmd-affected.ts` (each keys success locally — `runQueryCmd`'s finally-block uses a `recipeQuerySucceeded` flag, NOT `process.exitCode`, so `--ci`'s deliberate exit-1-on-findings is recognised as success). Counts only successful runs; recency-write failures are swallowed with a stderr `[recency] write failed: ` warning so they NEVER block the recipe response. The 90-day rolling window is enforced eagerly on the write path (single indexed `DELETE` inside `recordRecipeRun` before the upsert); reads filter at SELECT time (`WHERE last_run_at >= cutoff`) and never mutate the DB so the catalog stays side-effect free for `--recipes-json` and the MCP `codemap://recipes` resources. Default ON; opt-out via `.codemap/config` `recipeRecency: false` (short-circuits before any DB write — no rows ever land). `recipe_id` is loose — matches bundled or project-recipe ids (no `recipes` SQLite table to FK against; project-shadow rows share the bundled row, since only one version is ever reachable per id). diff --git a/docs/plans/mcp-trace-explore-tools.md b/docs/plans/mcp-trace-explore-tools.md index 319e0129..f22b2f9d 100644 --- a/docs/plans/mcp-trace-explore-tools.md +++ b/docs/plans/mcp-trace-explore-tools.md @@ -1,10 +1,12 @@ # MCP trace & explore tools — plan -> **Status:** open · **Priority:** P1 · **Effort:** M (~2 weeks) +> **Status:** shipped · **Priority:** P1 · **Effort:** M (~2 weeks) > > **Motivator:** Agents often need call-path and multi-symbol survey answers in one round-trip. Codemap has `impact` (radius walk) and `snippet` but no shortest-path or budget-capped multi-file survey. MCP wrappers must not erode Moat A — every wrapper ships with a recipe twin. > > **Roadmap:** [§ Backlog — Agent surface & ops](./agent-surface-and-ops.md#p1) · related [call-path-type-hierarchy-recipes](./call-path-type-hierarchy-recipes.md) +> +> **Shipped:** recipes [#131](https://github.com/stainless-code/codemap/pull/131); MCP tools [#134](https://github.com/stainless-code/codemap/pull/134) --- @@ -14,12 +16,12 @@ | --- | ---------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------ | | L.1 | **Recipe twins required** before MCP tools: `call-path`, `symbol-neighborhood` (bundled SQL). | [Moat A](../roadmap.md#moats-load-bearing) | | L.2 | MCP tools **`trace`**, **`explore`**, **`node`** are thin composers over recipes + `snippet` + existing engines — not opaque graph APIs. | Moat A | -| L.3 | **Output budgets** — cap total response chars (e.g. 15k); truncate with explicit `truncated: true` in JSON. | Agent context economics | -| L.4 | No NL task parsing — `trace` takes `from` / `to` symbol names; `explore` takes symbol list or recipe result rows. | [Floor — No LLM in the box](../roadmap.md#floors-v1-product-shape) | +| L.3 | **Output budgets** — cap snippet `source` chars (default 15k) + explore row cap (500); `truncated: true` with `truncation` detail. | Agent context economics | +| L.4 | No NL task parsing — `trace` takes `from` / `to` symbol names; `explore` takes symbol name list. | [Floor — No LLM in the box](../roadmap.md#floors-v1-product-shape) | --- -## Recipe specs (ship first) +## Recipe specs (shipped #131) ### `call-path` @@ -35,38 +37,27 @@ --- -## MCP tool specs (ship second) - -| Tool | Composes | -| --------- | ---------------------------------------------------------------------------- | -| `trace` | `query_recipe call-path` + `snippet` for each hop | -| `explore` | `query_recipe symbol-neighborhood` (multi-name) + `snippet` with char budget | -| `node` | `show` + one-hop `symbol-neighborhood` + optional inline snippets | - -Register in `mcp-server.ts`; document chains in [mcp-server-instructions](./mcp-server-instructions.md). - ---- +## MCP tool specs (shipped #134) -## Implementation steps +| Tool | Composes | +| --------- | --------------------------------------------------------------------------------- | +| `trace` | `query_recipe call-path` + cross-file `snippet` for hop symbols | +| `explore` | `query_recipe symbol-neighborhood` (deduped multi-name) + snippet budget | +| `node` | `show` + scoped one-hop `symbol-neighborhood` + optional center+neighbor snippets | -1. Add `templates/recipes/call-path.sql` + `.md` frontmatter -2. Add `templates/recipes/symbol-neighborhood.sql` + `.md` -3. Golden-query tests for both recipes -4. Implement MCP handlers in `tool-handlers.ts` (or dedicated module) -5. Output budget helper shared by explore/trace -6. Update agent-content skill with SQL equivalents +Register in `mcp-server.ts` + `http-server.ts`; document chains in [mcp-instructions](../templates/agent-content/mcp-instructions.md). --- ## Acceptance -- [ ] `codemap query --recipe call-path --params from=foo,to=bar` works -- [ ] MCP `trace` returns same path + snippets, respects budget -- [ ] Instructions document recipe-first fallback +- [x] `codemap query --recipe call-path --params from=foo,to=bar` works ([#131](https://github.com/stainless-code/codemap/pull/131)) +- [x] MCP/HTTP `trace` returns path + snippets, respects budget ([#134](https://github.com/stainless-code/codemap/pull/134)) +- [x] Instructions document recipe-first fallback ([#134](https://github.com/stainless-code/codemap/pull/134)) --- ## Dependencies -- [mcp-server-instructions](./mcp-server-instructions.md) should land first or in same PR +- [mcp-server-instructions](./mcp-server-instructions.md) — landed [#126](https://github.com/stainless-code/codemap/pull/126) - [call-path-type-hierarchy-recipes](./call-path-type-hierarchy-recipes.md) may extend CTE patterns later diff --git a/docs/roadmap.md b/docs/roadmap.md index 74abe4e5..601c122d 100644 --- a/docs/roadmap.md +++ b/docs/roadmap.md @@ -65,7 +65,7 @@ Prioritized agent & indexing ops queue (2026-05). Index: [`plans/agent-surface-a **P1 — medium** -- [ ] **MCP trace / explore / node** — recipe twins + thin MCP composers. Plan: [`plans/mcp-trace-explore-tools.md`](./plans/mcp-trace-explore-tools.md). Effort: M. +- [x] **MCP trace / explore / node** — recipe twins + thin MCP composers. Plan: [`plans/mcp-trace-explore-tools.md`](./plans/mcp-trace-explore-tools.md). [#134](https://github.com/stainless-code/codemap/pull/134). Effort: M. - [ ] **Agents init MCP wiring** — `agents init --mcp` + permissions. Plan: [`plans/agents-init-mcp-wiring.md`](./plans/agents-init-mcp-wiring.md). Effort: M. - [x] **Affected tests recipe** — dep-graph test selection + stdin + MCP `affected` tool. Plan: [`plans/affected-tests-recipe.md`](./plans/affected-tests-recipe.md). Shipped #132 + #133. - [ ] **Index lock + error log** — cross-process lock, `unlock`, `errors.log`. Plan: [`plans/index-lock-and-error-log.md`](./plans/index-lock-and-error-log.md). Effort: M. diff --git a/src/application/http-server.test.ts b/src/application/http-server.test.ts index 458507b7..868bf6d3 100644 --- a/src/application/http-server.test.ts +++ b/src/application/http-server.test.ts @@ -469,6 +469,58 @@ describe("http-server — POST /tool/{other tools}", () => { ).toBe(true); }); + it("trace with non-integer max_depth → 400 (Zod rejects)", async () => { + serverHandle = await startServer(); + const r = await postTool(serverHandle.port, "trace", { + from: "a", + to: "b", + max_depth: 1.5, + }); + expect(r.status).toBe(400); + expect(r.json.error).toContain('"trace"'); + }); + + it("explore with empty names → 400 (Zod rejects)", async () => { + serverHandle = await startServer(); + const r = await postTool(serverHandle.port, "explore", { names: [] }); + expect(r.status).toBe(400); + expect(r.json.error).toContain('"explore"'); + }); + + it("trace sets truncated when budget_chars is tiny", async () => { + seedTraceGraph(); + serverHandle = await startServer(); + const r = await postTool(serverHandle.port, "trace", { + from: "alpha", + to: "beta", + budget_chars: 1, + }); + expect(r.status).toBe(200); + expect(r.json.truncated).toBe(true); + expect(r.json.truncation?.snippets).toBe(true); + }); + + it("records recipe recency after trace", async () => { + seedTraceGraph(); + serverHandle = await startServer(); + const r = await postTool(serverHandle.port, "trace", { + from: "alpha", + to: "beta", + }); + expect(r.status).toBe(200); + const db = openDb(); + try { + const row = db + .query<{ run_count: number }>( + "SELECT run_count FROM recipe_recency WHERE recipe_id = 'call-path'", + ) + .get(); + expect(row?.run_count).toBeGreaterThanOrEqual(1); + } finally { + closeDb(db); + } + }); + it("list_baselines returns array (empty when none saved)", async () => { serverHandle = await startServer(); const r = await postTool(serverHandle.port, "list_baselines", {}); @@ -646,6 +698,21 @@ describe("http-server — Zod input validation at HTTP boundary", () => { }); expect(r.status).toBe(400); }); + + it("trace without from → 400 with structured error", async () => { + serverHandle = await startServer(); + const r = await postTool(serverHandle.port, "trace", { to: "bar" }); + expect(r.status).toBe(400); + expect(r.json.error).toContain('"trace"'); + expect(r.json.error).toContain("from"); + }); + + it("node with name=number → 400 (not deep handler crash)", async () => { + serverHandle = await startServer(); + const r = await postTool(serverHandle.port, "node", { name: 1 }); + expect(r.status).toBe(400); + expect(r.json.error).toContain("name"); + }); }); describe("http-server — GET /resources", () => { diff --git a/src/application/mcp-server.test.ts b/src/application/mcp-server.test.ts index 0d0c913e..b014dec7 100644 --- a/src/application/mcp-server.test.ts +++ b/src/application/mcp-server.test.ts @@ -1518,4 +1518,98 @@ describe("MCP server — trace / explore / node tools", () => { await server.close(); } }); + + it("trace returns isError on non-integer max_depth (Zod rejects)", async () => { + const { client, server } = await makeClient(); + try { + const r = await client.callTool({ + name: "trace", + arguments: { from: "a", to: "b", max_depth: 1.5 }, + }); + expect((r as { isError?: boolean }).isError).toBe(true); + } finally { + await server.close(); + } + }); + + it("explore returns isError on empty names array", async () => { + const { client, server } = await makeClient(); + try { + const r = await client.callTool({ + name: "explore", + arguments: { names: [] }, + }); + expect((r as { isError?: boolean }).isError).toBe(true); + } finally { + await server.close(); + } + }); + + it("trace sets truncated when budget_chars is tiny", async () => { + seedTraceGraph(); + const { client, server } = await makeClient(); + try { + const r = await client.callTool({ + name: "trace", + arguments: { from: "foo", to: "bar", budget_chars: 1 }, + }); + const json = readJson(r) as { + truncated: boolean; + truncation?: { snippets?: boolean }; + }; + expect(json.truncated).toBe(true); + expect(json.truncation?.snippets).toBe(true); + } finally { + await server.close(); + } + }); + + it("records recipe recency after trace and explore", async () => { + seedTraceGraph(); + const { client, server } = await makeClient(); + try { + await client.callTool({ + name: "trace", + arguments: { from: "foo", to: "bar" }, + }); + await client.callTool({ + name: "explore", + arguments: { names: ["foo"] }, + }); + const db = openDb(); + try { + const callPath = db + .query<{ run_count: number }>( + "SELECT run_count FROM recipe_recency WHERE recipe_id = 'call-path'", + ) + .get(); + const neighborhood = db + .query<{ run_count: number }>( + "SELECT run_count FROM recipe_recency WHERE recipe_id = 'symbol-neighborhood'", + ) + .get(); + expect(callPath?.run_count).toBeGreaterThanOrEqual(1); + expect(neighborhood?.run_count).toBeGreaterThanOrEqual(1); + } finally { + closeDb(db); + } + } finally { + await server.close(); + } + }); + + it("respects CODEMAP_MCP_TOOLS allowlist for trace tools", async () => { + const { client, server } = await makeClient({ + CODEMAP_MCP_TOOLS: "query,trace", + }); + try { + const tools = await client.listTools(); + const names = tools.tools.map((t) => t.name); + expect(names).toContain("trace"); + expect(names).not.toContain("explore"); + expect(names).not.toContain("node"); + } finally { + await server.close(); + } + }); }); diff --git a/src/application/mcp-server.ts b/src/application/mcp-server.ts index d8952b5b..ed3bddb5 100644 --- a/src/application/mcp-server.ts +++ b/src/application/mcp-server.ts @@ -326,7 +326,7 @@ function registerTraceTool(server: McpServer, opts: ServerOpts): void { "trace", { description: - "Shortest call path between two symbols plus budget-capped snippets. Composes `call-path` recipe + disk reads. Args: from, to (symbol names), max_depth (optional), via (calls|dependencies|all), budget_chars (default 15000). Returns {from, to, via?, path: [{file_path, caller_name, callee_name, line_start, hop, via}], snippets: [{name, file_path, source, stale, missing, ...}], truncated}. Fall back to `query_recipe` with recipe call-path when unsure.", + "Shortest call path between two symbols plus budget-capped snippets. Composes `call-path` recipe + disk reads (cross-file callee lookup). Args: from, to, max_depth?, via (calls|dependencies|all), budget_chars (default 15000, snippet source text only). Returns {from, to, via?, path, snippets, truncated, truncation?, snippets_skipped_reason?}. `truncated` is true when snippet budget or explore row cap hit; dependency hops omit auto-snippets. Fall back to `query_recipe` call-path when unsure.", inputSchema: traceArgsSchema, }, (args) => wrapToolResult(handleTrace(args, opts.root)), @@ -338,7 +338,7 @@ function registerExploreTool(server: McpServer, opts: ServerOpts): void { "explore", { description: - "Multi-symbol neighborhood survey with budget-capped snippets. Composes `symbol-neighborhood` (once per name) + disk reads. Args: names (non-empty array), depth (optional hop budget), kind (optional filter), budget_chars (default 15000). Returns {names, rows: [...], snippets: [...], truncated}. Fall back to `query_recipe` with recipe symbol-neighborhood.", + "Multi-symbol neighborhood survey with budget-capped snippets. Composes `symbol-neighborhood` (once per deduped name) + disk reads. Args: names (non-empty array), depth?, kind?, budget_chars (default 15000, snippet source only). Returns {names, rows, snippets, truncated, truncation?} — `truncation.rows` when row cap (500) hit, `truncation.snippets` when budget hit. Fall back to `query_recipe` symbol-neighborhood.", inputSchema: exploreArgsSchema, }, (args) => wrapToolResult(handleExplore(args, opts.root)), @@ -350,7 +350,7 @@ function registerNodeTool(server: McpServer, opts: ServerOpts): void { "node", { description: - "One-hop symbol survey: `show` center match + depth-1 `symbol-neighborhood` + optional inline snippets. Args: name, kind?, in? (path filter), include_snippets (default false), budget_chars (default 15000 when snippets enabled). Returns {center: {matches, disambiguation?}, neighborhood: [...], snippets: [...], truncated}.", + "One-hop symbol survey: `show` center + scoped depth-1 `symbol-neighborhood` + optional inline snippets. When center is unique (`in` or single match), neighborhood filters to that instance's connected files. Args: name, kind?, in?, include_snippets (default false), budget_chars? (default 15000 when snippets enabled; snippet source only). Returns {center, neighborhood, snippets, truncated, truncation?}.", inputSchema: nodeArgsSchema, }, (args) => wrapToolResult(handleNode(args, opts.root)), diff --git a/src/application/tool-handlers.ts b/src/application/tool-handlers.ts index 68e4e4e3..6ed312e1 100644 --- a/src/application/tool-handlers.ts +++ b/src/application/tool-handlers.ts @@ -863,7 +863,6 @@ export function handleTrace(args: TraceArgs, root: string): ToolResult { pathResult.kind === "internal" ? 500 : undefined, ); } - tryRecordRecipeRun("call-path"); const payload = composeTraceResult({ root, from: args.from, @@ -872,6 +871,7 @@ export function handleTrace(args: TraceArgs, root: string): ToolResult { path: pathResult.rows, budgetChars: args.budget_chars, }); + tryRecordRecipeRun("call-path"); return ok(payload); } catch (e) { return err(e instanceof Error ? e.message : String(e), 500); diff --git a/src/application/trace-engine.test.ts b/src/application/trace-engine.test.ts index d9df6c15..a3a84af2 100644 --- a/src/application/trace-engine.test.ts +++ b/src/application/trace-engine.test.ts @@ -10,6 +10,7 @@ import { composeExploreResult, composeNodeResult, composeTraceResult, + dedupeNames, executeCallPath, executeSymbolNeighborhood, } from "./trace-engine"; @@ -125,6 +126,40 @@ describe("composeExploreResult", () => { }); }); +function seedHomonymGraph() { + writeFileSync( + join(benchDir, "src", "a.ts"), + "export function helper() {\n return onlyA();\n}\nfunction onlyA() {\n return 1;\n}\n", + ); + writeFileSync( + join(benchDir, "src", "b.ts"), + "export function helper() {\n return onlyB();\n}\nfunction onlyB() {\n return 2;\n}\n", + ); + const db = openDb(); + try { + createTables(db); + db.run( + `INSERT INTO files (path, content_hash, size, line_count, language, last_modified, indexed_at) + VALUES ('src/a.ts', 'h1', 100, 6, 'typescript', 1, 1), + ('src/b.ts', 'h2', 100, 6, 'typescript', 1, 1)`, + ); + db.run( + `INSERT INTO symbols (name, kind, file_path, line_start, line_end, signature, is_exported, parent_name, visibility) + VALUES ('helper', 'function', 'src/a.ts', 1, 3, 'helper()', 1, NULL, 'export'), + ('onlyA', 'function', 'src/a.ts', 4, 6, 'onlyA()', 0, NULL, NULL), + ('helper', 'function', 'src/b.ts', 1, 3, 'helper()', 1, NULL, 'export'), + ('onlyB', 'function', 'src/b.ts', 4, 6, 'onlyB()', 0, NULL, NULL)`, + ); + db.run( + `INSERT INTO calls (file_path, caller_name, caller_scope, callee_name, line_start, column_start, column_end) + VALUES ('src/a.ts', 'helper', 'helper', 'onlyA', 2, 0, 0), + ('src/b.ts', 'helper', 'helper', 'onlyB', 2, 0, 0)`, + ); + } finally { + closeDb(db); + } +} + describe("composeNodeResult", () => { it("returns show envelope and one-hop neighborhood", () => { seedCallGraph(); @@ -146,5 +181,108 @@ describe("composeNodeResult", () => { expect(r.ok).toBe(true); if (!r.ok) return; expect(r.result.snippets.length).toBeGreaterThan(0); + expect(r.result.snippets.some((s) => s.name === "foo")).toBe(true); + }); + + it("scopes neighborhood to inPath when center is unique", () => { + seedHomonymGraph(); + const r = composeNodeResult({ + root: benchDir, + name: "helper", + inPath: "src/a.ts", + }); + expect(r.ok).toBe(true); + if (!r.ok) return; + expect(r.result.center.matches).toHaveLength(1); + expect(r.result.center.matches[0]?.file_path).toBe("src/a.ts"); + expect(r.result.neighborhood.some((row) => row.name === "onlyA")).toBe( + true, + ); + expect(r.result.neighborhood.some((row) => row.name === "onlyB")).toBe( + false, + ); + }); +}); + +function seedCrossFileCallGraph() { + writeFileSync( + join(benchDir, "src", "a.ts"), + "import { bar } from './b';\nexport function foo() {\n return bar();\n}\n", + ); + writeFileSync( + join(benchDir, "src", "b.ts"), + "export function bar() {\n return 1;\n}\n", + ); + const db = openDb(); + try { + createTables(db); + db.run( + `INSERT INTO files (path, content_hash, size, line_count, language, last_modified, indexed_at) + VALUES ('src/a.ts', 'h1', 100, 4, 'typescript', 1, 1), + ('src/b.ts', 'h2', 100, 3, 'typescript', 1, 1)`, + ); + db.run( + `INSERT INTO symbols (name, kind, file_path, line_start, line_end, signature, is_exported, parent_name, visibility) + VALUES ('foo', 'function', 'src/a.ts', 2, 4, 'foo()', 1, NULL, 'export'), + ('bar', 'function', 'src/b.ts', 1, 3, 'bar()', 1, NULL, 'export')`, + ); + db.run( + `INSERT INTO calls (file_path, caller_name, caller_scope, callee_name, line_start, column_start, column_end) + VALUES ('src/a.ts', 'foo', 'foo', 'bar', 3, 0, 0)`, + ); + } finally { + closeDb(db); + } +} + +describe("dedupeNames", () => { + it("preserves order and drops duplicates", () => { + expect(dedupeNames(["foo", "bar", "foo"])).toEqual(["foo", "bar"]); + }); +}); + +describe("composeTraceResult cross-file", () => { + it("resolves callee snippets from another file", () => { + seedCrossFileCallGraph(); + const path = executeCallPath({ root: benchDir, from: "foo", to: "bar" }); + expect(path.ok).toBe(true); + if (!path.ok) return; + const composed = composeTraceResult({ + root: benchDir, + from: "foo", + to: "bar", + path: path.rows, + }); + expect( + composed.snippets.some( + (s) => s.name === "bar" && s.file_path === "src/b.ts", + ), + ).toBe(true); + }); + + it("sets truncated when snippet budget is tiny", () => { + seedCallGraph(); + const path = executeCallPath({ root: benchDir, from: "foo", to: "bar" }); + expect(path.ok).toBe(true); + if (!path.ok) return; + const composed = composeTraceResult({ + root: benchDir, + from: "foo", + to: "bar", + path: path.rows, + budgetChars: 1, + }); + expect(composed.truncated).toBe(true); + expect(composed.truncation?.snippets).toBe(true); + }); +}); + +describe("composeExploreResult dedupe", () => { + it("dedupes duplicate seed names", () => { + seedCallGraph(); + const r = composeExploreResult({ root: benchDir, names: ["foo", "foo"] }); + expect(r.ok).toBe(true); + if (!r.ok) return; + expect(r.result.names).toEqual(["foo"]); }); }); diff --git a/src/application/trace-engine.ts b/src/application/trace-engine.ts index 69551e4a..951df51c 100644 --- a/src/application/trace-engine.ts +++ b/src/application/trace-engine.ts @@ -3,6 +3,7 @@ * over bundled `call-path` / `symbol-neighborhood` recipes plus `show` / snippet reads. */ +import type { CodemapDatabase } from "../db"; import { closeDb, openDb } from "../db"; import { applySourceCharBudget, @@ -24,6 +25,9 @@ import type { ShowResult, SnippetMatch, SymbolMatch } from "./show-engine"; export type TraceFailureKind = "param" | "query" | "internal"; +/** Default row cap for explore before `rows_truncated` (structural payload guard). */ +export const DEFAULT_EXPLORE_ROW_LIMIT = 500; + export interface CallPathHop { file_path: string; caller_name: string; @@ -45,6 +49,11 @@ export interface SymbolNeighborhoodRow { via: string; } +export interface TraceTruncation { + snippets?: boolean; + rows?: boolean; +} + function executeBundledRecipe(opts: { recipeId: string; root: string; @@ -142,6 +151,18 @@ export function executeSymbolNeighborhood(opts: { return { ok: true, rows: result.rows as unknown as SymbolNeighborhoodRow[] }; } +/** Preserve first-occurrence order; drop duplicate seed names. */ +export function dedupeNames(names: Iterable): string[] { + const seen = new Set(); + const out: string[] = []; + for (const name of names) { + if (seen.has(name)) continue; + seen.add(name); + out.push(name); + } + return out; +} + function symbolKey(name: string, filePath: string): string { return `${name}\0${filePath}`; } @@ -151,10 +172,11 @@ function isCallHopSnippetEligible(hop: CallPathHop): boolean { } function snippetsForSymbolMatches(opts: { - db: ReturnType; + db: CodemapDatabase; matches: SymbolMatch[]; projectRoot: string; }): SnippetMatch[] { + if (opts.matches.length === 0) return []; return buildSnippetResult({ db: opts.db, matches: opts.matches, @@ -163,7 +185,7 @@ function snippetsForSymbolMatches(opts: { } function lookupSymbolInFile( - db: ReturnType, + db: CodemapDatabase, name: string, filePath: string, ): SymbolMatch | undefined { @@ -171,8 +193,21 @@ function lookupSymbolInFile( return matches[0]; } +/** Prefer `preferredFile`, then fall back to global name lookup (cross-file callees). */ +function lookupSymbolForName( + db: CodemapDatabase, + name: string, + preferredFile?: string, +): SymbolMatch | undefined { + if (preferredFile !== undefined && preferredFile.length > 0) { + const local = lookupSymbolInFile(db, name, preferredFile); + if (local !== undefined) return local; + } + return findSymbolsByName(db, { name })[0]; +} + function snippetsForNeighborhoodRows(opts: { - db: ReturnType; + db: CodemapDatabase; rows: SymbolNeighborhoodRow[]; projectRoot: string; }): SnippetMatch[] { @@ -201,6 +236,91 @@ function snippetsForNeighborhoodRows(opts: { }); } +function mergeSnippetMatches( + primary: SnippetMatch[], + secondary: SnippetMatch[], +): SnippetMatch[] { + const seen = new Set(); + const out: SnippetMatch[] = []; + for (const item of [...primary, ...secondary]) { + const key = symbolKey(item.name, item.file_path); + if (seen.has(key)) continue; + seen.add(key); + out.push(item); + } + return out; +} + +/** Files connected to a scoped center symbol (call sites + definition files + deps). */ +function collectNeighborFilesForCenter( + db: CodemapDatabase, + center: SymbolMatch, +): Set { + const files = new Set([center.file_path]); + const calls = db + .query<{ file_path: string; caller_name: string; callee_name: string }>( + `SELECT file_path, caller_name, callee_name FROM calls + WHERE file_path = ? AND (caller_name = ? OR callee_name = ?)`, + ) + .all(center.file_path, center.name, center.name) as { + file_path: string; + caller_name: string; + callee_name: string; + }[]; + for (const call of calls) { + files.add(call.file_path); + for (const otherName of [call.caller_name, call.callee_name]) { + if (otherName === center.name) continue; + for (const def of findSymbolsByName(db, { name: otherName })) { + files.add(def.file_path); + } + } + } + const deps = db + .query<{ from_path: string; to_path: string }>( + `SELECT from_path, to_path FROM dependencies + WHERE from_path = ? OR to_path = ?`, + ) + .all(center.file_path, center.file_path) as { + from_path: string; + to_path: string; + }[]; + for (const dep of deps) { + files.add(dep.from_path); + files.add(dep.to_path); + } + return files; +} + +function filterNeighborhoodForCenter( + db: CodemapDatabase, + centerMatches: SymbolMatch[], + rows: SymbolNeighborhoodRow[], +): SymbolNeighborhoodRow[] { + if (centerMatches.length !== 1) return rows; + const allowed = collectNeighborFilesForCenter(db, centerMatches[0]!); + return rows.filter((row) => allowed.has(row.file_path)); +} + +function traceSnippetsSkippedReason( + path: CallPathHop[], + snippetCount: number, +): string | undefined { + if (path.length === 0 || snippetCount > 0) return undefined; + if (path.every((hop) => !isCallHopSnippetEligible(hop))) { + return "Path uses file-level dependency hops; use query_recipe call-path rows with show/snippet per hop."; + } + return "No indexed symbol definitions matched hop names; path rows are still valid."; +} + +function applyRowCap( + rows: T[], + limit: number, +): { rows: T[]; rowsTruncated: boolean } { + if (rows.length <= limit) return { rows, rowsTruncated: false }; + return { rows: rows.slice(0, limit), rowsTruncated: true }; +} + export interface TraceComposeResult { from: string; to: string; @@ -208,6 +328,8 @@ export interface TraceComposeResult { path: CallPathHop[]; snippets: SnippetMatch[]; truncated: boolean; + truncation?: TraceTruncation; + snippets_skipped_reason?: string; } export function composeTraceResult(opts: { @@ -228,9 +350,10 @@ export function composeTraceResult(opts: { for (const name of [hop.caller_name, hop.callee_name]) { const key = symbolKey(name, hop.file_path); if (seen.has(key)) continue; - seen.add(key); - const match = lookupSymbolInFile(db, name, hop.file_path); - if (match !== undefined) matches.push(match); + const match = lookupSymbolForName(db, name, hop.file_path); + if (match === undefined) continue; + seen.add(symbolKey(match.name, match.file_path)); + matches.push(match); } } const allSnippets = snippetsForSymbolMatches({ @@ -239,6 +362,10 @@ export function composeTraceResult(opts: { projectRoot: opts.root, }); const budgeted = applySourceCharBudget(allSnippets, budget); + const snippetsSkippedReason = traceSnippetsSkippedReason( + opts.path, + budgeted.items.length, + ); return { from: opts.from, to: opts.to, @@ -246,6 +373,12 @@ export function composeTraceResult(opts: { path: opts.path, snippets: budgeted.items, truncated: budgeted.truncated, + ...(budgeted.truncated + ? { truncation: { snippets: true } satisfies TraceTruncation } + : {}), + ...(snippetsSkippedReason !== undefined + ? { snippets_skipped_reason: snippetsSkippedReason } + : {}), }; } finally { closeDb(db, { readonly: true }); @@ -257,6 +390,7 @@ export interface ExploreComposeResult { rows: SymbolNeighborhoodRow[]; snippets: SnippetMatch[]; truncated: boolean; + truncation?: TraceTruncation; } export function composeExploreResult(opts: { @@ -265,12 +399,14 @@ export function composeExploreResult(opts: { depth?: number | undefined; kind?: string | undefined; budgetChars?: number | undefined; + rowLimit?: number | undefined; }): | { ok: true; result: ExploreComposeResult } | { ok: false; error: string; kind: TraceFailureKind } { + const names = dedupeNames(opts.names); const merged: SymbolNeighborhoodRow[] = []; const seenRows = new Set(); - for (const name of opts.names) { + for (const name of names) { const neighborhood = executeSymbolNeighborhood({ root: opts.root, name, @@ -286,22 +422,29 @@ export function composeExploreResult(opts: { } } + const rowLimit = opts.rowLimit ?? DEFAULT_EXPLORE_ROW_LIMIT; + const rowCapped = applyRowCap(merged, rowLimit); + const budget = opts.budgetChars ?? DEFAULT_OUTPUT_CHAR_BUDGET; const db = openDb(); try { const allSnippets = snippetsForNeighborhoodRows({ db, - rows: merged, + rows: rowCapped.rows, projectRoot: opts.root, }); const budgeted = applySourceCharBudget(allSnippets, budget); + const truncation: TraceTruncation = {}; + if (budgeted.truncated) truncation.snippets = true; + if (rowCapped.rowsTruncated) truncation.rows = true; return { ok: true, result: { - names: opts.names, - rows: merged, + names, + rows: rowCapped.rows, snippets: budgeted.items, - truncated: budgeted.truncated, + truncated: budgeted.truncated || rowCapped.rowsTruncated, + ...(Object.keys(truncation).length > 0 ? { truncation } : {}), }, }; } finally { @@ -314,6 +457,7 @@ export interface NodeComposeResult { neighborhood: SymbolNeighborhoodRow[]; snippets: SnippetMatch[]; truncated: boolean; + truncation?: TraceTruncation; } export function composeNodeResult(opts: { @@ -342,17 +486,28 @@ export function composeNodeResult(opts: { inPath: opts.inPath, }); const center = buildShowResult(matches); + const scopedNeighborhood = filterNeighborhoodForCenter( + db, + matches, + neighborhood.rows, + ); let snippets: SnippetMatch[] = []; let truncated = false; if (opts.includeSnippets === true) { const budget = opts.budgetChars ?? DEFAULT_OUTPUT_CHAR_BUDGET; - const allSnippets = snippetsForNeighborhoodRows({ + const centerSnippets = snippetsForSymbolMatches({ + db, + matches, + projectRoot: opts.root, + }); + const neighborSnippets = snippetsForNeighborhoodRows({ db, - rows: neighborhood.rows, + rows: scopedNeighborhood, projectRoot: opts.root, }); - const budgeted = applySourceCharBudget(allSnippets, budget); + const merged = mergeSnippetMatches(centerSnippets, neighborSnippets); + const budgeted = applySourceCharBudget(merged, budget); snippets = budgeted.items; truncated = budgeted.truncated; } @@ -361,9 +516,10 @@ export function composeNodeResult(opts: { ok: true, result: { center, - neighborhood: neighborhood.rows, + neighborhood: scopedNeighborhood, snippets, truncated, + ...(truncated ? { truncation: { snippets: true } } : {}), }, }; } finally { diff --git a/templates/agent-content/mcp-instructions.md b/templates/agent-content/mcp-instructions.md index 12edd108..fda93f37 100644 --- a/templates/agent-content/mcp-instructions.md +++ b/templates/agent-content/mcp-instructions.md @@ -10,22 +10,22 @@ Operational playbook injected into the MCP initialize handshake. Full schema, re ## Common tasks -| Goal | MCP tool | Recipe twin (`query_recipe`) | -| ----------------------------- | ----------------------------------------------------------------------- | ------------------------------------------------------------------------------ | -| Exact symbol lookup | **`show`** (`name`, optional `in`) | `find-symbol-definitions` | -| Kind / pattern lookup | **`query_recipe`** | `find-symbol-by-kind` | -| Source at symbol | **`snippet`** | same rows as `show` + disk text | -| Blast radius | **`impact`** (`target`, `direction`, `via`, `depth`) | `fan-in` for file hubs; symbol call graph via SQL or `impact` | -| Call path + snippets | **`trace`** (`from`, `to`, `via?`, `max_depth?`, `budget_chars?`) | `call-path` | -| Multi-symbol survey | **`explore`** (`names`, `depth?`, `kind?`, `budget_chars?`) | `symbol-neighborhood` (once per name) | -| One-hop symbol card | **`node`** (`name`, `kind?`, `in?`, `include_snippets?`) | `show` + `symbol-neighborhood` with `depth=1` | -| Affected tests | **`affected`** (`paths?`, `changed_since?`, `test_glob?`, `max_depth?`) | `affected-tests` (RS-delimit multiple paths in `query_recipe` params) | -| CI / SARIF | **`query_recipe`** + `format: "sarif"` | `deprecated-symbols`, `boundary-violations`, … | -| Ad-hoc SQL | **`query`** | — | -| N statements / one round-trip | **`query_batch`** (MCP-only) | N × `query` | -| Index freshness | **`validate`** | — | -| Drift vs baseline | **`audit`** | saved via `save_baseline` + `query_recipe` / `query` | -| Apply recipe diff rows | **`apply`** | recipe must emit `{file_path, line_start, before_pattern, after_pattern}` rows | +| Goal | MCP tool | Recipe twin (`query_recipe`) | +| ----------------------------- | ------------------------------------------------------------------------- | ------------------------------------------------------------------------------ | +| Exact symbol lookup | **`show`** (`name`, optional `in`) | `find-symbol-definitions` | +| Kind / pattern lookup | **`query_recipe`** | `find-symbol-by-kind` | +| Source at symbol | **`snippet`** | same rows as `show` + disk text | +| Blast radius | **`impact`** (`target`, `direction`, `via`, `depth`) | `fan-in` for file hubs; symbol call graph via SQL or `impact` | +| Call path + snippets | **`trace`** (`from`, `to`, `via?`, `max_depth?`, `budget_chars?`) | `call-path` | +| Multi-symbol survey | **`explore`** (`names`, `depth?`, `kind?`, `budget_chars?`) | `symbol-neighborhood` (once per name) | +| One-hop symbol card | **`node`** (`name`, `kind?`, `in?`, `include_snippets?`, `budget_chars?`) | `show` + `symbol-neighborhood` with `depth=1` | +| Affected tests | **`affected`** (`paths?`, `changed_since?`, `test_glob?`, `max_depth?`) | `affected-tests` (RS-delimit multiple paths in `query_recipe` params) | +| CI / SARIF | **`query_recipe`** + `format: "sarif"` | `deprecated-symbols`, `boundary-violations`, … | +| Ad-hoc SQL | **`query`** | — | +| N statements / one round-trip | **`query_batch`** (MCP-only) | N × `query` | +| Index freshness | **`validate`** | — | +| Drift vs baseline | **`audit`** | saved via `save_baseline` + `query_recipe` / `query` | +| Apply recipe diff rows | **`apply`** | recipe must emit `{file_path, line_start, before_pattern, after_pattern}` rows | ## Chains diff --git a/templates/agent-content/skill/10-recipes-context.md b/templates/agent-content/skill/10-recipes-context.md index df3433d3..038e7769 100644 --- a/templates/agent-content/skill/10-recipes-context.md +++ b/templates/agent-content/skill/10-recipes-context.md @@ -47,9 +47,9 @@ Each emitted delta carries its own `base` metadata so mixed-baseline audits are - **`show`** — `{name, kind?, in?}`. Exact symbol lookup → `{matches, disambiguation?}`. Fuzzy lookup belongs in `query` with `LIKE`. - **`snippet`** — same shape as `show` but each match also carries `source` (file text) + `stale` / `missing` flags. No reindex side-effects. - **`impact`** — `{target, direction?, via?, depth?, limit?, summary?}`. Symbol/file blast-radius walker (replaces hand-composed `WITH RECURSIVE`). Auto-resolves symbol vs file target; `via` defaults to every backend compatible with the kind. -- **`trace`** — `{from, to, max_depth?, via?, budget_chars?}`. Shortest call path + budget-capped snippets (`call-path` recipe twin). -- **`explore`** — `{names, depth?, kind?, budget_chars?}`. Multi-name neighborhood survey + snippets (`symbol-neighborhood` per name). -- **`node`** — `{name, kind?, in?, include_snippets?, budget_chars?}`. `show` center + depth-1 neighborhood; optional inline snippets. +- **`trace`** — `{from, to, max_depth?, via?, budget_chars?}`. Shortest call path + budget-capped snippets (`call-path` recipe twin). `truncated` when snippet budget hit; dependency hops omit auto-snippets. +- **`explore`** — `{names, depth?, kind?, budget_chars?}`. Multi-name neighborhood survey + snippets (`symbol-neighborhood` per deduped name). `truncation.rows` when row cap (500) hit; `truncation.snippets` when budget hit. +- **`node`** — `{name, kind?, in?, include_snippets?, budget_chars?}`. `show` center + scoped depth-1 neighborhood; optional center+neighbor snippets when `include_snippets: true`. - **`affected`** — `{paths?, changed_since?, test_glob?, max_depth?}`. Reverse-dependency walk from changed files to test paths (same preprocessor as **`codemap affected`** → **`affected-tests`** recipe). Explicit `paths` (including `paths: []` for empty — skips git) wins over git discovery; omit `paths` for working tree vs `changed_since` (default `HEAD`). When both `paths` and `changed_since` are sent, `paths` wins (mirrors CLI positional + `--changed-since`). - **`apply`** — `{recipe, params?, dry_run?, yes?}`. Executes the diff hunks a recipe row produces (`{file_path, line_start, before_pattern, after_pattern}`). **All-or-nothing**: any conflict aborts before any file is written. Over MCP/HTTP `yes: true` is required for the write path; `dry_run` and `yes` are mutually exclusive. From d346985d732353fb3b39f1adf0091942adf179de Mon Sep 17 00:00:00 2001 From: Sutu Sebastian Date: Mon, 25 May 2026 17:07:57 +0300 Subject: [PATCH 5/5] fix(mcp): close final PR #134 review nits Per-tool truncation docs, recency JSDoc, explore row-cap test, and HTTP/MCP Zod + recency parity. --- docs/architecture.md | 2 +- src/application/http-server.test.ts | 28 +++++++++++++++++++ src/application/mcp-server.test.ts | 26 +++++++++++++++++ src/application/mcp-server.ts | 4 +-- src/application/recipe-recency.ts | 9 ++++-- src/application/trace-engine.test.ts | 14 ++++++++++ src/application/trace-engine.ts | 2 +- templates/agent-content/mcp-instructions.md | 2 +- .../agent-content/skill/10-recipes-context.md | 6 ++-- 9 files changed, 82 insertions(+), 11 deletions(-) diff --git a/docs/architecture.md b/docs/architecture.md index fc5acb9d..e8383f15 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -135,7 +135,7 @@ A local SQLite database (`.codemap/index.db`) indexes the project tree and store **Affected wiring:** **`src/cli/cmd-affected.ts`** (argv — positional paths / `--stdin` / `--changed-since ` / `--params test_glob|max_depth` + `--json`; bootstrap absorbs `--root`/`--config`) + **`src/application/affected-engine.ts`** (engine — `resolveAffectedChangedPaths` + `executeAffectedTests`; pure recipe composer over bundled `affected-tests` SQL). CLI / MCP / HTTP dispatch the same engine via `tool-handlers.ts`'s `handleAffected` (MCP/HTTP) and `runAffectedCmd` (CLI). Path precedence: explicit paths (CLI positional / MCP `paths` array) → CLI `--stdin` → git vs `changed_since` / `HEAD` (`paths: []` on MCP/HTTP skips git). Result envelope: JSON array of `{test_path, impact_depth, actions?}` — file paths only; CI composes the runner command. **`tryRecordRecipeRun("affected-tests")`** lives at the orchestration layer (`handleAffected` + `runAffectedCmd`), not in the engine — same boundary discipline as `query_recipe` (see [§ `recipe_recency`](#recipe_recency--per-recipe-last-run--run-count-user-data-strict-without-rowid)). Recency records only when at least one changed path was resolved and the recipe SQL ran (empty path sets return `[]` without a recency write). -**Trace / explore / node wiring (MCP + HTTP only):** **`src/application/trace-engine.ts`** (engine — `executeCallPath` / `executeSymbolNeighborhood` recipe composers + `composeTraceResult` / `composeExploreResult` / `composeNodeResult` snippet batching) + **`src/application/output-budget.ts`** (`applySourceCharBudget`, default 15k chars on snippet `source` text). MCP/HTTP dispatch via `tool-handlers.ts`'s `handleTrace` / `handleExplore` / `handleNode`. **`trace`** → `call-path` recipe + disk snippets per hop (cross-file symbol lookup); **`explore`** → deduped `symbol-neighborhood` per `names[]` entry, row cap 500, budget-capped snippets; **`node`** → `show` center + scoped depth-1 neighborhood (filters to center instance when unique) + optional center+neighbor snippets. `truncated` is true when snippet budget and/or explore row cap hit (`truncation.snippets` / `truncation.rows`). Recipe twins remain the Moat A fallback (`query_recipe call-path`, `query_recipe symbol-neighborhood`). **`tryRecordRecipeRun`** at orchestration only (`call-path` on trace success; `symbol-neighborhood` on explore/node success). +**Trace / explore / node wiring (MCP + HTTP only):** **`src/application/trace-engine.ts`** (engine — `executeCallPath` / `executeSymbolNeighborhood` recipe composers + `composeTraceResult` / `composeExploreResult` / `composeNodeResult` snippet batching) + **`src/application/output-budget.ts`** (`applySourceCharBudget`, default 15k chars on snippet `source` text). MCP/HTTP dispatch via `tool-handlers.ts`'s `handleTrace` / `handleExplore` / `handleNode`. **`trace`** → `call-path` recipe + disk snippets per hop (cross-file symbol lookup); snippet budget only (`truncation.snippets`; `snippets_skipped_reason` on dependency hops). **`explore`** → deduped `symbol-neighborhood` per `names[]` entry, row cap 500, budget-capped snippets (`truncation.rows` / `truncation.snippets`). **`node`** → `show` center + scoped depth-1 neighborhood (filters to center instance when unique) + optional center+neighbor snippets (`truncated` only when `include_snippets: true`). Recipe twins remain the Moat A fallback (`query_recipe call-path`, `query_recipe symbol-neighborhood`). **`tryRecordRecipeRun`** at orchestration only (`call-path` on trace success; `symbol-neighborhood` on explore/node success). **Apply wiring:** **`src/cli/cmd-apply.ts`** (argv — `` + `--params` + `--dry-run` + `--yes` + `--json`; bootstrap absorbs `--root`/`--config`) + **`src/application/apply-engine.ts`** (engine — `applyDiffPayload({rows, projectRoot, dryRun})`). Pure transport-agnostic substrate-shaped fix executor: consumes the existing `--format diff-json` row contract from any recipe (`{file_path, line_start, before_pattern, after_pattern}`), validates each row against current disk, and either previews (dry-run) or writes (apply). CLI / MCP / HTTP all dispatch the same engine via `tool-handlers.ts`'s `handleApply`. **Phase 1** (always) resolves the project root via `path.resolve(projectRoot)` once, then for each row: rejects absolute `file_path` inputs and any candidate whose `path.resolve(resolvedRoot, file_path)` lands outside `resolvedRoot` (conflict `path escapes project root` — guards CLI + MCP + HTTP write paths against `../escape.ts`-style traversal); rejects duplicate `(file_path, line_start)` tuples (conflict `duplicate edit on same line` — without this, two phase-1-passing rows targeting the same line would split the run mid-phase-2 because the first replace invalidates the second's substring assertion, leaving Q2 (c) cross-file partial state). Reads each file at most once into `sourceCache`, splits on `/\r?\n/` for conflict reporting, checks `actual.includes(before_pattern)` (substring match — mirrors `buildDiffJson`'s contract; `rename-preview` emits `before_pattern = old_name` as the bare identifier, so whole-line exact match would conflict every time). Conflicts collect five reasons (`file missing` / `line out of range` / `line content drifted` / `path escapes project root` / `duplicate edit on same line`) — Q3 scan-and-collect, not fail-fast. **Phase 2** (gated on `!dryRun && conflicts.length === 0`) re-splits the cached source on raw `"\n"` (preserves CRLF as trailing `\r` per line; rejoining with `"\n"` round-trips losslessly), applies each file's edits in descending line order via `actual.replace(before, after)` with `$`-pre-escape (`replace(/\$/g, "$$$$")` — matches `buildDiffJson`'s GetSubstitution defence so identifiers like `$inject` round-trip safely), writes to a sibling temp path (`.codemap-apply-.tmp`), then `renameSync` into place — POSIX-atomic per file; concurrent readers see either pre-rename or post-rename content, never a torn write. **Q2 (c) all-or-nothing (semantic)**: any phase-1 conflict aborts phase 2 entirely before any file is touched. Phase-2 I/O failures (`writeFileSync` / `renameSync`) are NOT transactional across files — per-file atomicity holds (temp + rename), but a crash on file N leaves files `1..N-1` already renamed with no rollback; cross-file rollback would require pre-write backups + restore-on-throw and is deferred to a future PR. **Q6 gate**: TTY no `--yes` → phase-1 preview + `Proceed? [y/N]` prompt on stderr (default-N, `node:readline/promises`); TTY `--yes` → no prompt; non-TTY (CI / agents / MCP) without `--yes`/`--dry-run` rejected with stderr message. `--dry-run` + `--yes` mutually exclusive (parse-time error). MCP/HTTP transports (`handleApply`) require `yes: true` for the write path — there's no prompt to fall back on; `dry_run + yes` rejected as mutually exclusive. Result envelope (Q5; identical across modes): `{mode: 'dry-run'|'apply', applied: bool, files: [{file_path, rows_applied, warnings?}], conflicts: [{file_path, line_start, before_pattern, actual_at_line, reason}], summary: {files, files_modified, rows, rows_applied, conflicts, files_with_conflicts}}`. `applied: true` only when `mode === 'apply'` AND zero conflicts AND at least one row applied. Q7 idempotency: re-running on already-applied code reports a `line content drifted` conflict with `actual_at_line` showing the post-rename content; the user reads it and re-runs `codemap` to refresh the index → next run produces 0 rows (recipe finds nothing to rename) → vacuous clean apply. **Same-line ambiguity caveat (documented limitation):** `actual.replace(before_pattern, after_pattern)` rewrites only the **first** occurrence on the line. When `before_pattern` appears twice (e.g. `const foo = foo();` with `before = "foo"`) only the leftmost is replaced; the engine still reports `applied: true`. This mirrors `buildDiffJson`'s formatter contract verbatim — recipe authors who hit it normalise their SQL to emit a more specific pattern, or accept it (the formatter's `--format diff` preview shows the same shape). Promotion path: tighten phase-1 to conflict on ambiguity in a future PR if real users complain, but only alongside the formatter so preview and execution stay in lockstep. SARIF / annotations not supported (write action, not findings). TOCTOU: phase-1 reads through `sourceCache`; phase-2 transforms the cached source and writes — the gap between read and rename is a deliberate v1 simplification (apply isn't adversarial). Per Q10, only `cli/cmd-apply.ts` + `application/tool-handlers.ts` (+ the test files) may import `apply-engine.ts` for production execution — re-runnable forbidden-edge query at [§ Boundary verification — apply write path](#boundary-verification--apply-write-path). diff --git a/src/application/http-server.test.ts b/src/application/http-server.test.ts index 868bf6d3..d55029b1 100644 --- a/src/application/http-server.test.ts +++ b/src/application/http-server.test.ts @@ -521,6 +521,26 @@ describe("http-server — POST /tool/{other tools}", () => { } }); + it("records recipe recency after explore", async () => { + seedTraceGraph(); + serverHandle = await startServer(); + const r = await postTool(serverHandle.port, "explore", { + names: ["alpha"], + }); + expect(r.status).toBe(200); + const db = openDb(); + try { + const row = db + .query<{ run_count: number }>( + "SELECT run_count FROM recipe_recency WHERE recipe_id = 'symbol-neighborhood'", + ) + .get(); + expect(row?.run_count).toBeGreaterThanOrEqual(1); + } finally { + closeDb(db); + } + }); + it("list_baselines returns array (empty when none saved)", async () => { serverHandle = await startServer(); const r = await postTool(serverHandle.port, "list_baselines", {}); @@ -707,6 +727,14 @@ describe("http-server — Zod input validation at HTTP boundary", () => { expect(r.json.error).toContain("from"); }); + it("trace without to → 400 with structured error", async () => { + serverHandle = await startServer(); + const r = await postTool(serverHandle.port, "trace", { from: "foo" }); + expect(r.status).toBe(400); + expect(r.json.error).toContain('"trace"'); + expect(r.json.error).toContain("to"); + }); + it("node with name=number → 400 (not deep handler crash)", async () => { serverHandle = await startServer(); const r = await postTool(serverHandle.port, "node", { name: 1 }); diff --git a/src/application/mcp-server.test.ts b/src/application/mcp-server.test.ts index b014dec7..c96750d6 100644 --- a/src/application/mcp-server.test.ts +++ b/src/application/mcp-server.test.ts @@ -1532,6 +1532,32 @@ describe("MCP server — trace / explore / node tools", () => { } }); + it("trace returns isError when from is missing (Zod rejects)", async () => { + const { client, server } = await makeClient(); + try { + const r = await client.callTool({ + name: "trace", + arguments: { to: "b" }, + }); + expect((r as { isError?: boolean }).isError).toBe(true); + } finally { + await server.close(); + } + }); + + it("trace returns isError when to is missing (Zod rejects)", async () => { + const { client, server } = await makeClient(); + try { + const r = await client.callTool({ + name: "trace", + arguments: { from: "a" }, + }); + expect((r as { isError?: boolean }).isError).toBe(true); + } finally { + await server.close(); + } + }); + it("explore returns isError on empty names array", async () => { const { client, server } = await makeClient(); try { diff --git a/src/application/mcp-server.ts b/src/application/mcp-server.ts index ed3bddb5..d036d651 100644 --- a/src/application/mcp-server.ts +++ b/src/application/mcp-server.ts @@ -326,7 +326,7 @@ function registerTraceTool(server: McpServer, opts: ServerOpts): void { "trace", { description: - "Shortest call path between two symbols plus budget-capped snippets. Composes `call-path` recipe + disk reads (cross-file callee lookup). Args: from, to, max_depth?, via (calls|dependencies|all), budget_chars (default 15000, snippet source text only). Returns {from, to, via?, path, snippets, truncated, truncation?, snippets_skipped_reason?}. `truncated` is true when snippet budget or explore row cap hit; dependency hops omit auto-snippets. Fall back to `query_recipe` call-path when unsure.", + "Shortest call path between two symbols plus budget-capped snippets. Composes `call-path` recipe + disk reads (cross-file callee lookup). Args: from, to, max_depth?, via (calls|dependencies|all), budget_chars (default 15000, snippet source text only). Returns {from, to, via?, path, snippets, truncated, truncation?, snippets_skipped_reason?}. `truncated` is true when snippet budget hit (`truncation.snippets`); dependency hops omit auto-snippets (`snippets_skipped_reason`). Fall back to `query_recipe` call-path when unsure.", inputSchema: traceArgsSchema, }, (args) => wrapToolResult(handleTrace(args, opts.root)), @@ -350,7 +350,7 @@ function registerNodeTool(server: McpServer, opts: ServerOpts): void { "node", { description: - "One-hop symbol survey: `show` center + scoped depth-1 `symbol-neighborhood` + optional inline snippets. When center is unique (`in` or single match), neighborhood filters to that instance's connected files. Args: name, kind?, in?, include_snippets (default false), budget_chars? (default 15000 when snippets enabled; snippet source only). Returns {center, neighborhood, snippets, truncated, truncation?}.", + "One-hop symbol survey: `show` center + scoped depth-1 `symbol-neighborhood` + optional inline snippets. When center is unique (`in` or single match), neighborhood filters to that instance's connected files. Args: name, kind?, in?, include_snippets (default false), budget_chars? (default 15000 when snippets enabled; snippet source only). Returns {center, neighborhood, snippets, truncated, truncation?}. `truncated` only when `include_snippets: true` and snippet budget hit.", inputSchema: nodeArgsSchema, }, (args) => wrapToolResult(handleNode(args, opts.root)), diff --git a/src/application/recipe-recency.ts b/src/application/recipe-recency.ts index 653d0d2f..5dd630d2 100644 --- a/src/application/recipe-recency.ts +++ b/src/application/recipe-recency.ts @@ -7,7 +7,9 @@ import { STATE_DIR_DEFAULT } from "./state-dir"; /** * Write-path imports (`tryRecordRecipeRun` / `recordRecipeRun`) are restricted - * to `tool-handlers.ts` + `cli/cmd-query.ts` + `cli/cmd-affected.ts` (+ the test file). Re-runnable + * to `tool-handlers.ts` (`handleQueryRecipe`, `handleAffected`, `handleTrace`, + * `handleExplore`, `handleNode`) + `cli/cmd-query.ts` + `cli/cmd-affected.ts` + * (+ the test file). Re-runnable * forbidden-edge query lives at [`docs/architecture.md` § Boundary verification — * `recipe_recency` write path](../../docs/architecture.md#boundary-verification--recipe_recency-write-path). * Read-path imports (`enrichWithRecency` / `loadRecipeRecency`) are unrestricted. @@ -55,8 +57,9 @@ export function recordRecipeRun(opts: RecordRunOpts): void { } /** - * Orchestration-layer wrapper (`handleQueryRecipe`, `handleAffected`, `runQueryCmd`, - * `runAffectedCmd`). Opens its own DB because `executeQuery` runs with + * Orchestration-layer wrapper (`handleQueryRecipe`, `handleAffected`, + * `handleTrace`, `handleExplore`, `handleNode`, `runQueryCmd`, `runAffectedCmd`). + * Opens its own DB because `executeQuery` runs with * `PRAGMA query_only = 1` and can't double as the writer. Swallows every error — * recency-write failures NEVER block the recipe response. * diff --git a/src/application/trace-engine.test.ts b/src/application/trace-engine.test.ts index a3a84af2..316e7edd 100644 --- a/src/application/trace-engine.test.ts +++ b/src/application/trace-engine.test.ts @@ -285,4 +285,18 @@ describe("composeExploreResult dedupe", () => { if (!r.ok) return; expect(r.result.names).toEqual(["foo"]); }); + + it("sets truncation.rows when rowLimit exceeded", () => { + seedCallGraph(); + const r = composeExploreResult({ + root: benchDir, + names: ["foo", "bar"], + rowLimit: 1, + }); + expect(r.ok).toBe(true); + if (!r.ok) return; + expect(r.result.rows).toHaveLength(1); + expect(r.result.truncated).toBe(true); + expect(r.result.truncation?.rows).toBe(true); + }); }); diff --git a/src/application/trace-engine.ts b/src/application/trace-engine.ts index 951df51c..55825d6c 100644 --- a/src/application/trace-engine.ts +++ b/src/application/trace-engine.ts @@ -25,7 +25,7 @@ import type { ShowResult, SnippetMatch, SymbolMatch } from "./show-engine"; export type TraceFailureKind = "param" | "query" | "internal"; -/** Default row cap for explore before `rows_truncated` (structural payload guard). */ +/** Default row cap for explore before `truncation.rows` (structural payload guard). */ export const DEFAULT_EXPLORE_ROW_LIMIT = 500; export interface CallPathHop { diff --git a/templates/agent-content/mcp-instructions.md b/templates/agent-content/mcp-instructions.md index fda93f37..6aeab0ac 100644 --- a/templates/agent-content/mcp-instructions.md +++ b/templates/agent-content/mcp-instructions.md @@ -30,7 +30,7 @@ Operational playbook injected into the MCP initialize handshake. Full schema, re ## Chains - Rename: `find-symbol-definitions` → `find-symbol-references` (both via **`query_recipe`**). -- Call path: **`trace`** (`from`, `to`) or **`query_recipe`** `call-path`; add snippets via **`trace`** / **`node`** / **`explore`** (budget-capped) or **`snippet`** per row. +- Call path: **`trace`** (`from`, `to`) or **`query_recipe`** `call-path`; add snippets via **`trace`** / **`node`** / **`explore`** (budget-capped) or **`snippet`** per row. Dependency hops may return `snippets_skipped_reason` — fall back to **`query_recipe`** + **`snippet`** per hop. - Refactor risk: `fan-in` + `refactor-risk-ranking`. - Edit path: **`show`** → **`snippet`**; if `stale: true`, line range may have drifted. diff --git a/templates/agent-content/skill/10-recipes-context.md b/templates/agent-content/skill/10-recipes-context.md index 038e7769..45114767 100644 --- a/templates/agent-content/skill/10-recipes-context.md +++ b/templates/agent-content/skill/10-recipes-context.md @@ -47,9 +47,9 @@ Each emitted delta carries its own `base` metadata so mixed-baseline audits are - **`show`** — `{name, kind?, in?}`. Exact symbol lookup → `{matches, disambiguation?}`. Fuzzy lookup belongs in `query` with `LIKE`. - **`snippet`** — same shape as `show` but each match also carries `source` (file text) + `stale` / `missing` flags. No reindex side-effects. - **`impact`** — `{target, direction?, via?, depth?, limit?, summary?}`. Symbol/file blast-radius walker (replaces hand-composed `WITH RECURSIVE`). Auto-resolves symbol vs file target; `via` defaults to every backend compatible with the kind. -- **`trace`** — `{from, to, max_depth?, via?, budget_chars?}`. Shortest call path + budget-capped snippets (`call-path` recipe twin). `truncated` when snippet budget hit; dependency hops omit auto-snippets. -- **`explore`** — `{names, depth?, kind?, budget_chars?}`. Multi-name neighborhood survey + snippets (`symbol-neighborhood` per deduped name). `truncation.rows` when row cap (500) hit; `truncation.snippets` when budget hit. -- **`node`** — `{name, kind?, in?, include_snippets?, budget_chars?}`. `show` center + scoped depth-1 neighborhood; optional center+neighbor snippets when `include_snippets: true`. +- **`trace`** — `{from, to, max_depth?, via?, budget_chars?}`. Shortest call path + budget-capped snippets (`call-path` recipe twin). `truncated` when snippet budget hit (`truncation.snippets`); dependency hops set `snippets_skipped_reason` instead of auto-snippets. +- **`explore`** — `{names, depth?, kind?, budget_chars?}`. Multi-name neighborhood survey + snippets (`symbol-neighborhood` per deduped name). `truncated` when row cap (500) and/or snippet budget hit (`truncation.rows` / `truncation.snippets`). +- **`node`** — `{name, kind?, in?, include_snippets?, budget_chars?}`. `show` center + scoped depth-1 neighborhood; optional center+neighbor snippets when `include_snippets: true` (`truncated` / `truncation.snippets` only then). - **`affected`** — `{paths?, changed_since?, test_glob?, max_depth?}`. Reverse-dependency walk from changed files to test paths (same preprocessor as **`codemap affected`** → **`affected-tests`** recipe). Explicit `paths` (including `paths: []` for empty — skips git) wins over git discovery; omit `paths` for working tree vs `changed_since` (default `HEAD`). When both `paths` and `changed_since` are sent, `paths` wins (mirrors CLI positional + `--changed-since`). - **`apply`** — `{recipe, params?, dry_run?, yes?}`. Executes the diff hunks a recipe row produces (`{file_path, line_start, before_pattern, after_pattern}`). **All-or-nothing**: any conflict aborts before any file is written. Over MCP/HTTP `yes: true` is required for the write path; `dry_run` and `yes` are mutually exclusive.