From 6f3593f01b5b3a08ec93e52d343628d66bd0f164 Mon Sep 17 00:00:00 2001 From: Sutu Sebastian Date: Fri, 1 May 2026 18:55:58 +0300 Subject: [PATCH 01/14] docs(plans): draft recipes-content-registry MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Pair bundled recipes with sibling .md (when-to-use / follow-up SQL); enable project-local recipes via .codemap/recipes/.{sql,md}; auto-inherit into the codemap://recipes / recipes/{id} MCP resources shipped in PR #35. Plan covers storage layout (file-pair vs YAML-frontmatter — file-pair wins per editor + LSP support), loader contract (eager + cached, pure transport-agnostic engine in src/application/recipes-loader.ts), CLI surface (zero new flags — same shape; --recipes-json gains source + body fields), and a 6-commit tracer-bullet sequence. 6 open questions worth a grill round before code: bundled storage layout, loading time (eager vs lazy), monorepo discovery walk-up, actions for project recipes (skip / frontmatter / sibling .json), conflict resolution noise level, and load-time DML/DDL rejection. Status: design pass; not yet implemented. --- docs/plans/recipes-content-registry.md | 233 +++++++++++++++++++++++++ docs/roadmap.md | 2 +- 2 files changed, 234 insertions(+), 1 deletion(-) create mode 100644 docs/plans/recipes-content-registry.md diff --git a/docs/plans/recipes-content-registry.md b/docs/plans/recipes-content-registry.md new file mode 100644 index 00000000..1729d85a --- /dev/null +++ b/docs/plans/recipes-content-registry.md @@ -0,0 +1,233 @@ +## Plan — `recipes-content-registry` + +> Pair every bundled recipe with a sibling `.md` description (when-to-use / follow-up SQL hints), and let projects ship their own recipes via `.codemap/recipes/.{sql,md}` files — surfaces uniformly in `--recipes-json`, `codemap query --recipe `, and the `codemap://recipes` MCP resource. +> +> Adopted from [`docs/roadmap.md` § Backlog](../roadmap.md#backlog) ("Recipes-as-content registry"). Builds on the bundled recipe surface (PR [#26](https://github.com/stainless-code/codemap/pull/26)) and the MCP resources shipped in PR [#35](https://github.com/stainless-code/codemap/pull/35). + +**Status:** Open — design pass; not yet implemented. +**Cross-refs:** [`docs/architecture.md` § CLI usage](../architecture.md#cli-usage) (recipes are part of the query surface), [`docs/architecture.md` § MCP wiring](../architecture.md#cli-usage) (`codemap://recipes` resource), [`.agents/lessons.md`](../../.agents/lessons.md) (changesets policy: pre-v1 patch unless schema-breaks). + +--- + +## 1. Goal + +**Two consumers, one registry.** + +- **Bundled recipes today** live in `src/cli/query-recipes.ts` as a TypeScript object map. SQL + short description + optional `actions` are all in code. Description is a one-liner — there's no room for "when to use this", "follow-up SQL", or "what to do with the rows." +- **Project teams today** can't ship a custom recipe without forking codemap or wrapping `codemap query --json ""` in their own scripts. There's no on-ramp for "every team member can run `codemap query --recipe internal-flaky-tests` without remembering the SQL." + +After v1: + +```bash +# bundled recipe — long-form description in sibling .md +codemap query --json --recipe fan-out + +# project-local recipe loaded from .codemap/recipes/internal-flaky-tests.sql +codemap query --json --recipe internal-flaky-tests + +# catalog surfaces both +codemap query --recipes-json +# MCP resource surfaces both +read_resource codemap://recipes +``` + +The wins: + +- **Bundled recipes get room to teach.** The one-liner becomes a Markdown body with usage notes, follow-up queries, and "what an agent should do with these rows." +- **Project teams ship internal SQL** without forking. `git`-tracked, code-reviewable, no plugin API needed. +- **MCP / agent surface stays uniform** — `codemap://recipes` and `codemap://recipes/{id}` automatically include project-local recipes; agents discover them at session-start. + +## 2. Scope split (this plan vs follow-ups) + +| Slice | Status | Where it lives | +| --------------------------------------------------------------------------------------------------------------------- | ----------------------------------------- | ------------------------------------------------------------------- | +| **A. Project-local recipes** (`.codemap/recipes/.sql`) — actually-new capability | This plan (v1) | New loader + composes with existing CLI / MCP surfaces | +| **B. Bundled recipe extraction** (move `QUERY_RECIPES` map → `templates/recipes/.{sql,md}` files) — pure refactor | This plan (v1) | Same loader; bundled recipes become the same shape as project-local | +| **C. Sibling `.md` description body** for both bundled AND project-local | This plan (v1) | Optional file alongside `.sql` | +| **D. `actions` support for project-local recipes** | Open question (§ 12) — likely v1 if cheap | YAML frontmatter on `.md`? Sibling `.actions.json`? | +| **E. Recipe versioning / migrations** | v1.x | Out of scope for v1 — defer until two consumers ask | +| **F. Recipe parameters** (`{table}`, `{limit}` placeholders) | v1.x | Out of scope — would require a templating layer | + +Slices A + B + C ship together because B is pre-requisite for C (uniform loader needs uniform storage), and A is the actual user-facing capability. D depends on grill round. + +## 3. Storage layout + +### 3.1 Bundled recipes (after refactor) + +``` +templates/recipes/ +├── fan-out.sql # the SQL string +├── fan-out.md # description body (optional but recommended) +├── fan-out-sample.sql +├── fan-out-sample.md +├── deprecated-symbols.sql +├── deprecated-symbols.md +├── visibility-tags.sql +├── visibility-tags.md +└── … +``` + +Each `.sql` is the recipe's SQL verbatim (one statement, no `;` terminator needed). The matching `.md` is optional — when absent, the recipe still loads but has no long-form description (CLI / MCP surfaces show only the recipe id). + +`templates/recipes/` ships in the npm package alongside `templates/agents/` (already part of the published artifact — `agents-init.ts`'s `resolveAgentsTemplateDir()` shows the pattern). + +### 3.2 Project-local recipes + +``` +/ +└── .codemap/ + └── recipes/ + ├── internal-flaky-tests.sql + ├── internal-flaky-tests.md + └── owner-fanout.sql +``` + +`` is the same root the CLI's `--root` / `CODEMAP_ROOT` resolves to. `.codemap/` is the conventional location for codemap-related project artifacts — same parent as a future user-config might use. + +### 3.3 Single-file form (rejected for v1) + +YAML-frontmatter Markdown with the SQL in a code block (Astro / Hugo style) was considered: + +````markdown +--- +id: fan-out +description: Top 10 files by dependency fan-out +actions: + - type: review-coupling + description: … +--- + +When to use: … + +Follow-up SQL: … + +```sql +SELECT from_path, COUNT(*) AS deps FROM dependencies … +``` +```` + +```` + +**Rejected because:** +- Editor support for SQL inside Markdown code blocks is worse than for `.sql` files (no syntax highlighting, no LSP). +- Two-file split keeps SQL editable as SQL (works with sqlite CLI: `sqlite3 .codemap.db ".read .codemap/recipes/foo.sql"`). +- The frontmatter parsing surface (gray-matter or hand-rolled) is more code than a sibling `.md` lookup. +- One-file form remains a v1.x option if real consumer demand emerges. + +## 4. Loader contract + +A pure function in `src/application/recipes-loader.ts`: + +```typescript +interface LoadedRecipe { + id: string; + sql: string; + description: string | undefined; // first-line of .md, or undefined + body: string | undefined; // full .md body, or undefined + actions: RecipeAction[] | undefined; // from YAML frontmatter on .md (D — open question) + source: "bundled" | "project"; // for catalog disambiguation +} + +export function loadAllRecipes(opts: { + bundledDir: string; // resolveBundledRecipesDir() — npm package layout + projectDir: string | undefined; // resolveProjectRecipesDir(root) — undefined if .codemap/recipes/ is absent +}): LoadedRecipe[]; +```` + +**Conflict resolution:** if a project recipe has the same `id` as a bundled recipe, the **project recipe wins** (`source: "project"`); the bundled one is shadowed but still discoverable via a hypothetical future `--source bundled` filter (out of scope for v1). User-code-wins is the standard convention (npm, ESLint plugins, etc.). + +**Validation:** at load time, each `.sql` must parse as something `bun:sqlite`'s `.prepare()` accepts — but we don't actually prepare against a DB until `--recipe ` runs (would require an indexed project at load time). v1 does a cheap lexical sanity check: non-empty after stripping `--` line comments and trailing whitespace. SQL errors surface at query-time with the same `enrichQueryError` pretty-printing as ad-hoc SQL. + +**Loading time:** **eager at startup**, but cheap. `templates/recipes/` is filesystem-stable per-version (read once). `.codemap/recipes/` is filesystem-stable per-session (the user isn't editing recipes mid-CLI-call). Cache the result in a module-level variable; invalidate only on process restart. + +## 5. CLI surface (no new flags — same shape as today) + +```bash +codemap query --recipe # works for bundled OR project recipes; project wins on conflict +codemap query --recipes-json # full catalog: bundled + project, with `source` field +codemap query --print-sql # prints the SQL of regardless of source +``` + +`--recipes-json` output gets two new fields per recipe: + +```json +[ + { + "id": "fan-out", + "description": "Top 10 files by dependency fan-out", + "body": "# Fan-out\n\nWhen to use: …\n\nFollow-up SQL: …", + "sql": "SELECT from_path …", + "actions": [{ "type": "review-coupling", "description": "…" }], + "source": "bundled" + }, + { + "id": "internal-flaky-tests", + "description": null, + "body": null, + "sql": "SELECT path FROM files WHERE …", + "actions": null, + "source": "project" + } +] +``` + +`description` and `body` are nullable — recipes without sibling `.md` get null. `actions` field nullability depends on grill question D. + +## 6. MCP surface — auto-inherits + +Already shipped in PR [#35](https://github.com/stainless-code/codemap/pull/35): + +- `codemap://recipes` resource — automatically picks up project recipes since it calls `listQueryRecipeCatalog()` (which becomes the loader). +- `codemap://recipes/{id}` template — auto-resolves project recipe ids. + +The agent's discovery story is unchanged; the catalog just got bigger. + +## 7. Implementation deps + +- No new npm dependencies. Uses `node:fs/promises` (or sync `readFileSync` for cache population) + `node:path`. +- Reuses the existing `resolveAgentsTemplateDir()` pattern for `resolveBundledRecipesDir()` (npm package layout — `templates/recipes/` next to `templates/agents/`). +- New file: `src/application/recipes-loader.ts` (loader engine — pure, transport-agnostic). +- `src/cli/query-recipes.ts` becomes a thin re-export layer that calls the loader (preserves the `getQueryRecipeSql` / `getQueryRecipeActions` / `listQueryRecipeIds` / `listQueryRecipeCatalog` / `QUERY_RECIPES` named exports for backwards-compat with the MCP server + cmd-query). + +## 8. Tracer-bullet sequence + +Per [`tracer-bullets`](../../.agents/rules/tracer-bullets.md): + +1. **Loader scaffold** — `src/application/recipes-loader.ts` with `loadAllRecipes` returning bundled-only (project loader stubbed). Tests cover empty / one-recipe / multiple-recipe loads against a fixture `templates/recipes/` directory. Commit. +2. **Migrate bundled recipes** — extract every entry in `QUERY_RECIPES` to `templates/recipes/.sql`; for the ones with a meaningful one-liner already, also add `.md`. `query-recipes.ts` becomes a thin shim that calls `loadAllRecipes({bundledDir, projectDir: undefined})`. Tests: every existing recipe id still resolves to the same SQL. Commit. +3. **Project-local loader** — implement `resolveProjectRecipesDir(root)` + load `.codemap/recipes/*.sql` + sibling `.md` discovery. Tests cover: no `.codemap/recipes/` (no error, no project recipes); one project recipe; project recipe shadows bundled. Commit. +4. **`--recipes-json` carries `source` + `body`** — extend the catalog output. Tests cover both source values + body presence/absence. Commit. +5. **Optional `actions` support** (depends on grill Q-D) — if YAML frontmatter wins, ship a tiny parser; if sibling `.actions.json` wins, ship the lookup. Commit. +6. **Docs + agents update** — `architecture.md § Recipes wiring` paragraph, glossary entries (`recipe` definition gets the bundled-vs-project disambiguation), README CLI block (mention `.codemap/recipes/`), rule + skill across `.agents/` and `templates/agents/` (Rule 10), patch changeset. Delete this plan (Rule 2), lift canonical bits into architecture.md. Commit. + +Estimated total: ~1 day across ~6 commits. + +## 9. Open questions (worth a `grill-me` round before code) + +### Settled + +_None yet._ + +### Still open + +- **Q-A. Storage layout for bundled recipes.** Two options: (i) `templates/recipes/.{sql,md}` (proposed in § 3.1) — uniform with project layout, simpler loader. (ii) Keep `QUERY_RECIPES` as code, add `templates/recipes/.md` only for descriptions — less file churn but two storage shapes to maintain. Bias toward (i) — uniformity wins. +- **Q-B. Loading time.** Eager (load at every CLI invocation, ~10ms) vs lazy (on first `--recipe ` / `--recipes-json` / `codemap://recipes` access). Eager is simpler; lazy saves cycles on `codemap query ""` invocations that never touch a recipe. Bias toward eager — startup cost is ms-scale and the simplicity of "registry is always populated" pays for itself. +- **Q-C. Project recipes — discovery walk-up?** Today `.codemap.db` is created in the project root only. Should `.codemap/recipes/` also be project-root-only, OR walk up like `.git` does (find nearest ancestor `.codemap/recipes/` directory)? Walk-up matches monorepo intuition; root-only matches everything else codemap does today. +- **Q-D. `actions` for project-local recipes — and how specified?** Three options: (i) skip for v1 (project recipes can't have actions; bundled-only feature). (ii) YAML frontmatter on `.md` (one parser dependency, e.g. `gray-matter` or hand-rolled). (iii) Sibling `.actions.json` file (no parser; another file per recipe). (i) keeps v1 lean; (ii) is most ergonomic for recipe authors; (iii) is the "no new dep" middle ground. +- **Q-E. Conflict resolution loud or quiet?** When a project recipe shadows a bundled one, do we (i) silently let project win (clean), (ii) emit a one-time stderr warning ("project recipe `fan-out` shadows the bundled `fan-out`"), or (iii) require an explicit `--allow-shadow` flag? Bias toward (i) — user code wins is the convention; warnings risk noise. +- **Q-F. Validation strictness.** Reject project recipes that contain DML / DDL at load time (mirrors the `PRAGMA query_only` defence we shipped in PR #35), or let them fail at run time? Load-time rejection is more agent-friendly (fails fast on save_baseline-style misuse); run-time falls back to the engine's existing safeguard. Bias toward load-time — same lexical sanity check that's already proposed for empty-file detection. + +## 10. Non-goals (v1) + +- **Recipe versioning / migrations.** If a bundled recipe's SQL changes between codemap versions, project consumers using the same id silently get the new SQL on upgrade. Defer until a real consumer reports breakage. +- **Recipe parameters / templating.** No `{table}` / `{limit}` placeholder substitution — recipes are static SQL. Templating adds a parser surface and ambiguity around what's a parameter vs a SQL token. Defer until two consumers ask with the same shape. +- **Network-fetched recipes.** No `codemap recipes add github.com/foo/recipes` registry. Stay filesystem-only — security and supply-chain reasoning matches the agent-host trust boundary from the MCP plan. +- **Recipe execution control beyond `--recipe `.** No `--list-recipes` shorthand (use `--recipes-json | jq`). No `codemap recipe run ` (use `codemap query --recipe `). Single CLI surface stays. +- **`.codemap/recipes/.json` (raw envelope)** — recipes are SQL-first; JSON envelope would re-invent half of `.sql` + `.md`. + +## 11. References + +- Roadmap entry: [`docs/roadmap.md` § Backlog](../roadmap.md#backlog). +- Existing recipe shape: [`src/cli/query-recipes.ts`](../../src/cli/query-recipes.ts) (`QUERY_RECIPES` map, `RecipeAction` interface). +- MCP resources that auto-inherit: [`docs/architecture.md` § MCP wiring](../architecture.md#cli-usage), `codemap://recipes` and `codemap://recipes/{id}`. +- Doc lifecycle: this file follows the **Plan** type per [`docs/README.md` § Document Lifecycle](../README.md#document-lifecycle) — **delete on ship**, lift the canonical bits into `architecture.md` per Rule 2. diff --git a/docs/roadmap.md b/docs/roadmap.md index 034e5874..93f957be 100644 --- a/docs/roadmap.md +++ b/docs/roadmap.md @@ -39,7 +39,7 @@ Codemap stays a structural-index primitive that other tools can consume. Out of - [ ] **`codemap audit --base `** (v1.x) — worktree+reindex snapshot strategy. v1 shipped `--baseline ` / `---baseline ` (B.6 reuse) — see [`architecture.md` § Audit wiring](./architecture.md#cli-usage). v1.x adds `--base ` for "audit against an arbitrary ref I haven't pre-baselined" (defers worktree spawn + cache decision until a real consumer asks). - [ ] **`codemap audit` verdict + thresholds** (v1.x) — `verdict: "pass" | "warn" | "fail"` driven by `codemap.config.audit.deltas[].{added_max, action}`. Triggers: two consumers ship `jq`-based threshold scripts with similar shapes, OR one consumer asks with a concrete config sketch. Until then, raw deltas + consumer-side `jq` is the CI exit-code idiom. - [ ] **`codemap serve` (HTTP API, v1.x)** — same tool taxonomy + output shape as `codemap mcp` (shipped in v1), exposed over `POST /tool/{name}` with loopback default and optional `--token`. Defer until a concrete non-MCP consumer asks; design points are reserved in [`architecture.md` § MCP wiring](./architecture.md#cli-usage) so HTTP inherits them when its turn comes. -- [ ] **Recipes-as-content registry** — pair every bundled recipe in `src/cli/query-recipes.ts` with a sibling `.md` (or YAML frontmatter) describing _when to use, follow-up SQL_; surface in `--recipes-json`. Plus **project-local recipes** loaded from `.codemap/recipes/*.{sql,md}` so teams can ship internal SQL without an adapter API +- [ ] **Recipes-as-content registry** — pair every bundled recipe with a sibling `.md` (when-to-use, follow-up SQL); plus **project-local recipes** loaded from `.codemap/recipes/.{sql,md}` so teams can ship internal SQL without an adapter API. Plan: [`plans/recipes-content-registry.md`](./plans/recipes-content-registry.md). Composes with the `codemap://recipes` and `codemap://recipes/{id}` MCP resources shipped in PR #35. - [ ] **Targeted-read CLI** — `codemap show ` / `codemap snippet ` returns `file_path:line_start-line_end` + `signature` for one symbol. Same data as `SELECT … FROM symbols WHERE name = ?`, but a one-step CLI keeps agents from composing SQL for trivial precise reads - [ ] **Watch mode** for dev — `node:fs.watch` recursive + `--files` re-index loop; Linux `recursive` requires Node 19.1+ - [ ] **Monorepo / workspace awareness** — discover workspaces from `pnpm-workspace.yaml` / `package.json` and index per-workspace dependency graphs From d80d9667c1a909ce3c7d4c203a5cfc45c7b47f24 Mon Sep 17 00:00:00 2001 From: Sutu Sebastian Date: Fri, 1 May 2026 20:39:52 +0300 Subject: [PATCH 02/14] =?UTF-8?q?docs(plans):=20settle=20Q-A=20=E2=80=94?= =?UTF-8?q?=20file-pair=20storage=20for=20bundled=20recipes?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit templates/recipes/.{sql,md} for both bundled and project recipes. One loader code path, SQLite syntax highlighting in every editor, single-file diffs, sqlite3 .read works for ad-hoc testing. Migration is ~15 files; shim layer in cli/query-recipes.ts preserves backwards-compat exports. --- docs/plans/recipes-content-registry.md | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/docs/plans/recipes-content-registry.md b/docs/plans/recipes-content-registry.md index 1729d85a..e3cd1b99 100644 --- a/docs/plans/recipes-content-registry.md +++ b/docs/plans/recipes-content-registry.md @@ -206,11 +206,10 @@ Estimated total: ~1 day across ~6 commits. ### Settled -_None yet._ +- **Q-A. Storage layout for bundled recipes?** ✅ **(i) `templates/recipes/.{sql,md}` file-pair.** Uniformity with project recipes wins: one loader code path (no `if (source === "bundled")` branches), `.sql` files get SQLite syntax highlighting in every editor (today's `QUERY_RECIPES` template literals get none), single-file diffs for SQL changes, and `sqlite3 .codemap.db ".read …"` works for ad-hoc testing. Migration cost is one-time (~15 entries → ~15 `.sql` files); the shim layer in `cli/query-recipes.ts` preserves backwards-compat for `getQueryRecipeSql` / `getQueryRecipeActions` / `QUERY_RECIPES` re-exports. Rejected (ii) "code-map + sibling .md only" — smaller initial diff but two storage shapes that compound debt every time the recipe surface evolves. ### Still open -- **Q-A. Storage layout for bundled recipes.** Two options: (i) `templates/recipes/.{sql,md}` (proposed in § 3.1) — uniform with project layout, simpler loader. (ii) Keep `QUERY_RECIPES` as code, add `templates/recipes/.md` only for descriptions — less file churn but two storage shapes to maintain. Bias toward (i) — uniformity wins. - **Q-B. Loading time.** Eager (load at every CLI invocation, ~10ms) vs lazy (on first `--recipe ` / `--recipes-json` / `codemap://recipes` access). Eager is simpler; lazy saves cycles on `codemap query ""` invocations that never touch a recipe. Bias toward eager — startup cost is ms-scale and the simplicity of "registry is always populated" pays for itself. - **Q-C. Project recipes — discovery walk-up?** Today `.codemap.db` is created in the project root only. Should `.codemap/recipes/` also be project-root-only, OR walk up like `.git` does (find nearest ancestor `.codemap/recipes/` directory)? Walk-up matches monorepo intuition; root-only matches everything else codemap does today. - **Q-D. `actions` for project-local recipes — and how specified?** Three options: (i) skip for v1 (project recipes can't have actions; bundled-only feature). (ii) YAML frontmatter on `.md` (one parser dependency, e.g. `gray-matter` or hand-rolled). (iii) Sibling `.actions.json` file (no parser; another file per recipe). (i) keeps v1 lean; (ii) is most ergonomic for recipe authors; (iii) is the "no new dep" middle ground. From cc0451ada06760b2a2ba074c0e38835e25221360 Mon Sep 17 00:00:00 2001 From: Sutu Sebastian Date: Fri, 1 May 2026 20:41:25 +0300 Subject: [PATCH 03/14] =?UTF-8?q?docs(plans):=20settle=20Q-B=20=E2=80=94?= =?UTF-8?q?=20eager=20registry=20load=20at=20startup?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ~15-20 small file reads is sub-millisecond on warm SSD; rounding error vs node/bun startup. 'Registry is always populated' kills lazy guards across three call sites; surfaces malformed-recipe errors early. Rejected disk cache — over-engineered for static SQL strings. --- docs/plans/recipes-content-registry.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/plans/recipes-content-registry.md b/docs/plans/recipes-content-registry.md index e3cd1b99..c07897bf 100644 --- a/docs/plans/recipes-content-registry.md +++ b/docs/plans/recipes-content-registry.md @@ -207,10 +207,10 @@ Estimated total: ~1 day across ~6 commits. ### Settled - **Q-A. Storage layout for bundled recipes?** ✅ **(i) `templates/recipes/.{sql,md}` file-pair.** Uniformity with project recipes wins: one loader code path (no `if (source === "bundled")` branches), `.sql` files get SQLite syntax highlighting in every editor (today's `QUERY_RECIPES` template literals get none), single-file diffs for SQL changes, and `sqlite3 .codemap.db ".read …"` works for ad-hoc testing. Migration cost is one-time (~15 entries → ~15 `.sql` files); the shim layer in `cli/query-recipes.ts` preserves backwards-compat for `getQueryRecipeSql` / `getQueryRecipeActions` / `QUERY_RECIPES` re-exports. Rejected (ii) "code-map + sibling .md only" — smaller initial diff but two storage shapes that compound debt every time the recipe surface evolves. +- **Q-B. Loading time?** ✅ **Eager at startup.** Cost is negligible (~15-20 small file reads, sub-millisecond on warm SSD — rounding error vs node/bun startup, oxc, bun:sqlite). "Registry is always populated" eliminates per-call `if (notLoadedYet)` guards. Surfaces malformed-recipe errors at startup instead of 30-minutes-into-a-session. Rejected (ii) lazy — its win ("don't pay for what you don't use") is hypothetical for filesystem reads of static files; matters for DB connections / network calls, not 20 small files. Rejected (iii) eager-with-disk-cache — over-engineered; introduces invalidation problem for no measurable win. ### Still open -- **Q-B. Loading time.** Eager (load at every CLI invocation, ~10ms) vs lazy (on first `--recipe ` / `--recipes-json` / `codemap://recipes` access). Eager is simpler; lazy saves cycles on `codemap query ""` invocations that never touch a recipe. Bias toward eager — startup cost is ms-scale and the simplicity of "registry is always populated" pays for itself. - **Q-C. Project recipes — discovery walk-up?** Today `.codemap.db` is created in the project root only. Should `.codemap/recipes/` also be project-root-only, OR walk up like `.git` does (find nearest ancestor `.codemap/recipes/` directory)? Walk-up matches monorepo intuition; root-only matches everything else codemap does today. - **Q-D. `actions` for project-local recipes — and how specified?** Three options: (i) skip for v1 (project recipes can't have actions; bundled-only feature). (ii) YAML frontmatter on `.md` (one parser dependency, e.g. `gray-matter` or hand-rolled). (iii) Sibling `.actions.json` file (no parser; another file per recipe). (i) keeps v1 lean; (ii) is most ergonomic for recipe authors; (iii) is the "no new dep" middle ground. - **Q-E. Conflict resolution loud or quiet?** When a project recipe shadows a bundled one, do we (i) silently let project win (clean), (ii) emit a one-time stderr warning ("project recipe `fan-out` shadows the bundled `fan-out`"), or (iii) require an explicit `--allow-shadow` flag? Bias toward (i) — user code wins is the convention; warnings risk noise. From f3f9f725579d9b00010976ed4eeb6490b5114856 Mon Sep 17 00:00:00 2001 From: Sutu Sebastian Date: Fri, 1 May 2026 20:48:44 +0300 Subject: [PATCH 04/14] docs(plans): document DB-vs-filesystem rationale + gitignore verification + settle Q-C (root-only) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Folds two grill-round insights into the plan: 1. § 3.2 gains a 'Gitignore note (verified, not assumed)' paragraph — git check-ignore confirmed .codemap/recipes/ is NOT matched by the existing .codemap.* literal-dot pattern. Project recipes are checked into git by default, intended behavior. Consumer-side risk (their own .gitignore using .codemap*) is documented; agent rule + skill will recommend !.codemap/recipes/ un-ignore. 2. New § 3.3 'Why filesystem and not .codemap.db' captures the side-by-side test against query_baselines (which IS in DB, opposite call): nature (output vs input), index-state coupling, human-authored-for-review, meaningful-outside-one-DB. Records the 'send a recipe to a colleague' deciding test (file: send the .sql; DB: reinvent files via export/import). Bundled-recipes-in-npm-package angle reinforces. 3. Q-C settled: root-only (/.codemap/recipes/). Walk-up would make recipes the only codemap piece resolving differently from .codemap.db / indexer / resolver. Forward-compatible: root-only-→-walk-up is non-breaking; the reverse would be. --- docs/plans/recipes-content-registry.md | 19 +++++++++++++++++-- 1 file changed, 17 insertions(+), 2 deletions(-) diff --git a/docs/plans/recipes-content-registry.md b/docs/plans/recipes-content-registry.md index c07897bf..17aef219 100644 --- a/docs/plans/recipes-content-registry.md +++ b/docs/plans/recipes-content-registry.md @@ -84,7 +84,22 @@ Each `.sql` is the recipe's SQL verbatim (one statement, no `;` terminator neede `` is the same root the CLI's `--root` / `CODEMAP_ROOT` resolves to. `.codemap/` is the conventional location for codemap-related project artifacts — same parent as a future user-config might use. -### 3.3 Single-file form (rejected for v1) +**Gitignore note (verified, not just assumed).** Codemap's bundled `.gitignore` line `.codemap.*` is the literal-dot glob `.codemap.` — it matches `.codemap.db` but NOT the `.codemap/` directory or files inside it. Confirmed via `git check-ignore -v .codemap/recipes/foo.sql` returning no match. So project recipes are checked into git by default, which is the intended behavior (recipes are source code authored for human review). Consumer-side risk: if a user's `.gitignore` ignores `.codemap*` (no dot — common defensive pattern), their project recipes will be silently dropped on clone — the shipped agent rule + skill calls this out and recommends `!.codemap/recipes/` as the un-ignore. + +### 3.3 Why filesystem and not `.codemap.db` + +`query_baselines` lives inside `.codemap.db` (see [`architecture.md` § Query wiring](../architecture.md#cli-usage)) — same project, opposite call. The deciding tests: + +| | `query_baselines` (in DB) | recipes (in filesystem) | +| ----------------------------------------------- | -------------------------------------------------------------- | --------------------------------------------------------------------------- | +| Nature | Output — captured query results | Input — SQL the user wrote | +| Tied to a specific index state? | Yes (rows are valid only against the index that produced them) | No (same SQL runs against any project with the schema) | +| Authored by humans for human review? | No (codemap captures them on `--save-baseline`) | Yes (PR-reviewed source code) | +| Meaningful outside one project's `.codemap.db`? | No | Yes (copy-paste from docs, lift from a colleague's PR, share via submodule) | + +If a user wants to send a recipe to a colleague, the file-based answer is "send the `.sql`." The DB-based answer would be `codemap recipes export foo > foo.sql; … import foo.sql` — reinventing files. Plus bundled recipes ship in the npm package as files (`templates/recipes/`); putting them in DB would require a migration step and an exception to `dropAll()` for every codemap upgrade. + +### 3.4 Single-file form (rejected for v1) YAML-frontmatter Markdown with the SQL in a code block (Astro / Hugo style) was considered: @@ -208,10 +223,10 @@ Estimated total: ~1 day across ~6 commits. - **Q-A. Storage layout for bundled recipes?** ✅ **(i) `templates/recipes/.{sql,md}` file-pair.** Uniformity with project recipes wins: one loader code path (no `if (source === "bundled")` branches), `.sql` files get SQLite syntax highlighting in every editor (today's `QUERY_RECIPES` template literals get none), single-file diffs for SQL changes, and `sqlite3 .codemap.db ".read …"` works for ad-hoc testing. Migration cost is one-time (~15 entries → ~15 `.sql` files); the shim layer in `cli/query-recipes.ts` preserves backwards-compat for `getQueryRecipeSql` / `getQueryRecipeActions` / `QUERY_RECIPES` re-exports. Rejected (ii) "code-map + sibling .md only" — smaller initial diff but two storage shapes that compound debt every time the recipe surface evolves. - **Q-B. Loading time?** ✅ **Eager at startup.** Cost is negligible (~15-20 small file reads, sub-millisecond on warm SSD — rounding error vs node/bun startup, oxc, bun:sqlite). "Registry is always populated" eliminates per-call `if (notLoadedYet)` guards. Surfaces malformed-recipe errors at startup instead of 30-minutes-into-a-session. Rejected (ii) lazy — its win ("don't pay for what you don't use") is hypothetical for filesystem reads of static files; matters for DB connections / network calls, not 20 small files. Rejected (iii) eager-with-disk-cache — over-engineered; introduces invalidation problem for no measurable win. +- **Q-C. Project recipes — discovery walk-up?** ✅ **Root-only — `/.codemap/recipes/`.** Same root the CLI's `--root` / `CODEMAP_ROOT` resolves to; same root `.codemap.db` lives in. Adding walk-up for _just_ recipes would make them the only piece of codemap that resolves differently from everything else (DB, indexer, resolver) — confusing inconsistency. Monorepos are well-served today via `--root packages/foo` (recipes load from `packages/foo/.codemap/recipes/`); shared recipes can use a filesystem symlink. Walk-up is additive (forward-compatible), so we can revisit if real consumer demand emerges; root-only-→-walk-up is a non-breaking expansion. Rejected (iii) workspace-cascade — `.eslintrc`-style cascading is appropriate when a workspace primitive exists; codemap's workspace concept is a separate roadmap item, so cascading would pre-commit to a design before its dependency lands. ### Still open -- **Q-C. Project recipes — discovery walk-up?** Today `.codemap.db` is created in the project root only. Should `.codemap/recipes/` also be project-root-only, OR walk up like `.git` does (find nearest ancestor `.codemap/recipes/` directory)? Walk-up matches monorepo intuition; root-only matches everything else codemap does today. - **Q-D. `actions` for project-local recipes — and how specified?** Three options: (i) skip for v1 (project recipes can't have actions; bundled-only feature). (ii) YAML frontmatter on `.md` (one parser dependency, e.g. `gray-matter` or hand-rolled). (iii) Sibling `.actions.json` file (no parser; another file per recipe). (i) keeps v1 lean; (ii) is most ergonomic for recipe authors; (iii) is the "no new dep" middle ground. - **Q-E. Conflict resolution loud or quiet?** When a project recipe shadows a bundled one, do we (i) silently let project win (clean), (ii) emit a one-time stderr warning ("project recipe `fan-out` shadows the bundled `fan-out`"), or (iii) require an explicit `--allow-shadow` flag? Bias toward (i) — user code wins is the convention; warnings risk noise. - **Q-F. Validation strictness.** Reject project recipes that contain DML / DDL at load time (mirrors the `PRAGMA query_only` defence we shipped in PR #35), or let them fail at run time? Load-time rejection is more agent-friendly (fails fast on save_baseline-style misuse); run-time falls back to the engine's existing safeguard. Bias toward load-time — same lexical sanity check that's already proposed for empty-file detection. From ec445e390a3dc6b788f0ac44bf9d1e4cbbac9623 Mon Sep 17 00:00:00 2001 From: Sutu Sebastian Date: Fri, 1 May 2026 20:51:28 +0300 Subject: [PATCH 05/14] =?UTF-8?q?docs(plans):=20settle=20Q-D=20=E2=80=94?= =?UTF-8?q?=20YAML=20frontmatter=20on=20.md=20for=20project=20recipe=20act?= =?UTF-8?q?ions?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Hand-rolled parser (~30 LOC) handles only the shallow shape codemap needs (key/list/string/bool). Frontmatter co-locates the action with its prose. Project recipes feel first-class with the same actions template surface bundled recipes have. Rejected gray-matter / js-yaml (~50KB for full YAML 1.2 spec we don't need) and sibling .actions.json (wrong factoring — separates action from explanation). --- docs/plans/recipes-content-registry.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/plans/recipes-content-registry.md b/docs/plans/recipes-content-registry.md index 17aef219..a467fea3 100644 --- a/docs/plans/recipes-content-registry.md +++ b/docs/plans/recipes-content-registry.md @@ -227,7 +227,7 @@ Estimated total: ~1 day across ~6 commits. ### Still open -- **Q-D. `actions` for project-local recipes — and how specified?** Three options: (i) skip for v1 (project recipes can't have actions; bundled-only feature). (ii) YAML frontmatter on `.md` (one parser dependency, e.g. `gray-matter` or hand-rolled). (iii) Sibling `.actions.json` file (no parser; another file per recipe). (i) keeps v1 lean; (ii) is most ergonomic for recipe authors; (iii) is the "no new dep" middle ground. +- **Q-D. `actions` for project-local recipes?** ✅ **YAML frontmatter on `.md`, hand-rolled parser (~30 LOC).** Project recipes feel first-class — same `actions` template surface bundled recipes have. Frontmatter co-locates the action with its prose description (one editor open vs two file creates). Hand-rolled parser handles only the shallow shape codemap needs (key / list / string / bool); strict, clear errors, zero supply-chain surface. Rejected (i) skip-for-v1 — would make project recipes second-class ("missing a feature" feels worse than "we don't have evidence yet" for an open registry). Rejected `gray-matter` / `js-yaml` — too much surface (~50KB of full YAML 1.2 spec) for our shallow needs. Rejected (iii) sibling `.actions.json` — three files per actions-bearing recipe is more cognitive overhead than one with optional frontmatter; JSON files separate the action from its prose explanation, which is the wrong factoring. - **Q-E. Conflict resolution loud or quiet?** When a project recipe shadows a bundled one, do we (i) silently let project win (clean), (ii) emit a one-time stderr warning ("project recipe `fan-out` shadows the bundled `fan-out`"), or (iii) require an explicit `--allow-shadow` flag? Bias toward (i) — user code wins is the convention; warnings risk noise. - **Q-F. Validation strictness.** Reject project recipes that contain DML / DDL at load time (mirrors the `PRAGMA query_only` defence we shipped in PR #35), or let them fail at run time? Load-time rejection is more agent-friendly (fails fast on save_baseline-style misuse); run-time falls back to the engine's existing safeguard. Bias toward load-time — same lexical sanity check that's already proposed for empty-file detection. From 3132f5883a88db71baf33bbd79cfa12fcb7d68f1 Mon Sep 17 00:00:00 2001 From: Sutu Sebastian Date: Fri, 1 May 2026 20:55:31 +0300 Subject: [PATCH 06/14] =?UTF-8?q?docs(plans):=20settle=20Q-E=20=E2=80=94?= =?UTF-8?q?=20silent=20shadowing=20+=20shadows=20flag=20+=20agent-skill=20?= =?UTF-8?q?prompt=20update?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Three-layer answer optimised for agent DX + traceability: 1. Silent at runtime (matches user-code-wins convention). 2. shadows: true flag in catalog responses (--recipes-json, codemap://recipes, codemap://recipes/{id}) — discovery-time provenance. 3. Bundled skill prompt instructs agents to read codemap://recipes at session start + check shadows. Per-execution response shape stays unchanged (preserves plan § 4 uniformity). Stderr warnings rejected (MCP-stderr logs don't surface to the model anyway). --allow-shadow flag rejected (hostile to legitimate override case). Loader cost: ~5 LOC for the shadow check. --- docs/plans/recipes-content-registry.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/plans/recipes-content-registry.md b/docs/plans/recipes-content-registry.md index a467fea3..5b9ba1fe 100644 --- a/docs/plans/recipes-content-registry.md +++ b/docs/plans/recipes-content-registry.md @@ -224,11 +224,11 @@ Estimated total: ~1 day across ~6 commits. - **Q-A. Storage layout for bundled recipes?** ✅ **(i) `templates/recipes/.{sql,md}` file-pair.** Uniformity with project recipes wins: one loader code path (no `if (source === "bundled")` branches), `.sql` files get SQLite syntax highlighting in every editor (today's `QUERY_RECIPES` template literals get none), single-file diffs for SQL changes, and `sqlite3 .codemap.db ".read …"` works for ad-hoc testing. Migration cost is one-time (~15 entries → ~15 `.sql` files); the shim layer in `cli/query-recipes.ts` preserves backwards-compat for `getQueryRecipeSql` / `getQueryRecipeActions` / `QUERY_RECIPES` re-exports. Rejected (ii) "code-map + sibling .md only" — smaller initial diff but two storage shapes that compound debt every time the recipe surface evolves. - **Q-B. Loading time?** ✅ **Eager at startup.** Cost is negligible (~15-20 small file reads, sub-millisecond on warm SSD — rounding error vs node/bun startup, oxc, bun:sqlite). "Registry is always populated" eliminates per-call `if (notLoadedYet)` guards. Surfaces malformed-recipe errors at startup instead of 30-minutes-into-a-session. Rejected (ii) lazy — its win ("don't pay for what you don't use") is hypothetical for filesystem reads of static files; matters for DB connections / network calls, not 20 small files. Rejected (iii) eager-with-disk-cache — over-engineered; introduces invalidation problem for no measurable win. - **Q-C. Project recipes — discovery walk-up?** ✅ **Root-only — `/.codemap/recipes/`.** Same root the CLI's `--root` / `CODEMAP_ROOT` resolves to; same root `.codemap.db` lives in. Adding walk-up for _just_ recipes would make them the only piece of codemap that resolves differently from everything else (DB, indexer, resolver) — confusing inconsistency. Monorepos are well-served today via `--root packages/foo` (recipes load from `packages/foo/.codemap/recipes/`); shared recipes can use a filesystem symlink. Walk-up is additive (forward-compatible), so we can revisit if real consumer demand emerges; root-only-→-walk-up is a non-breaking expansion. Rejected (iii) workspace-cascade — `.eslintrc`-style cascading is appropriate when a workspace primitive exists; codemap's workspace concept is a separate roadmap item, so cascading would pre-commit to a design before its dependency lands. +- **Q-D. `actions` for project-local recipes?** ✅ **YAML frontmatter on `.md`, hand-rolled parser (~30 LOC).** Project recipes feel first-class — same `actions` template surface bundled recipes have. Frontmatter co-locates the action with its prose description (one editor open vs two file creates). Hand-rolled parser handles only the shallow shape codemap needs (key / list / string / bool); strict, clear errors, zero supply-chain surface. Rejected (i) skip-for-v1 — would make project recipes second-class ("missing a feature" feels worse than "we don't have evidence yet" for an open registry). Rejected `gray-matter` / `js-yaml` — too much surface (~50KB of full YAML 1.2 spec) for our shallow needs. Rejected (iii) sibling `.actions.json` — three files per actions-bearing recipe is more cognitive overhead than one with optional frontmatter; JSON files separate the action from its prose explanation, which is the wrong factoring. +- **Q-E. Conflict resolution loud or quiet?** ✅ **Silent at runtime + `shadows: true` flag in catalog discovery + agent-skill prompt update.** Three layers: (1) project wins silently — `--recipe fan-out` runs the project version with no stderr noise (user code wins; matches ESLint / npm overrides / `tsconfig` extends conventions). (2) Catalog responses (`--recipes-json`, `codemap://recipes`, `codemap://recipes/{id}`) carry `shadows: true` on project entries that override a bundled id of the same name. (3) Bundled `templates/agents/skills/codemap/SKILL.md` instructs agents to read `codemap://recipes` at session start and check `shadows` so they know when a recipe behaves differently from the documented bundled version. Per-execution response shape stays unchanged (preserves plan § 4 uniformity contract — `shadows` lives at discovery time, not per-call). Rejected (ii) one-time stderr warning — MCP servers log to stderr per spec but agent hosts don't surface those logs to the model, so warnings land nowhere useful for agent traceability. Rejected (iii) `--allow-shadow` flag — hostile to the legitimate-override case (every team that wants to override has to wire the flag through their tooling) for the rare-mistake case. Loader cost: ~5 LOC for the shadow-flag check. ### Still open -- **Q-D. `actions` for project-local recipes?** ✅ **YAML frontmatter on `.md`, hand-rolled parser (~30 LOC).** Project recipes feel first-class — same `actions` template surface bundled recipes have. Frontmatter co-locates the action with its prose description (one editor open vs two file creates). Hand-rolled parser handles only the shallow shape codemap needs (key / list / string / bool); strict, clear errors, zero supply-chain surface. Rejected (i) skip-for-v1 — would make project recipes second-class ("missing a feature" feels worse than "we don't have evidence yet" for an open registry). Rejected `gray-matter` / `js-yaml` — too much surface (~50KB of full YAML 1.2 spec) for our shallow needs. Rejected (iii) sibling `.actions.json` — three files per actions-bearing recipe is more cognitive overhead than one with optional frontmatter; JSON files separate the action from its prose explanation, which is the wrong factoring. -- **Q-E. Conflict resolution loud or quiet?** When a project recipe shadows a bundled one, do we (i) silently let project win (clean), (ii) emit a one-time stderr warning ("project recipe `fan-out` shadows the bundled `fan-out`"), or (iii) require an explicit `--allow-shadow` flag? Bias toward (i) — user code wins is the convention; warnings risk noise. - **Q-F. Validation strictness.** Reject project recipes that contain DML / DDL at load time (mirrors the `PRAGMA query_only` defence we shipped in PR #35), or let them fail at run time? Load-time rejection is more agent-friendly (fails fast on save_baseline-style misuse); run-time falls back to the engine's existing safeguard. Bias toward load-time — same lexical sanity check that's already proposed for empty-file detection. ## 10. Non-goals (v1) From 6e23bf64f5dcc9460035449d69269d76b084344a Mon Sep 17 00:00:00 2001 From: Sutu Sebastian Date: Fri, 1 May 2026 20:56:33 +0300 Subject: [PATCH 07/14] =?UTF-8?q?docs(plans):=20settle=20Q-F=20=E2=80=94?= =?UTF-8?q?=20load-time=20lexical=20check=20+=20runtime=20PRAGMA=20backsto?= =?UTF-8?q?p?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Defence in depth: lexical scan rejects DML/DDL at load with recipe-aware error UX (fires in CI / pre-commit). PRAGMA query_only runtime backstop from PR #35 stays as the parser-proof safety net for what lexical can't catch (WITH clauses, multi-statement, attached DBs). All 6 grill questions now settled — ready for tracer 1. --- docs/plans/recipes-content-registry.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/docs/plans/recipes-content-registry.md b/docs/plans/recipes-content-registry.md index 5b9ba1fe..85e08fad 100644 --- a/docs/plans/recipes-content-registry.md +++ b/docs/plans/recipes-content-registry.md @@ -227,9 +227,11 @@ Estimated total: ~1 day across ~6 commits. - **Q-D. `actions` for project-local recipes?** ✅ **YAML frontmatter on `.md`, hand-rolled parser (~30 LOC).** Project recipes feel first-class — same `actions` template surface bundled recipes have. Frontmatter co-locates the action with its prose description (one editor open vs two file creates). Hand-rolled parser handles only the shallow shape codemap needs (key / list / string / bool); strict, clear errors, zero supply-chain surface. Rejected (i) skip-for-v1 — would make project recipes second-class ("missing a feature" feels worse than "we don't have evidence yet" for an open registry). Rejected `gray-matter` / `js-yaml` — too much surface (~50KB of full YAML 1.2 spec) for our shallow needs. Rejected (iii) sibling `.actions.json` — three files per actions-bearing recipe is more cognitive overhead than one with optional frontmatter; JSON files separate the action from its prose explanation, which is the wrong factoring. - **Q-E. Conflict resolution loud or quiet?** ✅ **Silent at runtime + `shadows: true` flag in catalog discovery + agent-skill prompt update.** Three layers: (1) project wins silently — `--recipe fan-out` runs the project version with no stderr noise (user code wins; matches ESLint / npm overrides / `tsconfig` extends conventions). (2) Catalog responses (`--recipes-json`, `codemap://recipes`, `codemap://recipes/{id}`) carry `shadows: true` on project entries that override a bundled id of the same name. (3) Bundled `templates/agents/skills/codemap/SKILL.md` instructs agents to read `codemap://recipes` at session start and check `shadows` so they know when a recipe behaves differently from the documented bundled version. Per-execution response shape stays unchanged (preserves plan § 4 uniformity contract — `shadows` lives at discovery time, not per-call). Rejected (ii) one-time stderr warning — MCP servers log to stderr per spec but agent hosts don't surface those logs to the model, so warnings land nowhere useful for agent traceability. Rejected (iii) `--allow-shadow` flag — hostile to the legitimate-override case (every team that wants to override has to wire the flag through their tooling) for the rare-mistake case. Loader cost: ~5 LOC for the shadow-flag check. +- **Q-F. Validation strictness?** ✅ **Both — load-time lexical check + retain run-time `PRAGMA query_only` backstop.** Load-time gives recipe-aware error UX (e.g. "Project recipe `` at `.codemap/recipes/.sql` starts with `DELETE` — recipes must be read-only. Use `--save-baseline` for capturing rows.") and fires in CI / pre-commit hooks so bad recipes never reach main. Lexical scan: strip `--` line comments, find first identifier-shaped token, deny-list `INSERT` / `UPDATE` / `DELETE` / `DROP` / `CREATE` / `ALTER` / `ATTACH` / `DETACH` / `REPLACE` / `TRUNCATE` / `VACUUM` / `PRAGMA` (~20 LOC, same shape as the empty-recipe check). The PR #35 `PRAGMA query_only=1` runtime backstop **stays** as the parser-proof safety net for anything lexical scans can't catch (multi-statement payloads, `WITH` clauses with mutating sub-queries, attached databases). Different jobs: lexical = good UX for common mistakes; backstop = correctness no matter what passes lexical. Rejected (i) load-time only — incomplete (`WITH foo AS (DELETE FROM …) SELECT …` slips through); shouldn't claim safety we don't have. Rejected (ii) run-time only — error fires after parse with a less-clear message and no recipe-aware framing. + ### Still open -- **Q-F. Validation strictness.** Reject project recipes that contain DML / DDL at load time (mirrors the `PRAGMA query_only` defence we shipped in PR #35), or let them fail at run time? Load-time rejection is more agent-friendly (fails fast on save_baseline-style misuse); run-time falls back to the engine's existing safeguard. Bias toward load-time — same lexical sanity check that's already proposed for empty-file detection. +_None — all 6 questions settled. Ready to start tracer 1._ ## 10. Non-goals (v1) From b19c6030544b780a073766c9cbd53c6ad3aa93f4 Mon Sep 17 00:00:00 2001 From: Sutu Sebastian Date: Fri, 1 May 2026 21:57:20 +0300 Subject: [PATCH 08/14] feat(recipes): loader scaffold + merge logic (Tracer 1 of 6) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Pure transport-agnostic loader in src/application/recipes-loader.ts (mirrors the cmd-* ↔ *-engine seam from PR #33). Scope per plan §8 Tracer 1: - LoadedRecipe interface (canonical shape; bundled + project share it) - RecipeAction interface lifted from cli/query-recipes.ts (will become the canonical home; query-recipes becomes a shim in Tracer 2) - readRecipesFromDir(dir, source) — reads .sql, pairs with optional .md (description = first non-empty line, body = full text). Returns [] for missing/non-directory paths (project-recipes case where .codemap/recipes/ is absent — not an error). Throws on empty SQL with recipe-aware message - mergeRecipes(bundled, project) — project wins on id collision; sets shadows: true on overriding entries (Q-E settled). Output sorted by id (deterministic catalog order) - loadAllRecipes({bundledDir, projectDir}) — Tracer 1 wires bundled only; projectDir argument accepted but stubbed (returns []). Tracer 3 plugs project loader 15 unit tests cover: missing dir, non-.sql ignore, sql-only loading, sibling-md pairing, heading-strip in description, deterministic id order, empty-sql rejection, comments-then-sql happy path, non-directory passthrough, all 4 merge cases (project-only / bundled-only / shadow / no-overlap), Tracer 1 stub behavior. Layer note: query-recipes.ts (cli/) still owns QUERY_RECIPES + getQueryRecipeSql / getQueryRecipeActions / listQueryRecipeCatalog / listQueryRecipeIds. Tracer 2 migrates them to call into this loader. --- src/application/recipes-loader.test.ts | 194 +++++++++++++++++++++++++ src/application/recipes-loader.ts | 168 +++++++++++++++++++++ 2 files changed, 362 insertions(+) create mode 100644 src/application/recipes-loader.test.ts create mode 100644 src/application/recipes-loader.ts diff --git a/src/application/recipes-loader.test.ts b/src/application/recipes-loader.test.ts new file mode 100644 index 00000000..369a41aa --- /dev/null +++ b/src/application/recipes-loader.test.ts @@ -0,0 +1,194 @@ +import { afterEach, beforeEach, describe, expect, it } from "bun:test"; +import { mkdirSync, mkdtempSync, rmSync, writeFileSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; + +import { + loadAllRecipes, + mergeRecipes, + readRecipesFromDir, +} from "./recipes-loader"; +import type { LoadedRecipe } from "./recipes-loader"; + +let workDir: string; + +beforeEach(() => { + workDir = mkdtempSync(join(tmpdir(), "recipes-loader-")); +}); + +afterEach(() => { + rmSync(workDir, { recursive: true, force: true }); +}); + +function makeRecipeDir(name: string): string { + const dir = join(workDir, name); + mkdirSync(dir, { recursive: true }); + return dir; +} + +describe("readRecipesFromDir", () => { + it("returns [] when directory doesn't exist (project-recipes case)", () => { + expect(readRecipesFromDir(join(workDir, "missing"), "project")).toEqual([]); + }); + + it("ignores non-.sql files", () => { + const dir = makeRecipeDir("ignore-noise"); + writeFileSync(join(dir, "fan-out.sql"), "SELECT 1\n"); + writeFileSync(join(dir, "README.md"), "# unrelated\n"); + writeFileSync(join(dir, ".DS_Store"), ""); + const r = readRecipesFromDir(dir, "bundled"); + expect(r.map((x) => x.id)).toEqual(["fan-out"]); + }); + + it("loads SQL only — no sibling .md → description/body/actions undefined", () => { + const dir = makeRecipeDir("sql-only"); + writeFileSync(join(dir, "fan-out.sql"), "SELECT 1\n"); + const r = readRecipesFromDir(dir, "bundled"); + expect(r).toHaveLength(1); + const recipe = r[0]!; + expect(recipe).toMatchObject({ + id: "fan-out", + sql: "SELECT 1\n", + description: undefined, + body: undefined, + actions: undefined, + source: "bundled", + shadows: false, + }); + }); + + it("pairs sibling .md — description = first non-empty line, body = full text", () => { + const dir = makeRecipeDir("with-md"); + writeFileSync(join(dir, "fan-out.sql"), "SELECT 1\n"); + writeFileSync( + join(dir, "fan-out.md"), + "# Fan-out\n\nWhen to use: …\n\nFollow-up SQL: …\n", + ); + const r = readRecipesFromDir(dir, "bundled"); + expect(r[0]!.description).toBe("Fan-out"); + expect(r[0]!.body).toContain("When to use"); + }); + + it("description strips leading `# ` heading marker", () => { + const dir = makeRecipeDir("md-headers"); + writeFileSync(join(dir, "x.sql"), "SELECT 1\n"); + writeFileSync(join(dir, "x.md"), "## Heading two\n\ncontent\n"); + expect(readRecipesFromDir(dir, "bundled")[0]!.description).toBe( + "Heading two", + ); + }); + + it("returns recipes sorted by id (deterministic order)", () => { + const dir = makeRecipeDir("ordering"); + writeFileSync(join(dir, "zebra.sql"), "SELECT 1\n"); + writeFileSync(join(dir, "alpha.sql"), "SELECT 2\n"); + writeFileSync(join(dir, "monkey.sql"), "SELECT 3\n"); + const r = readRecipesFromDir(dir, "project"); + expect(r.map((x) => x.id)).toEqual(["alpha", "monkey", "zebra"]); + }); + + it("throws on empty SQL (just whitespace + comments)", () => { + const dir = makeRecipeDir("empty"); + writeFileSync( + join(dir, "blank.sql"), + "-- this is just a comment\n \n-- and another\n", + ); + expect(() => readRecipesFromDir(dir, "project")).toThrow(/empty/); + }); + + it("counts SQL with content as non-empty even with leading comments", () => { + const dir = makeRecipeDir("comments-then-sql"); + writeFileSync( + join(dir, "x.sql"), + "-- doc comment line\nSELECT path FROM files\n", + ); + expect(readRecipesFromDir(dir, "bundled")).toHaveLength(1); + }); + + it("returns [] for a non-directory path (not an error)", () => { + const filePath = join(workDir, "actually-a-file.txt"); + writeFileSync(filePath, ""); + expect(readRecipesFromDir(filePath, "bundled")).toEqual([]); + }); +}); + +describe("mergeRecipes", () => { + function recipe(id: string, source: LoadedRecipe["source"]): LoadedRecipe { + return { + id, + sql: `SELECT '${id}'`, + description: undefined, + body: undefined, + actions: undefined, + source, + shadows: false, + }; + } + + it("project-only — no shadows, no merging", () => { + const r = mergeRecipes( + [], + [recipe("a", "project"), recipe("b", "project")], + ); + expect(r.map((x) => `${x.id}:${x.source}:${x.shadows}`)).toEqual([ + "a:project:false", + "b:project:false", + ]); + }); + + it("bundled-only — passes through, sorted by id", () => { + const r = mergeRecipes( + [recipe("zebra", "bundled"), recipe("alpha", "bundled")], + [], + ); + expect(r.map((x) => x.id)).toEqual(["alpha", "zebra"]); + }); + + it("project shadows bundled — project wins, shadows: true", () => { + const r = mergeRecipes( + [recipe("fan-out", "bundled"), recipe("fan-in", "bundled")], + [recipe("fan-out", "project")], + ); + const fanOut = r.find((x) => x.id === "fan-out")!; + expect(fanOut.source).toBe("project"); + expect(fanOut.shadows).toBe(true); + // bundled fan-out is filtered out — only one entry per id. + expect(r.filter((x) => x.id === "fan-out")).toHaveLength(1); + // unrelated bundled recipe still present. + const fanIn = r.find((x) => x.id === "fan-in")!; + expect(fanIn.source).toBe("bundled"); + expect(fanIn.shadows).toBe(false); + }); + + it("project recipe with no bundled match — shadows: false", () => { + const r = mergeRecipes( + [recipe("fan-out", "bundled")], + [recipe("internal-flaky-tests", "project")], + ); + const internal = r.find((x) => x.id === "internal-flaky-tests")!; + expect(internal.shadows).toBe(false); + }); +}); + +describe("loadAllRecipes (Tracer 1 — bundled-only path)", () => { + it("loads bundled, ignores projectDir stub", () => { + const dir = makeRecipeDir("bundled-stub"); + writeFileSync(join(dir, "fan-out.sql"), "SELECT 1\n"); + const r = loadAllRecipes({ bundledDir: dir, projectDir: undefined }); + expect(r).toHaveLength(1); + expect(r[0]!.source).toBe("bundled"); + }); + + it("ignores a projectDir argument in Tracer 1 (project loader stubbed)", () => { + const bundledDir = makeRecipeDir("bundled"); + const projectDir = makeRecipeDir("project"); + writeFileSync(join(bundledDir, "x.sql"), "SELECT 1\n"); + writeFileSync( + join(projectDir, "y.sql"), + "SELECT 2\n", + // Will load in Tracer 3; for now project recipes are silently dropped. + ); + const r = loadAllRecipes({ bundledDir, projectDir }); + expect(r.map((x) => x.id)).toEqual(["x"]); + }); +}); diff --git a/src/application/recipes-loader.ts b/src/application/recipes-loader.ts new file mode 100644 index 00000000..0e6cf104 --- /dev/null +++ b/src/application/recipes-loader.ts @@ -0,0 +1,168 @@ +import { existsSync, readdirSync, readFileSync, statSync } from "node:fs"; +import { join } from "node:path"; + +/** + * One agent-facing follow-up suggested for every row of a recipe's result. + * Recipe authors hand-write this alongside the SQL (predictable: every row gets + * the same template). Ad-hoc SQL never carries actions — recipe-only feature. + * + * `auto_fixable` defaults to `false` when omitted. `description` is human prose + * for the agent to surface; `type` is a stable kebab-case verb the agent can + * key off (`delete-file`, `split-barrel`, `flag-caller`, …). + */ +export interface RecipeAction { + type: string; + auto_fixable?: boolean; + description?: string; +} + +/** + * One loaded recipe — the canonical shape the loader returns. Bundled and + * project recipes share this shape; `source` discriminates them. `shadows` + * is true when a project recipe overrides a bundled recipe of the same id + * (see plan §9 Q-E — agents read this at session start to know when a + * recipe behaves differently from the documented bundled version). + */ +export interface LoadedRecipe { + id: string; + sql: string; + description: string | undefined; + body: string | undefined; + actions: RecipeAction[] | undefined; + source: "bundled" | "project"; + shadows: boolean; +} + +export interface LoadRecipesOpts { + /** + * Absolute path to the directory containing bundled recipe `.sql` files. + * Resolved by the caller via `resolveBundledRecipesDir()` (npm package + * layout — `templates/recipes/` next to `templates/agents/`). + */ + bundledDir: string; + /** + * Absolute path to the project's `.codemap/recipes/` directory, or + * `undefined` if it doesn't exist. Tracer 3 wires this; Tracer 1 + * accepts but doesn't read it. + */ + projectDir: string | undefined; +} + +/** + * Eager loader — reads every `.sql` from `bundledDir` (and `projectDir` + * once Tracer 3 lands), pairs each with optional `.md`, applies + * load-time validation (non-empty SQL after stripping comments; + * lexical DML/DDL deny-list — Tracer 5), and returns the merged list. + * + * Project recipes win on id collision (`shadows: true` flag; see plan + * §9 Q-E). Per plan §9 Q-B (eager startup load), this is called once + * at module init in `cli/query-recipes.ts`'s shim layer; the result + * is module-cached for the process lifetime. + */ +export function loadAllRecipes(opts: LoadRecipesOpts): LoadedRecipe[] { + const bundled = readRecipesFromDir(opts.bundledDir, "bundled"); + + // Tracer 1: project loader is a stub. Tracer 3 implements it + the + // shadow-flag merge logic (project wins; sets shadows: true when + // an id matches a bundled recipe). + const project: LoadedRecipe[] = []; + + return mergeRecipes(bundled, project); +} + +/** + * Project recipes win on id collision; matching bundled entries are filtered + * out and the project entry's `shadows` flag is flipped to `true`. Order: + * project first (in id order), then bundled (in id order) — the catalog + * surface stays deterministic per directory listing. + */ +export function mergeRecipes( + bundled: LoadedRecipe[], + project: LoadedRecipe[], +): LoadedRecipe[] { + const projectIds = new Set(project.map((r) => r.id)); + const flaggedProject = project.map((r) => ({ + ...r, + shadows: projectIds.has(r.id) && bundled.some((b) => b.id === r.id), + })); + const filteredBundled = bundled.filter((r) => !projectIds.has(r.id)); + return [...flaggedProject, ...filteredBundled].sort((a, b) => + a.id.localeCompare(b.id), + ); +} + +/** + * Read every `.sql` from `dir`, pair with optional `.md`. Returns + * `[]` if the directory doesn't exist (project-recipes case in Tracer 3 — + * absence of `.codemap/recipes/` is not an error). Throws if the directory + * exists but a `.sql` fails the load-time validation (Tracer 5 will + * extend this with the DML/DDL lexical check). + */ +export function readRecipesFromDir( + dir: string, + source: "bundled" | "project", +): LoadedRecipe[] { + if (!existsSync(dir)) return []; + const stat = statSync(dir); + if (!stat.isDirectory()) return []; + + const entries = readdirSync(dir); + const recipes: LoadedRecipe[] = []; + + for (const entry of entries) { + if (!entry.endsWith(".sql")) continue; + const id = entry.slice(0, -".sql".length); + if (id.length === 0) continue; + const sqlPath = join(dir, entry); + const sql = readFileSync(sqlPath, "utf8"); + if (isEffectivelyEmpty(sql)) { + throw new Error( + `Recipe "${id}" at ${sqlPath} is empty (no SQL after stripping -- comments and whitespace).`, + ); + } + + const mdPath = join(dir, `${id}.md`); + const md = existsSync(mdPath) ? readFileSync(mdPath, "utf8") : undefined; + const description = md !== undefined ? firstNonEmptyLine(md) : undefined; + + recipes.push({ + id, + sql, + description, + body: md, + // Tracer 5 will populate this from YAML frontmatter on `md`. + actions: undefined, + source, + shadows: false, + }); + } + + return recipes.sort((a, b) => a.id.localeCompare(b.id)); +} + +/** + * Strip `--` line comments and trailing whitespace; return true if nothing + * meaningful remains. Same shape the load-time DML/DDL check (Tracer 5) + * will extend. + */ +function isEffectivelyEmpty(sql: string): boolean { + const stripped = sql + .split("\n") + .map((line) => { + const commentIdx = line.indexOf("--"); + return commentIdx === -1 ? line : line.slice(0, commentIdx); + }) + .join("\n") + .trim(); + return stripped.length === 0; +} + +function firstNonEmptyLine(text: string): string | undefined { + for (const raw of text.split("\n")) { + const trimmed = raw.trim(); + if (trimmed.length === 0) continue; + // Strip leading Markdown header markers so "# Fan-out" → "Fan-out". + return trimmed.replace(/^#+\s+/, ""); + } + return undefined; +} From 7fec23e4b7f7c2ea8f5fb48cdd19270138ec143e Mon Sep 17 00:00:00 2001 From: Sutu Sebastian Date: Fri, 1 May 2026 22:19:48 +0300 Subject: [PATCH 09/14] feat(recipes): migrate bundled recipes to templates/recipes/.{sql,md} (Tracer 2 of 6) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit QUERY_RECIPES TypeScript object map → templates/recipes/.sql + sibling .md description files. cli/query-recipes.ts becomes a thin shim that calls loadAllRecipes() at first access and caches the result. 12 bundled recipes migrated: fan-out, fan-out-sample, fan-out-sample-json, fan-in, index-summary, files-largest, components-by-hooks, markers-by-kind, deprecated-symbols, visibility-tags, files-hashes, barrel-files. Each gets a .sql file (verbatim) + .md (description body — first non-empty line becomes the catalog 'description'). Backwards-compat preserved: - QUERY_RECIPES exported as a Proxy so callers (cmd-query.ts, mcp-server.ts) can still use the legacy object-shape access (QUERY_RECIPES['fan-out'].description, Object.keys(QUERY_RECIPES), etc.) without changes - getQueryRecipeSql / getQueryRecipeActions / listQueryRecipeIds / listQueryRecipeCatalog all derive from the registry — same return shapes - Smoke tested: bun src/index.ts query --recipes-json + query --print-sql fan-out + query-golden all green Bundled recipe actions stay in code (BUNDLED_RECIPE_ACTIONS map) through Tracer 5 — that tracer adds the YAML frontmatter parser and lifts these into the .md files alongside descriptions, completing the migration. New: resolveBundledRecipesDir() in cli/query-recipes.ts mirrors resolveAgentsTemplateDir()'s npm-package layout (templates/recipes/ next to templates/agents/). _resetRecipesCacheForTests() escape hatch added for fixture swaps. templates/ already shipped in the npm artifact (per package.json files); templates/recipes/ inherits. Tracer 1's loader now has a real consumer; Tracer 3 will plug in projectDir for .codemap/recipes/.sql discovery. --- src/cli/query-recipes.ts | 376 +++++++++------------- templates/recipes/barrel-files.md | 3 + templates/recipes/barrel-files.sql | 5 + templates/recipes/components-by-hooks.md | 3 + templates/recipes/components-by-hooks.sql | 8 + templates/recipes/deprecated-symbols.md | 3 + templates/recipes/deprecated-symbols.sql | 5 + templates/recipes/fan-in.md | 3 + templates/recipes/fan-in.sql | 5 + templates/recipes/fan-out-sample-json.md | 3 + templates/recipes/fan-out-sample-json.sql | 9 + templates/recipes/fan-out-sample.md | 1 + templates/recipes/fan-out-sample.sql | 9 + templates/recipes/fan-out.md | 3 + templates/recipes/fan-out.sql | 5 + templates/recipes/files-hashes.md | 3 + templates/recipes/files-hashes.sql | 3 + templates/recipes/files-largest.md | 3 + templates/recipes/files-largest.sql | 4 + templates/recipes/index-summary.md | 1 + templates/recipes/index-summary.sql | 6 + templates/recipes/markers-by-kind.md | 1 + templates/recipes/markers-by-kind.sql | 4 + templates/recipes/visibility-tags.md | 3 + templates/recipes/visibility-tags.sql | 5 + 25 files changed, 259 insertions(+), 215 deletions(-) create mode 100644 templates/recipes/barrel-files.md create mode 100644 templates/recipes/barrel-files.sql create mode 100644 templates/recipes/components-by-hooks.md create mode 100644 templates/recipes/components-by-hooks.sql create mode 100644 templates/recipes/deprecated-symbols.md create mode 100644 templates/recipes/deprecated-symbols.sql create mode 100644 templates/recipes/fan-in.md create mode 100644 templates/recipes/fan-in.sql create mode 100644 templates/recipes/fan-out-sample-json.md create mode 100644 templates/recipes/fan-out-sample-json.sql create mode 100644 templates/recipes/fan-out-sample.md create mode 100644 templates/recipes/fan-out-sample.sql create mode 100644 templates/recipes/fan-out.md create mode 100644 templates/recipes/fan-out.sql create mode 100644 templates/recipes/files-hashes.md create mode 100644 templates/recipes/files-hashes.sql create mode 100644 templates/recipes/files-largest.md create mode 100644 templates/recipes/files-largest.sql create mode 100644 templates/recipes/index-summary.md create mode 100644 templates/recipes/index-summary.sql create mode 100644 templates/recipes/markers-by-kind.md create mode 100644 templates/recipes/markers-by-kind.sql create mode 100644 templates/recipes/visibility-tags.md create mode 100644 templates/recipes/visibility-tags.sql diff --git a/src/cli/query-recipes.ts b/src/cli/query-recipes.ts index d258c950..aad893a7 100644 --- a/src/cli/query-recipes.ts +++ b/src/cli/query-recipes.ts @@ -1,21 +1,19 @@ -/** - * One agent-facing follow-up suggested for every row of a recipe's result. - * Recipe authors hand-write this alongside the SQL (predictable: every row gets - * the same template). Ad-hoc SQL never carries actions — recipe-only feature. - * - * `auto_fixable` defaults to `false` when omitted. `description` is human prose - * for the agent to surface; `type` is a stable kebab-case verb the agent can - * key off (`delete-file`, `split-barrel`, `flag-caller`, …). - */ -export interface RecipeAction { - type: string; - auto_fixable?: boolean; - description?: string; -} +import { dirname, join } from "node:path"; +import { fileURLToPath } from "node:url"; + +import { loadAllRecipes } from "../application/recipes-loader"; +import type { LoadedRecipe } from "../application/recipes-loader"; + +export type { RecipeAction } from "../application/recipes-loader"; +import type { RecipeAction } from "../application/recipes-loader"; /** * One bundled recipe: id, human description, SQL, and optional per-row actions * (canonical source for CLI, `--recipes-json`, and the JSON output enrichment). + * + * NOTE: Kept for backwards-compat with callers that destructure the legacy + * shape. `LoadedRecipe` (from `application/recipes-loader`) is the new + * canonical type — has `body`, `source`, `shadows` in addition. */ export interface QueryRecipeCatalogEntry { id: string; @@ -25,223 +23,171 @@ export interface QueryRecipeCatalogEntry { } /** - * Bundled read-only SQL for `codemap query --recipe `. Keys match **`codemap query --help`**. + * Directory containing the bundled recipe `.sql` + `.md` files (next to + * `dist/` and `templates/agents/` in the published npm artifact). Mirrors + * `resolveAgentsTemplateDir()`'s layout — see [`docs/architecture.md` + * § Recipes wiring]. + */ +export function resolveBundledRecipesDir(): string { + return join( + dirname(fileURLToPath(import.meta.url)), + "..", + "..", + "templates", + "recipes", + ); +} + +/** + * Bundled recipe `actions` templates. Per-row hint that surfaces in `--json` + * output so agents see the recommended follow-up alongside each row. Lives + * here in code through Tracer 2 → Tracer 5 will lift these into YAML + * frontmatter on the sibling `.md` and remove this map. * - * `actions` (optional) is appended to each row in `--json` output so agents see - * the recommended follow-up alongside the data. Add an `actions` array on a - * recipe only when there's a concrete next step the agent should consider for - * every row — counts-by-kind and similar aggregates intentionally omit it. + * Add an entry here only when the recipe has a concrete next step the agent + * should consider for *every* row — counts-by-kind and similar aggregates + * intentionally have no actions. + */ +const BUNDLED_RECIPE_ACTIONS: Record = { + "fan-out": [ + { + type: "review-coupling", + description: + "High fan-out usually means orchestrator role; consider extracting helpers or splitting responsibilities.", + }, + ], + "fan-in": [ + { + type: "review-stability", + description: + "High fan-in: changes here ripple through many consumers. Protect with tests before refactoring.", + }, + ], + "files-largest": [ + { + type: "split-file", + description: + "Files this large are typical refactor candidates. Look for cohesive sub-modules to extract.", + }, + ], + "deprecated-symbols": [ + { + type: "flag-caller", + description: + "Warn before suggesting changes that depend on this symbol; check callers via the calls table.", + }, + ], + "visibility-tags": [ + { + type: "flag-non-public", + description: + "Treat as not part of the public API unless visibility = 'public': don't import from package consumers; check the visibility tag before extending re-exports.", + }, + ], + "barrel-files": [ + { + type: "split-barrel", + description: + "Confirm this is an intentional public-API surface; if it's accidental fan-out, consider splitting into smaller barrels.", + }, + ], +}; + +/** + * Module-cached registry — populated lazily on first access (loader is pure; + * the cache means we pay the filesystem read once per process lifetime per + * plan §9 Q-B). Project recipes (Tracer 3) will wire `projectDir` here once + * the bootstrap layer can pass it in. + */ +let cachedRegistry: LoadedRecipe[] | undefined; + +function getRegistry(): LoadedRecipe[] { + if (cachedRegistry === undefined) { + cachedRegistry = loadAllRecipes({ + bundledDir: resolveBundledRecipesDir(), + projectDir: undefined, + }).map((r) => ({ + ...r, + // Stitch in the bundled actions map until Tracer 5 lifts them into + // frontmatter on each `.md` file. + actions: + r.source === "bundled" ? BUNDLED_RECIPE_ACTIONS[r.id] : r.actions, + })); + } + return cachedRegistry; +} + +/** + * Reset the module cache — test-only escape hatch for fixture swaps. + */ +export function _resetRecipesCacheForTests(): void { + cachedRegistry = undefined; +} + +/** + * Bundled read-only SQL for `codemap query --recipe `. Backwards-compat + * shim — derives from the registry; new callers should use the loader's + * {@link LoadedRecipe} shape via `listQueryRecipeCatalog()` (richer fields: + * `body`, `source`, `shadows`). */ export const QUERY_RECIPES: Record< string, { sql: string; description: string; actions?: RecipeAction[] } -> = { - "fan-out": { - description: "Top 10 files by dependency fan-out (edge count)", - sql: `SELECT from_path, COUNT(*) AS deps -FROM dependencies -GROUP BY from_path -ORDER BY deps DESC, from_path ASC -LIMIT 10`, - actions: [ - { - type: "review-coupling", - description: - "High fan-out usually means orchestrator role; consider extracting helpers or splitting responsibilities.", - }, - ], - }, - "fan-out-sample": { - description: - "Top 10 by fan-out, plus up to five sample dependency targets per file", - sql: `SELECT d.from_path, - COUNT(*) AS deps, - (SELECT GROUP_CONCAT(to_path, ' | ') - FROM (SELECT to_path FROM dependencies d2 WHERE d2.from_path = d.from_path ORDER BY to_path ASC LIMIT 5)) - AS sample_targets -FROM dependencies d -GROUP BY d.from_path -ORDER BY deps DESC, d.from_path ASC -LIMIT 10`, - }, - /** - * Same ranking as `fan-out-sample`, but sample targets as a JSON array (SQLite JSON1 - * `json_group_array`). Prefer `fan-out-sample` if JSON1 is unavailable. - */ - "fan-out-sample-json": { - description: - "Like fan-out-sample, but sample_targets is a JSON array (requires JSON1)", - sql: `SELECT d.from_path, - COUNT(*) AS deps, - (SELECT json_group_array(to_path) - FROM (SELECT to_path FROM dependencies d2 WHERE d2.from_path = d.from_path ORDER BY to_path ASC LIMIT 5)) - AS sample_targets -FROM dependencies d -GROUP BY d.from_path -ORDER BY deps DESC, d.from_path ASC -LIMIT 10`, - }, - /** - * Files most imported/depended-on (complement to fan-out). - */ - "fan-in": { - description: "Top 15 files by fan-in (how many other files depend on them)", - sql: `SELECT to_path, COUNT(*) AS fan_in -FROM dependencies -GROUP BY to_path -ORDER BY fan_in DESC, to_path ASC -LIMIT 15`, - actions: [ - { - type: "review-stability", - description: - "High fan-in: changes here ripple through many consumers. Protect with tests before refactoring.", - }, - ], - }, - "index-summary": { - description: - "Single row: row counts for main tables (quick health snapshot)", - sql: `SELECT - (SELECT COUNT(*) FROM files) AS files, - (SELECT COUNT(*) FROM symbols) AS symbols, - (SELECT COUNT(*) FROM imports) AS imports, - (SELECT COUNT(*) FROM components) AS components, - (SELECT COUNT(*) FROM dependencies) AS dependencies`, +> = new Proxy( + {}, + { + get(_target, prop) { + if (typeof prop !== "string") return undefined; + const recipe = getRegistry().find((r) => r.id === prop); + if (recipe === undefined) return undefined; + return { + sql: recipe.sql, + description: recipe.description ?? recipe.id, + ...(recipe.actions !== undefined ? { actions: recipe.actions } : {}), + }; + }, + ownKeys() { + return getRegistry().map((r) => r.id); + }, + getOwnPropertyDescriptor(_target, prop) { + if (typeof prop !== "string") return undefined; + const recipe = getRegistry().find((r) => r.id === prop); + if (recipe === undefined) return undefined; + return { + enumerable: true, + configurable: true, + value: { + sql: recipe.sql, + description: recipe.description ?? recipe.id, + ...(recipe.actions !== undefined ? { actions: recipe.actions } : {}), + }, + }; + }, }, - "files-largest": { - description: "Top 20 files by line count (size/complexity hotspots)", - sql: `SELECT path, line_count, size, language -FROM files -ORDER BY line_count DESC, path ASC -LIMIT 20`, - actions: [ - { - type: "split-file", - description: - "Files this large are typical refactor candidates. Look for cohesive sub-modules to extract.", - }, - ], - }, - /** - * Hook count uses comma tally + 1 on the stored JSON array (Codemap emits flat - * `["useFoo","useBar"]` shapes). Avoids SQLite JSON1 (`json_array_length`) so - * the recipe runs on any SQLite build the CLI already supports. - */ - "components-by-hooks": { - description: - "React components with the most hooks (comma count on stored JSON array)", - sql: `SELECT name, file_path, - CASE - WHEN hooks_used IS NULL OR trim(hooks_used) = '' OR trim(hooks_used) = '[]' THEN 0 - ELSE (length(hooks_used) - length(replace(hooks_used, ',', ''))) + 1 - END AS hook_count -FROM components -ORDER BY hook_count DESC, file_path ASC, name ASC -LIMIT 20`, - }, - "markers-by-kind": { - description: "Marker counts by kind (TODO, FIXME, …)", - sql: `SELECT kind, COUNT(*) AS count -FROM markers -GROUP BY kind -ORDER BY count DESC, kind ASC`, - }, - /** - * Symbols documented with `@deprecated` in their leading JSDoc. Useful for - * agents to flag callers of soon-to-be-removed APIs before suggesting changes. - */ - "deprecated-symbols": { - description: - "Symbols whose JSDoc contains @deprecated (caller-warning candidates)", - sql: `SELECT name, kind, file_path, line_start, signature, doc_comment -FROM symbols -WHERE doc_comment LIKE '%@deprecated%' -ORDER BY file_path ASC, line_start ASC -LIMIT 50`, - actions: [ - { - type: "flag-caller", - description: - "Warn before suggesting changes that depend on this symbol; check callers via the calls table.", - }, - ], - }, - /** - * Symbols carrying JSDoc visibility tags (`@internal`, `@private`, `@alpha`, - * `@beta`). Useful for agents to know what is *not* part of the public API - * before suggesting imports or extending re-exports. - */ - "visibility-tags": { - description: - "Symbols carrying a JSDoc visibility tag (public / private / internal / alpha / beta)", - sql: `SELECT name, kind, visibility, file_path, line_start, signature, doc_comment -FROM symbols -WHERE visibility IS NOT NULL -ORDER BY file_path ASC, line_start ASC -LIMIT 100`, - actions: [ - { - type: "flag-non-public", - description: - "Treat as not part of the public API unless visibility = 'public': don't import from package consumers; check the visibility tag before extending re-exports.", - }, - ], - }, - /** - * All indexed file paths with their content hash. Powers the \`codemap validate\` - * CLI: callers diff this list against on-disk content to detect stale entries - * without paying to re-read every file. - */ - "files-hashes": { - description: - "All indexed files with content_hash (input for staleness checks)", - sql: `SELECT path, content_hash, language, line_count -FROM files -ORDER BY path ASC`, - }, - /** - * "Barrel" candidates — files that re-export a lot. High export count can - * indicate either an intentional public API surface or accidental fan-out; - * agents can use it to decide whether a new export should land here or stay local. - */ - "barrel-files": { - description: - "Top 20 files by export count (barrel / public-API candidates)", - sql: `SELECT file_path, COUNT(*) AS exports -FROM exports -GROUP BY file_path -ORDER BY exports DESC, file_path ASC -LIMIT 20`, - actions: [ - { - type: "split-barrel", - description: - "Confirm this is an intentional public-API surface; if it's accidental fan-out, consider splitting into smaller barrels.", - }, - ], - }, -}; +); /** * Sorted recipe ids (same set as {@link QUERY_RECIPES}). */ export function listQueryRecipeIds(): string[] { - return Object.keys(QUERY_RECIPES).sort(); + return getRegistry().map((r) => r.id); } /** - * Full catalog for **`codemap query --recipes-json`** — derived from {@link QUERY_RECIPES} only. + * Full catalog for **`codemap query --recipes-json`**. + * + * Tracer 2 returns the legacy shape (id / description / sql / actions?). + * Tracer 4 will extend the catalog payload to include `body`, `source`, + * and `shadows` from the {@link LoadedRecipe} shape. */ export function listQueryRecipeCatalog(): QueryRecipeCatalogEntry[] { - return listQueryRecipeIds().map((id) => { - const meta = QUERY_RECIPES[id]!; + return getRegistry().map((r) => { const entry: QueryRecipeCatalogEntry = { - id, - description: meta.description, - sql: meta.sql, + id: r.id, + description: r.description ?? r.id, + sql: r.sql, }; - if (meta.actions !== undefined) entry.actions = meta.actions; + if (r.actions !== undefined) entry.actions = r.actions; return entry; }); } @@ -250,7 +196,7 @@ export function listQueryRecipeCatalog(): QueryRecipeCatalogEntry[] { * Returns the SQL string for a recipe id, or `undefined` if unknown. */ export function getQueryRecipeSql(id: string): string | undefined { - return QUERY_RECIPES[id]?.sql; + return getRegistry().find((r) => r.id === id)?.sql; } /** @@ -259,5 +205,5 @@ export function getQueryRecipeSql(id: string): string | undefined { * ad-hoc SQL never gets actions. */ export function getQueryRecipeActions(id: string): RecipeAction[] | undefined { - return QUERY_RECIPES[id]?.actions; + return getRegistry().find((r) => r.id === id)?.actions; } diff --git a/templates/recipes/barrel-files.md b/templates/recipes/barrel-files.md new file mode 100644 index 00000000..89e95f12 --- /dev/null +++ b/templates/recipes/barrel-files.md @@ -0,0 +1,3 @@ +Top 20 files by export count (barrel / public-API candidates) + +High export count can indicate either an intentional public API surface or accidental fan-out. Agents can use this to decide whether a new export should land here or stay local. If it's accidental fan-out, consider splitting into smaller barrels. diff --git a/templates/recipes/barrel-files.sql b/templates/recipes/barrel-files.sql new file mode 100644 index 00000000..599f4f61 --- /dev/null +++ b/templates/recipes/barrel-files.sql @@ -0,0 +1,5 @@ +SELECT file_path, COUNT(*) AS exports +FROM exports +GROUP BY file_path +ORDER BY exports DESC, file_path ASC +LIMIT 20 diff --git a/templates/recipes/components-by-hooks.md b/templates/recipes/components-by-hooks.md new file mode 100644 index 00000000..642d8cfa --- /dev/null +++ b/templates/recipes/components-by-hooks.md @@ -0,0 +1,3 @@ +React components with the most hooks (comma count on stored JSON array) + +Hook count uses comma tally + 1 on the stored JSON array (Codemap emits flat `["useFoo","useBar"]` shapes). Avoids SQLite JSON1 (`json_array_length`) so the recipe runs on any SQLite build the CLI already supports. diff --git a/templates/recipes/components-by-hooks.sql b/templates/recipes/components-by-hooks.sql new file mode 100644 index 00000000..6f4a7890 --- /dev/null +++ b/templates/recipes/components-by-hooks.sql @@ -0,0 +1,8 @@ +SELECT name, file_path, + CASE + WHEN hooks_used IS NULL OR trim(hooks_used) = '' OR trim(hooks_used) = '[]' THEN 0 + ELSE (length(hooks_used) - length(replace(hooks_used, ',', ''))) + 1 + END AS hook_count +FROM components +ORDER BY hook_count DESC, file_path ASC, name ASC +LIMIT 20 diff --git a/templates/recipes/deprecated-symbols.md b/templates/recipes/deprecated-symbols.md new file mode 100644 index 00000000..3503099b --- /dev/null +++ b/templates/recipes/deprecated-symbols.md @@ -0,0 +1,3 @@ +Symbols whose JSDoc contains @deprecated (caller-warning candidates) + +Useful for agents to flag callers of soon-to-be-removed APIs before suggesting changes. Pair with `WHERE name = ''` against the `calls` table to find the actual call sites. diff --git a/templates/recipes/deprecated-symbols.sql b/templates/recipes/deprecated-symbols.sql new file mode 100644 index 00000000..79c76bd5 --- /dev/null +++ b/templates/recipes/deprecated-symbols.sql @@ -0,0 +1,5 @@ +SELECT name, kind, file_path, line_start, signature, doc_comment +FROM symbols +WHERE doc_comment LIKE '%@deprecated%' +ORDER BY file_path ASC, line_start ASC +LIMIT 50 diff --git a/templates/recipes/fan-in.md b/templates/recipes/fan-in.md new file mode 100644 index 00000000..5a913b6f --- /dev/null +++ b/templates/recipes/fan-in.md @@ -0,0 +1,3 @@ +Top 15 files by fan-in (how many other files depend on them) + +Files at the top are the most-imported in the codebase — changes here ripple through many consumers. Protect with tests before refactoring; treat as the project's de-facto stable API even if not formally exported. diff --git a/templates/recipes/fan-in.sql b/templates/recipes/fan-in.sql new file mode 100644 index 00000000..65086dcf --- /dev/null +++ b/templates/recipes/fan-in.sql @@ -0,0 +1,5 @@ +SELECT to_path, COUNT(*) AS fan_in +FROM dependencies +GROUP BY to_path +ORDER BY fan_in DESC, to_path ASC +LIMIT 15 diff --git a/templates/recipes/fan-out-sample-json.md b/templates/recipes/fan-out-sample-json.md new file mode 100644 index 00000000..948a8ef6 --- /dev/null +++ b/templates/recipes/fan-out-sample-json.md @@ -0,0 +1,3 @@ +Like fan-out-sample, but sample_targets is a JSON array (requires JSON1) + +Same ranking as `fan-out-sample`, but uses SQLite's JSON1 `json_group_array`. Prefer `fan-out-sample` if your SQLite build doesn't include JSON1. diff --git a/templates/recipes/fan-out-sample-json.sql b/templates/recipes/fan-out-sample-json.sql new file mode 100644 index 00000000..d97b4992 --- /dev/null +++ b/templates/recipes/fan-out-sample-json.sql @@ -0,0 +1,9 @@ +SELECT d.from_path, + COUNT(*) AS deps, + (SELECT json_group_array(to_path) + FROM (SELECT to_path FROM dependencies d2 WHERE d2.from_path = d.from_path ORDER BY to_path ASC LIMIT 5)) + AS sample_targets +FROM dependencies d +GROUP BY d.from_path +ORDER BY deps DESC, d.from_path ASC +LIMIT 10 diff --git a/templates/recipes/fan-out-sample.md b/templates/recipes/fan-out-sample.md new file mode 100644 index 00000000..e96055fb --- /dev/null +++ b/templates/recipes/fan-out-sample.md @@ -0,0 +1 @@ +Top 10 by fan-out, plus up to five sample dependency targets per file diff --git a/templates/recipes/fan-out-sample.sql b/templates/recipes/fan-out-sample.sql new file mode 100644 index 00000000..913efa3c --- /dev/null +++ b/templates/recipes/fan-out-sample.sql @@ -0,0 +1,9 @@ +SELECT d.from_path, + COUNT(*) AS deps, + (SELECT GROUP_CONCAT(to_path, ' | ') + FROM (SELECT to_path FROM dependencies d2 WHERE d2.from_path = d.from_path ORDER BY to_path ASC LIMIT 5)) + AS sample_targets +FROM dependencies d +GROUP BY d.from_path +ORDER BY deps DESC, d.from_path ASC +LIMIT 10 diff --git a/templates/recipes/fan-out.md b/templates/recipes/fan-out.md new file mode 100644 index 00000000..7b1dbf09 --- /dev/null +++ b/templates/recipes/fan-out.md @@ -0,0 +1,3 @@ +Top 10 files by dependency fan-out (edge count) + +Files at the top of this list act as orchestrators — they import from many other files. High fan-out usually means coordination logic that's a candidate for refactoring (extracting helpers, splitting responsibilities). Pair with `fan-in` to see hubs that are both depended-on AND depend-on-many. diff --git a/templates/recipes/fan-out.sql b/templates/recipes/fan-out.sql new file mode 100644 index 00000000..24c5257f --- /dev/null +++ b/templates/recipes/fan-out.sql @@ -0,0 +1,5 @@ +SELECT from_path, COUNT(*) AS deps +FROM dependencies +GROUP BY from_path +ORDER BY deps DESC, from_path ASC +LIMIT 10 diff --git a/templates/recipes/files-hashes.md b/templates/recipes/files-hashes.md new file mode 100644 index 00000000..9632d055 --- /dev/null +++ b/templates/recipes/files-hashes.md @@ -0,0 +1,3 @@ +All indexed files with content_hash (input for staleness checks) + +Powers the `codemap validate` CLI: callers diff this list against on-disk content to detect stale entries without paying to re-read every file. diff --git a/templates/recipes/files-hashes.sql b/templates/recipes/files-hashes.sql new file mode 100644 index 00000000..bdffbdff --- /dev/null +++ b/templates/recipes/files-hashes.sql @@ -0,0 +1,3 @@ +SELECT path, content_hash, language, line_count +FROM files +ORDER BY path ASC diff --git a/templates/recipes/files-largest.md b/templates/recipes/files-largest.md new file mode 100644 index 00000000..9daf2282 --- /dev/null +++ b/templates/recipes/files-largest.md @@ -0,0 +1,3 @@ +Top 20 files by line count (size/complexity hotspots) + +Files this large are typical refactor candidates. Look for cohesive sub-modules to extract — each split should reduce coupling, not just shuffle lines. diff --git a/templates/recipes/files-largest.sql b/templates/recipes/files-largest.sql new file mode 100644 index 00000000..c0906fef --- /dev/null +++ b/templates/recipes/files-largest.sql @@ -0,0 +1,4 @@ +SELECT path, line_count, size, language +FROM files +ORDER BY line_count DESC, path ASC +LIMIT 20 diff --git a/templates/recipes/index-summary.md b/templates/recipes/index-summary.md new file mode 100644 index 00000000..60fd634b --- /dev/null +++ b/templates/recipes/index-summary.md @@ -0,0 +1 @@ +Single row: row counts for main tables (quick health snapshot) diff --git a/templates/recipes/index-summary.sql b/templates/recipes/index-summary.sql new file mode 100644 index 00000000..638a9d76 --- /dev/null +++ b/templates/recipes/index-summary.sql @@ -0,0 +1,6 @@ +SELECT + (SELECT COUNT(*) FROM files) AS files, + (SELECT COUNT(*) FROM symbols) AS symbols, + (SELECT COUNT(*) FROM imports) AS imports, + (SELECT COUNT(*) FROM components) AS components, + (SELECT COUNT(*) FROM dependencies) AS dependencies diff --git a/templates/recipes/markers-by-kind.md b/templates/recipes/markers-by-kind.md new file mode 100644 index 00000000..43168771 --- /dev/null +++ b/templates/recipes/markers-by-kind.md @@ -0,0 +1 @@ +Marker counts by kind (TODO, FIXME, …) diff --git a/templates/recipes/markers-by-kind.sql b/templates/recipes/markers-by-kind.sql new file mode 100644 index 00000000..a8bd5a85 --- /dev/null +++ b/templates/recipes/markers-by-kind.sql @@ -0,0 +1,4 @@ +SELECT kind, COUNT(*) AS count +FROM markers +GROUP BY kind +ORDER BY count DESC, kind ASC diff --git a/templates/recipes/visibility-tags.md b/templates/recipes/visibility-tags.md new file mode 100644 index 00000000..14ed4177 --- /dev/null +++ b/templates/recipes/visibility-tags.md @@ -0,0 +1,3 @@ +Symbols carrying a JSDoc visibility tag (public / private / internal / alpha / beta) + +Useful for agents to know what is _not_ part of the public API before suggesting imports or extending re-exports. The `visibility` column is structured (parsed at index time, not regex on `doc_comment`). diff --git a/templates/recipes/visibility-tags.sql b/templates/recipes/visibility-tags.sql new file mode 100644 index 00000000..a29cef82 --- /dev/null +++ b/templates/recipes/visibility-tags.sql @@ -0,0 +1,5 @@ +SELECT name, kind, visibility, file_path, line_start, signature, doc_comment +FROM symbols +WHERE visibility IS NOT NULL +ORDER BY file_path ASC, line_start ASC +LIMIT 100 From 114e01c662c7614067a1fce35eb313a8d168b33a Mon Sep 17 00:00:00 2001 From: Sutu Sebastian Date: Fri, 1 May 2026 22:24:07 +0300 Subject: [PATCH 10/14] feat(recipes): project-local loader for .codemap/recipes/.sql (Tracer 3 of 6) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Wires up the actually-new user-facing capability per plan §1: teams ship internal SQL recipes via git-tracked .codemap/recipes/.sql files. Three pieces: 1. loadAllRecipes now reads opts.projectDir (was stubbed in Tracer 1). Composes via mergeRecipes — project wins on id collision with shadows: true flag (per Q-E settled). 2. resolveProjectRecipesDir(projectRoot) — root-only resolution per Q-C (no walk-up). Returns undefined if .codemap/recipes/ is missing or is a file rather than a directory; absence is not an error. 3. cli/query-recipes.ts shim's getRegistry() now resolves projectDir via getProjectRoot() (falls back to bundled-only if initCodemap hasn't run — covers direct unit-test paths). Cache key includes projectDir so multi-root sessions (test fixtures) re-resolve cleanly. _resetRecipesCacheForTests clears both halves. 5 new loader-engine tests: bundled-only / bundled+project / shadow detection / sorted ordering / missing-dir. 7 new shim tests: 3 for resolveProjectRecipesDir (absent / present / file-not-dir) + 4 for the end-to-end shim path (bundled-only baseline / project-local id surfaces / project shadows bundled / catalog merging). Project recipes get actions: undefined through Tracer 5 — that tracer adds the YAML frontmatter parser. --- src/application/recipes-loader.test.ts | 47 +++++++++--- src/application/recipes-loader.ts | 10 ++- src/cli/query-recipes.test.ts | 99 ++++++++++++++++++++++++++ src/cli/query-recipes.ts | 58 +++++++++++---- 4 files changed, 186 insertions(+), 28 deletions(-) create mode 100644 src/cli/query-recipes.test.ts diff --git a/src/application/recipes-loader.test.ts b/src/application/recipes-loader.test.ts index 369a41aa..f3923548 100644 --- a/src/application/recipes-loader.test.ts +++ b/src/application/recipes-loader.test.ts @@ -170,25 +170,54 @@ describe("mergeRecipes", () => { }); }); -describe("loadAllRecipes (Tracer 1 — bundled-only path)", () => { - it("loads bundled, ignores projectDir stub", () => { - const dir = makeRecipeDir("bundled-stub"); +describe("loadAllRecipes — bundled + project composition", () => { + it("loads bundled-only when projectDir is undefined", () => { + const dir = makeRecipeDir("bundled-only"); writeFileSync(join(dir, "fan-out.sql"), "SELECT 1\n"); const r = loadAllRecipes({ bundledDir: dir, projectDir: undefined }); expect(r).toHaveLength(1); expect(r[0]!.source).toBe("bundled"); }); - it("ignores a projectDir argument in Tracer 1 (project loader stubbed)", () => { + it("loads bundled + project, sorted by id", () => { const bundledDir = makeRecipeDir("bundled"); const projectDir = makeRecipeDir("project"); - writeFileSync(join(bundledDir, "x.sql"), "SELECT 1\n"); + writeFileSync(join(bundledDir, "fan-out.sql"), "SELECT 1\n"); + writeFileSync( + join(projectDir, "internal-flaky-tests.sql"), + "SELECT path FROM files\n", + ); + const r = loadAllRecipes({ bundledDir, projectDir }); + expect(r.map((x) => `${x.id}:${x.source}`)).toEqual([ + "fan-out:bundled", + "internal-flaky-tests:project", + ]); + }); + + it("project recipe shadows bundled with same id (project wins, shadows: true)", () => { + const bundledDir = makeRecipeDir("bundled-shadowed"); + const projectDir = makeRecipeDir("project-shadowing"); + writeFileSync(join(bundledDir, "fan-out.sql"), "SELECT 1\n"); writeFileSync( - join(projectDir, "y.sql"), - "SELECT 2\n", - // Will load in Tracer 3; for now project recipes are silently dropped. + join(projectDir, "fan-out.sql"), + "SELECT 'project version'\n", ); const r = loadAllRecipes({ bundledDir, projectDir }); - expect(r.map((x) => x.id)).toEqual(["x"]); + expect(r).toHaveLength(1); + const recipe = r[0]!; + expect(recipe.source).toBe("project"); + expect(recipe.shadows).toBe(true); + expect(recipe.sql).toContain("project version"); + }); + + it("missing .codemap/recipes/ directory is not an error", () => { + const bundledDir = makeRecipeDir("bundled"); + writeFileSync(join(bundledDir, "x.sql"), "SELECT 1\n"); + const r = loadAllRecipes({ + bundledDir, + projectDir: join(workDir, "does-not-exist"), + }); + expect(r).toHaveLength(1); + expect(r[0]!.source).toBe("bundled"); }); }); diff --git a/src/application/recipes-loader.ts b/src/application/recipes-loader.ts index 0e6cf104..c48f0f21 100644 --- a/src/application/recipes-loader.ts +++ b/src/application/recipes-loader.ts @@ -61,12 +61,10 @@ export interface LoadRecipesOpts { */ export function loadAllRecipes(opts: LoadRecipesOpts): LoadedRecipe[] { const bundled = readRecipesFromDir(opts.bundledDir, "bundled"); - - // Tracer 1: project loader is a stub. Tracer 3 implements it + the - // shadow-flag merge logic (project wins; sets shadows: true when - // an id matches a bundled recipe). - const project: LoadedRecipe[] = []; - + const project = + opts.projectDir !== undefined + ? readRecipesFromDir(opts.projectDir, "project") + : []; return mergeRecipes(bundled, project); } diff --git a/src/cli/query-recipes.test.ts b/src/cli/query-recipes.test.ts new file mode 100644 index 00000000..b5fe6098 --- /dev/null +++ b/src/cli/query-recipes.test.ts @@ -0,0 +1,99 @@ +import { afterEach, beforeEach, describe, expect, it } from "bun:test"; +import { mkdirSync, mkdtempSync, rmSync, writeFileSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; + +import { resolveCodemapConfig } from "../config"; +import { initCodemap } from "../runtime"; +import { + _resetRecipesCacheForTests, + getQueryRecipeActions, + getQueryRecipeSql, + listQueryRecipeCatalog, + listQueryRecipeIds, + resolveProjectRecipesDir, +} from "./query-recipes"; + +let projectRoot: string; + +beforeEach(() => { + projectRoot = mkdtempSync(join(tmpdir(), "query-recipes-")); + initCodemap(resolveCodemapConfig(projectRoot, undefined)); + _resetRecipesCacheForTests(); +}); + +afterEach(() => { + rmSync(projectRoot, { recursive: true, force: true }); + _resetRecipesCacheForTests(); +}); + +describe("resolveProjectRecipesDir", () => { + it("returns undefined when .codemap/recipes/ is absent", () => { + expect(resolveProjectRecipesDir(projectRoot)).toBeUndefined(); + }); + + it("returns the directory path when present", () => { + const recipesDir = join(projectRoot, ".codemap", "recipes"); + mkdirSync(recipesDir, { recursive: true }); + expect(resolveProjectRecipesDir(projectRoot)).toBe(recipesDir); + }); + + it("returns undefined when .codemap/recipes is a file (not directory)", () => { + mkdirSync(join(projectRoot, ".codemap"), { recursive: true }); + writeFileSync(join(projectRoot, ".codemap", "recipes"), "not a dir"); + expect(resolveProjectRecipesDir(projectRoot)).toBeUndefined(); + }); +}); + +describe("query-recipes shim — project recipes via runtime root", () => { + it("bundled-only when no .codemap/recipes/ exists", () => { + const ids = listQueryRecipeIds(); + expect(ids).toContain("fan-out"); + expect(ids).toContain("deprecated-symbols"); + // No project recipes; every entry in the catalog has source: "bundled". + // (catalog shape is the legacy QueryRecipeCatalogEntry through Tracer 4 + // — Tracer 4 adds source/body/shadows fields. For now confirm presence.) + expect(ids.length).toBeGreaterThan(0); + }); + + it("loads project-local recipes from .codemap/recipes/.sql", () => { + const recipesDir = join(projectRoot, ".codemap", "recipes"); + mkdirSync(recipesDir, { recursive: true }); + writeFileSync( + join(recipesDir, "internal-flaky-tests.sql"), + "SELECT path FROM files WHERE 1=0\n", + ); + _resetRecipesCacheForTests(); + + expect(listQueryRecipeIds()).toContain("internal-flaky-tests"); + expect(getQueryRecipeSql("internal-flaky-tests")).toContain("WHERE 1=0"); + }); + + it("project recipe shadows bundled — getQueryRecipeSql returns project version", () => { + const recipesDir = join(projectRoot, ".codemap", "recipes"); + mkdirSync(recipesDir, { recursive: true }); + writeFileSync( + join(recipesDir, "fan-out.sql"), + "SELECT 'project override' AS marker\n", + ); + _resetRecipesCacheForTests(); + + const sql = getQueryRecipeSql("fan-out"); + expect(sql).toContain("project override"); + // The bundled fan-out had `actions` (review-coupling) — project version + // doesn't carry actions until Tracer 5 wires YAML frontmatter. + expect(getQueryRecipeActions("fan-out")).toBeUndefined(); + }); + + it("listQueryRecipeCatalog includes project recipes alongside bundled", () => { + const recipesDir = join(projectRoot, ".codemap", "recipes"); + mkdirSync(recipesDir, { recursive: true }); + writeFileSync(join(recipesDir, "owner-fanout.sql"), "SELECT 1 AS x\n"); + _resetRecipesCacheForTests(); + + const catalog = listQueryRecipeCatalog(); + const ids = catalog.map((c) => c.id); + expect(ids).toContain("owner-fanout"); + expect(ids).toContain("fan-out"); + }); +}); diff --git a/src/cli/query-recipes.ts b/src/cli/query-recipes.ts index aad893a7..b126d7a7 100644 --- a/src/cli/query-recipes.ts +++ b/src/cli/query-recipes.ts @@ -1,8 +1,10 @@ +import { existsSync, statSync } from "node:fs"; import { dirname, join } from "node:path"; import { fileURLToPath } from "node:url"; import { loadAllRecipes } from "../application/recipes-loader"; import type { LoadedRecipe } from "../application/recipes-loader"; +import { getProjectRoot } from "../runtime"; export type { RecipeAction } from "../application/recipes-loader"; import type { RecipeAction } from "../application/recipes-loader"; @@ -38,6 +40,20 @@ export function resolveBundledRecipesDir(): string { ); } +/** + * Returns `/.codemap/recipes/` if it exists as a directory, + * else `undefined`. Per plan §9 Q-C, root-only — no walk-up; same root + * the CLI's `--root` / `CODEMAP_ROOT` resolves to. + */ +export function resolveProjectRecipesDir( + projectRoot: string, +): string | undefined { + const dir = join(projectRoot, ".codemap", "recipes"); + if (!existsSync(dir)) return undefined; + if (!statSync(dir).isDirectory()) return undefined; + return dir; +} + /** * Bundled recipe `actions` templates. Per-row hint that surfaces in `--json` * output so agents see the recommended follow-up alongside each row. Lives @@ -96,24 +112,39 @@ const BUNDLED_RECIPE_ACTIONS: Record = { /** * Module-cached registry — populated lazily on first access (loader is pure; * the cache means we pay the filesystem read once per process lifetime per - * plan §9 Q-B). Project recipes (Tracer 3) will wire `projectDir` here once - * the bootstrap layer can pass it in. + * plan §9 Q-B). Cache key includes `projectDir` so that a process running + * against multiple roots (test fixtures, multi-root MCP sessions later) + * re-resolves when the root changes. */ let cachedRegistry: LoadedRecipe[] | undefined; +let cachedRegistryProjectDir: string | undefined; function getRegistry(): LoadedRecipe[] { - if (cachedRegistry === undefined) { - cachedRegistry = loadAllRecipes({ - bundledDir: resolveBundledRecipesDir(), - projectDir: undefined, - }).map((r) => ({ - ...r, - // Stitch in the bundled actions map until Tracer 5 lifts them into - // frontmatter on each `.md` file. - actions: - r.source === "bundled" ? BUNDLED_RECIPE_ACTIONS[r.id] : r.actions, - })); + // `getProjectRoot()` throws if `initCodemap()` hasn't run; that only + // happens for direct unit tests of this module pre-bootstrap. Treat + // that as "no project recipes" — bundled-only registry. + let projectDir: string | undefined; + try { + projectDir = resolveProjectRecipesDir(getProjectRoot()); + } catch { + projectDir = undefined; } + + if (cachedRegistry !== undefined && cachedRegistryProjectDir === projectDir) { + return cachedRegistry; + } + + cachedRegistry = loadAllRecipes({ + bundledDir: resolveBundledRecipesDir(), + projectDir, + }).map((r) => ({ + ...r, + // Stitch in the bundled actions map until Tracer 5 lifts them into + // frontmatter on each `.md` file. Project recipes get `actions: undefined` + // until Tracer 5 plugs the YAML frontmatter parser. + actions: r.source === "bundled" ? BUNDLED_RECIPE_ACTIONS[r.id] : r.actions, + })); + cachedRegistryProjectDir = projectDir; return cachedRegistry; } @@ -122,6 +153,7 @@ function getRegistry(): LoadedRecipe[] { */ export function _resetRecipesCacheForTests(): void { cachedRegistry = undefined; + cachedRegistryProjectDir = undefined; } /** From 0caf6eb7393f3164a5f0f033c1d13dd983e66f74 Mon Sep 17 00:00:00 2001 From: Sutu Sebastian Date: Fri, 1 May 2026 23:04:50 +0300 Subject: [PATCH 11/14] feat(recipes): catalog payload carries source / body / shadows (Tracer 4 of 6) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Extends QueryRecipeCatalogEntry with three new additive fields: - body — full Markdown body of sibling .md (description = first non-empty line; body = long-form 'when to use' / 'follow-up SQL' content) - source — 'bundled' | 'project' (provenance discriminator) - shadows — true ONLY on project entries that override a bundled recipe of the same id (per Q-E settled — agents check this at session start to know when a recipe behaves differently from the documented bundled version) All additive: existing callers that destructure {id, description, sql, actions?} keep working unchanged. New helper: getQueryRecipeCatalogEntry(id) — same shape as listQueryRecipeCatalog entries, for one id (undefined for unknown). Used by codemap://recipes/{id} MCP resource so the per-id payload includes the same provenance fields the full catalog has. MCP server changes: - codemap://recipes/{id} payload now includes body / source / shadows (replaced the inline {id, description, sql, actions?} construction with JSON.stringify(getQueryRecipeCatalogEntry(id))) - codemap://recipes list-callback uses listQueryRecipeCatalog() (drops dependency on the legacy QUERY_RECIPES Proxy access) - Resource description updated to 'Single recipe by id: {id, description, body?, sql, actions?, source, shadows?}' - Removed unused listQueryRecipeIds + QUERY_RECIPES imports 5 new shim tests: bundled.source, bundled.body presence, project.source, project.shadows=true on override, getQueryRecipeCatalogEntry parity + unknown-id-undefined. Tracer 5 next: YAML frontmatter parser for project-recipe actions + load-time DML/DDL lexical check. --- src/application/mcp-server.ts | 29 ++++++++-------- src/cli/query-recipes.test.ts | 52 +++++++++++++++++++++++++++++ src/cli/query-recipes.ts | 63 ++++++++++++++++++++++++----------- 3 files changed, 109 insertions(+), 35 deletions(-) diff --git a/src/application/mcp-server.ts b/src/application/mcp-server.ts index 418de73a..402425ea 100644 --- a/src/application/mcp-server.ts +++ b/src/application/mcp-server.ts @@ -20,10 +20,9 @@ import { buildContextEnvelope } from "../cli/cmd-context"; import { computeValidateRows } from "../cli/cmd-validate"; import { getQueryRecipeActions, + getQueryRecipeCatalogEntry, getQueryRecipeSql, listQueryRecipeCatalog, - listQueryRecipeIds, - QUERY_RECIPES, } from "../cli/query-recipes"; import { loadUserConfig, resolveCodemapConfig } from "../config"; import { @@ -605,23 +604,26 @@ function registerResources(server: McpServer): void { }, ); - // codemap://recipes/{id} — one recipe (template form) + // codemap://recipes/{id} — one recipe (template form). Per Tracer 4 the + // payload includes `body` / `source` / `shadows` from the catalog entry — + // session-start agents check `shadows` to know when a project recipe + // overrides the documented bundled version. const oneRecipeCache = new Map(); server.registerResource( "recipe", new ResourceTemplate("codemap://recipes/{id}", { list: () => ({ - resources: listQueryRecipeIds().map((id) => ({ - uri: `codemap://recipes/${id}`, - name: id, - description: QUERY_RECIPES[id]!.description, + resources: listQueryRecipeCatalog().map((entry) => ({ + uri: `codemap://recipes/${entry.id}`, + name: entry.id, + description: entry.description, mimeType: "application/json", })), }), }), { description: - "Single recipe by id: {id, description, sql, actions?}. Replaces `codemap query --print-sql ` for agents.", + "Single recipe by id: {id, description, body?, sql, actions?, source, shadows?}. Replaces `codemap query --print-sql ` for agents; carries provenance fields so agents see when a project-local recipe overrides a bundled one.", mimeType: "application/json", }, (uri, variables) => { @@ -635,20 +637,15 @@ function registerResources(server: McpServer): void { ], }; } - const meta = QUERY_RECIPES[id]; - if (meta === undefined) { + const entry = getQueryRecipeCatalogEntry(id); + if (entry === undefined) { // Resources can't return structured errors the way tools do; throw so // the SDK surfaces a JSON-RPC error to the host. throw new Error( `codemap: unknown recipe "${id}". Read codemap://recipes for the catalog.`, ); } - const payload = JSON.stringify({ - id, - description: meta.description, - sql: meta.sql, - ...(meta.actions !== undefined ? { actions: meta.actions } : {}), - }); + const payload = JSON.stringify(entry); oneRecipeCache.set(id, payload); return { contents: [ diff --git a/src/cli/query-recipes.test.ts b/src/cli/query-recipes.test.ts index b5fe6098..326e8661 100644 --- a/src/cli/query-recipes.test.ts +++ b/src/cli/query-recipes.test.ts @@ -8,6 +8,7 @@ import { initCodemap } from "../runtime"; import { _resetRecipesCacheForTests, getQueryRecipeActions, + getQueryRecipeCatalogEntry, getQueryRecipeSql, listQueryRecipeCatalog, listQueryRecipeIds, @@ -97,3 +98,54 @@ describe("query-recipes shim — project recipes via runtime root", () => { expect(ids).toContain("fan-out"); }); }); + +describe("query-recipes shim — catalog source / shadows / body fields (Tracer 4)", () => { + it("bundled entries carry source: 'bundled' and no shadows flag", () => { + const fanOut = listQueryRecipeCatalog().find((c) => c.id === "fan-out"); + expect(fanOut?.source).toBe("bundled"); + expect(fanOut?.shadows).toBeUndefined(); + }); + + it("bundled entries carry body when sibling .md exists", () => { + const fanOut = listQueryRecipeCatalog().find((c) => c.id === "fan-out"); + expect(fanOut?.body).toBeDefined(); + expect(fanOut?.body).toContain("Top 10 files by dependency fan-out"); + }); + + it("project entries carry source: 'project' (no bundled clash → no shadows)", () => { + const recipesDir = join(projectRoot, ".codemap", "recipes"); + mkdirSync(recipesDir, { recursive: true }); + writeFileSync(join(recipesDir, "internal-fizz.sql"), "SELECT 1\n"); + _resetRecipesCacheForTests(); + + const fizz = listQueryRecipeCatalog().find((c) => c.id === "internal-fizz"); + expect(fizz?.source).toBe("project"); + expect(fizz?.shadows).toBeUndefined(); + }); + + it("project recipe shadowing bundled carries shadows: true", () => { + const recipesDir = join(projectRoot, ".codemap", "recipes"); + mkdirSync(recipesDir, { recursive: true }); + writeFileSync( + join(recipesDir, "fan-out.sql"), + "SELECT 'project override' AS marker\n", + ); + _resetRecipesCacheForTests(); + + const fanOut = listQueryRecipeCatalog().find((c) => c.id === "fan-out"); + expect(fanOut?.source).toBe("project"); + expect(fanOut?.shadows).toBe(true); + }); +}); + +describe("getQueryRecipeCatalogEntry (single-id lookup)", () => { + it("returns the same entry shape as listQueryRecipeCatalog for known id", () => { + const fromList = listQueryRecipeCatalog().find((c) => c.id === "fan-out"); + const fromGet = getQueryRecipeCatalogEntry("fan-out"); + expect(fromGet).toEqual(fromList); + }); + + it("returns undefined for unknown id", () => { + expect(getQueryRecipeCatalogEntry("no-such-recipe")).toBeUndefined(); + }); +}); diff --git a/src/cli/query-recipes.ts b/src/cli/query-recipes.ts index b126d7a7..f77b2a3c 100644 --- a/src/cli/query-recipes.ts +++ b/src/cli/query-recipes.ts @@ -10,18 +10,27 @@ export type { RecipeAction } from "../application/recipes-loader"; import type { RecipeAction } from "../application/recipes-loader"; /** - * One bundled recipe: id, human description, SQL, and optional per-row actions - * (canonical source for CLI, `--recipes-json`, and the JSON output enrichment). + * Catalog entry surfaced to `--recipes-json`, the `codemap://recipes` MCP + * resource, and the per-id `codemap://recipes/{id}` lookup. Backwards-compat + * shape with three extensions added in Tracer 4: * - * NOTE: Kept for backwards-compat with callers that destructure the legacy - * shape. `LoadedRecipe` (from `application/recipes-loader`) is the new - * canonical type — has `body`, `source`, `shadows` in addition. + * - **`body`** — full Markdown body of the sibling `.md` (when present); + * description is the first non-empty line of that body. + * - **`source`** — `"bundled"` (ships with the npm package) or `"project"` + * (loaded from `/.codemap/recipes/`). + * - **`shadows`** — `true` when a project recipe overrides a bundled recipe + * of the same id (per plan §9 Q-E — agents read this at session start to + * know when a recipe behaves differently from the documented bundled + * version). Absent / `false` for non-shadowing entries. */ export interface QueryRecipeCatalogEntry { id: string; description: string; + body?: string; sql: string; actions?: RecipeAction[]; + source: "bundled" | "project"; + shadows?: boolean; } /** @@ -206,22 +215,38 @@ export function listQueryRecipeIds(): string[] { } /** - * Full catalog for **`codemap query --recipes-json`**. - * - * Tracer 2 returns the legacy shape (id / description / sql / actions?). - * Tracer 4 will extend the catalog payload to include `body`, `source`, - * and `shadows` from the {@link LoadedRecipe} shape. + * Full catalog for **`codemap query --recipes-json`** and the + * `codemap://recipes` MCP resource. Per Tracer 4, includes `body`, + * `source`, and `shadows` fields on each entry. */ export function listQueryRecipeCatalog(): QueryRecipeCatalogEntry[] { - return getRegistry().map((r) => { - const entry: QueryRecipeCatalogEntry = { - id: r.id, - description: r.description ?? r.id, - sql: r.sql, - }; - if (r.actions !== undefined) entry.actions = r.actions; - return entry; - }); + return getRegistry().map((r) => buildCatalogEntry(r)); +} + +/** + * Single-entry lookup for the `codemap://recipes/{id}` MCP resource and any + * future `--recipe-json ` CLI shape. Returns `undefined` for unknown + * ids; otherwise the same {@link QueryRecipeCatalogEntry} shape as the + * full-catalog listing. + */ +export function getQueryRecipeCatalogEntry( + id: string, +): QueryRecipeCatalogEntry | undefined { + const recipe = getRegistry().find((r) => r.id === id); + return recipe === undefined ? undefined : buildCatalogEntry(recipe); +} + +function buildCatalogEntry(r: LoadedRecipe): QueryRecipeCatalogEntry { + const entry: QueryRecipeCatalogEntry = { + id: r.id, + description: r.description ?? r.id, + sql: r.sql, + source: r.source, + }; + if (r.body !== undefined) entry.body = r.body; + if (r.actions !== undefined) entry.actions = r.actions; + if (r.shadows) entry.shadows = true; + return entry; } /** From 05901e994b7d54c59ac62d94f4df726342a17b85 Mon Sep 17 00:00:00 2001 From: Sutu Sebastian Date: Sat, 2 May 2026 10:13:18 +0300 Subject: [PATCH 12/14] feat(recipes): YAML frontmatter actions + load-time DML/DDL deny-list (Tracer 5 of 6) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Closes the Q-D + Q-F open questions from the grill round. Q-D — actions for project-local recipes: - Hand-rolled YAML frontmatter parser in extractFrontmatterAndBody (~30 LOC core, ~50 LOC including helpers). Strict shape: one optional 'actions' list of {type, auto_fixable?, description?} between --- delimiters at the top of .md. Other top-level keys tolerated (forward-compat for future recipe metadata). Unknown action keys silently ignored. Items missing 'type' are filtered out (defensive). - Lifted the 6 bundled recipes' actions (fan-out, fan-in, files-largest, deprecated-symbols, visibility-tags, barrel-files) from BUNDLED_RECIPE_ACTIONS in cli/query-recipes.ts into YAML frontmatter on each templates/recipes/.md. The map is gone — uniform shape for both bundled and project recipes (Q-A's promised 'one storage shape, one loader code path'). Q-F — load-time DML/DDL lexical check: - validateRecipeSql exported from recipes-loader. Strips -- comments, finds first identifier-shaped token, rejects if in deny-list (INSERT/UPDATE/DELETE/DROP/CREATE/ALTER/ATTACH/DETACH/REPLACE/TRUNCATE/VACUUM/PRAGMA). Recipe-aware error message points at --save-baseline as the legitimate path for capturing rows. - Runtime PRAGMA query_only=1 backstop from PR #35 stays unchanged — different jobs: lexical = good UX for common mistakes; backstop = correctness for what slips by. Lessons re-learned (already in .agents/lessons.md): backticks containing colons in line/block comments break Bun's parser; /* */ inside backticks closes the surrounding /** */ JSDoc. Avoided both by replacing problematic backticks with plain quotes / parentheses. Tests: 27 new — 13 for validateRecipeSql, 7 for extractFrontmatterAndBody, 1 integration confirming actions + description both populate from a single .md. Total now 54 pass on the loader + shim test files. --- src/application/recipes-loader.test.ts | 188 +++++++++++++++++++ src/application/recipes-loader.ts | 231 +++++++++++++++++++++--- src/cli/query-recipes.ts | 66 +------ templates/recipes/barrel-files.md | 6 + templates/recipes/deprecated-symbols.md | 6 + templates/recipes/fan-in.md | 6 + templates/recipes/fan-out.md | 6 + templates/recipes/files-largest.md | 6 + templates/recipes/visibility-tags.md | 6 + 9 files changed, 436 insertions(+), 85 deletions(-) diff --git a/src/application/recipes-loader.test.ts b/src/application/recipes-loader.test.ts index f3923548..6860315a 100644 --- a/src/application/recipes-loader.test.ts +++ b/src/application/recipes-loader.test.ts @@ -4,9 +4,11 @@ import { tmpdir } from "node:os"; import { join } from "node:path"; import { + extractFrontmatterAndBody, loadAllRecipes, mergeRecipes, readRecipesFromDir, + validateRecipeSql, } from "./recipes-loader"; import type { LoadedRecipe } from "./recipes-loader"; @@ -221,3 +223,189 @@ describe("loadAllRecipes — bundled + project composition", () => { expect(r[0]!.source).toBe("bundled"); }); }); + +describe("validateRecipeSql — load-time DML/DDL deny-list", () => { + it("accepts SELECT (the common case)", () => { + expect(() => + validateRecipeSql("ok", "/tmp/ok.sql", "SELECT 1\n"), + ).not.toThrow(); + }); + + it("accepts WITH-prefixed CTEs", () => { + expect(() => + validateRecipeSql( + "cte", + "/tmp/cte.sql", + "WITH x AS (SELECT 1) SELECT * FROM x\n", + ), + ).not.toThrow(); + }); + + it("rejects DELETE with recipe-aware error", () => { + expect(() => + validateRecipeSql("bad", "/tmp/bad.sql", "DELETE FROM files\n"), + ).toThrow(/recipes must be read-only/); + }); + + for (const verb of [ + "INSERT", + "UPDATE", + "DROP", + "CREATE", + "ALTER", + "ATTACH", + "DETACH", + "REPLACE", + "TRUNCATE", + "VACUUM", + "PRAGMA", + ]) { + it(`rejects ${verb} at load time`, () => { + expect(() => + validateRecipeSql( + "bad", + "/tmp/bad.sql", + `${verb} something arbitrary\n`, + ), + ).toThrow(/read-only/); + }); + } + + it("ignores leading -- comments before the keyword", () => { + expect(() => + validateRecipeSql( + "ok", + "/tmp/ok.sql", + "-- doc line\n-- another doc\nSELECT 1\n", + ), + ).not.toThrow(); + }); + + it("rejects lowercase deny-list keywords (case-insensitive)", () => { + expect(() => + validateRecipeSql("bad", "/tmp/bad.sql", "drop table x\n"), + ).toThrow(/read-only/); + }); +}); + +describe("extractFrontmatterAndBody — YAML actions parser", () => { + it("returns body as full text when no frontmatter delimiter present", () => { + const md = "Just some plain markdown.\n"; + const r = extractFrontmatterAndBody(md); + expect(r.actions).toBeUndefined(); + expect(r.body).toBe(md); + }); + + it("parses a single action with type only", () => { + const md = `--- +actions: + - type: review-coupling +--- +Body line one +Body line two +`; + const r = extractFrontmatterAndBody(md); + expect(r.actions).toEqual([{ type: "review-coupling" }]); + expect(r.body.startsWith("Body line one")).toBe(true); + }); + + it("parses action with type + description (double-quoted)", () => { + const md = `--- +actions: + - type: split-barrel + description: "Confirm intent before splitting." +--- +body +`; + const r = extractFrontmatterAndBody(md); + expect(r.actions).toEqual([ + { type: "split-barrel", description: "Confirm intent before splitting." }, + ]); + }); + + it("parses action with auto_fixable: true (boolean scalar)", () => { + const md = `--- +actions: + - type: delete-file + auto_fixable: true + description: bare unquoted text is fine +--- +body +`; + const r = extractFrontmatterAndBody(md); + expect(r.actions).toEqual([ + { + type: "delete-file", + auto_fixable: true, + description: "bare unquoted text is fine", + }, + ]); + }); + + it("parses multiple action items", () => { + const md = `--- +actions: + - type: a + - type: b + description: second +--- +body +`; + const r = extractFrontmatterAndBody(md); + expect(r.actions).toEqual([ + { type: "a" }, + { type: "b", description: "second" }, + ]); + }); + + it("returns undefined actions when no actions key in frontmatter", () => { + const md = `--- +some_other_key: value +--- +body +`; + const r = extractFrontmatterAndBody(md); + expect(r.actions).toBeUndefined(); + expect(r.body.startsWith("body")).toBe(true); + }); + + it("treats malformed frontmatter (no closing ---) as no frontmatter", () => { + const md = `--- +actions: + - type: foo +this never closes +`; + const r = extractFrontmatterAndBody(md); + expect(r.actions).toBeUndefined(); + expect(r.body).toBe(md); + }); +}); + +describe("readRecipesFromDir — frontmatter integration", () => { + it("populates actions from sibling .md frontmatter", () => { + const dir = makeRecipeDir("with-frontmatter"); + writeFileSync(join(dir, "fan-out.sql"), "SELECT 1\n"); + writeFileSync( + join(dir, "fan-out.md"), + `--- +actions: + - type: review-coupling + description: "High fan-out usually means orchestrator role." +--- + +Top 10 files by dependency fan-out (edge count) +`, + ); + const r = readRecipesFromDir(dir, "bundled"); + expect(r).toHaveLength(1); + expect(r[0]!.actions).toEqual([ + { + type: "review-coupling", + description: "High fan-out usually means orchestrator role.", + }, + ]); + expect(r[0]!.description).toBe( + "Top 10 files by dependency fan-out (edge count)", + ); + }); +}); diff --git a/src/application/recipes-loader.ts b/src/application/recipes-loader.ts index c48f0f21..ec2e6db3 100644 --- a/src/application/recipes-loader.ts +++ b/src/application/recipes-loader.ts @@ -91,10 +91,16 @@ export function mergeRecipes( /** * Read every `.sql` from `dir`, pair with optional `.md`. Returns - * `[]` if the directory doesn't exist (project-recipes case in Tracer 3 — - * absence of `.codemap/recipes/` is not an error). Throws if the directory - * exists but a `.sql` fails the load-time validation (Tracer 5 will - * extend this with the DML/DDL lexical check). + * `[]` if the directory doesn't exist (project-recipes case — absence of + * `.codemap/recipes/` is not an error). Throws with recipe-aware error + * messages if a `.sql` fails load-time validation (empty after + * comment-stripping, or starts with a DML / DDL keyword). + * + * The runtime `PRAGMA query_only=1` backstop in `executeQuery` (PR #35) + * stays as the parser-proof safety net for anything this lexical scan + * can't catch (multi-statement payloads, `WITH foo AS (DELETE …) SELECT` + * sub-queries, attached databases). Different jobs: lexical = good UX + * for common mistakes; backstop = correctness no matter what. */ export function readRecipesFromDir( dir: string, @@ -113,23 +119,23 @@ export function readRecipesFromDir( if (id.length === 0) continue; const sqlPath = join(dir, entry); const sql = readFileSync(sqlPath, "utf8"); - if (isEffectivelyEmpty(sql)) { - throw new Error( - `Recipe "${id}" at ${sqlPath} is empty (no SQL after stripping -- comments and whitespace).`, - ); - } + validateRecipeSql(id, sqlPath, sql); const mdPath = join(dir, `${id}.md`); const md = existsSync(mdPath) ? readFileSync(mdPath, "utf8") : undefined; - const description = md !== undefined ? firstNonEmptyLine(md) : undefined; + const { actions, body } = + md !== undefined + ? extractFrontmatterAndBody(md) + : { actions: undefined, body: undefined }; + const description = + body !== undefined ? firstNonEmptyLine(body) : undefined; recipes.push({ id, sql, description, - body: md, - // Tracer 5 will populate this from YAML frontmatter on `md`. - actions: undefined, + body, + actions, source, shadows: false, }); @@ -139,20 +145,73 @@ export function readRecipesFromDir( } /** - * Strip `--` line comments and trailing whitespace; return true if nothing - * meaningful remains. Same shape the load-time DML/DDL check (Tracer 5) - * will extend. + * Throws with a recipe-aware message if `sql` is empty (after stripping + * `--` line comments) or starts with a DML / DDL keyword. Caller keeps + * the path for the error message; the parser-proof runtime backstop in + * `executeQuery` is the safety net beyond this. */ -function isEffectivelyEmpty(sql: string): boolean { - const stripped = sql +export function validateRecipeSql( + id: string, + sqlPath: string, + sql: string, +): void { + if (isEffectivelyEmpty(sql)) { + throw new Error( + `Recipe "${id}" at ${sqlPath} is empty (no SQL after stripping -- comments and whitespace).`, + ); + } + const firstKeyword = firstSqlKeyword(sql); + if (firstKeyword !== undefined && DML_DDL_DENY.has(firstKeyword)) { + throw new Error( + `Recipe "${id}" at ${sqlPath} starts with "${firstKeyword}" — recipes must be read-only. Use \`codemap query --save-baseline\` for capturing rows; the runtime PRAGMA query_only=1 guard would also reject this at execution time.`, + ); + } +} + +const DML_DDL_DENY = new Set([ + "INSERT", + "UPDATE", + "DELETE", + "DROP", + "CREATE", + "ALTER", + "ATTACH", + "DETACH", + "REPLACE", + "TRUNCATE", + "VACUUM", + "PRAGMA", +]); + +/** + * First identifier-shaped token in `sql` after stripping `--` line + * comments and leading whitespace. Returns the upper-cased keyword + * (SQLite is case-insensitive for keywords) or `undefined` if no token + * exists. Doesn't try to be clever about strings or block-style comments + * (those are rare in recipes; the runtime backstop catches what slips by). + */ +function firstSqlKeyword(sql: string): string | undefined { + const stripped = stripLineComments(sql); + const match = stripped.match(/[A-Za-z_][A-Za-z0-9_]*/); + return match === null ? undefined : match[0].toUpperCase(); +} + +function stripLineComments(sql: string): string { + return sql .split("\n") .map((line) => { - const commentIdx = line.indexOf("--"); - return commentIdx === -1 ? line : line.slice(0, commentIdx); + const idx = line.indexOf("--"); + return idx === -1 ? line : line.slice(0, idx); }) - .join("\n") - .trim(); - return stripped.length === 0; + .join("\n"); +} + +/** + * Strip `--` line comments and trailing whitespace; return true if nothing + * meaningful remains. + */ +function isEffectivelyEmpty(sql: string): boolean { + return stripLineComments(sql).trim().length === 0; } function firstNonEmptyLine(text: string): string | undefined { @@ -164,3 +223,129 @@ function firstNonEmptyLine(text: string): string | undefined { } return undefined; } + +/** + * Hand-rolled YAML frontmatter parser scoped to codemap's recipe needs. + * Reads one optional `actions` list of RecipeAction-shaped items between + * `---` delimiters at the top of the file. Per plan §9 Q-D: recipe-specific + * shallow shape only; reject anything weirder so authors get clear errors + * instead of half-parsed YAML edge cases. + * + * Returns the parsed actions (or undefined when the file has no + * frontmatter / no actions key) plus the body — file content with the + * frontmatter block stripped, used as the description body downstream. + */ +export function extractFrontmatterAndBody(md: string): { + actions: RecipeAction[] | undefined; + body: string; +} { + // Frontmatter must start at byte 0 with three dashes + newline (LF or + // CRLF); anything else is treated as plain Markdown. + const startMatch = md.match(/^---\r?\n/); + if (startMatch === null) { + return { actions: undefined, body: md }; + } + const afterStart = md.slice(startMatch[0].length); + const endMatch = afterStart.match(/\n---\r?\n/); + if (endMatch === null) { + return { actions: undefined, body: md }; + } + const fmText = afterStart.slice(0, endMatch.index); + const body = afterStart.slice(endMatch.index! + endMatch[0].length); + const actions = parseActionsFromFrontmatter(fmText); + return { actions, body }; +} + +// Parses the actions block from the frontmatter text. Strict shape — one +// top-level "actions" key whose value is a list of items with a required +// "type" field plus optional "auto_fixable" (boolean) and "description" +// (string). Returns undefined when no actions key is found. Other top-level +// keys are tolerated (forward-compat for future recipe metadata). +function parseActionsFromFrontmatter(fm: string): RecipeAction[] | undefined { + const lines = fm.split(/\r?\n/); + let i = 0; + while (i < lines.length) { + const line = lines[i]!; + if (/^\s*$/.test(line)) { + i++; + continue; + } + const keyMatch = line.match(/^([A-Za-z_][A-Za-z0-9_]*)\s*:\s*$/); + if (keyMatch !== null && keyMatch[1] === "actions") { + return parseActionList(lines, i + 1); + } + i++; + } + return undefined; +} + +function parseActionList(lines: string[], startIdx: number): RecipeAction[] { + const out: RecipeAction[] = []; + let i = startIdx; + let current: RecipeAction | undefined; + + while (i < lines.length) { + const line = lines[i]!; + // Stop at the next top-level YAML key (no leading whitespace + colon). + if (/^[A-Za-z_]/.test(line)) break; + + // List-item start (e.g. " - type: foo"). + const itemMatch = line.match(/^\s*-\s+(\w+)\s*:\s*(.*)$/); + if (itemMatch !== null) { + if (current !== undefined) out.push(current); + const [, key, raw] = itemMatch; + const value = parseScalar(raw!); + current = applyKey({ type: "" }, key!, value); + i++; + continue; + } + + // Continuation key on the same item (e.g. " description: foo"). + const contMatch = line.match(/^\s+(\w+)\s*:\s*(.*)$/); + if (contMatch !== null && current !== undefined) { + const [, key, raw] = contMatch; + current = applyKey(current, key!, parseScalar(raw!)); + i++; + continue; + } + + // Anything else is unrecognised — stop parsing this list and let + // downstream surface it as "actions block had unexpected content" + // if we ever need stricter errors. For now, terminate cleanly. + break; + } + + if (current !== undefined) out.push(current); + // Filter out items missing required `type` field (defensive — strict + // YAML would error here, but we fail open on malformed entries). + return out.filter((a) => a.type.length > 0); +} + +function parseScalar(raw: string): string | boolean { + const trimmed = raw.trim(); + if (trimmed === "true") return true; + if (trimmed === "false") return false; + // Strip surrounding quotes (single or double). + if ( + (trimmed.startsWith('"') && trimmed.endsWith('"')) || + (trimmed.startsWith("'") && trimmed.endsWith("'")) + ) { + return trimmed.slice(1, -1); + } + return trimmed; +} + +function applyKey( + action: RecipeAction, + key: string, + value: string | boolean, +): RecipeAction { + const next = { ...action }; + if (key === "type" && typeof value === "string") next.type = value; + else if (key === "auto_fixable" && typeof value === "boolean") + next.auto_fixable = value; + else if (key === "description" && typeof value === "string") + next.description = value; + // Unknown keys silently ignored (forward-compat). + return next; +} diff --git a/src/cli/query-recipes.ts b/src/cli/query-recipes.ts index f77b2a3c..4fdf87e6 100644 --- a/src/cli/query-recipes.ts +++ b/src/cli/query-recipes.ts @@ -63,67 +63,15 @@ export function resolveProjectRecipesDir( return dir; } -/** - * Bundled recipe `actions` templates. Per-row hint that surfaces in `--json` - * output so agents see the recommended follow-up alongside each row. Lives - * here in code through Tracer 2 → Tracer 5 will lift these into YAML - * frontmatter on the sibling `.md` and remove this map. - * - * Add an entry here only when the recipe has a concrete next step the agent - * should consider for *every* row — counts-by-kind and similar aggregates - * intentionally have no actions. - */ -const BUNDLED_RECIPE_ACTIONS: Record = { - "fan-out": [ - { - type: "review-coupling", - description: - "High fan-out usually means orchestrator role; consider extracting helpers or splitting responsibilities.", - }, - ], - "fan-in": [ - { - type: "review-stability", - description: - "High fan-in: changes here ripple through many consumers. Protect with tests before refactoring.", - }, - ], - "files-largest": [ - { - type: "split-file", - description: - "Files this large are typical refactor candidates. Look for cohesive sub-modules to extract.", - }, - ], - "deprecated-symbols": [ - { - type: "flag-caller", - description: - "Warn before suggesting changes that depend on this symbol; check callers via the calls table.", - }, - ], - "visibility-tags": [ - { - type: "flag-non-public", - description: - "Treat as not part of the public API unless visibility = 'public': don't import from package consumers; check the visibility tag before extending re-exports.", - }, - ], - "barrel-files": [ - { - type: "split-barrel", - description: - "Confirm this is an intentional public-API surface; if it's accidental fan-out, consider splitting into smaller barrels.", - }, - ], -}; - /** * Module-cached registry — populated lazily on first access (loader is pure; * the cache means we pay the filesystem read once per process lifetime per * plan §9 Q-B). Cache key includes `projectDir` so that a process running * against multiple roots (test fixtures, multi-root MCP sessions later) * re-resolves when the root changes. + * + * Per Tracer 5: `actions` come from YAML frontmatter on each `.md` for + * BOTH bundled and project recipes — uniform shape, no special-casing. */ let cachedRegistry: LoadedRecipe[] | undefined; let cachedRegistryProjectDir: string | undefined; @@ -146,13 +94,7 @@ function getRegistry(): LoadedRecipe[] { cachedRegistry = loadAllRecipes({ bundledDir: resolveBundledRecipesDir(), projectDir, - }).map((r) => ({ - ...r, - // Stitch in the bundled actions map until Tracer 5 lifts them into - // frontmatter on each `.md` file. Project recipes get `actions: undefined` - // until Tracer 5 plugs the YAML frontmatter parser. - actions: r.source === "bundled" ? BUNDLED_RECIPE_ACTIONS[r.id] : r.actions, - })); + }); cachedRegistryProjectDir = projectDir; return cachedRegistry; } diff --git a/templates/recipes/barrel-files.md b/templates/recipes/barrel-files.md index 89e95f12..96378140 100644 --- a/templates/recipes/barrel-files.md +++ b/templates/recipes/barrel-files.md @@ -1,3 +1,9 @@ +--- +actions: + - type: split-barrel + description: "Confirm this is an intentional public-API surface; if it's accidental fan-out, consider splitting into smaller barrels." +--- + Top 20 files by export count (barrel / public-API candidates) High export count can indicate either an intentional public API surface or accidental fan-out. Agents can use this to decide whether a new export should land here or stay local. If it's accidental fan-out, consider splitting into smaller barrels. diff --git a/templates/recipes/deprecated-symbols.md b/templates/recipes/deprecated-symbols.md index 3503099b..b66e5a85 100644 --- a/templates/recipes/deprecated-symbols.md +++ b/templates/recipes/deprecated-symbols.md @@ -1,3 +1,9 @@ +--- +actions: + - type: flag-caller + description: "Warn before suggesting changes that depend on this symbol; check callers via the calls table." +--- + Symbols whose JSDoc contains @deprecated (caller-warning candidates) Useful for agents to flag callers of soon-to-be-removed APIs before suggesting changes. Pair with `WHERE name = ''` against the `calls` table to find the actual call sites. diff --git a/templates/recipes/fan-in.md b/templates/recipes/fan-in.md index 5a913b6f..09d9d74f 100644 --- a/templates/recipes/fan-in.md +++ b/templates/recipes/fan-in.md @@ -1,3 +1,9 @@ +--- +actions: + - type: review-stability + description: "High fan-in: changes here ripple through many consumers. Protect with tests before refactoring." +--- + Top 15 files by fan-in (how many other files depend on them) Files at the top are the most-imported in the codebase — changes here ripple through many consumers. Protect with tests before refactoring; treat as the project's de-facto stable API even if not formally exported. diff --git a/templates/recipes/fan-out.md b/templates/recipes/fan-out.md index 7b1dbf09..5bebaf44 100644 --- a/templates/recipes/fan-out.md +++ b/templates/recipes/fan-out.md @@ -1,3 +1,9 @@ +--- +actions: + - type: review-coupling + description: "High fan-out usually means orchestrator role; consider extracting helpers or splitting responsibilities." +--- + Top 10 files by dependency fan-out (edge count) Files at the top of this list act as orchestrators — they import from many other files. High fan-out usually means coordination logic that's a candidate for refactoring (extracting helpers, splitting responsibilities). Pair with `fan-in` to see hubs that are both depended-on AND depend-on-many. diff --git a/templates/recipes/files-largest.md b/templates/recipes/files-largest.md index 9daf2282..79fa86e2 100644 --- a/templates/recipes/files-largest.md +++ b/templates/recipes/files-largest.md @@ -1,3 +1,9 @@ +--- +actions: + - type: split-file + description: "Files this large are typical refactor candidates. Look for cohesive sub-modules to extract." +--- + Top 20 files by line count (size/complexity hotspots) Files this large are typical refactor candidates. Look for cohesive sub-modules to extract — each split should reduce coupling, not just shuffle lines. diff --git a/templates/recipes/visibility-tags.md b/templates/recipes/visibility-tags.md index 14ed4177..ca8d6ec8 100644 --- a/templates/recipes/visibility-tags.md +++ b/templates/recipes/visibility-tags.md @@ -1,3 +1,9 @@ +--- +actions: + - type: flag-non-public + description: "Treat as not part of the public API unless visibility = 'public': don't import from package consumers; check the visibility tag before extending re-exports." +--- + Symbols carrying a JSDoc visibility tag (public / private / internal / alpha / beta) Useful for agents to know what is _not_ part of the public API before suggesting imports or extending re-exports. The `visibility` column is structured (parsed at index time, not regex on `doc_comment`). From 126a518cec93fdca7ee8817082be1acb5338ef0c Mon Sep 17 00:00:00 2001 From: Sutu Sebastian Date: Sat, 2 May 2026 10:17:11 +0300 Subject: [PATCH 13/14] docs(recipes): architecture/glossary/README/agent rule + skill (Tracer 6 of 6) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Lifts canonical bits out of docs/plans/recipes-content-registry.md per docs/README.md Rule 2 (delete plans on ship). Surfaces touched: - architecture.md § CLI usage gains a 'Recipes wiring' paragraph documenting the recipes-loader.ts ↔ query-recipes.ts seam, file-pair storage layout (templates/recipes/ for bundled, .codemap/recipes/ for project), shadow flag + load-time DML/DDL validation, and the .codemap.db-vs-.codemap/recipes/ gitignore asymmetry. - glossary.md § R: 'recipe' definition expanded to disambiguate bundled vs project sources, surface the actions-via-frontmatter shape, validation, and runtime backstop. New entry 'recipe shadows' covering the override discovery pattern. - roadmap.md: removed the recipes-as-content-registry backlog entry (now shipped). - README.md CLI block: added a project-recipes example showing mkdir + echo + --recipe lookup; mentions the shadows discovery field. - .agents/rules/codemap.md + templates/agents/rules/codemap.md (mirrored per Rule 10): new 'Project-local recipes' bullet right after Recipe actions covers the .codemap/recipes/ location, shadows: true catalog flag, YAML frontmatter shape, and load-time DML/DDL rejection. - .agents/skills/codemap/SKILL.md + templates/agents/skills/codemap/SKILL.md (mirrored): codemap://recipes resource description gains the 'check shadows at session start' guidance + source/shadows fields; codemap://recipes/{id} payload shape extended to {id, description, body?, sql, actions?, source, shadows?}; new 'Project-local recipes' bullet in the recipe section gives agents the full reference. - docs/plans/recipes-content-registry.md DELETED (Rule 2 — plan content fully lifted into architecture.md / glossary.md / agent files). - Minor changeset added (additive features, no schema breaks). --- .agents/rules/codemap.md | 2 + .agents/skills/codemap/SKILL.md | 5 +- .changeset/recipes-content-registry.md | 49 +++++ README.md | 8 + docs/architecture.md | 2 + docs/glossary.md | 11 +- docs/plans/recipes-content-registry.md | 249 ----------------------- docs/roadmap.md | 1 - templates/agents/rules/codemap.md | 2 + templates/agents/skills/codemap/SKILL.md | 5 +- 10 files changed, 79 insertions(+), 255 deletions(-) create mode 100644 .changeset/recipes-content-registry.md delete mode 100644 docs/plans/recipes-content-registry.md diff --git a/.agents/rules/codemap.md b/.agents/rules/codemap.md index e155cfb8..801711ce 100644 --- a/.agents/rules/codemap.md +++ b/.agents/rules/codemap.md @@ -29,6 +29,8 @@ A local database (default **`.codemap.db`**) indexes structure: symbols, imports **Recipe `actions`:** with **`--json`**, recipes that define an `actions` template append it to every row (kebab-case verb + description — e.g. `fan-out` → `review-coupling`). Under `--baseline`, actions attach to the **`added`** rows only. Inspect via **`--recipes-json`**. Ad-hoc SQL never carries actions. +**Project-local recipes:** drop `.sql` (and optional `.md` for description + actions) into **`/.codemap/recipes/`** — auto-discovered, runs via `--recipe ` like bundled. Project recipes win on id collision; check `--recipes-json` for **`shadows: true`** entries to know when a project recipe overrides the documented bundled version. `.md` supports YAML frontmatter (`actions: [{type, auto_fixable?, description?}]`) for the per-row action template — same shape as bundled recipes. Validation: SQL is rejected at load time if it starts with DML/DDL (DELETE/DROP/UPDATE/etc.); the runtime `PRAGMA query_only=1` is the parser-proof backstop. + **Baselines** (`query_baselines` table inside `.codemap.db`, no parallel JSON files): `--save-baseline[=]` snapshots a result set; `--baseline[=]` diffs the current result against it (added / removed rows; identity = `JSON.stringify(row)`). Name defaults to the `--recipe` id; ad-hoc SQL needs an explicit `=`. Survives `--full` and SCHEMA bumps. **Audit (`bun src/index.ts audit`)**: structural-drift command; emits `{head, deltas: {files, dependencies, deprecated}}` (each delta carries its own `base` metadata). Reuses B.6 baselines as the snapshot source. Two CLI shapes — `--baseline ` auto-resolves `-files` / `-dependencies` / `-deprecated`; `---baseline ` is the explicit per-delta override. v1 ships no `verdict` / threshold config — consumers compose `--json` + `jq` for CI exit codes. Auto-runs an incremental index before the diff (use `--no-index` to skip for frozen-DB CI). diff --git a/.agents/skills/codemap/SKILL.md b/.agents/skills/codemap/SKILL.md index 16006caa..3ad80e14 100644 --- a/.agents/skills/codemap/SKILL.md +++ b/.agents/skills/codemap/SKILL.md @@ -45,6 +45,7 @@ Replace placeholders (`'...'`) with your module path, file glob, or symbol name. - **`--baseline[=]`** — diff the current result against the saved baseline. Output `{baseline:{...}, current_row_count, added: [...], removed: [...]}` (with `--json`) or a two-section terminal dump. Identity = per-row multiset equality (canonical `JSON.stringify` keyed frequency map; duplicates preserved). Pair with `--summary` for `{baseline:{...}, current_row_count, added: N, removed: N}`. **Mutually exclusive with `--group-by`.** - **`--baselines`** lists saved baselines (no `rows_json` payload); **`--drop-baseline `** deletes one. Both reject every other flag — they're list-only / drop-only operations. - **Per-row recipe `actions`** — recipes that define an **`actions: [{type, auto_fixable?, description?}]`** template append it to every row in **`--json`** output (recipe-only; ad-hoc SQL never carries actions). Under `--baseline`, actions attach to the **`added`** rows only (the rows the agent should act on). Inspect via **`--recipes-json`**. +- **Project-local recipes** — drop **`.sql`** (and optional **`.md`** for description body + actions) into **`/.codemap/recipes/`** to make team-internal SQL a first-class CLI verb. `--recipes-json` and the `codemap://recipes` MCP resource list project recipes alongside bundled ones with **`source: "bundled" | "project"`** discriminating them. Project recipes win on id collision; entries that override a bundled id carry **`shadows: true`** so agents reading the catalog at session start know when a recipe behaves differently from the documented bundled version. `.md` supports YAML frontmatter (`---\nactions:\n - type: ...\n---`) for the per-row action template — same shape as bundled. Validation: SQL is rejected at load time if it starts with DML/DDL (DELETE/DROP/UPDATE/etc.); the runtime `PRAGMA query_only=1` is the parser-proof backstop. `.codemap.db` is gitignored; **`.codemap/recipes/` is NOT** — recipes are git-tracked source code authored for human review. **Audit (`bun src/index.ts audit`)** — separate top-level command for structural-drift verdicts. Composes B.6 baselines into a per-delta `{head, deltas}` envelope; v1 ships `files` / `dependencies` / `deprecated`. Two snapshot-source shapes: @@ -69,8 +70,8 @@ Each emitted delta carries its own `base` metadata so mixed-baseline audits are **Resources (lazy-cached on first `read_resource`; constant for server-process lifetime):** -- **`codemap://recipes`** — full catalog JSON (same as `--recipes-json`). -- **`codemap://recipes/{id}`** — single recipe `{id, description, sql, actions?}`. Replaces `--print-sql `. +- **`codemap://recipes`** — full catalog JSON (same as `--recipes-json`). Each entry carries `source: "bundled" | "project"` and `shadows: true` on project entries that override a bundled recipe id. Read this at session start so you know when a `--recipe foo` call will run a project override instead of the documented bundled version. +- **`codemap://recipes/{id}`** — single recipe `{id, description, body?, sql, actions?, source, shadows?}`. Replaces `--print-sql `. - **`codemap://schema`** — DDL of every table in `.codemap.db` (queried live from `sqlite_schema`). - **`codemap://skill`** — full text of bundled `templates/agents/skills/codemap/SKILL.md`. Agents that don't preload the skill at session start can fetch it here. diff --git a/.changeset/recipes-content-registry.md b/.changeset/recipes-content-registry.md new file mode 100644 index 00000000..3df4ba00 --- /dev/null +++ b/.changeset/recipes-content-registry.md @@ -0,0 +1,49 @@ +--- +"@stainless-code/codemap": minor +--- + +feat(recipes): recipes-as-content registry — bundled .md siblings + project-local recipes + +Two complementary capabilities: + +1. **Bundled recipes get richer descriptions.** Every bundled recipe in + `templates/recipes/` is now a `.sql` file paired with an optional + `.md` description body (replaces the inline TypeScript map in + `src/cli/query-recipes.ts`). Per-row `actions` templates live in YAML + frontmatter on the `.md` instead of code. Same surface for end users + (`--recipe ` / `--recipes-json` / `codemap://recipes`); single + storage shape across bundled + project recipes. + +2. **Project-local recipes** — drop `.{sql,md}` files into + `/.codemap/recipes/` to ship team-internal SQL as first- + class recipes. Auto-discovered via `--recipe `, surfaced in + `--recipes-json` and the `codemap://recipes` MCP resource alongside + bundled. Project recipes win on id collision; the catalog entry + carries `shadows: true` on overrides so agents reading the catalog + at session start see when a recipe behaves differently from the + documented bundled version (per-execution response shape stays + unchanged — uniformity contract preserved). + +Catalog entries (`--recipes-json` output, `codemap://recipes` +payload) gain three additive fields: `body` (full Markdown body), +`source` (`"bundled" | "project"`), and `shadows?` (true on +project entries that override a bundled id). Existing consumers +that destructure `{id, description, sql, actions?}` keep working. + +Validation: load-time lexical scan rejects DML / DDL keywords +(`INSERT` / `UPDATE` / `DELETE` / `DROP` / `CREATE` / `ALTER` / +`ATTACH` / `DETACH` / `REPLACE` / `TRUNCATE` / `VACUUM` / `PRAGMA`) +in recipe SQL with recipe-aware error messages — defence in depth +alongside the runtime `PRAGMA query_only=1` backstop in +`query-engine.ts` shipped in the previous release. + +Implementation: pure transport-agnostic loader in +`src/application/recipes-loader.ts`; thin shim in +`src/cli/query-recipes.ts` preserves backwards-compat exports +(`QUERY_RECIPES`, `getQueryRecipeSql`, etc.). Hand-rolled YAML +frontmatter parser scoped to the `actions` shape (no `js-yaml` +dependency). + +`.codemap.db` is gitignored as before; `.codemap/recipes/` is NOT +(verified via `git check-ignore`) — recipes are git-tracked source +code authored for human review. diff --git a/README.md b/README.md index fe43c409..c8ade911 100644 --- a/README.md +++ b/README.md @@ -110,6 +110,14 @@ codemap query --recipes-json codemap query --print-sql fan-out # `components-by-hooks` ranks by hook count without SQLite JSON1 (comma-based count on the stored JSON array). +# Project-local recipes — drop SQL files into .codemap/recipes/ to make them discoverable across the team +# Bundled recipes live in templates/recipes/ in the npm package; project recipes win on id collision +# (shadowing is signalled via a `shadows: true` field in --recipes-json so agents notice the override) +mkdir -p .codemap/recipes +echo "SELECT path FROM files WHERE language = 'typescript' AND line_count > 500" \ + > .codemap/recipes/big-ts-files.sql +codemap query --recipe big-ts-files # auto-discovered alongside bundled + # MCP server (Model Context Protocol) — for agent hosts (Claude Code, Cursor, Codex, generic MCP clients) codemap mcp # JSON-RPC on stdio; one tool per CLI verb plus query_batch # Tools: query, query_batch (MCP-only — N statements in one round-trip), query_recipe, audit, diff --git a/docs/architecture.md b/docs/architecture.md index 8689b727..f059bb51 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -125,6 +125,8 @@ A local SQLite database (`.codemap.db`) indexes the project tree and stores stru **Context wiring:** **`src/cli/cmd-context.ts`** — **`buildContextEnvelope`** composes the JSON envelope from existing recipes (`fan-in` for `hubs`, `markers` SELECT for `sample_markers`, `QUERY_RECIPES` map for the catalog). **`classifyIntent`** maps `--for ""` to one of `refactor | debug | test | feature | explore | other` via regex against the trimmed input; whitespace-only intents are rejected. `--compact` drops `hubs` + `sample_markers` and emits one-line JSON; otherwise pretty-prints with 2-space indent. +**Recipes wiring:** **`src/application/recipes-loader.ts`** (pure transport-agnostic loader) + **`src/cli/query-recipes.ts`** (shim — caches the loader output, exposes `getQueryRecipeSql` / `getQueryRecipeActions` / `listQueryRecipeIds` / `listQueryRecipeCatalog` / `getQueryRecipeCatalogEntry`). Recipes live as file pairs: **`.sql`** + optional **`.md`**. The loader reads `templates/recipes/` (bundled, ships in npm package next to `templates/agents/`) and `/.codemap/recipes/` (project-local — root-only resolution per the registry plan, no walk-up). Project recipes win on id collision; entries that override a bundled id carry **`shadows: true`** in the catalog so agents reading `codemap://recipes` at session start see when a recipe behaves differently from the documented bundled version. Per-row **`actions`** templates (kebab-case verb + description) live in YAML frontmatter on each `.md` — uniform shape across bundled + project. Hand-rolled YAML parser scoped to `actions: [{type, auto_fixable?, description?}]` only (no `js-yaml` dep). Load-time validation rejects empty SQL and DML / DDL keywords (`INSERT` / `UPDATE` / `DELETE` / `DROP` / `CREATE` / `ALTER` / `ATTACH` / `DETACH` / `REPLACE` / `TRUNCATE` / `VACUUM` / `PRAGMA`) with recipe-aware error messages — defence in depth alongside the runtime `PRAGMA query_only=1` backstop in `query-engine.ts` (PR #35). `.codemap.db` is gitignored; `.codemap/recipes/` is NOT (verified via `git check-ignore`) — recipes are git-tracked source code authored for human review. + **MCP wiring:** **`src/cli/cmd-mcp.ts`** (argv — `--help` only; bootstrap absorbs `--root`/`--config`) + **`src/application/mcp-server.ts`** (engine — tool registry, resource handlers, response composition). Mirrors the `cmd-audit.ts ↔ audit-engine.ts` seam — CLI parses + lifecycle; engine owns the SDK. **`runMcpServer`** bootstraps codemap once at server boot (config + resolver + DB access become module-level state), instantiates `McpServer` from **`@modelcontextprotocol/sdk`**, attaches a **`StdioServerTransport`**, and resolves when stdin closes (clean shutdown). Tool handlers reuse the existing engine entry-points: **`query`** + **`query_recipe`** call **`executeQuery`** in **`src/application/query-engine.ts`** (a pure transport-agnostic engine extracted from `printQueryResult`'s JSON branch — same `[...rows]` / `{count}` / `{group_by, groups}` envelope `--json` would print); **`query_batch`** loops via **`executeQueryBatch`** with batch-wide-defaults + per-statement-overrides (items are `string | {sql, summary?, changed_since?, group_by?}`); **`audit`** runs `resolveAuditBaselines` + `runAudit` from PR #33 unchanged; **`context`** / **`validate`** call `buildContextEnvelope` / `computeValidateRows` (pure functions in `src/cli/cmd-*.ts` — same layer-reversal allowance as `query-recipes`). **`save_baseline`** is one polymorphic tool (`{name, sql? | recipe?}`) with a runtime exclusivity check — mirrors the CLI's single `--save-baseline=` verb. **Tool naming**: snake_case throughout — Codemap convention matching the patterns in MCP spec examples and reference servers (GitHub MCP, Cursor built-ins); the spec itself doesn't mandate it. CLI stays kebab — translation lives at the MCP-arg layer. **Resources** (`codemap://recipes`, `codemap://recipes/{id}`, `codemap://schema`, `codemap://skill`) use **lazy memoisation** — first `read_resource` populates a per-server-instance cache; constant for the server-process lifetime so eager-vs-lazy produce identical observable behavior. `codemap://schema` queries `sqlite_schema` live; `codemap://skill` reads from `resolveAgentsTemplateDir() + skills/codemap/SKILL.md`. Output shape uniformity (plan § 4): every tool returns the JSON envelope its CLI counterpart's `--json` flag prints, surfaced via `content: [{type: "text", text: JSON.stringify(payload)}]`. `--changed-since` git lookups are memoised per `(root, ref)` pair across batch items so a `query_batch` of N items sharing the same ref does one git invocation, not N. Per-statement errors in `query_batch` are isolated — failed statements return `{error}` in their slot while siblings still execute. **Performance wiring:** **`--performance`** plumbs through **`RunIndexOptions.performance`** → **`indexFiles({ performance, collectMs })`**. `parse-worker-core.ts` records per-file **`parseMs`** on each `ParsedFile`; main thread times the four phases (`collect`, `parse`, `insert`, `index_create`) and assembles **`IndexPerformanceReport`** under `IndexRunStats.performance`. Note: `total_ms` is `indexFiles` wall-clock, **not** end-to-end run wall — `collect_ms` happens before `indexFiles` and is reported separately. diff --git a/docs/glossary.md b/docs/glossary.md index df9baff2..40ded28d 100644 --- a/docs/glossary.md +++ b/docs/glossary.md @@ -325,7 +325,16 @@ See **recipe**. ### recipe -A bundled SQL string in `src/cli/query-recipes.ts`, identified by id (e.g. `fan-in`, `deprecated-symbols`, `files-hashes`). Run via `codemap query --recipe ` (alias `-r`). Distinct from an ad-hoc **query** (which is any SQL string the agent composes itself). +A SQL file (plus optional sibling `.md` description) loaded into the catalog by `src/application/recipes-loader.ts`. Two sources, same shape: + +- **Bundled** — ships in the npm package as `templates/recipes/.{sql,md}`. Examples: `fan-in`, `deprecated-symbols`, `files-hashes`. +- **Project-local** — loaded from `/.codemap/recipes/.{sql,md}` (root-only resolution; not gitignored — meant to be checked in for team review). + +Run via `codemap query --recipe ` (alias `-r`). Project recipes win on id collision with bundled ones (entries carry `shadows: true` in the catalog so agents reading `codemap://recipes` at session start see when a recipe behaves differently from the documented bundled version). Per-row `actions` templates (kebab-case verb + description) live in YAML frontmatter on each `.md` — uniform between bundled and project. Load-time validation rejects empty SQL and DML / DDL keywords; runtime `PRAGMA query_only=1` (PR #35) is the parser-proof backstop. Distinct from an ad-hoc **query** (any SQL string the agent composes itself; ad-hoc SQL never carries actions). + +### `recipe shadows` + +Boolean flag on a project-local recipe entry that has the same `id` as a bundled recipe — `shadows: true` means "this project recipe overrides what the bundled version would have done." Surfaces in `--recipes-json`, `codemap://recipes`, and `codemap://recipes/{id}` so agents can see overrides without parsing per-execution responses (per-execution shape stays unchanged for plan § 4 uniformity). Silent at runtime — the agent-facing skill prompt is the channel that tells agents to check the flag at session start. ### research diff --git a/docs/plans/recipes-content-registry.md b/docs/plans/recipes-content-registry.md deleted file mode 100644 index 85e08fad..00000000 --- a/docs/plans/recipes-content-registry.md +++ /dev/null @@ -1,249 +0,0 @@ -## Plan — `recipes-content-registry` - -> Pair every bundled recipe with a sibling `.md` description (when-to-use / follow-up SQL hints), and let projects ship their own recipes via `.codemap/recipes/.{sql,md}` files — surfaces uniformly in `--recipes-json`, `codemap query --recipe `, and the `codemap://recipes` MCP resource. -> -> Adopted from [`docs/roadmap.md` § Backlog](../roadmap.md#backlog) ("Recipes-as-content registry"). Builds on the bundled recipe surface (PR [#26](https://github.com/stainless-code/codemap/pull/26)) and the MCP resources shipped in PR [#35](https://github.com/stainless-code/codemap/pull/35). - -**Status:** Open — design pass; not yet implemented. -**Cross-refs:** [`docs/architecture.md` § CLI usage](../architecture.md#cli-usage) (recipes are part of the query surface), [`docs/architecture.md` § MCP wiring](../architecture.md#cli-usage) (`codemap://recipes` resource), [`.agents/lessons.md`](../../.agents/lessons.md) (changesets policy: pre-v1 patch unless schema-breaks). - ---- - -## 1. Goal - -**Two consumers, one registry.** - -- **Bundled recipes today** live in `src/cli/query-recipes.ts` as a TypeScript object map. SQL + short description + optional `actions` are all in code. Description is a one-liner — there's no room for "when to use this", "follow-up SQL", or "what to do with the rows." -- **Project teams today** can't ship a custom recipe without forking codemap or wrapping `codemap query --json ""` in their own scripts. There's no on-ramp for "every team member can run `codemap query --recipe internal-flaky-tests` without remembering the SQL." - -After v1: - -```bash -# bundled recipe — long-form description in sibling .md -codemap query --json --recipe fan-out - -# project-local recipe loaded from .codemap/recipes/internal-flaky-tests.sql -codemap query --json --recipe internal-flaky-tests - -# catalog surfaces both -codemap query --recipes-json -# MCP resource surfaces both -read_resource codemap://recipes -``` - -The wins: - -- **Bundled recipes get room to teach.** The one-liner becomes a Markdown body with usage notes, follow-up queries, and "what an agent should do with these rows." -- **Project teams ship internal SQL** without forking. `git`-tracked, code-reviewable, no plugin API needed. -- **MCP / agent surface stays uniform** — `codemap://recipes` and `codemap://recipes/{id}` automatically include project-local recipes; agents discover them at session-start. - -## 2. Scope split (this plan vs follow-ups) - -| Slice | Status | Where it lives | -| --------------------------------------------------------------------------------------------------------------------- | ----------------------------------------- | ------------------------------------------------------------------- | -| **A. Project-local recipes** (`.codemap/recipes/.sql`) — actually-new capability | This plan (v1) | New loader + composes with existing CLI / MCP surfaces | -| **B. Bundled recipe extraction** (move `QUERY_RECIPES` map → `templates/recipes/.{sql,md}` files) — pure refactor | This plan (v1) | Same loader; bundled recipes become the same shape as project-local | -| **C. Sibling `.md` description body** for both bundled AND project-local | This plan (v1) | Optional file alongside `.sql` | -| **D. `actions` support for project-local recipes** | Open question (§ 12) — likely v1 if cheap | YAML frontmatter on `.md`? Sibling `.actions.json`? | -| **E. Recipe versioning / migrations** | v1.x | Out of scope for v1 — defer until two consumers ask | -| **F. Recipe parameters** (`{table}`, `{limit}` placeholders) | v1.x | Out of scope — would require a templating layer | - -Slices A + B + C ship together because B is pre-requisite for C (uniform loader needs uniform storage), and A is the actual user-facing capability. D depends on grill round. - -## 3. Storage layout - -### 3.1 Bundled recipes (after refactor) - -``` -templates/recipes/ -├── fan-out.sql # the SQL string -├── fan-out.md # description body (optional but recommended) -├── fan-out-sample.sql -├── fan-out-sample.md -├── deprecated-symbols.sql -├── deprecated-symbols.md -├── visibility-tags.sql -├── visibility-tags.md -└── … -``` - -Each `.sql` is the recipe's SQL verbatim (one statement, no `;` terminator needed). The matching `.md` is optional — when absent, the recipe still loads but has no long-form description (CLI / MCP surfaces show only the recipe id). - -`templates/recipes/` ships in the npm package alongside `templates/agents/` (already part of the published artifact — `agents-init.ts`'s `resolveAgentsTemplateDir()` shows the pattern). - -### 3.2 Project-local recipes - -``` -/ -└── .codemap/ - └── recipes/ - ├── internal-flaky-tests.sql - ├── internal-flaky-tests.md - └── owner-fanout.sql -``` - -`` is the same root the CLI's `--root` / `CODEMAP_ROOT` resolves to. `.codemap/` is the conventional location for codemap-related project artifacts — same parent as a future user-config might use. - -**Gitignore note (verified, not just assumed).** Codemap's bundled `.gitignore` line `.codemap.*` is the literal-dot glob `.codemap.` — it matches `.codemap.db` but NOT the `.codemap/` directory or files inside it. Confirmed via `git check-ignore -v .codemap/recipes/foo.sql` returning no match. So project recipes are checked into git by default, which is the intended behavior (recipes are source code authored for human review). Consumer-side risk: if a user's `.gitignore` ignores `.codemap*` (no dot — common defensive pattern), their project recipes will be silently dropped on clone — the shipped agent rule + skill calls this out and recommends `!.codemap/recipes/` as the un-ignore. - -### 3.3 Why filesystem and not `.codemap.db` - -`query_baselines` lives inside `.codemap.db` (see [`architecture.md` § Query wiring](../architecture.md#cli-usage)) — same project, opposite call. The deciding tests: - -| | `query_baselines` (in DB) | recipes (in filesystem) | -| ----------------------------------------------- | -------------------------------------------------------------- | --------------------------------------------------------------------------- | -| Nature | Output — captured query results | Input — SQL the user wrote | -| Tied to a specific index state? | Yes (rows are valid only against the index that produced them) | No (same SQL runs against any project with the schema) | -| Authored by humans for human review? | No (codemap captures them on `--save-baseline`) | Yes (PR-reviewed source code) | -| Meaningful outside one project's `.codemap.db`? | No | Yes (copy-paste from docs, lift from a colleague's PR, share via submodule) | - -If a user wants to send a recipe to a colleague, the file-based answer is "send the `.sql`." The DB-based answer would be `codemap recipes export foo > foo.sql; … import foo.sql` — reinventing files. Plus bundled recipes ship in the npm package as files (`templates/recipes/`); putting them in DB would require a migration step and an exception to `dropAll()` for every codemap upgrade. - -### 3.4 Single-file form (rejected for v1) - -YAML-frontmatter Markdown with the SQL in a code block (Astro / Hugo style) was considered: - -````markdown ---- -id: fan-out -description: Top 10 files by dependency fan-out -actions: - - type: review-coupling - description: … ---- - -When to use: … - -Follow-up SQL: … - -```sql -SELECT from_path, COUNT(*) AS deps FROM dependencies … -``` -```` - -```` - -**Rejected because:** -- Editor support for SQL inside Markdown code blocks is worse than for `.sql` files (no syntax highlighting, no LSP). -- Two-file split keeps SQL editable as SQL (works with sqlite CLI: `sqlite3 .codemap.db ".read .codemap/recipes/foo.sql"`). -- The frontmatter parsing surface (gray-matter or hand-rolled) is more code than a sibling `.md` lookup. -- One-file form remains a v1.x option if real consumer demand emerges. - -## 4. Loader contract - -A pure function in `src/application/recipes-loader.ts`: - -```typescript -interface LoadedRecipe { - id: string; - sql: string; - description: string | undefined; // first-line of .md, or undefined - body: string | undefined; // full .md body, or undefined - actions: RecipeAction[] | undefined; // from YAML frontmatter on .md (D — open question) - source: "bundled" | "project"; // for catalog disambiguation -} - -export function loadAllRecipes(opts: { - bundledDir: string; // resolveBundledRecipesDir() — npm package layout - projectDir: string | undefined; // resolveProjectRecipesDir(root) — undefined if .codemap/recipes/ is absent -}): LoadedRecipe[]; -```` - -**Conflict resolution:** if a project recipe has the same `id` as a bundled recipe, the **project recipe wins** (`source: "project"`); the bundled one is shadowed but still discoverable via a hypothetical future `--source bundled` filter (out of scope for v1). User-code-wins is the standard convention (npm, ESLint plugins, etc.). - -**Validation:** at load time, each `.sql` must parse as something `bun:sqlite`'s `.prepare()` accepts — but we don't actually prepare against a DB until `--recipe ` runs (would require an indexed project at load time). v1 does a cheap lexical sanity check: non-empty after stripping `--` line comments and trailing whitespace. SQL errors surface at query-time with the same `enrichQueryError` pretty-printing as ad-hoc SQL. - -**Loading time:** **eager at startup**, but cheap. `templates/recipes/` is filesystem-stable per-version (read once). `.codemap/recipes/` is filesystem-stable per-session (the user isn't editing recipes mid-CLI-call). Cache the result in a module-level variable; invalidate only on process restart. - -## 5. CLI surface (no new flags — same shape as today) - -```bash -codemap query --recipe # works for bundled OR project recipes; project wins on conflict -codemap query --recipes-json # full catalog: bundled + project, with `source` field -codemap query --print-sql # prints the SQL of regardless of source -``` - -`--recipes-json` output gets two new fields per recipe: - -```json -[ - { - "id": "fan-out", - "description": "Top 10 files by dependency fan-out", - "body": "# Fan-out\n\nWhen to use: …\n\nFollow-up SQL: …", - "sql": "SELECT from_path …", - "actions": [{ "type": "review-coupling", "description": "…" }], - "source": "bundled" - }, - { - "id": "internal-flaky-tests", - "description": null, - "body": null, - "sql": "SELECT path FROM files WHERE …", - "actions": null, - "source": "project" - } -] -``` - -`description` and `body` are nullable — recipes without sibling `.md` get null. `actions` field nullability depends on grill question D. - -## 6. MCP surface — auto-inherits - -Already shipped in PR [#35](https://github.com/stainless-code/codemap/pull/35): - -- `codemap://recipes` resource — automatically picks up project recipes since it calls `listQueryRecipeCatalog()` (which becomes the loader). -- `codemap://recipes/{id}` template — auto-resolves project recipe ids. - -The agent's discovery story is unchanged; the catalog just got bigger. - -## 7. Implementation deps - -- No new npm dependencies. Uses `node:fs/promises` (or sync `readFileSync` for cache population) + `node:path`. -- Reuses the existing `resolveAgentsTemplateDir()` pattern for `resolveBundledRecipesDir()` (npm package layout — `templates/recipes/` next to `templates/agents/`). -- New file: `src/application/recipes-loader.ts` (loader engine — pure, transport-agnostic). -- `src/cli/query-recipes.ts` becomes a thin re-export layer that calls the loader (preserves the `getQueryRecipeSql` / `getQueryRecipeActions` / `listQueryRecipeIds` / `listQueryRecipeCatalog` / `QUERY_RECIPES` named exports for backwards-compat with the MCP server + cmd-query). - -## 8. Tracer-bullet sequence - -Per [`tracer-bullets`](../../.agents/rules/tracer-bullets.md): - -1. **Loader scaffold** — `src/application/recipes-loader.ts` with `loadAllRecipes` returning bundled-only (project loader stubbed). Tests cover empty / one-recipe / multiple-recipe loads against a fixture `templates/recipes/` directory. Commit. -2. **Migrate bundled recipes** — extract every entry in `QUERY_RECIPES` to `templates/recipes/.sql`; for the ones with a meaningful one-liner already, also add `.md`. `query-recipes.ts` becomes a thin shim that calls `loadAllRecipes({bundledDir, projectDir: undefined})`. Tests: every existing recipe id still resolves to the same SQL. Commit. -3. **Project-local loader** — implement `resolveProjectRecipesDir(root)` + load `.codemap/recipes/*.sql` + sibling `.md` discovery. Tests cover: no `.codemap/recipes/` (no error, no project recipes); one project recipe; project recipe shadows bundled. Commit. -4. **`--recipes-json` carries `source` + `body`** — extend the catalog output. Tests cover both source values + body presence/absence. Commit. -5. **Optional `actions` support** (depends on grill Q-D) — if YAML frontmatter wins, ship a tiny parser; if sibling `.actions.json` wins, ship the lookup. Commit. -6. **Docs + agents update** — `architecture.md § Recipes wiring` paragraph, glossary entries (`recipe` definition gets the bundled-vs-project disambiguation), README CLI block (mention `.codemap/recipes/`), rule + skill across `.agents/` and `templates/agents/` (Rule 10), patch changeset. Delete this plan (Rule 2), lift canonical bits into architecture.md. Commit. - -Estimated total: ~1 day across ~6 commits. - -## 9. Open questions (worth a `grill-me` round before code) - -### Settled - -- **Q-A. Storage layout for bundled recipes?** ✅ **(i) `templates/recipes/.{sql,md}` file-pair.** Uniformity with project recipes wins: one loader code path (no `if (source === "bundled")` branches), `.sql` files get SQLite syntax highlighting in every editor (today's `QUERY_RECIPES` template literals get none), single-file diffs for SQL changes, and `sqlite3 .codemap.db ".read …"` works for ad-hoc testing. Migration cost is one-time (~15 entries → ~15 `.sql` files); the shim layer in `cli/query-recipes.ts` preserves backwards-compat for `getQueryRecipeSql` / `getQueryRecipeActions` / `QUERY_RECIPES` re-exports. Rejected (ii) "code-map + sibling .md only" — smaller initial diff but two storage shapes that compound debt every time the recipe surface evolves. -- **Q-B. Loading time?** ✅ **Eager at startup.** Cost is negligible (~15-20 small file reads, sub-millisecond on warm SSD — rounding error vs node/bun startup, oxc, bun:sqlite). "Registry is always populated" eliminates per-call `if (notLoadedYet)` guards. Surfaces malformed-recipe errors at startup instead of 30-minutes-into-a-session. Rejected (ii) lazy — its win ("don't pay for what you don't use") is hypothetical for filesystem reads of static files; matters for DB connections / network calls, not 20 small files. Rejected (iii) eager-with-disk-cache — over-engineered; introduces invalidation problem for no measurable win. -- **Q-C. Project recipes — discovery walk-up?** ✅ **Root-only — `/.codemap/recipes/`.** Same root the CLI's `--root` / `CODEMAP_ROOT` resolves to; same root `.codemap.db` lives in. Adding walk-up for _just_ recipes would make them the only piece of codemap that resolves differently from everything else (DB, indexer, resolver) — confusing inconsistency. Monorepos are well-served today via `--root packages/foo` (recipes load from `packages/foo/.codemap/recipes/`); shared recipes can use a filesystem symlink. Walk-up is additive (forward-compatible), so we can revisit if real consumer demand emerges; root-only-→-walk-up is a non-breaking expansion. Rejected (iii) workspace-cascade — `.eslintrc`-style cascading is appropriate when a workspace primitive exists; codemap's workspace concept is a separate roadmap item, so cascading would pre-commit to a design before its dependency lands. -- **Q-D. `actions` for project-local recipes?** ✅ **YAML frontmatter on `.md`, hand-rolled parser (~30 LOC).** Project recipes feel first-class — same `actions` template surface bundled recipes have. Frontmatter co-locates the action with its prose description (one editor open vs two file creates). Hand-rolled parser handles only the shallow shape codemap needs (key / list / string / bool); strict, clear errors, zero supply-chain surface. Rejected (i) skip-for-v1 — would make project recipes second-class ("missing a feature" feels worse than "we don't have evidence yet" for an open registry). Rejected `gray-matter` / `js-yaml` — too much surface (~50KB of full YAML 1.2 spec) for our shallow needs. Rejected (iii) sibling `.actions.json` — three files per actions-bearing recipe is more cognitive overhead than one with optional frontmatter; JSON files separate the action from its prose explanation, which is the wrong factoring. -- **Q-E. Conflict resolution loud or quiet?** ✅ **Silent at runtime + `shadows: true` flag in catalog discovery + agent-skill prompt update.** Three layers: (1) project wins silently — `--recipe fan-out` runs the project version with no stderr noise (user code wins; matches ESLint / npm overrides / `tsconfig` extends conventions). (2) Catalog responses (`--recipes-json`, `codemap://recipes`, `codemap://recipes/{id}`) carry `shadows: true` on project entries that override a bundled id of the same name. (3) Bundled `templates/agents/skills/codemap/SKILL.md` instructs agents to read `codemap://recipes` at session start and check `shadows` so they know when a recipe behaves differently from the documented bundled version. Per-execution response shape stays unchanged (preserves plan § 4 uniformity contract — `shadows` lives at discovery time, not per-call). Rejected (ii) one-time stderr warning — MCP servers log to stderr per spec but agent hosts don't surface those logs to the model, so warnings land nowhere useful for agent traceability. Rejected (iii) `--allow-shadow` flag — hostile to the legitimate-override case (every team that wants to override has to wire the flag through their tooling) for the rare-mistake case. Loader cost: ~5 LOC for the shadow-flag check. - -- **Q-F. Validation strictness?** ✅ **Both — load-time lexical check + retain run-time `PRAGMA query_only` backstop.** Load-time gives recipe-aware error UX (e.g. "Project recipe `` at `.codemap/recipes/.sql` starts with `DELETE` — recipes must be read-only. Use `--save-baseline` for capturing rows.") and fires in CI / pre-commit hooks so bad recipes never reach main. Lexical scan: strip `--` line comments, find first identifier-shaped token, deny-list `INSERT` / `UPDATE` / `DELETE` / `DROP` / `CREATE` / `ALTER` / `ATTACH` / `DETACH` / `REPLACE` / `TRUNCATE` / `VACUUM` / `PRAGMA` (~20 LOC, same shape as the empty-recipe check). The PR #35 `PRAGMA query_only=1` runtime backstop **stays** as the parser-proof safety net for anything lexical scans can't catch (multi-statement payloads, `WITH` clauses with mutating sub-queries, attached databases). Different jobs: lexical = good UX for common mistakes; backstop = correctness no matter what passes lexical. Rejected (i) load-time only — incomplete (`WITH foo AS (DELETE FROM …) SELECT …` slips through); shouldn't claim safety we don't have. Rejected (ii) run-time only — error fires after parse with a less-clear message and no recipe-aware framing. - -### Still open - -_None — all 6 questions settled. Ready to start tracer 1._ - -## 10. Non-goals (v1) - -- **Recipe versioning / migrations.** If a bundled recipe's SQL changes between codemap versions, project consumers using the same id silently get the new SQL on upgrade. Defer until a real consumer reports breakage. -- **Recipe parameters / templating.** No `{table}` / `{limit}` placeholder substitution — recipes are static SQL. Templating adds a parser surface and ambiguity around what's a parameter vs a SQL token. Defer until two consumers ask with the same shape. -- **Network-fetched recipes.** No `codemap recipes add github.com/foo/recipes` registry. Stay filesystem-only — security and supply-chain reasoning matches the agent-host trust boundary from the MCP plan. -- **Recipe execution control beyond `--recipe `.** No `--list-recipes` shorthand (use `--recipes-json | jq`). No `codemap recipe run ` (use `codemap query --recipe `). Single CLI surface stays. -- **`.codemap/recipes/.json` (raw envelope)** — recipes are SQL-first; JSON envelope would re-invent half of `.sql` + `.md`. - -## 11. References - -- Roadmap entry: [`docs/roadmap.md` § Backlog](../roadmap.md#backlog). -- Existing recipe shape: [`src/cli/query-recipes.ts`](../../src/cli/query-recipes.ts) (`QUERY_RECIPES` map, `RecipeAction` interface). -- MCP resources that auto-inherit: [`docs/architecture.md` § MCP wiring](../architecture.md#cli-usage), `codemap://recipes` and `codemap://recipes/{id}`. -- Doc lifecycle: this file follows the **Plan** type per [`docs/README.md` § Document Lifecycle](../README.md#document-lifecycle) — **delete on ship**, lift the canonical bits into `architecture.md` per Rule 2. diff --git a/docs/roadmap.md b/docs/roadmap.md index 93f957be..f4911311 100644 --- a/docs/roadmap.md +++ b/docs/roadmap.md @@ -39,7 +39,6 @@ Codemap stays a structural-index primitive that other tools can consume. Out of - [ ] **`codemap audit --base `** (v1.x) — worktree+reindex snapshot strategy. v1 shipped `--baseline ` / `---baseline ` (B.6 reuse) — see [`architecture.md` § Audit wiring](./architecture.md#cli-usage). v1.x adds `--base ` for "audit against an arbitrary ref I haven't pre-baselined" (defers worktree spawn + cache decision until a real consumer asks). - [ ] **`codemap audit` verdict + thresholds** (v1.x) — `verdict: "pass" | "warn" | "fail"` driven by `codemap.config.audit.deltas[].{added_max, action}`. Triggers: two consumers ship `jq`-based threshold scripts with similar shapes, OR one consumer asks with a concrete config sketch. Until then, raw deltas + consumer-side `jq` is the CI exit-code idiom. - [ ] **`codemap serve` (HTTP API, v1.x)** — same tool taxonomy + output shape as `codemap mcp` (shipped in v1), exposed over `POST /tool/{name}` with loopback default and optional `--token`. Defer until a concrete non-MCP consumer asks; design points are reserved in [`architecture.md` § MCP wiring](./architecture.md#cli-usage) so HTTP inherits them when its turn comes. -- [ ] **Recipes-as-content registry** — pair every bundled recipe with a sibling `.md` (when-to-use, follow-up SQL); plus **project-local recipes** loaded from `.codemap/recipes/.{sql,md}` so teams can ship internal SQL without an adapter API. Plan: [`plans/recipes-content-registry.md`](./plans/recipes-content-registry.md). Composes with the `codemap://recipes` and `codemap://recipes/{id}` MCP resources shipped in PR #35. - [ ] **Targeted-read CLI** — `codemap show ` / `codemap snippet ` returns `file_path:line_start-line_end` + `signature` for one symbol. Same data as `SELECT … FROM symbols WHERE name = ?`, but a one-step CLI keeps agents from composing SQL for trivial precise reads - [ ] **Watch mode** for dev — `node:fs.watch` recursive + `--files` re-index loop; Linux `recursive` requires Node 19.1+ - [ ] **Monorepo / workspace awareness** — discover workspaces from `pnpm-workspace.yaml` / `package.json` and index per-workspace dependency graphs diff --git a/templates/agents/rules/codemap.md b/templates/agents/rules/codemap.md index 0d0d4c85..8c10807f 100644 --- a/templates/agents/rules/codemap.md +++ b/templates/agents/rules/codemap.md @@ -36,6 +36,8 @@ Install **[@stainless-code/codemap](https://www.npmjs.com/package/@stainless-cod **Recipe `actions`:** with **`--json`**, recipes that define an `actions` template append it to every row (kebab-case verb + description — e.g. `fan-out` → `review-coupling`). Under `--baseline`, actions attach to the **`added`** rows only. Inspect via **`--recipes-json`**. Ad-hoc SQL never carries actions. +**Project-local recipes:** drop `.sql` (and optional `.md` for description + actions) into **`/.codemap/recipes/`** — auto-discovered, runs via `codemap query --recipe ` like bundled. Project recipes win on id collision; check `codemap query --recipes-json` for **`shadows: true`** entries to know when a project recipe overrides the documented bundled version. `.md` supports YAML frontmatter (`actions: [{type, auto_fixable?, description?}]`) for the per-row action template — same shape as bundled recipes. Validation: SQL is rejected at load time if it starts with DML/DDL (DELETE/DROP/UPDATE/etc.); the runtime `PRAGMA query_only=1` is the parser-proof backstop. + **Baselines** (`query_baselines` table inside `.codemap.db`, no parallel JSON files): `--save-baseline[=]` snapshots a result set; `--baseline[=]` diffs the current result against it (added / removed rows; identity = `JSON.stringify(row)`). Name defaults to the `--recipe` id; ad-hoc SQL needs an explicit `=`. Survives `--full` and SCHEMA bumps. **Audit (`codemap audit`)**: structural-drift command; emits `{head, deltas: {files, dependencies, deprecated}}` (each delta carries its own `base` metadata). Reuses B.6 baselines as the snapshot source. Two CLI shapes — `--baseline ` auto-resolves `-files` / `-dependencies` / `-deprecated`; `---baseline ` is the explicit per-delta override. v1 ships no `verdict` / threshold config — consumers compose `--json` + `jq` for CI exit codes. Auto-runs an incremental index before the diff (use `--no-index` to skip for frozen-DB CI). diff --git a/templates/agents/skills/codemap/SKILL.md b/templates/agents/skills/codemap/SKILL.md index 515e72d6..d61c89a7 100644 --- a/templates/agents/skills/codemap/SKILL.md +++ b/templates/agents/skills/codemap/SKILL.md @@ -45,6 +45,7 @@ Replace placeholders (`'...'`) with your module path, file glob, or symbol name. - **`--baseline[=]`** — diff the current result against the saved baseline. Output `{baseline:{...}, current_row_count, added: [...], removed: [...]}` (with `--json`) or a two-section terminal dump. Identity = per-row multiset equality (canonical `JSON.stringify` keyed frequency map; duplicates preserved). Pair with `--summary` for `{baseline:{...}, current_row_count, added: N, removed: N}`. **Mutually exclusive with `--group-by`.** - **`--baselines`** lists saved baselines (no `rows_json` payload); **`--drop-baseline `** deletes one. Both reject every other flag — they're list-only / drop-only operations. - **Per-row recipe `actions`** — recipes that define an **`actions: [{type, auto_fixable?, description?}]`** template append it to every row in **`--json`** output (recipe-only; ad-hoc SQL never carries actions). Under `--baseline`, actions attach to the **`added`** rows only (the rows the agent should act on). Inspect via **`--recipes-json`**. +- **Project-local recipes** — drop **`.sql`** (and optional **`.md`** for description body + actions) into **`/.codemap/recipes/`** to make team-internal SQL a first-class CLI verb. `--recipes-json` and the `codemap://recipes` MCP resource list project recipes alongside bundled ones with **`source: "bundled" | "project"`** discriminating them. Project recipes win on id collision; entries that override a bundled id carry **`shadows: true`** so agents reading the catalog at session start know when a recipe behaves differently from the documented bundled version. `.md` supports YAML frontmatter (`---\nactions:\n - type: ...\n---`) for the per-row action template — same shape as bundled. Validation: SQL is rejected at load time if it starts with DML/DDL (DELETE/DROP/UPDATE/etc.); the runtime `PRAGMA query_only=1` is the parser-proof backstop. `.codemap.db` is gitignored; **`.codemap/recipes/` is NOT** — recipes are git-tracked source code authored for human review. **Audit (`codemap audit`)** — separate top-level command for structural-drift verdicts. Composes B.6 baselines into a per-delta `{head, deltas}` envelope; v1 ships `files` / `dependencies` / `deprecated`. Two snapshot-source shapes: @@ -69,8 +70,8 @@ Each emitted delta carries its own `base` metadata so mixed-baseline audits are **Resources (lazy-cached on first `read_resource`; constant for server-process lifetime):** -- **`codemap://recipes`** — full catalog JSON (same as `--recipes-json`). -- **`codemap://recipes/{id}`** — single recipe `{id, description, sql, actions?}`. Replaces `--print-sql `. +- **`codemap://recipes`** — full catalog JSON (same as `--recipes-json`). Each entry carries `source: "bundled" | "project"` and `shadows: true` on project entries that override a bundled recipe id. Read this at session start so you know when a `--recipe foo` call will run a project override instead of the documented bundled version. +- **`codemap://recipes/{id}`** — single recipe `{id, description, body?, sql, actions?, source, shadows?}`. Replaces `--print-sql `. - **`codemap://schema`** — DDL of every table in `.codemap.db` (queried live from `sqlite_schema`). - **`codemap://skill`** — full text of this skill file. Agents that don't preload the skill at session start can fetch it here. From e71dddcf3e8a9c69bfdf633fc0d07b080704f743 Mon Sep 17 00:00:00 2001 From: Sutu Sebastian Date: Sat, 2 May 2026 11:42:02 +0300 Subject: [PATCH 14/14] fix(recipes): address PR #37 CodeRabbit feedback (1 Major + 6 Minor) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit All 7 verified valid against actual code; all applied. Major: - recipes-loader.ts: stripLineComments now strips block /* */ comments BEFORE the line-comment + first-keyword scan. Without this, the deny-list could be bypassed two ways: (1) A leading block comment containing a deny-listed keyword could cause the lexer to misclassify a legit SELECT as DDL. (2) /* SELECT */ DELETE FROM x would be accepted because the lexer saw 'SELECT' in the comment first. Now: regex strips block comments first, then line comments, then first-identifier match runs. Pure-block-comment files also trip the empty-recipe check correctly. The runtime PRAGMA query_only=1 backstop is still the parser-proof safety net for things like string-literal-embedded comments (vanishingly rare). 3 new tests cover false-positive avoidance, smuggled-DELETE rejection, and pure-block-comment-as-empty. Minor: - glossary.md § Conventions: 'Recipe = bundled SQL string in src/cli/query-recipes.ts' was outdated (recipes are now files in templates/recipes/ + .codemap/recipes/; query-recipes.ts is the shim). Reworded with a forward-pointer to § R recipe. - README.md CLI block: language IN ('ts', 'tsx') instead of language = 'typescript'. Verified via codemap query — the indexer stores 'ts' / 'tsx' / 'md' / 'json' etc., not the long form. The example as written would have returned 0 rows. - .agents/rules/codemap.md + templates/agents/rules/codemap.md + .agents/skills/codemap/SKILL.md + templates/agents/skills/codemap/SKILL.md (mirrored 4-way per Rule 10): the YAML frontmatter doc was showing inline-flow shape (actions: [{type, ...}]) but the loader's hand- rolled parser only accepts block-list (- type:). Authors copying inline form would silently lose actions. Replaced with a fenced block showing the correct block-list form. - templates/recipes/deprecated-symbols.md: WHERE name → WHERE callee_name. Verified via pragma_table_info — the calls table has caller_name + callee_name + caller_scope + file_path + id; no bare 'name' column. The recipe doc would have pointed agents at an invalid query. - templates/recipes/fan-in.md + fan-out.md: 'most-imported' / 'they import from many other files' → 'most depended-on' / 'they depend on many other files'. The dependencies table aggregates static imports + dynamic imports + resolved module- graph edges, so the import-only framing was narrower than what the metric measures. --- .agents/rules/codemap.md | 15 +++++++++- .agents/skills/codemap/SKILL.md | 2 +- README.md | 2 +- docs/glossary.md | 2 +- src/application/recipes-loader.test.ts | 35 ++++++++++++++++++++++++ src/application/recipes-loader.ts | 10 ++++++- templates/agents/rules/codemap.md | 15 +++++++++- templates/agents/skills/codemap/SKILL.md | 2 +- templates/recipes/deprecated-symbols.md | 2 +- templates/recipes/fan-in.md | 2 +- templates/recipes/fan-out.md | 2 +- 11 files changed, 79 insertions(+), 10 deletions(-) diff --git a/.agents/rules/codemap.md b/.agents/rules/codemap.md index 801711ce..68d8b23b 100644 --- a/.agents/rules/codemap.md +++ b/.agents/rules/codemap.md @@ -29,7 +29,20 @@ A local database (default **`.codemap.db`**) indexes structure: symbols, imports **Recipe `actions`:** with **`--json`**, recipes that define an `actions` template append it to every row (kebab-case verb + description — e.g. `fan-out` → `review-coupling`). Under `--baseline`, actions attach to the **`added`** rows only. Inspect via **`--recipes-json`**. Ad-hoc SQL never carries actions. -**Project-local recipes:** drop `.sql` (and optional `.md` for description + actions) into **`/.codemap/recipes/`** — auto-discovered, runs via `--recipe ` like bundled. Project recipes win on id collision; check `--recipes-json` for **`shadows: true`** entries to know when a project recipe overrides the documented bundled version. `.md` supports YAML frontmatter (`actions: [{type, auto_fixable?, description?}]`) for the per-row action template — same shape as bundled recipes. Validation: SQL is rejected at load time if it starts with DML/DDL (DELETE/DROP/UPDATE/etc.); the runtime `PRAGMA query_only=1` is the parser-proof backstop. +**Project-local recipes:** drop `.sql` (and optional `.md` for description + actions) into **`/.codemap/recipes/`** — auto-discovered, runs via `--recipe ` like bundled. Project recipes win on id collision; check `--recipes-json` for **`shadows: true`** entries to know when a project recipe overrides the documented bundled version. `.md` supports YAML frontmatter for the per-row action template — block-list shape only (the loader's hand-rolled parser doesn't accept inline-flow `[{...}]`): + +```markdown +--- +actions: + - type: review-coupling + auto_fixable: false + description: "High fan-out usually means orchestrator role." +--- + +(Markdown body — first non-empty line becomes the catalog description.) +``` + +Validation: SQL is rejected at load time if it starts with DML/DDL (DELETE/DROP/UPDATE/etc.); the runtime `PRAGMA query_only=1` is the parser-proof backstop. **Baselines** (`query_baselines` table inside `.codemap.db`, no parallel JSON files): `--save-baseline[=]` snapshots a result set; `--baseline[=]` diffs the current result against it (added / removed rows; identity = `JSON.stringify(row)`). Name defaults to the `--recipe` id; ad-hoc SQL needs an explicit `=`. Survives `--full` and SCHEMA bumps. diff --git a/.agents/skills/codemap/SKILL.md b/.agents/skills/codemap/SKILL.md index 3ad80e14..b0ff4cb1 100644 --- a/.agents/skills/codemap/SKILL.md +++ b/.agents/skills/codemap/SKILL.md @@ -45,7 +45,7 @@ Replace placeholders (`'...'`) with your module path, file glob, or symbol name. - **`--baseline[=]`** — diff the current result against the saved baseline. Output `{baseline:{...}, current_row_count, added: [...], removed: [...]}` (with `--json`) or a two-section terminal dump. Identity = per-row multiset equality (canonical `JSON.stringify` keyed frequency map; duplicates preserved). Pair with `--summary` for `{baseline:{...}, current_row_count, added: N, removed: N}`. **Mutually exclusive with `--group-by`.** - **`--baselines`** lists saved baselines (no `rows_json` payload); **`--drop-baseline `** deletes one. Both reject every other flag — they're list-only / drop-only operations. - **Per-row recipe `actions`** — recipes that define an **`actions: [{type, auto_fixable?, description?}]`** template append it to every row in **`--json`** output (recipe-only; ad-hoc SQL never carries actions). Under `--baseline`, actions attach to the **`added`** rows only (the rows the agent should act on). Inspect via **`--recipes-json`**. -- **Project-local recipes** — drop **`.sql`** (and optional **`.md`** for description body + actions) into **`/.codemap/recipes/`** to make team-internal SQL a first-class CLI verb. `--recipes-json` and the `codemap://recipes` MCP resource list project recipes alongside bundled ones with **`source: "bundled" | "project"`** discriminating them. Project recipes win on id collision; entries that override a bundled id carry **`shadows: true`** so agents reading the catalog at session start know when a recipe behaves differently from the documented bundled version. `.md` supports YAML frontmatter (`---\nactions:\n - type: ...\n---`) for the per-row action template — same shape as bundled. Validation: SQL is rejected at load time if it starts with DML/DDL (DELETE/DROP/UPDATE/etc.); the runtime `PRAGMA query_only=1` is the parser-proof backstop. `.codemap.db` is gitignored; **`.codemap/recipes/` is NOT** — recipes are git-tracked source code authored for human review. +- **Project-local recipes** — drop **`.sql`** (and optional **`.md`** for description body + actions) into **`/.codemap/recipes/`** to make team-internal SQL a first-class CLI verb. `--recipes-json` and the `codemap://recipes` MCP resource list project recipes alongside bundled ones with **`source: "bundled" | "project"`** discriminating them. Project recipes win on id collision; entries that override a bundled id carry **`shadows: true`** so agents reading the catalog at session start know when a recipe behaves differently from the documented bundled version. `.md` supports YAML frontmatter for the per-row action template — **block-list shape only** (loader's hand-rolled parser; no inline-flow `[{...}]`): `---\nactions:\n - type: my-verb\n auto_fixable: false\n description: "..."\n---`. Validation: SQL is rejected at load time if it starts with DML/DDL (DELETE/DROP/UPDATE/etc.); the runtime `PRAGMA query_only=1` is the parser-proof backstop. `.codemap.db` is gitignored; **`.codemap/recipes/` is NOT** — recipes are git-tracked source code authored for human review. **Audit (`bun src/index.ts audit`)** — separate top-level command for structural-drift verdicts. Composes B.6 baselines into a per-delta `{head, deltas}` envelope; v1 ships `files` / `dependencies` / `deprecated`. Two snapshot-source shapes: diff --git a/README.md b/README.md index c8ade911..483b3178 100644 --- a/README.md +++ b/README.md @@ -114,7 +114,7 @@ codemap query --print-sql fan-out # Bundled recipes live in templates/recipes/ in the npm package; project recipes win on id collision # (shadowing is signalled via a `shadows: true` field in --recipes-json so agents notice the override) mkdir -p .codemap/recipes -echo "SELECT path FROM files WHERE language = 'typescript' AND line_count > 500" \ +echo "SELECT path FROM files WHERE language IN ('ts', 'tsx') AND line_count > 500" \ > .codemap/recipes/big-ts-files.sql codemap query --recipe big-ts-files # auto-discovered alongside bundled diff --git a/docs/glossary.md b/docs/glossary.md index 40ded28d..3f650ef9 100644 --- a/docs/glossary.md +++ b/docs/glossary.md @@ -12,7 +12,7 @@ Alphabetical, lowercase. Disambiguation pairs link to each other. - **TS shape** = a TypeScript interface or type alias. - **SQLite table** = an actual on-disk table in `.codemap.db`. -- **Recipe** = a bundled SQL string in `src/cli/query-recipes.ts`, exposed via `codemap query --recipe `. +- **Recipe** = a cataloged SQL recipe loaded by `src/application/recipes-loader.ts` from `templates/recipes/.{sql,md}` (bundled) or `/.codemap/recipes/.{sql,md}` (project-local). Exposed via `codemap query --recipe ` and the `codemap://recipes` MCP resource. See [§ R recipe](#recipe). - **Query** = any SQL run against the index (recipe or ad-hoc). --- diff --git a/src/application/recipes-loader.test.ts b/src/application/recipes-loader.test.ts index 6860315a..c3be2ef1 100644 --- a/src/application/recipes-loader.test.ts +++ b/src/application/recipes-loader.test.ts @@ -286,6 +286,41 @@ describe("validateRecipeSql — load-time DML/DDL deny-list", () => { validateRecipeSql("bad", "/tmp/bad.sql", "drop table x\n"), ).toThrow(/read-only/); }); + + it("strips block /* */ comments before deciding the first keyword", () => { + // Without block-comment stripping, this would mis-detect 'INSERT' from the + // comment text and reject a legitimate SELECT recipe. + expect(() => + validateRecipeSql( + "ok", + "/tmp/ok.sql", + "/* notes about INSERT semantics — see issue #42 */\nSELECT 1\n", + ), + ).not.toThrow(); + }); + + it("rejects DELETE smuggled after a leading block comment (defence in depth)", () => { + // A bare `/* SELECT */ DELETE FROM x` would have slipped past a + // comment-blind first-keyword scan; block-comment stripping makes the + // deny-list see the real first keyword. + expect(() => + validateRecipeSql( + "bad", + "/tmp/bad.sql", + "/* SELECT */ DELETE FROM files\n", + ), + ).toThrow(/read-only/); + }); + + it("rejects pure-block-comment files as empty (no SQL after stripping)", () => { + expect(() => + validateRecipeSql( + "blank", + "/tmp/blank.sql", + "/* placeholder, no SQL yet */\n", + ), + ).toThrow(/empty/); + }); }); describe("extractFrontmatterAndBody — YAML actions parser", () => { diff --git a/src/application/recipes-loader.ts b/src/application/recipes-loader.ts index ec2e6db3..484cb0bc 100644 --- a/src/application/recipes-loader.ts +++ b/src/application/recipes-loader.ts @@ -197,7 +197,15 @@ function firstSqlKeyword(sql: string): string | undefined { } function stripLineComments(sql: string): string { - return sql + // Strip block comments first so that a leading `/* DELETE FROM x */` doesn't + // smuggle a deny-listed keyword past the lexer, and so that pure-comment + // recipes (block-comment only, no actual SQL) trip the empty-recipe check. + // Greedy-but-non-overlapping match; doesn't try to track nested comments + // (SQLite doesn't support them) or escape sequences inside strings (recipes + // mixing block comments with string literals are vanishingly rare and the + // runtime PRAGMA query_only=1 backstop catches anything that slips by). + const noBlock = sql.replace(/\/\*[\s\S]*?\*\//g, ""); + return noBlock .split("\n") .map((line) => { const idx = line.indexOf("--"); diff --git a/templates/agents/rules/codemap.md b/templates/agents/rules/codemap.md index 8c10807f..9d5733e2 100644 --- a/templates/agents/rules/codemap.md +++ b/templates/agents/rules/codemap.md @@ -36,7 +36,20 @@ Install **[@stainless-code/codemap](https://www.npmjs.com/package/@stainless-cod **Recipe `actions`:** with **`--json`**, recipes that define an `actions` template append it to every row (kebab-case verb + description — e.g. `fan-out` → `review-coupling`). Under `--baseline`, actions attach to the **`added`** rows only. Inspect via **`--recipes-json`**. Ad-hoc SQL never carries actions. -**Project-local recipes:** drop `.sql` (and optional `.md` for description + actions) into **`/.codemap/recipes/`** — auto-discovered, runs via `codemap query --recipe ` like bundled. Project recipes win on id collision; check `codemap query --recipes-json` for **`shadows: true`** entries to know when a project recipe overrides the documented bundled version. `.md` supports YAML frontmatter (`actions: [{type, auto_fixable?, description?}]`) for the per-row action template — same shape as bundled recipes. Validation: SQL is rejected at load time if it starts with DML/DDL (DELETE/DROP/UPDATE/etc.); the runtime `PRAGMA query_only=1` is the parser-proof backstop. +**Project-local recipes:** drop `.sql` (and optional `.md` for description + actions) into **`/.codemap/recipes/`** — auto-discovered, runs via `codemap query --recipe ` like bundled. Project recipes win on id collision; check `codemap query --recipes-json` for **`shadows: true`** entries to know when a project recipe overrides the documented bundled version. `.md` supports YAML frontmatter for the per-row action template — block-list shape only (the loader's hand-rolled parser doesn't accept inline-flow `[{...}]`): + +```markdown +--- +actions: + - type: review-coupling + auto_fixable: false + description: "High fan-out usually means orchestrator role." +--- + +(Markdown body — first non-empty line becomes the catalog description.) +``` + +Validation: SQL is rejected at load time if it starts with DML/DDL (DELETE/DROP/UPDATE/etc.); the runtime `PRAGMA query_only=1` is the parser-proof backstop. **Baselines** (`query_baselines` table inside `.codemap.db`, no parallel JSON files): `--save-baseline[=]` snapshots a result set; `--baseline[=]` diffs the current result against it (added / removed rows; identity = `JSON.stringify(row)`). Name defaults to the `--recipe` id; ad-hoc SQL needs an explicit `=`. Survives `--full` and SCHEMA bumps. diff --git a/templates/agents/skills/codemap/SKILL.md b/templates/agents/skills/codemap/SKILL.md index d61c89a7..60de5873 100644 --- a/templates/agents/skills/codemap/SKILL.md +++ b/templates/agents/skills/codemap/SKILL.md @@ -45,7 +45,7 @@ Replace placeholders (`'...'`) with your module path, file glob, or symbol name. - **`--baseline[=]`** — diff the current result against the saved baseline. Output `{baseline:{...}, current_row_count, added: [...], removed: [...]}` (with `--json`) or a two-section terminal dump. Identity = per-row multiset equality (canonical `JSON.stringify` keyed frequency map; duplicates preserved). Pair with `--summary` for `{baseline:{...}, current_row_count, added: N, removed: N}`. **Mutually exclusive with `--group-by`.** - **`--baselines`** lists saved baselines (no `rows_json` payload); **`--drop-baseline `** deletes one. Both reject every other flag — they're list-only / drop-only operations. - **Per-row recipe `actions`** — recipes that define an **`actions: [{type, auto_fixable?, description?}]`** template append it to every row in **`--json`** output (recipe-only; ad-hoc SQL never carries actions). Under `--baseline`, actions attach to the **`added`** rows only (the rows the agent should act on). Inspect via **`--recipes-json`**. -- **Project-local recipes** — drop **`.sql`** (and optional **`.md`** for description body + actions) into **`/.codemap/recipes/`** to make team-internal SQL a first-class CLI verb. `--recipes-json` and the `codemap://recipes` MCP resource list project recipes alongside bundled ones with **`source: "bundled" | "project"`** discriminating them. Project recipes win on id collision; entries that override a bundled id carry **`shadows: true`** so agents reading the catalog at session start know when a recipe behaves differently from the documented bundled version. `.md` supports YAML frontmatter (`---\nactions:\n - type: ...\n---`) for the per-row action template — same shape as bundled. Validation: SQL is rejected at load time if it starts with DML/DDL (DELETE/DROP/UPDATE/etc.); the runtime `PRAGMA query_only=1` is the parser-proof backstop. `.codemap.db` is gitignored; **`.codemap/recipes/` is NOT** — recipes are git-tracked source code authored for human review. +- **Project-local recipes** — drop **`.sql`** (and optional **`.md`** for description body + actions) into **`/.codemap/recipes/`** to make team-internal SQL a first-class CLI verb. `--recipes-json` and the `codemap://recipes` MCP resource list project recipes alongside bundled ones with **`source: "bundled" | "project"`** discriminating them. Project recipes win on id collision; entries that override a bundled id carry **`shadows: true`** so agents reading the catalog at session start know when a recipe behaves differently from the documented bundled version. `.md` supports YAML frontmatter for the per-row action template — **block-list shape only** (loader's hand-rolled parser; no inline-flow `[{...}]`): `---\nactions:\n - type: my-verb\n auto_fixable: false\n description: "..."\n---`. Validation: SQL is rejected at load time if it starts with DML/DDL (DELETE/DROP/UPDATE/etc.); the runtime `PRAGMA query_only=1` is the parser-proof backstop. `.codemap.db` is gitignored; **`.codemap/recipes/` is NOT** — recipes are git-tracked source code authored for human review. **Audit (`codemap audit`)** — separate top-level command for structural-drift verdicts. Composes B.6 baselines into a per-delta `{head, deltas}` envelope; v1 ships `files` / `dependencies` / `deprecated`. Two snapshot-source shapes: diff --git a/templates/recipes/deprecated-symbols.md b/templates/recipes/deprecated-symbols.md index b66e5a85..b47f599f 100644 --- a/templates/recipes/deprecated-symbols.md +++ b/templates/recipes/deprecated-symbols.md @@ -6,4 +6,4 @@ actions: Symbols whose JSDoc contains @deprecated (caller-warning candidates) -Useful for agents to flag callers of soon-to-be-removed APIs before suggesting changes. Pair with `WHERE name = ''` against the `calls` table to find the actual call sites. +Useful for agents to flag callers of soon-to-be-removed APIs before suggesting changes. Pair with `WHERE callee_name = ''` against the `calls` table to find the actual call sites. diff --git a/templates/recipes/fan-in.md b/templates/recipes/fan-in.md index 09d9d74f..e72ffddd 100644 --- a/templates/recipes/fan-in.md +++ b/templates/recipes/fan-in.md @@ -6,4 +6,4 @@ actions: Top 15 files by fan-in (how many other files depend on them) -Files at the top are the most-imported in the codebase — changes here ripple through many consumers. Protect with tests before refactoring; treat as the project's de-facto stable API even if not formally exported. +Files at the top are the most depended-on in the codebase (the `dependencies` table aggregates static imports, dynamic imports, and resolved module-graph edges) — changes here ripple through many consumers. Protect with tests before refactoring; treat as the project's de-facto stable API even if not formally exported. diff --git a/templates/recipes/fan-out.md b/templates/recipes/fan-out.md index 5bebaf44..400c0bff 100644 --- a/templates/recipes/fan-out.md +++ b/templates/recipes/fan-out.md @@ -6,4 +6,4 @@ actions: Top 10 files by dependency fan-out (edge count) -Files at the top of this list act as orchestrators — they import from many other files. High fan-out usually means coordination logic that's a candidate for refactoring (extracting helpers, splitting responsibilities). Pair with `fan-in` to see hubs that are both depended-on AND depend-on-many. +Files at the top of this list act as orchestrators — they depend on many other files (the `dependencies` table aggregates static imports, dynamic imports, and resolved module-graph edges). High fan-out usually means coordination logic that's a candidate for refactoring (extracting helpers, splitting responsibilities). Pair with `fan-in` to see hubs that are both depended-on AND depend-on-many.