diff --git a/.agents/skills/harden-pr/SKILL.md b/.agents/skills/harden-pr/SKILL.md index 2f24546d..07635be2 100644 --- a/.agents/skills/harden-pr/SKILL.md +++ b/.agents/skills/harden-pr/SKILL.md @@ -53,16 +53,16 @@ Default to **lite** when invoked immediately after a slice commit. Default to ** Reviewers optimize for this bar on in-scope files. **Full** mode applies it to the entire `origin/main...HEAD` diff; **lite** to the slice diff. -| Area | Pristine = | -| --------------- | ------------------------------------------------------------------------------------------------------------------ | -| **Correctness** | No known bugs or unhandled edge cases in changed paths; behavior matches intent anchor | -| **Tests** | Changed behavior covered; affected tests pass | -| **Checks** | Format, lint, typecheck clean on touched files ([`verify-after-each-step`](../../rules/verify-after-each-step.md)) | -| **Docs** | User-visible changes reflected in docs, changesets, help text — no drift | -| **Surfaces** | No maintainer leaks into consumer surfaces ([`consumer-surfaces`](../../rules/consumer-surfaces.md)) | -| **Structure** | No boundary violations or barrel bypasses in the diff | -| **Hygiene** | No dead code, TODO slop, or sloppy naming in touched files; errors actionable | -| **Ship shape** | A reviewer could merge without "fix before ship" notes (except deferred out-of-scope nits) | +| Area | Pristine = | +| --------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| **Correctness** | No known bugs or unhandled edge cases in changed paths; behavior matches intent anchor | +| **Tests** | Changed behavior covered; affected tests pass | +| **Checks** | Format, lint, typecheck clean on touched files ([`verify-after-each-step`](../../rules/verify-after-each-step.md)) | +| **Docs** | User-visible changes reflected in docs, changesets, help text — no drift; **shipped `docs/plans/.md` deleted + lifted** in the same PR ([`docs-governance`](../docs-governance/SKILL.md) § Closing a plan) | +| **Surfaces** | No maintainer leaks into consumer surfaces ([`consumer-surfaces`](../../rules/consumer-surfaces.md)) | +| **Structure** | No boundary violations or barrel bypasses in the diff | +| **Hygiene** | No dead code, TODO slop, or sloppy naming in touched files; errors actionable | +| **Ship shape** | A reviewer could merge without "fix before ship" notes (except deferred out-of-scope nits) | If a finding moves the bar toward pristine and stays in-bounds → **fix it**, including nits in touched files. @@ -80,7 +80,9 @@ Reviewers treat the anchor as contract. Findings that would violate it → **rep **Fix:** bugs, missing tests, docs/changeset drift, lint/type/format, error-handling gaps, edge cases, **behavior-preserving refactors in touched files**, in-scope nits (naming, comment hygiene, cheap lint fixes). -**Report only:** redesign, new capabilities, semantic API changes, nits outside the diff, refactors unrelated to a flagged issue. +**Report only:** redesign, semantic API changes, nits outside the diff, refactors unrelated to a flagged issue. + +**Do not defer complements:** agent-surface parity (rule/skill/MCP), glossary/architecture/golden-queries contracts, script/golden tests for acceptance criteria, and cross-links named in the plan ship in the **same PR** — not "optional v2" or post-merge unless the plan **Out of scope** section explicitly excludes them. ## Reviewer roster @@ -89,7 +91,7 @@ Spawn applicable reviewers **in parallel** via subagents in **one batch per pass ### Core (always) 1. **Correctness** — gaps vs production bar; bugs, edge cases, missing tests in changed paths -2. **Ship-readiness** — gaps vs production bar; docs, changesets, consumer-surface leaks, error messages; run [`verify-after-each-step`](../../rules/verify-after-each-step.md) checks on touched files +2. **Ship-readiness** — gaps vs production bar; docs, changesets, consumer-surface leaks, error messages; **grep inbound refs → delete shipped plan file → lift to `golden-queries.md` / `architecture.md` / `roadmap.md`**; run [`verify-after-each-step`](../../rules/verify-after-each-step.md) checks on touched files 3. **Structure (lite)** — gaps vs production bar; boundary smells on the diff (imports across declared layers, barrel bypasses); query codemap per [`codemap`](../codemap/SKILL.md) ### Extended (adaptive — spawn when diff triggers match) @@ -122,7 +124,7 @@ loop: pass += 1 goto loop capped: - emit deferred-nits list + emit deferred-nits list (each nit must cite plan Out of scope or cross-PR blocker — not "optional") done: if uncommitted fixes → git commit -m "harden: …" emit final report (include babysit one-liner if full mode) diff --git a/.changeset/high-crap-score.md b/.changeset/high-crap-score.md new file mode 100644 index 00000000..91b825f7 --- /dev/null +++ b/.changeset/high-crap-score.md @@ -0,0 +1,7 @@ +--- +"@stainless-code/codemap": patch +--- + +Add `high-crap-score` recipe: CRAP ranking with measured coverage when ingested, or graph-estimated 85/40/0% tiers from test reachability otherwise. + +Extend `unimported-exports` with `unresolved_import_blind_spot` reason and `evidence_json` (unresolved import hop) so dead-export / high-CRAP triage does not over-trust the graph past alias blind spots. diff --git a/docs/architecture.md b/docs/architecture.md index 26e8f994..ef11ffda 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -192,6 +192,8 @@ Three **mutually exclusive** CLI entry shapes; all converge on `applyDiffPayload **Evidence columns (high-judgment recipes):** Some bundled recipes add optional **`reason`** and **`evidence_json`** TEXT columns on each result row — factual detection path for agents, not pass/fail verdicts. Contract: [golden-queries.md § Evidence columns](./golden-queries.md#evidence-columns-high-judgment-recipes). +**Coverage columns (CRAP recipes):** `high-crap-score` adds **`coverage_source`** and **`effective_coverage_pct`** — measured vs graph-estimated undertest signal. Contract: [golden-queries.md § Coverage columns](./golden-queries.md#coverage-columns-crap--enrichment-recipes). + **Recipes wiring:** **`src/application/recipes-loader.ts`** (pure transport-agnostic loader) + **`src/application/query-recipes.ts`** (cache + public API — `getQueryRecipeSql` / `getQueryRecipeActions` / `getQueryRecipeParams` / `listQueryRecipeIds` / `listQueryRecipeCatalog` / `getQueryRecipeCatalogEntry`, shared by CLI + MCP). Recipes live as file pairs: **`.sql`** + optional **`.md`**. The loader reads `templates/recipes/` (bundled, ships in npm package next to `templates/agents/`) and `/recipes/` (project-local — default `.codemap/recipes/`; honors `--state-dir` / `CODEMAP_STATE_DIR`; root-only resolution per the registry plan, no walk-up). Project recipes win on id collision; entries that override a bundled id carry **`shadows: true`** in the catalog so agents reading `codemap://recipes` at session start see when a recipe behaves differently from the documented bundled version. Per-row **`actions`** templates and recipe **`params`** declarations live in YAML frontmatter on each `.md` — uniform shape across bundled + project. Param types are `string | number | boolean`; CLI passes values via repeatable `--params key=value[,key=value]`, MCP / HTTP pass nested `params: {key: value}` to `query_recipe`. Validation runs before SQL binding; missing / unknown / malformed params return the same `{error}` envelope as query failures. Hand-rolled YAML parser is scoped to block-list `actions:` and `params:` only (no `js-yaml` dep). Load-time validation rejects empty SQL and DML / DDL keywords (`INSERT` / `UPDATE` / `DELETE` / `DROP` / `CREATE` / `ALTER` / `ATTACH` / `DETACH` / `REPLACE` / `TRUNCATE` / `VACUUM` / `PRAGMA`) with recipe-aware error messages — defence in depth alongside the runtime `PRAGMA query_only=1` backstop in `query-engine.ts` (PR #35). `/index.db` is gitignored; `/recipes/` is NOT (verified via `git check-ignore`) — recipes are git-tracked source code authored for human review. **Tool / resource handlers (transport-agnostic):** **`src/application/tool-handlers.ts`** + **`src/application/resource-handlers.ts`** — pure functions that take the args object an MCP tool / resource URI accepts and return a discriminated **`ToolResult`** (`{ok: true, format: 'json'|'sarif'|'annotations'|'mermaid'|'diff'|'diff-json'|'codeclimate'|'badge', payload}` — badge arm also carries `badgeStyle`; `{ok: false, error}`) or a **`ResourcePayload`** (`{mimeType, text}`). MCP and HTTP both wrap the same handlers — MCP translates to `{content: [{type: "text", text}]}`, HTTP translates to `(status, body)` with the right `Content-Type`. Engine layer untouched; transport changes don't ripple into the SQL. diff --git a/docs/glossary.md b/docs/glossary.md index 03840b73..741d170a 100644 --- a/docs/glossary.md +++ b/docs/glossary.md @@ -151,6 +151,10 @@ Opt-in FTS5 virtual table over file content (`tokenize='porter unicode61'`). Alw Output mode rendering `{from, to, label?, kind?}` rows as a Mermaid `flowchart LR` diagram. Sibling of `--format sarif` / `--format annotations` in `application/output-formatters.ts`. **Bounded-input contract** (50-edge ceiling; `MERMAID_MAX_EDGES`) — unbounded inputs reject with a scope-suggestion error naming the recipe + count + scoping knobs (`LIMIT` / `--via` / `WHERE`). Auto-truncation explicitly out of scope (would be a verdict masquerading as output mode, violating the predicate-as-API moat). Recipes / ad-hoc SQL must alias columns to the `{from, to}` shape (e.g. `SELECT from_path AS "from", to_path AS "to" FROM dependencies LIMIT 50`). +### CRAP score / `high-crap-score` / `coverage_source` / `effective_coverage_pct` + +Change Risk Anti-Patterns score per published formula: `CC² × (1 - effective_coverage/100)³ + CC` where `CC = symbols.complexity`. Bundled recipe **`high-crap-score`** ranks symbols at or above `min_crap` (default 30). **`effective_coverage_pct`** uses ingested **`coverage.coverage_pct`** when a row exists (**`coverage_source: measured`**), else graph-estimated tiers from test reachability (**`coverage_source: estimated`**: 85% direct test reference, 40% file reachable from tests via value-only **`dependencies`**, 0% otherwise). Heuristic only — not execution coverage; prefer **`codemap ingest-coverage`** before CI gates. Complements **`high-complexity-untested`** when coverage is not ingested. + ### `coverage` (table) Statement coverage ingested from Istanbul JSON, LCOV, or V8 runtime (`NODE_V8_COVERAGE=...` directory via `--runtime`) via `codemap ingest-coverage `. Natural-key PK `(file_path, name, line_start)` — intentionally **not** a FK to `symbols.id` because `symbols` re-creates with fresh AUTOINCREMENT ids on every `--full` reindex; the natural-key approach lets coverage rows survive that churn (`coverage` is also intentionally absent from `dropAll()`, joins the `query_baselines` precedent). Columns: `coverage_pct REAL` (`NULL` when `total_statements = 0` — "untested" and "no testable code" are different signals), `hit_statements`, `total_statements`. Orphan rows (file deleted from project) are cleaned by an explicit `DELETE FROM coverage WHERE file_path NOT IN (SELECT path FROM files)` at the end of every ingest. Three meta keys (`coverage_last_ingested_at` / `_path` / `_format`) record freshness — single ingest at a time, so format is meta-level not per-row. diff --git a/docs/golden-queries.md b/docs/golden-queries.md index 2ee5aad2..489c8303 100644 --- a/docs/golden-queries.md +++ b/docs/golden-queries.md @@ -68,7 +68,11 @@ Scenarios live in **`fixtures/golden/scenarios.json`** (Tier A) or optional **`s ### Evidence columns (high-judgment recipes) -Some bundled recipes add optional **`reason`** (TEXT) and **`evidence_json`** (TEXT, JSON array) columns on each row — factual detection path for agents, not engine verdicts. See [plans/evidence-chains-on-recipe-rows.md](./plans/evidence-chains-on-recipe-rows.md). Goldens assert these columns when the recipe ships evidence (`boundary-violations`, `deprecated-symbols`, `unimported-exports`). +Some bundled recipes add optional **`reason`** (TEXT) and **`evidence_json`** (TEXT, JSON array) columns on each row — factual detection path for agents, not engine verdicts (Moat A — not pass/fail verdicts). Bounded subqueries cap evidence at three hops; list caps append `{"truncated":true}`. `unimported-exports` reasons: `no_direct_import`, `reexport_chain_possible`, `unresolved_import_blind_spot`. Goldens assert these columns when the recipe ships evidence (`boundary-violations`, `deprecated-symbols`, `unimported-exports`). + +### Coverage columns (CRAP / enrichment recipes) + +`high-crap-score` adds **`coverage_source`** (`measured` \| `estimated`) and **`effective_coverage_pct`** on each row — measured when `coverage` has a matching symbol row after `ingest-coverage`; otherwise graph-estimated tiers from test reachability. Goldens assert `coverage_source` when the recipe ships coverage semantics (`high-crap-score`); measured override is covered by `scripts/high-crap-score-measured.test.mjs`. --- diff --git a/docs/plans/agent-enrichment-wave.md b/docs/plans/agent-enrichment-wave.md index 03e8dfbc..90df7ccf 100644 --- a/docs/plans/agent-enrichment-wave.md +++ b/docs/plans/agent-enrichment-wave.md @@ -1,52 +1,29 @@ -# Agent enrichment wave — tracer workflow (plans 1–4) +# Agent enrichment wave — tracer workflow (plans 3–4) -> **Status:** in-flight · **Scope:** four P2 plans ranked by consumer/agent ROI +> **Status:** in-flight · **Scope:** remaining P2 plans ranked by consumer/agent ROI > > **Goal:** Ship tracer bullets that cut agent round-trips, improve answer trust, and sharpen PR/CI deltas — all Moat-A (predicate columns, no verdict primitives). > -> **Plans (execution order):** [evidence-chains](./evidence-chains-on-recipe-rows.md) → [graph-estimated-crap](./graph-estimated-crap.md) → [coverage-deletion-confidence](./coverage-deletion-confidence.md) → [audit-delta-attribution](./audit-delta-attribution.md) +> **Shipped (plans retired):** Evidence chains ([#174](https://github.com/stainless-code/codemap/pull/174)) · Graph-estimated CRAP ([#175](https://github.com/stainless-code/codemap/pull/175)) — durable contract in `golden-queries.md` + `architecture.md`; plan files deleted per [docs-governance](../../.agents/skills/docs-governance/SKILL.md) § Closing a plan. +> +> **Remaining:** [coverage-deletion-confidence](./coverage-deletion-confidence.md) → [audit-delta-attribution](./audit-delta-attribution.md) --- ## Shared conventions (locked) -| Convention | Applies to | -| ------------------------------------------------------------------------------------- | ---------- | -| **Moat A** — no `pass`/`fail` engine verdict; extra columns only | All four | -| **`reason` TEXT** — machine code + short clause where useful | #1, #3 | -| **`evidence_json` TEXT** — bounded JSON array (≤3 hops) | #1 | -| **`confidence` / `coverage_source` / `attribution`** — recipe-specific enums | #2, #3, #4 | -| **Golden update per slice** — `fixtures/golden/minimal/*.json` + `scenarios.json` | All | -| **`/harden-pr lite`** after each tracer commit; **`/harden-pr full`** before PR merge | All | - -**Cross-plan synergy:** #1 `reason` on recipes complements #4 `attribution` on audit `added` rows (optional merge in evidence plan v2). #2 and #3 both touch coverage semantics — ship #2 before #3 so agents have CRAP tiers before deletion-confidence narrows rows. - ---- - -## Plan 1 — Evidence chains (`evidence-chains-on-recipe-rows.md`) - -| Slice | Deliverable | Verify | -| ----------------------------- | ------------------------------------------------------------------ | --------------------- | -| **1.0 contract** | `docs/golden-queries.md` § evidence columns; one architecture line | doc review | -| **1.1 `boundary-violations`** | `reason` + `evidence_json` in SQL; `.md` + golden | `bun run test:golden` | -| **1.2 `deprecated-symbols`** | caller hops in `evidence_json` | golden + matrix | -| **1.3 `unimported-exports`** | `re_export_chains` LEFT JOIN; `reason` variants | golden | -| **1.4 agent surface** | `templates/agent-content/rule/00-full.md` one-liner | consumer check | - -**Open decisions (locked for v1):** E.2 `evidence_json` only (not typed columns); E.1 SQL-only (no query-engine post-processor). - ---- - -## Plan 2 — Graph-estimated CRAP (`graph-estimated-crap.md`) - -| Slice | Deliverable | Verify | -| ------------------------- | --------------------------------------------------------------- | ----------------- | -| **2.0 spike** | Reachability CTE on `fixtures/minimal` (script or ad-hoc query) | manual row counts | -| **2.1 recipe** | `high-crap-score.sql` + `.md`; `scenarios.json` | `test:golden` | -| **2.2 measured override** | golden with `ingest-coverage` setup | golden matrix | -| **2.3 cross-link** | `high-complexity-untested.md` points at CRAP when no ingest | doc | +| Convention | Applies to | +| ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------- | +| **Moat A** — no `pass`/`fail` engine verdict; extra columns only | All | +| **`reason` TEXT** — machine code + short clause where useful | #3 | +| **`evidence_json` TEXT** — bounded JSON array (≤3 hops) | shipped #1 | +| **`confidence` / `coverage_source` / `attribution`** — recipe-specific enums | #3, #4 | +| **Golden update per slice** — `fixtures/golden/minimal/*.json` + `scenarios.json` | All | +| **`/harden-pr lite`** after each tracer commit; **`/harden-pr full`** before PR merge | All | +| **Retire plan on merge** — delete `docs/plans/.md` + lift to reference docs/roadmap in the **same PR** (never leave shipped plans as leftovers) | All | +| **No deferring complements** — agent surfaces (rule/skill/MCP), glossary, golden/script tests, and plan acceptance items ship **in the same PR** unless explicitly listed under plan **Out of scope** | All | -**Grill before 2.1 if spike ambiguous:** Q1 type-only imports in walk (default: value edges only); Q2 recipe id `high-crap-score`. +**Cross-plan synergy:** shipped evidence `reason` complements #4 `attribution` on audit `added` rows. CRAP `coverage_source` (#175) ships before #3 so deletion-confidence can narrow rows with coverage semantics. --- @@ -77,18 +54,15 @@ ## PR cadence -| PR | Contents | Changeset | -| -------------------------- | ------------------------------------ | --------- | -| **#A Evidence wave 1** | Slices 1.0–1.1 (boundary-violations) | patch | -| **#B Evidence wave 2** | Slices 1.2–1.4 | patch | -| **#C CRAP recipe** | Plan 2 complete | patch | -| **#D Deletion confidence** | Plan 3 complete | patch | -| **#E Audit attribution** | Plan 4 complete | patch | +| PR | Contents | Changeset | Retire plan on merge | +| -------------------------- | --------------- | --------- | --------------------------------- | +| **#D Deletion confidence** | Plan 3 complete | patch | `coverage-deletion-confidence.md` | +| **#E Audit attribution** | Plan 4 complete | patch | `audit-delta-attribution.md` | -Each PR: `harden-pr full` → merge. Do not batch plans 1–4 into one PR. +Each PR: `harden-pr full` (includes plan retirement) → merge. Do not batch plans 3–4 into one PR. --- ## Current slice -**Active:** Plan 1 shipped in [**PR #174**](https://github.com/stainless-code/codemap/pull/174) (awaiting merge) — next: Plan 2 spike **2.0** (`graph-estimated-crap.md`). +**Active:** Plan 3 slice **3.1** on `feat/high-crap-score` or fresh branch from `main` after **#175** merges — `coverage-confirmed-dead` recipe fork. diff --git a/docs/plans/apply-write-safety.md b/docs/plans/apply-write-safety.md index 7b84c4c7..66443354 100644 --- a/docs/plans/apply-write-safety.md +++ b/docs/plans/apply-write-safety.md @@ -106,4 +106,4 @@ bun src/index.ts apply --dry-run ## Dependencies - Shipped: `apply-engine.ts`, `apply-run.ts`, `hashContent`, apply confirmation gates (`--yes`) -- Synergy: [evidence-chains-on-recipe-rows](./evidence-chains-on-recipe-rows.md) (agents should dry-run then apply with safety net) +- Synergy: [golden-queries.md § Evidence columns](../golden-queries.md#evidence-columns-high-judgment-recipes) (agents parse `reason` / `evidence_json` before `apply`) diff --git a/docs/plans/evidence-chains-on-recipe-rows.md b/docs/plans/evidence-chains-on-recipe-rows.md deleted file mode 100644 index 79acb6d1..00000000 --- a/docs/plans/evidence-chains-on-recipe-rows.md +++ /dev/null @@ -1,123 +0,0 @@ -# Evidence chains on recipe rows — plan - -> **Status:** open · **Priority:** P2 · **Effort:** M–L (~2–4 weeks, phased per recipe) -> -> **Motivator:** High-judgment recipes (`unimported-exports`, `boundary-violations`, `deprecated-symbols`, …) return locatable rows but agents still re-query (`barrel-chains`, `fan-in`, `find-symbol-references`) to learn **why** a row appeared before `apply` or manual edits. Enriching rows with compact `reason` + optional `evidence` fields cuts round-trips and reduces false-positive deletions. -> -> **Roadmap:** [§ Recipe & audit enrichment](../roadmap.md#recipe--audit-enrichment) - ---- - -## Agent start here - -Start with **`boundary-violations`** (rule metadata already in row — add `reason` + thin `evidence_json`). Then `deprecated-symbols`, then `unimported-exports` (needs `re_export_chains`). Update golden queries per wave. - -### Key touchpoints - -| File | What to read | -| ---------------------------------------------------------------------------------------------- | --------------------------------------------- | -| [`templates/recipes/boundary-violations.sql`](../../templates/recipes/boundary-violations.sql) | Simplest JOIN to extend | -| [`templates/recipes/unimported-exports.sql`](../../templates/recipes/unimported-exports.sql) | Re-export false-positive class | -| [`templates/recipes/deprecated-symbols.sql`](../../templates/recipes/deprecated-symbols.sql) | Caller subquery target | -| [`src/db.ts`](../../src/db.ts) | `re_export_chains`, `boundary_rules`, `calls` | -| [`docs/golden-queries.md`](../golden-queries.md) | Assert new columns in JSON rows | -| [`docs/architecture.md`](../architecture.md) § Recipes wiring | Recipe `.sql` + `.md` pair contract | - -### Architecture - -```text -recipe SQL SELECT …, reason, evidence_json - → query-engine (unchanged transport) - → JSON rows (extra columns; SARIF/annotations ignore unless mapped later) -``` - -Evidence is **in-SQL**, not a post-processor — same Moat-A path as the recipe. - -### Tracer bullet (slice 1) - -`boundary-violations`: add `reason` constant + `evidence_json` with rule tuple; one golden query. Ship before touching `unimported-exports` re-export subquery. **Orchestration:** [agent-enrichment-wave.md](./agent-enrichment-wave.md) § Plan 1 slice 1.1 — shipped `edeee68`. - -### Out of scope (v1) - -`untested-and-dead` / `visibility-tags` evidence (v2); audit `added` row attribution merge (optional v2 with [audit-delta-attribution](./audit-delta-attribution.md)); post-query enrichment engine (Q2). - ---- - -## Pre-locked decisions - -| # | Decision | Source | -| --- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------ | -| E.1 | **SQL columns, not a post-processor** — evidence lives in the recipe `SELECT` (JOINs / scalar subqueries). Same Moat-A path as the recipe itself; no hidden engine verdict. | [Moat A](../roadmap.md#moats-load-bearing) | -| E.2 | **Standard column names** — `reason TEXT` (short machine-readable code + human clause) and optional `evidence_json TEXT` (JSON array of hop objects) on allowlisted recipes only. | Recipe contract extension | -| E.3 | **Bounded evidence** — cap chain depth (e.g. 3 hops) and JSON length in SQL (`LIMIT` in subqueries); truncate with `{truncated: true}` in JSON. | Token budget / agent ergonomics | -| E.4 | **Complements `actions[]`** — frontmatter `actions` stay the UX hint; `reason`/`evidence_json` are factual substrate the agent can cite. | Shipped recipe frontmatter pattern | -| E.5 | **Phased rollout** — v1: `unimported-exports`, `boundary-violations`, `deprecated-symbols`; v2: `untested-and-dead`, `visibility-tags` as patterns prove out. | Tracer bullets | -| E.6 | **Not a confidence verdict** — `reason` explains detection path; optional `confidence_hint: low\|medium` only where recipes already document false-positive classes. | No pass/fail primitive | - ---- - -## Per-recipe evidence (v1) - -| Recipe | `reason` examples | `evidence_json` shape (sketch) | -| --------------------- | ------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------- | -| `unimported-exports` | `no_direct_import` · `reexport_chain_possible` · `unresolved_import_blind_spot` | `[{kind:"reexport",from_file,to_file,hops}]` from `re_export_chains` when export name appears in a chain | -| `boundary-violations` | `boundary_deny_match` | `[{rule_name,from_glob,to_glob}]` (rule metadata already in row — evidence duplicates for stable JSON contract) | -| `deprecated-symbols` | `deprecated_jsdoc` · `has_callers` · `no_callers` | `[{kind:"caller",name,file_path,line_start}]` top-N from `calls` / `references` | - ---- - -## Implementation steps - -**Wave 1 — contract** - -1. Document evidence column convention in `docs/golden-queries.md` (and one line in `docs/architecture.md` § Recipes if warranted). -2. Add golden-query scenarios asserting `reason` present on v1 recipes. - -**Wave 2 — `boundary-violations`** (simplest — rule join already explicit) 3. Add `reason`, `evidence_json` columns to SQL + update `.md` frontmatter description. - -**Wave 3 — `unimported-exports`** 4. LEFT JOIN / subquery against `re_export_chains` for same export `name`; set `reason` when chain exists. 5. Subquery count of `imports` with `resolved_path IS NULL` targeting same module as `unresolved_import_blind_spot` hint (optional v1.1). - -**Wave 4 — `deprecated-symbols`** 6. Attach top 3 caller rows via correlated subquery or CTE; `reason` reflects caller count bucket. - -**Wave 5 — agent surfaces** 7. One line in `templates/agent-content/skill/` — "high-judgment recipe rows may include `reason` / `evidence_json`; parse before `apply`." 8. MCP `query_recipe` tool description note (no new tool). - ---- - -### Verification - -```bash -bun src/index.ts query --recipe boundary-violations --json # wave 2 -bun src/index.ts query --recipe unimported-exports --json # wave 3 -bun src/index.ts query --recipe deprecated-symbols --json # wave 4 -bun test scripts/query-golden-coverage-matrix.test.mjs # golden rows assert reason / evidence_json -``` - -Ship one recipe per wave; verify before moving to the next. - ---- - -## Acceptance - -- [x] `codemap query --recipe unimported-exports --json` rows include `reason`; re-export false-positive class includes non-empty `evidence_json` when chain exists -- [x] `boundary-violations` rows include stable `reason: boundary_deny_match` -- [x] `deprecated-symbols` rows with callers include `evidence_json` caller hops -- [x] Golden queries updated; no new CLI verb -- [x] SARIF / annotations unchanged (extra columns ignored by formatters unless future mapping added) - ---- - -## Open decisions (impl PR) - -| # | Question | Lock (wave 2026-06) | -| --- | ------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------- | -| Q1 | Single `evidence_json` vs separate typed columns (`reexport_hops`, `caller_count`)? | **`evidence_json` only** (E.2) — one JSON contract per recipe row | -| Q2 | Post-query enrichment in `query-engine` for recipes that opt in via frontmatter `evidence: true`? | **SQL-only** (E.1) — no post-processor in v1 | -| Q3 | Include `binding_kind` from `bindings` for rename-preview synergy in v1 or v2? | **v2** | - ---- - -## Dependencies - -- Shipped: `re_export_chains`, `dependencies`, `boundary_rules`, `calls`, `references`, `bindings`, recipe `actions` frontmatter -- Synergy: [audit-delta-attribution](./audit-delta-attribution.md) (attribution + reason on audit `added` rows — optional merge in v2) -- Weaker until: [c9-plugin-layer](./c9-plugin-layer.md) (reachability reasons for `untested-and-dead` evidence in v2) diff --git a/docs/plans/graph-estimated-crap.md b/docs/plans/graph-estimated-crap.md deleted file mode 100644 index 2f5f5323..00000000 --- a/docs/plans/graph-estimated-crap.md +++ /dev/null @@ -1,129 +0,0 @@ -# Graph-estimated CRAP score — plan - -> **Status:** open · **Priority:** P2 · **Effort:** M (~2 weeks) -> -> **Motivator:** CRAP ranks **complex and undertested** functions. Codemap has `symbols.complexity` + ingested `coverage`, but `high-complexity-untested` is **misleading without ingest** (`COALESCE(coverage_pct, 0)` treats missing as 0%). Graph-estimated tiers (85/40/0%) from test reachability when measured coverage is absent. -> -> **Roadmap:** [§ Recipe & audit enrichment](../roadmap.md#recipe--audit-enrichment) - ---- - -## Agent start here - -Spike the **reachability CTE** on `fixtures/minimal` (or codemap self-index) before authoring recipe files. Reuse **`test_suites`** + `affected-tests` glob conventions for test-file seeds. Recipe-only v1 — no schema migration. - -### Key touchpoints - -| File | What to read | -| -------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------- | -| [`templates/recipes/high-complexity-untested.sql`](../../templates/recipes/high-complexity-untested.sql) | Coverage `COALESCE` pattern to replace/extend | -| [`templates/recipes/affected-tests.sql`](../../templates/recipes/affected-tests.sql) | Test file / `test_suites` detection | -| [`templates/recipes/`](../../templates/recipes/) | Frontmatter + param conventions | -| [`src/db.ts`](../../src/db.ts) | `coverage`, `dependencies`, `calls`, `references`, `symbols` DDL | -| [`docs/golden-queries.md`](../golden-queries.md) | Golden scenario registration | - -### Architecture - -```text -recipe high-crap-score (SQL only) - → test_files CTE (test_suites + globs) - → reachable_files (deps from test_files) - → per symbol: measured coverage OR tier 85/40/0 - → CRAP formula → rows + coverage_source column -``` - -### Tracer bullet (slice 1) - -Recipe SQL + `.md` on fixture index without coverage ingest (tiers only). Golden row asserting `coverage_source: estimated`. Second golden with `ingest-coverage` → `measured` overrides. - -### Out of scope (v1) - -`symbols.estimated_coverage_pct` materialised column; new CLI verb; treating estimates as CI gate (document as heuristic only). - ---- - -## Pre-locked decisions - -| # | Decision | Source | -| --- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------ | -| G.1 | **Recipe-first v1** — bundled `high-crap-score` (or `crap-score`) as SQL over existing tables; no new verdict CLI verb. | [Moat A](../roadmap.md#moats-load-bearing) | -| G.2 | **CRAP formula** — `crap_score = CC² × (1 - effective_coverage/100)³ + CC` where `CC = symbols.complexity`; threshold param default 30 (industry convention). | Published CRAP definition | -| G.3 | **Coverage precedence** — per symbol: use ingested `coverage.coverage_pct` when a row exists; else use graph-estimated tier. Column `coverage_source: measured \| estimated`. | Composes with `ingest-coverage` | -| G.4 | **Estimation tiers (v1)** — when no measured coverage row: **85%** if symbol is directly referenced from a test file (`calls` / `references` from `test_suites.file_path`); **40%** if source `file_path` is dependency-reachable from any test file but symbol is not directly referenced; **0%** otherwise. | Static graph heuristic — document false-positive class | -| G.5 | **Test file detection** — seed set = `test_suites.file_path` UNION paths matching default test suffix globs (`*.test.*`, `*.spec.*`, same as `affected-tests`). | Shipped `test_suites` + recipe precedent | -| G.6 | **Heuristic disclaimer** — recipe `.md` states estimates ≠ execution; prefer `ingest-coverage` for CI gates. | Agent-facing honesty | - ---- - -## SQL shape (sketch) - -```sql --- illustrative; final recipe may use CTEs for test_files, reachable_files, symbol_refs -WITH test_files AS ( … ), - reachable_files AS ( … recursive deps from test_files … ), - effective AS ( - SELECT s.*, - COALESCE(c.coverage_pct, - CASE - WHEN direct_test_ref THEN 85 - WHEN file_reachable THEN 40 - ELSE 0 - END) AS effective_coverage_pct, - CASE WHEN c.coverage_pct IS NOT NULL THEN 'measured' ELSE 'estimated' END AS coverage_source - FROM symbols s - LEFT JOIN coverage c ON … - … - ) -SELECT name, file_path, complexity, effective_coverage_pct, coverage_source, - (complexity * complexity * POWER(1 - effective_coverage_pct / 100.0, 3) + complexity) AS crap_score -FROM effective -WHERE complexity IS NOT NULL AND crap_score >= :min_crap -ORDER BY crap_score DESC; -``` - ---- - -## Implementation steps - -1. Spike reachability CTE on `fixtures/minimal` + codemap self-index — validate tier counts vs intuition. -2. Author `templates/recipes/high-crap-score.sql` + `.md` (params: `min_crap`, optional `limit`). -3. Golden scenario with + without `ingest-coverage` on same fixture (measured overrides estimated). -4. Cross-link from `high-complexity-untested.md` as alternative when coverage not ingested. -5. Optional v2 (defer): materialise `symbols.estimated_coverage_pct` at index time if recipe CTE is too slow on 10k+ trees — trigger-gated per perf plan discipline. - ---- - -### Verification - -```bash -bun src/index.ts query --recipe high-crap-score --json --root fixtures/minimal -bun src/index.ts ingest-coverage && bun src/index.ts query --recipe high-crap-score --json -bun test scripts/query-golden-coverage-matrix.test.mjs # after golden scenario added -``` - ---- - -## Acceptance - -- [ ] Without coverage ingest: symbols in files imported by tests get tier 40/85; isolated files get 0% -- [ ] With coverage ingest: `coverage_source = measured` and CRAP uses real `coverage_pct` -- [ ] `codemap query --recipe high-crap-score --json` works; SARIF compatible via `--format sarif` -- [ ] No new pass/fail primitive - ---- - -## Open decisions (impl PR) - -| # | Question | -| --- | ------------------------------------------------------------------------------------------------ | -| Q1 | Include type-only imports in reachability walk? (default: value edges only, mirror import graph) | -| Q2 | Recipe id: `high-crap-score` vs `crap-score`? | -| Q3 | Materialised column at index time vs recipe-only — measure CTE cost on self-index first. | - ---- - -## Dependencies - -- Shipped: `symbols.complexity`, `coverage`, `dependencies`, `calls`, `references`, `test_suites`, `affected-tests` glob conventions -- Synergy: [`symbols.cognitive_complexity`](../glossary.md#symbolscognitive_complexity--cognitive-complexity) (optional second axis in same recipe later); [coverage-deletion-confidence](./coverage-deletion-confidence.md) (opposite signal — dead + zero coverage) -- Weaker until: [c9-plugin-layer](./c9-plugin-layer.md) (framework test files may be misclassified) diff --git a/docs/roadmap.md b/docs/roadmap.md index 24165a84..9e041100 100644 --- a/docs/roadmap.md +++ b/docs/roadmap.md @@ -87,9 +87,9 @@ Long-running MCP / HTTP sessions dominate agent workflows; one-shot CLI keeps th Predicate-as-API only — enrich row shape and audit deltas; no standalone pass/fail verdict primitive ([Moat A](./roadmap.md#moats-load-bearing)). - [ ] **Audit delta attribution** — on `audit --base ` (and matching MCP/HTTP audit), tag each `added` row with **`attribution: introduced | inherited`** via stable finding keys (`requiredColumns` → deterministic key) diffed against the sha-keyed audit-cache index at the merge base. Per-delta `summary` counts (`added_introduced`, `added_inherited`) optional when `summary: true`. Reuses shipped `audit-worktree` / `git archive` cache — no new verdict primitive ([Moat A](./roadmap.md#moats-load-bearing)). Complements deferred **`codemap audit` verdict + thresholds** (consumer filters `introduced` via `jq`). Plan: [`plans/audit-delta-attribution.md`](./plans/audit-delta-attribution.md). Effort: M. -- [ ] **Evidence chains on recipe rows** — extend high-judgment recipe SQL with standard columns `reason` (short detection code + clause) and optional `evidence_json` (bounded hop array): e.g. `unimported-exports` → `re_export_chains` summary / unresolved-import blind-spot hint; `boundary-violations` → matched deny rule; `deprecated-symbols` → top caller sites from `calls` / `references`. Phased v1 on three recipes; complements frontmatter `actions[]` — agents cite evidence before `apply` / manual edits ([Moat A](./roadmap.md#moats-load-bearing)). Plan: [`plans/evidence-chains-on-recipe-rows.md`](./plans/evidence-chains-on-recipe-rows.md). Effort: M–L. +- [x] **Evidence chains on recipe rows** — shipped on `boundary-violations`, `deprecated-symbols`, `unimported-exports` (`reason` + `evidence_json`; includes `reexport_chain_possible` and `unresolved_import_blind_spot`). Contract: [golden-queries.md § Evidence columns](./golden-queries.md#evidence-columns-high-judgment-recipes). v2 optional: audit `added` attribution merge ([Moat A](./roadmap.md#moats-load-bearing)). - [ ] **Tiered lookup fast paths** — `show` / exact-name recipe paths hit covering indexes first; document latency expectations in MCP tool descriptions. FTS and broad scans remain explicit fallbacks. Effort: S–M. -- [ ] **Graph-estimated CRAP recipe** — bundled `high-crap-score`: CRAP = `CC² × (1 - coverage/100)³ + CC` using `symbols.complexity`; **measured** `coverage` when ingested, else **graph-estimated** tiers (85% / 40% / 0% from test-file reachability over `dependencies` / `calls` / `test_suites`). Rows expose `coverage_source: measured | estimated`. Complements `high-complexity-untested` when no coverage file exists. Plan: [`plans/graph-estimated-crap.md`](./plans/graph-estimated-crap.md). Effort: M. +- [x] **Graph-estimated CRAP recipe** — bundled `high-crap-score`: CRAP = `CC² × (1 - coverage/100)³ + CC` using `symbols.complexity`; **measured** `coverage` when ingested, else **graph-estimated** tiers (85% / 40% / 0% from test-file reachability over `dependencies` / `calls` / `test_suites`). Rows expose `coverage_source: measured | estimated`. Contract: [golden-queries.md § Coverage columns](./golden-queries.md#coverage-columns-crap--enrichment-recipes). Complements `high-complexity-untested` when no coverage file exists. - [ ] **Coverage-confirmed dead recipe** — bundled `coverage-confirmed-dead`: JOIN static dead-code predicate (uncalled exports, suppression-aware) with ingested `coverage` — rows carry `confidence: high` when callers = 0 and `coverage_pct = 0`, `medium` when coverage not ingested. Predicate columns only, no verdict primitive ([Moat A](./roadmap.md#moats-load-bearing)). Plan: [`plans/coverage-deletion-confidence.md`](./plans/coverage-deletion-confidence.md). Effort: L–M. ### Distribution & evaluation depth diff --git a/fixtures/CAPABILITIES.json b/fixtures/CAPABILITIES.json index 2a341ec5..e19595cd 100644 --- a/fixtures/CAPABILITIES.json +++ b/fixtures/CAPABILITIES.json @@ -158,12 +158,18 @@ { "id": "coverage.recipes", "description": "coverage ingest and killer / risk recipes", - "fixtureFiles": ["coverage/coverage-final.json", "src/orphan.ts"], + "fixtureFiles": [ + "coverage/coverage-final.json", + "src/lib/alias-blind-spot.ts", + "src/lib/complexity-fixture.ts", + "src/orphan.ts" + ], "goldenScenarios": [ "coverage-rows-after-ingest", "untested-and-dead", "refactor-risk-ranking", - "files-by-coverage" + "files-by-coverage", + "high-crap-score" ], "setup": ["ingest-coverage"] }, diff --git a/fixtures/golden/minimal/files-count.json b/fixtures/golden/minimal/files-count.json index 39271588..beb71b49 100644 --- a/fixtures/golden/minimal/files-count.json +++ b/fixtures/golden/minimal/files-count.json @@ -1,5 +1,5 @@ [ { - "n": 42 + "n": 43 } ] diff --git a/fixtures/golden/minimal/files-hashes.json b/fixtures/golden/minimal/files-hashes.json index a6976a20..899c3f14 100644 --- a/fixtures/golden/minimal/files-hashes.json +++ b/fixtures/golden/minimal/files-hashes.json @@ -13,9 +13,9 @@ }, { "path": "README.md", - "content_hash": "4f6f0b898fbc9cecd08c66bf33b54cbc3a8d176da86fdeda8860a16e1c1cd3f6", + "content_hash": "2ddba8d7d07352340f7d76a8bbe542335cc84ef3a32681afb01ad3734929391c", "language": "md", - "line_count": 58 + "line_count": 59 }, { "path": "package.json", @@ -149,6 +149,12 @@ "language": "ts", "line_count": 4 }, + { + "path": "src/lib/alias-blind-spot.ts", + "content_hash": "34f2847180126455a500387ae3b2f49f67cc4e13290cc677c3e148827a6564ff", + "language": "ts", + "line_count": 6 + }, { "path": "src/lib/cache.ts", "content_hash": "964f6d3ddb8a7782b1273a38ed0bbde8cd886eb672c1f20466f0aa88ff49205f", diff --git a/fixtures/golden/minimal/files-largest.json b/fixtures/golden/minimal/files-largest.json index 8587c12a..a1ef33ed 100644 --- a/fixtures/golden/minimal/files-largest.json +++ b/fixtures/golden/minimal/files-largest.json @@ -7,8 +7,8 @@ }, { "path": "README.md", - "line_count": 58, - "size": 9521, + "line_count": 59, + "size": 9819, "language": "md" }, { diff --git a/fixtures/golden/minimal/high-crap-score.json b/fixtures/golden/minimal/high-crap-score.json new file mode 100644 index 00000000..4962adb1 --- /dev/null +++ b/fixtures/golden/minimal/high-crap-score.json @@ -0,0 +1,13 @@ +[ + { + "name": "labyrinth", + "kind": "function", + "file_path": "src/lib/complexity-fixture.ts", + "line_start": 22, + "line_end": 83, + "complexity": 19, + "effective_coverage_pct": 85, + "coverage_source": "estimated", + "crap_score": 20.22 + } +] diff --git a/fixtures/golden/minimal/index-summary.json b/fixtures/golden/minimal/index-summary.json index 6b95d210..6f0cb243 100644 --- a/fixtures/golden/minimal/index-summary.json +++ b/fixtures/golden/minimal/index-summary.json @@ -1,8 +1,8 @@ [ { - "files": 42, + "files": 43, "symbols": 106, - "imports": 25, + "imports": 26, "components": 5, "dependencies": 23 } diff --git a/fixtures/golden/minimal/index-table-stats.json b/fixtures/golden/minimal/index-table-stats.json index 46cb62ab..47f6445c 100644 --- a/fixtures/golden/minimal/index-table-stats.json +++ b/fixtures/golden/minimal/index-table-stats.json @@ -1,8 +1,8 @@ [ { - "files": 42, + "files": 43, "symbols": 106, - "imports": 25, + "imports": 26, "exports": 62, "components": 5, "dependencies": 23, @@ -13,17 +13,17 @@ "css_vars": 2, "css_classes": 2, "css_keyframes": 1, - "scopes": 107, - "ref_count": 317, - "bindings": 274, - "import_specifiers": 30, + "scopes": 108, + "ref_count": 319, + "bindings": 276, + "import_specifiers": 31, "function_params": 18, "runtime_markers": 6, "test_suites": 7, "re_export_chains": 4, "module_cycles": 2, "dynamic_imports": 1, - "file_metrics": 34, + "file_metrics": 35, "unresolved_calls": 1, "jsx_elements": 10, "async_calls": 1, diff --git a/fixtures/golden/minimal/source-fts-row-count.json b/fixtures/golden/minimal/source-fts-row-count.json index 39271588..beb71b49 100644 --- a/fixtures/golden/minimal/source-fts-row-count.json +++ b/fixtures/golden/minimal/source-fts-row-count.json @@ -1,5 +1,5 @@ [ { - "n": 42 + "n": 43 } ] diff --git a/fixtures/golden/minimal/unimported-exports.json b/fixtures/golden/minimal/unimported-exports.json index e8b86c33..537358f5 100644 --- a/fixtures/golden/minimal/unimported-exports.json +++ b/fixtures/golden/minimal/unimported-exports.json @@ -212,8 +212,8 @@ "file_path": "src/orphan.ts", "is_default": 0, "re_export_source": null, - "reason": "no_direct_import", - "evidence_json": "[]" + "reason": "unresolved_import_blind_spot", + "evidence_json": "[{\"kind\":\"unresolved_import\",\"file_path\":\"src/lib/alias-blind-spot.ts\",\"source\":\"@codemap-fixture/unresolved-orphan\",\"specifier\":\"orphanHelper\"}]" }, { "name": "ImportedMammal", diff --git a/fixtures/golden/scenarios.json b/fixtures/golden/scenarios.json index 2b41c951..688264e2 100644 --- a/fixtures/golden/scenarios.json +++ b/fixtures/golden/scenarios.json @@ -593,6 +593,12 @@ "prompt": "High cyclomatic complexity + low coverage", "recipe": "high-complexity-untested" }, + { + "id": "high-crap-score", + "prompt": "High CRAP score with graph-estimated coverage tiers (min_crap=15)", + "recipe": "high-crap-score", + "params": { "min_crap": 15 } + }, { "id": "text-in-deprecated-functions", "prompt": "FTS TODO/FIXME/HACK in @deprecated functions with low coverage (requires fts5)", diff --git a/fixtures/minimal/README.md b/fixtures/minimal/README.md index 335cad89..5681d3ce 100644 --- a/fixtures/minimal/README.md +++ b/fixtures/minimal/README.md @@ -28,6 +28,7 @@ Stable tree exercising every codemap surface — used by `test:golden`, `test:ag | **`fts5`** | `fts5: true` in config — powers `text-in-deprecated-functions` | | **`coverage`** | Istanbul + LCOV under `coverage/` — killer recipes + refactor-risk | | **`orphan` exports** | `orphan.ts` — `unimported-exports` | +| **Unresolved import blind spot** | `lib/alias-blind-spot.ts` — `@codemap-fixture/unresolved-orphan` → `unimported-exports` `unresolved_import_blind_spot` on `orphanHelper` | | **Project-local recipes** | `.codemap/recipes/shop-symbols.{sql,md}` | | **`CODEOWNERS`** | `--group-by owner` | diff --git a/fixtures/minimal/src/lib/alias-blind-spot.ts b/fixtures/minimal/src/lib/alias-blind-spot.ts new file mode 100644 index 00000000..5621d6c9 --- /dev/null +++ b/fixtures/minimal/src/lib/alias-blind-spot.ts @@ -0,0 +1,5 @@ +// Unresolved path alias — resolver leaves `imports.resolved_path IS NULL`. +// Exercises `unimported-exports` `reason=unresolved_import_blind_spot` on `orphanHelper`. +import { orphanHelper } from "@codemap-fixture/unresolved-orphan"; + +void orphanHelper; diff --git a/scripts/high-crap-score-measured.test.mjs b/scripts/high-crap-score-measured.test.mjs new file mode 100644 index 00000000..3d4f3987 --- /dev/null +++ b/scripts/high-crap-score-measured.test.mjs @@ -0,0 +1,104 @@ +import { describe, expect, it } from "bun:test"; +import { unlinkSync, writeFileSync } from "node:fs"; +import { join } from "node:path"; + +/** + * Plan 2 slice 2.2 — measured coverage overrides graph tiers when ingest ran. + * Run via `bun run test:scripts` (golden runner already ingests coverage in setup). + */ +import { $ } from "bun"; + +const REPO_ROOT = join(import.meta.dir, ".."); + +describe("high-crap-score measured override", () => { + it("uses coverage_source measured when coverage row exists (now @ 100%)", async () => { + await $`bun src/index.ts ingest-coverage coverage/coverage-final.json --root fixtures/minimal` + .cwd(REPO_ROOT) + .quiet(); + const result = + await $`bun src/index.ts query --recipe high-crap-score --json --params min_crap=1 --root fixtures/minimal` + .cwd(REPO_ROOT) + .quiet(); + expect(result.exitCode).toBe(0); + const rows = JSON.parse(result.stdout.toString()); + const nowRow = rows.find( + (r) => r.name === "now" && r.file_path === "src/utils/date.ts", + ); + expect(nowRow).toBeDefined(); + expect(nowRow.coverage_source).toBe("measured"); + expect(nowRow.effective_coverage_pct).toBe(100); + }); + + it("measured 0% overrides graph estimated tier (labyrinth @ 85% without ingest)", async () => { + await $`rm -f fixtures/minimal/.codemap/index.db fixtures/minimal/.codemap/index.db-shm fixtures/minimal/.codemap/index.db-wal` + .cwd(REPO_ROOT) + .quiet(); + await $`bun src/index.ts --full --root fixtures/minimal` + .cwd(REPO_ROOT) + .quiet(); + + const before = + await $`bun src/index.ts query --recipe high-crap-score --json --params min_crap=1 --root fixtures/minimal` + .cwd(REPO_ROOT) + .quiet(); + expect(before.exitCode).toBe(0); + const beforeRows = JSON.parse(before.stdout.toString()); + const labBefore = beforeRows.find( + (r) => + r.name === "labyrinth" && + r.file_path === "src/lib/complexity-fixture.ts", + ); + expect(labBefore).toBeDefined(); + expect(labBefore.coverage_source).toBe("estimated"); + expect(labBefore.effective_coverage_pct).toBe(85); + + const overlayPath = join( + REPO_ROOT, + "fixtures/minimal/coverage/coverage-labyrinth-zero.json", + ); + writeFileSync( + overlayPath, + JSON.stringify({ + "src/lib/complexity-fixture.ts": { + path: "src/lib/complexity-fixture.ts", + statementMap: { + 0: { + start: { line: 25, column: 0 }, + end: { line: 25, column: 1 }, + }, + }, + s: { 0: 0 }, + }, + }), + ); + await $`bun src/index.ts ingest-coverage coverage/coverage-labyrinth-zero.json --root fixtures/minimal` + .cwd(REPO_ROOT) + .quiet(); + + const after = + await $`bun src/index.ts query --recipe high-crap-score --json --params min_crap=1 --root fixtures/minimal` + .cwd(REPO_ROOT) + .quiet(); + expect(after.exitCode).toBe(0); + const afterRows = JSON.parse(after.stdout.toString()); + const labAfter = afterRows.find( + (r) => + r.name === "labyrinth" && + r.file_path === "src/lib/complexity-fixture.ts", + ); + expect(labAfter).toBeDefined(); + expect(labAfter.coverage_source).toBe("measured"); + expect(labAfter.effective_coverage_pct).toBe(0); + + unlinkSync(overlayPath); + await $`rm -f fixtures/minimal/.codemap/index.db fixtures/minimal/.codemap/index.db-shm fixtures/minimal/.codemap/index.db-wal` + .cwd(REPO_ROOT) + .quiet(); + await $`bun src/index.ts --full --root fixtures/minimal` + .cwd(REPO_ROOT) + .quiet(); + await $`bun src/index.ts ingest-coverage coverage/coverage-final.json --root fixtures/minimal` + .cwd(REPO_ROOT) + .quiet(); + }); +}); diff --git a/scripts/spike-crap-reachability.sql b/scripts/spike-crap-reachability.sql new file mode 100644 index 00000000..57c03ebf --- /dev/null +++ b/scripts/spike-crap-reachability.sql @@ -0,0 +1,71 @@ +-- Plan 2 slice 2.0 spike: graph-estimated coverage tiers on fixtures/minimal. +-- Run: codemap query --json "$(cat scripts/spike-crap-reachability.sql)" --root fixtures/minimal +-- Expected function/method tier counts: 85% → labyrinth (direct test ref); 40% → complexity-fixture peers (reachable); 0% → rest. +WITH RECURSIVE +test_files(path) AS ( + SELECT DISTINCT f.path + FROM files f + WHERE EXISTS ( + SELECT 1 + FROM test_suites ts + WHERE ts.file_path = f.path + ) + OR f.path GLOB '*.test.ts' + OR f.path GLOB '*.test.tsx' + OR f.path GLOB '*.spec.ts' + OR f.path GLOB '*.spec.tsx' + OR f.path GLOB '*.test.js' + OR f.path GLOB '*.spec.js' + OR f.path GLOB '*.test.jsx' + OR f.path GLOB '*.spec.jsx' +), +reachable_files(file_path, depth, visited) AS ( + SELECT path, 0, char(30) || path || char(30) + FROM test_files + UNION ALL + SELECT + d.to_path, + rf.depth + 1, + rf.visited || d.to_path || char(30) + FROM dependencies d + JOIN reachable_files rf ON d.from_path = rf.file_path + WHERE rf.depth < 50 + AND instr(rf.visited, char(30) || d.to_path || char(30)) = 0 +), +symbol_tiers AS ( + SELECT + s.name, + s.file_path, + s.complexity, + CASE + WHEN EXISTS ( + SELECT 1 + FROM "references" r + JOIN bindings b ON b.reference_id = r.id + JOIN test_files tf ON tf.path = r.file_path + WHERE b.resolved_symbol_id = s.id + ) + OR EXISTS ( + SELECT 1 + FROM calls c2 + JOIN test_files tf ON tf.path = c2.file_path + WHERE c2.callee_symbol_id = s.id + AND (c2.provenance IS NULL OR c2.provenance = 'ast') + ) + THEN 85 + WHEN EXISTS ( + SELECT 1 + FROM reachable_files rf + WHERE rf.file_path = s.file_path + ) + THEN 40 + ELSE 0 + END AS estimated_pct + FROM symbols s + WHERE s.complexity IS NOT NULL + AND s.kind IN ('function', 'method') +) +SELECT estimated_pct, COUNT(*) AS symbol_count +FROM symbol_tiers +GROUP BY estimated_pct +ORDER BY estimated_pct DESC diff --git a/scripts/spike-crap-reachability.test.mjs b/scripts/spike-crap-reachability.test.mjs new file mode 100644 index 00000000..33926cb9 --- /dev/null +++ b/scripts/spike-crap-reachability.test.mjs @@ -0,0 +1,32 @@ +import { describe, expect, it } from "bun:test"; +/** + * Plan 2 slice 2.0 — locks reachability tier counts on fixtures/minimal. + * Run via `bun run test:scripts`. + */ +import { readFileSync } from "node:fs"; +import { join } from "node:path"; + +import { $ } from "bun"; + +const REPO_ROOT = join(import.meta.dir, ".."); +const SPIKE_SQL = readFileSync( + join(REPO_ROOT, "scripts/spike-crap-reachability.sql"), + "utf-8", +); + +describe("spike-crap-reachability (fixtures/minimal)", () => { + it("assigns 85/40/0% tiers to 1/4/39 function-shaped symbols", async () => { + const result = + await $`bun src/index.ts query --json ${SPIKE_SQL} --root fixtures/minimal` + .cwd(REPO_ROOT) + .quiet(); + expect(result.exitCode).toBe(0); + const rows = JSON.parse(result.stdout.toString()); + const byTier = Object.fromEntries( + rows.map((r) => [r.estimated_pct, r.symbol_count]), + ); + expect(byTier[85]).toBe(1); + expect(byTier[40]).toBe(4); + expect(byTier[0]).toBe(39); + }); +}); diff --git a/templates/agent-content/mcp-instructions.md b/templates/agent-content/mcp-instructions.md index 97f79700..68b863d7 100644 --- a/templates/agent-content/mcp-instructions.md +++ b/templates/agent-content/mcp-instructions.md @@ -29,30 +29,32 @@ Key fields: `pending_sync` (watcher debounce queue or in-flight reindex), `commi ## Common tasks -| Goal | MCP tool | Recipe twin (`query_recipe`) | -| -------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| Exact symbol lookup | **`show`** (`name`, optional `in`) | `find-symbol-definitions` | -| Field-qualified symbol discovery | **`show`** or **`snippet`** (`query` with `kind:` / `name:` / `path:` / `in:` + free text) | `find-symbol-by-kind` for kind-heavy patterns; CLI `codemap show --query '…' --print-sql` to inspect generated SQL (no MCP `print_sql` arg) | -| Kind / pattern lookup | **`query_recipe`** | `find-symbol-by-kind` | -| Source at symbol | **`snippet`** | same rows as `show` + disk text | -| Blast radius | **`impact`** (`target`, `direction`, `via`, `depth`) | `fan-in` for file hubs; symbol call graph via SQL or `impact` | -| Call path + snippets | **`trace`** (`from`, `to`, `via?`, `max_depth?`, `budget_chars?`) — adaptive snippet caps 15k/10k/6k when omitted | `call-path` | -| Type extends / implements chain | **`query_recipe`** | `type-ancestors`, `type-descendants` (`file_path` when homonyms; on `type-descendants` also scopes output to that file) | -| Multi-symbol survey | **`explore`** (`names`, `depth?`, `kind?`, `budget_chars?`) — row cap always adaptive (500/250/125); snippets 15k/10k/6k when `budget_chars` omitted | `symbol-neighborhood` (once per name) | -| One-hop symbol card | **`node`** (`name`, `kind?`, `in?`, `include_snippets?`, `budget_chars?`) — adaptive snippet caps when snippets enabled | `show` + `symbol-neighborhood` with `depth=1` | -| Affected tests | **`affected`** (`paths?`, `changed_since?`, `test_glob?`, `max_depth?`) | `affected-tests` (RS-delimit multiple paths in `query_recipe` params) | -| CI / SARIF | **`query_recipe`** + `format: "sarif"` | `deprecated-symbols`, `boundary-violations`, … | -| GitLab Code Quality | **`query_recipe`** + `format: "codeclimate"` | `boundary-violations`, … — locatable rows only; flat `minor` severity | -| CI badge / issue count | **`query_recipe`** + `format: "badge"` (+ `badge_style: "json"` for gates) | presentation only — triage via JSON rows / `--summary` | -| Ad-hoc SQL | **`query`** | — | -| N statements / one round-trip | **`query_batch`** | **`codemap query batch`** | -| Index freshness (index-level) | **`context`** (`index_freshness`) + tool metadata above | — | -| Per-file staleness | **`validate`** | — | -| Drift vs baseline | **`audit`** (`baseline_prefix` and/or per-delta `baselines`) or **`query`** / **`query_recipe`** + `baseline` (one-shot row diff vs `query_baselines`; incompatible with non-`json` `format` / `group_by`) | save via **`save_baseline`**; `summary: true` → count-only diff | -| Load coverage data | **`ingest_coverage`** (`path`, optional `runtime` for V8 dirs; auto-detects Istanbul `.json` / LCOV `.info`) | enables `worst-covered-exports`, `files-by-coverage`, `untested-and-dead` | -| Apply recipe diff rows | **`apply`** (`recipe`, `params?`, `dry_run?`, `yes?`, `force?`, `until_empty?`, `max_passes?`, `commit_message?`) | recipe must emit `{file_path, line_start, before_pattern, after_pattern}` rows; `yes: true` required for writes; non-`auto_fixable` recipes need `force: true` | -| Apply agent/codemod rows | **`apply_rows`** (`rows`, `dry_run?`, `yes?`) | same row contract; bypasses recipe `auto_fixable` / allowlist gates | -| Apply unified diff text | **`apply_diff_input`** (`diff_text`, `dry_run?`, `yes?`, `commit_message?`) | parses git-style hunks; same executor as `apply_rows` | +| Goal | MCP tool | Recipe twin (`query_recipe`) | +| --------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------ | +| Exact symbol lookup | **`show`** (`name`, optional `in`) | `find-symbol-definitions` | +| Field-qualified symbol discovery | **`show`** or **`snippet`** (`query` with `kind:` / `name:` / `path:` / `in:` + free text) | `find-symbol-by-kind` for kind-heavy patterns; CLI `codemap show --query '…' --print-sql` to inspect generated SQL (no MCP `print_sql` arg) | +| Kind / pattern lookup | **`query_recipe`** | `find-symbol-by-kind` | +| Source at symbol | **`snippet`** | same rows as `show` + disk text | +| Blast radius | **`impact`** (`target`, `direction`, `via`, `depth`) | `fan-in` for file hubs; symbol call graph via SQL or `impact` | +| Call path + snippets | **`trace`** (`from`, `to`, `via?`, `max_depth?`, `budget_chars?`) — adaptive snippet caps 15k/10k/6k when omitted | `call-path` | +| Type extends / implements chain | **`query_recipe`** | `type-ancestors`, `type-descendants` (`file_path` when homonyms; on `type-descendants` also scopes output to that file) | +| Multi-symbol survey | **`explore`** (`names`, `depth?`, `kind?`, `budget_chars?`) — row cap always adaptive (500/250/125); snippets 15k/10k/6k when `budget_chars` omitted | `symbol-neighborhood` (once per name) | +| One-hop symbol card | **`node`** (`name`, `kind?`, `in?`, `include_snippets?`, `budget_chars?`) — adaptive snippet caps when snippets enabled | `show` + `symbol-neighborhood` with `depth=1` | +| Affected tests | **`affected`** (`paths?`, `changed_since?`, `test_glob?`, `max_depth?`) | `affected-tests` (RS-delimit multiple paths in `query_recipe` params) | +| CI / SARIF | **`query_recipe`** + `format: "sarif"` | `deprecated-symbols`, `boundary-violations`, … | +| GitLab Code Quality | **`query_recipe`** + `format: "codeclimate"` | `boundary-violations`, … — locatable rows only; flat `minor` severity | +| CI badge / issue count | **`query_recipe`** + `format: "badge"` (+ `badge_style: "json"` for gates) | presentation only — triage via JSON rows / `--summary` | +| Ad-hoc SQL | **`query`** | — | +| N statements / one round-trip | **`query_batch`** | **`codemap query batch`** | +| Index freshness (index-level) | **`context`** (`index_freshness`) + tool metadata above | — | +| Per-file staleness | **`validate`** | — | +| Drift vs baseline | **`audit`** (`baseline_prefix` and/or per-delta `baselines`) or **`query`** / **`query_recipe`** + `baseline` (one-shot row diff vs `query_baselines`; incompatible with non-`json` `format` / `group_by`) | save via **`save_baseline`**; `summary: true` → count-only diff | +| Load coverage data | **`ingest_coverage`** (`path`, optional `runtime` for V8 dirs; auto-detects Istanbul `.json` / LCOV `.info`) | enables `worst-covered-exports`, `files-by-coverage`, `untested-and-dead`; **`high-crap-score`** uses measured rows (`coverage_source: measured`) over graph tiers | +| Complex + undertested (no ingest) | **`query_recipe`** `high-crap-score` | graph-estimated 85/40/0% tiers — parse `coverage_source` before CI gates; prefer after **`ingest_coverage`** when possible | +| High-judgment recipe triage | **`query_recipe`** `unimported-exports`, `boundary-violations`, `deprecated-symbols` | rows include `reason` / `evidence_json` — cite before **`apply`** or deletion | +| Apply recipe diff rows | **`apply`** (`recipe`, `params?`, `dry_run?`, `yes?`, `force?`, `until_empty?`, `max_passes?`, `commit_message?`) | recipe must emit `{file_path, line_start, before_pattern, after_pattern}` rows; `yes: true` required for writes; non-`auto_fixable` recipes need `force: true` | +| Apply agent/codemod rows | **`apply_rows`** (`rows`, `dry_run?`, `yes?`) | same row contract; bypasses recipe `auto_fixable` / allowlist gates | +| Apply unified diff text | **`apply_diff_input`** (`diff_text`, `dry_run?`, `yes?`, `commit_message?`) | parses git-style hunks; same executor as `apply_rows` | ## Chains @@ -72,6 +74,6 @@ Key fields: `pending_sync` (watcher debounce queue or in-flight reindex), `commi ## Recipe ids cited here -`find-symbol-definitions`, `find-symbol-by-kind`, `find-symbol-references`, `fan-in`, `call-path`, `symbol-neighborhood`, `type-ancestors`, `type-descendants`, `affected-tests`, `deprecated-symbols`, `boundary-violations`, `refactor-risk-ranking`, `calls-including-heuristic` (opt-in; requires `synthesis.heuristicCalls: true` in config). Default call-graph recipes exclude `provenance = 'heuristic'`. Others: list via **`codemap://recipes`** before **`query_recipe`**. +`find-symbol-definitions`, `find-symbol-by-kind`, `find-symbol-references`, `fan-in`, `call-path`, `symbol-neighborhood`, `type-ancestors`, `type-descendants`, `affected-tests`, `deprecated-symbols`, `boundary-violations`, `unimported-exports`, `high-crap-score`, `refactor-risk-ranking`, `calls-including-heuristic` (opt-in; requires `synthesis.heuristicCalls: true` in config). Default call-graph recipes exclude `provenance = 'heuristic'`. Others: list via **`codemap://recipes`** before **`query_recipe`**. - + diff --git a/templates/agent-content/rule/00-full.md b/templates/agent-content/rule/00-full.md index 79599d80..43bc2ac6 100644 --- a/templates/agent-content/rule/00-full.md +++ b/templates/agent-content/rule/00-full.md @@ -20,7 +20,9 @@ codemap query --recipes-json # canonical list of every bundled + p **Row count:** no cap; add `LIMIT` and `ORDER BY` when you need bounded output. On failure, stdout is `{"error": "..."}` and the process exits 1. -**Evidence columns:** Some recipe rows (e.g. `boundary-violations`, `deprecated-symbols`, `unimported-exports`) add **`reason`** and **`evidence_json`** — factual detection path for agents, not pass/fail verdicts. +**Evidence columns:** Some recipe rows (e.g. `boundary-violations`, `deprecated-symbols`, `unimported-exports`) add **`reason`** and **`evidence_json`** — factual detection path for agents, not pass/fail verdicts (`unimported-exports` includes `unresolved_import_blind_spot` when unresolved imports name the export). + +**Coverage columns:** `high-crap-score` rows add **`coverage_source`** (`measured` \| `estimated`) and **`effective_coverage_pct`** — measured when `ingest-coverage` has a symbol row; else graph tiers 85/40/0% from test reachability (heuristic, not execution). ## Trigger patterns @@ -60,7 +62,8 @@ If the question matches any of these, use the index instead of grepping: | "Which exports has nobody imported?" | `--recipe unimported-exports` | | "Which components touch deprecated APIs?" | `--recipe components-touching-deprecated` | | "What's risky to refactor right now?" | `--recipe refactor-risk-ranking` | -| "What's high-complexity AND undertested?" | `--recipe high-complexity-untested` | +| "What's high-complexity AND undertested?" | `--recipe high-complexity-untested` (needs `ingest-coverage`; without ingest prefer `high-crap-score`) | +| "Complex + undertested without coverage ingest?" | `--recipe high-crap-score` (graph-estimated tiers; `coverage_source: estimated`) | | "What's cognitively complex (nesting-heavy)?" | `--recipe high-cognitive-complexity` (default `min_score=15`; `--params min_score=20` to tighten) | ## Quick reference queries diff --git a/templates/agent-content/skill/10-recipes-context.md b/templates/agent-content/skill/10-recipes-context.md index ace45f7f..eae47cc7 100644 --- a/templates/agent-content/skill/10-recipes-context.md +++ b/templates/agent-content/skill/10-recipes-context.md @@ -16,7 +16,8 @@ Replace placeholders (`'...'`) with your module path, file glob, or symbol name. - **`--save-baseline[=]`** — snapshot the result rows to the **`query_baselines`** table inside `/index.db` (default `.codemap/index.db`; no parallel JSON files; survives `--full` and SCHEMA bumps). Name defaults to the `--recipe` id; ad-hoc SQL needs an explicit `=`. Re-saving with the same name overwrites in place. - **`--baseline[=]`** — diff the current result against the saved baseline. Output `{baseline:{...}, current_row_count, added: [...], removed: [...]}` (with `--json`) or a two-section terminal dump. Identity = per-row multiset equality (canonical `JSON.stringify` keyed frequency map; duplicates preserved). Pair with `--summary` for `{baseline:{...}, current_row_count, added: N, removed: N}`. **Mutually exclusive with `--group-by`.** - **`--baselines`** lists saved baselines (no `rows_json` payload); **`--drop-baseline `** deletes one. Both reject every other flag — they're list-only / drop-only operations. -- **Evidence columns** — high-judgment recipes (`boundary-violations`, `deprecated-symbols`, `unimported-exports`, …) may add **`reason`** and **`evidence_json`** on each row — factual detection path; parse before `apply` or deletion. +- **Evidence columns** — high-judgment recipes (`boundary-violations`, `deprecated-symbols`, `unimported-exports`, …) may add **`reason`** and **`evidence_json`** on each row — factual detection path (consumer hop, re-export chain, or unresolved-import blind spot on `unimported-exports`); parse before `apply` or deletion. +- **Coverage columns** — `high-crap-score` adds **`coverage_source`** (`measured` \| `estimated`) and **`effective_coverage_pct`**; `estimated` is graph reachability, not execution — prefer `ingest-coverage` before CI gates. - **Per-row recipe `actions`** — recipes that define an **`actions: [{type, auto_fixable?, description?, command?}]`** template append it to every row in **`--json`** output (recipe-only; ad-hoc SQL never carries actions). Rendered **`command`** lines substitute `{{param}}` from bound recipe params — param **names vary by recipe** (`old`/`new` on `rename-preview`; `old_source`/`new_source` on `migrate-import-source`; `symbol`/`replacement` on `migrate-deprecated`; see each `.md` frontmatter). Under `--baseline`, actions attach to the **`added`** rows only (the rows the agent should act on). Inspect via **`--recipes-json`**. - **Boundary violations (config-driven)** — declare `boundaries: [{name, from_glob, to_glob, action?}]` in `.codemap/config.ts` and run `codemap query --recipe boundary-violations [--format sarif|codeclimate|badge]`. GitLab CI: `--format codeclimate`; README/CI summary: `--format badge`. The `action` field defaults to `"deny"` (the only shape v1 surfaces); rules are reconciled into the `boundary_rules` table on every index pass and joined against `dependencies` via SQLite `GLOB`. - **Project-local recipes** — drop **`.sql`** (and optional **`.md`** for description body, params, and actions) into **`/recipes/`** (default `.codemap/recipes/`; honors `--state-dir` / `CODEMAP_STATE_DIR`) to make team-internal SQL a first-class CLI verb. `--recipes-json` and the `codemap://recipes` MCP resource list project recipes alongside bundled ones with **`source: "bundled" | "project"`** discriminating them. Project recipes win on id collision; entries that override a bundled id carry **`shadows: true`** so agents reading the catalog at session start know when a recipe behaves differently from the documented bundled version. `.md` supports YAML frontmatter for `params:` and per-row `actions:` — **block-list shape only** (loader's hand-rolled parser; no inline-flow `[{...}]`). Param types: `string | number | boolean`; pass values with `--params key=value[,key=value]` (repeatable; last value wins). Example: `codemap query --json --recipe find-symbol-by-kind --params kind=function,name_pattern=%Query%`. Validation: SQL is rejected at load time if it starts with DML/DDL (DELETE/DROP/UPDATE/etc.); params validate before SQL binding; runtime `PRAGMA query_only=1` is the parser-proof backstop. `/index.db` is gitignored; **`/recipes/` is NOT** — recipes are git-tracked source code authored for human review. diff --git a/templates/recipes/high-complexity-untested.md b/templates/recipes/high-complexity-untested.md index 45cd2959..3e007107 100644 --- a/templates/recipes/high-complexity-untested.md +++ b/templates/recipes/high-complexity-untested.md @@ -22,6 +22,10 @@ McCabe formula: `1 + (decision points)`. Branching nodes counted by Codemap's pa Each row also includes **SonarSource cognitive complexity** for the same symbol (nesting-heavy control flow scores higher than flat branch chains). The recipe **filter** still uses cyclomatic `>= 10`; use `high-cognitive-complexity` when cognitive score alone is the gate. +## Without `ingest-coverage` + +`COALESCE(coverage_pct, 0)` treats missing coverage as **0%**, so every high-complexity symbol appears undertested. Prefer **`high-crap-score`** when coverage is not ingested — it uses graph-estimated tiers (85/40/0%) from test reachability instead of assuming zero coverage. + ## Why the joint signal - High complexity alone surfaces too many false positives — a heavily-branched config-loader or visitor pattern is fine if it's well-tested. diff --git a/templates/recipes/high-crap-score.md b/templates/recipes/high-crap-score.md new file mode 100644 index 00000000..7b37059e --- /dev/null +++ b/templates/recipes/high-crap-score.md @@ -0,0 +1,31 @@ +--- +params: + - name: min_crap + type: number + required: false + default: 30 + description: Minimum CRAP score threshold (industry default 30) +actions: + - type: review-crap-score + auto_fixable: false + description: "High CRAP (complex + undertested) — add tests or simplify before refactor. Check coverage_source: measured rows used ingested coverage; estimated rows use graph tiers only." +--- + +# high-crap-score + +Ranks symbols by **CRAP score** — `CC² × (1 - effective_coverage/100)³ + CC` where `CC = symbols.complexity`. + +**Coverage precedence:** ingested `coverage` rows win (`coverage_source: measured`) — including **0% measured**, which overrides graph tiers even when tests reference the symbol. Otherwise graph-estimated tiers (`coverage_source: estimated`) via value-only `dependencies` fan-out (type-only imports are excluded at index time): + +| Tier | When | +| ------- | --------------------------------------------------------------------------------------------- | +| **85%** | Symbol directly referenced from a test file (`bindings`-resolved `references` or AST `calls`) | +| **40%** | Symbol's `file_path` is dependency-reachable from any test file | +| **0%** | Otherwise | + +Estimates are **heuristics**, not execution coverage — prefer `codemap ingest-coverage` before CI gates. Composes with `high-complexity-untested` (cyclomatic + measured-only today). + +```bash +codemap query --recipe high-crap-score --json +codemap query --recipe high-crap-score --params min_crap=15 --json +``` diff --git a/templates/recipes/high-crap-score.sql b/templates/recipes/high-crap-score.sql new file mode 100644 index 00000000..4ec09893 --- /dev/null +++ b/templates/recipes/high-crap-score.sql @@ -0,0 +1,112 @@ +-- CRAP score (complexity × undertest risk) with measured or graph-estimated coverage. +-- Formula: CC² × (1 - effective_coverage/100)³ + CC (CC = symbols.complexity). +-- Without ingest-coverage, effective coverage uses static tiers from test reachability: +-- 85% direct reference from a test file; 40% file dependency-reachable from tests; 0% otherwise. +WITH RECURSIVE +params(min_crap) AS ( + SELECT COALESCE(?, 30) +), +test_files(path) AS ( + SELECT DISTINCT f.path + FROM files f + WHERE EXISTS ( + SELECT 1 + FROM test_suites ts + WHERE ts.file_path = f.path + ) + OR f.path GLOB '*.test.ts' + OR f.path GLOB '*.test.tsx' + OR f.path GLOB '*.spec.ts' + OR f.path GLOB '*.spec.tsx' + OR f.path GLOB '*.test.js' + OR f.path GLOB '*.spec.js' + OR f.path GLOB '*.test.jsx' + OR f.path GLOB '*.spec.jsx' +), +reachable_files(file_path, depth, visited) AS ( + SELECT path, 0, char(30) || path || char(30) + FROM test_files + UNION ALL + SELECT + d.to_path, + rf.depth + 1, + rf.visited || d.to_path || char(30) + FROM dependencies d + JOIN reachable_files rf ON d.from_path = rf.file_path + WHERE rf.depth < 50 + AND instr(rf.visited, char(30) || d.to_path || char(30)) = 0 +), +effective AS ( + SELECT + s.name, + s.kind, + s.file_path, + s.line_start, + s.line_end, + s.complexity, + ROUND( + COALESCE( + c.coverage_pct, + CASE + WHEN EXISTS ( + SELECT 1 + FROM "references" r + JOIN bindings b ON b.reference_id = r.id + JOIN test_files tf ON tf.path = r.file_path + WHERE b.resolved_symbol_id = s.id + ) + OR EXISTS ( + SELECT 1 + FROM calls c2 + JOIN test_files tf ON tf.path = c2.file_path + WHERE c2.callee_symbol_id = s.id + AND (c2.provenance IS NULL OR c2.provenance = 'ast') + ) + THEN 85.0 + WHEN EXISTS ( + SELECT 1 + FROM reachable_files rf + WHERE rf.file_path = s.file_path + ) + THEN 40.0 + ELSE 0.0 + END + ), + 1 + ) AS effective_coverage_pct, + CASE + WHEN c.coverage_pct IS NOT NULL THEN 'measured' + ELSE 'estimated' + END AS coverage_source + FROM symbols s + LEFT JOIN coverage c + ON c.file_path = s.file_path + AND c.name = s.name + AND c.line_start = s.line_start + WHERE s.complexity IS NOT NULL +), +scored AS ( + SELECT + e.*, + ROUND( + e.complexity * e.complexity * POWER(1 - e.effective_coverage_pct / 100.0, 3) + + e.complexity, + 2 + ) AS crap_score + FROM effective e +) +SELECT + s.name, + s.kind, + s.file_path, + s.line_start, + s.line_end, + s.complexity, + s.effective_coverage_pct, + s.coverage_source, + s.crap_score +FROM scored s +CROSS JOIN params p +WHERE s.crap_score >= p.min_crap +ORDER BY s.crap_score DESC, s.complexity DESC, s.file_path, s.name +LIMIT 50 diff --git a/templates/recipes/unimported-exports.md b/templates/recipes/unimported-exports.md index cecada41..73f27ce6 100644 --- a/templates/recipes/unimported-exports.md +++ b/templates/recipes/unimported-exports.md @@ -7,14 +7,14 @@ actions: Exports that have no row in `imports` referencing their file AND name. Surfaces the **direct-use-only** subset of "unused exports" — useful as a starting candidate list, but **NEVER as a "safe to delete" list** without manual verification. -Rows include **`reason`** (`no_direct_import` \| `reexport_chain_possible`) and **`evidence_json`** (barrel hops from `re_export_chains` when a re-export path may explain the false positive). +Rows include **`reason`** (`no_direct_import` \| `reexport_chain_possible` \| `unresolved_import_blind_spot`) and **`evidence_json`** (barrel hops or unresolved-import hints when a false positive may be explained). ## V1 limitations (false-positive classes) The recipe ships intentionally simple. Three known classes of false positive: 1. **Re-export chains** — the recipe still matches **direct** `imports` → `exports` only; it does not walk consumers through barrels. If `src/index.ts` re-exports `bar` from `src/bar.ts`, and consumers import `bar` from the barrel, `bar` in `src/bar.ts` can still appear as unimported. Rows with a matching `re_export_chains` hop get **`reason=reexport_chain_possible`** and barrel hops in **`evidence_json`** — triage those before deletion; they are not exclusions from the result set. -2. **Unresolved imports** — when `imports.resolved_path IS NULL` (e.g. `tsconfig.json` path aliases codemap's resolver can't resolve, or external-package imports), those rows are ignored. If the unresolved import actually targets the export, it's a false positive. Codemap's resolver covers most TS / JS shapes; this is a corner case for unusual config. +2. **Unresolved imports** — when `imports.resolved_path IS NULL` but specifiers name the export, rows get **`reason=unresolved_import_blind_spot`** and importer hops in **`evidence_json`** (`kind: unresolved_import`). Matching is by **imported name** only (homonyms elsewhere are not excluded). Namespace `*` on unresolved sources is intentionally not treated as a global blind spot. External packages and unresolvable aliases both qualify — triage before deletion; not proof of use. 3. **Default exports skipped** — `is_default = 0` filter. Default exports are commonly framework entry points (Next.js `page.tsx`, Storybook stories, `vite.config.ts`) that codemap doesn't model; flagging them produces high false-positive noise. To include them, drop the `AND e.is_default = 0` clause in a project-local override. ## What's NOT covered (orthogonal recipes) diff --git a/templates/recipes/unimported-exports.sql b/templates/recipes/unimported-exports.sql index cc2588e2..cc967e22 100644 --- a/templates/recipes/unimported-exports.sql +++ b/templates/recipes/unimported-exports.sql @@ -3,14 +3,11 @@ -- file AND the specifiers JSON contains its name (or "*" for namespace imports). -- -- V1 limitations (documented in unimported-exports.md): --- 1. Re-export chains: if A re-exports `bar` from B, and consumers import `bar` --- from A, the recipe doesn't follow the chain — false positive on B.bar. --- Rows with a matching `re_export_chains` hop get `reason=reexport_chain_possible`. --- 2. Unresolved imports (`resolved_path IS NULL`, e.g. tsconfig path aliases --- that codemap's resolver can't resolve) get IGNORED — false positives if --- they actually reference an export. --- 3. Default exports skipped (often framework entry points like Next.js --- page.tsx, Storybook stories, vite.config.ts). +-- 1. Re-export chains: rows with a matching `re_export_chains` hop get +-- `reason=reexport_chain_possible`. +-- 2. Unresolved imports: `resolved_path IS NULL` with a matching specifier get +-- `reason=unresolved_import_blind_spot` (hint only — may be external package). +-- 3. Default exports skipped (often framework entry points). WITH direct_uses AS ( SELECT DISTINCT e.id FROM exports e @@ -50,55 +47,126 @@ SELECT AND r.to_name = u.name ) THEN 'reexport_chain_possible' + WHEN EXISTS ( + SELECT 1 + FROM imports i + CROSS JOIN json_each(i.specifiers) j + WHERE i.resolved_path IS NULL + AND j.value = u.name + ) + THEN 'unresolved_import_blind_spot' ELSE 'no_direct_import' END AS reason, - COALESCE( - ( - SELECT - CASE - WHEN chain_total > 3 - THEN json_insert(chain_hops, '$[#]', json_object('truncated', json('true'))) - ELSE chain_hops - END - FROM ( + CASE + WHEN EXISTS ( + SELECT 1 + FROM re_export_chains r + WHERE r.to_file = u.file_path + AND r.to_name = u.name + ) + THEN COALESCE( + ( SELECT - ( - SELECT COUNT(*) - FROM re_export_chains r - WHERE r.to_file = u.file_path - AND r.to_name = u.name - ) AS chain_total, - COALESCE( + CASE + WHEN chain_total > 3 + THEN json_insert(chain_hops, '$[#]', json_object('truncated', json('true'))) + ELSE chain_hops + END + FROM ( + SELECT ( - SELECT json_group_array( - json_object( - 'kind', - 'reexport', - 'from_file', - from_file, - 'to_file', - to_file, - 'hops', - hops, - 'truncated', - truncated + SELECT COUNT(*) + FROM re_export_chains r + WHERE r.to_file = u.file_path + AND r.to_name = u.name + ) AS chain_total, + COALESCE( + ( + SELECT json_group_array( + json_object( + 'kind', + 'reexport', + 'from_file', + from_file, + 'to_file', + to_file, + 'hops', + hops, + 'truncated', + truncated + ) ) - ) - FROM ( - SELECT r.from_file, r.to_file, r.hops, r.truncated - FROM re_export_chains r - WHERE r.to_file = u.file_path - AND r.to_name = u.name - ORDER BY r.from_file, r.hops - LIMIT 3 - ) - ), - '[]' - ) AS chain_hops - ) - ), - '[]' - ) AS evidence_json + FROM ( + SELECT r.from_file, r.to_file, r.hops, r.truncated + FROM re_export_chains r + WHERE r.to_file = u.file_path + AND r.to_name = u.name + ORDER BY r.from_file, r.hops + LIMIT 3 + ) + ), + '[]' + ) AS chain_hops + ) + ), + '[]' + ) + WHEN EXISTS ( + SELECT 1 + FROM imports i + CROSS JOIN json_each(i.specifiers) j + WHERE i.resolved_path IS NULL + AND j.value = u.name + ) + THEN COALESCE( + ( + SELECT + CASE + WHEN hop_total > 3 + THEN json_insert(unresolved_hops, '$[#]', json_object('truncated', json('true'))) + ELSE unresolved_hops + END + FROM ( + SELECT + ( + SELECT COUNT(*) + FROM imports i + CROSS JOIN json_each(i.specifiers) j + WHERE i.resolved_path IS NULL + AND j.value = u.name + ) AS hop_total, + COALESCE( + ( + SELECT json_group_array( + json_object( + 'kind', + 'unresolved_import', + 'file_path', + file_path, + 'source', + source, + 'specifier', + specifier + ) + ) + FROM ( + SELECT i.file_path, i.source, j.value AS specifier + FROM imports i + CROSS JOIN json_each(i.specifiers) j + WHERE i.resolved_path IS NULL + AND j.value = u.name + ORDER BY i.file_path, i.source + LIMIT 3 + ) + ), + '[]' + ) AS unresolved_hops + ) + ), + '[]' + ) + ELSE '[]' + END AS evidence_json FROM unimported u ORDER BY u.file_path, u.name LIMIT 50