Explain why read_files blocks gitignored files

[nordicnode]

Summary

read_files returns a bare [BLOCKED] when a file is excluded by ignore rules, with no reason and no remediation — the model can't tell the user why, and often can't even tell the file exists (glob/code_search omit it too). This makes the ignore-rule block self-describing, following the existing FILE_TOO_LARGE precedent of sentinel + trailing explanation. Addresses #1151 (the interactive permission prompt the reporter also asks for is host-app side; see Out of scope).

Root cause

sdk/src/tools/read-files.ts returns FILE_READ_STATUS.IGNORED ('[BLOCKED]') verbatim from three distinct block paths: the .env policy, a host fileFilter, and the gitignore/.codebuffignore/built-in-defaults check. Only the third is the silent, surprising one users hit — and it gave the model nothing to work with. The sentinel is consumed via startsWith (HIDDEN_FILE_READ_STATUS / toOptionalFile in common/src/constants/paths.ts) and rendered pass-through by agent-runtime, so a trailing explanation is safe; TOO_LARGE already ships one.

Changes

1. sdk/src/tools/read-files.ts — in the isFileIgnored branch only, the result becomes [BLOCKED]: <reason> <remediation>:
   • Names the rule sources (.gitignore, .codebuffignore, built-in defaults) and states it is not an OS permission issue.
   • Claims the file exists on disk only when a fs.stat confirms it (the ignore check itself never touches the file, so an ignored-and-deleted path must not get the existence claim).
   • For the env-template sub-case, where isFileIgnored can fail closed on an unreadable ignore file without any rule matching (common/src/project-file-tree.ts, throwOnReadError: allowEnvTemplate), the wording hedges to "blocked by ignore-rule checking" instead of asserting a match.
   • Remediation says to adjust or negate the matching rule in .codebuffignore, with the explicit caveat that a file-level negation cannot re-include a path under an excluded directory (isIgnoredByIgnoreChain returns on the first ignored parent prefix, so !build/out.js is inert when build/ itself is ignored).
   • The .env-policy and custom-fileFilter blocks stay bare [BLOCKED] — those are deliberate security/host decisions that shouldn't leak reasons.
2. sdk/src/tools/read-files.ts — the explanatory suffix is suppressed for sensitive env paths even on the internal-edit path (enforceEnvPolicy: false, which skips the top env gate), so a secret blocked via built-in ignore defaults still gets a bare [BLOCKED].
3. sdk/README.md — documents that ignore-rule blocks carry a trailing reason and that .codebuffignore (.gitignore syntax, checked alongside) is the project-level escape hatch; previously undocumented repo-wide.
4. sdk/src/__tests__/read-files.test.ts — gitignore-path assertions switched from exact toBe to sentinel-prefix startsWith (env-policy/filter-block assertions remain exact, guarding the opacity); new regression test asserts the reason text, the directory-negation caveat, the omitted existence claim for an ignored-and-deleted file, and that the .env policy block stays opaque.
5. sdk/src/__tests__/run-file-filter.test.ts — one gitignore-path assertion switched to prefix match.

Out of scope: (a) an interactive permission prompt — no request-permission primitive exists in the public tree (contracts, client action, and the desktop UI are all private-side); this PR makes the tool layer say why so a host (or the model, in prose) can offer the .codebuffignore escape the reporter found manually. (b) changing glob/code_search output — separate surface, separate PR. The issue's "agent cannot detect the file exists" complaint is addressed here at the tool layer: the block message now states the file was omitted by discovery for the same reason, and that it exists on disk when a stat confirms it.

Tests

• The two touched test files (read-files.test.ts, run-file-filter.test.ts): 44 pass / 0 fail.
• Regression proof: with the source change stashed, the new test fails (ignore-rule block explains reason and unblock path... → red); with it applied, green. The new test also covers the ignored-and-deleted case (no existence claim) and the internal-edit secret gate.

Validation

On mirror 0d2d7a085 (branch rebased onto latest main; branch base f5adf610f touched no PR file — git diff f5adf610f..0d2d7a085 --name-only -- sdk/ lists only sdk/test/setup-env.ts, re-gated after rebase):

• cd sdk && CI=true bun run test — 537 pass / 0 fail (sdk is clean on main; unchanged).
• cd sdk && bun run typecheck — 0 errors (baseline 0).
• cd cli && CI=true bun run test — 2848 pass / 13 fail / 11 errors of 2871; failure set identical to the known-red baseline (3 × * release wrapper contains only product configuration and package loading + unloadable packages/internal/tar files). No new failures.
• bun x prettier --check on the touched files (incl. sdk/README.md): read-files.ts, run-file-filter.test.ts, and README.md clean; read-files.test.ts has exactly one dirty line, the pre-existing bigFile wrap at ~line 841 that this diff does not touch — no new warnings from changed lines.
• bun run build:sdk and bun run build:freebuff — both succeed.
• Consumer audit: no exact === '[BLOCKED]' comparisons exist anywhere in the public tree (grep for the literal finds only the constant definition); toOptionalFile/HIDDEN_FILE_READ_STATUS use startsWith; agent-runtime (render-read-files-result.ts, getPreviouslyReadFiles) passes content through unmodified.
• Runtime: gates re-verified under the repo-pinned Bun 1.3.14 (.bun-version) via bunx bun@1.3.14 — sdk suite, typecheck, prettier, and the cli failure set match the numbers above, which were first measured on local Bun 1.4.0.

[codebuff-team]

Good diagnosis: read_files returning a bare [BLOCKED] for gitignore/.codebuffignore exclusions genuinely is a dead end for the model, and following the FILE_TOO_LARGE sentinel+suffix precedent is the right pattern rather than inventing a new format. I like that you were careful to:

• Keep .env-policy and custom fileFilter blocks opaque (read-files.ts, the isSensitiveEnvFilePath branch) so this doesn't leak info about deliberately-hidden files.
• Only assert 'exists on disk' when fs.stat actually confirms it, rather than always claiming existence for an ignored path (covers the ignored-then-deleted case).
• Call out the real gotcha that !build/out.js can't re-include a path once build/ itself is excluded — that's a subtlety that would otherwise generate a support ticket at whoever ports this.
• Cover the enforceEnvPolicy: false internal-edit path explicitly with a test, since it bypasses the top-level env gate.

One thing I couldn't verify from the diff: isSensitiveEnvFilePath is referenced in the new branch but no import is shown being added in read-files.ts. If this helper isn't already in scope in that file (e.g. used by the existing top-level .env gate), this won't compile. Please double check that and mention where it's defined/imported in the PR description if it's pre-existing.

Minor nit: the generated message is quite long and packs several facts into one sentence (rule sources, OS-permission disclaimer, existence, glob/code_search behavior, remediation, and the negation caveat). Consider whether the model actually needs all of that every time, or whether some of it belongs in the tool's static description instead of a per-call string. Not a blocker, just something a maintainer may want to trim.

Overall: focused, tested, and addresses a real usability gap. Worth porting once the import is confirmed.