Skip to content

Skills infrastructure: .claude/skills/, verify:skills, skills:eval #4863

Closed

Description

@cliffhall

Part of the agentic software factory tracker #4858, Wave 2. Proposed as S2 in docs/agent-guidance-inception.md (PR #4861) §9, from #4859.

Depends on

None beyond #4859. Can run in parallel with the rest of Wave 2.

Scope

  • Port from the Inspector's v2/main, with their *.test.mjs files:

  • Add root npm scripts: verify:skills, verify:skills:cli, skills:eval, test:scripts.

  • Copy docs/skill-authoring.md (paths only).

  • Add the "Maintaining the skills" rules to AGENTS.md:

    • explicit disable-model-invocation, defaulting to false
    • eval cases with ≥ 5 positives plus negatives
    • the listing budget
    • paths only when a skill is useless outside the matched files, with the trade-off stated in the PR
    • quote any # or : in a description

    If Part 2 (AGENTS.md) hasn't merged yet, hand these rules to it instead.

Acceptance criteria

  • The ported verifier keeps its "no skills found" failure. Because this issue lands before any skill, it ships an explicit, temporary bootstrap allowance for an empty .claude/skills/, which the first skill-adding PR removes: whichever of Part 6, 7, 10, 11 or 12 lands first.
  • It fails on a fixture with malformed frontmatter, or with no disable-model-invocation.
  • npm run skills:eval runs against Claude, and against Copilot with AGENT=copilot.
  • verify:skills and verify:skills:cli run on every PR in a new root-guards CI job created here (in typescript.yml or a small repo-level workflow; this repo has no main.yml). Part 4 (TypeScript workspace gate: Prettier, ESLint, root validate, CI #4864) extends it.
  • The verifier's wiring checks (checkWiring) are ported against this repo's real files. Two links don't exist yet, so each gets an explicit, temporary allowance: root validate reaching verify:format-coverage (removed by Part 4, TypeScript workspace gate: Prettier, ESLint, root validate, CI #4864) and local:gate reaching verify:skills:cli (removed by Part 11, Add local:gate, pre-push-gate skill #4871).

Branch

Targets v2/main. Closes #N won't auto-close an issue on a non-default branch, so close this one by hand on merge and move the card to Done.

Activity

  1. added this to the v2.0.0 milestone on Sep 27, 2026
  2. self-assigned this
    on Sep 27, 2026
  3. changed the title [-]Agentic software factory Part 3: Skills infrastructure: .claude/skills/, verify:skills, skills:eval[/-] [+]Skills infrastructure: .claude/skills/, verify:skills, skills:eval[/+] on Oct 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

agent guidanceRules, skills, and quality gates for agentsenhancementNew feature or requestv2

Type

No type

Projects

No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions