fix(agentic-engineering): forbid transferring a gh --json field name from a REST or GraphQL surface - #199
Conversation
…vocabulary-cross-surface-190
…from a REST or GraphQL surface The surveyor's vocabulary rule forbade transferring a `--json` field name between subcommands, but 22 of 25 real `Unknown JSON field` failures measured in a consuming deployment came from a different API surface: `path` learned from the REST `actions/runs` payload the prescribed classifier consumes, then spent on `gh run list --json`, which rejects the whole read on one unknown name — an entire default-branch breakage pass returning nothing for every repository. Widen the rule to name the cross-surface case explicitly, call out `path` and `created_at` where the definition points at that payload, and pin the widened wording in the manifest validator with its own neutralising fixture (RED: nine pin rejections of the widened text under the old pin; GREEN after). The plugin moves to 4.4.24 on top of the 4.4.23 base. Fixes #190 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Readiness record — tried and evaluated as a user at
|
@coderabbitai review |
|
|
|
Warning Review limit reachedNext included review available in 59 minutes. View limit detailsLimit details: You’ve used the included review currently available. You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. Review configuration: ⚙️ Run configurationConfiguration used: Organization UI Review profile: ASSERTIVE Plan: Team Run ID: 📒 Files selected for processing (8)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
CodeRabbit refused this head on quota ( @codex review |
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
|
Codex Review: Didn't find any major issues. More of your lovely PRs please. Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
…vocabulary-cross-surface-190 # Conflicts: # plugins/agentic-engineering/resources/provider-neutral.desired-state.json
…main now carries Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The merge with @coderabbitai review |
|
Tip For best results, initiate chat on the files or code changes. 🧩 Analysis chain🏁 Script executed: #!/bin/bash
set -euo pipefail
printf '%s\n' '== HEAD =='
git rev-parse HEAD
git status --short
git show --no-patch --format=fuller HEAD
printf '%s\n' '== Changed files against merge base with main =='
base="$(git merge-base HEAD origin/main)"
printf 'merge-base: %s\n' "$base"
git diff --name-status "$base"...HEAD
git diff --stat "$base"...HEAD
printf '%s\n' '== Diff =='
git diff --find-renames --find-copies --no-ext-diff "$base"...HEAD
printf '%s\n' '== Outlines of changed source-like files =='
git diff --name-only "$base"...HEAD | while IFS= read -r file; do
case "$file" in
*.js|*.mjs|*.cjs|*.ts|*.tsx|*.json|*.yaml|*.yml)
printf '\n-- %s --\n' "$file"
ast-grep outline "$file" --items all 2>/dev/null || true
;;
esac
doneLength of output: 18680 🏁 Script executed: #!/bin/bash
set -euo pipefail
printf '%s\n' '== Validator contract and surrounding desired-state checks =='
sed -n '410,485p' scripts/validate-manifests.sh
printf '%s\n' '== Fixture setup, assertion helper, and changed tests =='
sed -n '620,710p' scripts/validate-manifests.test.sh
sed -n '915,985p' scripts/validate-manifests.test.sh
rg -n -C 3 'check_fail\(|make_desired_state\(|validate-manifests' scripts/validate-manifests.test.sh
printf '%s\n' '== Complete changed agent section =='
sed -n '1,110p' plugins/agentic-engineering/agents/portfolio-surveyor.agent.md
printf '%s\n' '== Agent definition digest verification =='
sha256sum plugins/agentic-engineering/agents/portfolio-surveyor.agent.md
python3 - <<'PY'
import json
with open("plugins/agentic-engineering/resources/provider-neutral.desired-state.json", encoding="utf-8") as f:
data = json.load(f)
print(data["agents"]["portfolio-surveyor"]["definitionSha256"])
PY
printf '%s\n' '== Version declarations and contract references =='
rg -n -C 1 '"version": "4\.4\.(24|25)"|agentic-engineering.*4\.4\.(24|25)' \
-g '*.json' -g '*.md' -g '*.sh' .
rg -n -C 2 'Every `gh --json` vocabulary is local to its subcommand|actions/runs|gh run list --json' \
-g '*.md' -g '*.sh' -g '*.json' .Length of output: 37108 🏁 Script executed: #!/bin/bash
set -euo pipefail
printf '%s\n' '== Desired-state JSON shape and SHA references =='
jq 'paths(scalars) as $p | select(($p | join(".")) | test("portfolio-surveyor|definitionSha256")) | {path: $p, value: getpath($p)}' \
plugins/agentic-engineering/resources/provider-neutral.desired-state.json
rg -n -C 4 'portfolio_surveyor_sha256|actual_portfolio_surveyor_sha256|definitionSha256|portfolio-surveyor.*digest' \
scripts/validate-manifests.sh scripts/validate-manifests.test.sh
printf '%s\n' '== Plugin version declarations =='
rg -n -C 1 '"version": "4\.4\.(24|25)"|agentic-engineering.*4\.4\.(24|25)' \
-g '*.json' -g '*.md' -g '*.sh' . || true
printf '%s\n' '== JSON vocabulary contract references =='
rg -n -C 2 'Every `gh --json` vocabulary is local to its subcommand|actions/runs|gh run list --json' \
-g '*.md' -g '*.sh' -g '*.json' . || true
printf '%s\n' '== Full validator assertion block =='
rg -n -A 90 -B 8 'portfolio_survey_json_vocabulary_contract' scripts/validate-manifests.sh
printf '%s\n' '== Test harness assertion helpers =='
sed -n '130,175p' scripts/validate-manifests.test.shLength of output: 40668
I found no issues. The surveyor rule, validator pin, negative fixture, version manifests, and desired-state SHA-256 are consistent. The fixture rejects the permitted cross-surface transfer as required. You are interacting with an AI system. |
Readiness addendum at |
CodeRabbit answered the request as a chat prompt rather than a review (no review ran at this head), so the round advances to the next lane. Codex completed clean on this change at the previous head @codex review |
|
Codex Review: Didn't find any major issues. Delightful! Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
1 participant
Why
The surveyor's rule against reusing
gh --jsonfield names only spoke of transfer between subcommands, yet the failures actually measured in a consuming deployment came overwhelmingly from a different API surface — a field name that is real in a REST payload being spent on aghsubcommand, which then rejects the entire read. One such read fanned across eight repositories turned a whole breakage sweep into "no evidence".What
The rule now names the cross-surface case explicitly and calls out the two REST fields where the confusion originates, and the manifest validator pins the widened wording with its own failing fixture so it cannot regress silently.
Fixes #190
🤖 Generated with Claude Code