Fix catalog-latest-url-bypass: require tag-pinned catalog download URLs by Shaurya2k06 · Pull Request #4194 · github/spec-kit

Shaurya2k06 · 2026-08-18T19:22:17Z

Bug fix — catalog-latest-url-bypass

Proposed fix for issue #4185, applying the remediation from the bug assessment.

Verdict: valid · Severity: high

Summary

Community catalog agent workflows treated tag-pinned download_url as advisory and accepted floating releases/latest/ aliases after an HTTP 200 check. The three workflows now MUST use a tag-pinned URL, reject releases/latest/ before any HTTP check, and require the URL tag to match the submitted version (vX.Y.Z or X.Y.Z).

Changes

File	Change	Notes
`.github/workflows/add-community-extension.md`	modified	Step 2d: MUST + `releases/latest/` reject + tag/version match
`.github/workflows/add-community-preset.md`	modified	Step 2e: same checklist as extensions
`.github/workflows/add-community-bundle.md`	modified	Step 2c: aligned wording; still release-asset URLs only
`tests/test_github_workflows.py`	added test	Static assertions on the three workflow sources

Tests Added or Updated

tests/test_github_workflows.py::test_community_submission_workflows_require_tag_pinned_download_urls — asserts each community workflow drops “should follow the pattern”, forbids releases/latest/, and requires tag/version match language

Local Verification

Commands run: .venv/bin/python -m pytest tests/test_github_workflows.py -q → 6 passed
Commands run: .venv/bin/specify --help → CLI help rendered
gh aw compile was not available locally; lock YAML left unchanged because compiled workflows runtime-import the .md prompt bodies

Deviations from Assessment

None. Scope is the three workflow files plus a static test, as preferred. No shared checklist doc, no Python URL validator, no skill/template changes.

Risks & Review Notes

Instruction-only change: future submissions only; existing catalog entries are untouched.
LLM agents can still ignore MUST language; a deterministic validator remains out of scope.

Refs #4185

Testing

Tested locally with uv run specify --help
Ran existing tests with uv sync && uv run pytest (focused: tests/test_github_workflows.py)
Tested with a sample project (if applicable)

AI Disclosure

I did not use AI assistance for this contribution
I did use AI assistance (describe below)

Drafted and implemented under human supervision by Cursor Grok 4.6. I reviewed the code, ran tests, and the commit myself. I take full responsibility of the code generated by me.

Reject floating releases/latest URLs in the community catalog agent workflows and require the URL tag to match the submitted version. Refs github#4185 Assisted-by: Cursor Grok 4.6 (supervised) Signed-off-by: shaurya2k06 <shaurya2k06@gmail.com>

Copilot

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Copilot

Review details

Files reviewed: 4/4 changed files
Comments generated: 1
Review effort level: Balanced

mnriem

Please address Copilot feedback

Separate substring checks for vX.Y.Z and X.Y.Z were not independent. Refs github#4185 Assisted-by: Cursor Grok 4.6 (supervised) Signed-off-by: shaurya2k06 <shaurya2k06@gmail.com>

Copilot

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Copilot

Review details

Suppressed comments (1)

Previously missed (1) — in code that hasn't changed since the last review.

tests/test_github_workflows.py:126

These independent whole-document checks do not actually guard the required clauses. For example, changing MUST follow to may follow in all three workflows still passes because the tag-match sentence supplies another MUST; similarly, reject can be satisfied elsewhere in each document. Assert the complete normalized validation statements so weakening either high-severity requirement fails this regression test.

        assert "should follow the pattern" not in lowered
        assert "releases/latest/" in source_text
        assert "reject" in lowered
        assert "`vX.Y.Z` or `X.Y.Z`" in source_text
        assert "MUST" in source_text or "must" in lowered

Files reviewed: 4/4 changed files
Comments generated: 0 new
Review effort level: Balanced

mnriem · 2026-08-20T12:49:09Z

Thanks for tightening these catalog checks—the direction looks good. Before approving, could you please address a few remaining concerns?

Preserve existing scoped tags such as aide-v1.0.0.
Require extension and preset download URLs to belong to the submitted repository.
Strengthen the regression test to assert complete validation clauses rather than independent keywords.
Add artifact checksum validation, or track it as explicit follow-up work.

Posted on behalf of @mnriem by GitHub Copilot (model: GPT-5.6 Sol).

Shaurya2k06 · 2026-08-20T13:12:32Z

oh yeah, for sure! I'll make the changes needed asap

Keep tag-pinned URLs but accept suffixes like aide-v1.0.0, require download_url to match the submitted repository, and treat sha256 as optional follow-up rather than a hard catalog gate. Refs github#4185 Assisted-by: Cursor Grok 4.6 (supervised) Signed-off-by: shaurya2k06 <shaurya2k06@gmail.com>

Copilot

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Copilot

Review details

Files reviewed: 4/4 changed files
Comments generated: 3
Review effort level: Balanced

Skip the HTTP check for a floating releases/latest URL without aborting the rest of Step 2. Refs github#4185 Assisted-by: Cursor Grok 4.6 (supervised) Signed-off-by: shaurya2k06 <shaurya2k06@gmail.com>

Copilot

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Shaurya2k06 · 2026-08-20T13:28:28Z

valid points, updated accordingly. All three workflows now say mark the pinning check failed, skip the HTTP request for that URL, then continue the remaining validations.

Copilot

Review details

Files reviewed: 4/4 changed files
Comments generated: 0 new
Review effort level: Balanced

Shaurya2k06 · 2026-08-24T11:16:06Z

Hey @mnriem , could I get a review on this?

Copilot

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Copilot

🟡 Changes recommended

The unconditional SHA verification conflicts with the requirement not to fetch URLs that fail pinning validation.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details

Files reviewed: 4/4 changed files
Comments generated: 3
Review effort level: Balanced

mnriem

Please address Copilot feedback

Keep archive fetching and optional hash verification behind successful URL pinning checks. Refs github#4185 Assisted-by: Codex (model: GPT-5, autonomous) Signed-off-by: shaurya2k06 <shaurya2k06@gmail.com>

Copilot

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Shaurya2k06 · 2026-09-10T06:41:01Z

Updated all three catalog workflow prompts in a1a7967 so downloads—including optional SHA verification—run only after every pinning check succeeds. Tightened the shared regression assertion to keep the gate and SHA rule together.

Checks: focused workflow tests (6 passed), Ruff, and the CLI smoke test pass. The full local suite completed with 6,957 passed and 9 unrelated script-parity failures on macOS.

@mnriem, could you please re-review?

Posted on behalf of @Shaurya2k06 by Codex (model: GPT-5).

mnriem · 2026-09-10T15:05:56Z

Thanks @Shaurya2k06 — confirmed the pinning/HTTP-skip and sha256-gating fixes are applied consistently across the extension, preset, and bundle validation workflows, and all threads are resolved. This is now on me: I'll re-review to clear my earlier changes-requested and trigger CI.

Copilot

🔵 Needs a closer look

The regression test does not enforce the mandatory wording central to the security fix.

Review details

Suppressed comments (1)

Previously missed (1) — in code that hasn't changed since the last review.

tests/test_github_workflows.py:152

This regression test does not actually enforce the central MUST follow ... tag-pinned pattern requirement: it only rejects the former exact phrase, so changing the workflow to advisory wording such as “may follow the pattern” would still pass while reintroducing the weakness from #4185. Assert that each source contains the mandatory The download URL MUST follow clause as well.

Files reviewed: 4/4 changed files
Comments generated: 0 new
Review effort level: Balanced

mnriem · 2026-09-10T18:16:26Z

Thank you!

Shaurya2k06 requested a review from mnriem as a code owner August 18, 2026 19:22

Copilot AI balanced review requested due to automatic review settings August 18, 2026 19:22

Copilot AI reviewed Aug 18, 2026

mnriem requested a balanced review from Copilot August 19, 2026 12:19

Copilot started reviewing on behalf of mnriem August 19, 2026 12:19 View session

Copilot AI reviewed Aug 19, 2026

View reviewed changes

Comment thread tests/test_github_workflows.py Outdated

mnriem requested changes Aug 19, 2026

View reviewed changes

test: assert both catalog tag forms together

efed88d

Separate substring checks for vX.Y.Z and X.Y.Z were not independent. Refs github#4185 Assisted-by: Cursor Grok 4.6 (supervised) Signed-off-by: shaurya2k06 <shaurya2k06@gmail.com>

Copilot AI review requested due to automatic review settings August 19, 2026 15:53

Copilot AI reviewed Aug 19, 2026

Shaurya2k06 requested a review from mnriem August 19, 2026 15:57

mnriem requested a balanced review from Copilot August 19, 2026 22:11

Copilot started reviewing on behalf of mnriem August 19, 2026 22:12 View session

Copilot AI reviewed Aug 19, 2026

View reviewed changes

Copilot AI review requested due to automatic review settings August 20, 2026 13:15

Copilot AI reviewed Aug 20, 2026

mnriem requested a balanced review from Copilot August 20, 2026 13:21

Copilot started reviewing on behalf of mnriem August 20, 2026 13:22 View session

Copilot AI reviewed Aug 20, 2026

View reviewed changes

Comment thread .github/workflows/add-community-extension.md Outdated

Comment thread .github/workflows/add-community-preset.md Outdated

Comment thread .github/workflows/add-community-bundle.md Outdated

fix: keep collecting catalog validation failures after a latest URL

45295ea

Skip the HTTP check for a floating releases/latest URL without aborting the rest of Step 2. Refs github#4185 Assisted-by: Cursor Grok 4.6 (supervised) Signed-off-by: shaurya2k06 <shaurya2k06@gmail.com>

Copilot AI review requested due to automatic review settings August 20, 2026 13:27

Copilot AI reviewed Aug 20, 2026

mnriem requested a balanced review from Copilot August 20, 2026 13:39

Copilot started reviewing on behalf of mnriem August 20, 2026 13:39 View session

Copilot AI reviewed Aug 20, 2026

View reviewed changes

Shaurya2k06 requested a balanced review from Copilot August 29, 2026 19:38

Copilot AI reviewed Aug 29, 2026

mnriem requested a balanced review from Copilot August 31, 2026 22:48

Copilot started reviewing on behalf of mnriem August 31, 2026 22:48 View session

Copilot AI reviewed Aug 31, 2026

View reviewed changes

Comment thread .github/workflows/add-community-extension.md Outdated

Comment thread .github/workflows/add-community-preset.md Outdated

Comment thread .github/workflows/add-community-bundle.md Outdated

mnriem requested changes Sep 1, 2026

View reviewed changes

fix: gate catalog SHA checks on URL pinning

a1a7967

Keep archive fetching and optional hash verification behind successful URL pinning checks. Refs github#4185 Assisted-by: Codex (model: GPT-5, autonomous) Signed-off-by: shaurya2k06 <shaurya2k06@gmail.com>

Copilot AI review requested due to automatic review settings September 10, 2026 06:40

Copilot AI reviewed Sep 10, 2026

Shaurya2k06 requested a review from mnriem September 10, 2026 06:43

mnriem added the triage-nice-to-have Verdict: evidence-backed fix or greenlit feature — land after review label Sep 10, 2026

mnriem requested a balanced review from Copilot September 10, 2026 15:06

Copilot started reviewing on behalf of mnriem September 10, 2026 15:06 View session

Copilot AI reviewed Sep 10, 2026

View reviewed changes

mnriem approved these changes Sep 10, 2026

View reviewed changes

mnriem merged commit c173bf1 into github:main Sep 10, 2026
15 checks passed

Uh oh!

Conversation

Shaurya2k06 commented Aug 18, 2026

Bug fix — catalog-latest-url-bypass

Summary

Changes

Tests Added or Updated

Local Verification

Deviations from Assessment

Risks & Review Notes

Testing

AI Disclosure

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Review details

Uh oh!

Uh oh!

mnriem left a comment

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Review details

Uh oh!

mnriem commented Aug 20, 2026

Uh oh!

Shaurya2k06 commented Aug 20, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Review details

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Uh oh!

Shaurya2k06 commented Aug 20, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Review details

Uh oh!

Shaurya2k06 commented Aug 24, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟡 Changes recommended

Uh oh!

Uh oh!

Uh oh!

Uh oh!

mnriem left a comment

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Uh oh!

Shaurya2k06 commented Sep 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mnriem commented Sep 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Shaurya2k06 commented Sep 10, 2026 •

edited

Loading

mnriem commented Sep 10, 2026 •

edited

Loading