fix(java): remove duplicate codeql-java-extensions dependency causing fatal analysis crash#155
Merged
Merged
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
Fixes a fatal CodeQL Java analysis crash caused by resolving githubsecuritylab/codeql-java-extensions twice (once transitively via java/lib and once directly via published configs), by removing the redundant hard dependency from the Java libs pack and bumping its version so the fix publishes.
Changes:
- Remove
githubsecuritylab/codeql-java-extensionsfromjava/lib/qlpack.ymldependencies to avoid duplicate-pack resolution. - Bump
githubsecuritylab/codeql-java-libsversion from0.2.1to0.2.2for publishing.
Show a summary per file
| File | Description |
|---|---|
java/lib/qlpack.yml |
Removes the redundant extensions dependency and bumps the pack version to publish the fix. |
Review details
Tip
Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
- Files reviewed: 1/1 changed files
- Comments generated: 0
- Review effort level: Low
felickz
force-pushed
the
felickz-fix-java-duplicate-extensions-pack
branch
from
b009584 to
362fe7d
Compare
… fatal analysis crash Rebased onto main (0.2.2 already synced/published for real via #158) instead of the stale #126 branch. Bumps java/lib to 0.2.3 so the fix actually publishes, since 0.2.2 is already live on GHCR with this bug present. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
felickz
force-pushed
the
felickz-fix-java-duplicate-extensions-pack
branch
from
362fe7d to
db2cd3d
Compare
Contributor
Author
|
Update: Rebased and retargeted this PR to Since #158 already synced everything to Diff is now minimal and clean against library: true
name: githubsecuritylab/codeql-java-libs
-version: 0.2.2
+version: 0.2.3
dependencies:
codeql/java-all: '*'
- githubsecuritylab/codeql-java-extensions: '*'Bumping to |
👍
felickz
added a commit
that referenced
this pull request
Jul 10, 2026
.release.yml's version was bumped to 0.2.3 directly (not via a pack version bump), and java/lib was independently bumped to 0.2.3 by PR #155's hotfix. This closes the gap by bringing every other pack's own qlpack.yml version in line, so merging this triggers the real batch publish of everything to 0.2.3 (java/lib is already live and will be a no-op). Also bumps go/ext and python/ext/qlpack.yml for consistency with .release.yml's 'CodeQL Pack Versions' location pattern, even though those two packs aren't wired into publish.yml's matrix yet (tracked by open PR #144). Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
felickz
added a commit
that referenced
this pull request
Jul 10, 2026
…lease process - publish.yml's push trigger now only fires when .release.yml changes (paths filter), not on every push to main. workflow_dispatch remains as the manual escape hatch for one-off hotfix publishes. - Rewrite CONTRIBUTING.md's Releases & publishing section to match: shipping a pack change now just stages it until the next release cut or a manual dispatch; add a Cutting a release walkthrough for update-release.yml/patch-release-me now that .release.yml's CodeQL Pack Versions location (#158) actually drives every pack's version; document the -alpha.N + workflow_dispatch hotfix convention and note #155 as an accepted one-off exception; note #159 as the recovery path if .release.yml's version is ever hand-edited directly. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
felickz
added a commit
that referenced
this pull request
Jul 10, 2026
…+ docs (#160) * feat(publish): add publish-summary table + GitHub Release upsert publish.yml now records a per-pack JSON result fragment (published / up-to-date / failed) from each of the 4 jobs' matrix entries, uploads them as artifacts, and a new summary job downloads and merges them into a unified markdown table (languages x pack types) that is: 1. written to the Actions run summary ($GITHUB_STEP_SUMMARY) 2. upserted into the matching GitHub Release body (.release.yml's version -> vX.Y.Z tag), inside marker comments so "What's Changed" and other release content is preserved across runs New scripts: - .github/scripts/build-publish-summary.sh - .github/scripts/upsert-release-table.sh Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> * fix(publish): avoid tee+pipe fragility in Build summary table step The previous form piped the script's stdout through tee and redirected tee's own stdout (not the file write) into GITHUB_STEP_SUMMARY. Combined with 'bash -e' (no pipefail) at the step level, this masked failures and produced an empty summary.md in a real run, upserting empty publish-summary markers into the v0.2.2 release. Switch to plain sequential redirects and add debug output (fragment listing, line/byte counts) to make future failures visible in the step log. * feat(publish): gate auto-publish on .release.yml changes; document release process - publish.yml's push trigger now only fires when .release.yml changes (paths filter), not on every push to main. workflow_dispatch remains as the manual escape hatch for one-off hotfix publishes. - Rewrite CONTRIBUTING.md's Releases & publishing section to match: shipping a pack change now just stages it until the next release cut or a manual dispatch; add a Cutting a release walkthrough for update-release.yml/patch-release-me now that .release.yml's CodeQL Pack Versions location (#158) actually drives every pack's version; document the -alpha.N + workflow_dispatch hotfix convention and note #155 as an accepted one-off exception; note #159 as the recovery path if .release.yml's version is ever hand-edited directly. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> * fix: address Copilot review feedback on PR #160 - Anchor CURRENT_VERSION extraction to '^version:' (with head -1) in all four publish.yml jobs so a stray 'version' substring elsewhere in a qlpack.yml (e.g. a comment) can't be matched instead of the real field. - upsert-release-table.sh: only take the in-place marker-replacement path when BOTH the start and end marker are present in the existing release body; otherwise append, so a corrupted/partial marker pair can't cause the awk script to silently truncate the rest of the release notes. - CONTRIBUTING.md: grammar fix (find/replaces -> finds and replaces). Verified gh release create --notes-file + --generate-notes together is NOT mutually exclusive in gh 2.96.0 (contrary to a review comment) -- confirmed empirically: the v0.2.3 release this script created has both the publish-summary table (from --notes-file) and the auto-generated 'What's Changed' changelog (from --generate-notes) in its body. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> * feat(publish): add CodeQL standard library versions cross-check table Every publish.yml run now builds and upserts a second table into the matching GitHub Release: for each language, the locked codeql/<lang>-all version (from lib/codeql-pack.lock.yml) versus the version github/codeql itself bundles with the pinned CodeQL CLI (.codeqlversion), read from its own <lang>/ql/lib/qlpack.yml at tag codeql-cli/v<version>. Mismatches are flagged with an actions warning annotation and a table marker, giving a machine-checkable tripwire for the drift risk documented in CONTRIBUTING.md (CI's 'codeql pack install' step is non-resolving, so a forgotten 'codeql pack upgrade <dir>' after a .codeqlversion bump silently pins a language to a stale library version forever). - Add .github/scripts/build-codeql-lib-versions-table.sh - Parameterize upsert-release-table.sh with an optional block-name arg so multiple independent marker-comment blocks can coexist in one release body without clobbering each other - Wire both into publish.yml's summary job - Document the new table and cross-check in CONTRIBUTING.md Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> * fix: mark build-codeql-lib-versions-table.sh executable git-created file lost the exec bit, causing publish.yml's new step to fail with 'Permission denied' (exit 126). Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> * fix(build-codeql-lib-versions-table): check src lock file, not lib CONTRIBUTING.md's 'Supported CodeQL versions' table already documents <language>/src/codeql-pack.lock.yml as the source-of-truth lock file (the queries pack CI actually compiles/tests against), not lib. Reading from lib/codeql-pack.lock.yml instead meant the tripwire could miss drift in the src lock, or flag drift that didn't actually affect the compiled queries. Switch the check (and its remediation guidance) to src, per Copilot review feedback on PR #160. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> * feat(publish): commit/tag-anchored links + queries pack in lib-versions table - Discover direct codeql/* dependencies dynamically from each language's src/qlpack.yml instead of hardcoding codeql/<lang>-all, so this also covers codeql/<lang>-queries for C++/C# (which additionally depend on the upstream standard queries pack). - Link every "our locked" version to the exact commit/file/line in this repo (blob/<GITHUB_SHA>/...#L<n>) and every "upstream" version to the exact file/line in github/codeql at the matching codeql-cli/vX tag, so each row can be verified with one click instead of trusting the table. - Rename the section/table to "CodeQL standard library & query pack versions" and update CONTRIBUTING.md references accordingly. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> * feat: automate CodeQL CLI version bump detection and dependency refresh Adds the two workflows needed to close the gap identified while reviewing PR #118 (codeql pack upgrade weekly cron): - detect-codeql-release.yml: weekly (+ workflow_dispatch) tripwire that compares .codeqlversion against github/codeql-cli-binaries' latest release and keeps a single tracking issue open/updated while behind, auto-closing it once caught up. Never opens a PR itself. - update-codeql-version.yml: workflow_dispatch with a codeql_version input. Bumps .codeqlversion, runs codeql pack upgrade <dir> for every pack, and opens a PR via the same GitHub App token pattern as update-release.yml so CI actually runs on it (a GITHUB_TOKEN-authored PR would not trigger downstream workflows). Together these cover detection + mechanical dependency refresh; fixing compile/test breakage from upstream API changes and bumping each pack's own version: still needs a human (or delegated Copilot coding agent) - documented as a 3-step process in CONTRIBUTING.md. Rewrites CONTRIBUTING.md's "Updating the pinned CodeQL CLI/library version" section to describe this new process and adds reference-style links for both new workflows and codeql-cli-binaries. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> * fix(publish): gate release-table upsert to push trigger, clarify failed-check cell Two fixes from Copilot review on PR #160: - publish.yml: only run the "Upsert release table" step on the push (release-cut) trigger, not workflow_dispatch. A manual one-off hotfix publish (see CONTRIBUTING.md) still points .release.yml at the last cut release, so upserting that run's tables into it would overwrite an already-published release's notes with a misleading summary of an unrelated hotfix run. - build-publish-summary.sh: when a pack's version-check step itself fails, previous_version can be empty/null, which rendered as the confusing "publish failed (still ``)". Now shows an explicit "previous version unknown" message in that case. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> * fix(publish): check both src and lib lock files for CodeQL version drift src/qlpack.yml and lib/qlpack.yml each declare their own codeql/<lang>-all dependency and resolve it into their own independent codeql-pack.lock.yml. Nothing keeps these two lock files in sync automatically - codeql pack upgrade resolves each pack directory separately, so running it against one and not the other silently lets them drift. The standard-library-versions table previously only checked src as a 'coarse signal', which meant a stale lib lock could go undetected while the table showed all green. Now checks both src and lib for every language (ext/ext-library-sources are still out of scope: they pin via extensionTargets, not dependencies, so CodeQL never resolves a locked version for them). Adds a Pack column to the table to distinguish rows. Also updates CONTRIBUTING.md's Supported CodeQL versions section, which similarly implied a single lock file was the source of truth per language. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> * docs: fix publish.yml job count in CONTRIBUTING.md (five jobs, not four) CONTRIBUTING.md said publish.yml is organized as four jobs, but it also has a fifth 'summary' job (needs: [queries, library, extensions, library_sources_extensions], if: always()) that aggregates results into the publish-summary and CodeQL lib-versions tables and upserts them into the Release notes. Clarify the doc to describe all five. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> * fix(publish): don't fail check_version when a pack's GHCR container doesn't exist yet All four publish jobs (queries/library/extensions/library_sources_extensions) read the currently-published version via 'gh api .../versions'. run: steps default to bash -e, so a 404 (package never published, e.g. a brand-new language/pack) aborted the step and failed the job before it could ever publish that pack for the first time. Move '|| true' inside the command substitution so a fetch failure just leaves PUBLISHED_VERSION empty (correctly differing from CURRENT_VERSION, triggering a first publish) instead of killing the step under set -e. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Contributor
Author
|
Verified fixed. Re-ran the exact repro scenario against dsp-testing/SecurityShepherd: https://github.com/dsp-testing/SecurityShepherd/actions/runs/29121889021/job/86458636453 The java-kotlin Analyze job completed successfully in ~5 minutes with no fatal error. Log shows a clean sequence with no duplicate pack conflict: \ No \CatastrophicError, no \Encountered two packs with the same name. This previously crashed at exactly this point (see the original repro run https://github.com/dsp-testing/SecurityShepherd/actions/runs/28809261039/job/85432641504). Confirmed the fix resolves the issue in a real-world CI run using the published config. 🎉 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Bug
Any Java scan using one of this repo's published configs (
configs/default.yml,configs/synthetics.yml,configs/audit.yml) fails with a fatal CodeQL error:Confirmed against a real, 100%-reproducible failing GitHub Actions run using
configs/synthetics.ymlverbatim from this repo: dsp-testing/SecurityShepherd run 28809261039, job 85432641504. The crash happens during the java-kotlinanalyzestep, right after "Resolving data extensions".Root cause
java/lib/qlpack.ymlhard-pinnedgithubsecuritylab/codeql-java-extensions: '0.2.1'as a dependency. All three published configs also listgithubsecuritylab/codeql-java-extensionsdirectly in theirpacks:list (to load its MaD data extension models). This means the same pack gets resolved twice in a single evaluation:codeql-java-queries→codeql-java-libs→codeql-java-extensionspacks:listThe CodeQL CLI can't reconcile the two resolved instances of the same named pack and fatally errors.
Why only Java is affected:
csharp/lib/qlpack.ymlonly depends oncodeql/csharp-all—codeql-csharp-extensionsis not baked intocsharp-libs, it's purely opt-in via config, same as every other language. Java is the only lib pack that hard-pinned its extensions pack as a dependency (added in #8f4ee8a, "feat: add utility predicates and classes for Java library").Nothing in
java/libactually needscodeql-java-extensionsat QL-compile time:java/lib/ghsl/Utils.qllonly imports standardsemmle.code.java.*library modules. The extensions pack is a pure MaD data pack (seejava/ext/qlpack.yml) with no QL code to import — the dependency existed only to bundle/pull in the pack, which is redundant since every shipped config already lists it explicitly.Fix
githubsecuritylab/codeql-java-extensions: '0.2.1'line fromjava/lib/qlpack.yml'sdependencies:, bringing Java in line with C# (and every other language) where the extensions pack is purely opt-in via config.java/libversion to0.2.3(see stacking note above re: why not0.2.2) so the fix gets published (.github/workflows/publish.ymlcompares each pack'sqlpack.ymlversion against what's already published on GHCR, independently per pack).No other
qlpack.ymlunderjava/needs changes:java/src/qlpack.ymldepends ongithubsecuritylab/codeql-java-libs: '*', unaffected.java/test/qlpack.ymlintentionally keeps its own directgithubsecuritylab/codeql-java-extensions: '*'dependency (for testing the extension models directly) — untouched.java/ext/qlpack.yml(the extensions pack itself) is unchanged.Lock files (
codeql-pack.lock.yml) don't need updating — they only record resolved published-registry packs (e.g.codeql/java-all); local in-workspace packs likecodeql-java-libs/codeql-java-extensionsare resolved viacodeql-workspace.ymland never appeared in the lock files, so removing this dependency doesn't change lock content.Verification
Local sandbox only had CodeQL CLI 2.25.6 available, not the repo's pinned 2.21.1 (
.codeqlversion), so local results can't perfectly reproduce or validate against the exact pinned toolchain:java/lib/ghsl.qll) withcodeql query compile— succeeds cleanly after the fix, no dependency/syntax errors.codeql test run) — passing.--additional-packs); this didn't reproduce because local resolution dedupes identical filesystem paths, whereas the production failure occurs when the registry resolves the same named pack into two distinct locations (once as a transitive dependency, once as a directly-requested config pack).This repo's CI (
.github/workflows/ci.yml) installs the actual pinned 2.21.1 CLI and runs the real Java analyze/compile/test jobs — that run is the authoritative validation of this fix, since it uses the correct pinned toolchain rather than the newer, mismatched local CLI.