Skip to content

CodeQL Extension submits empty block to multi-repository variant analysis (MRVA) #4059

Closed

Description

@mrigankpawagi

Describe the bug
When I select a query for MRVA, the extension submits an empty bundle. Every file that should have been included in the bundle, like my query, the qlpack.yml file, and the files related to all dependencies, get "ignored" because they "are outside the pack root" — as logged by the extension when it runs codeql pack bundle. As a result, and as one would expect, the MRVA jobs fail.

I have tried this with the example queries (no matter the language) in the vscode-codeql-starter repository too (after launching VSCode from their workspace) but even that gave me the same issue.

Any help will be highly appreciated.

Version
CodeQL CLI: 2.22.1
CodeQL VSCode Extension version: 1.17.3
VSCode version: 1.101.2

Activity

  1. koesie10 commented on Jun 27, 2025

    @koesie10
    Member

    @mrigankpawagi Do you have a CodeQL config file configured? If so, could you please share whether that includes any options? If not, could you please share some logs that show that the files are ignored and share some screenshots of the error that's shown in VS Code/Actions?

  2. mrigankpawagi commented on Jun 27, 2025

    @mrigankpawagi
    Author

    Thanks for your comment, @koesie10.

    I have no config file.

    For the example, I am running the example.ql query in getting-started/codeql-extra-queries-cpp from the vscode-codeql-starter repository.

    CodeQL Extension Logs (since startup): gist. The very end of the logs show all the ignored files. The second last line says Variant analysis Empty block submitted for processing.

    The results panel:

    Image

    Logs from the "Run Query Task" from the "run dmlc/xgboost" job:

    Run github/codeql-variant-analysis-action/query@main
    Setup CodeQL CLI
    Getting query pack
    /usr/bin/tar xz --warning=no-unknown-keyword --overwrite -C /home/runner/work/_temp/50d23035-f91e-423b-9046-a73124ef8998 -f query_pack.tar.gz
    {
      "productName" : "CodeQL",
      "vendor" : "GitHub",
      "version" : "2.22.0",
      "sha" : "d46e51feae8a9a26d032706673b3eea25c5bed1f",
      "branches" : [
        "codeql-cli-2.22.0"
      ],
      "copyright" : "Copyright (C) 2019-2025 GitHub, Inc.",
      "unpackedLocation" : "/opt/hostedtoolcache/CodeQL/2.22.0/x64/codeql",
      "configFileLocation" : "/home/runner/.config/codeql/config",
      "configFileFound" : false,
      "features" : {
        "analysisSummaryV2Default" : true,
        "buildModeOption" : true,
        "bundleSupportsIncludeDiagnostics" : true,
        "bundleSupportsIncludeLogs" : true,
        "databaseInterpretResultsSupportsSarifRunProperty" : true,
        "featuresInVersionResult" : true,
        "indirectTracingSupportsStaticBinaries" : false,
        "informsAboutUnsupportedPathFilters" : true,
        "supportsPython312" : true,
        "mrvaPackCreate" : true,
        "threatModelOption" : true,
        "traceCommandUseBuildMode" : true,
        "v2ramSizing" : true,
        "mrvaPackCreateMultipleQueries" : true,
        "setsCodeqlRunnerEnvVar" : true,
        "sarifMergeRunsFromEqualCategory" : true,
        "forceOverwrite" : true,
        "generateSummarySymbolMap" : true,
        "pythonDefaultIsToNotExtractStdlib" : true,
        "queryServerRunQueries" : true
      }
    }
    {
      "productName" : "CodeQL",
      "vendor" : "GitHub",
      "version" : "2.22.0",
      "sha" : "d46e51feae8a9a26d032706673b3eea25c5bed1f",
      "branches" : [
        "codeql-cli-2.22.0"
      ],
      "copyright" : "Copyright (C) 2019-2025 GitHub, Inc.",
      "unpackedLocation" : "/opt/hostedtoolcache/CodeQL/2.22.0/x64/codeql",
      "configFileLocation" : "/home/runner/.config/codeql/config",
      "configFileFound" : false,
      "features" : {
        "analysisSummaryV2Default" : true,
        "buildModeOption" : true,
        "bundleSupportsIncludeDiagnostics" : true,
        "bundleSupportsIncludeLogs" : true,
        "databaseInterpretResultsSupportsSarifRunProperty" : true,
        "featuresInVersionResult" : true,
        "indirectTracingSupportsStaticBinaries" : false,
        "informsAboutUnsupportedPathFilters" : true,
        "supportsPython312" : true,
        "mrvaPackCreate" : true,
        "threatModelOption" : true,
        "traceCommandUseBuildMode" : true,
        "v2ramSizing" : true,
        "mrvaPackCreateMultipleQueries" : true,
        "setsCodeqlRunnerEnvVar" : true,
        "sarifMergeRunsFromEqualCategory" : true,
        "forceOverwrite" : true,
        "generateSummarySymbolMap" : true,
        "pythonDefaultIsToNotExtractStdlib" : true,
        "queryServerRunQueries" : true
      }
    }
    
    /home/runner/work/_actions/github/codeql-variant-analysis-action/main/dist/query.js:80897
        throw new Error(`Path '${queryPackPath}' is missing a qlpack file.`);
              ^
    
    Error: Path '/home/runner/work/_temp/50d23035-f91e-423b-9046-a73124ef8998' is missing a qlpack file.
        at getQueryPackName (/home/runner/work/_actions/github/codeql-variant-analysis-action/main/dist/query.js:80897:11)
        at getQueryPackInfo (/home/runner/work/_actions/github/codeql-variant-analysis-action/main/dist/query.js:80860:16)
        at run (/home/runner/work/_actions/github/codeql-variant-analysis-action/main/dist/query.js:81366:31)
        at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    
    Node.js v20.19.1
    
  3. koesie10 commented on Jun 27, 2025

    @koesie10
    Member

    @mrigankpawagi Thank you, this is very useful information! Does the USERNAME in your logs show up as two different strings, or are they all the same string? It seems like Windows short paths might be enabled on your system and we aren't resolving that correctly. If they do all show up as the same string, do they contain a ~ character or are they your full username?

  4. mrigankpawagi commented on Jun 27, 2025

    @mrigankpawagi
    Author

    @koesie10 Hi, yeah it does appear in two different ways just like what you said. I redacted my username but replaced both kinds with the same word "USERNAME".

  5. koesie10 commented on Jun 27, 2025

    @koesie10
    Member

    @mrigankpawagi Would you perhaps be able to share the logs again, but replacing the variant that contains ~ with something like USER~NAME, and the other variant with USERNAME? That will allow us to see where exactly the translation is happening.

  6. mrigankpawagi commented on Jun 27, 2025

    @mrigankpawagi
    Author

    @koesie10 I have updated the logs at the same link with the usernames. My username was something like a-bcdefgh with the variant being A-BCDE~1.

  7. mrigankpawagi commented on Jun 30, 2025

    @mrigankpawagi
    Author

    Hi @koesie10, is there any update on this? Thank you so much for your time!

  8. koesie10 commented on Jul 1, 2025

    @koesie10
    Member

    @mrigankpawagi Thanks for the updated logs! Those confirm that the issue is with 8.3 short filenames. I'm trying to investigate whether there's a solution we can use for this, and it seems like we should be able to call the Windows function GetLongPathNameW from the extension to determine the full path name, which should fix the issue. I'll test whether this works and let you know.

  9. koesie10 commented on Jul 1, 2025

    @koesie10
    Member

    @mrigankpawagi I've implemented a fix for this issue in this branch and have kicked off an Actions run to produce a VSIX file. You should be able to download the vscode-codeql-extension artifact from this Actions run, which should include a .vsix file. You can install this VSIX file in VS Code. When you run a variant analysis using this version of the extension, does it correctly produce the query pack and result in a successful MRVA?

  10. mrigankpawagi commented on Jul 1, 2025

    @mrigankpawagi
    Author

    Hi @koesie10, thanks so much for the patch. Unfortunately I am unable to test it because I am not able to set the controller repository with your patched version. It says

    An error occurred while setting up the controller repository: Error getting controller repository "username/repository": Bad credentials - https://docs.github.com/rest

    I installed your version after uninstalling the previous one and deleting all its data (from both .vscode and %APPDATA%). There was no prompt by VSCode for me to authenticate myself for the controller repository.

    Thanks!

  11. koesie10 commented on Jul 1, 2025

    @koesie10
    Member

    @mrigankpawagi Are you able to sign out/sign in from the profile menu in the bottom-left corner of your VS Code window?

  12. mrigankpawagi commented on Jul 1, 2025

    @mrigankpawagi
    Author
  13. mrigankpawagi commented on Jul 2, 2025

    @mrigankpawagi
    Author

    Just to confirm with you: Is this the source of the problem?

    base: A-BCDE~1
    Expanding short path component: A-BCDE~1
    No inode info available. Skipping.
    

    Everything worked correctly on my other PC where my username is 5-characters long (and so I guess there is no need for an 8.3 name).

    Edit: I am still looking for a resolution to the problem because the other PC is my personal machine, but I wish to use MRVA on my work machine 😃!

  14. koesie10 commented on Jul 2, 2025

    @koesie10
    Member

    Just to confirm with you: Is this the source of the problem?

    base: A-BCDE~1
    Expanding short path component: A-BCDE~1
    No inode info available. Skipping.
    

    Everything worked correctly on my other PC where my username is 5-characters long (and so I guess there is no need for an 8.3 name).

    Edit: I am still looking for a resolution to the problem because the other PC is my personal machine, but I wish to use MRVA on my work machine 😃!

    @mrigankpawagi Yes, exactly. The CodeQL CLI only works if given the "long path", so the extension tries to convert the short path to the long path. This usually works by finding the long path name in the directory listing, but it seems like in your case it cannot find the path. I'm not sure why that would happen, I'm unable to reproduce it on one of my machines with a long username.

    I'm also not sure why the extension would be unable to authenticate you since there are no differences between in authentication between the released version of the extension and that VSIX file. Could you perhaps try using VS Code Insiders and seeing if it works there (just to see whether it works in a different installation of VS Code)?

  15. mrigankpawagi commented on Jul 2, 2025

    @mrigankpawagi
    Author

    Thanks @koesie10!

    I did as you said -- VSCode Insiders prompted me to login and I was actually able to run MRVA properly :-)

  16. koesie10 commented on Jul 2, 2025

    @koesie10
    Member

    @mrigankpawagi Great, then I'll create a PR to hopefully get this change into the extension. Just to make sure, did your logs show something like "Short path was not resolved to long path, using native method", followed by "Expanded short path ... to ..."?

  17. mrigankpawagi commented on Jul 2, 2025

    @mrigankpawagi
    Author

    @koesie10 yes it does:

    Short path was not resolved to long path, using native method
    Expanded short path C:\Users\A-BCDE~1\AppData\Local\Temp\queries_-30528-get429If3v26 to C:\Users\a-bcdefgh\AppData\Local\Temp\queries_-30528-get429If3v2

  18. mrigankpawagi commented on Jul 2, 2025

    @mrigankpawagi
    Author

    And of course, thanks for the rather quick response. I was very excited to run some queries with MRVA :-)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    VSCodebugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions