Skip to content

[CI failure] CI scheduled runs #2745

Closed
Closed
[CI failure] CI scheduled runs#2745
Assignees
rwgkjpascucci-nv
Labels
P0High priority - Must do!ci-workflow-healthAssigned to rolling issues managed by the ci-workflow-health.yml workflow.
Milestone
cuda.core 1.3.0

Description

@github-actions
github-actions
bot
opened on Sep 1, 2026

This issue tracks an unhealthy unattended CI workflow.
Subsequent failures are recorded in comments; a successful run in the
same trigger scope closes the issue automatically.

First unhealthy run

CI run #5545, attempt 1 concluded with cancelled.

  • Trigger: schedule
  • Branch: main
  • Commit: 0244727
  • Completed: 2026-09-01T01:28:45Z

Created automatically by the CI workflow-health monitor.

Activity

  1. github-actions
    added
    bugSomething isn't working
    triageNeeds the team's attention
    CI/CDCI/CD infrastructure
    ci-workflow-healthAssigned to rolling issues managed by the ci-workflow-health.yml workflow.
    on Sep 1, 2026

  2. github-actions commented on Sep 2, 2026

    @github-actions
    github-actionsBot
    on Sep 2, 2026 – with GitHub Actions
    ContributorAuthor

    Another unhealthy run

    CI run #5569, attempt 1 concluded with failure.

    • Trigger: schedule
    • Branch: main
    • Commit: b81f9d6
    • Completed: 2026-09-02T01:52:57Z

  3. jpascucci-nv commented on Sep 2, 2026

    @jpascucci-nv
    jpascucci-nv
    on Sep 2, 2026
    Contributor

    There seems to be two problems here. One is #2725, which looks like it's in progress. The other appears to be a transient:
    unknown: failed to copy: httpReadSeeker: failed open: unexpected status from GET request to https://quay.io/v2/pypa/manylinux_2_28_aarch64/blobs/sha256:f417c91485c002aaf565f940ec46da5f0a23b04219433e3cdec0382a34456d2e: 502 Bad Gateway - I was just able to get that URL from my desktop.

  4. jpascucci-nv
    added
    duplicateThis issue or pull request already exists
    on Sep 2, 2026

  5. github-actions commented on Sep 3, 2026

    @github-actions
    github-actionsBot
    on Sep 3, 2026 – with GitHub Actions
    ContributorAuthor

    Another unhealthy run

    CI run #5586, attempt 1 concluded with cancelled.

    • Trigger: schedule
    • Branch: main
    • Commit: 53b4374
    • Completed: 2026-09-03T02:16:51Z

  6. jpascucci-nv commented on Sep 3, 2026

    @jpascucci-nv
    jpascucci-nv
    on Sep 3, 2026
    Contributor

    This appears to be failing in a different way, I'm going to remove the duplicate tag.

    Windows fatal exception: access violation

<Cannot show all threads while the GIL is disabled>
Stack (most recent call first):
  File "C:\actions-runner\_work\cuda-python\cuda-python\cuda_core\tests\test_module.py", line 289 in test_get_kernel
  File "C:\actions-runner\_work\cuda-python\cuda-python\cuda_core\tests\conftest.py", line 102 in wrapper
  File "C:\actions-runner\_work\_tool\Python\3.14.7\x64-freethreaded\Lib\site-packages\pytest_run_parallel\plugin.py", line 83 in closure
  File "C:\actions-runner\_work\_tool\Python\3.14.7\x64-freethreaded\Lib\threading.py", line 1024 in run
  File "C:\actions-runner\_work\_tool\Python\3.14.7\x64-freethreaded\Lib\threading.py", line 1082 in _bootstrap_inner
  File "C:\actions-runner\_work\_tool\Python\3.14.7\x64-freethreaded\Lib\threading.py", line 1044 in _bootstrap

Current thread's C stack trace (most recent call first):
  <cannot get C stack on this system>
/c/actions-runner/_work/cuda-python/cuda-python/ci/tools/run-tests: line 197:   825 Segmentation fault         ${SANITIZER_CMD} pytest -rxXs -v --randomly-dont-reorganize "${PYTEST_PARALLEL_ARGS[@]}" tests/
tests/test_module.py::test_get_kernel[1-5]
  7. jpascucci-nv
    removed
    duplicateThis issue or pull request already exists
    on Sep 3, 2026

  8. github-actions commented on Sep 4, 2026

    @github-actions
    github-actionsBot
    on Sep 4, 2026 – with GitHub Actions
    ContributorAuthor

    Another unhealthy run

    CI run #5599, attempt 1 concluded with cancelled.

    • Trigger: schedule
    • Branch: main
    • Commit: fb57dbf
    • Completed: 2026-09-04T01:28:56Z

  9. github-actions commented on Sep 5, 2026

    @github-actions
    github-actionsBot
    on Sep 5, 2026 – with GitHub Actions
    ContributorAuthor

    Another unhealthy run

    CI run #5621, attempt 1 concluded with cancelled.

    • Trigger: schedule
    • Branch: main
    • Commit: f84a7cf
    • Completed: 2026-09-05T02:02:43Z

  10. github-actions commented on Sep 6, 2026

    @github-actions
    github-actionsBot
    on Sep 6, 2026 – with GitHub Actions
    ContributorAuthor

    Another unhealthy run

    CI run #5625, attempt 1 concluded with cancelled.

    • Trigger: schedule
    • Branch: main
    • Commit: f84a7cf
    • Completed: 2026-09-06T01:18:30Z

  11. github-actions commented on Sep 7, 2026

    @github-actions
    github-actionsBot
    on Sep 7, 2026 – with GitHub Actions
    ContributorAuthor

    Another unhealthy run

    CI run #5627, attempt 1 concluded with cancelled.

    • Trigger: schedule
    • Branch: main
    • Commit: f84a7cf
    • Completed: 2026-09-07T01:17:32Z

  12. github-actions commented on Sep 8, 2026

    @github-actions
    github-actionsBot
    on Sep 8, 2026 – with GitHub Actions
    ContributorAuthor

    Another unhealthy run

    CI run #5629, attempt 1 concluded with cancelled.

    • Trigger: schedule
    • Branch: main
    • Commit: f84a7cf
    • Completed: 2026-09-08T03:32:16Z

  13. 44 remaining items

  14. github-actions commented on Oct 8, 2026

    @github-actions
    github-actionsBot
    on Oct 8, 2026 – with GitHub Actions
    ContributorAuthor

    Another unhealthy run

    CI run #6342, attempt 1 concluded with failure.

    • Trigger: schedule
    • Branch: main
    • Commit: 93eb6d5
    • Completed: 2026-10-08T02:42:49Z

  15. github-actions commented on Oct 9, 2026

    @github-actions
    github-actionsBot
    on Oct 9, 2026 – with GitHub Actions
    ContributorAuthor

    Another unhealthy run

    CI run #6359, attempt 1 concluded with cancelled.

    • Trigger: schedule
    • Branch: main
    • Commit: a2b1e31
    • Completed: 2026-10-09T01:18:54Z

  16. github-actions commented on Oct 10, 2026

    @github-actions
    github-actionsBot
    on Oct 10, 2026 – with GitHub Actions
    ContributorAuthor

    Another unhealthy run

    CI run #6381, attempt 1 concluded with cancelled.

    • Trigger: schedule
    • Branch: main
    • Commit: c7e6f09
    • Completed: 2026-10-10T01:16:47Z

  17. github-actions commented on Oct 11, 2026

    @github-actions
    github-actionsBot
    on Oct 11, 2026 – with GitHub Actions
    ContributorAuthor

    Another unhealthy run

    CI run #6383, attempt 1 concluded with cancelled.

    • Trigger: schedule
    • Branch: main
    • Commit: c7e6f09
    • Completed: 2026-10-11T01:14:52Z

  18. github-actions commented on Oct 12, 2026

    @github-actions
    github-actionsBot
    on Oct 12, 2026 – with GitHub Actions
    ContributorAuthor

    Another unhealthy run

    CI run #6384, attempt 1 concluded with cancelled.

    • Trigger: schedule
    • Branch: main
    • Commit: c7e6f09
    • Completed: 2026-10-12T01:22:23Z

  19. github-actions commented on Oct 13, 2026

    @github-actions
    github-actionsBot
    on Oct 13, 2026 – with GitHub Actions
    ContributorAuthor

    Another unhealthy run

    CI run #6406, attempt 1 concluded with cancelled.

    • Trigger: schedule
    • Branch: main
    • Commit: 5fe0749
    • Completed: 2026-10-13T01:17:41Z

  20. github-actions commented on Oct 14, 2026

    @github-actions
    github-actionsBot
    on Oct 14, 2026 – with GitHub Actions
    ContributorAuthor

    Another unhealthy run

    CI run #6442, attempt 1 concluded with failure.

    • Trigger: schedule
    • Branch: main
    • Commit: 9dc5f33
    • Completed: 2026-10-14T01:38:10Z

  21. github-actions commented on Oct 15, 2026

    @github-actions
    github-actionsBot
    on Oct 15, 2026 – with GitHub Actions
    ContributorAuthor

    Another unhealthy run

    CI run #6478, attempt 1 concluded with cancelled.

    • Trigger: schedule
    • Branch: main
    • Commit: 1f41281
    • Completed: 2026-10-15T01:16:14Z

  22. github-actions commented on Oct 16, 2026

    @github-actions
    github-actionsBot
    on Oct 16, 2026 – with GitHub Actions
    ContributorAuthor

    Another unhealthy run

    CI run #6511, attempt 1 concluded with failure.

    • Trigger: schedule
    • Branch: main
    • Commit: 2224499
    • Completed: 2026-10-16T01:00:29Z

  23. github-actions commented on Oct 17, 2026

    @github-actions
    github-actionsBot
    on Oct 17, 2026 – with GitHub Actions
    ContributorAuthor

    Another unhealthy run

    CI run #6547, attempt 1 concluded with failure.

    • Trigger: schedule
    • Branch: main
    • Commit: 2b394fb
    • Completed: 2026-10-17T01:02:19Z

  24. github-actions commented on Oct 18, 2026

    @github-actions
    github-actionsBot
    on Oct 18, 2026 – with GitHub Actions
    ContributorAuthor

    Another unhealthy run

    CI run #6556, attempt 1 concluded with cancelled.

    • Trigger: schedule
    • Branch: main
    • Commit: 576ef68
    • Completed: 2026-10-18T01:37:20Z

  25. github-actions commented on Oct 19, 2026

    @github-actions
    github-actionsBot
    on Oct 19, 2026 – with GitHub Actions
    ContributorAuthor

    Another unhealthy run

    CI run #6561, attempt 1 concluded with failure.

    • Trigger: schedule
    • Branch: main
    • Commit: 576ef68
    • Completed: 2026-10-19T02:56:38Z
  26. jpascucci-nv
    added a commit that references this issue on Oct 19, 2026

    Remove flaky test_concurrent_eviction_does_not_delete_replaced_file (#…

    9d903cd

  27. github-actions commented on Oct 20, 2026

    @github-actions
    github-actionsBot
    on Oct 20, 2026 – with GitHub Actions
    ContributorAuthor

    Recovered

    CI run #6581, attempt 1 concluded with success.

    • Trigger: schedule
    • Branch: main
    • Commit: b1e2788
    • Completed: 2026-10-20T01:21:55Z

    Closing this incident automatically.

  28. github-actions
    closed this as completedon Oct 20, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

P0High priority - Must do!ci-workflow-healthAssigned to rolling issues managed by the ci-workflow-health.yml workflow.

Type

No type

Projects

No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions