Skip to content

checkGitHubCI blocks indefinitely on orphaned check suites stuck in QUEUED #1160

Description

@trivenay

Problem

checkGitHubCI() in pr_checker.js blocks landing with ✘ GitHub CI is still running when any GitHub Actions check suite has status !== 'COMPLETED'. However, GitHub occasionally creates check suites that remain stuck in QUEUED status with 0 check runs indefinitely — they are never dispatched and will never complete.

This has blocked at least two recent PRs on nodejs/node:

  • #64991Test macOS suite stuck 3+ days, landed manually
  • #64830 — same Test macOS suite stuck, landed manually

In both cases, the orphaned suite:

  • Has status: QUEUED, conclusion: null
  • Has 0 check runs (never started executing)
  • Has a valid workflowRun (Test macOS, workflow ID 858952)
  • Was created at the same time as sibling suites that completed fine
  • Never gets re-triggered by request-ci because GitHub sees it as already "queued"
  • The only workaround is a collaborator manually running git node land --yes

The commit-queue.sh pre-check (gh pr checks | grep pending) passes fine because it evaluates individual check runs (all completed), not suite containers.

Data

PR #64991, SHA 94a47d6f15b2:

  • 25 GitHub Actions suites total
  • 24 completed (success/skipped)
  • 1 orphaned: suite 84102428609, Test macOS, QUEUED, 0 check runs, created 2026-08-05T13:19:48Z

No newer run of Test macOS exists for this SHA — request-ci does not re-trigger a workflow GitHub considers already "queued."

Proposed fix

Consider a suite as orphaned and skip it if:

  • status !== 'COMPLETED'
  • checkRuns.nodes.length === 0 (nothing has ever executed inside it)
  • Suite age exceeds a configurable threshold (suggesting 3 hours since createdAt — legitimate suites dispatch runs within seconds)
if (status !== 'COMPLETED') {
  const runCount = checkRuns?.nodes?.length ?? 0;
  const ageMs = Date.now() - new Date(createdAt).getTime();
  if (runCount === 0 && ageMs > ORPHANED_SUITE_TIMEOUT) {
    cli.warn(`Ignoring orphaned check suite with no runs (status: ${status}, age: ${Math.round(ageMs / 3600000)}h)`);
    continue;
  }
  pendingJobs.push({ app: app.slug, status, conclusion });
  continue;
}

GraphQL change: add createdAt to the checkSuites query in PRCommits.gql.

Trade-offs

What this fixes: Orphaned suites no longer block the commit queue forever.

Risk: If someone adds commit-queue before request-ci has finished triggering all workflows, a suite that should have run but hasn't started yet (0 runs, within the threshold) could age past the timeout and get skipped — landing the PR without that test suite's results.

However, the current workaround (git node land --yes) already bypasses ALL checks — including real failures. A targeted skip of empty aged-out suites is strictly safer than the current manual workaround.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions