Skip to content

Stack split across merge groups closes the pull request at the boundary instead of continuing it in the next group #444

Open

Description

@smani

Summary

When a stack is split across consecutive merge groups, the pull request at the
split boundary is closed instead of continuing in the next group. Its base
branch (the head branch of the last PR in the merged group) is deleted by the
merge, GitHub re-targets the PR one link up the chain — onto the base's base,
which the same merge also deleted — and then closes it because that branch no
longer exists.

The stack never finishes landing, and the closed PR cannot be recovered through
the API or the UI: each obvious route is refused by a different validation (see
Impact). Recovery required force-pushing two deleted refs back to the remote.

Observed twice in the same stack, ~50 minutes apart, with gh stack v0.1.0.

Pull requests are referred to below as PR A … PR E on branches a … e;
timings, error strings, configuration values and documentation quotes are
verbatim from the real occurrences.

Environment

  • gh stack v0.1.0
  • Base branch main uses a merge queue:
    mergeMethod=SQUASH, mergingStrategy=ALLGREEN,
    maximumEntriesToBuild=8, maximumEntriesToMerge=5, minimumEntriesToMerge=1
  • Repository settings: delete_branch_on_merge: true,
    squash_merge_commit_message: PR_BODY
  • No ruleset on the base branch — GET /repos/{owner}/{repo}/rules/branches/main
    returns [], so require_last_push_approval and dismiss_stale_reviews_on_push
    are both off. This distinguishes this report from Stack merge hangs after partial merge invalidates latest-push approval #174 and Can't merge a stacked PR with no clear reason (all PRs show mergeable) #323, where
    invalidated latest-push approval is the stated mechanism. Every PR here kept
    reviewDecision: APPROVED throughout, including across force-pushes.
  • All PRs APPROVED, MERGEABLE, required checks green when enqueued.
  • Every pull request in the stack was in the queue and building concurrently
    when the merge happened — added_to_merge_queue is recorded for all of them at
    the same second, and none was ejected before the merge. The stack was not too
    large for the group either: in the second occurrence it was 5 pull requests
    against maximumEntriesToMerge=5, so the documented 50-percent buffer was not
    even needed. Only the first three merged.

Reproduction

  1. Create a 5-PR stack: main <- a (PR A) <- b (PR B) <- c (PR C) <- d (PR D) <- e (PR E).
  2. Merge the whole stack (gh stack merge --yes). All five enter the queue
    together — every PR records added_to_merge_queue at the same second — and
    all five build concurrently (maximumEntriesToBuild=8).
  3. The queue merges only PR A, PR B, PR C, while PR D and PR E are
    still in the queue.

Actual

T+0s   PR A, PR B, PR C   merged                     (all three, same second)
T+2s   PR D               base_ref_deleted
T+2s   PR D               closed
T+2s   PR E               removed_from_merge_queue   (actor: github-merge-queue[bot])
  • PR D's base was c. After the merge it reads b — the base of its base,
    deleted in the same operation — and the PR is closed.
  • PR E being ejected is expected and documented; it stays open because its own
    base branch d still exists.
  • The remainder of the stack does not continue in a subsequent merge group. It
    cannot: the PR at the boundary is closed.
  • Why the queue stopped after three is unexplained by any configuration value —
    all five were enqueued and building, and five is within
    maximumEntriesToMerge. Whatever the reason for the boundary, the failure
    reported here is what happens at it.

The first occurrence was identical with an 8-PR stack: three merged, the fourth
closed 2 seconds later, the rest left open.

Expected

Three documentation statements describe what should happen, and the observed
behavior contradicts the first two:

A large stack may also split across consecutive merge groups: the merge queue
allows a merge group to exceed its configured maximum size by up to 50 percent
to keep a stack together, and any pull requests that don't fit continue in
subsequent groups until the full stack has landed.

— Troubleshooting stacked pull requests

Merge part of the stack by merging a mid-stack pull request. The pull requests
below it merge too, and the pull requests above stay open and automatically
re-target the stack's base branch.

— About stacked pull requests

Stacks are kept together in the merge queue. If a pull request is removed or
ejected from the queue, all pull requests above it in the stack are also
ejected and removed.

— Troubleshooting stacked pull requests (this part works as documented)

So the PR at a group boundary should either continue in the next group, or at
minimum stay open re-targeted to the stack's base branch. Re-targeting to
the immediate parent's base is only safe when that branch still exists, which it
never does when the stack is merged as a unit with delete_branch_on_merge
enabled.

Any of these would fix it: re-target survivors to the stack's base rather than
one link up; re-target before deleting merged head branches; or skip head-branch
deletion for a branch that is the base of an open stacked pull request.

Impact: the closed pull request cannot be recovered

Four recovery routes, each refused by a different validation:

PATCH /repos/{owner}/{repo}/pulls/{D}  base=main state=open
  422  Cannot change the base branch because the pull request is part of a stack.

PATCH /repos/{owner}/{repo}/pulls/{D}  state=open
  422  state cannot be changed. The b branch has been deleted.

# after `gh stack unstack <stack>` lifted the stack restriction:
PATCH /repos/{owner}/{repo}/pulls/{D}  base=main state=open
  422  Cannot change the base branch of a closed pull request.

# and where the head had been force-pushed after closing (e.g. by `gh stack sync`):
PATCH /repos/{owner}/{repo}/pulls/{D}  state=open
  422  state cannot be changed. The d branch was force-pushed or recreated.

Changing the base requires the PR to be open; reopening requires the base branch
to exist; and reopening also requires the head to be at the SHA recorded when the
PR closed. With the base branch deleted, there is no ordering of these calls that
succeeds.

Recovery that worked

  1. git push origin <sha of b>:refs/heads/b — restore the deleted base branch.
    The SHA survives only in refs/pull/{B}/head.
  2. Where the head had moved since closing, rewind it to the recorded
    headRefOid (also recoverable from refs/pull/{D}/head).
  3. PATCH /pulls/{D} state=open — now permitted.
  4. gh stack unstack <stack> — required before the base can be changed.
  5. PATCH /pulls/{D} base=main.
  6. Restore the real head, delete the scaffold branch, and gh stack link to
    rebuild the stack.

A user who did not know the SHAs live in refs/pull/{n}/head would have to
abandon the pull request and open a new one, losing its review history.

Notes

  • gh stack merge --help says that under a merge queue the pull requests "are
    added to the queue together but merge as the queue processes them, so they may
    land in separate groups rather than all at once", while Merging stacked pull
    requests
    describes the selected PR and everything below it landing "together
    as a single operation". Both can be true, but with delete_branch_on_merge
    enabled the first has a consequence the docs don't mention: any group boundary
    inside a stack closes the pull request at that boundary.
  • gh stack unstack <stack> reported
    ⚠ Some pull requests are queued for merge or have auto-merge enabled while no
    pull request had either — that is unstack shows a generic/incorrect error instead of the real API rejection reason #372.
  • gh stack view --json reported the closed pull request as "state": "OPEN",
    so neither view nor --short surfaced the closure; only the REST/GraphQL API
    showed CLOSED. Filing separately.

Activity

  1. windsornguyen commented on Aug 16, 2026

    @windsornguyen

    I was able to reproduce this on 2026-08-15 PDT with gh stack v0.1.0.

    Environment

    • Seven pull requests in one stack.
    • Merge queue: ALLGREEN, SQUASH, maximumEntriesToBuild=5, maximumEntriesToMerge=100, minimumEntriesToMerge=1, and no minimum wait.
    • delete_branch_on_merge=true.
    • One approval required, require_last_push_approval=false, and every pull request remained approved.
    • All pull-request checks were green before enqueue.
    • Merge-group checks for layers 1 and 2 ran normally. Layers 3 through 5 were deliberately delayed by 60 seconds. Layers 6 and 7 therefore remained beyond the build-capacity boundary.

    Observed

    All seven pull requests entered the queue at the same second.

    T+50s  layers 1 and 2 removed from queue
    T+51s  layers 1 and 2 merged
    T+53s  layer 3 base_ref_deleted, closed, and removed from queue
    T+53s  layers 4 through 7 removed from queue and left open
    

    After closure, layer 3 reported layer 1 as its base: GitHub retargeted it one link down onto another branch deleted by the same merge. This matches the mechanism in the issue exactly and is independent of latest-push approval behavior.

    Controls

    Two controls behaved correctly:

    1. A five-layer stack with five build slots and similar check durations split into 3 + 2 and fully merged.
    2. A five-layer stack with five build slots and the same delay at layer 3 merged layers 1 and 2, preserved layer 2's branch while layer 3 was building, then merged the remaining three.

    In this fixture, exceeding merge-queue build capacity while a mergeable prefix landed made the failure reproducible. The controls show that automatic branch deletion and consecutive merge groups do not fail unconditionally. The unsafe deletion is sensitive to queue/build state above the boundary.

  2. FilippoVigani commented on Aug 18, 2026

    @FilippoVigani

    Hit this today (as well as a few days ago), gh stack v0.1.0, and the mechanism you describe lines up with what our timeline API shows. Two things I can add: our trigger has nothing to do with build capacity, and there's an ordering trap in the recovery.

    Ours was an 8-PR stack, but only the bottom three were queued — call them A, B, C on a, b, c. Queue config:

    mergeMethod                    SQUASH
    mergingStrategy                HEADGREEN
    maximumEntriesToBuild          15
    maximumEntriesToMerge          5
    minimumEntriesToMerge          1
    minimumEntriesToMergeWaitTime  300
    delete_branch_on_merge         true
    

    All three entered the queue in the same second, all three got merge groups seconds later, and all three group checks passed. Twenty minutes later (times UTC):

    15:07:40   A, B, C   added_to_merge_queue (same second)
    15:08:02   C         merge-group check completed, success
    15:16:36             group entry above C cancelled
    15:25:26             group entry above C cancelled again
    15:27:57   A, B      merged (same second)
    15:27:59   C         automatic_base_change_succeeded   (base b -> a)
    15:28:00   C         base_ref_deleted
    15:28:02   C         closed
    15:28:02   C         removed_from_merge_queue          (github-merge-queue[bot])
    

    Same retarget one link down: once closed, C's base read a, which the same merge had just deleted.

    What made it clear that C wasn't rejected for any reason of its own: its merge-group commit was built on B's group commit and went green at 15:08:02, and the group chain that got rebuilt at 15:28:16 starts from B's merged commit instead. So C was quietly dropped when the group re-formed, twenty minutes after its own check passed.

    The bit I think is new is that there was no capacity pressure anywhere. Three entries against maximumEntriesToBuild=15 and maximumEntriesToMerge=5, nothing ejected before the merge, and the queue still split after two. HEADGREEN and the 300s minimum wait are the settings that differ from the two cases above, if that helps narrow it down.

    On recovery: after the closure I did the obvious next thing and rebased the five surviving PRs onto the new trunk, which force-pushes them. That moves the head off the headRefOid recorded at close, so your step 2 stops being optional. And gh won't tell you which precondition you tripped:

    API call failed: GraphQL: Could not open the pull request. (reopenPullRequest)
    

    REST at least names one of them:

    422  state cannot be changed. The <base> branch has been deleted.
    

    Restoring the deleted base branch wasn't enough on its own for me, which fits the head having moved in the meantime. Since gh stack sync and gh stack rebase will do that force-push for you, it's easy to lose the cheap recovery before you find out it exists — probably worth a line in your recovery notes saying to rescue the closed PR before touching anything above it. I gave up and opened a replacement, so the review history is gone, as you predicted.

    Two smaller things, both matching yours: gh stack unstack warned about queued or auto-merge PRs when there were none (#372), though it did let the base change through afterwards, and gh stack view --json reported the closed PR as OPEN, along with the two already-merged ones.

    Happy to pass repo and PR identifiers to someone at GitHub privately if that would help trace the group decision.

  3. added
    bugBug Reports
    topic: merge queueStacked PRs with merge queues: enqueue only bottom PR, invalid_merge_commit, incompatibility.
    on Aug 24, 2026
  4. broskoTT commented on Aug 27, 2026

    @broskoTT

    Another reproduction, on a different repository and with a different merge-queue
    grouping strategy. Same mechanism and the same T+2s signature, so this is not
    specific to ALLGREEN or to hitting the group-size limit.

    Everything below is publicly verifiable in
    tenstorrent/tt-umd.

    Environment

    • gh stack v0.1.0, gh v2.97.0
    • Stack created with gh stack link; the four PRs were enqueued manually from the
      GitHub UI
      , not via gh stack merge
    • Merge queue on main: merge_method=SQUASH, grouping_strategy=HEADGREEN,
      max_entries_to_build=10, max_entries_to_merge=5, min_entries_to_merge=1,
      min_entries_to_merge_wait_minutes=5
    • delete_branch_on_merge: true

    The stack, bottom to top:

    PR head branch outcome
    #3225 brosko/tt_device_model_skeleton merged 29b93e68
    #3235 brosko/tt_device_is_remote merged 58011604
    #3236 brosko/tt_device_model_protocol merged d8ed3be1
    #3237 brosko/tt_device_model_soc_arch closed, not merged
    #3254 brosko/tt_device_model_arch_impl never queued, still open
    #3264 brosko/tt_device_model_hang_detector never queued, still open

    Only the bottom four were enqueued, all recording added_to_merge_queue at the
    same second. Note this rules out capacity as the trigger: 4 entries against
    max_entries_to_merge=5. The boundary came from the grouping strategy merging the
    green prefix, not from the group being full.

    Actual

    2026-08-27T06:40:12Z  #3225 #3235 #3236 #3237  added_to_merge_queue
    2026-08-27T09:08:10Z  #3225 #3235 #3236        merged
    2026-08-27T09:08:12Z  #3237                    automatic_base_change_succeeded
    2026-08-27T09:08:12Z  #3237                    base_ref_deleted
    2026-08-27T09:08:13Z  #3237                    closed
    2026-08-27T09:08:13Z  #3237                    removed_from_merge_queue  github-merge-queue[bot]
    

    automatic_base_change_succeeded lands in the same second as
    base_ref_deleted, which makes the mechanism this issue describes directly
    visible rather than inferred. #3237 was opened against brosko/tt_device_model_protocol
    (#3236's head); the batch merge deleted it, GitHub retargeted #3237 one link down
    onto brosko/tt_device_is_remote (#3235's head) — deleted by the very same
    batch
    — and closed the PR one second later. gh pr view 3237 still reports
    baseRefName: brosko/tt_device_is_remote today, pointing at a branch that has
    never existed since.

    Unlike the original report, the boundary PR was itself removed_from_merge_queue
    rather than the PR above it, because the PRs above the boundary were never
    enqueued. #3254 and #3264 kept intact bases and stayed open, so the damage was
    confined to the single PR at the boundary.

    Reopen refusal reproduces verbatim

    $ gh pr reopen 3237 --repo tenstorrent/tt-umd
      422  state cannot be changed. The brosko/tt_device_is_remote branch has been deleted.
    

    Recovery preconditions still held

    Corroborating the documented procedure: #3237's head was still at the headRefOid
    recorded when it closed (8c54b537, matching refs/pull/3237/head), and the
    deleted bases were recoverable from refs/pull/3235/head (422fde3e) and
    refs/pull/3236/head (5565b076). So route 1 in Recovery that worked is viable
    here — but only because nothing force-pushed the head after the closure, and only
    for someone who knows refs/pull/{n}/head exists.

    Suggestion

    Since the retarget and the deletion are observably in the same second, a
    stack-aware merge could retarget every downstream PR to the stack's trunk
    before the merged branches are deleted, rather than relying on GitHub's
    one-link-up retarget — which is unsound by construction when a whole prefix of the
    stack merges at once. Failing that, a warning when delete_branch_on_merge is enabled and the stack may
    split across queue groups would at least make the outcome predictable — though note
    that in our case the enqueue happened outside gh stack entirely, so a client-side
    warning would not have fired. gh stack sync detecting a base that points at a
    deleted branch and offering the restack would cover that path too.

  5. voxeljorge commented on Oct 1, 2026

    @voxeljorge

    Another one, gh stack v0.1.0, ALLGREEN / SQUASH, maximumEntriesToBuild=5,
    maximumEntriesToMerge=5, checkResponseTimeout=3600, delete_branch_on_merge: true.
    Four PRs A ← B ← C ← D enqueued in the same second:

    07:21:17  A, B  merged
    07:21:17  C     removed_from_merge_queue  reason: checks_timed_out
    07:21:18  D     removed_from_merge_queue  reason: stack_invalidated
    07:21:20  C     automatic_base_change_succeeded (b -> a), base_ref_deleted (a)
    07:21:21  C     closed
    

    New here: the boundary PR carries an explicit reason (RemovedFromMergeQueueEvent.reason
    in GraphQL), so in this case the split was caused by C's merge-group checks timing out
    (63 min after enqueue, against a 60-min timeout), not by group sizing. Any timed-out check
    above the bottom of a queued stack seems to be enough.

    Recovery route 1 worked as written. gh stack unstack printed the #372 warning and said the
    stack was left in place, but it had in fact removed C and D; the base change succeeded
    right after.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugBug Reportstopic: merge queueStacked PRs with merge queues: enqueue only bottom PR, invalid_merge_commit, incompatibility.

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions