Add `--stagger <duration>` to space out branch pushes and avoid duplicate CI runs

[david-waltermire]

The problem in one paragraph

gh stack sync pushes every branch in the stack at once, in a single atomic push. When that happens, GitHub sometimes sends two or three "the PR changed" events for the same commit instead of one. Each extra event starts an extra CI run. Those extra runs get cancelled, but GitHub keeps every check it has ever posted for a commit — so the cancelled ones stay on the pull request's Checks tab forever, showing red. Reviewers open the PR, see red, and think it is broken when it is not.

Adding a short pause between pushes makes the problem go away completely. I measured this, and about 3 seconds was enough.

Why the extra events happen

Every PR in a stack sits on top of the one below it. So when the whole stack is pushed at once, a middle PR has two things change at the same moment: its own branch, and the branch it is based on.

GitHub appears to process those two changes in parallel, and both end up announcing "this PR's commits changed." You get a duplicate event for a commit that only changed once.

The bottom PR of a stack never has this problem, because its base is the trunk and the trunk is not part of the push. Every PR above the bottom is affected.

Why this cannot be worked around in CI

I tried the obvious fixes first. None of them work:

You cannot filter the duplicate out in a workflow. I checked what GitHub records for each run, and every run created by the push — the real one and the duplicates, across all our workflows — points at the same push. Same event, same commit, same push. There is no field that tells the duplicate apart from the original, so there is nothing to write a condition against.

Concurrency settings do not help. A concurrency group can cancel a running job or make it wait, but it cannot stop the duplicate run from being created — and the checks appear as soon as the run is created. Turning off cancel-in-progress is worse: it doubles the CI bill, and a third event still cancels the queued second one.

Deleting the runs is the only cleanup GitHub offers, and deleting CI history to tidy up a display problem is not something we are willing to do.

Why it matters

• Reviewers misread healthy PRs as broken. One of our PRs showed 59 passing checks, 36 cancelled, 8 skipped, and 2 failed. Both failures were old entries already replaced by passing ones — the PR was genuinely green. But you cannot see that from the Checks tab. On a tall stack this happens to every layer, every time we restack.
• It wastes a lot of CI. Each duplicate run starts its full set of jobs (about 40 for us) before being cancelled. On a 17-PR stack that is several hundred wasted job starts per restack, on a shared runner pool.

What I measured

This is a 17-PR stack in a private repository, restacked several times in one day.

Before — one atomic push of the whole stack:

Out of 8 branches I sampled, 6 got duplicate CI runs.

After — same stack, same branches, pushed one at a time with a pause between each.

Each pause is its own experiment, because what matters for a given PR is the delay between pushing the branch below it and pushing the branch itself. I used pauses from 2 to 45 seconds. The times below are the real measured intervals, which include the time the push itself took:

Pause between pushes	Branches	Duplicate runs
3 seconds	1	0
6–7 seconds	2	0
11–12 seconds	2	0
16–17 seconds	2	0
21–22 seconds	2	0
31 seconds	2	0
46–47 seconds	2	0

Zero duplicates in 14 tries in a row, compared with roughly 3-in-4 branches duplicating when pushed all at once. If the old rate still applied, getting 14 clean branches by luck would be about a 1-in-250-million shot.

Two honest caveats:

• I ran the shortest pauses first. If there were some warm-up effect, it would have made the short pauses look bad — they were clean, so that does not explain the result.
• This was one restack during a quiet period. I would not claim 3 seconds is a guaranteed floor. What I would claim is that a few seconds is plenty, and a minute is unnecessary.

What I am asking for

An opt-in flag that keeps today's behaviour by default:

gh stack sync   --stagger 5s
gh stack submit --stagger 5s
gh stack push   --stagger 5s

With the flag set, push the branches one at a time from the bottom up, pausing for the given duration between each, instead of pushing them all at once.

For a 15-branch stack at 5 seconds, that is a little over a minute — much cheaper than the CI the duplicate runs burn, and far cheaper than the confusion they cause reviewers.

The trade-off, stated plainly

Pausing between pushes means giving up the atomic push, so a failure partway through would leave the stack half pushed. That is already how gh stack push behaves today — its help text says "Updates are not atomic: a branch may update even if another branch is rejected" — so this would not introduce a new kind of failure. It is a good reason to keep the flag opt-in and to mention the trade-off in its help text.

Things I considered and ruled out

Idea	Why it does not work
Filter out the duplicate event in the workflow	The duplicate is identical to the original, right down to the push it points at. Nothing to filter on
Change the concurrency settings	Cannot stop the run being created, and the checks appear with the run
Delete the cancelled runs	Works, but destroys CI history to fix a cosmetic problem
Document it and tell reviewers to check elsewhere	We already do. Reviewers look at the Checks tab, not at our contributing guide
Write our own staggered push script	This is what we are doing now. It works, but it means stepping outside the tool and losing its safety behaviour — which is why I would rather see it supported

Environment

• gh-stack v0.1.0
• GitHub.com, self-hosted runners
• Stack size: 17 PRs

[david-waltermire]

I have an implementation of this ready on a branch, but I cannot open the pull request myself: POST /repos/github/gh-stack/pulls returns 404 for a fork head, and none of the last 100 PRs on this repo originate from a fork. It looks like outside contributions are restricted, which is worth knowing given CONTRIBUTING.md invites forks. Happy to be told I have got that wrong.

The branch is public, so a maintainer can open it directly:

main...david-waltermire:gh-stack:david-waltermire/stagger

Branch: david-waltermire/stagger on david-waltermire/gh-stack, one commit, based on main at ab00aa4.

What it does

Adds --stagger <duration> to sync, submit, and push. With the flag absent or zero, behaviour is unchanged: sync still does its single --force-with-lease --atomic multi-ref push. With a positive duration, branches are pushed one at a time bottom-up with per-branch --force-with-lease, sleeping between pushes and never after the last one.

The three commands have distinct push sites, so all three take the flag, but the multi-ref path routes through one new helper (pushBranches in cmd/utils.go) rather than duplicating the logic. submit already pushed per-branch in a loop interleaved with PR creation, so it only gained the sleep.

Negative durations are rejected with ErrInvalidArgs, and the help text states the trade-off: staggering necessarily gives up --atomic, so a mid-sequence failure leaves the stack partly pushed. That is already the documented behaviour of gh stack push, which is why the flag is opt-in.

Tests

Table-driven, in the existing testify and git.SetOps style, covering flag parsing including negative and malformed values, that zero keeps the single multi-ref push, that a positive value pushes per-branch in bottom-up order, and that no sleep occurs after the final push. Sleeping is faked through a package-level sleepBetweenPushes variable, mirroring the existing git.SetOps swap idiom, since the codebase has no clock abstraction and a real sleep cannot prove the absence of a trailing one.

The three core tests were confirmed failing against unwired code before the commands were routed through the helper, so they do guard the behaviour rather than merely accompany it.

Diff is roughly 82 lines of production code and 429 lines of tests across cmd/, plus --stagger rows and a short note in docs/src/content/docs/reference/cli.md.

go build ./..., go vet ./... and go test -race -count=1 ./... all pass. Two files, internal/modify/state.go and internal/stack/lock_windows.go, already fail gofmt -l on unmodified main; I left them alone.

Glad to revise the approach, the flag name, or the surface it applies to.