Skip to content

commit-reach: terminate merge-base walk when one paint side is exhausted#2149

Draft
spkrka wants to merge 5 commits into
gitgitgadget:masterfrom
spkrka:side-exhaust-pr
Draft

commit-reach: terminate merge-base walk when one paint side is exhausted#2149
spkrka wants to merge 5 commits into
gitgitgadget:masterfrom
spkrka:side-exhaust-pr

Conversation

@spkrka

@spkrka spkrka commented Jun 13, 2026

Copy link
Copy Markdown

Optimize paint_down_to_common() for merge-base queries that hit
large one-sided histories.

When the walk from one side reaches a commit with a very low
generation number that the other side never paints, max_nonstale
stays alive and forces the walk to drain most of the graph. A
common trigger is a repository import that grafts a separate
history with its own root, but any merge that introduces a
low-generation commit never painted by the other side has the
same effect.

A new merge-base candidate can only be discovered when exclusive
PARENT1 and PARENT2 paint meet. This series teaches
paint_down_to_common() to stop as soon as one side has no exclusive
commits left in the queue; once one side is exhausted, no further
candidates can appear.

  origin/HEAD  o   o  PR HEAD
               |   |
     (import)  o   :
              / \ /
             |   o  merge-base
             |   |
             :   :  (~2.5M commits)
             |   |
  import root   main root

This was discussed in an earlier RFC thread [1]. Derrick Stolee
provided a criss-cross counterexample that sharpened the halt
condition, and Elijah Newren independently discovered the same
optimization and shared an implementation in PR #2150 [2]. Patch 3
incorporates test cases from Elijah's branch.

This series implements the optimization only after the walk enters
the finite-generation region, where generation ordering guarantees
that paint on visited commits is final.

Patch organization

1/5 commit-reach: introduce struct paint_queue with per-side counters
2/5 commit-reach: stop the walk when one side is exhausted
3/5 t6600: add side-exhaustion edge-case tests
4/5 t6099, t6600: add side-exhaustion regression tests
5/5 Documentation/technical: document paint_down_to_common()

Patch 5 adds a technical document covering the algorithm, the
INFINITY/finite region boundary, and the termination conditions,
to help reviewers and future contributors follow the reasoning.

Benchmarks

Measured on a 2.6M-commit monorepo with commit-graph (baseline
v2.55-rc1):

merge-base --all  (across import)       4.293s ->    8ms  (537x)
merge-tree        (across import)       5.345s ->   13ms  (411x)
merge-base --all  (1000 commits apart)  5.404s ->    7ms  (772x)

No regression on linux.git (1.4M commits, commit-graph):

merge-base HEAD HEAD~1000                 38ms ->   40ms
merge-base --all HEAD HEAD~1000           87ms ->   36ms
merge-base --is-ancestor HEAD~1000 HEAD   11ms ->   11ms
merge-base --all HEAD HEAD~10000         626ms ->  428ms

[1] https://lore.kernel.org/git/CAL71e4Ps-2_0+uuZu43N9pFnXBemoAohPs_eyRJf8taXHJPAXQ@mail.gmail.com/T/#u
[2] #2150

@spkrka spkrka force-pushed the side-exhaust-pr branch 5 times, most recently from 33092ed to e5d2f58 Compare June 18, 2026 15:26
Replace the nonstale_queue abstraction in paint_down_to_common() with
a new paint_queue struct that tracks per-side commit counts. Each
non-stale queued commit occupies exactly one counter bucket based on
its paint flags: PARENT1-only, PARENT2-only, or both sides (a pending
merge-base candidate).

The counters are maintained by paint_count_transition() which handles
all flag changes as bucket transfers: remove from the old bucket, add
to the new one. Either step is a no-op when the respective state has
no bucket (stale or zero).

The loop condition changes from pointer-based (max_nonstale) to
counter-based (while any counter is positive), which is equivalent.

ahead_behind() is decoupled to use its own local tracking, since it
does not need per-side counters. No behavior change.

Signed-off-by: Kristofer Karlsson <krka@spotify.com>
@spkrka spkrka force-pushed the side-exhaust-pr branch from e5d2f58 to 712f1df Compare June 18, 2026 15:31
spkrka and others added 2 commits June 18, 2026 17:40
Add an early termination check to paint_down_to_common() using the
per-side counters introduced in the previous commit. Once the walk
enters the finite-generation region, terminate early when one side's
exclusive count drops to zero -- no new merge-base can form without
both paint sides meeting.

The check also waits for pending_merge_bases to reach zero, ensuring
all merge-base candidates have been popped and recorded before
exiting.

The INFINITY gate ensures correctness: commits without a commit-graph
entry have GENERATION_NUMBER_INFINITY and are ordered by commit date,
which is not topologically reliable. The optimization only fires
once the walk enters the finite-generation region where ordering
guarantees hold.

On large repositories with commit-graph, this yields 100-1000x
speedups for merge-base queries where one side (e.g. a PR branch) is
much smaller than the other.

Helped-by: Derrick Stolee <stolee@gmail.com>
Helped-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Kristofer Karlsson <krka@spotify.com>
Add test cases to t6600-test-reach.sh that exercise edge cases in the
side-exhaustion optimization for paint_down_to_common():

 - in_merge_bases_many:self: commit is both A and one of the X inputs
 - get_merge_bases_many:duplicate-twos: duplicate entries in X list
 - get_merge_bases_many:pending-stale: STALE transition on an
   already-painted commit (ps-* diamond topology)
 - get_merge_bases_many:infinity-both-sides: both tips outside the
   commit-graph with non-monotonic dates (pi-* topology)

Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Kristofer Karlsson <krka@spotify.com>
@spkrka spkrka force-pushed the side-exhaust-pr branch 2 times, most recently from b0a9ae9 to ff6ca03 Compare June 18, 2026 19:59
spkrka added 2 commits June 19, 2026 09:47
Add t6099 to test the case where multiple merge-base candidates exist
and one is an ancestor of another. This exercises the side-exhaustion
optimization in paint_down_to_common together with the
remove_redundant safety net in get_merge_bases_many_0.

Add a mixed finite/INFINITY test to t6600 where one tip is outside
the commit-graph (INFINITY generation) and the other is inside.
This exercises the region transition: the walk starts in the
INFINITY region where side-exhaustion is disabled, then crosses
into the finite region where it can fire.

Signed-off-by: Kristofer Karlsson <krka@spotify.com>
Add a technical document describing merge-base computation
and specifically paint_down_to_common() implementation.

Signed-off-by: Kristofer Karlsson <krka@spotify.com>
@spkrka spkrka force-pushed the side-exhaust-pr branch from ff6ca03 to 7d5b1bb Compare June 19, 2026 07:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants