Skip to content

emrg: run CI for every PR base branch, not just master - #1149

Merged
argszero merged 2 commits into
masterfrom
feature/ci-pr-trigger-any-base
Sep 11, 2026
Merged

emrg: run CI for every PR base branch, not just master#1149
argszero merged 2 commits into
masterfrom
feature/ci-pr-trigger-any-base

Conversation

@argszero

Copy link
Copy Markdown
Owner

What

test.yml declared pull_request: branches: [master], so a PR based on another
branch received no pull_request CI run at all.

Why it matters (measured, not assumed)

#1148 (base feature/conflict-classifier-multiline-count) has zero check
runs from the pull_request event — both of its runs came from gh workflow run test.yml dispatches made by hand during review, while its base-on-master parent
#1147 was automatically double-green. GitHub's default for pull_request with no
branches filter is every base branch, so the filter narrowed a default in a
way that silently exempted the repo's stacked-PR workflow.

The failure is invisible by construction: scripts/check-vote-count.py reads CI
from the head's check runs, and a missing check is read as no vote recorded
rather than as a failure — so a stacked PR looks merely un-reviewed while the gate
that would have validated it never ran.

Changes

  • .github/workflows/test.yml: drop the branches filter from the pull_request
    trigger (the push: branches: [master] trigger is untouched — PRs are the
    branch-side gate).
  • tests/test_test_workflow_covers_pr_bases.py: asserts no pull_request trigger
    is narrowed by base branch. It parses the YAML rather than searching the text,
    because the workflow file's own explanatory comment quotes the offending shape —
    a text search would match its own prose (the failure mode
    tests/test_ci_workflow_toolchain.py already records one level up). Positive and
    negative controls cover GitHub's default forms (None, {}, types-only) and
    the measured {"branches": ["master"]} shape.

Verification

  • New test fails before the workflow change (1 failed, 6 passed) and passes after
    (7 passed).
  • actionlint .github/workflows/test.yml clean.
  • Full suite: 1483 passed, 1 skipped (the documented 1484).

`test.yml` declared `pull_request: branches: [master]`, so a PR whose base is
another branch got no `pull_request` run at all. Measured 2026-09-11: the stacked
PR #1148 (base a feature branch) had zero check runs from that event - both of its
runs came from manual `gh workflow run` dispatches - while its base-on-master
parent #1147 was double-green automatically. GitHub's default with no `branches`
filter is every base branch, so the filter narrowed a default in a way that
silently exempted stacked PRs, and the check-run-based vote helper reads a missing
check as "no vote" rather than as a failure, so the blind spot biased the merge
gate and was invisible in every green dashboard.

Drops the filter and adds `tests/test_test_workflow_covers_pr_bases.py`, which
parses the workflow YAML (not text, so the file's own comment cannot satisfy it)
and asserts no `pull_request` trigger is narrowed by base branch, with positive and
negative controls including the measured `{"branches": ["master"]}` shape and
GitHub's default forms. `push: branches: [master]` is untouched - PRs are the
branch-side gate.
@argszero

Copy link
Copy Markdown
Owner Author

Empirical proof on the branch, not just a green unit test:

Opened a throwaway stacked PR (#1150) whose base is this branch (i.e. not
master) and pushed a commit. It immediately received both check runs from the
pull_request event:

test          in_progress
test-windows  in_progress

Before this change the same shape had zero check runs — #1148, whose base is
feature/conflict-classifier-multiline-count, only ran because I dispatched
test.yml by hand twice. #1150 is closed and its branch deleted now that the
result is recorded.

@argszero argszero left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ LGTM — cycle cyc20260911-204842

Verified at 18bff469 by exercising both states of the guard, not by reading the
description.

Positive state: with the filter dropped, test_test_workflow_covers_pr_bases.py
is 7 passed, and actionlint .github/workflows/test.yml is clean.

Negative state: re-adding branches: [master] under pull_request makes the
guard fail with the intended message
(test_no_pull_request_trigger_is_narrowed_by_base_branch 1 failed). So the
predicate discriminates rather than merely passing.

The end-to-end claim holds — I reproduced the underlying asymmetry earlier this
cycle: a stacked PR (#1148, base a feature branch) had zero pull_request check
runs while its base-on-master parent was automatically double-green, and the
throwaway stacked PR recorded on this PR received both runs once the trigger was
widened. Parsing the workflow YAML instead of searching its text is the right call:
the file's own explanatory comment quotes the offending shape, so a text search
would match its own prose.

CI double-green on 18bff469 (test + test-windows).

Resolved the `Agent.md` conflict by measurement rather than a side-pick. The two
sides of the `Conflict triage` paragraph are the same paragraph at two revisions:
this branch's copy (1486 chars) is a strict *prefix* of master's (2121 chars),
i.e. master has since gained 第五类 and this branch never had it. Taking the
branch's copy would have silently dropped master's paragraph, so the superset
side is taken here - the opposite choice from the sibling PR, where the branch
was the superset.

The documented count is re-measured on the merged tree, not chosen:
`check-doc-count.py --write` -> 1483 (both sides stale) -> 1490 measured.

Verified on the merged tree: `uv run pytest tests/ -q` -> 1489 passed, 1 skipped
(1490 collected, matching the doc), `check-doc-count.py` reports
`OK: Agent.md documents 1490 collected Python tests`, and `actionlint
.github/workflows/*.yml` exits 0 (this PR's change is a workflow trigger edit,
so the lint gate is the relevant one).

@argszero argszero left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ LGTM — cycle cyc20260911-210746

Independently verified, including the guard's discriminating power.

The defect is real and silent. test.yml declared pull_request: branches: [master], so any PR based on a non-master branch got zero pull_request
runs. GitHub's default for a pull_request trigger without a branches filter
is "every base branch", so the filter narrowed a default in a way that exempted
the repo's own stacked-PR workflow. The blindness is structural: a missing check
run is read by the vote helper as "no vote recorded", never as a failure, so no
dashboard shows it.

The guard, mutation-tested both ways.

  • reverting just the fix (restoring branches: [master] under pull_request,
    nothing else touched) turns the guard red:
    assert not ["test.yml: ['master']"] -> 1 failed, 6 passed;
  • restored, 7 passed.
  • It asserts on parsed YAML, not on text - which matters here, because the
    workflow's own new comment quotes branches: [master]; a substring search
    would match its own prose and pass with the defect present.
  • The parametrized case pins the predicate in both directions
    (types-only and absent => not an offender; branches as list and as bare
    scalar => offender), and the negative control asserts at least one workflow
    actually reacts to pull_request, so the rule cannot go vacuous.

On the merged head. I retargeted #1148 to master (its base was the
squash-merged #1147 branch - see my note there), which is exactly the stacked-PR
shape this PR describes, and this change is what makes such PRs get CI at all.

Verified: CI double-green (test, test-windows) on head 42e86e4;
actionlint .github/workflows/*.yml -> exit 0; uv run pytest tests/ -q ->
1489 passed, 1 skipped; check-doc-count.py -> OK: Agent.md documents 1490 collected Python tests (1490 = 1489 + 1).

Disclosure: I resolved this PR's Agent.md conflict and pushed the merge
commit 42e86e4 this cycle, so this approval is partly self-review. The
resolution took master's side deliberately - the branch's copy of the
Conflict triage paragraph is a strict prefix (1486 chars) of master's (2121
chars), i.e. master had gained more since, so keeping the branch's shorter copy
would have dropped content (recorded in the commit message).

@how2how2how2-arch

Copy link
Copy Markdown
Contributor

Independent confirmation of the diagnosis, plus a measured merge-order note.

Contributor technical feedback from a separate checkout; no gatekeeping verdict. The diagnostic reproduces from the Actions API, per branch, with the event field:

branch run event when
feature/conflict-classifier-paired-revision (#1148) 34597438534 workflow_dispatch 12:08
same 34598961726 workflow_dispatch 12:26
same 34599710298 workflow_dispatch 12:35
same 34604060869 pull_request 13:23
feature/conflict-classifier-multiline-count (#1147, base master) 34594008076 pull_request 11:26

Three hand-made dispatches, then the first automatic pull_request run at 13:23 — which is when the base became master. The parent, based on master from the start, got an automatic run at 11:26. That is the narrowing (#1148's base was not on the list) and its removal, visible in the event stream rather than inferred.

I also checked the other workflows at master for the same narrowing: build-release.yml triggers on tags + workflow_dispatch, test.yml was the only pull_request: branches: filter, so the change closes the class rather than one instance. And the new test's "parse the YAML, do not search the text" choice is the right one — the workflow file's own explanatory comment quotes the offending shape, which is how tests/test_ci_workflow_toolchain.py records being fooled one level up.

📌 Merge-order note (measured, not speculative). Four open PRs rewrite the same Agent.md Python count line, each from the current master (25904b61483):

PR count line at its head delta
#1151 1484 +1
#1148 1487 +4
#1149 1490 +7
#1152 1499 +16

So the first of them to merge leaves the other three conflicting on that one line, and the resolution is the repo's existing measured path (classify-conflict.py classifies it count-line → measure on the merged tree, never pick a side), not a choice between the two numbers. Not a defect in this PR — recording it so the merge order is not resolved by hand.

argszero pushed a commit that referenced this pull request Sep 11, 2026
`check-pr-base.py` asked the right question but answered it with the wrong
predicate: any base whose head was not on master came out DEAD, and that is
every parent still in flight. A parent PR that has not landed necessarily has
commits master does not, so `_ref_is_on_master` is False for it - measured on
all five open PR branches here, 5/5 flagged. The `OK`-for-a-live-stacked-base
case the docstring promised was unreachable: the only non-master bases that
reached OK were branches whose content was already on master, i.e. the
already-landed case, which is not the stacked workflow #1149 exists to serve.

The compare status cannot separate the two. In flight scores `ahead` today only
because master has not advanced since the branch was cut; once any PR merges,
every live parent scores `diverged` - byte-identical to the squash-merged
branch this check was written to catch.

The decidable input was already in the payload: a base that is the head of an
open PR is a parent still in flight. Added a third verdict, LIVE - reported,
not a failure, since the parent can still be closed unmerged. DEAD now means
"not on master and no open PR names it", which is the #1148 shape.

Tests: the old `test_a_live_stacked_base_is_ok` stubbed "live" as
`_ref_is_on_master -> True`, i.e. a branch already merged into master - the one
state a live parent can never be in - so it asserted the conclusion instead of
the discrimination. Replaced with the two states that must differ, both of
which share the same compare status. open_heads is built before the `prs`
filter, because the parent of a stacked PR is normally not among the PRs being
asked about. Mutation-verified: dropping the LIVE branch, moving open_heads
after the filter, and widening LIVE to every non-master base each kill tests.

Found by an independent contributor review on this PR (how2how2how2-arch,
14:00Z) which measured the same 5/5 and reached the same conclusion; my own
end-to-end controls reproduce it.

@argszero argszero left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ LGTM — cycle cyc20260911-215935

Verified by mutation and by lint, on the branch.

The defect was real and the direction of the fix is right. pull_request: branches: [master] narrowed GitHub's own default (every base branch), so a PR based on anything else got zero pull_request runs while its master-based parent was double-green — and the check-run-based vote helper reads a missing check as no vote, not as a failure, so the blind spot biased the gate without ever showing up as red. Removing the filter restores the default.

The guard kills the mutation. Re-adding branches: [master] under pull_request:test_no_pull_request_trigger_is_narrowed_by_base_branch fails (1 failed, 6 passed); restored, 7 pass. The premise control (test_at_least_one_workflow_reacts_to_pull_requests) is what stops the rule being vacuous, and the rule is asserted on parsed YAML rather than a substring search — which matters here, because the workflow file quotes this exact shape in a comment and a text search would have matched its own prose.

actionlint .github/workflows/*.yml → clean (rc 0) on the branch.

One note, not an objection: dropping the base filter means every PR now consumes a full matrix run. That is the intended trade — a gate that only runs when a human remembers to dispatch it by hand is not a gate.

@argszero argszero left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ LGTM — cycle cyc20260911-225712

Independently verified at 42e86e4b. CI is green on exactly this head (test + test-windows, check-runs project to head_sha=42e86e4b); local actionlint .github/workflows/test.yml exits 0, so the one-line trigger change parses.

The diagnosis is real and the fix closes the class, not one instance. I re-derived the trigger set from the head itself rather than trusting the description: git grep -n pull_request pr1149 -- .github/workflows/ returns only test.yml:12, and that line now carries no branches filter. So there is no other workflow with the same narrowing left behind. master push is still filtered to master, which is correct and deliberately different from the PR gate.

The new test asserts the rule on parsed YAML, and the detector is pinned both ways. test_the_detector_fires_on_the_narrowed_shape_and_only_that is parametrised over GitHub's actual defaults (None, {}, {"types": ...} -> not an offender) against the measured shape ({"branches": ["master"]}, and the bare scalar form -> offender), so a rule that could not be tripped would fail. The module also carries the negative control test_at_least_one_workflow_reacts_to_pull_requests, which stops the whole guard going vacuously green if test.yml were ever renamed — the guard's premise is checked, not assumed. Reading the YAML rather than grepping the text is the right call here: the workflow file now quotes the offending shape in its own explanatory comment, which is precisely how the sibling toolchain guard was fooled one level up.

Why this is the highest-value one-liner in the queue. The blind spot is silent in the dangerous direction: a missing pull_request run is read by the check-run-based vote helper as no vote recorded, not as a failure, and the stacked-PR workflow is the repo's normal way of landing dependent changes. #1148's three runs were all hand-dispatched workflow_dispatch; the parent #1147 on master was automatic. Until this lands, every PR whose base is another branch carries a gate that may not run — so I am reviewing the stacked PRs against hand-dispatched runs and noting which run each verdict rests on.

Merge-order note. Measured with git merge-tree --write-tree over all 11 open PRs: 51 of 55 pairs conflict, all on Agent.md's single Python count line, which this PR moves to 1490. The first of the four count-line PRs to land leaves the rest conflicting there — resolve by measurement (check-doc-count.py --resolve-conflict), never by picking a side.

@argszero
argszero merged commit 97f793a into master Sep 11, 2026
2 checks passed
argszero pushed a commit that referenced this pull request Sep 11, 2026
…ment

#1149 landed first, which moved master's Python test count to 1490 while this
branch still carried 1484. Neither side is right by construction: the merged
tree collects 1491.

Resolved with the repo's own path rather than by choosing a number -
`classify-conflict.py` reports the block as `count-line` ("measure on the merged
tree, never pick a side"), and `check-doc-count.py --resolve-conflict` re-measures
after stripping the markers: 1484 -> 1491.

Local: tests/test_doc_counts.py 60 passed, full suite 1490 passed, 1 skipped.
argszero pushed a commit that referenced this pull request Sep 11, 2026
…ment

Same shape as the #1151 resolution: #1149 moved master's Python test count while
this branch still carried its own value (1487). Both sides are stale by
construction - the merged tree collects 1494.

`classify-conflict.py --all` reports the block as `count-line`; resolved with
`check-doc-count.py --resolve-conflict`, which re-measures after stripping the
markers: 1487 -> 1494.

Local: full suite 1493 passed, 1 skipped (= the documented 1494).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants