Skip to content

Consume unchanged effect-free subtrees during convergence passes - #6251

Closed
ondrejmirtes wants to merge 1 commit into
2.2.xfrom
consume-convergence
Closed

ondrejmirtes wants to merge 1 commit into
2.2.xfrom
consume-convergence

Conversation

@ondrejmirtes

Copy link
Copy Markdown
Member

Third extraction chunk from the single-pass analyser branch (follow-up to #6249).

Convergence passes re-walked the whole loop body every round. Now each pass chains the previous pass's storage as its baseline and keeps the full ExpressionResult of every expression it walks in a per-site frame: an expression whose stored result is effect-free (no scope derived, no throw/impure points) and whose read state is unchanged (variables and tracked expression holders, both attribute-cached per node) is consumed instead of re-walked — only the loop-carried parts of the body re-walk.

To keep #6249's replay firing on consuming passes, every walked result tags its emission segment [recording, start, end) and a consumption splices that segment from the pass that last walked the subtree into the consuming pass's recording; only a segment-less consumption gaps the pass and clears its replay candidacy.

One subtlety: the old-world pricing walks that resolveType() starts for short-circuit operators re-enter processExprNode() mid-pass with a fresh throwaway storage. They emit nowhere, so they may consume stored results but must not splice, store, or tag — keying those operations on the pass's own storage identity fixed emission duplication the naive version produced.

On PHPStan's own handler-heavy code, ~29% of pass expression walks are consumed. The CPU effect is modest — the expensive subtrees (calls with throw/impure points) fail the effect-free gate by construction, so what is consumable is also cheap to re-walk:

corpus before after Δ
self-analysis (--debug src, 3+3 interleaved) 80.30s 79.30s −1.2%
WordPress core (e2e corpus, level 0) ~173s ~172s ~0 to −2%, noise-limited

Analysis output is byte-identical on full self-analysis with the feature on vs off; full test suite, make phpstan, and CS are green. No turbo-ext changes (the per-site frames are plain PHP state; the shadowed storage class is untouched).

🤖 Generated with Claude Code

https://claude.ai/code/session_01GnwgpaeUXRkgSDyg95tfK8

A fixpoint re-walk (loop body convergence, backward-goto replay, by-ref
closure convergence) re-walked the whole body every round. Each pass now
chains the previous pass's storage as its baseline and keeps the full
ExpressionResult of every expression it walks in a per-site frame: an
expression whose stored result is effect-free (the walk derived no scope
and carries no throw/impure points whose recorded scopes would replay
stale state into catch merges) and whose read state - variables and
tracked expression holders - did not change since the pass that walked it
is consumed instead of re-walked. Only the loop-carried parts of the body
re-walk. The final walk is untouched: it runs outside the pass mode on
the outer storage.

A consumption skips the subtree's recorded emissions, which would leave
the fixpoint pass's recording incomplete and stop it from replacing the
final walk. So every walked result also tags its emission segment
[recording, start, end) and a consumption splices that segment from the
pass that last walked the subtree into the consuming pass's recording -
only a segment-less consumption gaps the pass and clears its replay
candidacy. The spliced scopes agree with the replay on everything the
consumed subtree reads - exactly what the consume gate certifies - and
the chained pass storages carry the matching before-scopes.

Consumption, tagging and frame stores apply only to walks carrying the
pass's own storage: the old-world pricing walks that resolveType()
starts for short-circuit operators re-enter processExprNode() with a
fresh throwaway storage - they emit nowhere, so they may consume stored
results but must not splice segments (a mispriced splice duplicated
emissions in the pass recording), store results, or tag brackets.

The site and pass frames live on NodeScopeResolver like the storage
stack; the public processNodes()/processStmtNodes() entries suspend
them, so a fresh walk an extension starts mid-analysis does not inherit
the interrupted walk's convergence state. Unlike the originating branch,
no recording clearing is needed: full results live only in site-scoped
frames that pop when the site's convergence loop finishes, so a segment
can never outlive its recording.

(adapted from commits 4a25082 and
3e9d22e)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GnwgpaeUXRkgSDyg95tfK8
@ondrejmirtes

Copy link
Copy Markdown
Member Author

Closing unmerged: A/B measurements show the consume mechanism buys at most ~1% anywhere — upstream (−1.2% self-analysis, ~0 on WordPress) and on the single-pass branch (0 to −1%, noise-limited, point estimate even slightly negative). The expensive subtrees fail the effect-free gate by construction, so what is consumable is also cheap to re-walk, and the gate itself costs comparably. The replay (#6249) stays as the convergence win; this complexity does not pay for itself.

@ondrejmirtes
ondrejmirtes deleted the consume-convergence branch August 24, 2026 19:12
ondrejmirtes added a commit that referenced this pull request Aug 24, 2026
The branch's own fixpoint-replay commits were dropped during the rebase
over the merged extraction chunks (#6249); this restores the replay in
upstream's final form - raw-recorded pairs wrapped at replay time, the
RecordingNodeCallback short-circuit in callNodeCallback - woven into the
branch's handler shapes (ambient storage push around pass walks, the
deferred While_ statement callback, the on-demand falsey cond
re-pricing). replayRecording() binds the storage through the scope's
push/pop like every other branch-side ambient binding. The consume
mechanism (#6251, closed unmerged) is gone entirely: convergence
behavior on this branch is now byte-for-byte upstream's algorithm.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GnwgpaeUXRkgSDyg95tfK8
ondrejmirtes added a commit that referenced this pull request Aug 24, 2026
The branch's own fixpoint-replay commits were dropped during the rebase
over the merged extraction chunks (#6249); this restores the replay in
upstream's final form - raw-recorded pairs wrapped at replay time, the
RecordingNodeCallback short-circuit in callNodeCallback - woven into the
branch's handler shapes (ambient storage push around pass walks, the
deferred While_ statement callback, the on-demand falsey cond
re-pricing). replayRecording() binds the storage through the scope's
push/pop like every other branch-side ambient binding. The consume
mechanism (#6251, closed unmerged) is gone entirely: convergence
behavior on this branch is now byte-for-byte upstream's algorithm.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GnwgpaeUXRkgSDyg95tfK8
ondrejmirtes added a commit that referenced this pull request Aug 24, 2026
The branch's own fixpoint-replay commits were dropped during the rebase
over the merged extraction chunks (#6249); this restores the replay in
upstream's final form - raw-recorded pairs wrapped at replay time, the
RecordingNodeCallback short-circuit in callNodeCallback - woven into the
branch's handler shapes (ambient storage push around pass walks, the
deferred While_ statement callback, the on-demand falsey cond
re-pricing). replayRecording() binds the storage through the scope's
push/pop like every other branch-side ambient binding. The consume
mechanism (#6251, closed unmerged) is gone entirely: convergence
behavior on this branch is now byte-for-byte upstream's algorithm.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GnwgpaeUXRkgSDyg95tfK8
ondrejmirtes added a commit that referenced this pull request Aug 30, 2026
The branch's own fixpoint-replay commits were dropped during the rebase
over the merged extraction chunks (#6249); this restores the replay in
upstream's final form - raw-recorded pairs wrapped at replay time, the
RecordingNodeCallback short-circuit in callNodeCallback - woven into the
branch's handler shapes (ambient storage push around pass walks, the
deferred While_ statement callback, the on-demand falsey cond
re-pricing). replayRecording() binds the storage through the scope's
push/pop like every other branch-side ambient binding. The consume
mechanism (#6251, closed unmerged) is gone entirely: convergence
behavior on this branch is now byte-for-byte upstream's algorithm.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GnwgpaeUXRkgSDyg95tfK8
ondrejmirtes added a commit that referenced this pull request Aug 30, 2026
The branch's own fixpoint-replay commits were dropped during the rebase
over the merged extraction chunks (#6249); this restores the replay in
upstream's final form - raw-recorded pairs wrapped at replay time, the
RecordingNodeCallback short-circuit in callNodeCallback - woven into the
branch's handler shapes (ambient storage push around pass walks, the
deferred While_ statement callback, the on-demand falsey cond
re-pricing). replayRecording() binds the storage through the scope's
push/pop like every other branch-side ambient binding. The consume
mechanism (#6251, closed unmerged) is gone entirely: convergence
behavior on this branch is now byte-for-byte upstream's algorithm.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GnwgpaeUXRkgSDyg95tfK8
ondrejmirtes added a commit that referenced this pull request Aug 31, 2026
The branch's own fixpoint-replay commits were dropped during the rebase
over the merged extraction chunks (#6249); this restores the replay in
upstream's final form - raw-recorded pairs wrapped at replay time, the
RecordingNodeCallback short-circuit in callNodeCallback - woven into the
branch's handler shapes (ambient storage push around pass walks, the
deferred While_ statement callback, the on-demand falsey cond
re-pricing). replayRecording() binds the storage through the scope's
push/pop like every other branch-side ambient binding. The consume
mechanism (#6251, closed unmerged) is gone entirely: convergence
behavior on this branch is now byte-for-byte upstream's algorithm.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GnwgpaeUXRkgSDyg95tfK8
ondrejmirtes added a commit that referenced this pull request Aug 31, 2026
The branch's own fixpoint-replay commits were dropped during the rebase
over the merged extraction chunks (#6249); this restores the replay in
upstream's final form - raw-recorded pairs wrapped at replay time, the
RecordingNodeCallback short-circuit in callNodeCallback - woven into the
branch's handler shapes (ambient storage push around pass walks, the
deferred While_ statement callback, the on-demand falsey cond
re-pricing). replayRecording() binds the storage through the scope's
push/pop like every other branch-side ambient binding. The consume
mechanism (#6251, closed unmerged) is gone entirely: convergence
behavior on this branch is now byte-for-byte upstream's algorithm.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GnwgpaeUXRkgSDyg95tfK8
ondrejmirtes added a commit that referenced this pull request Sep 1, 2026
The branch's own fixpoint-replay commits were dropped during the rebase
over the merged extraction chunks (#6249); this restores the replay in
upstream's final form - raw-recorded pairs wrapped at replay time, the
RecordingNodeCallback short-circuit in callNodeCallback - woven into the
branch's handler shapes (ambient storage push around pass walks, the
deferred While_ statement callback, the on-demand falsey cond
re-pricing). replayRecording() binds the storage through the scope's
push/pop like every other branch-side ambient binding. The consume
mechanism (#6251, closed unmerged) is gone entirely: convergence
behavior on this branch is now byte-for-byte upstream's algorithm.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GnwgpaeUXRkgSDyg95tfK8
ondrejmirtes added a commit that referenced this pull request Sep 2, 2026
The branch's own fixpoint-replay commits were dropped during the rebase
over the merged extraction chunks (#6249); this restores the replay in
upstream's final form - raw-recorded pairs wrapped at replay time, the
RecordingNodeCallback short-circuit in callNodeCallback - woven into the
branch's handler shapes (ambient storage push around pass walks, the
deferred While_ statement callback, the on-demand falsey cond
re-pricing). replayRecording() binds the storage through the scope's
push/pop like every other branch-side ambient binding. The consume
mechanism (#6251, closed unmerged) is gone entirely: convergence
behavior on this branch is now byte-for-byte upstream's algorithm.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GnwgpaeUXRkgSDyg95tfK8
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant