Skip to content

fix(test): consume runtime-app-reload frames monotonically (#111) - #114

Closed
ScriptedAlchemy wants to merge 3 commits into
mainfrom
fix/overview-e2e-reload-frames
Closed

fix(test): consume runtime-app-reload frames monotonically (#111)#114
ScriptedAlchemy wants to merge 3 commits into
mainfrom
fix/overview-e2e-reload-frames

Conversation

@ScriptedAlchemy

@ScriptedAlchemy ScriptedAlchemy commented Sep 1, 2026

Copy link
Copy Markdown
Owner

Fixes #111

Problem

packages/workbench/tests/overview.e2e.test.ts (offers the host-owned MCP playground handoff...) failed intermittently with an extra runtime-app-reload generation frame beyond the exact HMR message list it pinned. Reproduced 1-in-5 on unmodified main under Node 26, and it was the dominant local-ci gate tax.

Captured green sequences on this machine were [generation 0 (connect replay), generation 1 (changed App compile)]. A later generation-0 frame appears only after MCP playground handoff opens a second proxy socket. The flake is an extra frame of an already-seen or late ordinal landing between the post-edit snapshot and config-reconcile equality.

Fix

Two complementary shapes, both required by the evidence:

  1. Channel: the Replace raw Rsbuild HMR frames with a provider-owned Runtime App reload channel #73 App reload plugin no longer mints a frame from hashless/empty-hash App completions. Treating the stats object identity as a hash emitted a spurious reload on every unidentifiable completion. Regression in dev-provider.integration.test.ts (emits one owned App reload...).
  2. Assertion (PR fix: root-cause the CI flakes instead of rerunning them #20 shape): overview e2e consumes generations monotonically (non-decreasing observed sequence, duplicate-tolerant unique ordinals as a prefix). An extra at-least-once frame or a late compile from the source edit no longer fails config-reconcile. Failures dump generation ordinals, raw frames, and SSE hub {latestSequence, subscriptionCount}.

Product contract is unchanged: one owned reload per later successful changed App compilation. Hashless compiles were never a proven generation.

Test plan

  • Plugin unit: hashless/empty-hash completions do not emit
  • 20/20 consecutive overview handoff e2e runs on Node 22.19.0, loaded machine (loadavg 7.4-11.0 on 8 cores, AGENT_BUNDLE_TEST_TIME_SCALE=4)
  • Node 24/26 not installed on this box (only v22.19.0 at /home/box/.local/node/current and system v20.19.2)

The overview e2e counted HMR frames exactly, so a late extra generation
from an at-least-once reload (or a hashless compile treated as unique)
failed 1-in-5 under load. Skip unidentifiable App hashes in the #73
channel and assert unique ordinals as a monotonic prefix instead.
@changeset-bot

changeset-bot Bot commented Sep 1, 2026

Copy link
Copy Markdown

⚠️ No Changeset found

Latest commit: 71547f3

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 1, 2026

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review Completed 2026-09-01T01:00:43.978527Z e1423bf PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: e1423bf887

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +106 to +107
const hash = stats.hash;
if (typeof hash !== 'string' || hash.length === 0) return;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Preserve reloads for successful hashless App builds

When Rsbuild reports a later successful App completion without a hash, this now returns before onAppReload(), even if that completion wrote changed App output. Hash absence prevents deduplication but does not establish that the build is unchanged, so under the same loaded conditions that produce hashless completions, the user-facing example can leave its iframe running stale assets indefinitely until another hashed App compilation occurs, contrary to this option's contract of notifying each later successful changed compilation. Retain a conservative reload or derive another reliable completion identity instead of silently dropping the event.

AGENTS.md reference: AGENTS.md:L10-L12

Useful? React with 👍 / 👎.

Unique-sorted ordinals are tautologically monotonic. Check the actual
runtime-app-reload sequence is non-decreasing and include SSE hub
sequence in the failure diagnostic.
A later successful App compile without stats.hash is unidentifiable, not
unchanged. Skipping onAppReload left the example iframe stale. Dedupe
hashed completions only; hashless success still notifies (at-least-once).
expect(runtimePreviewHmrMessages).toEqual(hmrMessagesBeforeConfigReconcile);
expectMonotonicReloadFrames();
const reloadOrdinals = ownedReloadOrdinals();
expect(reloadOrdinals.slice(0, reloadOrdinalsBeforeConfigReconcile.length)).toEqual(reloadOrdinalsBeforeConfigReconcile);

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P2] Reject reload ordinals minted during config reconciliation

This prefix assertion allows arbitrary new ordinals after the saved prefix. If the two initial watched-source edits coalesce into generation 1, then a spurious reload caused by the later finite, invalid, or repaired config write can become generation 2 and this helper still passes. That weakens the prior contract that those reconciliations retain the existing preview without emitting another App reload. First wait for the edit-triggered sequence to settle, snapshot it, and require the sequence to remain exactly unchanged throughout the config-reconcile phase (while still tolerating duplicate frames for already-seen ordinals).

ScriptedAlchemy added a commit that referenced this pull request Sep 1, 2026
…ity (ported from #114)

A hashless App completion fell back to the stats object as the dedupe key:
every such completion compared unequal (always minting) and clobbered the
retained hash, so the next unchanged hashed completion minted a spurious
frame too. Hashless success still reloads (unidentifiable is not unchanged),
but the retained hash survives it. Regression covers hashless/empty-hash
minting and the unchanged-hash-after-hashless dedupe.
@ScriptedAlchemy

Copy link
Copy Markdown
Owner Author

Reconciled with #120 (Wave 1 lane 0), which supersedes this PR for issue #111; closing per the owner's instruction (branch left intact).

Adopted from this PR, with credit: the runtimeAppReloadPlugin change and its regression coverage, ported into #120 (commit b60da26). Verified in the plugin harness on unmodified main that the defect is real: a hashless success falls back to stats object identity, so it always mints ([1,2,3][1,2,3,4]) and clobbers the retained hash, making the next unchanged hashed completion mint a spurious frame too (→ [1,2,3,4,5]). One correction to this PR's description: the fix does not stop hashless completions from minting (your own regression asserts they still reload, correctly — unidentifiable is not proven unchanged); what it fixes is the object-identity fallback and the retained-hash clobbering. #120 adds a regression for that clobbering case (unchanged hash after hashless stays deduped), verified to fail against the unfixed plugin.

Not adopted: the overview e2e assertion shape. #120's captured failing sequence (generation 0, 1, +2 with the compiler log showing the two watched-source writes splitting into two App compilations) shows the extra frame is a real new generation, so #120 bounds the maximum generation by the edit budget in addition to monotonic consumption — a config reconcile that reloads the retained App still fails once the edit generations are spent, which the unbounded prefix check here would let pass. #120 also carries the 20/20-per-line proof across four Node lines (22.19.0 / 22.23.1 / 24.19.0 / 26.8.1, load 37–92), covering the Node 24/26 gap noted in this PR's test plan.

ScriptedAlchemy added a commit that referenced this pull request Sep 1, 2026
…ity (ported from #114)

A hashless App completion fell back to the stats object as the dedupe key:
every such completion compared unequal (always minting) and clobbered the
retained hash, so the next unchanged hashed completion minted a spurious
frame too. Hashless success still reloads (unidentifiable is not unchanged),
but the retained hash survives it. Regression covers hashless/empty-hash
minting and the unchanged-hash-after-hashless dedupe.
ScriptedAlchemy added a commit that referenced this pull request Sep 1, 2026
…ls in the overview e2e (#120)

* fix(test): consume owned reload frames monotonically over generation ordinals

Two replaced watched sources legitimately compile as one coalesced or two
split App generations under load, so the overview e2e's exact frame-snapshot
equality raced the second frame. Assert monotonic, duplicate-tolerant,
budget-bounded generation consumption instead (issue #111, same shape as the
dev-provider fix from #20).

* fix(rsc-runtime-demo): never dedupe App reloads by stats object identity (ported from #114)

A hashless App completion fell back to the stats object as the dedupe key:
every such completion compared unequal (always minting) and clobbered the
retained hash, so the next unchanged hashed completion minted a spurious
frame too. Hashless success still reloads (unidentifiable is not unchanged),
but the retained hash survives it. Regression covers hashless/empty-hash
minting and the unchanged-hash-after-hashless dedupe.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

flake: overview.e2e sees an extra runtime-app-reload generation frame (1-in-5 on main, Node 26) — dominant local-ci gate tax

1 participant