Skip to content

fix: consume owned reload frames monotonically over generation ordinals in the overview e2e - #120

Merged
ScriptedAlchemy merged 4 commits into
mainfrom
wave1/111-reload-frame-flake
Sep 1, 2026
Merged

fix: consume owned reload frames monotonically over generation ordinals in the overview e2e#120
ScriptedAlchemy merged 4 commits into
mainfrom
wave1/111-reload-frame-flake

Conversation

@ScriptedAlchemy

@ScriptedAlchemy ScriptedAlchemy commented Sep 1, 2026

Copy link
Copy Markdown
Owner

Summary

overview.e2e.test.ts :: offers the host-owned MCP playground handoff... intermittently observed one extra runtime-app-reload generation frame (issue #111, reproduced first-run on unmodified main under Node 26 at load ~76).

Diagnosis: the channel behaves as designed for the observed flake; the test over-asserted (candidate 2 from the issue). The captured failing frame sequence:

- Expected
+ Received
  [
    "{\"generation\":0,\"kind\":\"runtime-app-reload\"}",
    "{\"generation\":1,\"kind\":\"runtime-app-reload\"}",
+   "{\"generation\":2,\"kind\":\"runtime-app-reload\"}",
  ]

The extra frame is not a duplicate — it is a new, strictly monotonic ordinal. The failing run's compiler log shows the two watched-source writes (src/widget/App.tsx + src/widget/styles.css, replaced via Promise.all) landing in two separate App compilations under load (start building src/widget/App.tsxstart building src/widget/styles.css) instead of coalescing into one. Each frame corresponded to a real changed App generation; the test's exact frame-array snapshot (hmrMessagesBeforeConfigReconcile + toEqual) raced the second legitimate frame. Deduping by generation ordinal in the channel would not have removed this frame — its ordinal is new.

Fixes

1. Test (the #111 flake): same shape PR #20 applied to dev-provider.integration.test.ts ("one source change may burn more than one generation ordinal … under load"): the assertion now consumes the owned channel monotonically over generation ordinals — connect replay is generation 0, duplicates tolerated, each advance is exactly one, and the maximum generation is bounded by the number of watched-source edits (2) — so a genuinely wrong generation (regression, gap, foreign traffic on the channel, or a config reconcile reloading the retained App once the edit budget is spent) still fails. Extended in place in the existing HMR/dev e2e; no parallel test file.

2. Plugin (secondary defect, ported from #114 during reconciliation): runtimeAppReloadPlugin deduped hashed completions correctly, but a hashless/empty-hash success fell back to stats object identity as the dedupe key: every such completion compared unequal (always minting) and clobbered the retained hash, so the next unchanged hashed completion also minted a spurious frame. Proven in the plugin harness on unmodified main: after [1,2,3], a hashless completion minted 4, then a repeat of the already-seen hash app-change-b minted 5. The fix keeps the retained hash across hashless completions; hashless success still reloads (unidentifiable is not proven unchanged — at-least-once, consumed monotonically by the e2e). Regression extended in dev-provider.integration.test.ts, including the unchanged-hash-after-hashless dedupe case, verified to fail against the unfixed plugin. Note this was NOT the mechanism of the captured #111 flake (those frames were real changed compiles) — it is a latent correctness fix at the same seam.

Changeset

None: the workbench change is test-only, and the plugin fix lives in @agent-bundle/rsc-agent-runtime-demo, which is private: true.changeset/config.json sets privatePackages.version: false, so private packages are not versioned or published by the Changesets workflow.

Secondary flakes from the issue

  • mcp-app-real.e2e.test.ts does not share the mechanism: it drives the reload channel through mocked Playwright WebSocket routes (synthetic frames) rather than counting real provider frames — left untouched.
  • dev-watcher.test.ts does not share the mechanism: its flake is chokidar coalescing a create event at the filesystem-watcher layer, not reload-channel frame accounting — left untouched.

Proof: 20 consecutive runs per Node line, loaded machine (1-min load 37–92 throughout)

Node line Result
22.19.0 (mise, hosted 22 line) 20/20 pass
22.23.1 (default PATH) 20/20 pass
24.19.0 (mise) 20/20 pass
26.8.1 (mise) 20/20 pass

Baseline on unmodified main under Node 26 failed on run 1 with the frame sequence above.

Fixes #111

@changeset-bot

changeset-bot Bot commented Sep 1, 2026

Copy link
Copy Markdown

⚠️ No Changeset found

Latest commit: 6f2e9bb

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 1, 2026

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review Completed 2026-09-01T01:18:31.111587Z b2616a3 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: b2616a3133

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread packages/workbench/tests/overview.e2e.test.ts
…ordinals

Two replaced watched sources legitimately compile as one coalesced or two
split App generations under load, so the overview e2e's exact frame-snapshot
equality raced the second frame. Assert monotonic, duplicate-tolerant,
budget-bounded generation consumption instead (issue #111, same shape as the
dev-provider fix from #20).
…ity (ported from #114)

A hashless App completion fell back to the stats object as the dedupe key:
every such completion compared unequal (always minting) and clobbered the
retained hash, so the next unchanged hashed completion minted a spurious
frame too. Hashless success still reloads (unidentifiable is not unchanged),
but the retained hash survives it. Regression covers hashless/empty-hash
minting and the unchanged-hash-after-hashless dedupe.
@pkg-pr-new

pkg-pr-new Bot commented Sep 1, 2026

Copy link
Copy Markdown
npm i https://pkg.pr.new/ScriptedAlchemy/agent-bundle@120
npm i https://pkg.pr.new/ScriptedAlchemy/agent-bundle/create-agent-bundle@120
npm i https://pkg.pr.new/ScriptedAlchemy/agent-bundle/@agent-bundle/rsc-runtime@120

commit: bc2a325

@ScriptedAlchemy
ScriptedAlchemy merged commit 69546b2 into main Sep 1, 2026
6 of 9 checks passed
@ScriptedAlchemy
ScriptedAlchemy deleted the wave1/111-reload-frame-flake branch September 1, 2026 05:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

flake: overview.e2e sees an extra runtime-app-reload generation frame (1-in-5 on main, Node 26) — dominant local-ci gate tax

1 participant