diff --git a/.changeset/README.md b/.changeset/README.md index efaf0773f..63925c4b9 100644 --- a/.changeset/README.md +++ b/.changeset/README.md @@ -13,6 +13,7 @@ versions; `changeset publish` (when enabled) publishes the result. | ----------------------- | ------------------------------ | ---------------------------------------- | | `agent-bundle` | `packages/agent-bundle` | publishable | | `@agent-bundle/runtime` | `packages/rsc-runtime` | publishable | +| `rsc-markdown-stream` | `packages/rsc-markdown-stream` | publishable | | `create-agent-bundle` | `packages/create-agent-bundle` | publishable | | `agent-bundle-workbench`| `packages/workbench` | private, ignored | | `@agent-bundle-example/*`, `@agent-bundle/rsc-agent-runtime-demo` | `examples/*` | private, ignored | diff --git a/.changeset/jsx-markdown-content.md b/.changeset/jsx-markdown-content.md new file mode 100644 index 000000000..320543832 --- /dev/null +++ b/.changeset/jsx-markdown-content.md @@ -0,0 +1,19 @@ +--- +"@agent-bundle/runtime": patch +"rsc-markdown-stream": patch +"agent-bundle": patch +--- + +Add the async `MarkdownContent` component and the `renderToMarkdown` / +`renderToMarkdownStream` exports to `@agent-bundle/runtime`, so routes author +rich Markdown blocks — headings, lists, GFM tables, task lists, nested async +components, escaped text — as JSX lowered into `Agent.Markdown` instead of +hand-concatenated strings. The renderer behind them, `rsc-markdown-stream`, is +now a package of this repository and is published from it (it was previously +only installable from its git URL), so `@agent-bundle/runtime` depends on it +by version. `agent-bundle build` now follows symlinked (workspace) dependencies +transitively when attributing bundle provenance, resolving each one the way +Node does, so a project whose linked dependency links another package — +including one hoisted to an ancestor `node_modules` — no longer fails with +`AB5000`, and a dependency that links back onto the project never hides the +project's own sources from provenance. (#344) diff --git a/AGENTS.md b/AGENTS.md index aaa20e56e..df277720d 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -95,8 +95,9 @@ ## Changesets - Every PR that changes a publishable package (`packages/agent-bundle`, - `packages/rsc-runtime`, `packages/create-agent-bundle` — anything except - `tests/**`) must include exactly one changeset: `pnpm changeset` or a + `packages/rsc-runtime`, `packages/rsc-markdown-stream`, + `packages/create-agent-bundle` — anything except `tests/**`) must include + exactly one changeset: `pnpm changeset` or a hand-written `.changeset/.md`. Private packages (`packages/workbench`, `examples/*`, `website`) are ignored and never named in a changeset. - Pre-1.0 semver: `minor` = breaking, `patch` = everything else (features diff --git a/NOTICE b/NOTICE index 5c429f6cd..a78e8e84f 100644 --- a/NOTICE +++ b/NOTICE @@ -12,6 +12,12 @@ license and notices, which are preserved unmodified alongside that material: src/mcp/APP-RENDERER-LICENSE, which the agent-bundle package ships under dist/workbench/. + - The rsc-markdown-stream package (packages/rsc-markdown-stream) was + imported from https://github.com/ScriptedAlchemy/rsc-markdown-stream at + commit eba2ea0b930493b80b9f4f9bb2c582041b0a3f47, where it was + distributed under the MIT License. That notice is preserved as + UPSTREAM-LICENSE in the package and its published tarball. + The repository's vendored reference checkouts under repos/ are read-only reference material, retain their own upstream licenses, and are not part of any published package. diff --git a/docs/preview-packages.md b/docs/preview-packages.md index 9e9968367..9f778a58a 100644 --- a/docs/preview-packages.md +++ b/docs/preview-packages.md @@ -11,7 +11,9 @@ release owner must resolve the repository-wide `"access": "restricted"` policy for `agent-bundle`, which does not currently override it with `publishConfig.access`. Until then pkg.pr.new is the release channel. Every CI package-preview run publishes real, -installable tarballs of all three publishable workspace packages to [pkg.pr.new](https://pkg.pr.new) +installable tarballs of all four publishable workspace packages (`agent-bundle`, +`@agent-bundle/runtime`, `rsc-markdown-stream`, `create-agent-bundle`) to +[pkg.pr.new](https://pkg.pr.new) — a free continuous-release registry keyed by commit SHA and pull request. These are the packages to install until a first npm release is cut. @@ -24,6 +26,15 @@ npm i https://pkg.pr.new/ScriptedAlchemy/agent-bundle/agent-bundle@1 npm i https://pkg.pr.new/ScriptedAlchemy/agent-bundle/@agent-bundle/runtime@1 ``` +`@agent-bundle/runtime` depends on `rsc-markdown-stream`, the Markdown +renderer behind `MarkdownContent`; its preview tarball points that dependency +at the renderer's own preview of the same commit, so npm fetches it without a +separate install. Install the renderer directly only to use it on its own: + +```sh +npm i https://pkg.pr.new/ScriptedAlchemy/agent-bundle/rsc-markdown-stream@1 +``` + The `create-agent-bundle` scaffolder is published to the same channel and is meant to be run rather than installed: @@ -46,6 +57,7 @@ work), which is the right form for lockfiles and reproducible setups: ```sh npm i https://pkg.pr.new/ScriptedAlchemy/agent-bundle/agent-bundle@5685521 npm i https://pkg.pr.new/ScriptedAlchemy/agent-bundle/@agent-bundle/runtime@5685521 +npm i https://pkg.pr.new/ScriptedAlchemy/agent-bundle/rsc-markdown-stream@5685521 ``` pnpm and yarn accept the same URLs (`pnpm add `, `yarn add agent-bundle@`). @@ -55,8 +67,11 @@ Previews carry the version string `0.0.0-preview-`, and the publish package to that exact preview version inside the preview tarballs. Today that is the optional `@agent-bundle/runtime` peer declared by `agent-bundle` (`@agent-bundle/runtime` itself no longer declares an `agent-bundle` peer; -its peers are `react`, `react-dom`, and `@rspack/core`). Installing both -packages from the same sha therefore works with stock npm — no +its peers are `react`, `react-dom`, and `@rspack/core`). A regular +`dependencies` entry that names a sibling workspace package is rewritten to +that sibling's same-sha tarball URL: `@agent-bundle/runtime`'s +`rsc-markdown-stream` dependency resolves to the renderer preview of the same +commit. Installing both packages from the same sha therefore works with stock npm — no `--legacy-peer-deps` needed. Mixing two different shas fails with `ERESOLVE` by design; use one sha (or one PR number) for both URLs. Previews published before the peer rewrite landed (PR #46, fixing #45) still carry the original @@ -83,7 +98,7 @@ installable artifacts. `.github/workflows/package-preview.yml` runs `pnpm preview:publish` (`pkg-pr-new publish --previewVersion --peerDeps --no-compact --no-template './packages/agent-bundle' './packages/rsc-runtime' -'./packages/create-agent-bundle'`) +'./packages/rsc-markdown-stream' './packages/create-agent-bundle'`) after a full build, on every pull request and on every push to `main`. Runs for `main` pushes use a per-commit concurrency group, so overlapping pushes cannot cancel one another and every `main` commit has an installable diff --git a/examples/audiobook-curator/src/components/library-analysis.tsx b/examples/audiobook-curator/src/components/library-analysis.tsx index abbcc10b5..4a6618cd4 100644 --- a/examples/audiobook-curator/src/components/library-analysis.tsx +++ b/examples/audiobook-curator/src/components/library-analysis.tsx @@ -1,6 +1,6 @@ import { stat } from 'node:fs/promises'; -import { Agent } from '@agent-bundle/runtime'; +import { Agent, MarkdownContent } from '@agent-bundle/runtime'; import React from 'react'; import type { LibraryAuditReceipt } from '../library.ts'; @@ -21,6 +21,26 @@ interface MeasuredFile { const errorMessage = (error: unknown): string => error instanceof Error ? error.message : 'File metadata is unavailable.'; +/** JSX-authored GFM table lowered to Markdown by the runtime's renderer. */ +const MeasuredFilesTable = ({ measured }: { readonly measured: readonly MeasuredFile[] }) => ( + + + + + + + {measured.map((file) => ( + + + + + + ))} + +
FileBytesStatus
{file.path}{file.bytes === undefined ? '' : String(file.bytes)}{file.error ?? 'measured'}
+ +); + const measureFiles = async ( files: readonly string[], signal: AbortSignal, @@ -62,6 +82,7 @@ export const LibraryAnalysis = async ({ receipt, signal }: LibraryAnalysisProps) { label: 'Measured files', value: available.length }, { label: 'Reclaimable bytes', value: reclaimableBytes }, ]} /> + {unavailable.length > 0 ? ( <> diff --git a/examples/audiobook-curator/src/components/primitives.tsx b/examples/audiobook-curator/src/components/primitives.tsx index 0d5922eaa..a4ee01a1e 100644 --- a/examples/audiobook-curator/src/components/primitives.tsx +++ b/examples/audiobook-curator/src/components/primitives.tsx @@ -1,4 +1,4 @@ -import { Agent } from '@agent-bundle/runtime'; +import { Agent, MarkdownContent } from '@agent-bundle/runtime'; import React from 'react'; export interface Field { @@ -10,10 +10,18 @@ export interface DataListProps { readonly fields: readonly Field[]; } +const singleLine = (value: Field['value']): string => String(value).replaceAll(/\s*\n\s*/gu, ' '); + export const DataList = ({ fields }: DataListProps) => ( - - {fields.map(({ label, value }) => `- **${label}:** ${String(value).replaceAll(/\s*\n\s*/gu, ' ')}`).join('\n')} - + +
    + {fields.map(({ label, value }) => ( +
  • + {label}: {singleLine(value)} +
    • + ))} +
+ ); export interface FileListProps { @@ -21,7 +29,11 @@ export interface FileListProps { } export const FileList = ({ files }: FileListProps) => ( - {files.map((file) => `- ${file}`).join('\n')} + +
    + {files.map((file) =>
  • {file}
    • )} +
+ ); export interface CalloutProps { diff --git a/examples/audiobook-curator/tests/route-unit/streaming.test.ts b/examples/audiobook-curator/tests/route-unit/streaming.test.ts index 5211e6844..d42e34526 100644 --- a/examples/audiobook-curator/tests/route-unit/streaming.test.ts +++ b/examples/audiobook-curator/tests/route-unit/streaming.test.ts @@ -44,6 +44,10 @@ it('streams library analysis after the audit shell while preserving the canonica && documentText(document).includes('"kind":"progress"'))).toBe(true); expectDocument(rendered) .toContainMarkdown('**Reclaimable bytes:** 4') + // The JSX-authored measured-files table lowers to one GFM table block. + .toContainMarkdown('| File | Bytes | Status |') + .toContainMarkdown(`| ${join(library, 'Shared title.flac')} | 15 | measured |`) + .toContainMarkdown(`| ${join(library, 'Shared title.mp3')} | 4 | measured |`) .toContainMarkdown(`- ${join(library, 'Shared title.flac')}\n- ${join(library, 'Shared title.mp3')}`) .toContainContext('Duplicate candidate group') .toHaveValue(rendered.result); diff --git a/package.json b/package.json index 332483433..36d6b6c22 100644 --- a/package.json +++ b/package.json @@ -9,7 +9,7 @@ }, "packageManager": "pnpm@11.23.0", "scripts": { - "build": "pnpm --filter @agent-bundle/runtime build && pnpm --filter agent-bundle build && pnpm --filter create-agent-bundle build", + "build": "pnpm --filter rsc-markdown-stream build && pnpm --filter @agent-bundle/runtime build && pnpm --filter agent-bundle build && pnpm --filter create-agent-bundle build", "test": "pnpm test:unit && pnpm test:route-unit && pnpm test:projection && pnpm test:integration", "test:unit": "rstest --config rstest.unit.config.ts", "test:route-unit": "rstest --config rstest.route-unit.config.ts", @@ -21,7 +21,7 @@ "test:watch": "rstest --config rstest.config.ts --watch", "lint": "rslint .", "bench:hook-cold-start": "node scripts/measure-hook-cold-start.mjs", - "typecheck": "tsc --noEmit && tsc --project packages/workbench/tsconfig.json && tsc --project packages/create-agent-bundle/tsconfig.json", + "typecheck": "tsc --noEmit && tsc --project packages/workbench/tsconfig.json && tsc --project packages/create-agent-bundle/tsconfig.json && tsc --project packages/rsc-markdown-stream/tsconfig.json", "check": "pnpm build && pnpm test:unit && pnpm test:route-unit && pnpm test:projection && pnpm test:integration:run && pnpm lint && pnpm typecheck", "check:local-ci": "node scripts/local-ci.mjs", "check:host-cli": "node scripts/host-cli-pins.mjs verify", @@ -46,7 +46,7 @@ "changeset": "changeset", "version-packages": "changeset version", "release": "pnpm check:release && changeset publish", - "preview:publish": "pkg-pr-new publish --previewVersion --peerDeps --no-compact --no-template './packages/agent-bundle' './packages/rsc-runtime' './packages/create-agent-bundle'", + "preview:publish": "pkg-pr-new publish --previewVersion --peerDeps --no-compact --no-template './packages/agent-bundle' './packages/rsc-runtime' './packages/rsc-markdown-stream' './packages/create-agent-bundle'", "pack:dry-run": "pnpm build && npm pack ./packages/agent-bundle --dry-run --json", "lint:release": "attw --pack --profile esm-only packages/agent-bundle", "check:release": "pnpm pack:dry-run && pnpm lint:release && pnpm test:packed:release", diff --git a/packages/agent-bundle/src/build/rslib.ts b/packages/agent-bundle/src/build/rslib.ts index ec8b6663a..1a3638447 100644 --- a/packages/agent-bundle/src/build/rslib.ts +++ b/packages/agent-bundle/src/build/rslib.ts @@ -4,10 +4,12 @@ import { pluginReact } from '@rsbuild/plugin-react'; import { createRslib, mergeRslibConfig, rspack, type LibConfig, type Rspack } from '@rslib/core'; import { readFile, realpath } from 'node:fs/promises'; -import { join, resolve } from 'node:path'; +import { createRequire } from 'node:module'; +import { dirname, join, resolve, sep } from 'node:path'; import { sha256Hex } from '../core/digest.ts'; import { isErrno } from '../core/errors.ts'; +import { exists } from '../core/paths.ts'; import { isRecord } from '../core/strict-json.ts'; import type { AgentBundleToolsConfig } from '../core/types.ts'; import type { AgentBundleMeta } from '../meta.ts'; @@ -290,31 +292,107 @@ const assertNoResidualReservedImports = async ( })); }; -const declaredDependencyRoots = async (cwd: string): Promise => { - let bytes: string; +const projectDependencyFields = ['dependencies', 'devDependencies', 'optionalDependencies', 'peerDependencies'] as const; +/** What a dependency's bundle can pull in: its devDependencies never ship. */ +const runtimeDependencyFields = ['dependencies', 'optionalDependencies', 'peerDependencies'] as const; + +const declaredDependencyNames = (manifest: Record, fields: readonly string[]): readonly string[] => { + const names = new Set(); + for (const field of fields) { + const dependencies = manifest[field]; + if (dependencies === null || typeof dependencies !== 'object' || Array.isArray(dependencies)) continue; + for (const name of Object.keys(dependencies)) { + if (/^(?:@[a-z0-9._-]+\/)?[a-z0-9._-]+$/iu.test(name)) names.add(name); + } + } + return [...names]; +}; + +const readManifest = async (packageRoot: string): Promise | undefined> => { try { - bytes = await readFile(resolve(cwd, 'package.json'), 'utf8'); + return JSON.parse(await readFile(resolve(packageRoot, 'package.json'), 'utf8')) as Record; } catch (error) { - if (isErrno(error, 'ENOENT')) return Object.freeze([]); + if (isErrno(error, 'ENOENT')) return undefined; throw error; } - const manifest = JSON.parse(bytes) as Record; - const names = new Set(); - for (const field of ['dependencies', 'devDependencies', 'optionalDependencies', 'peerDependencies']) { - const dependencies = manifest[field]; - if (dependencies === null || typeof dependencies !== 'object' || Array.isArray(dependencies)) continue; - for (const name of Object.keys(dependencies)) names.add(name); +}; + +const isBeneathNodeModules = (path: string): boolean => path.split(sep).includes('node_modules'); + +/** + * The manifest of dependency `name` as Node resolves it from `packageRoot`, + * which honours hoisting: npm, Yarn, and pnpm with a hoist pattern place a + * workspace dependency in an ancestor `node_modules`, where Rspack finds it + * too. A package whose `exports` map hides `package.json` makes that lookup + * throw, so the same ancestor walk is then performed by hand. + */ +const dependencyManifestPath = async (packageRoot: string, name: string): Promise => { + try { + return createRequire(join(packageRoot, 'package.json')).resolve(`${name}/package.json`); + } catch (error) { + if (isErrno(error, 'MODULE_NOT_FOUND')) return undefined; + if (!isErrno(error, 'ERR_PACKAGE_PATH_NOT_EXPORTED')) throw error; } - const roots = await Promise.all([...names].map(async (name) => { - if (!/^(?:@[a-z0-9._-]+\/)?[a-z0-9._-]+$/iu.test(name)) return undefined; - try { - return await realpath(resolve(cwd, 'node_modules', ...name.split('/'))); - } catch (error) { - if ((error as NodeJS.ErrnoException).code === 'ENOENT') return undefined; - throw error; - } - })); - return Object.freeze(roots.filter((root): root is string => root !== undefined)); + let directory = packageRoot; + while (true) { + const candidate = join(directory, 'node_modules', ...name.split('/'), 'package.json'); + if (await exists(candidate)) return candidate; + const parent = dirname(directory); + if (parent === directory) return undefined; + directory = parent; + } +}; + +/** + * The project root as Rspack records it, so a dependency link back onto the + * project compares equal. A directory that does not exist has no manifest and + * therefore no dependency roots; it is kept as given rather than failing here. + */ +const canonicalProjectRoot = async (cwd: string): Promise => { + const root = resolve(cwd); + try { + return await realpath(root); + } catch (error) { + if (isErrno(error, 'ENOENT')) return root; + throw error; + } +}; + +/** + * Real roots of the project's declared dependencies, followed transitively + * through each linked dependency's own runtime dependencies. Provenance already + * discards modules beneath a `node_modules` directory, but pnpm links workspace + * packages by symlink and Rspack records their modules at real paths, which + * carry no such segment: `@agent-bundle/runtime` resolved to + * `packages/rsc-runtime` must be excluded by root, and so must the workspace + * packages *it* depends on (`rsc-markdown-stream`), which the project never + * declares itself. Registry packages resolve beneath `node_modules`, so their + * trees are never walked. The project itself is never a root: a dependency + * cycle back onto it (A → B → A) must not turn every authored module into an + * ignored one. Only the root itself is exempt — a dependency linked from + * inside the project (`/packages/dep`, `file:./vendor/dep`) is still + * a dependency and is excluded like any other. + */ +const declaredDependencyRoots = async (cwd: string): Promise => { + const projectRoot = await canonicalProjectRoot(cwd); + const roots = new Set(); + const visited = new Set(); + const visit = async (packageRoot: string, fields: readonly string[]): Promise => { + if (visited.has(packageRoot)) return; + visited.add(packageRoot); + const manifest = await readManifest(packageRoot); + if (manifest === undefined) return; + await Promise.all(declaredDependencyNames(manifest, fields).map(async (name) => { + const manifestPath = await dependencyManifestPath(packageRoot, name); + if (manifestPath === undefined) return; + const root = await realpath(dirname(manifestPath)); + if (root === projectRoot) return; + roots.add(root); + if (!isBeneathNodeModules(root)) await visit(root, runtimeDependencyFields); + })); + }; + await visit(projectRoot, projectDependencyFields); + return Object.freeze([...roots].sort((left, right) => left.localeCompare(right))); }; interface InspectedBundlerConfig { diff --git a/packages/agent-bundle/tests/build.test.ts b/packages/agent-bundle/tests/build.test.ts index 4f9401016..ceec3224a 100644 --- a/packages/agent-bundle/tests/build.test.ts +++ b/packages/agent-bundle/tests/build.test.ts @@ -1,5 +1,5 @@ import { supportedCapabilities } from './support/adapter-capabilities.ts'; -import { chmod, mkdtemp, mkdir, readFile, readdir, rename, rm, stat, writeFile } from 'node:fs/promises'; +import { chmod, mkdtemp, mkdir, readFile, readdir, rename, rm, stat, symlink, writeFile } from 'node:fs/promises'; import { tmpdir } from 'node:os'; import { dirname, join, relative } from 'node:path'; import { fileURLToPath, pathToFileURL } from 'node:url'; @@ -1162,6 +1162,137 @@ it('inlines reserved specifiers through exact-match aliases and virtual generate } }, 20_000); +/** + * A workspace layout as a package manager lays it out: the project's dependency + * `linked-a` is a symlink to a sibling package, and `linked-a`'s own dependency + * `linked-b` is another symlink. Rspack records both at their real paths, which + * carry no `node_modules` segment, so provenance must exclude them by root — + * including the transitive one the project never declares + * (`@agent-bundle/runtime` → `rsc-markdown-stream` in this repository). + * + * `hoisted` places the `linked-b` link in the workspace-root `node_modules`, + * as npm, Yarn, and a hoisting pnpm do, rather than beneath `linked-a`. + * `cyclic` gives `linked-b` a dependency back onto the project itself. + * `nested` keeps both linked packages inside the project directory, as a + * root package linking `/packages/*` or a `file:./vendor/dep` does. + */ +interface LinkedWorkspaceLayout { + readonly cyclic?: boolean; + readonly hoisted?: boolean; + readonly nested?: boolean; +} + +const linkedWorkspaceProject = async ( + layout: LinkedWorkspaceLayout = {}, +): Promise<{ readonly entry: RslibEntry; readonly root: string }> => { + const parent = await mkdtemp(join(tmpdir(), 'agent-bundle-linked-workspace-')); + const root = join(parent, 'project'); + const packages = layout.nested === true ? join(root, 'packages') : join(parent, 'packages'); + const linkedA = join(packages, 'linked-a'); + const linkedB = join(packages, 'linked-b'); + const linkedBLink = layout.hoisted === true + ? join(parent, 'node_modules', 'linked-b') + : join(linkedA, 'node_modules', 'linked-b'); + await Promise.all([ + mkdir(join(root, 'src'), { recursive: true }), + mkdir(join(root, 'node_modules'), { recursive: true }), + mkdir(linkedA, { recursive: true }), + mkdir(dirname(linkedBLink), { recursive: true }), + mkdir(join(linkedB, 'node_modules'), { recursive: true }), + ]); + await Promise.all([ + writeFile(join(root, 'package.json'), '{"name":"project","type":"module","dependencies":{"linked-a":"workspace:*"}}\n'), + writeFile(join(root, 'src', 'entry.ts'), "import { a } from 'linked-a';\nimport { local } from './local.ts';\nconsole.log(a, local);\n"), + writeFile(join(root, 'src', 'local.ts'), "export const local = 'local-marker';\n"), + // `linked-a` exposes its manifest through `exports`, so Node's resolver + // finds it; `linked-b` hides it, exercising the ancestor `node_modules` walk. + writeFile( + join(linkedA, 'package.json'), + '{"name":"linked-a","type":"module","exports":{".":"./index.js","./package.json":"./package.json"},"dependencies":{"linked-b":"workspace:*"}}\n', + ), + writeFile(join(linkedA, 'index.js'), "import { b } from 'linked-b';\nexport const a = `a:${b}`;\n"), + writeFile( + join(linkedB, 'package.json'), + `{"name":"linked-b","type":"module","exports":"./index.js"${layout.cyclic === true ? ',"dependencies":{"project":"workspace:*"}' : ''}}\n`, + ), + writeFile(join(linkedB, 'index.js'), "export const b = 'linked-b-marker';\n"), + ]); + await Promise.all([ + symlink(linkedA, join(root, 'node_modules', 'linked-a'), 'dir'), + symlink(linkedB, linkedBLink, 'dir'), + ...(layout.cyclic === true ? [symlink(root, join(linkedB, 'node_modules', 'project'), 'dir')] : []), + ]); + return { + entry: { + name: 'linked', + outputRelativePath: 'scripts/linked.mjs', + source: join(root, 'src', 'entry.ts'), + sourceInputs: [join(root, 'src', 'entry.ts')], + }, + root: parent, + }; +}; + +/** Builds the linked-workspace project and returns its bundle text and evidence. */ +const buildLinkedWorkspaceProject = async ( + layout: LinkedWorkspaceLayout = {}, +): Promise<{ readonly bundle: string; readonly evidence: Awaited>; readonly root: string }> => { + const { entry, root: parent } = await linkedWorkspaceProject(layout); + const root = join(parent, 'project'); + try { + const evidence = await buildWithRslib({ + cwd: root, + entries: [entry], + meta: testMeta, + outputRoot: join(root, 'dist'), + }); + const bundle = await readFile(join(root, 'dist', 'scripts', 'linked.mjs'), 'utf8'); + return { bundle, evidence, root }; + } finally { + await rm(parent, { force: true, recursive: true }); + } +}; + +const linkedWorkspaceSourceInputs = (root: string): readonly string[] => [ + join(root, 'src', 'entry.ts'), + join(root, 'src', 'local.ts'), +]; + +it('bundles symlinked workspace dependencies, transitively, without attributing them as project sources', async () => { + const { bundle, evidence, root } = await buildLinkedWorkspaceProject(); + expect(bundle).toContain('linked-b-marker'); + expect(evidence).toEqual([{ path: 'scripts/linked.mjs', sourceInputs: linkedWorkspaceSourceInputs(root) }]); +}, 20_000); + +it('excludes a transitive workspace dependency hoisted to an ancestor node_modules', async () => { + // npm, Yarn, and pnpm with a hoist pattern place `linked-b` beside the + // workspace root rather than beneath `linked-a`; Node and Rspack resolve it + // from there, so the provenance walk must resolve the same way instead of + // probing only `linked-a/node_modules`. + const { bundle, evidence, root } = await buildLinkedWorkspaceProject({ hoisted: true }); + expect(bundle).toContain('linked-b-marker'); + expect(evidence).toEqual([{ path: 'scripts/linked.mjs', sourceInputs: linkedWorkspaceSourceInputs(root) }]); +}, 20_000); + +it('keeps the project sources when a linked dependency depends back on the project', async () => { + // project → linked-a → linked-b → project: resolving the cycle must never + // register the project root as an ignored dependency root, or every module + // the project authored would silently vanish from provenance. + const { bundle, evidence, root } = await buildLinkedWorkspaceProject({ cyclic: true }); + expect(bundle).toContain('local-marker'); + expect(evidence).toEqual([{ path: 'scripts/linked.mjs', sourceInputs: linkedWorkspaceSourceInputs(root) }]); +}, 20_000); + +it('excludes linked dependencies that live inside the project directory', async () => { + // A root package linking `/packages/*` (or a `file:./vendor/dep` + // dependency) resolves to a directory beneath the project. Only the project + // root itself is exempt from the ignored roots; a dependency nested inside + // it is still a dependency, not authored source. + const { bundle, evidence, root } = await buildLinkedWorkspaceProject({ nested: true }); + expect(bundle).toContain('linked-b-marker'); + expect(evidence).toEqual([{ path: 'scripts/linked.mjs', sourceInputs: linkedWorkspaceSourceInputs(root) }]); +}, 20_000); + it('parses emitted bundles in full when a tools hatch could have rewritten them', async () => { // A compiler bundle is trusted to the ESM lexer only while its bytes are the // bundler's own. A hatch runs after Rspack parsed the source and can rewrite diff --git a/packages/agent-bundle/tests/license-metadata.test.ts b/packages/agent-bundle/tests/license-metadata.test.ts index df19be8cd..10373e590 100644 --- a/packages/agent-bundle/tests/license-metadata.test.ts +++ b/packages/agent-bundle/tests/license-metadata.test.ts @@ -8,7 +8,7 @@ const workspaceRoot = process.cwd(); const projectLicense = 'Apache-2.0'; /** SHA-256 of https://www.apache.org/licenses/LICENSE-2.0.txt (canonical text; .gitattributes pins LF endings). */ const canonicalApache2Sha256 = 'cfc7749b96f63bd31c3c42b5c471bf756814053e847c10f3eb003417bc523d30'; -const publishablePackages = ['agent-bundle', 'rsc-runtime', 'create-agent-bundle'] as const; +const publishablePackages = ['agent-bundle', 'rsc-runtime', 'rsc-markdown-stream', 'create-agent-bundle'] as const; interface Manifest { readonly files?: readonly string[]; diff --git a/packages/agent-bundle/tests/packed-stdio-projection.test.ts b/packages/agent-bundle/tests/packed-stdio-projection.test.ts index 74ee435db..7a0042ae6 100644 --- a/packages/agent-bundle/tests/packed-stdio-projection.test.ts +++ b/packages/agent-bundle/tests/packed-stdio-projection.test.ts @@ -45,9 +45,10 @@ interface McpJson { * the cost and explicitly does not claim any of this. */ it('serves compiled routes and durable state across packed process restarts', async () => { - const [agentBundle, runtime] = await Promise.all([ + const [agentBundle, runtime, markdownStream] = await Promise.all([ sharedPackedTarball('agent-bundle'), sharedPackedTarball('runtime'), + sharedPackedTarball('markdown-stream'), ]); const consumer = await mkdtemp(join(tmpdir(), 'agent-bundle-packed-stdio-')); const project = join(consumer, 'project'); @@ -79,6 +80,7 @@ it('serves compiled routes and durable state across packed process restarts', as await execFile('npm', ['install', ...cachedNpmInstallArguments, agentBundle.tarball, runtime.tarball, + markdownStream.tarball, 'react@19.2.8', 'react-dom@19.2.8', 'zod@4.4.3', diff --git a/packages/agent-bundle/tests/support/shared-pack.ts b/packages/agent-bundle/tests/support/shared-pack.ts index 3946dfcf5..372d6a094 100644 --- a/packages/agent-bundle/tests/support/shared-pack.ts +++ b/packages/agent-bundle/tests/support/shared-pack.ts @@ -23,12 +23,16 @@ export interface SharedPack { readonly tarball: string; } -export type SharedPackPackage = 'agent-bundle' | 'create-agent-bundle' | 'runtime'; +export type SharedPackPackage = 'agent-bundle' | 'create-agent-bundle' | 'markdown-stream' | 'runtime'; /** packages/ directory and npm package name for each shared-pack key. */ const sharedPackPackages: Readonly>> = { 'agent-bundle': { directory: 'agent-bundle', npmName: 'agent-bundle' }, 'create-agent-bundle': { directory: 'create-agent-bundle', npmName: 'create-agent-bundle' }, + // `@agent-bundle/runtime` depends on it by exact version; a consumer that + // installs the runtime tarball needs this one alongside until that version + // is on the registry (npm dedupes the runtime's edge onto it). + 'markdown-stream': { directory: 'rsc-markdown-stream', npmName: 'rsc-markdown-stream' }, runtime: { directory: 'rsc-runtime', npmName: '@agent-bundle/runtime' }, }; diff --git a/packages/create-agent-bundle/tests/scaffold-packed-matrix.e2e.test.ts b/packages/create-agent-bundle/tests/scaffold-packed-matrix.e2e.test.ts index d1a9f3130..b425bd3c6 100644 --- a/packages/create-agent-bundle/tests/scaffold-packed-matrix.e2e.test.ts +++ b/packages/create-agent-bundle/tests/scaffold-packed-matrix.e2e.test.ts @@ -7,8 +7,14 @@ import { promisify } from 'node:util'; import { afterAll, expect, it } from '@rstest/core'; -import { installedEnvironment, npmInstallArguments, packOutputFromJson } from '../../agent-bundle/tests/support/shared-pack.ts'; -import { cleanupScaffoldFixture, expectCleanValidate, npmRun, scaffoldProject } from './support/scaffold-fixture.ts'; +import { installedEnvironment, packOutputFromJson } from '../../agent-bundle/tests/support/shared-pack.ts'; +import { + cleanupScaffoldFixture, + expectCleanValidate, + installScaffoldedProject, + npmRun, + scaffoldProject, +} from './support/scaffold-fixture.ts'; const execFile = promisify(executeFile); @@ -26,10 +32,7 @@ afterAll(cleanupScaffoldFixture); */ it.concurrent('scaffolds the mcp-server template and serves the conventional entry from the artifact', async () => { const projectRoot = await scaffoldProject('mcp-server', 'status-plugin', ['--no-install']); - await execFile('npm', ['install', ...npmInstallArguments], { - cwd: projectRoot, - env: installedEnvironment(), - }); + await installScaffoldedProject(projectRoot); const checked = await npmRun(projectRoot, 'check'); await expectCleanValidate(projectRoot); @@ -76,10 +79,7 @@ it.concurrent('scaffolds the mcp-server template and serves the conventional ent it.concurrent('scaffolds the cli-tool template with a routed bin, lib, and artifact script', async () => { const projectRoot = await scaffoldProject('cli-tool', 'greeter', ['--no-install']); - await execFile('npm', ['install', ...npmInstallArguments], { - cwd: projectRoot, - env: installedEnvironment(), - }); + await installScaffoldedProject(projectRoot); // The template's own harness pool ran inside `check`, and it is asserted // positively — a silent `check` would also pass if the pool were dropped or diff --git a/packages/create-agent-bundle/tests/support/scaffold-fixture.ts b/packages/create-agent-bundle/tests/support/scaffold-fixture.ts index b6b571d7c..af0ef6fdd 100644 --- a/packages/create-agent-bundle/tests/support/scaffold-fixture.ts +++ b/packages/create-agent-bundle/tests/support/scaffold-fixture.ts @@ -6,12 +6,18 @@ import { promisify } from 'node:util'; import { expect } from '@rstest/core'; -import { cachedNpmInstallArguments, installedEnvironment, sharedPackedTarball } from '../../../agent-bundle/tests/support/shared-pack.ts'; +import { + cachedNpmInstallArguments, + installedEnvironment, + npmInstallArguments, + sharedPackedTarball, +} from '../../../agent-bundle/tests/support/shared-pack.ts'; const execFile = promisify(executeFile); interface PackedFixture { readonly frameworkTarball: string; + readonly markdownStreamTarball: string; readonly root: string; readonly runnerRoot: string; readonly scaffolderBin: string; @@ -28,10 +34,16 @@ interface PackedFixture { */ const packFixture = async (): Promise => { const root = await mkdtemp(join(tmpdir(), 'create-agent-bundle-e2e-')); - const [{ tarball: frameworkTarball }, { tarball: scaffolderTarball }, { tarball: runtimeTarball }] = await Promise.all([ + const [ + { tarball: frameworkTarball }, + { tarball: scaffolderTarball }, + { tarball: runtimeTarball }, + { tarball: markdownStreamTarball }, + ] = await Promise.all([ sharedPackedTarball('agent-bundle'), sharedPackedTarball('create-agent-bundle'), sharedPackedTarball('runtime'), + sharedPackedTarball('markdown-stream'), ]); const pairedRuntimeTarball = join( dirname(frameworkTarball), @@ -48,12 +60,30 @@ const packFixture = async (): Promise => { }); return { frameworkTarball, + markdownStreamTarball, root, runnerRoot, scaffolderBin: join(runnerRoot, 'node_modules', '.bin', 'create-agent-bundle'), }; }; +/** + * `npm install` for a scaffolded project whose template pins the paired + * local runtime tarball. That runtime depends on `rsc-markdown-stream` by + * exact version, which the registry cannot serve until this repository + * publishes it, so the run-level tarball is offered in the same install and + * npm dedupes the runtime's edge onto it. `--no-save` leaves the scaffolded + * manifest exactly as the scaffolder wrote it: a real consumer declares + * nothing extra. + */ +export const installScaffoldedProject = async (projectRoot: string): Promise => { + const { markdownStreamTarball } = await fixture(); + await execFile('npm', ['install', ...npmInstallArguments, '--no-save', markdownStreamTarball], { + cwd: projectRoot, + env: installedEnvironment(), + }); +}; + let fixturePromise: Promise | undefined; const fixture = (): Promise => { fixturePromise ??= packFixture(); diff --git a/packages/rsc-markdown-stream/README.md b/packages/rsc-markdown-stream/README.md new file mode 100644 index 000000000..4d1f50aae --- /dev/null +++ b/packages/rsc-markdown-stream/README.md @@ -0,0 +1,132 @@ +# rsc-markdown-stream + +Render React / RSC trees to **Markdown**, never HTML. + +This is a custom renderer in the spirit of [rsc-html-stream](https://github.com/devongovett/rsc-html-stream), but for the other half of the job: where you would normally hand your tree to `renderToReadableStream` from `react-dom/server`, hand it to `renderToMarkdownStream` instead and get a stream of GitHub Flavored Markdown. No HTML is ever produced, no RSC payload is injected, nothing hydrates — markdown *is* the output. + +Zero dependencies. `react` is the only peer (v19+). `react-dom` is not in the dependency graph at all. + +## Usage + +```js +import {renderToMarkdown} from 'rsc-markdown-stream'; + +let md = await renderToMarkdown(
); +// "# Hello\n\nSome **bold** text...\n" +``` + +Streaming, parallel to the SSR setup you already know — consume an RSC stream and render it to markdown instead of HTML: + +```js +import {renderToReadableStream} from 'react-server-dom-BUNDLER/server.edge'; +import {createFromReadableStream} from 'react-server-dom-BUNDLER/client.edge'; +import {renderToMarkdownStream} from 'rsc-markdown-stream'; + +let rscStream = renderToReadableStream(); + +let data; +function Content() { + data ??= createFromReadableStream(rscStream); + return React.use(data); +} + +let markdownStream = renderToMarkdownStream(); +// ReadableStream of UTF-8 markdown, emitted block by block +``` + +## Where it runs in agent-bundle + +This package is the renderer behind `MarkdownContent` in +[`@agent-bundle/runtime`](../rsc-runtime/README.md): routes author headings, lists, and GFM tables +as JSX, and the runtime lowers the rendered Markdown into an `Agent.Markdown` node inside a real +React Flight request compiled under the `react-server` condition. The upstream repository, +[ScriptedAlchemy/rsc-markdown-stream](https://github.com/ScriptedAlchemy/rsc-markdown-stream), +keeps a standalone Rsbuild example of the full pipeline — an RSC server writing raw Flight bytes +to stdout, a consumer decoding them with `react-server-dom-webpack/client` and handing the tree to +`renderToMarkdownStream` — plus a browser demo; neither ships with this package. + +Markdown blocks stream out progressively as each server component's data resolves: the header +arrives first, a table next, the slower subtrees last — the same progressive behavior you'd get +from streaming HTML SSR, but the output is markdown. + +## shadcn/ui to markdown + +Component-library trees — Radix primitives, cva variants, lucide icons and all — render to markdown surprisingly well (the upstream example renders a shadcn/ui dashboard this way): + +- shadcn's `Table` components are real `` elements underneath, so they come out as GFM tables. +- Radix `Checkbox` renders a hidden `` for form interop — inside `
  • ` that becomes a GFM task list (`- [x]`). +- Radix `AccordionTrigger` lives inside an `

    ` header, so triggers become real markdown headings; collapsed content and inactive `TabsContent` are unmounted by Radix and produce nothing, while the `defaultValue` panel renders. +- Radix state/hooks (`useState`, `useId`, context) run on the renderer's built-in dispatcher in their initial, uncontrolled state. Portal-based components (Dialog, Popover, Tooltip) are the ones that won't work. +- Styled containers (Card, Alert, Button) flatten to plain text blocks — map `button` to a custom serializer via `options.components` to keep adjacent button labels from running together. + +### How this relates to rsc-html-stream + +[rsc-html-stream](https://github.com/devongovett/rsc-html-stream) does not render anything: it is a ~130-line transport that interleaves Flight bytes into an HTML stream as `