` via Shiki decorations. Values that are not TypeScript (display
+ * names such as `HTTP/2 Headers Object`) are highlighted as plain text, so
+ * their prose is not coloured as operators and numeric literals.
+ *
+ * Falls back to the minimal handler when the type failed to parse or nothing
+ * resolved (no point paying for highlighting then).
+ *
+ * @param {import('mdast-util-to-hast').State} state
+ * @param {import('mdast').Node} node
+ * @returns {import('hast').Element}
+ */
+export const typeAnnotationToHighlightedHast = (state, node) => {
+ const links = node.data?.links ?? [];
+
+ if (node.data?.parseError || links.length === 0) {
+ return typeAnnotationToHast(state, node);
+ }
+
+ const root = highlighter.shiki.codeToHast(node.value, {
+ lang: node.data?.typescript ? 'typescript' : 'text',
+ themes: { light: lightTheme, dark: darkTheme },
+ decorations: links.map(({ start, end, href }) => ({
+ start,
+ end,
+ tagName: 'a',
+ properties: { href, class: 'type-link' },
+ alwaysWrap: true,
+ })),
+ });
+
+ // codeToHast wraps the highlighted line in ; re-shape that into
+ // a single inline element ("only the outermost type opens/closes
+ // the code fragment")
+ const [preElement] = root.children;
+ const [codeElement] = preElement.children;
+
+ const result = {
+ type: 'element',
+ tagName: 'code',
+ properties: {
+ class: `${preElement.properties.class} type`,
+ },
+ children: codeElement.children,
+ };
+
+ state.patch(node, result);
+
+ return state.applyData(node, result);
+};
diff --git a/packages/node-legacy/src/legacy-html/generate.mjs b/packages/node-legacy/src/legacy-html/generate.mjs
index 989c6d806..6ff25a816 100644
--- a/packages/node-legacy/src/legacy-html/generate.mjs
+++ b/packages/node-legacy/src/legacy-html/generate.mjs
@@ -7,7 +7,7 @@ import getConfig from '@doc-kit/core/utils/configuration/index.mjs';
import { writeFile } from '@doc-kit/core/utils/file.mjs';
import { groupNodesByModule } from '@doc-kit/core/utils/generators.mjs';
import { minifyHTML } from '@doc-kit/core/utils/html-minifier.mjs';
-import { getRemarkRehypeWithShiki as remark } from '@doc-kit/core/utils/remark.mjs';
+import { getRemarkRehypeWithShiki as remark } from '@doc-kit/core/utils/remark-shiki.mjs';
import buildContent from './utils/buildContent.mjs';
import { replaceTemplateValues } from './utils/replaceTemplateValues.mjs';
diff --git a/packages/node-legacy/src/legacy-html/utils/buildContent.mjs b/packages/node-legacy/src/legacy-html/utils/buildContent.mjs
index 090675c6f..37abac651 100644
--- a/packages/node-legacy/src/legacy-html/utils/buildContent.mjs
+++ b/packages/node-legacy/src/legacy-html/utils/buildContent.mjs
@@ -6,7 +6,7 @@ import {
populate,
} from '@doc-kit/core/utils/configuration/templates.mjs';
import { UNIST } from '@doc-kit/core/utils/queries/index.mjs';
-import { getRemarkRehypeWithShiki as remark } from '@doc-kit/core/utils/remark.mjs';
+import { getRemarkRehypeWithShiki as remark } from '@doc-kit/core/utils/remark-shiki.mjs';
import { h as createElement } from 'hastscript';
import { u as createTree } from 'unist-builder';
import { SKIP, visit } from 'unist-util-visit';
diff --git a/packages/react/src/html/README.md b/packages/react/src/html/README.md
index 183324c41..9459d20a9 100644
--- a/packages/react/src/html/README.md
+++ b/packages/react/src/html/README.md
@@ -1,11 +1,17 @@
# `html` Generator
-The `html` generator transforms JSX AST entries into complete web bundles. Its
-bundler adapter builds server-rendered HTML and client-side JavaScript, CSS, and
-imported assets, then writes the complete static site to `output`. Vite is the
-default adapter, but projects can supply an adapter for webpack or another
-bundler. The generator is output-only and does not return an in-memory copy of
-its HTML or CSS.
+The `html` generator turns the pages' JSX into a complete static site: the
+server-rendered HTML pages, the client-side JavaScript, CSS, and imported
+assets, written to `output`. Vite is the default bundler adapter, but projects
+can supply an adapter for webpack or another bundler. The generator is
+output-only and does not return an in-memory copy of its HTML or CSS.
+
+The site is built in pieces that are each as small as they can be. The bundler
+builds the component library once and the client assets once. Each page's
+program is then compiled — JSX to a plain module — and the worker pool imports,
+renders, templates, minifies and writes the pages one at a time, so memory
+scales with the largest page rather than with the site. `all.html` is assembled
+from the module pages' compiled content rather than built again from scratch.
## Configuring
@@ -38,9 +44,13 @@ its HTML or CSS.
JSX-in-MDX. See [`components`](#components). **Default:** `{}`.
- `navigation` {Object} Sidebar groups and navigation bar items. See
[`navigation`](#navigation). **Default:** `{}`.
-- `bundler` {WebBundler} Adapter that renders server entries and writes the
- client and HTML output. See [Bundler adapters](#bundler-adapters).
- **Default:** `createViteBundler()`.
+- `generateAllPage` {boolean} When `true`, writes `all.html`: every module
+ page's content on one page, in sidebar order, assembled from the module pages
+ rather than built again. Chunk pages and the index are left out.
+ **Default:** `true`.
+- `bundler` {WebBundler} Adapter that bundles the component library and the
+ client assets, and compiles page programs. See
+ [Bundler adapters](#bundler-adapters). **Default:** `createViteBundler()`.
### `head`
@@ -175,29 +185,42 @@ omitted rather than rendered empty.
### Bundler adapters
-- `getEntryId` {Function} Return the module identifier placed in every
- populated HTML page's client script tag.
-- `render` {Function} Bundle and execute the server `entries`, returning a `Map`
- of API name to rendered HTML.
-- `build` {Function} Bundle the client `entry`, process the populated `pages`,
- and write the complete output.
+- `buildServer` {Function} Bundle the component library for Node and return
+ the `file:` URL of the built module.
+- `compile` {Function} Turn one page program, a module using JSX, into plain
+ JavaScript Node can import.
+- `buildClient` {Function} Bundle the client `entry` into `config.output` and
+ return the assets every page loads.
The `bundler` option accepts a small Doc Kit adapter rather than configuration
for a particular build system.
-`render` receives `{ entries, virtualImports, config }`; `build` receives
-`{ entry, virtualImports, pages, minifyPages, config }`. The server entry map
-uses `${api}.jsx` keys and rendered server results use `api` keys. The client
-`entry` is a single program shared by every page, served at the identifier
-`getEntryId` returns. Page maps use output-relative HTML file names;
-`minifyPages` takes such a map and returns its minified counterpart, spreading
-the work across Doc Kit's worker pool — call it on the final HTML when
-`config.minify` is set. `config` is the resolved `html` configuration.
-
-The adapter must compile the generated Preact JSX and CSS imports and resolve
-the supplied theme aliases and virtual modules. The generated `#theme/config`
-module exports `server` as `true` for the server build and `false` for the
-client build.
+`buildServer` receives `{ entry, virtualImports, outDir, config }`. The `entry`
+is the component library's source: re-exports of every component a page may
+render, Preact's `h` and `Fragment`, and `renderToStringAsync`. It must be
+bundled into one self-contained module written under `outDir` (a temporary
+directory the generator removes afterwards), since the page programs import it
+from wherever they are compiled to.
+
+`compile(code, fileName)` receives one page program: a module that imports the
+library and exports the page's `content` and a default render function, written
+in JSX. It must return plain JavaScript. The JSX must compile with the classic
+runtime to calls of the `_jsx` and `_Fragment` bindings the program imports
+(these names are exported as `JSX_PRAGMA` and `JSX_PRAGMA_FRAG` from the
+generator's `constants.mjs`), so that the page and the library share one Preact.
+
+`buildClient` receives `{ entry, virtualImports, config }`. The client `entry`
+is a single program shared by every page. It must be bundled into
+`config.output` and the call must return
+`{ scripts, preloads, stylesheets }`: paths relative to the output root of the
+module scripts to load, the chunks they statically import (rendered as
+`modulepreload` hints), and the stylesheets. The generator renders those into
+every page, resolved against the page's location.
+
+`config` is the resolved `html` configuration. The adapter must compile the
+generated Preact JSX and CSS imports and resolve the supplied theme aliases and
+virtual modules. The generated `#theme/config` module exports `server` as
+`true` for the server build and `false` for the client build.
A webpack integration can live entirely in project configuration without
adding webpack to Doc Kit:
@@ -205,17 +228,20 @@ adding webpack to Doc Kit:
```js
// webpack-bundler.mjs
export const createWebpackBundler = webpackOptions => ({
- getEntryId: () => 'virtual:doc-kit/client/index.jsx',
+ async buildServer({ entry, virtualImports, outDir, config }) {
+ // Materialize or load the in-memory modules, run webpack's Node target
+ // with the entry, write one self-contained module under `outDir`, and
+ // return its `file:` URL.
+ },
- async render({ entries, virtualImports, config }) {
- // Materialize or load the in-memory modules, run webpack's server target,
- // execute each emitted entry, and return Map.
+ async compile(code, fileName) {
+ // Transform the page's JSX (classic runtime, pragma `_jsx`, fragment
+ // pragma `_Fragment`) and return the resulting module source.
},
- async build({ entry, virtualImports, pages, minifyPages, config }) {
- // Run webpack's browser target, inject its emitted assets into `pages`,
- // minify them with `minifyPages` when `config.minify` is set, and write
- // the HTML and assets to config.output.
+ async buildClient({ entry, virtualImports, config }) {
+ // Run webpack's browser target into config.output and return
+ // { scripts, preloads, stylesheets } as output-relative paths.
},
});
```
@@ -271,18 +297,22 @@ export default {
The generator owns the fields required to coordinate its builds: config-file
loading, app type and base, virtual inputs, Preact compatibility aliases and
automatic JSX runtime, the Lightning CSS transformer, output/write mode, SSR
-format and temporary output, and SSR dependency bundling. Values supplied for
-those fields are replaced after configuration is merged. User plugins are
-registered after the generator's virtual-module plugin; other Vite options are
-preserved.
+format and output, and SSR dependency bundling. Values supplied for those
+fields are replaced after configuration is merged. User plugins are registered
+after the generator's virtual-module plugin; other Vite options are preserved.
+
+Vite builds the client entry as a module, not the pages as HTML entries, so
+plugins see and can transform every module of the client and server builds but
+never the HTML pages. Customize the pages through the
+[HTML template](#html-template) instead.
-Vite manifests are optional. Pass `build: { manifest: true }` or a manifest file
-name to `createViteBundler` when another tool needs one. The generated HTML
-already references the correct hashed scripts, stylesheets, imported assets,
-and module preloads.
+The adapter reads the client asset names from Vite's manifest. A manifest is
+written either way; pass `build: { manifest: true }` (or a file name) to
+`createViteBundler` to keep it in the output for another tool.
-Function-valued plugins and hooks are supported because the `html` generator
-runs on the main thread and does not serialize the bundler to a worker.
+The adapter is only ever used on the main thread, so function-valued plugins
+and hooks are supported. Worker threads receive the `html` configuration with
+its function values removed.
### Default `imports`
@@ -438,7 +468,7 @@ export default ({ metadata }) => (
- `headings` {Array} Pre-computed table of contents heading entries.
- `readingTime` {string|undefined} Estimated reading time (e.g. `'5 min read'`).
Only present when the `jsx-ast` generator's `showReadingTime` option is
- enabled.
+ enabled. On `all.html` it is the sum of the module pages' reading times.
- `children` {ComponentChildren} Processed page content.
The `Layout` component receives the props above. Custom Layout components can use
@@ -453,8 +483,8 @@ The HTML template file (set via `templatePath`) uses JavaScript template literal
- `title` {string} Fully resolved page title (e.g.
`'File system | Node.js v22.x'`).
- `dehydrated` {string} Server-rendered HTML for the page content.
-- `entrypoint` {string} Adapter-provided module identifier for this page's
- hydration.
+- `assets` {string} The `
+${title} ${assets}
```
-The configured adapter processes each populated page. It must replace or
-resolve `entrypoint`, include that page's scripts and stylesheets, and write the
-final HTML.
+The populated page is the final HTML: it is minified when `minify` is set and
+written as is. Put `${assets}` in the ``, or the page loads no script and
+no stylesheet.
diff --git a/packages/react/src/html/__tests__/generate.test.mjs b/packages/react/src/html/__tests__/generate.test.mjs
index 8934595e7..c3d4cc663 100644
--- a/packages/react/src/html/__tests__/generate.test.mjs
+++ b/packages/react/src/html/__tests__/generate.test.mjs
@@ -1,8 +1,9 @@
import assert from 'node:assert/strict';
-import { mkdir, mkdtemp, readFile, rm, writeFile } from 'node:fs/promises';
+import { mkdtemp, readdir, readFile, rm, writeFile } from 'node:fs/promises';
import { tmpdir } from 'node:os';
-import { dirname, join } from 'node:path';
+import { join } from 'node:path';
import { describe, it } from 'node:test';
+import { pathToFileURL } from 'node:url';
import { setConfig } from '@doc-kit/core/utils/configuration/index.mjs';
import { jsx, toJs } from 'estree-util-to-js';
@@ -10,16 +11,16 @@ import { jsx, toJs } from 'estree-util-to-js';
import buildContent from '../../jsx-ast/utils/buildContent.mjs';
import { buildNotFoundPage } from '../../jsx-ast/utils/synthetic/404.mjs';
import { generate as chunk } from '../../section-pages/generate.mjs';
-import { createViteBundler } from '../bundlers/vite.mjs';
+import { compile, createViteBundler } from '../bundlers/vite.mjs';
import { generate } from '../generate.mjs';
/**
- * Converts a JSX AST entry into the `{ data, code }` shape `web` now consumes,
- * mirroring the conversion the jsx-ast worker performs before streaming.
+ * Converts a page's JSX AST into the `{ data, headings, readingTime, content }`
+ * shape `html` consumes, mirroring the conversion the jsx-ast worker performs.
*/
-const toCodeItem = content => ({
- data: content.data,
- code: toJs(content, { handlers: jsx }).value,
+const toPage = ({ content, ...page }) => ({
+ ...page,
+ content: toJs(content, { handlers: jsx }).value,
});
const createEntry = (
@@ -67,6 +68,9 @@ const createTestConfiguration = async (context, target = ['html']) => {
},
});
+ // The pages under test are the module pages themselves
+ config.html.generateAllPage = false;
+
return { config, output };
};
@@ -82,7 +86,7 @@ describe('web generate', () => {
buildContent(notFoundPage.entries, notFoundPage.head),
]);
- await generate(contents.map(toCodeItem));
+ await generate(contents.map(toPage));
const [fsHTML, notFoundHTML] = await Promise.all([
readFile(join(output, 'api/fs.html'), 'utf8'),
@@ -93,9 +97,41 @@ describe('web generate', () => {
assert.match(fsHTML, /href=fs\.json/);
assert.match(fsHTML, /href=fs\.md/);
assert.doesNotMatch(notFoundHTML, /View As/);
- assert.match(fsHTML, /src=\.\.\/assets\//);
- assert.match(notFoundHTML, /src=\.\/assets\//);
+ // Assets resolve from the page: relative for real pages, from the root
+ // for synthetic ones, which are served at any path
+ assert.match(fsHTML, /src=\.\.\/assets\/client-[^ ]+\.js/);
+ assert.match(fsHTML, /href=\.\.\/assets\/[^ ]+\.css/);
+ assert.match(notFoundHTML, /src=\/assets\/client-[^ ]+\.js/);
assert.match(fsHTML, /on:idle[^>]*data-island-name=SearchBox/);
+ // The manifest the asset tags were read from does not ship
+ assert.equal((await readdir(output)).includes('.vite'), false);
+ });
+
+ it('assembles all.html from the module pages, in sidebar order', async context => {
+ const { config, output } = await createTestConfiguration(context);
+ config.html.generateAllPage = true;
+
+ const entries = [
+ createEntry('zlib', 'Zlib'),
+ createEntry('fs', 'File system'),
+ createEntry('index', 'Index'),
+ ];
+
+ await generate(
+ await Promise.all(
+ entries.map(entry => buildContent([entry], entry))
+ ).then(contents => contents.map(toPage))
+ );
+
+ const html = await readFile(join(output, 'all.html'), 'utf8');
+
+ assert.match(html, /All \|/);
+ // Both modules' content, file system first, the index left out
+ assert.match(html, /File system body[\s\S]*Zlib body/);
+ assert.doesNotMatch(html, /Index body/);
+ // Their tables of contents, concatenated
+ assert.match(html, /href=#fs[\s\S]*href=#zlib/);
+ assert.doesNotMatch(html, /View As/);
});
it('renders chunk pages with navigation back to their module', async context => {
@@ -145,7 +181,7 @@ describe('web generate', () => {
[...pages.values()].map(group => buildContent(group, group[0]))
);
- await generate(contents.map(toCodeItem));
+ await generate(contents.map(toPage));
const [fsHTML, readFileHTML] = await Promise.all([
readFile(join(output, 'fs.html'), 'utf8'),
@@ -191,7 +227,7 @@ describe('web generate', () => {
};
const fs = createEntry('fs', 'File system');
- await generate([toCodeItem(await buildContent([fs], fs))]);
+ await generate([toPage(await buildContent([fs], fs))]);
const html = await readFile(join(output, 'fs.html'), 'utf8');
assert.match(html, /Custom project docs/);
@@ -202,44 +238,43 @@ describe('web generate', () => {
assert.match(html, /property=og:type content=website/);
});
- it('uses Vite base URLs for absolute client assets', async context => {
+ it('uses the base URL for absolute client assets', async context => {
const { config, output } = await createTestConfiguration(context);
config.html.useAbsoluteURLs = true;
config.html.baseURL = 'https://example.com/docs';
const notFoundPage = buildNotFoundPage();
const content = await buildContent(notFoundPage.entries, notFoundPage.head);
- await generate([toCodeItem(content)]);
+ await generate([toPage(content)]);
const html = await readFile(join(output, '404.html'), 'utf8');
assert.match(html, /src=https:\/\/example\.com\/docs\/assets\//);
assert.match(html, /href=https:\/\/example\.com\/docs\/assets\//);
});
- it('applies configured Vite plugins', async context => {
+ it('applies configured Vite plugins to the client build', async context => {
const { config, output } = await createTestConfiguration(context);
config.html.bundler = createViteBundler({
plugins: [
{
- name: 'test-html-transform',
- transformIndexHtml() {
- return [
- {
- tag: 'meta',
- attrs: { name: 'vite-plugin', content: 'enabled' },
- injectTo: 'head',
- },
- ];
+ name: 'test-transform',
+ transform(code, id) {
+ if (id.includes('client/index.jsx')) {
+ return `${code}\nglobalThis.__DOC_KIT_PLUGIN__ = "enabled";`;
+ }
},
},
],
});
const fs = createEntry('fs', 'File system');
- await generate([toCodeItem(await buildContent([fs], fs))]);
- const html = await readFile(join(output, 'fs.html'), 'utf8');
+ await generate([toPage(await buildContent([fs], fs))]);
+
+ const assets = await readdir(join(output, 'assets'));
+ const client = assets.find(file => /^client-.*\.js$/.test(file));
+ const code = await readFile(join(output, 'assets', client), 'utf8');
- assert.match(html, /name=vite-plugin/);
+ assert.match(code, /__DOC_KIT_PLUGIN__/);
});
it('uses a custom bundler adapter for server and client output', async context => {
@@ -247,51 +282,78 @@ describe('web generate', () => {
const calls = [];
config.html.bundler = {
- getEntryId() {
- calls.push('entry');
- return '/custom/index.js';
- },
-
- async render({ entries, virtualImports, config: receivedConfig }) {
+ async buildServer({ entry, virtualImports, outDir, config: received }) {
calls.push('server');
- assert.strictEqual(receivedConfig, config.html);
- assert.ok(entries.has('fs.jsx'));
+ assert.strictEqual(received, config.html);
+ assert.match(
+ entry,
+ /export \{ default as Layout \} from "#theme\/Layout";/
+ );
+ assert.match(entry, /export \{ h, Fragment \} from "preact";/);
assert.match(virtualImports['#theme/config'], /export const pages/);
assert.match(
virtualImports['#theme/config'],
/export const server = true;/
);
- return new Map([
- ['fs', 'Custom SSR '],
- ]);
+ // A stand-in library: the page renders to a fixed fragment
+ const library = join(outDir, 'library.mjs');
+ await writeFile(
+ library,
+ [
+ 'export const h = (type, props, ...children) => ({ type, props, children });',
+ 'export const Fragment = "Fragment";',
+ 'export const Layout = "Layout";',
+ 'export const renderToStringAsync = async ({ props }) =>',
+ ' `${props.metadata.api} `;',
+ ].join('\n')
+ );
+
+ return pathToFileURL(library).href;
+ },
+
+ compile(code, fileName) {
+ calls.push('compile');
+ assert.match(fileName, /^fs\.jsx$/);
+ assert.match(code, /export const content = \(\) => <>/);
+
+ return compile(code, fileName);
},
- async build({ entry, virtualImports, pages, config: receivedConfig }) {
+ async buildClient({ entry, virtualImports, config: received }) {
calls.push('client');
- assert.strictEqual(receivedConfig, config.html);
+ assert.strictEqual(received, config.html);
assert.match(entry, /registerIslands\(/);
assert.match(
virtualImports['#theme/config'],
/export const server = false;/
);
- await Promise.all(
- [...pages].map(async ([fileName, html]) => {
- const path = join(output, fileName);
- await mkdir(dirname(path), { recursive: true });
- await writeFile(path, html);
- })
- );
+ return {
+ scripts: ['custom/index.js'],
+ preloads: ['custom/shared.js'],
+ stylesheets: ['custom/index.css'],
+ };
},
};
const fs = createEntry('fs', 'File system');
- await generate([toCodeItem(await buildContent([fs], fs))]);
+ await generate([toPage(await buildContent([fs], fs))]);
const html = await readFile(join(output, 'fs.html'), 'utf8');
- assert.match(html, /data-custom-ssr/);
- assert.match(html, /src="\/custom\/index\.js"/);
- assert.deepStrictEqual(calls, ['server', 'entry', 'client']);
+ assert.match(html, /fs<\/article>/);
+ assert.match(
+ html,
+ /'],
- ])
+ new Map([['virtual:entry', 'export default 42;']])
);
const entryId = plugin.resolveId('virtual:entry');
assert.ok(entryId);
assert.strictEqual(plugin.load(entryId), 'export default 42;');
assert.strictEqual(plugin.resolveId('virtual:missing'), undefined);
- assert.strictEqual(plugin.resolveId(htmlId), htmlId);
- assert.strictEqual(
- plugin.load(htmlId),
- ''
- );
+ assert.strictEqual(plugin.load('/real/file.js'), undefined);
});
});
describe('Vite configuration', () => {
- it('uses the generated client entries and configured output', () => {
+ it('uses the generated client entry and configured output', () => {
const vite = {
base: '/custom/',
build: {
outDir: 'custom-output',
- manifest: true,
rolldownOptions: {
input: 'custom-entry.js',
},
},
};
- const input = { fs: 'virtual:doc-kit/client/fs.jsx' };
+ const input = { client: 'virtual:doc-kit/client/index.jsx' };
const config = createViteConfig({
sources: new Map(),
input,
server: false,
+ outDir: output,
config: getConfig('html'),
vite,
});
assert.strictEqual(config.base, './');
assert.strictEqual(config.build.outDir, output);
- assert.strictEqual(config.build.manifest, true);
+ // A manifest is always written for the client: the asset tags come from it
+ assert.strictEqual(config.build.manifest, '.vite/manifest.json');
assert.strictEqual(config.build.rolldownOptions.input, input);
});
- it('keeps the temporary SSR build self-contained', () => {
+ it('keeps a manifest the project asked for', () => {
+ const config = createViteConfig({
+ sources: new Map(),
+ input: {},
+ server: false,
+ outDir: output,
+ config: getConfig('html'),
+ vite: { build: { manifest: 'manifest.json' } },
+ });
+
+ assert.strictEqual(config.build.manifest, 'manifest.json');
+ });
+
+ it('keeps the server library self-contained', () => {
const vite = {
ssr: {
external: ['preact'],
@@ -85,19 +92,20 @@ describe('Vite configuration', () => {
},
build: {
minify: true,
+ manifest: true,
rolldownOptions: {
external: ['preact'],
},
},
};
- const input = { fs: 'virtual:doc-kit/server/fs.jsx' };
+ const input = { library: 'virtual:doc-kit/server/library.jsx' };
const serverOutput = join(tmpdir(), 'doc-kit-vite-ssr-test');
const config = createViteConfig({
sources: new Map(),
input,
server: true,
- serverOutDir: serverOutput,
+ outDir: serverOutput,
config: getConfig('html'),
vite,
});
@@ -105,34 +113,45 @@ describe('Vite configuration', () => {
assert.strictEqual(config.build.ssr, true);
assert.strictEqual(config.build.outDir, serverOutput);
assert.strictEqual(config.build.minify, false);
+ assert.strictEqual(config.build.manifest, false);
assert.deepStrictEqual(config.build.rolldownOptions.external, []);
assert.deepStrictEqual(config.ssr.external, []);
assert.strictEqual(config.ssr.noExternal, true);
});
});
-describe('Vite SSR temporary output', () => {
- it('always removes temporary output after a renderer throws', async () => {
- const temporaryDirectory = await mkdtemp(
- join(tmpdir(), 'doc-kit-vite-cleanup-test-')
+describe('Vite page compilation', () => {
+ it('compiles JSX to the runtime bindings a page program imports', async () => {
+ const code = await compile(
+ [
+ 'import { h as _jsx, Fragment as _Fragment, Layout } from "file:///library.mjs";',
+ 'export const content = () => <>Hi >;',
+ 'export default () => {content()} ;',
+ ].join('\n'),
+ 'fs.jsx'
);
- await assert.rejects(
- render({
- entries: new Map([
- [
- 'broken.jsx',
- 'export default () => { throw new Error("render failed"); };',
- ],
- ]),
- virtualImports: {},
- config: getConfig('html'),
- vite: {},
- createTemporaryDirectory: async () => temporaryDirectory,
- }),
- /render failed/
- );
+ assert.match(code, /_jsx\(_Fragment, null, .*_jsx\("h1", \{/s);
+ assert.match(code, /_jsx\(Layout, \{/);
+ // Nothing else is pulled in: the runtime is the program's own import
+ assert.doesNotMatch(code, /jsx-runtime/);
+ });
+
+ it('builds an importable library module from a virtual entry', async context => {
+ const outDir = await mkdtemp(join(tmpdir(), 'doc-kit-vite-library-test-'));
+ context.after(() => rm(outDir, { recursive: true, force: true }));
+
+ const url = await buildServer({
+ entry:
+ 'export { h, Fragment } from "preact"; export { answer } from "virtual:answer";',
+ virtualImports: { 'virtual:answer': 'export const answer = 42;' },
+ outDir,
+ config: getConfig('html'),
+ });
+
+ const library = await import(url);
- await assert.rejects(access(temporaryDirectory), { code: 'ENOENT' });
+ assert.strictEqual(library.answer, 42);
+ assert.strictEqual(typeof library.h, 'function');
});
});
diff --git a/packages/react/src/html/bundlers/vite.mjs b/packages/react/src/html/bundlers/vite.mjs
index 893dd66e7..b3e159cc4 100644
--- a/packages/react/src/html/bundlers/vite.mjs
+++ b/packages/react/src/html/bundlers/vite.mjs
@@ -1,6 +1,5 @@
-import { mkdtemp, rm } from 'node:fs/promises';
-import { tmpdir } from 'node:os';
-import { basename, isAbsolute, join, resolve } from 'node:path';
+import { readFile, rm, rmdir } from 'node:fs/promises';
+import { dirname, join, resolve } from 'node:path';
import { fileURLToPath, pathToFileURL } from 'node:url';
import {
@@ -8,10 +7,10 @@ import {
defaultClientConditions,
defaultServerConditions,
mergeConfig,
+ transformWithOxc,
} from 'vite';
-import { FONT_DIRECTORY } from '../constants.mjs';
-import { createPageMinifier } from '../utils/minify.mjs';
+import { FONT_DIRECTORY, JSX_PRAGMA, JSX_PRAGMA_FRAG } from '../constants.mjs';
const VIRTUAL_PREFIX = 'virtual:doc-kit/';
const RESOLVED_VIRTUAL_PREFIX = '\0doc-kit:';
@@ -21,6 +20,18 @@ const PACKAGE_ANCHOR = fileURLToPath(import.meta.url);
// entry chunk (plus its shared dependencies) for the whole site, rather than
// a copy per page.
const CLIENT_ENTRY_ID = `${VIRTUAL_PREFIX}client/index.jsx`;
+const CLIENT_NAME = 'client';
+
+// The server-side component library every page program imports from.
+const SERVER_ENTRY_ID = `${VIRTUAL_PREFIX}server/library.jsx`;
+const LIBRARY_NAME = 'library';
+
+// Where Vite writes the client manifest the asset tags are read from, unless
+// the project asked for a manifest of its own.
+const MANIFEST_NAME = '.vite/manifest.json';
+
+// Vite injects this into HTML entries; a module entry has to import it.
+const MODULE_PRELOAD_POLYFILL = 'vite/modulepreload-polyfill';
/**
* Resolves a package specifier
@@ -44,9 +55,7 @@ const resolveThemeAliases = (aliases, root) =>
);
/**
- * Creates a Vite plugin that serves an exact map of in-memory modules and HTML
- * entries. HTML keeps its absolute identifier so Vite emits it at the matching
- * path relative to the configured root.
+ * Creates a Vite plugin that serves an exact map of in-memory modules.
*
* @param {Map} sources
* @returns {import('vite').Plugin}
@@ -54,7 +63,7 @@ const resolveThemeAliases = (aliases, root) =>
export const createVirtualModulesPlugin = sources => {
// Package imports are anchored to the same importer whichever virtual
// module they come from, so each specifier resolves the same way every
- // time: resolve it once, not once per page that imports it.
+ // time: resolve it once, not once per module that imports it.
const anchored = new Map();
return {
@@ -70,9 +79,7 @@ export const createVirtualModulesPlugin = sources => {
*/
resolveId(id, importer) {
if (sources.has(id)) {
- return isAbsolute(id) && id.endsWith('.html')
- ? id
- : `${RESOLVED_VIRTUAL_PREFIX}${id}`;
+ return `${RESOLVED_VIRTUAL_PREFIX}${id}`;
}
if (
@@ -97,75 +104,11 @@ export const createVirtualModulesPlugin = sources => {
* @returns {string|undefined}
*/
load(id) {
- return sources.get(
- id.startsWith(RESOLVED_VIRTUAL_PREFIX)
- ? id.slice(RESOLVED_VIRTUAL_PREFIX.length)
- : id
- );
- },
- };
-};
-
-/**
- * Finalizes Vite's generated HTML before its normal write phase.
- *
- * @param {import('../types').ClientBundleOptions['minifyPages']} minifyPages
- * @returns {import('vite').Plugin}
- */
-const createHTMLFinalizerPlugin = minifyPages => ({
- name: 'doc-kit:finalize-html',
- /**
- * Minifies every generated HTML entry after Vite has injected its scripts,
- * stylesheets, and module preloads. The pages are handed over as one batch
- * so the minifier can spread them across the worker pool.
- */
- generateBundle: {
- order: 'post',
- /**
- * @param {object} _
- * @param {Record} bundle
- */
- async handler(_, bundle) {
- const assets = Object.values(bundle).filter(
- item => item.type === 'asset' && item.fileName.endsWith('.html')
- );
-
- const minified = await minifyPages(
- new Map(
- assets.map(asset => [
- asset.fileName,
- typeof asset.source === 'string'
- ? asset.source
- : Buffer.from(asset.source).toString('utf8'),
- ])
- )
- );
-
- for (const asset of assets) {
- asset.source = minified.get(asset.fileName);
+ if (id.startsWith(RESOLVED_VIRTUAL_PREFIX)) {
+ return sources.get(id.slice(RESOLVED_VIRTUAL_PREFIX.length));
}
},
- },
-});
-
-/**
- * Converts generated page programs into named Vite inputs and virtual modules
- * for the SSR build: each page needs a distinct virtual JSX path of its own.
- *
- * @param {Map} codeMap
- */
-const createServerEntries = codeMap => {
- const input = {};
- const sources = new Map();
-
- for (const [fileName, code] of codeMap) {
- const id = `${VIRTUAL_PREFIX}server/${fileName}`;
-
- input[basename(fileName, '.jsx')] = id;
- sources.set(id, code);
- }
-
- return { input, sources };
+ };
};
/**
@@ -174,10 +117,9 @@ const createServerEntries = codeMap => {
*
* @param {object} options
* @param {Map} options.sources
- * @param {Record|Array} options.input
+ * @param {Record} options.input
* @param {boolean} options.server
- * @param {string} [options.serverOutDir]
- * @param {import('../types').ClientBundleOptions['minifyPages']} [options.minifyPages]
+ * @param {string} options.outDir
* @param {import('../types').ResolvedWebConfiguration} options.config
* @param {import('vite').UserConfig} options.vite
* @returns {import('vite').InlineConfig}
@@ -186,8 +128,7 @@ export const createViteConfig = ({
sources,
input,
server,
- serverOutDir,
- minifyPages = createPageMinifier(),
+ outDir,
config: webConfig,
vite = {},
}) => {
@@ -209,14 +150,8 @@ export const createViteConfig = ({
logLevel: vite.logLevel ?? 'warn',
// Virtual entries must resolve before user plugins, while user plugins can
- // still transform every module and generated HTML page.
- plugins: [
- createVirtualModulesPlugin(sources),
- ...(vite.plugins ?? []),
- ...(!server && webConfig.minify
- ? [createHTMLFinalizerPlugin(minifyPages)]
- : []),
- ],
+ // still transform every module.
+ plugins: [createVirtualModulesPlugin(sources), ...(vite.plugins ?? [])],
resolve: mergeConfig(
{ resolve: vite.resolve },
@@ -265,16 +200,18 @@ export const createViteConfig = ({
build: {
...vite.build,
- // Both builds are complete Vite outputs. SSR uses a private directory
- // because its entries can share chunks; the client writes the final site.
- outDir: server ? serverOutDir : resolve(webConfig.output),
+ // Both builds are complete Vite outputs. The server library goes to a
+ // private directory; the client writes into the final site.
+ outDir,
write: true,
emptyOutDir: false,
copyPublicDir: false,
watch: null,
lib: false,
- ...(server ? { manifest: false } : {}),
+ // The client manifest is how the asset tags are found; a manifest the
+ // project asked for doubles as that.
+ manifest: server ? false : vite.build?.manifest || MANIFEST_NAME,
ssr: server,
// Islands make split CSS wrong: a component's stylesheet would arrive
@@ -283,8 +220,8 @@ export const createViteConfig = ({
// first paint, whenever — or whether — its islands load.
...(server ? {} : { cssCodeSplit: false }),
- // Browser output follows the generator's minification setting. Temporary
- // server output stays readable and disappears immediately after render.
+ // Browser output follows the generator's minification setting. The
+ // server library is only ever executed, never shipped.
minify: server ? false : (vite.build?.minify ?? webConfig.minify),
rolldownOptions: {
@@ -337,109 +274,141 @@ export const createViteConfig = ({
};
/**
- * Builds and executes the server entries through Vite's SSR pipeline.
+ * Bundles the component library through Vite's SSR pipeline, into one
+ * self-contained module Node can import from anywhere.
*
- * @param {object} options
- * @param {Map} options.entries
- * @param {Record} options.virtualImports
- * @param {import('../types').ResolvedWebConfiguration} options.config
- * @param {import('vite').UserConfig} options.vite
- * @param {() => Promise} [options.createTemporaryDirectory]
- * @returns {Promise>}
+ * @param {import('../types').ServerBundleOptions & { vite?: import('vite').UserConfig }} options
+ * @returns {Promise} The `file:` URL of the built library
*/
-export const render = async ({
- entries,
+export const buildServer = async ({
+ entry,
virtualImports,
- createTemporaryDirectory = () => mkdtemp(join(tmpdir(), 'doc-kit-vite-ssr-')),
+ outDir,
config,
vite = {},
}) => {
- const { input, sources } = createServerEntries(entries);
+ const sources = new Map([
+ [SERVER_ENTRY_ID, entry],
+ ...Object.entries(virtualImports),
+ ]);
- for (const [id, code] of Object.entries(virtualImports)) {
- sources.set(id, code);
- }
+ await viteBuild(
+ createViteConfig({
+ sources,
+ input: { [LIBRARY_NAME]: SERVER_ENTRY_ID },
+ server: true,
+ outDir,
+ config,
+ vite,
+ })
+ );
- // Vite writes the compiled SSR renderers here so Node can import and execute
- // them without mixing intermediate modules into the final site. The directory
- // is removed after rendering
- const temporaryDirectory = await createTemporaryDirectory();
+ return pathToFileURL(join(outDir, `${LIBRARY_NAME}.mjs`)).href;
+};
- try {
- await viteBuild(
- createViteConfig({
- sources,
- input,
- server: true,
- serverOutDir: temporaryDirectory,
- config,
- vite,
- })
- );
-
- const pages = new Map();
-
- await Promise.all(
- Object.keys(input).map(async name => {
- const module = await import(
- pathToFileURL(join(temporaryDirectory, `${name}.mjs`)).href
- );
-
- pages.set(name, await module.default());
- })
- );
-
- return pages;
- } finally {
- await rm(temporaryDirectory, { recursive: true, force: true });
+/**
+ * Compiles one page program's JSX to the calls it imports from the library.
+ * A single native transform, no module graph: this runs once per page.
+ *
+ * @param {string} code
+ * @param {string} fileName
+ * @returns {Promise}
+ */
+export const compile = async (code, fileName) => {
+ const result = await transformWithOxc(code, fileName, {
+ jsx: {
+ runtime: 'classic',
+ pragma: JSX_PRAGMA,
+ pragmaFrag: JSX_PRAGMA_FRAG,
+ },
+ });
+
+ return result.code;
+};
+
+/**
+ * Collects the chunks an entry statically imports, transitively, in the order
+ * Vite would preload them.
+ *
+ * @param {Record }>} manifest
+ * @param {{ imports?: Array }} chunk
+ * @param {Set} [seen]
+ * @returns {Array}
+ */
+const collectImports = (manifest, chunk, seen = new Set()) => {
+ for (const key of chunk.imports ?? []) {
+ if (!seen.has(key)) {
+ seen.add(key);
+ collectImports(manifest, manifest[key], seen);
+ }
}
+
+ return [...seen].map(key => manifest[key].file);
};
/**
- * Lets Vite transform the rendered pages as HTML entries. Vite injects their
- * hashed scripts, stylesheets, and module preloads, then writes the site.
+ * Bundles the client entry into the site and reads back, from Vite's
+ * manifest, the assets every page has to load.
*
- * @param {object} options
- * @param {string} options.entry
- * @param {Record} options.virtualImports
- * @param {Map} options.pages
- * @param {import('../types').ClientBundleOptions['minifyPages']} [options.minifyPages]
- * @param {import('../types').ResolvedWebConfiguration} options.config
- * @param {import('vite').UserConfig} options.vite
- * @returns {Promise}
+ * @param {import('../types').ClientBundleOptions & { vite?: import('vite').UserConfig }} options
+ * @returns {Promise}
*/
-export const build = async ({
+export const buildClient = async ({
entry,
virtualImports,
- pages,
- minifyPages,
config,
vite = {},
}) => {
- const sources = new Map([[CLIENT_ENTRY_ID, entry]]);
- const root = resolve(vite.root ?? process.cwd());
- const input = [];
-
- for (const [fileName, html] of pages) {
- const id = resolve(root, fileName);
- input.push(id);
- sources.set(id, html);
- }
+ const sources = new Map([
+ [
+ CLIENT_ENTRY_ID,
+ `import ${JSON.stringify(MODULE_PRELOAD_POLYFILL)};\n${entry}`,
+ ],
+ ...Object.entries(virtualImports),
+ ]);
- for (const [id, code] of Object.entries(virtualImports)) {
- sources.set(id, code);
- }
+ const outDir = resolve(config.output);
await viteBuild(
createViteConfig({
sources,
- input,
+ input: { [CLIENT_NAME]: CLIENT_ENTRY_ID },
server: false,
- minifyPages,
+ outDir,
config,
vite,
})
);
+
+ const requested = vite.build?.manifest;
+ const manifestPath = join(
+ outDir,
+ typeof requested === 'string' ? requested : MANIFEST_NAME
+ );
+
+ const manifest = JSON.parse(await readFile(manifestPath, 'utf8'));
+
+ // The manifest was only for us, so it does not ship with the site
+ if (!requested) {
+ await rm(manifestPath);
+ await rmdir(dirname(manifestPath)).catch(() => {});
+ }
+
+ const chunk = Object.values(manifest).find(item => item.isEntry);
+
+ return {
+ scripts: [chunk.file],
+ preloads: collectImports(manifest, chunk),
+ // With CSS code splitting off, the one stylesheet is its own manifest
+ // entry rather than being listed under the chunk that imports it.
+ stylesheets: [
+ ...new Set(
+ Object.values(manifest)
+ .map(({ file }) => file)
+ .filter(file => file.endsWith('.css'))
+ ),
+ ],
+ };
};
/**
@@ -450,19 +419,16 @@ export const build = async ({
*/
export const createViteBundler = (options = {}) => ({
/**
- * The client entry every page loads.
- */
- getEntryId: () => CLIENT_ENTRY_ID,
- /**
- * Runs the Vite server build.
+ * Bundles the server-side component library.
*
* @param {import('../types').ServerBundleOptions} context
*/
- render: context => render({ ...context, vite: options }),
+ buildServer: context => buildServer({ ...context, vite: options }),
+ compile,
/**
- * Runs the Vite client build.
+ * Bundles the client assets.
*
* @param {import('../types').ClientBundleOptions} context
*/
- build: context => build({ ...context, vite: options }),
+ buildClient: context => buildClient({ ...context, vite: options }),
});
diff --git a/packages/react/src/html/constants.mjs b/packages/react/src/html/constants.mjs
index 3850a2a38..f894e0611 100644
--- a/packages/react/src/html/constants.mjs
+++ b/packages/react/src/html/constants.mjs
@@ -75,6 +75,14 @@ export const JSX_IMPORTS = {
},
};
+/**
+ * The bindings a page program imports from the component library for its JSX,
+ * and which the bundler's `compile` must target (classic runtime): every
+ * `` becomes a `_jsx(...)` call, every `<>` a `_Fragment`.
+ */
+export const JSX_PRAGMA = '_jsx';
+export const JSX_PRAGMA_FRAG = '_Fragment';
+
/**
* Where the bundler emits fonts
*/
diff --git a/packages/react/src/html/generate.mjs b/packages/react/src/html/generate.mjs
index 0aa1d2f63..0e060d3b4 100644
--- a/packages/react/src/html/generate.mjs
+++ b/packages/react/src/html/generate.mjs
@@ -1,22 +1,37 @@
'use strict';
-import { readFile } from 'node:fs/promises';
+import { mkdir, mkdtemp, readFile, rm, writeFile } from 'node:fs/promises';
+import { tmpdir } from 'node:os';
+import { join } from 'node:path';
+import { pathToFileURL } from 'node:url';
+import logger from '@doc-kit/core/logger/index.mjs';
import getConfig from '@doc-kit/core/utils/configuration/index.mjs';
+import { resolveBundler } from './bundlers/index.mjs';
+import { buildAllPage } from './utils/all.mjs';
import { copyStaticAssets } from './utils/copying.mjs';
-import { createPageMinifier } from './utils/minify.mjs';
-import { createCodeConverter, processBundles } from './utils/processing.mjs';
+import createProgramBuilder, { moduleFileName } from './utils/generate.mjs';
+import { createVirtualImports } from './utils/processing.mjs';
+import { createPageWriter } from './utils/render.mjs';
+
+const htmlLogger = logger.child('html');
/**
- * Main generation function that sends per-page JSX code to the web bundler.
+ * Main generation function: turns the pages' JSX into the static site.
+ *
+ * Receives `jsx-ast`'s output as `{ data, headings, readingTime, content }`
+ * items, `content` being each page's JSX code. The site is then built in
+ * pieces that are each as small as they can be:
*
- * Receives `jsx-ast`'s output as `{ data, code }` items — the JSX AST was
- * already serialized to `code` in the jsx-ast worker, so no AST is held here.
- * Bundling and rendering then run once over the accumulated code, since shared
- * component chunks, CSS, and the sidebar need every entry together. The
- * worker pool only comes back into play for the final minification of the
- * rendered pages.
+ * 1. The component library is bundled once, for the server.
+ * 2. The client assets are bundled once; every page loads the same ones.
+ * 3. Each page's program is compiled (JSX to a plain module) and written to a
+ * temporary directory, one at a time, so no page is held longer than that.
+ * 4. The worker pool imports, renders, templates, minifies and writes the
+ * pages, one page in memory per worker.
+ * 5. `all.html`, when enabled, is a program that imports the module pages'
+ * content, so it is written last from what was already compiled.
*
* @type {import('./types').Generator['generate']}
*/
@@ -25,25 +40,83 @@ export async function generate(input, worker) {
const template = await readFile(config.templatePath, 'utf-8');
- const converter = createCodeConverter();
+ const pages = [...input];
+ const all = config.generateAllPage ? buildAllPage(pages) : undefined;
- // Per-page metadata, in render order. Each item is already just
- // `{ data, code }` — the heavy JSX AST was converted to `code` and discarded
- // in the jsx-ast worker, so nothing large is held here.
- const datas = [];
+ // Every page's metadata, in render order — the sidebar, the index and the
+ // cross links need the whole set.
+ const datas = [...pages, ...(all ? [all] : [])].map(({ data }) => data);
- for (const item of input) {
- converter.add(item);
- datas.push(item.data);
- }
+ const bundler = await resolveBundler(config.bundler);
+ const { buildLibraryProgram, buildPageProgram, clientProgram } =
+ createProgramBuilder();
+
+ // The built library and the compiled page programs live here until every
+ // page is written; the directory is removed afterwards
+ const outDir = await mkdtemp(join(tmpdir(), 'doc-kit-html-'));
+
+ try {
+ const libraryURL = await bundler.buildServer({
+ entry: buildLibraryProgram(),
+ virtualImports: createVirtualImports(datas, config.virtualImports, true),
+ outDir,
+ config,
+ });
+
+ htmlLogger.debug('Built the component library');
+
+ const assets = await bundler.buildClient({
+ entry: clientProgram,
+ virtualImports: createVirtualImports(datas, config.virtualImports, false),
+ config,
+ });
+
+ htmlLogger.debug('Built the client assets', assets);
+
+ const modulesDir = join(outDir, 'pages');
+ await mkdir(modulesDir);
- await processBundles({
- serverCodeMap: converter.serverCodeMap,
- clientProgram: converter.clientProgram,
- datas,
- template,
- minifyPages: createPageMinifier(worker),
- });
+ /**
+ * Compiles a page's program to disk and describes it for the workers.
+ *
+ * @param {import('./types').Page} page
+ * @returns {Promise}
+ */
+ const compile = async page => {
+ const file = join(modulesDir, moduleFileName(page.data.api));
+
+ await writeFile(
+ file,
+ await bundler.compile(
+ buildPageProgram(page, libraryURL),
+ `${page.data.api}.jsx`
+ )
+ );
+
+ return { moduleURL: pathToFileURL(file).href, data: page.data };
+ };
+
+ const tasks = [];
+
+ for (const page of pages) {
+ tasks.push(await compile(page));
+ }
+
+ htmlLogger.debug(`Compiled ${tasks.length} page programs`);
+
+ const writePages = createPageWriter(worker);
+ const extra = { template, assets };
+
+ await writePages(tasks, extra);
+
+ // The composed page imports the others' compiled programs, so it can only
+ // be rendered once those exist — which they now do.
+ if (all) {
+ await writePages([await compile(all)], extra);
+ }
+ } finally {
+ await rm(outDir, { recursive: true, force: true });
+ }
await copyStaticAssets(config);
}
diff --git a/packages/react/src/html/index.mjs b/packages/react/src/html/index.mjs
index aad9dcae4..607592a39 100644
--- a/packages/react/src/html/index.mjs
+++ b/packages/react/src/html/index.mjs
@@ -3,24 +3,24 @@
import { join } from 'node:path';
import { generate } from './generate.mjs';
-import { processChunk } from './utils/minify.mjs';
+import { processChunk } from './utils/render.mjs';
/**
- * Web generator - transforms JSX AST entries into complete web bundles.
+ * Web generator - transforms the pages' JSX into a complete static site.
*
- * This generator processes JSX AST entries and produces:
+ * This generator takes the `jsx-ast` output and produces:
* - Server-side rendered HTML pages
* - Client-side JavaScript with code splitting
* - Bundled CSS styles
*
- * The configured bundler writes the complete static site to the output
- * directory; this terminal generator does not return an in-memory copy.
+ * The configured bundler builds the component library and the client assets
+ * once each; the pages are then compiled, rendered, templated, minified and
+ * written one at a time by the worker pool, so memory scales with the largest
+ * page rather than with the site. `all.html` is assembled from the module
+ * pages' compiled content instead of being built again from scratch.
*
- * `jsx-ast` serializes each page's JSX AST to a `code` string inside its worker,
- * so this generator only ever handles small `{ data, code }` items — the heavy
- * ASTs (notably the giant `all` page) never reach the main thread. Bundling and
- * rendering run once over the accumulated code, since code-splitting and the
- * sidebar need every entry together.
+ * This terminal generator writes to the output directory and does not return
+ * an in-memory copy.
*
* @type {import('./types').Generator}
*/
@@ -82,14 +82,18 @@ export default {
showCrossLinks: false,
},
+ // Whether to write `all.html`: every module page's content on one page,
+ // assembled from the module pages rather than built again.
+ generateAllPage: true,
+
// When omitted, the Vite adapter is loaded lazily during generation.
bundler: undefined,
}),
generate,
- // Minifying the rendered pages is the one step of the bundle that scales
- // with the page count, so it is farmed out to the worker pool.
+ // Rendering, templating, minifying and writing the pages scales with the
+ // page count, so it is farmed out to the worker pool.
hasParallelProcessor: true,
processChunk,
diff --git a/packages/react/src/html/template.html b/packages/react/src/html/template.html
index 248b3abc0..bf8cee2c5 100644
--- a/packages/react/src/html/template.html
+++ b/packages/react/src/html/template.html
@@ -9,6 +9,8 @@
${preloads}
+ ${assets}
+
${head}
@@ -22,6 +24,5 @@
${dehydrated}
-
diff --git a/packages/react/src/html/types.d.ts b/packages/react/src/html/types.d.ts
index 54a8052da..1613ffeb2 100644
--- a/packages/react/src/html/types.d.ts
+++ b/packages/react/src/html/types.d.ts
@@ -1,4 +1,4 @@
-import type { JSXContent } from '../jsx-ast/utils/buildContent.mjs';
+import type { PageCode } from '../jsx-ast/types';
import type { GlobalConfiguration } from '@doc-kit/core/utils/configuration/types';
import type SideBar from '@node-core/ui-components/Containers/Sidebar';
import type NavBar from '@node-core/ui-components/Containers/NavBar';
@@ -30,35 +30,62 @@ export type HeadConfig = {
export type ResolvedWebConfiguration = Configuration & GlobalConfiguration;
+// A page assembled from other pages' content (`all.html`): `parts` lists the
+// `api`s whose compiled programs it imports, in order.
+export type ComposedPage = Omit & { parts: Array };
+
+// What the generator turns into a page program.
+export type Page = PageCode | ComposedPage;
+
+// A compiled page program, ready for a worker to import and render.
+export type PageTask = {
+ // `file:` URL of the compiled module.
+ moduleURL: string;
+ // The page's metadata.
+ data: PageCode['data'];
+};
+
+// The client assets every page loads, as paths relative to the output root.
+export type ClientAssets = {
+ // Module scripts, in load order.
+ scripts: Array;
+ // Chunks the scripts statically import, to preload.
+ preloads: Array;
+ // Stylesheets.
+ stylesheets: Array;
+};
+
export type ServerBundleOptions = {
- // Server-side JSX programs keyed by `${api}.jsx`.
- entries: Map;
- // In-memory modules that the bundler must make available to the entries.
+ // The component library's source: re-exports of every component, the JSX
+ // runtime (`h`, `Fragment`) and `renderToStringAsync`.
+ entry: string;
+ // In-memory modules that the bundler must make available to the entry.
virtualImports: Record;
+ // Where to write the built library.
+ outDir: string;
config: ResolvedWebConfiguration;
};
export type ClientBundleOptions = {
// The client-side program every page loads, hydrating its server-rendered
- // markup; the bundler serves it at the identifier `getEntryId()` returns.
+ // markup.
entry: string;
// In-memory modules that the bundler must make available to the entry.
virtualImports: Record;
- // Populated HTML keyed by its output-relative file name.
- pages: Map;
- // Minifies final pages (keyed like `pages`) across the worker pool. Bundlers
- // call it on the HTML they are about to write when `config.minify` is set.
- minifyPages: (pages: Map) => Promise>;
config: ResolvedWebConfiguration;
};
export type WebBundler = {
- // Returns the module identifier embedded in every page's client script tag.
- getEntryId(): string;
- // Returns rendered HTML keyed by API name.
- render(options: ServerBundleOptions): Promise>;
- // Bundles the client entries and writes the complete site.
- build(options: ClientBundleOptions): Promise;
+ // Bundles the component library for Node and returns the `file:` URL of the
+ // built module. Page programs import from it.
+ buildServer(options: ServerBundleOptions): Promise;
+ // Turns one page program — a module using JSX — into plain JavaScript Node
+ // can import. JSX must compile to calls of the `_jsx` and `_Fragment`
+ // bindings the program imports (the classic runtime; see `JSX_PRAGMA`).
+ compile(code: string, fileName: string): Promise;
+ // Bundles the client entry into `config.output` and returns the assets every
+ // page must load.
+ buildClient(options: ClientBundleOptions): Promise;
};
export type Configuration = {
@@ -83,11 +110,14 @@ export type Configuration = {
navbar?: ComponentProps['navItems'];
showCrossLinks?: boolean;
};
+ // Whether to write `all.html`, every module page's content on one page.
+ generateAllPage: boolean;
// Optional bundler adapter. When omitted, the Vite adapter is loaded lazily.
bundler?: WebBundler;
};
export type Generator = GeneratorMetadata<
Configuration,
- Generate, Promise>
+ Generate, Promise>,
+ ProcessChunk
>;
diff --git a/packages/react/src/html/utils/__tests__/all.test.mjs b/packages/react/src/html/utils/__tests__/all.test.mjs
new file mode 100644
index 000000000..4e4d9b78e
--- /dev/null
+++ b/packages/react/src/html/utils/__tests__/all.test.mjs
@@ -0,0 +1,64 @@
+import assert from 'node:assert/strict';
+import { describe, it } from 'node:test';
+
+import { buildAllPage } from '../all.mjs';
+
+const createPage = (
+ api,
+ name,
+ { depth = 1, chunk, synthetic, minutes } = {}
+) => ({
+ data: {
+ api,
+ path: `/${api}`,
+ basename: api,
+ chunk,
+ synthetic,
+ heading: { depth, data: { name, text: name, slug: api } },
+ },
+ headings: [{ depth, value: name, slug: api }],
+ readingTime:
+ minutes === undefined
+ ? undefined
+ : { text: `${minutes} min read`, minutes },
+ content: `<>${name} >`,
+});
+
+describe('buildAllPage', () => {
+ it('returns a synthetic `all` page made of the module pages, in sidebar order', () => {
+ const { data, headings, parts } = buildAllPage([
+ createPage('zlib', 'Zlib'),
+ createPage('index', 'Index'),
+ createPage('fs', 'File system'),
+ createPage('404', 'Page Not Found', { synthetic: true }),
+ createPage('fs-readfile', 'readFile', { chunk: { api: 'fs' } }),
+ ]);
+
+ assert.equal(data.api, 'all');
+ assert.equal(data.path, '/all');
+ assert.equal(data.heading.data.name, 'All');
+ assert.equal(data.synthetic, true);
+ // The index, the other synthetic pages and the chunk pages are left out
+ assert.deepEqual(parts, ['fs', 'zlib']);
+ assert.deepEqual(
+ headings.map(({ value }) => value),
+ ['File system', 'Zlib']
+ );
+ });
+
+ it('sums the reading time of its parts when it is shown', () => {
+ const { readingTime } = buildAllPage([
+ createPage('fs', 'File system', { minutes: 2.4 }),
+ createPage('zlib', 'Zlib', { minutes: 1.2 }),
+ ]);
+
+ assert.equal(readingTime.text, '4 min read');
+ assert.ok(Math.abs(readingTime.minutes - 3.6) < 1e-9);
+ });
+
+ it('has no reading time when the pages have none', () => {
+ const { readingTime } = buildAllPage([createPage('fs', 'File system')]);
+
+ assert.equal(readingTime, undefined);
+ });
+});
diff --git a/packages/react/src/html/utils/__tests__/processing.test.mjs b/packages/react/src/html/utils/__tests__/processing.test.mjs
index 3cab6c0fb..e3080def0 100644
--- a/packages/react/src/html/utils/__tests__/processing.test.mjs
+++ b/packages/react/src/html/utils/__tests__/processing.test.mjs
@@ -8,8 +8,10 @@ import {
import { FONTS } from '../../constants.mjs';
import {
+ buildAssetTags,
buildPreloads,
buildHead,
+ pageFileName,
populateWithEvaluation,
resolvePageRoot,
} from '../processing.mjs';
@@ -196,3 +198,46 @@ describe('buildHead', () => {
assert.strictEqual(buildHead({ meta: [], links: [], html: [] }), '');
});
});
+
+describe('buildAssetTags', () => {
+ const assets = {
+ scripts: ['assets/client-abc.js'],
+ preloads: ['assets/shared-def.js'],
+ stylesheets: ['assets/style-ghi.css'],
+ };
+
+ it('resolves every asset against the page root, scripts first', () => {
+ const tags = buildAssetTags(assets, '../').split('\n');
+
+ assert.deepStrictEqual(
+ tags.map(tag => tag.trim()),
+ [
+ '',
+ ' ',
+ ' ',
+ ]
+ );
+ });
+
+ it('keeps an absolute root absolute', () => {
+ const tags = buildAssetTags(assets, 'https://example.com/docs/');
+
+ assert.ok(
+ tags.includes('src="https://example.com/docs/assets/client-abc.js"')
+ );
+ });
+
+ it('renders nothing for an empty asset list', () => {
+ assert.strictEqual(
+ buildAssetTags({ scripts: [], preloads: [], stylesheets: [] }, './'),
+ ''
+ );
+ });
+});
+
+describe('pageFileName', () => {
+ it('derives the output file from the page path', () => {
+ assert.strictEqual(pageFileName({ path: '/api/fs' }), 'api/fs.html');
+ assert.strictEqual(pageFileName({ path: '/404' }), '404.html');
+ });
+});
diff --git a/packages/react/src/html/utils/all.mjs b/packages/react/src/html/utils/all.mjs
new file mode 100644
index 000000000..f52647400
--- /dev/null
+++ b/packages/react/src/html/utils/all.mjs
@@ -0,0 +1,42 @@
+'use strict';
+
+import { getSortedHeadNodes } from '../../jsx-ast/utils/getSortedHeadNodes.mjs';
+import { createSyntheticHead } from '../../jsx-ast/utils/synthetic/synthetic.mjs';
+
+/**
+ * Builds the `all.html` page from the module pages already generated.
+ *
+ * Its content is exactly the module pages' content, one after the other, so
+ * nothing is rebuilt: the page program imports each module's `content` export
+ * (see `buildPageProgram`) and the table of contents and reading time are the
+ * modules' own, concatenated and summed. Chunk pages and the index are left
+ * out, since the full module pages already carry their content.
+ *
+ * Modules are ordered as the sidebar lists them.
+ *
+ * @param {Array} pages - Every generated page
+ * @returns {import('../types').ComposedPage}
+ */
+export const buildAllPage = pages => {
+ const byApi = new Map(pages.map(page => [page.data.api, page]));
+
+ const parts = getSortedHeadNodes(
+ pages
+ .map(({ data }) => data)
+ .filter(data => !data.synthetic && !data.chunk && data.api !== 'index')
+ ).map(({ api }) => byApi.get(api));
+
+ const minutes = parts.reduce(
+ (sum, { readingTime }) => sum + (readingTime?.minutes ?? 0),
+ 0
+ );
+
+ return {
+ data: createSyntheticHead('all', 'All'),
+ headings: parts.flatMap(({ headings }) => headings),
+ readingTime: parts.some(({ readingTime }) => readingTime)
+ ? { text: `${Math.ceil(minutes)} min read`, minutes }
+ : undefined,
+ parts: parts.map(({ data }) => data.api),
+ };
+};
diff --git a/packages/react/src/html/utils/generate.mjs b/packages/react/src/html/utils/generate.mjs
index 74b4ec1f0..b6a8d7c00 100644
--- a/packages/react/src/html/utils/generate.mjs
+++ b/packages/react/src/html/utils/generate.mjs
@@ -1,8 +1,14 @@
import { resolve } from 'node:path';
import getConfig from '@doc-kit/core/utils/configuration/index.mjs';
+import { omitKeys } from '@doc-kit/core/utils/misc.mjs';
-import { JSX_IMPORTS, ROOT } from '../constants.mjs';
+import {
+ JSX_IMPORTS,
+ JSX_PRAGMA,
+ JSX_PRAGMA_FRAG,
+ ROOT,
+} from '../constants.mjs';
/**
* Normalizes a `components` config entry into the `JSXImportConfig` shape.
@@ -16,6 +22,14 @@ const normalizeComponent = ([tag, value]) =>
? { name: tag, source: value }
: { name: tag, isDefaultExport: true, ...value };
+/**
+ * Quotes a module source for an import/export statement, escaping backslashes
+ * so Windows paths are not treated as escape sequences.
+ *
+ * @param {string} source
+ */
+const quote = source => `"${source.replaceAll('\\', '\\\\')}"`;
+
/**
* Creates an ES Module `import` statement as a string, based on parameters.
*
@@ -29,23 +43,42 @@ export const createImportDeclaration = (
source,
useDefault = true
) => {
- // Escape backslashes to prevent treating them as escape characters
- source = source.replaceAll('\\', '\\\\');
-
// Side-effect-only import (e.g., CSS files)
if (!importName) {
- return `import "${source}";`;
+ return `import ${quote(source)};`;
}
// Default import: import Name from "source"
if (useDefault) {
- return `import ${importName} from "${source}";`;
+ return `import ${importName} from ${quote(source)};`;
}
// Named import: import { Name } from "source"
- return `import { ${importName} } from "${source}";`;
+ return `import { ${importName} } from ${quote(source)};`;
};
+/**
+ * Creates an ES Module re-export statement as a string.
+ *
+ * @param {import('../constants.mjs').JSXImportConfig} component
+ * @returns {string}
+ */
+const createExportDeclaration = ({ name, source, isDefaultExport = true }) =>
+ isDefaultExport
+ ? `export { default as ${name} } from ${quote(source)};`
+ : `export { ${name} } from ${quote(source)};`;
+
+/**
+ * The names a page's program imports from the component library besides the
+ * components it renders: the JSX runtime, the layout, and the renderer.
+ */
+const RUNTIME_IMPORTS = [
+ `h as ${JSX_PRAGMA}`,
+ `Fragment as ${JSX_PRAGMA_FRAG}`,
+ JSX_IMPORTS.Layout.name,
+ 'renderToStringAsync',
+];
+
/**
* Factory function that creates the page programs.
*/
@@ -59,39 +92,66 @@ export default () => {
];
/**
- * Declares only the components a page actually uses.
+ * The server-side component library: one module re-exporting every
+ * component a page may render, Preact's JSX runtime, and the renderer. The
+ * bundler builds it once; every page program imports from the result, so the
+ * components are compiled once rather than once per page.
*
- * @param {Array} imports
- * @param {string} code - The page's JSX expression.
- * @returns {Array}
+ * @returns {string} The library's source.
*/
- const declare = (imports, code) =>
- imports
- .filter(({ name }) => code.includes(`<${name}`))
- .map(({ name, source, isDefaultExport = true }) =>
- createImportDeclaration(name, source, isDefaultExport)
- );
+ const buildLibraryProgram = () =>
+ [
+ ...componentImports.map(createExportDeclaration),
+ 'export { h, Fragment } from "preact";',
+ 'export { renderToStringAsync } from "preact-render-to-string";',
+ ].join('\n');
/**
- * Builds a server-side rendering (SSR) program.
+ * Builds a page's server program: a module exporting the page's `content`
+ * (a function returning the JSX fragment, so each render gets fresh
+ * elements), its `headings`, and a default export rendering the page.
*
- * @param {string} componentCode - JSX component code expression.
- * @returns {string} Complete server-side JavaScript program.
+ * A composed page (`all.html`) has no content of its own: it imports the
+ * `content` of the pages it is made of, so their JSX is never rebuilt.
+ *
+ * @param {import('../types').Page} page
+ * @param {string} libraryURL - Where the built component library was written.
+ * @returns {string} The program, as JSX.
*/
- const buildServerProgram = componentCode => {
+ const buildPageProgram = (page, libraryURL) => {
+ const { data, headings, readingTime } = page;
+
+ const { imports, content } =
+ 'parts' in page
+ ? {
+ imports: page.parts.map(
+ (api, index) =>
+ `import { content as content${index} } from ${quote(`./${moduleFileName(api)}`)};`
+ ),
+ content: `<>${page.parts.map((_, index) => `{content${index}()}`).join('')}>`,
+ }
+ : { imports: [], content: page.content };
+
+ // Import only the components the page actually renders
+ const used = componentImports
+ .filter(({ name }) => content.includes(`<${name}`))
+ .map(({ name }) => name)
+ .filter(name => name !== JSX_IMPORTS.Layout.name);
+
+ // The metadata is the head without the node children
+ const metadata = omitKeys(data, [
+ 'content',
+ 'heading',
+ 'stability',
+ 'changes',
+ ]);
+
return [
- // Import the JSX components this page uses
- ...declare(componentImports, componentCode),
-
- // Import Preact's async SSR render function (named import)
- createImportDeclaration(
- 'renderToStringAsync',
- 'preact-render-to-string',
- false
- ),
-
- // Export a renderer that the server bundler can execute.
- `export default () => renderToStringAsync(${componentCode});`,
+ `import { ${[...RUNTIME_IMPORTS, ...used].join(', ')} } from ${quote(libraryURL)};`,
+ ...imports,
+ `export const headings = ${JSON.stringify(headings)};`,
+ `export const content = () => ${content};`,
+ `export default () => renderToStringAsync(<${JSX_IMPORTS.Layout.name} metadata={${JSON.stringify(metadata)}} headings={headings} readingTime={${JSON.stringify(readingTime?.text)}}>{content()}${JSX_IMPORTS.Layout.name}>);`,
].join('\n');
};
@@ -119,5 +179,13 @@ export default () => {
.join(', ')}});`,
].join('\n');
- return { buildServerProgram, clientProgram };
+ return { buildLibraryProgram, buildPageProgram, clientProgram };
};
+
+/**
+ * The file a page's compiled program is written to, next to the others, so a
+ * composed page can import its parts by relative path.
+ *
+ * @param {string} api
+ */
+export const moduleFileName = api => `${api}.mjs`;
diff --git a/packages/react/src/html/utils/minify.mjs b/packages/react/src/html/utils/minify.mjs
deleted file mode 100644
index 60d73f8c6..000000000
--- a/packages/react/src/html/utils/minify.mjs
+++ /dev/null
@@ -1,49 +0,0 @@
-'use strict';
-
-import { minifyHTML } from '@doc-kit/core/utils/html-minifier.mjs';
-
-/**
- * Minifies a chunk of rendered pages. This is the `html` generator's worker
- * entry point: minifying is the one CPU-bound step of the bundle that scales
- * with the number of pages, and it parallelizes trivially — so it runs in the
- * worker pool rather than serially on the main thread.
- *
- * @param {Array<[string, string]>} pages - `[fileName, html]` pairs
- * @param {Array} indices - The pairs to process
- * @returns {Promise>} The minified pairs
- */
-export const processChunk = (pages, indices) =>
- Promise.all(
- indices.map(async index => {
- const [fileName, html] = pages[index];
-
- return [fileName, await minifyHTML(html)];
- })
- );
-
-/**
- * Creates the page minifier handed to the bundler: it distributes the pages
- * across the worker pool and collects the results as they complete. Without a
- * pool — the generator was called directly rather than by the orchestrator —
- * the pages are minified on the calling thread instead.
- *
- * @param {ParallelWorker} [worker]
- * @returns {import('../types').ClientBundleOptions['minifyPages']}
- */
-export const createPageMinifier = worker => async pages => {
- const items = [...pages];
-
- const chunks = worker
- ? worker.stream(items)
- : [await processChunk(items, [...items.keys()])];
-
- const minified = new Map();
-
- for await (const chunk of chunks) {
- for (const [fileName, html] of chunk) {
- minified.set(fileName, html);
- }
- }
-
- return minified;
-};
diff --git a/packages/react/src/html/utils/processing.mjs b/packages/react/src/html/utils/processing.mjs
index 933e01317..6801abfa1 100644
--- a/packages/react/src/html/utils/processing.mjs
+++ b/packages/react/src/html/utils/processing.mjs
@@ -2,9 +2,7 @@ import getConfig from '@doc-kit/core/utils/configuration/index.mjs';
import { populate } from '@doc-kit/core/utils/configuration/templates.mjs';
import createConfigSource from './config.mjs';
-import createProgramBuilder from './generate.mjs';
import { relativeOrAbsolute } from './relativeOrAbsolute.mjs';
-import { resolveBundler } from '../bundlers/index.mjs';
import { FONT_DIRECTORY, FONTS, SPECULATION_RULES } from '../constants.mjs';
import { THEME_SCRIPT } from '../ui/theme-script.mjs';
@@ -16,7 +14,7 @@ import { THEME_SCRIPT } from '../ui/theme-script.mjs';
* @param {boolean} server
* @returns {Record}
*/
-const createVirtualImports = (datas, virtualImports, server) => ({
+export const createVirtualImports = (datas, virtualImports, server) => ({
...virtualImports,
'#theme/config': createConfigSource(datas, server),
});
@@ -116,113 +114,75 @@ export const buildHead = ({ meta = [], links = [], html = [] }) =>
].join('\n ');
/**
- * Creates an accumulator that wraps per-page JSX code into server and client
- * programs one at a time. The JSX AST has already been serialized to a code
- * string upstream (in the `jsx-ast` worker), so the heavy AST never reaches
- * the main thread — only the code string and page metadata stream in here.
+ * Renders the tags that load a page's client assets, resolved against the
+ * page's root: the entry script, the chunks it statically imports (preloaded,
+ * as the bundler would), and the stylesheets.
*
- * @returns {{ add: (item: { data: import('@doc-kit/core/generators/metadata/types').MetadataEntry, code: string }) => void, serverCodeMap: Map, clientProgram: string }}
+ * @param {import('../types').ClientAssets} assets - Output-relative asset paths
+ * @param {string} root - The page's root (see {@link resolvePageRoot})
+ * @returns {string}
+ */
+export const buildAssetTags = ({ scripts, preloads, stylesheets }, root) =>
+ [
+ ...scripts.map(
+ file => ``
+ ),
+ ...preloads.map(file =>
+ renderTag('link', {
+ rel: 'modulepreload',
+ crossorigin: true,
+ href: `${root}${file}`,
+ })
+ ),
+ ...stylesheets.map(file =>
+ renderTag('link', {
+ rel: 'stylesheet',
+ crossorigin: true,
+ href: `${root}${file}`,
+ })
+ ),
+ ].join('\n ');
+
+/**
+ * The output file of a page, relative to the output directory.
+ *
+ * @param {import('@doc-kit/core/generators/metadata/types').MetadataEntry} data
*/
-export function createCodeConverter() {
- const { buildServerProgram, clientProgram } = createProgramBuilder();
-
- const serverCodeMap = new Map();
-
- return {
- /**
- * Records the server program for a single page's JSX code.
- *
- * @param {{ data: import('@doc-kit/core/generators/metadata/types').MetadataEntry, code: string }} item
- */
- add: ({ data, code }) => {
- // Prepare code for server-side execution (wrapped for SSR)
- serverCodeMap.set(`${data.api}.jsx`, buildServerProgram(code));
- },
- serverCodeMap,
- // The client entry is the same module for every page: the pages differ
- // only in their server-rendered markup, which the entry hydrates.
- clientProgram,
- };
-}
+export const pageFileName = data => `${data.path.replace(/^\/+/, '')}.html`;
/**
- * Bundles pre-converted JSX code into complete HTML pages and client assets.
- * Conversion (JSX AST → code) happens upstream via
- * {@link createCodeConverter} so the heavy ASTs are already discarded; this
- * step needs every entry together for code-splitting and the shared sidebar.
+ * Populates the HTML template for one rendered page.
*
* @param {object} params
- * @param {Map} params.serverCodeMap - Server-side code per page.
- * @param {string} params.clientProgram - The client entry shared by every page.
- * @param {Array} params.datas - Per-page metadata, in render order.
- * @param {string} params.template - The HTML template string for the output pages.
- * @param {import('../types').ClientBundleOptions['minifyPages']} params.minifyPages - Minifies the final pages, off the main thread.
+ * @param {string} params.template - The HTML template
+ * @param {import('@doc-kit/core/generators/metadata/types').MetadataEntry} params.data - The page's metadata
+ * @param {string} params.dehydrated - The server-rendered page
+ * @param {import('../types').ClientAssets} params.assets - The client assets every page loads
+ * @returns {string}
*/
-export async function processBundles({
- serverCodeMap,
- clientProgram,
- datas,
- template,
- minifyPages,
-}) {
+export const populatePage = ({ template, data, dehydrated, assets }) => {
const config = getConfig('html');
- const bundler = await resolveBundler(config.bundler);
-
- const serverPages = await bundler.render({
- entries: serverCodeMap,
- virtualImports: createVirtualImports(datas, config.virtualImports, true),
- config,
- });
const titleSuffix = populate(config.title, {
...config,
version: config.version.version,
});
- // Pre-render the configurable `` markup once, since it is identical
- // across every page. Computed here (rather than inline in the template) so
- // template authors avoid nested template-literal escaping.
- const head = buildHead(config.head);
-
- // Render the templates with the client identifier supplied by the adapter.
- // The adapter then owns scripts, stylesheets, preloads, and imported assets.
- const entrypoint = bundler.getEntryId();
-
- const pages = new Map(
- datas.map(data => {
- const root = resolvePageRoot(data);
- const title = data.title ?? data.heading.data.name;
- const fileName = `${data.path.replace(/^\/+/, '')}.html`;
-
- return [
- fileName,
- populateWithEvaluation(template, {
- title: escapeHTML(
- title
- ? titleSuffix
- ? `${title} | ${titleSuffix}`
- : title
- : titleSuffix
- ),
- dehydrated: serverPages.get(data.api) ?? '',
- entrypoint,
- speculationRules: SPECULATION_RULES,
- themeScript: THEME_SCRIPT,
- preloads: buildPreloads(root),
- root,
- metadata: data,
- config,
- head,
- }),
- ];
- })
- );
-
- await bundler.build({
- entry: clientProgram,
- virtualImports: createVirtualImports(datas, config.virtualImports, false),
- pages,
- minifyPages,
+ const root = resolvePageRoot(data);
+ const title = data.title ?? data.heading.data.name;
+
+ return populateWithEvaluation(template, {
+ title: escapeHTML(
+ title ? (titleSuffix ? `${title} | ${titleSuffix}` : title) : titleSuffix
+ ),
+ dehydrated,
+ assets: buildAssetTags(assets, root),
+ speculationRules: SPECULATION_RULES,
+ themeScript: THEME_SCRIPT,
+ preloads: buildPreloads(root),
+ root,
+ metadata: data,
config,
+ head: buildHead(config.head),
});
-}
+};
diff --git a/packages/react/src/html/utils/render.mjs b/packages/react/src/html/utils/render.mjs
new file mode 100644
index 000000000..aa6beca9c
--- /dev/null
+++ b/packages/react/src/html/utils/render.mjs
@@ -0,0 +1,84 @@
+'use strict';
+
+import { mkdir, writeFile } from 'node:fs/promises';
+import { dirname, join } from 'node:path';
+
+import logger from '@doc-kit/core/logger/index.mjs';
+import getConfig from '@doc-kit/core/utils/configuration/index.mjs';
+import { minifyHTML } from '@doc-kit/core/utils/html-minifier.mjs';
+
+import { pageFileName, populatePage } from './processing.mjs';
+
+const renderLogger = logger.child('html');
+
+/**
+ * Renders and writes a chunk of pages. This is the `html` generator's worker
+ * entry point.
+ *
+ * Each page is a compiled program on disk (see `buildPageProgram`): it is
+ * imported, rendered to HTML, placed in the template, minified when configured,
+ * and written to the output directory — one page at a time, and nothing comes
+ * back but the file name. A worker therefore holds one page at once, plus the
+ * component library it imported the first time, and the whole run's memory
+ * scales with the largest page rather than with the site.
+ *
+ * @param {Array} tasks
+ * @param {Array} indices - The tasks to process
+ * @param {{ template: string, assets: import('../types').ClientAssets }} extra
+ * @returns {Promise>} The written file names
+ */
+export const processChunk = async (tasks, indices, { template, assets }) => {
+ const config = getConfig('html');
+
+ const written = [];
+
+ for (const index of indices) {
+ const { moduleURL, data } = tasks[index];
+
+ const { default: render } = await import(moduleURL);
+
+ let html = populatePage({
+ template,
+ data,
+ dehydrated: await render(),
+ assets,
+ });
+
+ if (config.minify) {
+ html = await minifyHTML(html);
+ }
+
+ const fileName = pageFileName(data);
+ const path = join(config.output, fileName);
+
+ await mkdir(dirname(path), { recursive: true });
+ await writeFile(path, html);
+
+ written.push(fileName);
+ }
+
+ return written;
+};
+
+/**
+ * Creates the page writer: it spreads the pages across the worker pool and
+ * waits for every one to be written. Without a pool — the generator was called
+ * directly rather than by the orchestrator — the pages are rendered on the
+ * calling thread instead.
+ *
+ * @param {ParallelWorker} [worker]
+ * @returns {(tasks: Array, extra: { template: string, assets: import('../types').ClientAssets }) => Promise}
+ */
+export const createPageWriter = worker => async (tasks, extra) => {
+ const chunks = worker
+ ? worker.stream(tasks, extra)
+ : [await processChunk(tasks, [...tasks.keys()], extra)];
+
+ let count = 0;
+
+ for await (const chunk of chunks) {
+ count += chunk.length;
+
+ renderLogger.debug(`Wrote ${count}/${tasks.length} pages`);
+ }
+};
diff --git a/packages/react/src/jsx-ast/README.md b/packages/react/src/jsx-ast/README.md
index 66369b4c2..aab9b9f8a 100644
--- a/packages/react/src/jsx-ast/README.md
+++ b/packages/react/src/jsx-ast/README.md
@@ -8,13 +8,20 @@ The `jsx-ast` generator converts MDAST (Markdown Abstract Syntax Tree) to JSX AS
**Default:** `'main'`.
- `index` {Array} Array of `{ section, api }` objects defining the
documentation structure.
-- `generateAllPage` {boolean} When `true`, creates a synthetic JSX AST entry
- for `all.html`. **Default:** `true`.
- `generateNotFoundPage` {boolean} When `true`, creates a synthetic JSX AST
entry for `404.html`. **Default:** `true`.
- `showReadingTime` {boolean} When `true`, computes an estimated reading time
for each page and displays it in the MetaBar. **Default:** `false`.
+## Output
+
+Each page is emitted as `{ data, headings, readingTime, content }`: the page's
+head entry, its table of contents, the optional reading time, and the processed
+content serialized to JSX code as one fragment. The page layout is not part of
+the content — the `html` generator wraps each page in ``, and assembles
+`all.html` from the module pages' content (see its `generateAllPage` option)
+rather than building every module a second time here.
+
## Index page
`index.html` is generated when an `index` document is part of the input, and
diff --git a/packages/react/src/jsx-ast/__tests__/generate.test.mjs b/packages/react/src/jsx-ast/__tests__/generate.test.mjs
index 7d09db400..40466dc41 100644
--- a/packages/react/src/jsx-ast/__tests__/generate.test.mjs
+++ b/packages/react/src/jsx-ast/__tests__/generate.test.mjs
@@ -60,22 +60,29 @@ const createWorker = seenItems => ({
});
describe('jsx-ast generate', () => {
- it('does not attach raw section entries to regular JSX content', async () => {
+ it('returns the page content as a JSX fragment alongside its ToC', async () => {
await setConfig({ target: ['jsx-ast'] });
const fs = createEntry('fs', 'File system');
- const [content] = await processChunk([{ head: fs, entries: [fs] }], [0]);
+ const [page] = await processChunk([{ head: fs, entries: [fs] }], [0]);
- assert.equal(content.data.api, 'fs');
- assert.equal('sectionEntries' in content, false);
+ assert.equal(page.data.api, 'fs');
+ assert.equal('sectionEntries' in page, false);
+ // The layout is the html generator's: only the content is serialized
+ assert.match(page.content, /^<>/);
+ assert.doesNotMatch(page.content, / value),
+ ['File system']
+ );
+ assert.equal(page.readingTime, undefined);
});
it('respects jsx-ast synthetic page flags', async () => {
await setConfig({ target: ['jsx-ast'] });
- const jsxAstConfig = getConfig('jsx-ast');
- jsxAstConfig.generateAllPage = false;
- jsxAstConfig.generateNotFoundPage = false;
+ getConfig('jsx-ast').generateNotFoundPage = false;
const seenItems = [];
const results = await collect(
diff --git a/packages/react/src/jsx-ast/generate.mjs b/packages/react/src/jsx-ast/generate.mjs
index bd073c980..fc5e77c18 100644
--- a/packages/react/src/jsx-ast/generate.mjs
+++ b/packages/react/src/jsx-ast/generate.mjs
@@ -5,34 +5,27 @@ import { jsx, toJs } from 'estree-util-to-js';
import buildContent from './utils/buildContent.mjs';
import { getSortedHeadNodes } from './utils/getSortedHeadNodes.mjs';
import { buildNotFoundPage } from './utils/synthetic/404.mjs';
-import { buildAllPage } from './utils/synthetic/all.mjs';
/**
- * Builds the `{ head, entries }` page descriptors for all configured synthetic
- * pages. The descriptors are cheap to build; the expensive `buildContent` step
- * runs later in a worker (via `processChunk`), so the very large synthetic
- * `all` page is never built on the main thread.
- *
- * @param {Array} input
+ * Builds the `{ head, entries }` page descriptors for the configured synthetic
+ * pages. `all.html` is not one of them: it is the module pages concatenated,
+ * so the `html` generator assembles it from their content instead of building
+ * every module a second time here.
*/
-const buildSyntheticDescriptors = input => {
+const buildSyntheticDescriptors = () => {
const config = getConfig('jsx-ast');
- return [
- config.generateAllPage && buildAllPage(input),
- config.generateNotFoundPage && buildNotFoundPage(),
- ].filter(Boolean);
+ return config.generateNotFoundPage ? [buildNotFoundPage()] : [];
};
/**
* Process a chunk of items in a worker thread.
*
- * Each item is a `{ head, entries }` descriptor (one module, or a synthetic
- * page). The JSX AST is built AND serialized to a code string here, inside the
- * worker, so the heavy AST — most notably the giant `all` page, which
- * concatenates every module — is dropped in the worker and never crosses back
- * to or accumulates on the main thread. Only the much smaller code string and
- * the page metadata are returned.
+ * Each item is a `{ head, entries }` descriptor (one module, one chunk page, or
+ * a synthetic page). The JSX AST is built AND serialized to a code string here,
+ * inside the worker, so the heavy AST is dropped in the worker and never
+ * crosses back to or accumulates on the main thread. Only the code string, the
+ * table of contents and the page metadata are returned.
*
* @type {import('./types').Generator['processChunk']}
*/
@@ -42,11 +35,9 @@ export async function processChunk(slicedInput, itemIndices) {
for (const idx of itemIndices) {
const { head, entries } = slicedInput[idx];
- const content = await buildContent(entries, head);
-
- const { value: code } = toJs(content, { handlers: jsx });
+ const { content, ...page } = await buildContent(entries, head);
- results.push({ data: content.data, code });
+ results.push({ ...page, content: toJs(content, { handlers: jsx }).value });
}
return results;
@@ -58,13 +49,6 @@ export async function processChunk(slicedInput, itemIndices) {
* @type {import('./types').Generator['generate']}
*/
export async function* generate(input, worker) {
- // The `index` page is only generated when an `index` document is part of
- // the input; the module list for the synthetic pages excludes it, as well
- // as chunk pages, whose content the full module pages already carry.
- const moduleInput = input.filter(
- entry => entry.api !== 'index' && !entry.chunk
- );
-
// Create sliced input: each item contains head + its module's entries
// This avoids sending all 4700+ entries to every worker
const groupedModules = groupNodesByModule(input);
@@ -73,9 +57,7 @@ export async function* generate(input, worker) {
entries: groupedModules.get(head.api),
}));
- // Process the synthetic pages through the worker pool as well, so their
- // (potentially enormous) content is built and converted off the main thread.
- descriptors.push(...buildSyntheticDescriptors(moduleInput));
+ descriptors.push(...buildSyntheticDescriptors());
for await (const chunkResult of worker.stream(descriptors)) {
yield chunkResult;
diff --git a/packages/react/src/jsx-ast/index.mjs b/packages/react/src/jsx-ast/index.mjs
index ab2026806..191e1934f 100644
--- a/packages/react/src/jsx-ast/index.mjs
+++ b/packages/react/src/jsx-ast/index.mjs
@@ -16,7 +16,6 @@ export default {
defaultConfiguration: {
ref: 'main',
- generateAllPage: true,
generateNotFoundPage: true,
showReadingTime: false,
},
diff --git a/packages/react/src/jsx-ast/types.d.ts b/packages/react/src/jsx-ast/types.d.ts
index 9d630e1a2..f70d84088 100644
--- a/packages/react/src/jsx-ast/types.d.ts
+++ b/packages/react/src/jsx-ast/types.d.ts
@@ -1,16 +1,15 @@
import type { MetadataEntry } from '@doc-kit/core/generators/metadata/types';
-import type { JSXContent } from './utils/buildContent.mjs';
+import type { PageContent } from './utils/buildContent.mjs';
+
+// What the worker returns for a page: the fragment serialized to JSX code.
+export type PageCode = Omit & { content: string };
export type Generator = GeneratorMetadata<
{
ref: string;
- generateAllPage: boolean;
generateNotFoundPage: boolean;
showReadingTime: boolean;
},
- Generate, AsyncGenerator>,
- ProcessChunk<
- { head: MetadataEntry; entries: Array },
- JSXContent
- >
+ Generate, AsyncGenerator>,
+ ProcessChunk<{ head: MetadataEntry; entries: Array }, PageCode>
>;
diff --git a/packages/react/src/jsx-ast/utils/buildContent.mjs b/packages/react/src/jsx-ast/utils/buildContent.mjs
index 17eb44f16..f33e6deb7 100644
--- a/packages/react/src/jsx-ast/utils/buildContent.mjs
+++ b/packages/react/src/jsx-ast/utils/buildContent.mjs
@@ -7,7 +7,6 @@ import {
populate,
} from '@doc-kit/core/utils/configuration/templates.mjs';
import { parseInline } from '@doc-kit/core/utils/inline.mjs';
-import { omitKeys } from '@doc-kit/core/utils/misc.mjs';
import { UNIST } from '@doc-kit/core/utils/queries/index.mjs';
import { transformNodesToString } from '@doc-kit/core/utils/unist.mjs';
import { h as createElement } from 'hastscript';
@@ -38,10 +37,19 @@ import {
} from './signature.mjs';
/**
+ * Estimates the reading time of a page's text. Both the display text and the
+ * raw minutes are kept: pages assembled from several others (`all.html`) sum
+ * the minutes rather than re-reading the text.
*
+ * @param {string} text
+ * @returns {Promise<{ text: string, minutes: number }>}
*/
const readingTime = text =>
- import('reading-time').then(({ default: rt }) => rt(text).text);
+ import('reading-time').then(({ default: rt }) => {
+ const { text: display, minutes } = rt(text);
+
+ return { text: display, minutes };
+ });
/**
* Processes lifecycle and change history data into a sorted array of change entries.
@@ -321,54 +329,58 @@ export const processEntry = entry => {
};
/**
- * Builds the overall document layout tree
+ * Builds a page's content: every entry processed and wrapped in one JSX
+ * fragment, plus the table of contents and reading time the layout needs.
+ *
+ * The layout itself (``) is not part of the content. The `html`
+ * generator wraps each page in it, which lets a page be assembled from other
+ * pages' content — `all.html` is the module pages concatenated — without
+ * building those modules a second time.
+ *
* @param {Array} entries - API documentation metadata entries
- * @param {Object} metadata - Raw page metadata from the head entry
*/
-export const createDocumentLayout = async (entries, metadata) => {
+export const createDocumentContent = async entries => {
// Collapse overloaded function headings into one stable ToC entry, tagging the
// underlying headings with compact anchors / overload flags read just below.
annotateOverloads(entries);
const { showReadingTime } = getConfig('jsx-ast');
- return createTree('root', [
- createJSXElement(JSX_IMPORTS.Layout.name, {
- metadata,
- headings: extractHeadings(entries),
- readingTime: showReadingTime
- ? await readingTime(extractTextContent(entries))
- : undefined,
- children: entries.map(processEntry),
- }),
- ]);
+ return {
+ headings: extractHeadings(entries),
+ readingTime: showReadingTime
+ ? await readingTime(extractTextContent(entries))
+ : undefined,
+ root: createTree('root', [
+ createJSXElement(null, {
+ inline: false,
+ children: entries.map(processEntry),
+ }),
+ ]),
+ };
};
/**
- * @typedef {import('estree').Node & { data: import('@doc-kit/core/generators/metadata/types').MetadataEntry }} JSXContent
+ * @typedef {Object} PageContent
+ * @property {import('@doc-kit/core/generators/metadata/types').MetadataEntry} data - The page's head entry
+ * @property {Array[number]>} headings - The table of contents
+ * @property {{ text: string, minutes: number } | undefined} readingTime - Set when `showReadingTime` is on
+ * @property {import('estree-jsx').JSXFragment} content - The processed entries, as one JSX fragment
*
- * Transforms API metadata entries into processed MDX content
+ * Transforms API metadata entries into a page's JSX content
* @param {Array} metadataEntries - API documentation metadata entries
* @param {import('@doc-kit/core/generators/metadata/types').MetadataEntry} head - Main API metadata entry with version information
- * @returns {Promise}
+ * @returns {Promise}
*/
const buildContent = async (metadataEntries, head) => {
- // The metadata is the heading without the node children
- const metadata = omitKeys(head, [
- 'content',
- 'heading',
- 'stability',
- 'changes',
- ]);
-
- // Create root document AST with all layout components and processed content
- const root = await createDocumentLayout(metadataEntries, metadata);
+ const { headings, readingTime, root } =
+ await createDocumentContent(metadataEntries);
// Run remark processor to transform AST (parse markdown, plugins, etc.)
const ast = await remark().run(root);
- // The final MDX content is the expression in the Program's first body node
- return { ...ast.body[0].expression, data: head };
+ // The fragment is the expression in the Program's first body node
+ return { data: head, headings, readingTime, content: ast.body[0].expression };
};
export default buildContent;
diff --git a/packages/react/src/jsx-ast/utils/plugins/__tests__/transformer.test.mjs b/packages/react/src/jsx-ast/utils/plugins/__tests__/transformer.test.mjs
index 4b7df0720..0af3f6a8d 100644
--- a/packages/react/src/jsx-ast/utils/plugins/__tests__/transformer.test.mjs
+++ b/packages/react/src/jsx-ast/utils/plugins/__tests__/transformer.test.mjs
@@ -4,10 +4,10 @@ import { describe, it } from 'node:test';
import transformer from '../transformer.mjs';
describe('jsx-ast transformer', () => {
- it('moves generated footnotes into the Layout children', () => {
- const layout = {
- type: 'mdxJsxTextElement',
- name: 'Layout',
+ it('moves generated footnotes into the page content fragment', () => {
+ const content = {
+ type: 'mdxJsxFlowElement',
+ name: null,
children: [{ type: 'element', tagName: 'p', children: [] }],
};
const footnotes = {
@@ -31,12 +31,12 @@ describe('jsx-ast transformer', () => {
};
const tree = {
type: 'root',
- children: [layout, { type: 'text', value: '\n' }, footnotes],
+ children: [content, { type: 'text', value: '\n' }, footnotes],
};
transformer()(tree);
assert.equal(tree.children.includes(footnotes), false);
- assert.equal(layout.children.at(-1), footnotes);
+ assert.equal(content.children.at(-1), footnotes);
});
});
diff --git a/packages/react/src/jsx-ast/utils/plugins/transformer.mjs b/packages/react/src/jsx-ast/utils/plugins/transformer.mjs
index 7d46e3abf..819b16307 100644
--- a/packages/react/src/jsx-ast/utils/plugins/transformer.mjs
+++ b/packages/react/src/jsx-ast/utils/plugins/transformer.mjs
@@ -14,11 +14,17 @@ const isFootnotesSection = node =>
node.properties?.className?.includes('footnotes'));
/**
- * Finds the generated page Layout node.
+ * Finds the JSX fragment wrapping the page content (see `buildContent`).
* @param {import('hast').Root} tree
*/
-const findLayout = tree =>
- tree.children.find(node => node.name === 'Layout' && node.children);
+const findPageContent = tree =>
+ tree.children.find(
+ node =>
+ (node.type === 'mdxJsxFlowElement' ||
+ node.type === 'mdxJsxTextElement') &&
+ node.name === null &&
+ node.children
+ );
/**
* @template {import('unist').Node} T
@@ -64,10 +70,10 @@ const transformer = tree => {
if (index !== -1) {
const [section] = tree.children.splice(index, 1);
- const layout = findLayout(tree);
+ const content = findPageContent(tree);
- if (layout) {
- layout.children.push(section);
+ if (content) {
+ content.children.push(section);
} else {
tree.children.push(section);
}
diff --git a/packages/react/src/jsx-ast/utils/remark.mjs b/packages/react/src/jsx-ast/utils/remark.mjs
index 6a0b7426b..59bafd01c 100644
--- a/packages/react/src/jsx-ast/utils/remark.mjs
+++ b/packages/react/src/jsx-ast/utils/remark.mjs
@@ -2,7 +2,7 @@
import { highlighter } from '@doc-kit/core/utils/highlighter.mjs';
import { lazy } from '@doc-kit/core/utils/misc.mjs';
-import { typeAnnotationToHighlightedHast } from '@doc-kit/core/utils/type-annotations/hast.mjs';
+import { typeAnnotationToHighlightedHast } from '@doc-kit/core/utils/type-annotations/highlighted.mjs';
import rehypeShikiji from '@node-core/rehype-shiki/plugin';
import recmaJsx from 'recma-jsx';
import recmaStringify from 'recma-stringify';
diff --git a/packages/react/src/jsx-ast/utils/synthetic/__tests__/all.test.mjs b/packages/react/src/jsx-ast/utils/synthetic/__tests__/all.test.mjs
deleted file mode 100644
index 4c3daf4d3..000000000
--- a/packages/react/src/jsx-ast/utils/synthetic/__tests__/all.test.mjs
+++ /dev/null
@@ -1,33 +0,0 @@
-import assert from 'node:assert/strict';
-import { describe, it } from 'node:test';
-
-import { buildAllPage } from '../all.mjs';
-
-describe('buildAllPage', () => {
- it('returns a synthetic `all` head with an "All" heading', () => {
- const { head } = buildAllPage([]);
-
- assert.equal(head.api, 'all');
- assert.equal(head.path, '/all');
- assert.equal(head.basename, 'all');
- assert.equal(head.heading.data.name, 'All');
- assert.equal(head.synthetic, true);
- });
-
- it('forwards the input entries as the page entries', () => {
- const a = { api: 'fs', heading: { depth: 1, data: {} } };
- const b = { api: 'http', heading: { depth: 1, data: {} } };
-
- const { entries } = buildAllPage([a, b]);
-
- assert.deepEqual(entries, [a, b]);
- });
-
- it('does not mutate the input array', () => {
- const input = [{ api: 'fs' }];
-
- buildAllPage(input);
-
- assert.equal(input.length, 1);
- });
-});
diff --git a/packages/react/src/jsx-ast/utils/synthetic/all.mjs b/packages/react/src/jsx-ast/utils/synthetic/all.mjs
deleted file mode 100644
index 1185dcd08..000000000
--- a/packages/react/src/jsx-ast/utils/synthetic/all.mjs
+++ /dev/null
@@ -1,13 +0,0 @@
-'use strict';
-
-import { createSyntheticHead } from './synthetic.mjs';
-
-/**
- * Builds the page descriptor for `all.html`
- *
- * @param {Array} entries
- */
-export const buildAllPage = entries => ({
- head: createSyntheticHead('all', 'All'),
- entries,
-});