diff --git a/.changeset/effect-filesystem-phase2-dev-server.md b/.changeset/effect-filesystem-phase2-dev-server.md new file mode 100644 index 000000000..9dfddcc8b --- /dev/null +++ b/.changeset/effect-filesystem-phase2-dev-server.md @@ -0,0 +1,5 @@ +--- +"agent-bundle": patch +--- + +Keep `agent-bundle dev` — the Workbench dev server, its host installs, MCP sessions and probes, hook / host-discovery / native / script playgrounds, skill documents, evals, and asset serving — behaving exactly as before while every service's file reads, temporary directories, and removals run on one platform runtime that `startDevServer` creates and the session's `close` releases after the last service has closed. Diagnostics, routes, and responses are unchanged. Two lifetimes are now explicit: an MCP session's plugin-data directory lives exactly as long as the session (removed when the session closes, or when an open fails before the session exists), and a script playground workspace that cannot be removed is still reported in the run result's `cleanupFailures` instead of replacing the script's outcome. `agent-bundle --version` / `--help` keep loading no Effect module. (#551) diff --git a/docs/effect-conventions.md b/docs/effect-conventions.md index 90967f44d..996242b1f 100644 --- a/docs/effect-conventions.md +++ b/docs/effect-conventions.md @@ -411,9 +411,23 @@ the first-party CLI's user-facing text — see `NodeTerminal` + `NodeStdio`, built on the first command write, see [Terminal and Stdio](#terminal-and-stdio-user-facing-cli-text)) and widens to `platformLayer` there when CLI code adopts the filesystem services; the - dev server (phase 2) gets one `makeScopedEffectRuntime(platformLayer)` in - `startDevServer`, disposed from the session's `close`. Never provide a - platform layer deep inside library code. + dev server has one `makeScopedEffectRuntime(platformLayer)`, created + inside `startDevServer` (never at module top level — `effect` is a CLI + cold-start cost) and disposed from the returned session's `close` after + every service has closed (`createDevPlatformRuntime` in + `src/dev/platform-run.ts`). Every dev service takes the runtime as an + optional `platformRuntime` constructor option typed as `DevPlatformRuntime` + (`src/dev/platform-runtime.ts`), a deliberately Effect-free handle with only + `close()`: service options sit on the package's public declaration graph, + which `public-api.test.ts` keeps free of `effect` imports. The service's + implementation resolves the handle to its `PlatformRun` edge with + `platformRunOf(options.platformRuntime)` (`runWithPlatform`'s signature + over the long-lived runtime, `PlatformError` unwrapped the same way); + absent a handle, `platformRunOf` returns `runWithPlatform`, so the services + stay usable on their own. Both modules live under `dev/`, not in + `platform.ts` — that module is bundled into the emitted installer, which + stays byte-identical. Never provide a platform layer deep inside library + code. - Errors: `PlatformError` flows through the Effect error channel and is mapped once, at the boundary, onto the existing contract. Where a user-facing AB#### diagnostic already exists for the failure, map to it @@ -450,6 +464,22 @@ the first-party CLI's user-facing text — see synchronous `existsSync` probe beside `createRequire`'s synchronous resolution. - Synchronous SQLite setup (`rsc-runtime/src/state/sqlite.ts`). +- Dev-server durable protocols and identity checks: `dev/epoch-store.ts`, + `dev/dev-lock.ts`, `dev/runtime-generation-store.ts`'s publish path + (`wx` manifest, non-recursive `mkdir`, `rename`, `lstat`-verified assets; + only its manifest / asset reads and recursive `mkdir`s are programs), + `dev/playground/native-playground-service.ts`'s catalog publication + (`link` / `open` / `rename` / `lstat`, rollback quarantine), + `dev/host-install-manager.ts`'s `lstat` rows, `mkdtemp` + `cp` + (`verbatimSymlinks`, `errorOnExist`) staging and atomic `rename` swap, + `dev/eval/eval-service.ts`'s `O_NOFOLLOW` evidence reader, the + `lstat` / `realpath` containment in `dev/skill-document-service.ts` and + `dev/project-service.ts`, `dev/package-build-service.ts`'s `rmdir` + pruning (no `FileSystem` equivalent with its "not empty" contract), and + `dev/playground/mcp-probe-service.ts`'s retrying teardown `rm` + (`maxRetries` / `retryDelay` have no `FileSystem.remove` option), and + `dev/playground/lifecycle-replay-service.ts`'s synchronous `existsSync` + probe. - `dev/watcher.ts`: chokidar stays. `FileSystem.watch` is a thin `fs.watch` with create/update/remove only — no `ignored` callbacks, readiness, or the other event kinds — and the watcher's `dev:ino` signatures need `stat` @@ -511,9 +541,28 @@ carve-outs above. The sibling `routes/graph.ts` reads stay raw: only its two async reads would add an Effect runtime per compile for nothing. Emitted artifacts, hook wrappers, compiler hot paths, and the modules `cli.ts` loads eagerly never import this module (`cli.test.ts` -fails if `--version` / `--help` resolve an `effect` module); the dev server -picks it up in phase 2's second PR through -`makeScopedEffectRuntime(platformLayer)`. +fails if `--version` / `--help` resolve an `effect` module). The dev server +(phase 2, second PR, 2026-09-03) runs on one +`makeScopedEffectRuntime(platformLayer)` created in `startDevServer` +(`dev/platform-run.ts`) and handed to every service as the Effect-free +`platformRuntime` handle (`dev/platform-runtime.ts`), resolved to its edge +with `platformRunOf`: the ordinary reads, temp directories, and removals of +`dev/project-service.ts`, `package-build-service.ts`, +`host-install-manager.ts`, `skill-document-service.ts`, +`workbench-assets.ts`, `runtime-provider-loader.ts`, +`playground/{hook,host-discovery,mcp-probe,native,script}-playground-service.ts`, +and `eval/eval-service.ts`. `runtime-generation-store.ts` reads through +`FileSystem` too but on `runWithPlatform`: providers construct it through +the public `createRuntimeGenerationStore` factory, whose effect-free options +contract (`runtime-store-contracts.ts`, exported from `agent-bundle/api`) +has no session runtime to hand it. Two directories outlive their call and are +therefore not `withTempDirectory` brackets: the MCP session's plugin-data +directory is acquired into its own session-lifetime `Scope` whose only +finalizer removes it — the session closes that scope from `close()`, and +until the session exists the open scope's release closes it instead — and +the script playground's workspace lease keeps `close` as a separate step so +a removal failure is reported in the result's `cleanupFailures`, not in +place of the script's outcome. ### Terminal and Stdio: user-facing CLI text @@ -688,7 +737,7 @@ wire contracts](#effect-schema-wire-contracts-schema-projections). | Module | Adopted in | Re-verify | | --- | --- | --- | | `effect/unstable/reactivity` (+ `@effect/atom-react` bindings) | Workbench Agent Document panel (#105 phase 1) and route editor (#105 phase 2) | re-pin bumps @effect/atom-react in lockstep; re-run disposal regression + bundle measurement; stream-backed derived atoms stay banned until the rc.112 disposal fix ships | -| `@effect/platform-node` (`NodeServices.layer`, `create-agent-bundle`) and `@effect/platform-node-shared` (`agent-bundle`'s `platformLayer`); `FileSystem` / `Path` services live in `effect` | **adopted** (2026-09-03) for ordinary I/O — `create-agent-bundle` scaffolder and the `agent-bundle` temp directories in `api.ts` / the Codex validator (phase 1); host-contracts validators, `services/*`, `eval/*`, and the post-build readers (phase 2, ordinary-I/O modules, 2026-09-03); see [Effect platform services](#effect-platform-services-effectplatform-node) for the keep-raw list and the consumer-footprint reason for the split | re-pin bumps both in lockstep with `effect`; re-check whether `@effect/platform-node` still forces a `redis` peer (if it stops, `agent-bundle` can move to `NodeServices.layer`); re-check whether `lstat` / `O_NOFOLLOW` / directory fsync landed (would shrink the keep-raw list) and the `runMain` 130/143 exit contract | +| `@effect/platform-node` (`NodeServices.layer`, `create-agent-bundle`) and `@effect/platform-node-shared` (`agent-bundle`'s `platformLayer`); `FileSystem` / `Path` services live in `effect` | **adopted** (2026-09-03) for ordinary I/O — `create-agent-bundle` scaffolder and the `agent-bundle` temp directories in `api.ts` / the Codex validator (phase 1); host-contracts validators, `services/*`, `eval/*`, and the post-build readers (phase 2, ordinary-I/O modules, 2026-09-03); the dev server's services on one session-scoped runtime created in `startDevServer` (phase 2, dev server, 2026-09-03); see [Effect platform services](#effect-platform-services-effectplatform-node) for the keep-raw list and the consumer-footprint reason for the split | re-pin bumps both in lockstep with `effect`; re-check whether `@effect/platform-node` still forces a `redis` peer (if it stops, `agent-bundle` can move to `NodeServices.layer`); re-check whether `lstat` / `O_NOFOLLOW` / directory fsync landed (would shrink the keep-raw list) and the `runMain` 130/143 exit contract | | `@effect/platform-node-shared` (`NodeTerminal` / `NodeStdio`) + `effect/Terminal`, `effect/Stdio` | first-party CLI command output, diagnostics, and machine output (`src/cli.ts`, `src/effect/terminal.ts`, `src/effect/cli-runtime.ts`), loaded lazily on the first command write (2026-09-03); Commander's help/version/argv-error text and the scaffolder's `--help` / flag-error text stay on synchronous process writes for the cold-start budget | re-pin re-checks `Terminal.display` stays stdout-only, `readLine` EOF → `QuitError`, the `Stdio` sink contract, and re-measures `agent-bundle --version` startup against the recorded ≈60 ms (`cli.test.ts` fails the build if the trivial invocations resolve an `effect` module) | | `Schema` / `SchemaAST` / `SchemaParser` projections (`toType` / `toEncoded`) for wire contracts | **declined** (2026-09-01) | revisit at Effect GA or on the first encoded/decoded-divergent wire contract; re-pin re-checks the projections API and the `onExcessProperty` parse-option default | diff --git a/packages/agent-bundle/src/dev/eval/eval-service.ts b/packages/agent-bundle/src/dev/eval/eval-service.ts index 01d45c913..2d7f4b193 100644 --- a/packages/agent-bundle/src/dev/eval/eval-service.ts +++ b/packages/agent-bundle/src/dev/eval/eval-service.ts @@ -1,10 +1,15 @@ import { createHash } from 'node:crypto'; import { constants, type Stats } from 'node:fs'; -import { lstat, open, realpath, rm } from 'node:fs/promises'; +import { lstat, open, realpath } from 'node:fs/promises'; import { basename, dirname, join, relative, resolve } from 'node:path'; import { Readable } from 'node:stream'; +import { Effect, FileSystem } from 'effect'; + import { createDefaultRegistry, type TargetRegistry } from '../../adapters/registry.ts'; +import { type PlatformRun } from '../../effect/platform.ts'; +import { platformRunOf } from '../platform-run.ts'; +import type { DevPlatformRuntime } from '../platform-runtime.ts'; import { loadConfig } from '../../config/load.ts'; import type { Diagnostic } from '../../core/diagnostics.ts'; import { digest } from '../../core/digest.ts'; @@ -152,6 +157,8 @@ export interface EvalServiceOptions { readonly now?: () => Date; readonly projectRoot: string; readonly registry?: TargetRegistry; + /** The dev server's session runtime; absent, each program runs on its own `platformLayer`. */ + readonly platformRuntime?: DevPlatformRuntime; readonly targets?: readonly string[]; } @@ -447,6 +454,7 @@ export class EvalService { readonly #now: () => Date; readonly #projectRoot: string; readonly #registry: TargetRegistry; + readonly #run: PlatformRun; readonly #targets: readonly string[] | undefined; readonly #eventSubscriptions = new Map>(); readonly #activeRuns = new Map(); @@ -461,6 +469,7 @@ export class EvalService { this.#now = options.now ?? (() => new Date()); this.#projectRoot = resolve(options.projectRoot); this.#registry = options.registry ?? createDefaultRegistry(); + this.#run = platformRunOf(options.platformRuntime); this.#targets = options.targets; } @@ -660,7 +669,7 @@ export class EvalService { const missing = missingArtifactTargets(planned, artifact); if (missing.length > 0) { // Nothing owns this directory yet, so the abandoned artifact copy is removed. - await rm(directory, { force: true, recursive: true }); + await this.#run(Effect.flatMap(FileSystem.FileSystem, (fs) => fs.remove(directory, { force: true, recursive: true }))); throw serviceError( 'EVAL_TARGET_MISSING', `The evaluated artifact has no target for ${JSON.stringify(missing)}. Build the pinned host targets before evaluating them.`, @@ -1057,6 +1066,8 @@ export class EvalService { ): Promise { const artifactRoot = join(directory, 'artifacts'); const target = join(directory, ...segments); + // Stays on `node:fs` (keep-raw list): an `O_NOFOLLOW` descriptor whose + // `dev`/`ino`/`nlink` identity is checked against the surrounding `lstat`s. await assertNoSymlinkedArtifactPath(this.#projectRoot, target); const before = await lstat(target); if (!before.isFile() || before.isSymbolicLink() || before.nlink !== 1 || before.size > maximumArtifactBytes) { diff --git a/packages/agent-bundle/src/dev/host-install-manager.ts b/packages/agent-bundle/src/dev/host-install-manager.ts index 4b10725ec..6d6ba436a 100644 --- a/packages/agent-bundle/src/dev/host-install-manager.ts +++ b/packages/agent-bundle/src/dev/host-install-manager.ts @@ -3,17 +3,20 @@ import { lstat, mkdir, mkdtemp, - readFile, readdir, rename, rm, symlink, - writeFile, } from 'node:fs/promises'; import { homedir, tmpdir } from 'node:os'; import { basename, join, relative, resolve } from 'node:path'; +import { Effect, FileSystem } from 'effect'; + import { stableJson } from '../core/digest.ts'; +import { isPlatformErrno, readFileString, type PlatformRun } from '../effect/platform.ts'; +import { platformRunOf } from './platform-run.ts'; +import type { DevPlatformRuntime } from './platform-runtime.ts'; import type { Diagnostic } from '../core/diagnostics.ts'; import { installBundle as defaultInstallBundle, @@ -44,6 +47,8 @@ export interface DevHostInstallManagerOptions { readonly hosts: readonly InstallHost[]; readonly installBundle?: (options: InstallBundleOptions) => Promise; readonly projectRoot: string; + /** The dev server's session runtime; absent, each program runs on its own `platformLayer`. */ + readonly platformRuntime?: DevPlatformRuntime; } interface InstalledDevHost { @@ -80,13 +85,14 @@ const rewriteMcpDocument = async ( bundleRoot: string, host: InstallHost, projectRoot: string, + run: PlatformRun, ): Promise => { const path = join(bundleRoot, mcpDocumentPath(host)); let document: unknown; try { - document = JSON.parse(await readFile(path, 'utf8')) as unknown; + document = JSON.parse(await run(readFileString(path))) as unknown; } catch (error) { - if (error instanceof Error && (error as NodeJS.ErrnoException).code === 'ENOENT') return; + if (isPlatformErrno(error, 'ENOENT')) return; throw error; } if (!isRecord(document) || !isRecord(document.mcpServers)) { @@ -103,7 +109,7 @@ const rewriteMcpDocument = async ( }, ] as const), )); - await writeFile(path, `${stableJson({ ...document, mcpServers })}\n`, 'utf8'); + await run(Effect.flatMap(FileSystem.FileSystem, (fs) => fs.writeFileString(path, `${stableJson({ ...document, mcpServers })}\n`))); }; const marker = ( @@ -117,18 +123,27 @@ const marker = ( schemaVersion: 1, }); +/** + * Stays on `node:fs` for the staging copy: `cp` with `verbatimSymlinks` and + * `errorOnExist` has no `FileSystem.copy` equivalent, and the parent's + * ownership transfers to the returned `cleanup`, so it is not a bracket. + */ const prepareDevBundle = async ( source: string, host: InstallHost, epochId: string, projectRoot: string, + run: PlatformRun, ): Promise Promise; readonly root: string }>> => { const parent = await mkdtemp(join(tmpdir(), `agent-bundle-dev-${host}-`)); const root = join(parent, 'bundle'); try { await cp(source, root, { errorOnExist: true, force: false, recursive: true, verbatimSymlinks: true }); - await rewriteMcpDocument(root, host, projectRoot); - await writeFile(join(root, DEV_INSTALL_MARKER), `${stableJson(marker(epochId, host, projectRoot))}\n`, 'utf8'); + await rewriteMcpDocument(root, host, projectRoot, run); + await run(Effect.flatMap(FileSystem.FileSystem, (fs) => fs.writeFileString( + join(root, DEV_INSTALL_MARKER), + `${stableJson(marker(epochId, host, projectRoot))}\n`, + ))); return Object.freeze({ cleanup: () => rm(parent, { force: true, recursive: true }), root, @@ -296,6 +311,7 @@ export class DevHostInstallManager { readonly #installBundle: (options: InstallBundleOptions) => Promise; readonly #installed = new Map(); readonly #projectRoot: string; + readonly #run: PlatformRun; #closed = false; #pending: Promise = Promise.resolve(); #subscription: ProjectEventSubscription | undefined; @@ -309,6 +325,7 @@ export class DevHostInstallManager { this.#hosts = Object.freeze([...new Set(options.hosts)]); this.#installBundle = options.installBundle ?? defaultInstallBundle; this.#projectRoot = resolve(options.projectRoot); + this.#run = platformRunOf(options.platformRuntime); } start(): void { @@ -384,7 +401,7 @@ export class DevHostInstallManager { } async #syncHost(epochRoot: string, epochId: string, host: InstallHost): Promise { - const prepared = await prepareDevBundle(join(epochRoot, host), host, epochId, this.#projectRoot); + const prepared = await prepareDevBundle(join(epochRoot, host), host, epochId, this.#projectRoot, this.#run); try { let installed = this.#installed.get(host); if (installed === undefined) { diff --git a/packages/agent-bundle/src/dev/mcp-session/mcp-session-service.ts b/packages/agent-bundle/src/dev/mcp-session/mcp-session-service.ts index 2b98f7cc2..81ae6369d 100644 --- a/packages/agent-bundle/src/dev/mcp-session/mcp-session-service.ts +++ b/packages/agent-bundle/src/dev/mcp-session/mcp-session-service.ts @@ -4,9 +4,8 @@ import { type Transport, } from '@modelcontextprotocol/client'; import { StdioClientTransport } from '@modelcontextprotocol/client/stdio'; -import { Cause, Effect, Exit } from 'effect'; +import { Cause, Effect, Exit, FileSystem, Scope } from 'effect'; import { randomUUID } from 'node:crypto'; -import { mkdtemp, readFile, rm } from 'node:fs/promises'; import { tmpdir } from 'node:os'; import { isAbsolute, resolve } from 'node:path'; @@ -17,6 +16,8 @@ import { joinArtifact } from '../../core/paths.ts'; import { isRecord, parseJsonWithoutDuplicateKeys } from '../../core/strict-json.ts'; import { runPromise } from '../../effect/boundary.ts'; import { liftPromise, liftTry } from '../../effect/lift.ts'; +import { readFileString, unwrapPlatformError, type PlatformRun } from '../../effect/platform.ts'; +import { platformRunOf } from '../platform-run.ts'; import { readTargetMcpServer, type ModernMcpServer, @@ -231,6 +232,7 @@ export class McpSessionService { readonly #epochStore: EpochStore; readonly #projectRoot: string; readonly #registry: TargetRegistry; + readonly #run: PlatformRun; readonly #traceSink: McpSessionTraceSink | undefined; readonly #openingSessions = new Set(); readonly #sessions = new Map(); @@ -249,6 +251,7 @@ export class McpSessionService { this.#epochStore = options.epochStore; this.#projectRoot = resolve(options.projectRoot); this.#registry = options.registry ?? createDefaultRegistry(); + this.#run = platformRunOf(options.platformRuntime); this.#traceSink = options.traceSink; } @@ -263,7 +266,7 @@ export class McpSessionService { let cleanupFailed = false; let cleanupFailure: unknown; try { - return await runPromise(this.#openEffect({ ...options, signal, timeoutMs }, (error) => { + return await this.#run(this.#openEffect({ ...options, signal, timeoutMs }, (error) => { if (!cleanupFailed) { cleanupFailed = true; cleanupFailure = error; @@ -284,11 +287,17 @@ export class McpSessionService { * a finalizer — only while the session has not been constructed. Once the * `McpSession` exists it owns every resource, the scope finalizers disarm, * and a later open failure is cleaned up by `session.close()` instead. + * + * The plugin-data directory is not a `withTempDirectory` bracket: it + * outlives this call. It is acquired into its own session-lifetime + * {@link Scope}, whose only finalizer removes it, and the session closes + * that scope from `close()` — until the session exists, the open scope's + * release closes it instead. */ #openEffect( options: OpenMcpSessionOptions, reportCleanupFailure: (error: unknown) => void, - ): Effect.Effect { + ): Effect.Effect { return Effect.suspend(() => { const cleanupFailures: unknown[] = []; let constructed: McpSession | undefined; @@ -321,9 +330,23 @@ export class McpSessionService { if (errors.length > 0) return yield* Effect.fail(new DiagnosticError(errors)); const targetRoot = yield* liftTry(() => joinArtifact(epochRoot, target)); const server = yield* liftPromise(() => this.#server(targetRoot, target, runtime, options.serverName)); + const fs = yield* FileSystem.FileSystem; + const pluginDataScope = yield* Scope.make(); + const releasePluginData = (): Promise => runPromise(Scope.close(pluginDataScope, Exit.void)); const pluginData = yield* Effect.acquireRelease( - liftPromise(() => mkdtemp(resolve(tmpdir(), 'agent-bundle-mcp-'))), - (directory) => releaseUnlessTransferred(() => rm(directory, { force: true, recursive: true })), + Effect.tap( + fs.makeTempDirectory({ directory: tmpdir(), prefix: 'agent-bundle-mcp-' }), + (directory) => Scope.addFinalizer( + pluginDataScope, + // The session's close step reports this failure (last-failure-wins); + // `runPromise` rethrows the defect as the unwrapped Node error. + fs.remove(directory, { force: true, recursive: true }).pipe( + Effect.mapError(unwrapPlatformError), + Effect.orDie, + ), + ), + ), + () => releaseUnlessTransferred(releasePluginData), ); const sessionId = randomUUID(); const session = yield* liftTry(() => new McpSession({ @@ -340,6 +363,7 @@ export class McpSessionService { onClose: () => this.#invalidateSession(sessionId, new Error('MCP session closed.')), onClosing: () => this.#invalidateSession(sessionId, new Error('MCP session is closing.')), pluginData, + releasePluginData, resolved: { runtime, server, target, targetRoot }, timeoutMs: options.timeoutMs, ...(this.#traceSink === undefined ? {} : { traceSink: this.#traceSink }), @@ -480,7 +504,7 @@ export class McpSessionService { const path = joinArtifact(targetRoot, runtime.manifestPath); let document: unknown; try { - document = parseJsonWithoutDuplicateKeys(await readFile(path, 'utf8')); + document = parseJsonWithoutDuplicateKeys(await this.#run(readFileString(path))); } catch (error) { if (error instanceof SyntaxError) { throw new Error(`MCP manifest for target ${JSON.stringify(target)} is not valid JSON.`, { cause: error }); diff --git a/packages/agent-bundle/src/dev/mcp-session/mcp-session-types.ts b/packages/agent-bundle/src/dev/mcp-session/mcp-session-types.ts index 74da48264..f66f5133a 100644 --- a/packages/agent-bundle/src/dev/mcp-session/mcp-session-types.ts +++ b/packages/agent-bundle/src/dev/mcp-session/mcp-session-types.ts @@ -13,6 +13,7 @@ import type { import type { Stream } from 'node:stream'; import type { TargetRegistry } from '../../adapters/registry.ts'; +import type { DevPlatformRuntime } from '../platform-runtime.ts'; import type { EpochStore } from '../epoch-store.ts'; import type { McpSessionBinding, @@ -172,6 +173,8 @@ export interface McpSessionServiceOptions { readonly epochStore: EpochStore; readonly projectRoot: string; readonly registry?: TargetRegistry; + /** The dev server's session runtime; absent, each program runs on its own `platformLayer`. */ + readonly platformRuntime?: DevPlatformRuntime; /** Optional observability sink. It receives safe trace categories, never changes session behavior. */ readonly traceSink?: McpSessionTraceSink; } diff --git a/packages/agent-bundle/src/dev/mcp-session/mcp-session.ts b/packages/agent-bundle/src/dev/mcp-session/mcp-session.ts index ff7ee26b9..51792a376 100644 --- a/packages/agent-bundle/src/dev/mcp-session/mcp-session.ts +++ b/packages/agent-bundle/src/dev/mcp-session/mcp-session.ts @@ -13,14 +13,14 @@ import { type Tool, type Transport, } from '@modelcontextprotocol/client'; -import { Effect, type Scope, Semaphore } from 'effect'; +import { Effect, FileSystem, type Scope, Semaphore } from 'effect'; import { randomUUID } from 'node:crypto'; -import { rm } from 'node:fs/promises'; import type { Stream } from 'node:stream'; import { isRecord, snapshotStrictJsonValue } from '../../core/strict-json.ts'; import { runPromise, runSync } from '../../effect/boundary.ts'; import { liftPromise, liftTry } from '../../effect/lift.ts'; +import { runWithPlatform } from '../../effect/platform.ts'; import type { EpochReference } from '../epoch-store.ts'; import type { McpSessionBinding, @@ -148,6 +148,7 @@ export class McpSession { readonly #onClose: () => void; readonly #onClosing: () => void; readonly #pluginData: string; + readonly #releasePluginData: () => Promise; readonly #resolved: ResolvedMcpSessionServer; readonly #launch: ResolvedMcpSessionLaunch; readonly #timeoutMs: number; @@ -186,6 +187,11 @@ export class McpSession { readonly onClose: () => void; readonly onClosing?: () => void; readonly pluginData: string; + /** + * Releases `pluginData` on close; the service passes the close of its + * session-lifetime scope. Default: remove the directory. + */ + readonly releasePluginData?: () => Promise; readonly resolved: ResolvedMcpSessionServer; readonly timeoutMs?: number; readonly traceSink?: McpSessionTraceSink; @@ -201,6 +207,10 @@ export class McpSession { this.#onClose = options.onClose; this.#onClosing = options.onClosing ?? (() => undefined); this.#pluginData = options.pluginData; + this.#releasePluginData = options.releasePluginData ?? (() => runWithPlatform(Effect.flatMap( + FileSystem.FileSystem, + (fs) => fs.remove(this.#pluginData, { force: true, recursive: true }), + ))); this.#resolved = options.resolved; this.#timeoutMs = resolveTimeoutMs(options.timeoutMs ?? defaultTimeoutMs); this.#traceLog = new McpSessionTraceLog(this.#binding, options.traceSink); @@ -543,7 +553,7 @@ export class McpSession { return Effect.gen({ self: this }, function* (this: McpSession) { this.#cancelAll('MCP session closed.'); yield* step(() => this.#closeClient()); - yield* step(() => rm(this.#pluginData, { force: true, recursive: true })); + yield* step(this.#releasePluginData); yield* step(() => this.#epochReference.close()); this.#onClose(); if (failures.length > 0) { diff --git a/packages/agent-bundle/src/dev/package-build-service.ts b/packages/agent-bundle/src/dev/package-build-service.ts index b67466cc5..e88a7ace3 100644 --- a/packages/agent-bundle/src/dev/package-build-service.ts +++ b/packages/agent-bundle/src/dev/package-build-service.ts @@ -1,11 +1,16 @@ -import { rm, rmdir } from 'node:fs/promises'; +import { rmdir } from 'node:fs/promises'; import { dirname, join } from 'node:path'; +import { Effect, FileSystem } from 'effect'; + import { toPosixRelative } from '../core/paths.ts'; import { buildPackageOutputs } from '../build/package-build.ts'; import type { Diagnostic } from '../core/diagnostics.ts'; import { digest } from '../core/digest.ts'; +import { type PlatformRun } from '../effect/platform.ts'; +import { platformRunOf } from './platform-run.ts'; +import type { DevPlatformRuntime } from './platform-runtime.ts'; import type { PreparedProject } from './project-service.ts'; import type { Invalidation } from './types.ts'; @@ -46,6 +51,8 @@ export interface DevPackageBuilder { export interface DevPackageBuildServiceOptions { /** Injectable only for deterministic unit tests. */ readonly buildOutputs?: typeof buildPackageOutputs; + /** The dev server's session runtime; absent, each program runs on its own `platformLayer`. */ + readonly platformRuntime?: DevPlatformRuntime; } /** Files that change the package build without appearing in bundle provenance. */ @@ -85,6 +92,7 @@ const relativePosix = toPosixRelative; export class DevPackageBuildService implements DevPackageBuilder { readonly #buildOutputs: typeof buildPackageOutputs; + readonly #run: PlatformRun; #last: Readonly<{ identity: string; inputs: ReadonlySet; @@ -95,6 +103,7 @@ export class DevPackageBuildService implements DevPackageBuilder { constructor(options: DevPackageBuildServiceOptions = {}) { this.#buildOutputs = options.buildOutputs ?? buildPackageOutputs; + this.#run = platformRunOf(options.platformRuntime); } async build(prepared: PreparedProject, invalidation: Invalidation): Promise { @@ -155,11 +164,16 @@ export class DevPackageBuildService implements DevPackageBuilder { if (published === undefined) return outcome('absent'); this.#published = undefined; try { - for (const path of published.paths) { - await rm(join(published.outputRoot, path), { force: true }); - } + const outputRoot = published.outputRoot; + await this.#run(Effect.flatMap(FileSystem.FileSystem, (fs) => Effect.forEach( + published.paths, + (path) => fs.remove(join(outputRoot, path), { force: true }), + { discard: true }, + ))); // Prune now-empty directories, deepest first; a directory that still - // holds files another producer wrote simply stays. + // holds files another producer wrote simply stays. `rmdir` has no + // `FileSystem` equivalent (`remove` is `rm`, which refuses directories + // without `recursive`), so this loop stays on `node:fs`. const directories = [...new Set(published.paths .map((path) => dirname(path)) .filter((directory) => directory !== '.'))] diff --git a/packages/agent-bundle/src/dev/platform-run.ts b/packages/agent-bundle/src/dev/platform-run.ts new file mode 100644 index 000000000..60f368b5f --- /dev/null +++ b/packages/agent-bundle/src/dev/platform-run.ts @@ -0,0 +1,46 @@ +import { Effect, type Layer } from 'effect'; + +import { makeScopedEffectRuntime } from '../effect/boundary.ts'; +import { + platformLayer, + runWithPlatform, + unwrapPlatformError, + type PlatformRun, + type PlatformServices, +} from '../effect/platform.ts'; +import type { DevPlatformRuntime } from './platform-runtime.ts'; + +/** + * The Effect-typed side of `DevPlatformRuntime`: `createDevPlatformRuntime` + * builds one `makeScopedEffectRuntime(platformLayer)` and hands back the + * Effect-free handle; `platformRunOf` resolves the handle to its `PlatformRun` + * edge (`PlatformError` unwrapped to its Node cause, like `runWithPlatform`). + * The edge is kept off the handle's type on purpose — see + * `./platform-runtime.ts`. Imported by services for their implementation only, + * so it never reaches an emitted `.d.ts`. Lives under `dev/`, not in + * `src/effect/platform.ts`, so the emitted installer that bundles that module + * stays byte-identical. + */ +const edges = new WeakMap(); + +export const createDevPlatformRuntime = ( + layer: Layer.Layer = platformLayer, +): DevPlatformRuntime => { + const runtime = makeScopedEffectRuntime(layer); + const run: PlatformRun = (effect, options) => runtime.run(Effect.mapError(effect, unwrapPlatformError), options); + const handle: DevPlatformRuntime = Object.freeze({ close: () => runtime.close() }); + edges.set(handle, run); + return handle; +}; + +/** + * The edge a service runs its platform programs through: the session + * runtime's when one was given, otherwise `runWithPlatform` (one layer per + * call), so every service stays constructible on its own. + */ +export const platformRunOf = (runtime: DevPlatformRuntime | undefined): PlatformRun => { + if (runtime === undefined) return runWithPlatform; + const run = edges.get(runtime); + if (run === undefined) throw new TypeError('platformRuntime must come from createDevPlatformRuntime.'); + return run; +}; diff --git a/packages/agent-bundle/src/dev/platform-runtime.ts b/packages/agent-bundle/src/dev/platform-runtime.ts new file mode 100644 index 000000000..2965ddcda --- /dev/null +++ b/packages/agent-bundle/src/dev/platform-runtime.ts @@ -0,0 +1,18 @@ +/** + * The dev server's platform runtime, as the services see it: one per + * `startDevServer` call, created inside that function (never at module top + * level — `effect` is a CLI cold-start cost, #530) and closed from the returned + * session's `close` after the last service that ran on it has closed. + * + * Deliberately Effect-free. Every dev service names this type in its exported + * options (`platformRuntime?: DevPlatformRuntime`), and those declarations sit + * on the package's public declaration graph, which must not import `effect` + * (`public-api.test.ts` "keeps every public declaration graph free of + * effect"). The Effect-typed edge lives in `./platform-run.ts` + * (`createDevPlatformRuntime`, `platformRunOf`), which services import for + * their implementation only. + */ +export interface DevPlatformRuntime { + /** Releases the runtime's Scope. Call after every service that ran on it has closed. */ + close(): Promise; +} diff --git a/packages/agent-bundle/src/dev/playground/hook-playground-service.ts b/packages/agent-bundle/src/dev/playground/hook-playground-service.ts index e670f554e..d3dd9bf9a 100644 --- a/packages/agent-bundle/src/dev/playground/hook-playground-service.ts +++ b/packages/agent-bundle/src/dev/playground/hook-playground-service.ts @@ -1,7 +1,9 @@ -import { cp, mkdtemp, readFile, readdir, rm } from 'node:fs/promises'; +import { cp } from 'node:fs/promises'; import { tmpdir } from 'node:os'; import { join } from 'node:path'; +import { Effect, FileSystem } from 'effect'; + import { canonicalHookEventFor, type TargetHookContract } from '../../adapters/hook-contract.ts'; import { createDefaultRegistry, TargetRegistry } from '../../adapters/registry.ts'; import type { ArtifactHook } from '../../build/hook-index.ts'; @@ -14,6 +16,10 @@ import { HookService } from '../../services/hook-service.ts'; import { EpochStore, type EpochReference } from '../epoch-store.ts'; import type { DevLogKindFor, DevLogSink } from '../logs/dev-log-service.ts'; import { deepFreeze } from '../../core/freeze.ts'; +import { liftPromise } from '../../effect/lift.ts'; +import { readFileString, withTempDirectory, type PlatformRun } from '../../effect/platform.ts'; +import { platformRunOf } from '../platform-run.ts'; +import type { DevPlatformRuntime } from '../platform-runtime.ts'; type CanonicalHookInput = Readonly>; @@ -102,6 +108,8 @@ export interface HookPlaygroundServiceOptions { /** Optional non-throwing producer-wide diagnostics sink. */ readonly logger?: DevLogSink; readonly registry?: TargetRegistry; + /** The dev server's session runtime; absent, each program runs on its own `platformLayer`. */ + readonly platformRuntime?: DevPlatformRuntime; } const epochStagingMarkerName = '.agent-bundle-epoch-stage.json'; @@ -154,10 +162,11 @@ const matcherFor = async ( hook: ArtifactHook, contract: TargetHookContract, nativeSelector: string, + run: PlatformRun, ): Promise => { let document: unknown; try { - document = JSON.parse(await readFile(join(artifact, hook.target, contract.manifestPath), 'utf8')); + document = JSON.parse(await run(readFileString(join(artifact, hook.target, contract.manifestPath)))); } catch (error) { if (isErrno(error, 'ENOENT')) return missingManifest(hook.target, hook.event, contract.manifestPath); return undefined; @@ -182,6 +191,7 @@ const hostMappingFor = async ( artifact: string, hook: ArtifactHook, contract: TargetHookContract, + run: PlatformRun, ): Promise => { const canonicalEvent = canonicalHookEventFor(hook.event); if (canonicalEvent === undefined) return unsupportedEvent(hook.target, hook.event); @@ -189,7 +199,7 @@ const hostMappingFor = async ( if (typeof nativeSelector !== 'string' || nativeSelector.trim().length === 0) { return unsupportedEvent(hook.target, hook.event); } - const matcher = await matcherFor(artifact, hook, contract, nativeSelector); + const matcher = await matcherFor(artifact, hook, contract, nativeSelector, run); if (typeof matcher === 'object' && matcher !== null) return matcher; return Object.freeze({ canonicalEvent, @@ -243,9 +253,11 @@ export class HookPlaygroundService { readonly #hookService: Pick; readonly #logger: DevLogSink | undefined; readonly #registry: TargetRegistry; + readonly #run: PlatformRun; constructor(options: HookPlaygroundServiceOptions) { this.#copy = options.copy ?? cp; + this.#run = platformRunOf(options.platformRuntime); this.#epochStore = options.epochStore; this.#registry = options.registry ?? createDefaultRegistry(); this.#hookService = options.hookService ?? new HookService({ registry: this.#registry }); @@ -287,7 +299,7 @@ export class HookPlaygroundService { if (clonedMatches.length !== 1) { throw new Error(`Expected exactly one ${target} hook matching ${JSON.stringify(options.hook)} in the simulation clone.`); } - const mapping = await hostMappingFor(simulationArtifact, clonedMatches[0]!, contract); + const mapping = await hostMappingFor(simulationArtifact, clonedMatches[0]!, contract, this.#run); if ('diagnostics' in mapping) return mapping; const canonicalResult = canonicalResultFor(await this.#hookService.simulate({ @@ -376,17 +388,23 @@ export class HookPlaygroundService { ): Promise { const targetDigest = storedTargetDigestFor(reference, target); await assertTargetDigest(reference.root, target, targetDigest); - const artifact = await mkdtemp(join(tmpdir(), 'agent-bundle-hook-playground-')); - try { - // Copies stay sequential: a failed copy must settle before the artifact directory is released. - for (const entry of await readdir(reference.root)) { - if (entry === epochStagingMarkerName) continue; - await this.#copy(join(reference.root, entry), join(artifact, entry), { recursive: true }); - } - await assertTargetDigest(artifact, target, targetDigest); - return await action(artifact); - } finally { - await rm(artifact, { force: true, recursive: true }); - } + // The simulation artifact lives exactly as long as the action: the + // `mkdtemp` + `finally rm` bracket. The clone copy stays on the injectable + // `fs.cp` (a test seam), sequential so a failed copy settles before the + // directory is released. + return this.#run(withTempDirectory( + { directory: tmpdir(), prefix: 'agent-bundle-hook-playground-' }, + (artifact) => Effect.flatMap( + Effect.flatMap(FileSystem.FileSystem, (fs) => fs.readDirectory(reference.root)), + (entries) => liftPromise(async () => { + for (const entry of entries) { + if (entry === epochStagingMarkerName) continue; + await this.#copy(join(reference.root, entry), join(artifact, entry), { recursive: true }); + } + await assertTargetDigest(artifact, target, targetDigest); + return action(artifact); + }), + ), + )); } } diff --git a/packages/agent-bundle/src/dev/playground/host-discovery-service.ts b/packages/agent-bundle/src/dev/playground/host-discovery-service.ts index 62cb638fb..574d9efec 100644 --- a/packages/agent-bundle/src/dev/playground/host-discovery-service.ts +++ b/packages/agent-bundle/src/dev/playground/host-discovery-service.ts @@ -1,4 +1,3 @@ -import { readFile } from 'node:fs/promises'; import { join } from 'node:path'; import { createDefaultRegistry, type TargetRegistry } from '../../adapters/registry.ts'; @@ -15,6 +14,9 @@ import type { HostDiscoveryReport, } from '../../contracts/discovery.ts'; import { parseJsonWithoutDuplicateKeys } from '../../core/strict-json.ts'; +import { readFileString, type PlatformRun } from '../../effect/platform.ts'; +import { platformRunOf } from '../platform-run.ts'; +import type { DevPlatformRuntime } from '../platform-runtime.ts'; import { runDoctor, type DoctorDurableStateReport, @@ -41,6 +43,8 @@ export interface HostDiscoveryServiceOptions { readonly manifestDigest?: string; }> | undefined; readonly registry?: TargetRegistry; + /** The dev server's session runtime; absent, each program runs on its own `platformLayer`. */ + readonly platformRuntime?: DevPlatformRuntime; } const discoveryDiagnostic = ( @@ -115,6 +119,7 @@ const discoveryMcpServer = (value: ModernMcpServerEntry): DiscoveryMcpServer => const enumerateMcpServers = async ( value: DoctorHostReport, registry: TargetRegistry, + run: PlatformRun, ): Promise => { const bundleRoot = value.bundle?.bundleRoot; if (bundleRoot === undefined) return undefined; @@ -122,7 +127,7 @@ const enumerateMcpServers = async ( const runtime = registry.mcpRuntime(value.host); if (runtime === undefined) return undefined; const document = parseJsonWithoutDuplicateKeys( - await readFile(join(bundleRoot, runtime.manifestPath), 'utf8'), + await run(readFileString(join(bundleRoot, runtime.manifestPath))), ); const result = readTargetMcpServers(runtime, document); if (result.status === 'invalid') return undefined; @@ -135,10 +140,11 @@ const enumerateMcpServers = async ( const hostReport = async ( value: DoctorHostReport, registry: TargetRegistry, + run: PlatformRun, ): Promise => Object.freeze({ ...(value.bundle === undefined ? {} - : { bundle: bundleFinding(value.bundle, await enumerateMcpServers(value, registry)) }), + : { bundle: bundleFinding(value.bundle, await enumerateMcpServers(value, registry, run)) }), diagnostics: Object.freeze(value.diagnostics.map(discoveryDiagnostic)), host: value.host, inventory: Object.freeze({ @@ -166,6 +172,7 @@ export class HostDiscoveryService implements HostDiscoveryRouteService { readonly #now: () => Date; readonly #prepared: NonNullable; readonly #registry: TargetRegistry; + readonly #run: PlatformRun; #inFlight: Promise | undefined; constructor(options: HostDiscoveryServiceOptions = {}) { @@ -174,6 +181,7 @@ export class HostDiscoveryService implements HostDiscoveryRouteService { this.#now = options.now ?? (() => new Date()); this.#prepared = options.prepared ?? (() => undefined); this.#registry = options.registry ?? createDefaultRegistry(); + this.#run = platformRunOf(options.platformRuntime); } discover(): Promise { @@ -196,7 +204,7 @@ export class HostDiscoveryService implements HostDiscoveryRouteService { ...(bundleSource ? { from: bundleSource } : {}), }); const hosts: readonly DiscoveryHostReport[] = Object.freeze( - await Promise.all(report.hosts.map((value) => hostReport(value, this.#registry))), + await Promise.all(report.hosts.map((value) => hostReport(value, this.#registry, this.#run))), ); const endpoints: DiscoveryEndpointReport = endpointReport(report.endpoints); const diagnostics: readonly DiscoveryDiagnostic[] = Object.freeze( diff --git a/packages/agent-bundle/src/dev/playground/lifecycle-replay-service.ts b/packages/agent-bundle/src/dev/playground/lifecycle-replay-service.ts index c3eec6f5d..0a6cafebe 100644 --- a/packages/agent-bundle/src/dev/playground/lifecycle-replay-service.ts +++ b/packages/agent-bundle/src/dev/playground/lifecycle-replay-service.ts @@ -182,6 +182,7 @@ const lazyRenderRouteEvents: typeof renderRouteEvents = async (target, options) return renderer.renderRouteEvents(target, options); }; +/** Synchronous probe from a sync resolver; stays on `node:fs` (no `FileSystem` sync API). */ const lifecycleRenderChildPath = (): string => { const current = fileURLToPath(import.meta.url); const candidates = current.endsWith('.ts') diff --git a/packages/agent-bundle/src/dev/playground/mcp-probe-service.ts b/packages/agent-bundle/src/dev/playground/mcp-probe-service.ts index 437b11cd2..16e45bb17 100644 --- a/packages/agent-bundle/src/dev/playground/mcp-probe-service.ts +++ b/packages/agent-bundle/src/dev/playground/mcp-probe-service.ts @@ -8,10 +8,12 @@ import { } from '@modelcontextprotocol/client'; import { StdioClientTransport } from '@modelcontextprotocol/client/stdio'; import { performance } from 'node:perf_hooks'; -import { mkdtemp, readFile, rm } from 'node:fs/promises'; +import { rm } from 'node:fs/promises'; import { tmpdir } from 'node:os'; import { resolve } from 'node:path'; +import { Effect, FileSystem } from 'effect'; + import { createDefaultRegistry, type TargetRegistry } from '../../adapters/registry.ts'; import type { McpProbeFailure, @@ -24,6 +26,9 @@ import type { } from '../../contracts/mcp-probe.ts'; import { redactCredentialText } from '../../core/credentials.ts'; import { parseJsonWithoutDuplicateKeys } from '../../core/strict-json.ts'; +import { readFileString, type PlatformRun } from '../../effect/platform.ts'; +import { platformRunOf } from '../platform-run.ts'; +import type { DevPlatformRuntime } from '../platform-runtime.ts'; import { resolveBundleRoot } from '../../install/doctor.ts'; import { readTargetMcpServer, @@ -126,6 +131,8 @@ export interface McpProbeServiceOptions { readonly prepared: () => Readonly<{ readonly bundleSource: string }> | undefined; readonly projectRoot: string; readonly registry?: TargetRegistry; + /** The dev server's session runtime; absent, each program runs on its own `platformLayer`. */ + readonly platformRuntime?: DevPlatformRuntime; readonly timeoutMs?: number; /** Testing seam for every probe delay; production keeps Node timers. */ readonly timers?: McpProbeTimers; @@ -312,13 +319,23 @@ const positiveTimeout = (value: number): number => { return value; }; +/** + * Teardown-owned removal. Stays on `node:fs`: `FileSystem.remove` has no + * `maxRetries` / `retryDelay`, and a transport that is still releasing its + * process needs them. + */ +const removePluginData = (pluginData: string): Promise => + rm(pluginData, { force: true, maxRetries: 3, recursive: true, retryDelay: 50 }); + +/** Plain removal for the paths where nothing was launched, so nothing can hold the directory. */ +const removeUnusedPluginData = (run: PlatformRun, pluginData: string): Promise => + run(Effect.flatMap(FileSystem.FileSystem, (fs) => fs.remove(pluginData, { force: true, recursive: true }))); + /** * Invoke a close callback so that both a synchronous throw and a rejection * become one settled promise; teardown chains must never depend on a close * being well behaved. */ -const removePluginData = (pluginData: string): Promise => - rm(pluginData, { force: true, maxRetries: 3, recursive: true, retryDelay: 50 }); const settledClose = (close: () => Promise): Promise => { try { @@ -342,6 +359,7 @@ export class McpProbeService { readonly #projectRoot: string; readonly #registry: TargetRegistry; readonly #removePluginData: (pluginData: string) => Promise; + readonly #runPlatform: PlatformRun; readonly #timeoutMs: number; readonly #timers: McpProbeTimers; @@ -352,8 +370,14 @@ export class McpProbeService { { name: 'agent-bundle', version: '0.1.0' }, { capabilities: {} }, )); + this.#runPlatform = platformRunOf(options.platformRuntime); + // Ownership of the directory passes to the transport teardown, so this is + // a plain `makeTempDirectory`, not a `withTempDirectory` bracket. this.#createPluginData = options.createPluginData ?? - (() => mkdtemp(resolve(tmpdir(), 'agent-bundle-mcp-probe-'))); + (() => this.#runPlatform(Effect.flatMap( + FileSystem.FileSystem, + (fs) => fs.makeTempDirectory({ directory: tmpdir(), prefix: 'agent-bundle-mcp-probe-' }), + ))); this.#createStdioTransport = options.createStdioTransport ?? ((stdioOptions) => new StdioClientTransport(stdioOptions)); this.#createStreamableHttpTransport = options.createStreamableHttpTransport ?? @@ -444,7 +468,7 @@ export class McpProbeService { ); } catch (error) { // No transport was opened, so nothing can still hold the directory. - await rm(pluginData, { force: true, recursive: true }); + await removeUnusedPluginData(this.#runPlatform, pluginData); throw error; } // From here on the transport teardown owns plugin-data removal (#execute): @@ -541,7 +565,7 @@ export class McpProbeService { serverName: string, ) { const document = parseJsonWithoutDuplicateKeys( - await readFile(resolve(bundleRoot, runtime.manifestPath), 'utf8'), + await this.#runPlatform(readFileString(resolve(bundleRoot, runtime.manifestPath))), ); const result = readTargetMcpServer(runtime, document, serverName); if (result.status === 'missing') { @@ -572,7 +596,7 @@ export class McpProbeService { transport = this.#transport(options.launch); } catch (error) { // Nothing was launched, so the directory cannot be in use. - await rm(options.pluginData, { force: true, recursive: true }); + await removeUnusedPluginData(this.#runPlatform, options.pluginData); throw error; } // One close promise per probe: a timeout starts the transport's TERM/KILL diff --git a/packages/agent-bundle/src/dev/playground/native-playground-service.ts b/packages/agent-bundle/src/dev/playground/native-playground-service.ts index 16bb4cd9c..4b00847f0 100644 --- a/packages/agent-bundle/src/dev/playground/native-playground-service.ts +++ b/packages/agent-bundle/src/dev/playground/native-playground-service.ts @@ -1,8 +1,13 @@ import { constants, type Stats } from 'node:fs'; -import { link, lstat, mkdir, mkdtemp, open, readdir, realpath, rename, rm, type FileHandle } from 'node:fs/promises'; +import { link, lstat, mkdir, mkdtemp, open, readdir, rename, rm, type FileHandle } from 'node:fs/promises'; import { basename, dirname, isAbsolute, join, relative, resolve } from 'node:path'; +import { Effect, FileSystem } from 'effect'; + import { digest, stableJson } from '../../core/digest.ts'; +import { type PlatformRun } from '../../effect/platform.ts'; +import { platformRunOf } from '../platform-run.ts'; +import type { DevPlatformRuntime } from '../platform-runtime.ts'; import { hasExactOwnKeys, isJsonRecord, parseJsonWithoutDuplicateKeys, snapshotStrictJsonValue, type JsonValue } from '../../core/strict-json.ts'; import { loadConfig } from '../../config/load.ts'; import type { PreparedEvalArtifact } from '../../eval/artifact.ts'; @@ -111,6 +116,8 @@ export interface NativePlaygroundServiceOptions { readonly projectRoot: string; /** @internal Deterministic cleanup seam for lifecycle tests. */ readonly removeWorkspace?: (root: string) => Promise; + /** The dev server's session runtime; absent, each program runs on its own `platformLayer`. */ + readonly platformRuntime?: DevPlatformRuntime; } export interface NativePlaygroundCatalogStorage { @@ -630,6 +637,7 @@ export class NativePlaygroundService { readonly #planFixture: NonNullable; readonly #projectRoot: string; readonly #removeWorkspace: NonNullable; + readonly #run: PlatformRun; readonly #stagingSettleDeadlineMs: number; #abortDispatchDepth = 0; #closePromise: Promise | undefined; @@ -641,7 +649,9 @@ export class NativePlaygroundService { this.#stagingSettleDeadlineMs = options.catalogStagingSettleDeadlineMs ?? stagingPublicationSettleDeadlineMs; this.#catalogMove = options.catalogStorage?.move ?? rename; this.#projectRoot = options.projectRoot; - this.#removeWorkspace = options.removeWorkspace ?? (async (root) => rm(root, { force: true, recursive: true })); + this.#run = platformRunOf(options.platformRuntime); + this.#removeWorkspace = options.removeWorkspace ?? + ((root) => this.#run(Effect.flatMap(FileSystem.FileSystem, (fs) => fs.remove(root, { force: true, recursive: true })))); this.#environment = options.environment; this.#native = options.native; this.#discover = options.discover ?? (async (projectRoot) => { @@ -1491,8 +1501,10 @@ export class NativePlaygroundService { } if (this.#catalogDirectory !== undefined) return; try { - const resolvedEpochRoot = await realpath(dirname(reference.root)); - const resolvedDirectory = await realpath(directory); + const [resolvedEpochRoot, resolvedDirectory] = await this.#run(Effect.flatMap( + FileSystem.FileSystem, + (fs) => Effect.all([fs.realPath(dirname(reference.root)), fs.realPath(directory)]), + )); if (!isInsideOrEqual(resolvedEpochRoot, resolvedDirectory)) { throw new Error('Native Playground catalog directory is invalid.'); } @@ -1502,10 +1514,13 @@ export class NativePlaygroundService { } } + /** Ownership passes to the operation's `#cleanupWorkspace`, so this is not a `withTempDirectory` bracket. */ async #createWorkspaceRoot(): Promise { const root = join(this.#projectRoot, '.agent-bundle'); - await mkdir(root, { recursive: true }); - return mkdtemp(join(root, 'native-playground-')); + return this.#run(Effect.flatMap(FileSystem.FileSystem, (fs) => Effect.andThen( + fs.makeDirectory(root, { recursive: true }), + fs.makeTempDirectory({ directory: root, prefix: 'native-playground-' }), + ))); } async #workspaceDiff(workspace: string, prepared: NativePlaygroundPrepared): Promise { diff --git a/packages/agent-bundle/src/dev/playground/script-playground-service.ts b/packages/agent-bundle/src/dev/playground/script-playground-service.ts index 9fe94cfa8..852536899 100644 --- a/packages/agent-bundle/src/dev/playground/script-playground-service.ts +++ b/packages/agent-bundle/src/dev/playground/script-playground-service.ts @@ -1,12 +1,16 @@ import { spawn, type ChildProcess } from 'node:child_process'; -import { mkdtemp, rm } from 'node:fs/promises'; -import { extname, join, resolve } from 'node:path'; +import { extname, resolve } from 'node:path'; import { tmpdir } from 'node:os'; +import { Effect, FileSystem } from 'effect'; + import { createDefaultRegistry, type TargetRegistry } from '../../adapters/registry.ts'; import { validateArtifactWithSnapshot } from '../../build/validate-artifact.ts'; import { isErrno } from '../../core/errors.ts'; import { isInside } from '../../core/paths.ts'; +import { type PlatformRun } from '../../effect/platform.ts'; +import { platformRunOf } from '../platform-run.ts'; +import type { DevPlatformRuntime } from '../platform-runtime.ts'; import { taskkill, terminateProcessTree, @@ -118,6 +122,8 @@ export interface ScriptPlaygroundServiceOptions { readonly resolveScript?: (request: Omit) => Promise; /** Internal test seam for the epoch lease paired with resolveScript. */ readonly releaseEpochReference?: () => Promise; + /** The dev server's session runtime; absent, each program runs on its own `platformLayer`. */ + readonly platformRuntime?: DevPlatformRuntime; /** Internal test seam; production invokes Windows taskkill directly. */ readonly taskkill?: ProcessTreeTaskkill; readonly timeoutMs?: number; @@ -137,12 +143,20 @@ const interpreterFor = (suffix: string): PlaygroundScriptInterpreter | undefined return undefined; }; -const workspace = async (): Promise => { - const path = await mkdtemp(join(tmpdir(), 'agent-bundle-playground-script-')); +/** + * Not a `withTempDirectory` bracket: the lease's `close` is a separate step + * of the run so that a workspace removal failure is reported in the result + * (`cleanupFailures`) instead of replacing the script's outcome. + */ +const workspace = async (run: PlatformRun): Promise => { + const path = await run(Effect.flatMap( + FileSystem.FileSystem, + (fs) => fs.makeTempDirectory({ directory: tmpdir(), prefix: 'agent-bundle-playground-script-' }), + )); let closePromise: Promise | undefined; return Object.freeze({ close: () => { - closePromise ??= rm(path, { force: true, recursive: true }); + closePromise ??= run(Effect.flatMap(FileSystem.FileSystem, (fs) => fs.remove(path, { force: true, recursive: true }))); return closePromise; }, path, @@ -389,7 +403,8 @@ export class ScriptPlaygroundService { const timeoutMs = options.timeoutMs ?? defaultTimeoutMs; if (!Number.isSafeInteger(outputLimit) || outputLimit < 1) throw new Error('Script playground output limit must be a positive safe integer.'); if (!Number.isSafeInteger(timeoutMs) || timeoutMs < 1) throw new Error('Script playground timeout must be a positive safe integer.'); - this.#createWorkspace = options.createWorkspace ?? workspace; + const run = platformRunOf(options.platformRuntime); + this.#createWorkspace = options.createWorkspace ?? (() => workspace(run)); this.#epochStore = options.epochStore; this.#outputLimit = outputLimit; this.#platform = options.platform ?? process.platform; diff --git a/packages/agent-bundle/src/dev/project-service.ts b/packages/agent-bundle/src/dev/project-service.ts index 542e70a73..af9309061 100644 --- a/packages/agent-bundle/src/dev/project-service.ts +++ b/packages/agent-bundle/src/dev/project-service.ts @@ -3,6 +3,9 @@ import { lstat, readFile, readdir, realpath } from 'node:fs/promises'; import { dirname, isAbsolute, join, relative, resolve, sep } from 'node:path'; import { createDefaultRegistry, type TargetRegistry } from '../adapters/registry.ts'; +import { readFileBytes } from '../effect/platform.ts'; +import { platformRunOf } from './platform-run.ts'; +import type { DevPlatformRuntime } from './platform-runtime.ts'; import { loadDevContractMatrix, type PreparedDevContractMatrix, @@ -59,6 +62,8 @@ export interface ProjectServiceOptions { readonly outputRoots?: readonly string[]; readonly registry?: TargetRegistry; readonly root: string; + /** The dev server's session runtime; absent, each program runs on its own `platformLayer`. */ + readonly platformRuntime?: DevPlatformRuntime; readonly targets?: readonly string[]; } @@ -155,6 +160,10 @@ const relativeSourcePath = (root: string, source: string): string => { return components.join('/'); }; +/** + * Stays on `node:fs`: the per-file `lstat` records link identity next to + * the bytes, and this runs once per project file on every preparation. + */ const sourceInput = async (root: string, source: string): Promise => { try { const resolvedSource = await realpath(source); @@ -737,7 +746,7 @@ export class ProjectService { const configPath = resolve(root, this.#options.configPath ?? 'agent-bundle.config.ts'); const configIdentity = async (): Promise => { try { - return createHash('sha256').update(await readFile(configPath)).digest('hex'); + return createHash('sha256').update(await platformRunOf(this.#options.platformRuntime)(readFileBytes(configPath))).digest('hex'); } catch { return undefined; } diff --git a/packages/agent-bundle/src/dev/runtime-generation-store.ts b/packages/agent-bundle/src/dev/runtime-generation-store.ts index 2610ef056..a016ee3f5 100644 --- a/packages/agent-bundle/src/dev/runtime-generation-store.ts +++ b/packages/agent-bundle/src/dev/runtime-generation-store.ts @@ -4,14 +4,16 @@ import { mkdir, open, readdir, - readFile, rename, rm, writeFile, } from 'node:fs/promises'; import { isAbsolute, join, relative, resolve, sep } from 'node:path'; +import { Effect, FileSystem } from 'effect'; + import { digest, stableJson } from '../core/digest.ts'; +import { readFileBytes, readFileString, runWithPlatform, type PlatformRun } from '../effect/platform.ts'; import { freezeJsonValue, type JsonObject, type JsonValue } from './types.ts'; import { YieldableFrameworkError } from '../effect/errors.ts'; import type { @@ -242,6 +244,7 @@ export class RuntimeGenerationStore implements DevRuntimeGe #prepared = new Map, PreparedRecord>(); readonly #remove: (path: string) => Promise; readonly #retainInactive: number; + readonly #run: PlatformRun; readonly #stagingRoot: string; readonly #storageRoot: string; readonly #validateMetadata: RuntimeGenerationValidator; @@ -275,6 +278,7 @@ export class RuntimeGenerationStore implements DevRuntimeGe this.#now = options.now ?? (() => new Date()); this.#retainInactive = options.retainInactive ?? defaultRetainInactive; this.#remove = options.remove ?? (async (path) => rm(path, { force: true, recursive: true })); + this.#run = runWithPlatform; } active(): RuntimeGeneration | undefined { @@ -537,12 +541,14 @@ export class RuntimeGenerationStore implements DevRuntimeGe if (this.#initialized) return; if (this.#initialization === undefined) { this.#initialization = (async () => { - await mkdir(this.#storageRoot, { recursive: true }); + const makeDirectory = (path: string): Promise => + this.#run(Effect.flatMap(FileSystem.FileSystem, (fs) => fs.makeDirectory(path, { recursive: true }))); + await makeDirectory(this.#storageRoot); await this.#remove(this.#stagingRoot); await this.#remove(this.#generationsRoot); await Promise.all([ - mkdir(this.#stagingRoot, { recursive: true }), - mkdir(this.#generationsRoot, { recursive: true }), + makeDirectory(this.#stagingRoot), + makeDirectory(this.#generationsRoot), ]); this.#initialized = true; })(); @@ -603,7 +609,7 @@ export class RuntimeGenerationStore implements DevRuntimeGe ): Promise> { const manifestPath = join(root, manifestFileName); assertInside(root, manifestPath); - const bytes = await readFile(manifestPath, 'utf8'); + const bytes = await this.#run(readFileString(manifestPath)); let parsed: JsonValue; try { parsed = freezeJsonValue(JSON.parse(bytes) as unknown); @@ -699,7 +705,7 @@ export class RuntimeGenerationStore implements DevRuntimeGe if (status.size !== asset.bytes) { throw invalid(`Runtime generation asset ${JSON.stringify(asset.path)} did not match its byte length.`); } - const bytes = await readFile(path); + const bytes = await this.#run(readFileBytes(path)); if (bytesSha256(bytes) !== asset.sha256) { throw invalid(`Runtime generation asset ${JSON.stringify(asset.path)} did not match its SHA-256 digest.`); } diff --git a/packages/agent-bundle/src/dev/runtime-provider-loader.ts b/packages/agent-bundle/src/dev/runtime-provider-loader.ts index e3cd256f2..056fba7a0 100644 --- a/packages/agent-bundle/src/dev/runtime-provider-loader.ts +++ b/packages/agent-bundle/src/dev/runtime-provider-loader.ts @@ -1,9 +1,12 @@ -import { realpath, stat } from 'node:fs/promises'; import { isAbsolute, relative, resolve } from 'node:path'; +import { Effect, FileSystem } from 'effect'; import { createJiti } from 'jiti'; import type { AgentBundleDevRuntimeConfig } from '../core/types.ts'; +import type { PlatformRun } from '../effect/platform.ts'; +import { platformRunOf } from './platform-run.ts'; +import type { DevPlatformRuntime } from './platform-runtime.ts'; import type { DevRuntimeDescriptor } from './runtime-protocol.ts'; import type { DevRuntimeProvider } from './runtime-provider.ts'; import { YieldableFrameworkError } from '../effect/errors.ts'; @@ -104,7 +107,11 @@ const namedProviderFactory = (module: ProviderModule): (() => unknown) => { return entry.value as () => unknown; }; -const containedProviderPath = async (projectRoot: string, declaration: AgentBundleDevRuntimeConfig): Promise => { +const containedProviderPath = async ( + projectRoot: string, + declaration: AgentBundleDevRuntimeConfig, + run: PlatformRun, +): Promise => { if (!nonemptyString(declaration.provider)) { throw new DevRuntimeProviderLoadError('Development runtime provider must be a nonempty project-relative module path.'); } @@ -118,10 +125,10 @@ const containedProviderPath = async (projectRoot: string, declaration: AgentBund let canonicalRoot: string; let canonicalProvider: string; try { - [canonicalRoot, canonicalProvider] = await Promise.all([ - realpath(lexicalRoot), - realpath(lexicalProvider), - ]); + [canonicalRoot, canonicalProvider] = await run(Effect.flatMap(FileSystem.FileSystem, (fs) => Effect.all([ + fs.realPath(lexicalRoot), + fs.realPath(lexicalProvider), + ], { concurrency: 'unbounded' }))); } catch { throw new DevRuntimeProviderLoadError('Development runtime provider must name an existing regular file inside the project root.'); } @@ -129,12 +136,13 @@ const containedProviderPath = async (projectRoot: string, declaration: AgentBund throw new DevRuntimeProviderLoadError('Development runtime provider must resolve inside the project root.'); } + let regularFile: boolean; try { - if (!(await stat(canonicalProvider)).isFile()) { - throw new DevRuntimeProviderLoadError('Development runtime provider must name an existing regular file.'); - } - } catch (error) { - if (error instanceof DevRuntimeProviderLoadError) throw error; + regularFile = (await run(Effect.flatMap(FileSystem.FileSystem, (fs) => fs.stat(canonicalProvider)))).type === 'File'; + } catch { + throw new DevRuntimeProviderLoadError('Development runtime provider must name an existing regular file.'); + } + if (!regularFile) { throw new DevRuntimeProviderLoadError('Development runtime provider must name an existing regular file.'); } return canonicalProvider; @@ -145,8 +153,9 @@ export const resolveDevRuntimeProvider = async ( projectRoot: string, declaration: AgentBundleDevRuntimeConfig, importer: DevRuntimeModuleImporter = importProviderModule, + platformRuntime?: DevPlatformRuntime, ): Promise => { - const providerPath = await containedProviderPath(projectRoot, declaration); + const providerPath = await containedProviderPath(projectRoot, declaration, platformRunOf(platformRuntime)); let loaded: ProviderModule; try { loaded = await importer(providerPath); diff --git a/packages/agent-bundle/src/dev/skill-document-service.ts b/packages/agent-bundle/src/dev/skill-document-service.ts index e6376ffcd..d53426c4f 100644 --- a/packages/agent-bundle/src/dev/skill-document-service.ts +++ b/packages/agent-bundle/src/dev/skill-document-service.ts @@ -1,4 +1,4 @@ -import { lstat, readFile, readdir, realpath } from 'node:fs/promises'; +import { lstat, readdir, realpath } from 'node:fs/promises'; import { extname, join, resolve } from 'node:path'; import { projectMeta } from '../build/meta.ts'; @@ -11,6 +11,9 @@ import { EpochStore } from './epoch-store.ts'; import { ProjectService } from './project-service.ts'; import { isInsideOrEqual } from '../core/paths.ts'; import { deepFreeze } from '../core/freeze.ts'; +import { readFileBytes, type PlatformRun } from '../effect/platform.ts'; +import { platformRunOf } from './platform-run.ts'; +import type { DevPlatformRuntime } from './platform-runtime.ts'; export type SkillDocumentErrorCode = @@ -78,6 +81,8 @@ export interface SkillDocumentServiceOptions { readonly epochStore: EpochStore; readonly projectService: ProjectService; readonly root: string; + /** The dev server's session runtime; absent, each program runs on its own `platformLayer`. */ + readonly platformRuntime?: DevPlatformRuntime; } const contentTypes: Readonly> = Object.freeze({ @@ -196,9 +201,11 @@ const assertedDirectory = async (root: string): Promise => { return realpath(root); }; +/** The `lstat` / `realpath` checks refuse symlinks and stay on `node:fs`; only the final read is ordinary. */ const readAllowedResource = async ( root: string, resource: SkillResource, + run: PlatformRun, ): Promise => { const realRoot = await assertedDirectory(root); const candidate = resolve(root, resource.relativePath); @@ -220,7 +227,7 @@ const readAllowedResource = async ( } const contentType = contentTypeFor(resource.relativePath); return Object.freeze({ - body: new Uint8Array(await readFile(realCandidate)), + body: new Uint8Array(await run(readFileBytes(realCandidate))), ...(contentDispositionFor(contentType) === undefined ? {} : { contentDisposition: 'attachment' as const }), contentType, relativePath: resource.relativePath, @@ -235,11 +242,13 @@ export class SkillDocumentService { readonly #epochStore: EpochStore; readonly #projectService: ProjectService; readonly #root: string; + readonly #run: PlatformRun; constructor(options: SkillDocumentServiceOptions) { this.#epochStore = options.epochStore; this.#projectService = options.projectService; this.#root = resolve(options.root); + this.#run = platformRunOf(options.platformRuntime); } async sourceTree(): Promise { @@ -269,7 +278,7 @@ export class SkillDocumentService { throw error; } const resource = resourceByPath(skill.resources, segments); - return readAllowedResource(skill.dir, resource); + return readAllowedResource(skill.dir, resource, this.#run); } async generatedTree(epochId: string, target: string): Promise { @@ -298,7 +307,7 @@ export class SkillDocumentService { return this.#withEpochTarget(epochId, target, async (targetRoot) => { const document = await this.#generatedParser(skillId, targetRoot); const resource = resourceByPath(document.resources, segments); - return readAllowedResource(document.dir, resource); + return readAllowedResource(document.dir, resource, this.#run); }); } diff --git a/packages/agent-bundle/src/dev/workbench-assets.ts b/packages/agent-bundle/src/dev/workbench-assets.ts index 1686d9259..0cc20caab 100644 --- a/packages/agent-bundle/src/dev/workbench-assets.ts +++ b/packages/agent-bundle/src/dev/workbench-assets.ts @@ -1,14 +1,19 @@ -import { realpath, stat, readFile } from 'node:fs/promises'; import { basename, extname, resolve } from 'node:path'; -import { isErrno } from '../core/errors.ts'; +import { Effect, FileSystem, Option } from 'effect'; + import { isInside } from '../core/paths.ts'; +import { isPlatformErrno, readFileBytes } from '../effect/platform.ts'; +import { platformRunOf } from './platform-run.ts'; +import type { DevPlatformRuntime } from './platform-runtime.ts'; import type { WorkbenchAssetSource } from './foreground-server.ts'; export interface WorkbenchAssetSourceOptions { /** Root of the prebuilt workbench asset tree. */ readonly root?: string; + /** The dev server's session runtime; absent, each program runs on its own `platformLayer`. */ + readonly platformRuntime?: DevPlatformRuntime; } const contentTypes: Readonly> = Object.freeze({ @@ -45,7 +50,14 @@ export const createWorkbenchAssetSource = ( options: WorkbenchAssetSourceOptions = {}, ): WorkbenchAssetSource => { const root = resolve(options.root ?? defaultRoot()); - const resolvedRoot = realpath(root); + const run = platformRunOf(options.platformRuntime); + // Resolved once, like the former eager `realpath(root)` promise: a missing + // root surfaces on the first read, as it did. + let resolvedRoot: Promise | undefined; + const resolveRoot = (): Promise => { + resolvedRoot ??= run(Effect.flatMap(FileSystem.FileSystem, (fs) => fs.realPath(root))); + return resolvedRoot; + }; // The tree is fixed once built, so successful reads are cached forever. // Misses are never cached: they carry no read cost worth saving, and a // hostile stream of request paths must not grow the cache. @@ -57,18 +69,30 @@ export const createWorkbenchAssetSource = ( if (!isInside(root, candidate)) return undefined; const cached = served.get(candidate); if (cached !== undefined) return cached; - try { - const [actualRoot, actualPath] = await Promise.all([resolvedRoot, realpath(candidate)]); - if (!isInside(actualRoot, actualPath)) return undefined; - const metadata = await stat(actualPath); - if (!metadata.isFile()) return undefined; - const asset = Object.freeze({ body: await readFile(actualPath), contentType: contentTypeFor(actualPath) }); - served.set(candidate, asset); - return asset; - } catch (error) { - if (isErrno(error, 'ENOENT')) return undefined; + const actualRoot = await resolveRoot().catch((error: unknown) => { + if (isPlatformErrno(error, 'ENOENT')) return undefined; throw error; - } + }); + if (actualRoot === undefined) return undefined; + const asset = await run(readContainedAsset(actualRoot, candidate)); + if (Option.isNone(asset)) return undefined; + served.set(candidate, asset.value); + return asset.value; }, }); }; + +/** `realPath` → containment → regular-file `stat` → bytes; `ENOENT` anywhere is a miss, other errors propagate. */ +const readContainedAsset = Effect.fnUntraced(function* (actualRoot: string, candidate: string) { + const fs = yield* FileSystem.FileSystem; + return yield* Effect.gen(function* () { + const actualPath = yield* fs.realPath(candidate); + if (!isInside(actualRoot, actualPath)) return Option.none>(); + const metadata = yield* fs.stat(actualPath); + if (metadata.type !== 'File') return Option.none>(); + const body = yield* readFileBytes(actualPath); + return Option.some(Object.freeze({ body, contentType: contentTypeFor(actualPath) })); + }).pipe(Effect.catch((error) => isPlatformErrno(error, 'ENOENT') + ? Effect.succeed(Option.none>()) + : Effect.fail(error))); +}); diff --git a/packages/agent-bundle/src/dev/workbench-server.ts b/packages/agent-bundle/src/dev/workbench-server.ts index 3933ff4e8..502eea373 100644 --- a/packages/agent-bundle/src/dev/workbench-server.ts +++ b/packages/agent-bundle/src/dev/workbench-server.ts @@ -6,6 +6,7 @@ import type { InstallHost } from '../install/install.ts'; import { AgentApi } from './agent-api.ts'; import { ArtifactInspectionService } from './artifacts/artifact-inspection-service.ts'; import { DevCoordinator } from './coordinator.ts'; +import { DevPackageBuildService } from './package-build-service.ts'; import { runDevEpochContracts } from './dev-contract-runner.ts'; import { EpochAdoptionPolicy } from './epoch-adoption-policy.ts'; import { DevLogService } from './logs/dev-log-service.ts'; @@ -47,6 +48,8 @@ import { McpSessionService } from './mcp-session/mcp-session-service.ts'; import { NativePlaygroundService } from './playground/native-playground-service.ts'; import { PlaygroundOrchestrationService } from './playground/playground-orchestration-service.ts'; import { PlaygroundStore as PlaygroundService } from './playground/playground-store.ts'; +import { createDevPlatformRuntime } from './platform-run.ts'; +import type { DevPlatformRuntime } from './platform-runtime.ts'; import { ProjectService } from './project-service.ts'; import { emptyCompiledRouteGraph } from '../routes/graph.ts'; import { routeManifestFor } from './routes/route-manifest.ts'; @@ -541,6 +544,44 @@ const withMcpSessionLifecycle = ( /** Starts one loopback foreground session over the current project services. */ export const startDevServer = async (options: StartDevServerOptions): Promise => { + // One platform runtime per dev-server session, created here rather than at + // module top level: `effect` is a CLI cold-start cost (#530), and the + // runtime's Scope is disposed from the returned session's `close`. + const platformRuntime = createDevPlatformRuntime(); + let session: DevServerSession; + try { + session = await startDevServerSession(options, platformRuntime); + } catch (error) { + const [cleanup] = await Promise.allSettled([platformRuntime.close()]); + if (cleanup?.status === 'rejected') { + throw new DevServerStartError([ + Object.freeze({ error, resource: 'start' }), + Object.freeze({ error: cleanup.reason, resource: 'cleanup' }), + ]); + } + throw error; + } + return Object.freeze({ + close: async (): Promise => { + // Every service that ran on the runtime closes first; only then is its + // Scope released. A session close failure is the report that matters, so + // it is rethrown as-is even when the disposal also fails; the disposal + // failure surfaces on its own only after a clean session close. + try { + await session.close(); + } catch (error) { + await Promise.allSettled([platformRuntime.close()]); + throw error; + } + await platformRuntime.close(); + }, + openRuntimeClientSurface: (surfaceId: string) => session.openRuntimeClientSurface(surfaceId), + status: () => session.status(), + url: session.url, + }); +}; + +const startDevServerSession = async (options: StartDevServerOptions, platformRuntime: DevPlatformRuntime): Promise => { const root = resolve(options.root); const registry = options.registry ?? createDefaultRegistry(); const openBrowser = options.openBrowser ?? openInBrowser; @@ -555,6 +596,7 @@ export const startDevServer = async (options: StartDevServerOptions): Promise { const prepared = latestValidPreparedProject; if (prepared?.model === undefined) return undefined; @@ -756,6 +801,7 @@ export const startDevServer = async (options: StartDevServerOptions): Promise( Effect.provide(effect, platformLayer).pipe(Effect.mapError(unwrapPlatformError)), options, ); + +/** + * The Promise edge a service runs its platform programs through: either + * `runWithPlatform` (one layer per call — the default for library callers) + * or a long-lived runtime's edge (`src/dev/platform-runtime.ts`, the dev + * server). Same failure contract either way: `PlatformError` unwrapped to + * its Node cause. A type only: this module is bundled into the emitted + * installer, and the runtime-backed edge must not ride along. + */ +export type PlatformRun = ( + effect: Effect.Effect, + options?: RunPromiseOptions, +) => Promise; diff --git a/packages/agent-bundle/tests/effect-filesystem-phase2-dev.test.ts b/packages/agent-bundle/tests/effect-filesystem-phase2-dev.test.ts new file mode 100644 index 000000000..1fb58ad91 --- /dev/null +++ b/packages/agent-bundle/tests/effect-filesystem-phase2-dev.test.ts @@ -0,0 +1,251 @@ +import { access, mkdtemp, readFile, rm, writeFile } from 'node:fs/promises'; +import { tmpdir } from 'node:os'; +import { join } from 'node:path'; + +import { Effect, FileSystem, Layer, Option } from 'effect'; +import { afterEach, describe, expect, it } from '@rstest/core'; + +import { createDefaultRegistry } from '../src/adapters/registry.ts'; +import { McpSession } from '../src/dev/mcp-session/mcp-session.ts'; +import { createDevPlatformRuntime, platformRunOf } from '../src/dev/platform-run.ts'; +import type { DevPlatformRuntime } from '../src/dev/platform-runtime.ts'; +import { ScriptPlaygroundService } from '../src/dev/playground/script-playground-service.ts'; +import { createWorkbenchAssetSource } from '../src/dev/workbench-assets.ts'; +import { platformLayer, runWithPlatform } from '../src/effect/platform.ts'; +import { mcpCatalogStub, stdioTransportStub } from './support/mcp-client-stub.ts'; + +/** + * Phase-2 FileSystem adoption, dev-server slice: every dev service takes a + * `platformRuntime` handle (the dev server passes its one session runtime) and + * the ordinary reads, temp directories, and removals run through its edge. + * These tests pin the seam with `FileSystem.layerNoop` runtimes and the OS + * semantics with real directories. + */ +const roots: string[] = []; +const scratch = async (prefix: string): Promise => { + const root = await mkdtemp(join(tmpdir(), prefix)); + roots.push(root); + return root; +}; + +const runtimes: DevPlatformRuntime[] = []; + +afterEach(async () => { + await Promise.all(runtimes.splice(0).map((runtime) => runtime.close())); + await Promise.all(roots.splice(0).map((root) => rm(root, { force: true, recursive: true }))); +}); + +/** + * A session runtime whose `FileSystem` is the given stub; every other platform + * service is real (`Context.mergeAll` keeps the later layer's service). + */ +const noopRuntime = (fileSystem: Layer.Layer): DevPlatformRuntime => { + const runtime = createDevPlatformRuntime(Layer.mergeAll(platformLayer, fileSystem)); + runtimes.push(runtime); + return runtime; +}; + +const fileInfo = (size: number): FileSystem.File.Info => ({ + atime: Option.none(), + birthtime: Option.none(), + blksize: Option.none(), + blocks: Option.none(), + dev: 0, + gid: Option.none(), + ino: Option.none(), + mode: 0o100644, + mtime: Option.none(), + nlink: Option.none(), + rdev: Option.none(), + size: FileSystem.Size(size), + type: 'File', + uid: Option.none(), +}); + +describe('workbench assets over the session runtime', () => { + it('resolves the root once and reads a contained asset through the given runner', async () => { + const calls: string[] = []; + const body = Buffer.from('body { color: red }'); + const run = noopRuntime(FileSystem.layerNoop({ + readFile: (path) => Effect.sync(() => { + calls.push(`readFile ${path}`); + return new Uint8Array(body); + }), + realPath: (path) => Effect.sync(() => { + calls.push(`realPath ${path}`); + return path; + }), + stat: (path) => Effect.sync(() => { + calls.push(`stat ${path}`); + return fileInfo(body.byteLength); + }), + })); + const assets = createWorkbenchAssetSource({ root: '/srv/workbench', platformRuntime: run }); + const first = await assets.read('styles/app.css'); + expect(first).toEqual({ body, contentType: 'text/css; charset=utf-8' }); + expect(await assets.read('styles/app.css')).toBe(first); + expect(await assets.read('../escape.css')).toBeUndefined(); + expect(calls).toEqual([ + 'realPath /srv/workbench', + 'realPath /srv/workbench/styles/app.css', + 'stat /srv/workbench/styles/app.css', + 'readFile /srv/workbench/styles/app.css', + ]); + }); + + it('treats a missing root or asset as a miss, exactly like the former ENOENT catch', async () => { + const root = await scratch('agent-bundle-fs-phase2-assets-'); + await writeFile(join(root, 'index.html'), ''); + const assets = createWorkbenchAssetSource({ root }); + expect(await assets.read('missing.js')).toBeUndefined(); + expect((await assets.read('index.html'))?.body).toEqual(Buffer.from('')); + const missingRoot = createWorkbenchAssetSource({ root: join(root, 'absent') }); + expect(await missingRoot.read('index.html')).toBeUndefined(); + }); +}); + +describe('script playground workspace lease', () => { + const temporaryScript = async (source: string): Promise => { + const root = await scratch('agent-bundle-fs-phase2-script-'); + const path = join(root, 'review.mjs'); + await writeFile(path, source); + return path; + }; + const request = { epochId: 'epoch-server-owned', scriptId: 'script:review', target: 'codex' } as unknown as Parameters[0]; + + it('creates the workspace and removes it through the runner as separate steps', async () => { + const script = await temporaryScript('process.stdout.write(process.cwd());\n'); + const workspace = await scratch('agent-bundle-fs-phase2-workspace-'); + const calls: string[] = []; + const run = noopRuntime(FileSystem.layerNoop({ + makeTempDirectory: (options) => Effect.sync(() => { + calls.push(`makeTempDirectory ${options?.directory ?? ''} ${options?.prefix ?? ''}`); + return workspace; + }), + remove: (path, options) => Effect.sync(() => { + calls.push(`remove ${path} ${String(options?.recursive)} ${String(options?.force)}`); + }), + })); + const service = new ScriptPlaygroundService({ + resolveScript: async () => Object.freeze({ + interpreter: Object.freeze({ args: Object.freeze([]), command: process.execPath }), + name: 'review', + path: script, + }), + platformRuntime: run, + }); + await expect(service.run(request)).resolves.toMatchObject({ exitCode: 0, stdout: workspace }); + expect(calls).toEqual([ + `makeTempDirectory ${tmpdir()} agent-bundle-playground-script-`, + `remove ${workspace} true true`, + ]); + }); + + it('reports a failed workspace removal in the result instead of replacing the script outcome', async () => { + const script = await temporaryScript('process.stdout.write("ok");\n'); + const workspace = await scratch('agent-bundle-fs-phase2-workspace-'); + const run = noopRuntime(FileSystem.layerNoop({ + makeTempDirectory: () => Effect.succeed(workspace), + remove: () => Effect.die(new Error('workspace removal failed')), + })); + const service = new ScriptPlaygroundService({ + resolveScript: async () => Object.freeze({ + interpreter: Object.freeze({ args: Object.freeze([]), command: process.execPath }), + name: 'review', + path: script, + }), + platformRuntime: run, + }); + await expect(service.run(request)).resolves.toMatchObject({ + cleanupFailures: [{ code: 'workspace-release-failed' }], + exitCode: 0, + stdout: 'ok', + }); + }); + + it('removes a real workspace on the default runner', async () => { + const script = await temporaryScript('process.stdout.write(process.cwd());\n'); + const service = new ScriptPlaygroundService({ + resolveScript: async () => Object.freeze({ + interpreter: Object.freeze({ args: Object.freeze([]), command: process.execPath }), + name: 'review', + path: script, + }), + }); + const result = await service.run(request); + expect(result.exitCode).toBe(0); + expect(result.stdout.startsWith(join(tmpdir(), 'agent-bundle-playground-script-'))).toBe(true); + await expect(access(result.stdout)).rejects.toMatchObject({ code: 'ENOENT' }); + }); +}); + +describe('MCP session plugin-data release', () => { + const sessionFor = (pluginData: string, releasePluginData?: () => Promise): McpSession => new McpSession({ + binding: { epochId: 'epoch-release', serverName: 'fixture', target: 'portable' }, + createClient: () => ({ + callTool: async () => ({ content: [] }), + close: async () => undefined, + connect: async () => undefined, + ...mcpCatalogStub(), + }), + createStdioTransport: () => stdioTransportStub() as never, + createStreamableHttpTransport: () => ({}) as never, + epochReference: { close: async () => undefined, root: '/tmp/agent-bundle-fs-phase2-epoch' } as never, + id: 'session-release', + onClose: () => undefined, + pluginData, + ...(releasePluginData === undefined ? {} : { releasePluginData }), + resolved: { + runtime: createDefaultRegistry().mcpRuntime('portable')!, + server: { args: [], command: 'node', kind: 'stdio' }, + target: 'portable', + targetRoot: '/tmp/agent-bundle-fs-phase2-epoch/portable', + }, + workspaceRoot: '/tmp/agent-bundle-fs-phase2-workspace', + }); + + it('removes the directory on close by default, through the platform layer', async () => { + const pluginData = await scratch('agent-bundle-fs-phase2-plugin-data-'); + const session = sessionFor(pluginData); + await session.close(); + await expect(access(pluginData)).rejects.toMatchObject({ code: 'ENOENT' }); + }); + + it('runs the service-owned release exactly once and reports its failure from close', async () => { + let releases = 0; + const failure = new Error('scope close failed'); + const session = sessionFor('/tmp/agent-bundle-fs-phase2-unowned', async () => { + releases += 1; + throw failure; + }); + await expect(session.close()).rejects.toBe(failure); + await expect(session.close()).rejects.toBe(failure); + expect(releases).toBe(1); + }); +}); + +describe('platformRunOf', () => { + it('resolves a session runtime to its edge and rejects a foreign handle', async () => { + const runtime = noopRuntime(FileSystem.layerNoop({ readFileString: () => Effect.succeed('stubbed') })); + await expect(platformRunOf(runtime)(Effect.flatMap(FileSystem.FileSystem, (fs) => fs.readFileString('/any')))).resolves.toBe('stubbed'); + expect(platformRunOf(undefined)).toBe(runWithPlatform); + expect(() => platformRunOf({ close: async () => undefined })).toThrow(TypeError); + }); + + it('unwraps PlatformError to the Node cause on the session runtime, like runWithPlatform', async () => { + const root = await scratch('agent-bundle-fs-phase2-errno-'); + const runtime = createDevPlatformRuntime(); + runtimes.push(runtime); + await expect(platformRunOf(runtime)(Effect.flatMap(FileSystem.FileSystem, (fs) => fs.readFileString(join(root, 'absent'))))) + .rejects.toMatchObject({ code: 'ENOENT' }); + }); +}); + +describe('runWithPlatform stays the default edge', () => { + it('reads through the shared layer when no runtime is given', async () => { + const root = await scratch('agent-bundle-fs-phase2-default-'); + await writeFile(join(root, 'a.txt'), 'a'); + const text = await runWithPlatform(Effect.flatMap(FileSystem.FileSystem, (fs) => fs.readFileString(join(root, 'a.txt')))); + expect(text).toBe(await readFile(join(root, 'a.txt'), 'utf8')); + }); +});