Skip to content

Consumer test harness: script proof level (runScript) and the re-homed Workbench-surface helpers — #103 remainder #370

Description

@ScriptedAlchemy

Why this exists

The closed-issue audit of #103 found two items from the issue text that its staged plan (#151#190#212#225#239/#244) never delivered and that no other issue carries:

  1. Script proof in the consumer harness. Generate an Agent Bundle framework test harness on top of Rstest #103's proposed public surface lists runScript beside renderRoute / invokeMcpTool / invokeCli, and test level 2 ("projection contract harness") says it "executes plain and rendered scripts". On origin/main (af1c18551) agent-bundle/test exports eight proof levels (route-unit, mcp-in-memory, cli-dispatch, packed-stdio, packed-deleted-source, browser-app, simulated, host-install) and none of them runs a conventional src/scripts/*.{ts,tsx} module. Script output modes are proven only inside the repository (packages/agent-bundle/tests/cli-routes-build.test.ts, Generate CLI, MCPorter, and script projections from the compiled route graph #102 stage 4), so a consumer cannot assert a rendered script's --json / --ndjson / piped-Markdown output without reconstructing the generated executable.
  2. Workbench-surface browser helpers. The Generate an Agent Bundle framework test harness on top of Rstest #103 stage-3 landing re-homed "Workbench-surface browser helpers and render-stream presentation" to Add an interactive Agent Workbench with local host discovery and lifecycle replay #105's stages. Add an interactive Agent Workbench with local host discovery and lifecycle replay #105 closed (all five stages) without adding any consumer-facing Workbench helper to agent-bundle/test/browser. If this is intentionally not a consumer surface (the Workbench is the framework's own dev tool, not a route the consumer authors), record that decision here and close; otherwise it needs an owner.

Scope (item 1)

  • runScript(name, argv, options) at a new honestly labeled proof level (in-process dispatch through the same render-session contract the generated scripts/<name>.mjs executable uses, mirroring how invokeCli reuses runGeneratedCliEntry): plain .ts scripts execute with ordinary stdout/stderr capture; rendered .tsx scripts expose cliJson / cliNdjson-style accessors plus piped Markdown.
  • The test manifest carries script descriptors from the same compiler pass (AgentBundleTestManifest.scripts), collision-checked like cliCommands.
  • Provenance stamps the level and prints it in every failure; a pass here is not packed or host evidence.
  • Adoption: one rendered-script test in examples/audiobook-curator or the route-harness fixture; packages/agent-bundle/README.md proof-level table row.
  • No packed-pool cost (per meta-framework: execution plan and sequencing #107): the packed journey already covers scripts.

Refs: #103 (proposed surface + level 2), #102 stage 4 (#278), #105.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestmeta-frameworkAgent Bundle compiler-coupled meta-frameworktestingFramework test harnesses and integration evidence

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions