diff --git a/.agents/architect.md b/.agents/architect.md index febcfbf00..a01efc2f5 100644 --- a/.agents/architect.md +++ b/.agents/architect.md @@ -20,6 +20,9 @@ outside the implementation can follow the decision on the first read. ## Responsibilities - Reconcile architecture, specifications, issues, milestones and PR stacks. +- Prepare the first interface, architecture and product-verification proposal + for each feature, then collaborate with the Product Owner where approved rules + do not settle it. - Review plans and implementations for architectural correctness, not coding style already covered by automated checks. - Identify the smallest coherent delivery order and work that can proceed in @@ -30,7 +33,79 @@ outside the implementation can follow the decision on the first read. project record. The Architect does not implement the reviewed change. It does not merge, -close, edit or comment on GitHub unless the user requests that action. +close, edit or comment on GitHub unless the user requests that action. Explicit +approval of a reusable rule authorizes the Architect to record that rule in its +rulebook, but does not authorize changes to another project artifact. + +## Product Owner collaboration + +The Product Owner leads the design of public interfaces, durable architecture +conventions and product verification. The Architect brings a concrete first +pass rather than asking the Product Owner to design from a blank page. + +Read and apply these rulebooks before finalizing a feature design: + +- [Product interface](product-interface.md) for anything a person or agent must + type or read while using XMD. +- [Architecture rules](architecture-rules.md) for lasting layers, ownership, + names, terms and reusable patterns. +- [Product verification](product-verification.md) for the behavior a feature + must show working. + +Approved rules delegate conforming decisions to the Architect. The Architect +shows the Product Owner a short inventory of those decisions and the rules that +settled them, but does not ask for approval again. Novelty, ambiguity, conflict +or an exception returns to the Product Owner through an interview. + +For an unresolved interface or architecture decision, review one item at a +time: + +1. **Current:** Show the existing surface or closest established pattern. +2. **Intent:** Explain the behavior or purpose in plain language. +3. **Assessment:** Apply existing rules and constraints, and name what remains + unsettled. +4. **Proposed:** Show the exact interface, wording or architecture decision. +5. **Feedback:** Ask one focused question about the proposal. + +## Communication rules + +1. When the Product Owner says they do not understand, stop and run the five-part understanding interview. + +When the Product Owner says they do not understand, stop the current design +discussion and run this interview before continuing: + +1. **Current:** Quote or restate what the Architect said. +2. **Intent:** Explain what the Architect meant in plain language. +3. **Reconsideration:** Explain why it was unclear in light of existing rules + and constraints. +4. **Proposed:** Give the exact replacement wording or design. +5. **Feedback:** Ask whether it now expresses the Product Owner's intent. + +Do not assume confusion is merely editorial. It may reveal unclear wording, an +unexplained concept or an unsettled product decision. + +## Maintaining rules + +Reusable rules come from explicit Product Owner approval. After an interview, +distinguish a decision local to the feature from a candidate rule. Propose the +exact candidate and where it applies; never generalize feedback silently. + +Record an approved rule immediately in the relevant rulebook and report its +exact wording and location. Each rulebook keeps rationale, examples and +constraints outside the normative rule. + +Maintained rules follow these format rules: + +1. Every maintained rule is one sentence of at most 160 characters; its Markdown list marker does not count. +2. Keep rationale, examples and constraints outside the rule. +3. Only explicit Product Owner approval promotes a feature decision into a reusable rule. +4. An approved rule delegates conforming decisions to the Architect. +5. Record an approved rule immediately in its rulebook. + +When a feature exposes existing debt, correct a surface the feature directly +changes or cannot remain coherent without. Record adjacent violations as +proposed follow-up Stories instead of expanding the feature, and never copy an +inconsistent pattern merely because it exists. ## Language diff --git a/.agents/architecture-rules.md b/.agents/architecture-rules.md new file mode 100644 index 000000000..98fa834f6 --- /dev/null +++ b/.agents/architecture-rules.md @@ -0,0 +1,25 @@ +# Architecture rules + +This rulebook records reusable decisions about system layers, ownership, +cross-package APIs, package responsibilities, shared terminology and patterns +future features should reuse. `architecture.md` and the specifications describe +the system itself; this file governs how new architecture is designed. + +## Approved rules + +1. Product Owner review covers system layers, ownership, cross-package APIs, package roles, shared terms and reusable patterns. +2. Local helpers and implementation structure stay delegated unless they introduce a new architectural concept. +3. Reuse established architecture, names and terminology before introducing a new pattern. +4. Do not copy an inconsistent existing pattern merely because it already exists. +5. Correct debt only when the feature changes that surface or cannot remain coherent without the correction. +6. Record adjacent debt as a proposed follow-up Story instead of expanding the feature. + +The Architect applies these rules without another approval when they settle a +design. A new concept, ambiguous fit, conflict or proposed exception returns to +the Product Owner through the interview in +[architect.md](architect.md). + +An architecture audit will expand this rulebook by interviewing the Product +Owner about the current documents and reconciling their layers, names, +terminology and patterns. Until then, sparse rules are not permission to infer a +convention from inconsistent precedent. diff --git a/.agents/component-descriptions.md b/.agents/component-descriptions.md index 47cd70449..f5e7674a2 100644 --- a/.agents/component-descriptions.md +++ b/.agents/component-descriptions.md @@ -1,5 +1,9 @@ # Component descriptions +A component description follows the general product-interface rules and review +process in [product-interface.md](product-interface.md). This guide adds the +constraints specific to the description rendered beside component metadata. + A component description helps a document author decide what to type. It appears inside `xmd syntax` beside the component's forms, props, captures, `as` behavior, return contract and origin. Write it as the short practical explanation those @@ -7,41 +11,77 @@ structured fields cannot provide, not as a second reference entry. ## Rules -1. **Lead with an imperative.** Start with what the author can do: “Read or write - a file,” not “Reads and writes UTF-8 text.” -2. **Lead with purpose, not mechanism.** `` launches a coding - agent with prepared context. Terminal ownership and provider setup are - secondary details. -3. **Show a representative invocation.** Demonstrate the spelling instead of - naming a “self-closing” or “paired” form. The example is concrete and can be - copied into a document. -4. **Write for the Markdown author.** Describe observable behavior rather than - resolution, error segments, expansion boundaries or other engine machinery. -5. **Do not repeat the structured fields.** Forms, Props, Captures, `as`, - Returns and Origin already answer their own questions. Mention one only when - it explains a choice or a surprising consequence. -6. **Cut what the reader can infer.** Saying that deleting a missing file does - not error already implies that deleting it twice succeeds. -7. **Do not defend an absence.** “There are no formatting options” and “there - is no unbounded loop” make an omitted capability sound like a feature being - defended. Leave it out unless the refusal changes how the component is used. -8. **Do not answer a question nobody asked.** Do not promise that a value stays - unchanged or travels by reference unless a reasonable author would expect a - transformation. -9. **Keep behavior that changes use.** Activation conditions, placement rules, - scope, transport choices, failure behavior and preflight-only forms belong in - the description when they determine what an author writes or observes. -10. **State conditions before apparently unconditional behavior.** Explain when - `` runs before saying what happens when it runs. -11. **Name scope precisely.** Say “in its content” rather than “inside.” State - when a component reaches work performed by nested components. -12. **Prefer the exact recognizable term.** “Compared with `===`” is clearer and - more accurate than “by identity” or “by reference.” Name `` rather - than “questions” when another question component follows a different path. -13. **Use parallel descriptions for related components.** Give `` and - `` the same sentence structure so their differing failure - behavior is visible at a glance. -14. **Describe present behavior.** Avoid roadmap qualifiers such as “currently.” +1. Lead with an imperative. + + Start with what the author can do: “Read or write a file,” not “Reads and + writes UTF-8 text.” + +2. Lead with purpose, not mechanism. + + `` launches a coding agent with prepared context. Terminal + ownership and provider setup are secondary details. + +3. Show a representative invocation. + + Demonstrate the spelling instead of naming a “self-closing” or “paired” + form. The example is concrete and can be copied into a document. + +4. Write for the Markdown author. + + Describe observable behavior rather than resolution, error segments, + expansion boundaries or other engine machinery. + +5. Do not repeat the structured fields. + + Forms, Props, Captures, `as`, Returns and Origin already answer their own + questions. Mention one only when it explains a choice or a surprising + consequence. + +6. Cut what the reader can infer. + + Saying that deleting a missing file does not error already implies that + deleting it twice succeeds. + +7. Do not defend an absence. + + “There are no formatting options” and “there is no unbounded loop” make an + omitted capability sound like a feature being defended. Leave it out unless + the refusal changes how the component is used. + +8. Do not answer a question nobody asked. + + Do not promise that a value stays unchanged or travels by reference unless a + reasonable author would expect a transformation. + +9. Keep behavior that changes use. + + Activation conditions, placement rules, scope, transport choices, failure + behavior and preflight-only forms belong when they determine what an author + writes or observes. + +10. State conditions before apparently unconditional behavior. + + Explain when `` runs before saying what happens when it runs. + +11. Name scope precisely. + + Say “in its content” rather than “inside.” State when a component reaches + work performed by nested components. + +12. Prefer the exact recognizable term. + + “Compared with `===`” is clearer and more accurate than “by identity” or “by + reference.” Name `` rather than “questions” when another question + component follows a different path. + +13. Use parallel descriptions for related components. + + Give `` and `` the same sentence structure so their + differing failure behavior is visible at a glance. + +14. Describe present behavior. + + Avoid roadmap qualifiers such as “currently.” A description is complete when it states the purpose, shows how to invoke the component, and includes only the additional facts an author needs to use it @@ -49,10 +89,9 @@ correctly. ## Refinement interview -Refine a set of descriptions as an interview between the author of the current -behavior and a reader. The interview separates the contract from its wording, -lets one editorial decision become a corpus-wide rule, and prevents a bulk -rewrite from hiding factual mistakes. +Use the Product Owner interview in [architect.md](architect.md). The interview +separates the component's behavior from its wording and prevents a bulk rewrite +from hiding factual mistakes. ### Prepare the pass @@ -64,14 +103,16 @@ rewrite from hiding factual mistakes. ### Review one component -Present exactly four things: +Apply the general five-part interview with these description-specific details: 1. **Current:** quote the existing description verbatim. 2. **Intent:** explain the facts it is trying to communicate without defending the current wording. -3. **Proposed:** offer a clearer and simpler description that follows the rules +3. **Assessment:** identify what is unclear or inconsistent under the general + product-interface rules and the description rules above. +4. **Proposed:** offer a clearer and simpler description that follows the rules above. -4. **Feedback:** ask one focused question about the judgment still unresolved, +5. **Feedback:** ask one focused question about the judgment still unresolved, or ask for approval when none remains. Do not move to the next component until the reader answers. Apply an accepted @@ -79,9 +120,9 @@ description immediately and keep a visible count of completed entries. ### Carry feedback through the corpus -When an edit establishes a general rule, name it and apply it to every later -proposal. Revisit earlier descriptions when the rule affects them; do not change -an approved description silently. +When feedback suggests a reusable rule, follow the explicit promotion process +in [architect.md](architect.md). Revisit earlier descriptions when an approved +rule affects them; do not change an approved description silently. Check a disputed claim against the implementation before revising the prose. The description must remain accurate even when the most approachable wording is diff --git a/.agents/product-interface.md b/.agents/product-interface.md new file mode 100644 index 000000000..f154e5309 --- /dev/null +++ b/.agents/product-interface.md @@ -0,0 +1,39 @@ +# Product interface + +This rulebook governs the words and structures a person or agent must type or +read while using XMD. Public syntax, CLI input, prompts, choices, help, +documentation, output, warnings, errors, refusals and generated artifacts shown +to people are in scope. Internal agent prompts are also reviewed because they +shape the product's output. + +Component descriptions also follow the specialized additions in +[component-descriptions.md](component-descriptions.md). + +## Approved rules + +1. Design public XMD interfaces so people understand what to write, what will happen, what it affects and what they receive. +2. Expected agent authorship never lowers the standard for human clarity. +3. Mixed human-agent surfaces are human-facing and meet the human standard. +4. Errors and refusals tell people what prevented progress and what they can do next. +5. Product Owner review covers internal agent prompts because they shape product output. +6. Show conforming interfaces and the rules that settled them without asking for approval again. +7. Review unresolved interfaces with the Product Owner one at a time. + +The first rule applies to both readings of an executable document: source a +person audits and output a person acts on. Agent flexibility does not justify +compressed, irregular, implicit or machine-oriented interfaces. + +## Feature review + +The Architect inventories every interface a feature adds or changes. The +inventory names the applicable approved rules and closest existing patterns. +Conforming interfaces remain visible in a short summary but need no new +approval. + +Review unresolved interfaces one at a time using the Product Owner interview in +[architect.md](architect.md). For a new interface, **Current** shows the closest +established pattern. **Proposed** shows the exact shape and wording together. + +Feedback applies first to the feature. When it appears reusable, the Architect +proposes a rule candidate separately. Only explicit Product Owner approval +promotes it into this file. diff --git a/.agents/product-verification.md b/.agents/product-verification.md new file mode 100644 index 000000000..fa96d86b2 --- /dev/null +++ b/.agents/product-verification.md @@ -0,0 +1,31 @@ +# Product verification + +Product verification describes the behavior a feature must show working. The +Product Owner leads this design for the foreseeable future. The Architect +prepares the first pass; the Planner and Implementor derive lower-level evidence +after the product behavior is approved. + +## Approved rules + +1. The Architect drafts product verification, then interviews the Product Owner to refine and approve the behavior it covers. +2. Product verification covers user journeys, observable behavior, important failures, recovery and incomplete implementations. +3. Interface and product verification approval precede finalized architecture or implementation handoffs. +4. The Architect and Planner derive lower-level tests from approved product verification without further approval. + +## Feature review + +Review one behavior at a time: + +1. Describe the behavior the feature must provide. +2. Show what a person does and observes. +3. Identify an incomplete implementation that could appear to work. +4. Propose verification that distinguishes the complete behavior. +5. Ask the Product Owner what should change. + +Product verification stays at the product level. Unit tests, boundary tests, +fixtures, runtime selection and focused commands are implementation evidence +derived afterward. + +A future corpus audit will use this interview to evaluate existing tests and +propose additional rules. Until that audit, the Product Owner continues to +approve product verification for every feature. diff --git a/.oxlintrc.json b/.oxlintrc.json index 2d9c6dc91..e92294a55 100644 --- a/.oxlintrc.json +++ b/.oxlintrc.json @@ -19,7 +19,8 @@ "local/no-sync-filesystem": "error", "local/no-yield-in-finally": "error", "local/prefer-effection-operation": "error", - "local/prefer-effection-result": "error" + "local/prefer-effection-result": "error", + "local/require-scope-bound-event-registration": "error" }, "overrides": [ diff --git a/AGENTS.md b/AGENTS.md index 5d324ab6c..efdc6f047 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -447,6 +447,28 @@ the corpus or enabling in-shard concurrency is not an answer to a miss. structural values for composition data and a contextual Api for operations. Security enforcement, durable identity, and reconciliation never trust replaceable context state. +16. An event listener an Effection operation installs has that operation's + lifetime. Wait for one event with `once()` from `@effectionx/node/events`, + never `emitter.once()` or `addEventListener(..., { once: true })`: cleanup + that waits for the event is no cleanup for a wait that is cancelled. A + longer subscription binds a stable handler and removes that same handler, + from that same receiver and event, in the owner's own teardown — `.off()` + for Node, `.removeEventListener()` with the matching capture mode for the + DOM. Removal is synchronous, so it belongs in a `finally` around the + subscription, an `ensure()` that **completed before** the subscription, or + the cleanup an `action()` returns; where teardown must wait on the event + itself, keep the handler through the wait and remove it in a synchronous + `finally` inside that same `ensure()`. `yield* ensure(...)` is itself a + suspension: an owner halted while it registers unwinds with no cleanup on + it at all, so an `ensure()` yielded *after* the subscription has not + established anything — nor may a native resource be created before the + cleanup that releases it, and only the resource's own closing event proves + it is finished, never an assigned exit status. The listener ordering is + enforced by the `local/require-scope-bound-event-registration` Oxlint rule + (`scripts/oxlint-rules/`), which does not autofix: which owner, which + handler and which order are lifecycle decisions. The resource half — a + child spawned before the cleanup that reaps it — is not something that rule + can see, and is held by each owner's focused lifecycle regression instead. ## Writing Guide diff --git a/architecture.md b/architecture.md index 8d50192f0..99f35f838 100644 --- a/architecture.md +++ b/architecture.md @@ -123,8 +123,9 @@ Existing documents and code get aligned to this section retroactively. | provider partition | one complete, independently owned agent-provider state — runtime, store, managed sessions, queues, coordinator, teardown — selected by the one installed factory at each dispatch. Production is the single-partition case of the same path; holding a partition grants work, never permission | | `JournalProvenance` | a non-operational, equality-only witness that a live publication stream descends from the exact journal backend a provider selected for one workflow run; it grants no append, read, execution, publication or reconciliation capability, and is meaningful only because the provider retains the witness it established and later requires exact equality | | result object | the value a component binds instead of failing: `{ok: true, value}` or `{ok: false, …}`, whose failure members the component declares | -| syntax catalog | the complete, versioned description of what a document may write in one directory under one host profile: every structural construct the engine reserves, and the one implementation selection chooses for every other name. It is observation — describing an environment installs no operational state, runs nothing and journals nothing — and it is produced once per request and projected, never rediscovered per format | -| run profile declarations | the component registrations a first-party package makes, held as plain values apart from the middleware, providers, activation and launchers its installer also arranges. The installer registers exactly those values and inspection reads exactly those values, so what a run installs and what the catalog reports cannot drift | +| syntax symbols | the complete, versioned description of what a document may write in one directory under one host profile: every structural construct the engine reserves, and the one implementation selection chooses for every other name. It is observation — describing an environment installs no operational state, runs nothing and journals nothing — and it is produced once per request and projected, never rediscovered per format. `xmd syntax` prints one for an environment nobody is running; canonical `` renders one for the site an element was written at, from the same construction and the same Markdown renderer | +| syntax reference | the engine-owned lexical answer to "what may a document write here", carried by value on canonical core's expansion authority beside the import authority. The execution builds one at its root from the selection inputs it captured before any installation, middleware or document code ran, or from the one set of symbols a trusted host stated for its profile; a trusted canonical evaluation boundary replaces it for the subtree it evaluates. It answers with text and carries no authority: a component the symbols name is not a component anything may run | +| run profile declarations | the component registrations a first-party package makes, held as plain values apart from the middleware, providers, activation and launchers its installer also arranges. The installer registers exactly those values and inspection reads exactly those values, so what a run installs and what the symbols report cannot drift | | origin-only | the inspectability of a component whose contract could only be learned by loading it: a repository TypeScript module, whose schemas live on its exports and whose top level would run. Such an entry carries name, category, origin and source kind, and no contract field at all — an absent contract is stated, never rendered as an empty one | | definition-owned return state | which value body a `` selects for: one ephemeral state per execution of one value root or Markdown value component. Structural directives keep the ambient one, a component invocation hides it from the invoked body, a nested value body installs its own, and caller-projected content restores the caller's. It travels down the expansion call stack as a local rather than through a context, and no exported function accepts another body's, so nothing a document can read, replace, or import acts on a live one. The first claim on it is atomic, so a second executed return fails the body rather than replacing its value, and it appends no durable event | @@ -1553,8 +1554,8 @@ nothing in that value: its own durable effect record is the account of what it did, and a synthetic receipt beside it would be a second thing to keep true. An observation names its durable operation after the invocation the engine -handed it, through the claimant this attachment's execution delivered to the -`` factory it built (*Capability-backed invocation identity*). Nothing +handed it, through the claimant this execution delivered to the protected +`` body it built (*Capability-backed invocation identity*). Nothing else carries authority: an Effection Context is addressed by name, a contextual Api handler installed outside an invocation answers ahead of the engine's own, and a definition or registry answer is a value a handler may keep from one @@ -1563,17 +1564,58 @@ durable name would each replay the other's admitted fragment, and an implementation running under another attachment's identity would commit against its own run's storage under that attachment's expansion. +An admitted element reaches the operations the host handed the profile, and no +others. It used to invoke the ordinary component — core's `` — which +resolves `API.Files` at the moment it runs, so the authority a fragment +exercised was whatever the document, a repository component or middleware had +installed by then. A profile now states a *capability* rather than a definition: +canonical capture reads each of the host's methods off once, binds it behind a +revocation the execution owns, and closes core's own body over the result. So a +fragment reaches five filesystem operations and one transport — not the rest of +the host's provider, not `API.Files`, `API.Fetch` or `API.Env`, and not for +longer than the execution that captured them. A directory an admitted fragment +creates scopes its content through the evaluation's own cursor rather than the +contextual environment, because the contextual environment is exactly what a +document can answer. + +That narrowing changed what these identities authorize, so core's entries state +revision 2 and the workflow's `` states revision 3. **A run suspended +under the earlier revisions refuses to resume rather than silently receiving the +narrower grant**, which is what a revision is for: a continuation resumes only +under the exact identities it was admitted with. + `` is one Agent turn. The host hides no Agent loop inside it. Ordinary authored Markdown owns iteration with ``, branches on whether a reply is another observation proposal or a final proposal, and makes exhaustion an -explicit document failure. `` is the workflow host's -component that invokes the evaluator, and the host does not register it: it -**declares** it to the execution, which mints the domain, calls the host's -factory once per attachment with that domain's claimant, and registers what -comes back. Being reachable from a trusted document is all registration decides; -every ceiling comes from values the host captured before any document existed, -and what an invocation may name comes from the execution that minted the -claimant and the resolution that selected the implementation it was built for. +explicit document failure. `` is canonical core's own protected +component, claimed ahead of every host and author tier, so the name means the +same thing in every execution and no registration, repository file, bundle +member, declared Markdown component, import handler or second loaded copy +answers for it. A host does not supply the component; it supplies the +*ceiling*, as one `ExecutionInstallation.evaluation` profile captured by value +before any installation runs. An execution accepts one and refuses two, and one +that was offered none has no evaluation at all — which `` says when it +is written rather than at startup. + +The document writes the program as `text`, or renders it as paired content: + +```mdx + + + + Read the changelog and report the version. + +``` + +The two are disjoint — an element stating both is stating the program twice, and +is refused rather than resolved by precedence. A paired producer renders under +the *narrowed* syntax reference, so an agent asked to write a fragment is told +what a fragment may contain rather than what the surrounding document may write, +while keeping the producer's own operational authority: narrowing what may be +*named* is not narrowing what the producer may *do*. The workflow profile also +accepts the released `source` spelling, silently, because documents were written +against it before `text` existed; the ordinary profile refuses it, and no +profile has ever had `program`. Within one run a logical Agent session is identified by the Agent/Session expansion identity the engine derived, and by nothing else. The authored @@ -1708,7 +1750,7 @@ still binds `{ observations: [], output: "" }`. Local Git, Git-host, issue, process, eval and exec, native-command, credential and external-write effects are outside the class. -The versioned Dir origin is a grant boundary, not a catalog decoration. The +The versioned Dir origin is a grant boundary, not a symbol decoration. The former `@executablemd/workflow/composition#Dir` identity authorized lexical placement that created nothing; it does not authorize the persistent mutation. A continuation whose retained write table names that former identity is refused @@ -1752,13 +1794,33 @@ nothing the admission was granted under. A run that no longer retains an admission root has lost the history the grant was made over and is refused before generated work. Every non-root term — the class selection, the selected pinned identities with their admitted forms, and the normalized request -ceilings — compares whole and exactly, in canonical order. Only the tables the -selection reached take part, so changing a write table a read-only admission -never drew on changes nothing it was granted under. Deciding the fragment -inside that durable effect is what makes this hold: a continuation restores -what was admitted without consulting the current candidate at all, so what -expands is the source this run admitted rather than whatever a later caller is -holding. +ceilings — compares whole and exactly, in canonical order. A pinned identity is +one closed tagged record — whether canonical core supplies the body or the +ordinary import chain answered for it, the origin, the component key and the +revision — rather than an assembled spelling, so a reader comparing two +admissions can say which term moved and no two hosts have to agree on a +separator. The untagged version-1 record holds one opaque string instead — +whatever the party that built the pinned entry chose, which for core was +`@executablemd/core#File:read` and for the workflow host was +`@executablemd/workflow/composition/dir-v2#Dir`, with no revision in either and +no convention to read back out. Nothing recovers four structural terms from one +of those, so nothing tries: a *capability* entry states the exact version-1 +strings it succeeds, and a retained string reconciles against that stated list +and against nothing else. Stating one is an assertion by whoever owns the entry +that the current grant is no wider than the old one; an entry that states none +refuses every version-1 record naming it. A component answer states no alias and +has nowhere to put one, because that arm did not exist when version-1 records +were written. New records always state the structural identity. The run's own +ceiling table is keyed by those terms encoded rather than joined, because a +joined key is only as unique as the separator is illegal and nothing makes a +space illegal in an origin or a component key: two entries whose terms differ +only in where a space falls each keep their own admitted requests. Only the +tables the selection reached take part, so changing a write table a read-only +admission never drew on changes nothing it was granted under. Deciding the +fragment inside that durable effect is what makes this hold: a continuation +restores what was admitted without consulting the current candidate at all, so +what expands is the source this run admitted rather than whatever a later caller +is holding. The admission and every durable effect the admitted fragment performs are yielded inline by the owning document expansion, in authored order and ahead @@ -2422,6 +2484,16 @@ hidden inside library objects that accumulate. One exception: metadata an author declares at module evaluation, about a value the author owns, may live on that value. +A callback registered with something outside the process — a listener on a +socket, a child process, a stream or a DOM target — is state of exactly this +kind, and the source it is attached to knows nothing about the operation that +attached it. The event arriving is not teardown: a cancelled wait is precisely +the case where it never arrives, and a losing race arm is one that will never +be told. So the owner detaches its handlers, on completion, failure, halt and +race loss alike, before it is considered closed, and an event delivered after +that reaches nothing and changes nothing. `local/require-scope-bound-event-registration` +holds source to it. + A Repository selection is composition data and is therefore replaceable: a document may bind one, render one, hand one to a child, and construct one that looks exactly like it. Nothing a repository provider does is authorized by the @@ -3547,11 +3619,14 @@ ships different bytes under a declared name therefore fails where it is installed rather than where a document happens to write the name. **The name is claimed, not offered.** Resolution places a declared component in -the same protected tier as a reserved registration, above the workflow component -bundle, repository files and every registered default. Two claims on one name -are refused rather than ordered, so the tier never chooses. A repository -`Policy.md`, a bundled `Policy`, an ordinary registration and another loaded -copy of the host package can none of them answer for a declared `Policy`. +the same host tier as a reserved registration, above the workflow component +bundle, repository files and every registered default — and below the canonical +protected tier, which is the engine's own claim rather than a host's. Two claims +on one name are refused rather than ordered, so the tier never chooses. A +repository `Policy.md`, a bundled `Policy`, an ordinary registration and another +loaded copy of the host package can none of them answer for a declared `Policy`; +and a host that declares a name canonical core owns is refused at admission, +before the root import. **Invocation is canonical, for the declared names.** A declaring execution imports through the same retention a bundled one does: canonical core keeps its @@ -3631,18 +3706,177 @@ and a name is not a secret, so a component could build one, reach the record and answer that everything is exact. It is reclaimed with its execution, reaches no public entrypoint, and marks nothing on the segments themselves. -## The syntax catalog boundary +## The canonical protected tier + +Resolution already had a protected tier — a reserved registration, or exact +Markdown a host declared. Both of those are *a host's* claim, made by whoever +assembled the run. Above them sits the engine's own, and a name in it means the +same thing in every execution: whichever host built it, whichever package +registered what, and whatever the repository holds. + +One component is in it. `` describes the vocabulary of the site it is +written at, and a description of a run's vocabulary that anything in the run +could answer for is a description of nothing — the value of the answer is +exactly that nobody but core produced it. + +**It is not a registration.** A registration is an answer a registry gives for a +name, and a registry is something a nested scope layers over, a host installs +into, and a handler can keep a record from and hand back somewhere else. So the +tier is the resolver's own table, consulted after structural syntax and before +every host or author tier, and consulted unconditionally: no option a caller +passes — or leaves out — puts a repository file, a bundle member or a +registration in front of it. + +Collision is refused where each claim is made, so the tier never has to choose: +a registration under the name is refused atomically at registration, a host's +declared Markdown and a workflow bundle member are each refused at admission +before the root import, and a repository `Syntax.md`, `Syntax.ts` or directory +candidate is never probed for because selection has already answered. + +**Invocation is canonical.** The execution mints one identity domain per +protected component, calls its factory with that domain's claimant, and registers +what comes back nowhere. Import is closed for exactly those names, on the terms a +declaration's names are closed: `Component.importComponent` middleware may +observe the import, delegate it and refuse it by throwing, and what it cannot do +is answer one — a witness is issued where canonical execution produced its answer +and verified where the component is invoked, and core's own copy is what runs. +Every other name in the execution stays the ordinary open import it has always +been. + +**Protection is about the answer, not about power.** A protected implementation +is handed the lexical syntax reference for its site and nothing else: no +component definitions, no import witness, no invocation capability, no policy +table, no provider and no registration handle. The body itself is kept in a table +private to the copy of core that built the implementation and reached only by +canonical expansion, so an implementation another loaded copy created — which is +an ordinary arrangement, because a component can be loaded from disk beside its +own copy — has no body here and no answer to give. + +**The named form is a second question.** Bare `` answers *what may I +write here*. `` answers *how do I use this one*, and +the two read different inputs on purpose. The reference therefore carries a +pair: what may **execute** at this site, which the bare form reports, and the +**enclosing authoring symbols**, which named selection reads. At a root they are +one set. Under a trusted evaluation boundary that narrows execution they are +not, and each rendered entry states whether it is available in the current +evaluation — so a nested author can be told how a component works where they may +not run one, without being left to infer that documentation implies authority. +#713 installs that boundary; this stack supplies and proves the seam. + +The documentation itself is the owning package's. Every registration boundary +that contributes public components keeps a `components.md` beside itself — core, +its Agent registrations, the CLI, testing, web and repository composition — with +a contribution derived from the same declarations it registers, so adding a +component demands documentation rather than relying on a hand-kept list. + +**A package's bootstrap installs its registrations and its documentation +together**, through one stable namespaced `Documentation` middleware, so a host +that bootstraps a package gets both from one call. Canonical core is the +terminal, and every wrapper delegates before appending its own — which is why +composition order decides how the list reads and nothing else. Canonical +execution collects **once**, after the trusted host's bootstrap and before the +root import or any document code, and snapshots the answer field by field. So a +document or component that composes around the Api afterwards composes into a +chain nothing reads, two executions assembled in sibling scopes each read their +own, and two contributions that *disagree* about one owning package and +component name refuse at collection, whichever order supplied them. Collection +is the boundary that catches them because it is the only one every execution +passes through: a document that writes bare ``, or none at all, builds +no index and would otherwise run to completion on an assembly nobody validated. + +**A contribution is four values** — the owning package, the asset identity, the +exact documentation text and the component-name set — and equality covers all +four, by value rather than by object identity, with a name set carrying no +order. A repetition of that same value adds nothing and succeeds: one package's +declarative vocabulary is deliberately entered at more than one layer, the +repository-composition set by an ordinary run's bootstrap and again inside a +workflow attachment because either may be the only one, and a nested run or +evaluation host the same way. The inner scope descends from the outer, so both +wrappers sit in one chain, and collection folds them to one — keeping every +layer's registrations and one documentation value. Comparing the asset alone +would instead coalesce two bootstraps that genuinely disagree and silently keep +whichever ran first. + +**Collection classifies duplicates, not the helper a bootstrap happened to +use.** The middleware namespace is public, so a package may compose it directly +and hand back two value-identical contributions rather than going through the +convenience helper; that assembly is exactly as valid. Canonical collection +therefore snapshots the composed chain by value, folds every completely +identical contribution, and only then classifies what is left — so correctness +does not depend on which spelling a package chose. + +A different owner is unequal but is not by itself a conflict, because +documentation joins by component name *and* origin — two packages may each +document a same-spelled component. Disjoint component sets from one owner +coexist for the same reason. + +The alternative — a host-maintained list of every package's documentation, kept +beside a host-maintained list of every package's registrations — is two lists +that drift, and they did: a nested run registered `` and then reported +it undocumented, because one list had been updated and the other had not. + +Assets are located from +their own module's URL rather than the working directory or `--include`, and +every distribution loads the same bytes; one shared entrypoint builds the +complete index and refuses a missing, unknown or duplicated section before any +distribution is produced. One validated index serves ``, `xmd syntax Elicit` and +#678's release reference, so three surfaces cannot describe one component three +ways. It joins by name *and* owning package: a repository `Elicit.md` has a +repository origin, which names no package, so the built-in's prose is never +attached to it. A first-party component with no +section, a heading naming something the package does not supply, one appearing +twice, and one that is not a component name each refuse the whole index rather +than producing a partial one — a reference with a hole in it cannot be told from +one whose components have nothing to say. The no-documentation sentence is for a +**custom** component instead, which no package governs and which takes its prose +from its own document's body when it has any. + +**It says so in the symbols.** A protected component reports its own origin +kind, `protected`, rather than borrowing `registered` with `reserved: true`. The +two answer a reader's actual question — *could I supply this name myself?* — +oppositely: a reserved registration is a host installing something under a name +it wants kept, so it can be absent from another run, replaced by a different +host, or refused when two hosts claim it, and none of that is true here. A +workflow bundle member gained its own kind for the same reason: reported as a +`repository` path it read as a file the reader could edit, when it is the exact +blob `sourceHash` names, fixed when the run was defined. Both are why the symbols +are version 2 rather than an addition to version 1 — a version-1 reader was +promised a closed set of origins, and the honest fix adds to that set. + +The durable record follows the ordinary rules. Selection itself records +`{ kind: "protected" }` and nothing else: there is no path, no origin to look up +and no implementation to serialize, so the record says only which tier answered +and replay asks the running execution for the implementation it built. An +execution that built none refuses rather than resolving the name again, because a +replay that fell back to the ordinary tiers would run whatever is offered under +that name today. Each occurrence then claims the identity this execution minted, +performs one `syntax_symbols` read, and retains exactly +`{ symbols: string }`. A continuation hostile-parses that record +and hands the same text back without consulting the filesystem, the registry, the +bundle, the host or the lexical reference again; a missing, additional or +mistyped member is stale input rather than a component failure, and refuses +before output or binding. Two authored occurrences are two identities and two +reads, and repeated reads of one binding read nothing again. + +## The syntax symbols boundary `xmd syntax` answers what a document may write here, and answering must cost nothing. The boundary that makes that true is one operation with no authority. -**One catalog, two projections.** Core produces a `SyntaxCatalog` — version 1, a -fixed three-category tuple, entries sorted by name — and the CLI's Markdown and -JSON renderers each take that value. Neither renderer discovers anything, and +**One symbols, two projections.** Core produces a `SyntaxSymbols` — version 2, a +fixed three-category tuple, entries sorted by name — and the Markdown and JSON +renderers each take that value. Neither renderer discovers anything, and neither parses the other's output, so the two formats cannot describe different -environments. JSON is the canonical, lossless projection; Markdown is written -for a person and labels a schema it cannot faithfully summarize rather than -inventing a type for it. +environments. JSON is the canonical, lossless projection and belongs to the CLI; +Markdown is written for a person and labels a schema it cannot faithfully +summarize rather than inventing a type for it. + +**The Markdown renderer is core's**, because two things print it: the command, +which describes an environment nobody is running, and canonical ``, +which hands the same text to a document that is running. One renderer is what +makes those the same bytes for the same site; two that agreed by hand would be +one release away from telling an operator and an agent different things about +one profile. **A declaration set is admitted on execution's terms, or not at all.** The identity components a host would declare to an execution are read here without @@ -3660,10 +3894,45 @@ exact Markdown a host would declare to an execution is admitted here on execution's terms too, and each declared name contributes one complete entry under built-in, reporting `declared-markdown` as its source kind and the declared origin and digest as its origin. Its private closure contributes -nothing: those names are not syntax a document may write, so a catalog that -listed them would describe an environment that does not exist. - -**Inspection is observation, never authority.** Producing a catalog installs +nothing: those names are not syntax a document may write, so a set of symbols +that listed them would describe an environment that does not exist. + +**An execution carries one of its own, lexically.** The symbols a running +document is shown travel by value on canonical core's private expansion +authority, beside the import authority and the identity domains — not through a +Context, because a context resolves by name and a name is not a secret, so a +document could build one and answer for the vocabulary it is shown. + +Canonical core builds it at the execution root from the selection inputs that +execution captured before any installation, middleware or document code ran: the +includes it resolves against, the registry it started with, the identity +components and exact Markdown its host declared, and the component bundle it is +closed over when it has one. A bundled name is described from the pinned bytes +already in hand — nothing is imported, executed, or read from a file to describe +one — and reported at the canonical repository-relative path that blob has in the +commit, which is where a reader of a workflow run looks for it. + +A trusted host may state the symbols its profile describes instead, on the terms +every other trusted-host value travels: captured by value with the rest of the +installation, before any installed code exists. One execution accepts one, and +two are refused rather than ordered, because ordering them would make which +profile a document is shown depend on assembly order. `xmd plan` states one — a +Plan is a program a later `xmd run` executes, so the vocabulary the agent must be +shown is that profile's rather than the authorship execution's, which searches no +repository and refuses almost every capability. An ordinary run states none and +describes itself. + +Nothing is built until an occurrence asks, so a run whose document never writes +`` enumerates no includes, parses no component and reads no +frontmatter. Each ask builds afresh, which is what makes an occurrence's retained +symbols its own rather than a copy of whichever one ran first. + +A trusted canonical evaluation boundary may install a narrower reference for +the subtree it evaluates, and leaving that subtree restores the enclosing one. It +adds nothing: the symbols it installs are the ones its own admission already +selected, so an entry absent from that admission is absent from the reference. + +**Inspection is observation, never authority.** Producing symbols installs only the declarative registration layer selection needs. It does not enter `execute()`, construct a durable stream, install a Files, Service, Agent or elicitation provider, start testing, reserve the terminal, mint an invocation @@ -3679,12 +3948,12 @@ candidate order, repository override and registered fallback are therefore not restated anywhere, and a repository component that overrides a default appears once, under user-provided, with its repository origin. -**A partial catalog is never presented as a complete one.** A missing include is +**A partial set of symbols is never presented as a complete one.** A missing include is ordinary absence. An include that exists but cannot be enumerated fails the whole request: a non-directory, an unreadable tree, a symbolic link where a directory was named, and a *selection-relevant* symbolic link to anything other than a file. Traversal never follows a link to a directory, so refusing is what -stops the catalog silently omitting everything beneath one. The diagnostic names +stops the symbols silently omitting everything beneath one. The diagnostic names the configured include and the logical entry, never the resolved host path. **Relevance is the link's own logical path**, because that path is what probing @@ -3700,8 +3969,8 @@ no name reaches. **Documented forms are canonical and checked.** A declaration says which authored forms a component accepts, and the spelling is closed: omission means both, and the only arrays are `["self-closing"]`, `["paired"]` and -`["self-closing", "paired"]`. One spelling per meaning is what lets a catalog be -compared without being normalized, so an empty array, a reversed pair, a +`["self-closing", "paired"]`. One spelling per meaning is what lets two entries +be compared without being normalized, so an empty array, a reversed pair, a repeated member and a form no invocation has are refused where the declaration is made — at registration and at the identity declarations inspection reads, because a declaration refused in only one of those places is refused only when a @@ -3826,12 +4095,13 @@ Status is measured against main. | Construct | Does | Status | | --- | --- | --- | -| `xmd syntax` | describes every structural construct and every selected component the production `run` profile would let a document write in the contextual working directory, as deterministic Markdown or as version-1 JSON, from one catalog. Inspection only: it registers the run profile's declarations in a bounded scope and reads the filesystem for which files exist and, for a selected Markdown component, that file's frontmatter. It runs no body, imports no repository TypeScript module, installs no provider, mints no authority and writes no journal. An include it cannot enumerate — a selection-relevant symbolic link to a directory beneath it included — fails the whole request rather than printing a healthy subset | built on the #632 stack | +| `xmd syntax` | describes every structural construct and every selected component the production `run` profile would let a document write in the contextual working directory, as deterministic Markdown or as version-2 JSON, from one construction. `xmd syntax Elicit` names one component instead and renders its symbol metadata followed by the long-form documentation the owning package ships, through the same selection, index and renderer `` uses. Inspection only: it registers the run profile's declarations in a bounded scope and reads the filesystem for which files exist and, for a selected Markdown component, that file's frontmatter. It runs no body, imports no repository TypeScript module, installs no provider, mints no authority and writes no journal. An include it cannot enumerate — a selection-relevant symbolic link to a directory beneath it included — fails the whole request rather than printing a healthy subset. The Markdown renderer is core's rather than this command's, because canonical `` prints the same symbols for a running document and the two must be the same bytes for the same site | built on the #632 stack, with the shared Markdown renderer added on this stack | +| `` | outputs the components and control-flow constructs a document may write at the site it is written at, as the Markdown `xmd syntax` prints — one construction and one renderer, so an operator and an agent are never told different things about one profile. Self-closing only, with one optional closed `names` prop, and a text component: the bare form lists the symbols available here, `` renders those components' metadata and the long-form documentation their owning package ships, and the ordinary `as` captures the same text and emits nothing — while a paired spelling, an unknown prop, an empty list, a duplicate or a non-string member refuses before anything is claimed or read. It is the one member of the canonical protected tier, resolved after structural syntax and ahead of every host or author tier: a repository `Syntax.md`, a bundled `Syntax`, an ordinary or reserved registration, a host's declared Markdown and an implementation from a second loaded copy can none of them answer for it, and `Component.importComponent` middleware may observe, delegate or refuse the import without being able to answer one. What the symbols say is the execution's own — built from the selection inputs it captured before any installation, middleware or document code ran, or from the one set of symbols a trusted host stated for its profile — and they are carried lexically on canonical core's expansion authority rather than through any context. The documentation the named form reads is collected the same way: each package's bootstrap contributes its own through the additive `Documentation` Api, canonical core is the terminal, and the execution collects once after the trusted host's bootstrap and snapshots by value before the root import, so middleware a running document installs composes into a chain nothing reads and two contributions that disagree about one owning package and component name refuse at collection whichever order they were bootstrapped in, while a repetition of the same four values — owner, asset, exact text and component set — adds nothing and succeeds, because one package is deliberately bootstrapped at more than one layer. Each occurrence claims the identity the execution minted, performs one `syntax_symbols` read, and retains exactly `{ symbols: string }`; a continuation hostile-parses that record and restores the text the run actually showed without rediscovering a moved environment, while a missing, additional or mistyped member is stale input that refuses before output or binding. A cancelled read completes its teardown and commits nothing. It reports itself under its own origin kind, `protected`, never as a reserved registration, and selection records `{ kind: "protected" }` alone — replay asks this execution for the implementation it built rather than resolving the name again. It carries no authority at all: a component the symbols name is neither registered, resolved nor authorized by being named | built on this stack; the narrower reference a trusted evaluation boundary installs for its subtree is the seam #713 fills | | document validation | validates one supplied root projection and the recursive Markdown source closure normal component selection discovers, returning deterministic version-1 document diagnostics and `valid`, `invalid` or `not-statically-checkable` invocation outcomes without evaluating document code or installing operational host behavior | built on the #654 stack | | `xmd plan` | turns one Prompt into approved Plan source and delivers it, by executing exactly one root document and starting no program. That root is the packaged plan command document, under the internal `` identity — an adapter that projects the request into `` and returns what comes back — and inside that, the packaged `` Component under the authorship profile its own `` installs: one enclosing Session, a host ceiling of one host-owned directory dedicated to that logical session — under `~/.xmd/plan/sessions` by default, keyed by the digest of the name, never the name, created empty and required to be empty before the provider exists or a session is materialized, refused rather than cleaned when it is not, durable when the caller named the session and handed back non-recursively after teardown when it did not — with no additional directories, no MCP servers, no native tools and a private strict denial no permission flag widens, no Files, command, service or network capability for that document, no repository component search, and the Component's own private ``, whose closed assessment reports the structural defects the draft authored. The invocation settles one structural check — `validateDocumentStructure` under the ordinary run-profile registry, the `` identity, the caller's includes and the run profile's declarations — and ``, `` and the command's own final gate all ask that one, so what they can disagree about is when it was asked rather than what was asked. There is no caller-source defect to tell them apart from: the grammar is fixed and complete before a draft exists, because no generated document adds an option to a command whose result *is* the document. Its instructions require every Plan to begin with one descriptive level-one title and to keep the Prompt's outcomes as readable steps with each component beside the step it performs, through repairs and revisions alike; that is an authorship and human-review requirement, and `` never enforces it. A tenth draft that still has problems may be stopped or explained: the explanation is one more ordinary turn in the same Session carrying only the final diagnostics, is inert text, reopens no draft limit, and ends the command. The host's instruction layer states only that an answer belongs to the message that asked for it, so which shape a turn wants stays in the document. Authorship reports itself as it happens: the command root's rendered transcript is progress on stderr — one Markdown phase announced before each piece of work, its attempt and repair ordinals derived from the two bounds `Plan.md` binds once, with `--verbose` adding every cleared draft and every failed check's structured findings — written through a private paired `` that renders its content, sends it through the current document-output operation and returns nothing, so a phase can never enter ``'s capture or the declaration's exact-source disposition. Which surface is asking and whether `--verbose` was written are sealed host facts, so the ordinary `` surface announces nothing and expands no progress body. The host owns the stream and the terminal alike: whitespace normalization for every invocation, terminal formatting only when the entrypoint states its own stderr is one, the transcript drained inside the scope that owns the execution, and a destination that stops accepting bytes cancelling the producer and waiting for every owned teardown before it reports — with no stdout fallback. Authorship's own durable stream is the host's choice and is written rather than read: a fresh invocation-owned in-memory one, or the file `--journal` exclusively created, holding the ordinary `serializeDurableEvent()` JSONL in commit order under the same serialized pre-append secret gate, so a rejected event reaches neither the file nor the committed sequence and the prefix before it stays readable. Nothing opens either as input, replays it, or resumes from it. Then, only after every provider, Prompt task and Elicitation resource inside the authorship frame has torn down, the Component's own `` structurally admits the exact approved bytes, and after that execution ends the host asks the same check once more — about a tree the whole teardown has had time to move, so a component the approved Plan names and something removed after that admission is refused here and nowhere else, and a Plan declaring properties a later run will supply is admitted rather than refused — and delivers it to exactly one sink: stdout byte for byte by default, or an exclusively created `--output` path. It starts no later root and retains no later execution, so the caller's own composition decides whether the program runs: `xmd plan … | xmd run -`, or an `--output` artifact a later `xmd run` names. Its grammar is what that leaves: one request, `--include`, `--agent-provider`, `--default-agent`, `--session`, `--timeout`, `--output`, `--verbose`, `--journal`, and ordinary help and version. The last two are spelled in full and observe this authorship alone — `-V` and `-j` are `xmd run`'s aliases for options about a program's run, and each is refused by naming the long spelling. Every option that configured the former execution — `--run`, the aggregate and generated root properties, `--raw`, the exec and fetch deadlines, the three permission flags and both secret-detection spellings — is refused by name in fixed preflight, before the general parser can drop or coerce a token and before `--help` can short-circuit the dispatch, in either order; `--run` reports the migration that names both compositions, and every other one reports that `xmd run` is where a program is configured. The same options are unchanged under `xmd run`. No permission mode is settled at all. The retired `prompt` spelling is not a command and is not absorbed by the default `run` grammar, which would read it as a document reference and execute a file of that name: an invocation whose exact first token is `prompt` is refused before any scan, selection or path lookup, establishing nothing, while `xmd run ./prompt` still executes a document legitimately called that | built on the #660 stack | -| `` | writes and reviews one Plan, from a Prompt an ordinary document wrote. `…` expands its paired body once with the capabilities the calling document already has — the complete untrimmed rendering is the Prompt, and it is never emitted separately — and produces the exact approved Plan source. It is a paired **exact text** component: the bare form emits that source into the calling document's own rendering, and the `as` form captures the same bytes instead. Neither form evaluates what it produced, and neither announces a phase: the progress `xmd plan` writes is a private side effect of the command surface, so an ordinary `` expands no progress body at all. It is the public name of the packaged `` Component: exact first-party Markdown declared to every ordinary run, so a repository `Plan.md`, a workflow bundle, a registration, `Component.importComponent` middleware and another loaded copy can none of them answer for it. Paired only, with one optional non-empty `session` prop and an optional `as`; a body that renders to nothing fails before any catalog, directory, Session, turn, review or check exists. The Agent writes under exactly the `xmd plan` fixed policy however broad the calling document's authority is, and a host that supplies no Agent context refuses before placement — including the `` child an `xmd test` document launches, which is the run profile and therefore resolves the same protected bytes rather than reporting a missing component, and which supplies one when it declares a canonical ``. An authored `session` keeps the existing durable named-directory lifetime, so the same name at the same site reaches the same conversation next time, while an omitted one is site- and iteration-unique, replay-stable and handed back after teardown; sibling sites stay distinct even when they write one name, and the name never becomes a path. Every turn, answer, check, approval and admission belongs to the enclosing document's journal, so a continuation restores completed authorship instead of repeating it; there is no second journal. Complete authorship teardown precedes structural admission, which precedes the emission or the binding — and the admission is structure alone, so a Plan declaring properties a later run will supply is produced rather than refused. It creates no file and executes nothing it produced | built on the #660 stack; no delivery, custom root, policy prop or replacement selector exists (#536 owns constrained caller-authored policy) | +| `` | writes and reviews one Plan, from a Prompt an ordinary document wrote. `…` expands its paired body once with the capabilities the calling document already has — the complete untrimmed rendering is the Prompt, and it is never emitted separately — and produces the exact approved Plan source. It is a paired **exact text** component: the bare form emits that source into the calling document's own rendering, and the `as` form captures the same bytes instead. Neither form evaluates what it produced, and neither announces a phase: the progress `xmd plan` writes is a private side effect of the command surface, so an ordinary `` expands no progress body at all. It is the public name of the packaged `` Component: exact first-party Markdown declared to every ordinary run, so a repository `Plan.md`, a workflow bundle, a registration, `Component.importComponent` middleware and another loaded copy can none of them answer for it. Paired only, with one optional non-empty `session` prop and an optional `as`; a body that renders to nothing fails before any inspection, directory, Session, turn, review or check exists. The Agent writes under exactly the `xmd plan` fixed policy however broad the calling document's authority is, and a host that supplies no Agent context refuses before placement — including the `` child an `xmd test` document launches, which is the run profile and therefore resolves the same protected bytes rather than reporting a missing component, and which supplies one when it declares a canonical ``. An authored `session` keeps the existing durable named-directory lifetime, so the same name at the same site reaches the same conversation next time, while an omitted one is site- and iteration-unique, replay-stable and handed back after teardown; sibling sites stay distinct even when they write one name, and the name never becomes a path. Every turn, answer, check, approval and admission belongs to the enclosing document's journal, so a continuation restores completed authorship instead of repeating it; there is no second journal. Complete authorship teardown precedes structural admission, which precedes the emission or the binding — and the admission is structure alone, so a Plan declaring properties a later run will supply is produced rather than refused. It creates no file and executes nothing it produced | built on the #660 stack; no delivery, custom root, policy prop or replacement selector exists (#536 owns constrained caller-authored policy) | | `` / `` | chooses one branch by comparing a value with `===`. `……` decides its whole case structure from source before evaluating anything, then evaluates the selector once and each non-default matcher at most once in source order, expands the first `===` match — or the final default, or nothing — inline and transparently, and appends no journal event | built on the #692 stack | -| `xmd upgrade` | replaces the standalone binary that ran it with a published release, by executing one root document: the packaged upgrade command document, under the internal `` identity, with an empty component search path and no Files, Process, Service, command, Fetch, Agent, Elicitation, workflow or repository capability. That document is an **ordinary streaming text root** — it declares no `returns` and uses neither `` nor `` — so its rendered body is the command's output: each root segment reaches the reader as it completes, and a branch the command did not take contributes no prose, no phase call and no result. Its durable events go to one invocation-local in-memory stream, or to the file `--journal` named and the CLI exclusively created; neither is ever read back, and neither grants any resume or retry authority. Markdown owns the whole of the policy — the exact-tag grammar, which release is selected, semantic-version comparison through the npm `semver` package, which consent an install needs, the status, already-current and installation branches, and the wording of every refusal and every report. A compiled macOS or Linux binary whose platform the release publishes for is the only host that declares the four phases that policy may reach, ``, ``, `` and ``, and it declares them to canonical execution rather than through any contextual Api, middleware, repository lookup, ordinary `xmd run` profile or public syntax catalog; every other entrypoint states its provenance and no authority at all, so an npm, Bun, Deno-source or compiled Windows invocation has no phase to reach and stops at its own refusal before release lookup or any filesystem change. That host alone owns the private half: the exact `process.execPath` spelling it will replace and never a link it resolved, one non-blocking exclusive advisory lock on a stable sidecar beside that file, the bounded anonymous GitHub reads under a scope-bound abort signal, the downloaded bytes, the digest, the staged candidate it runs for its version, and one same-directory rename. Opaque identity is the boundary between the two halves — a release identity per admitted release, then one candidate advancing `downloaded → verified → committed` exactly once, with one installation attempt per invocation — so the document chooses among the releases it was shown and can name no other release, target, asset or destination, skip verification or replay a phase. Before the rename every failure and cancellation leaves the installed file byte-identical; after it the candidate is authoritative and no cleanup restores the old bytes | built on the #659 stack | +| `xmd upgrade` | replaces the standalone binary that ran it with a published release, by executing one root document: the packaged upgrade command document, under the internal `` identity, with an empty component search path and no Files, Process, Service, command, Fetch, Agent, Elicitation, workflow or repository capability. That document is an **ordinary streaming text root** — it declares no `returns` and uses neither `` nor `` — so its rendered body is the command's output: each root segment reaches the reader as it completes, and a branch the command did not take contributes no prose, no phase call and no result. Its durable events go to one invocation-local in-memory stream, or to the file `--journal` named and the CLI exclusively created; neither is ever read back, and neither grants any resume or retry authority. Markdown owns the whole of the policy — the exact-tag grammar, which release is selected, semantic-version comparison through the npm `semver` package, which consent an install needs, the status, already-current and installation branches, and the wording of every refusal and every report. A compiled macOS or Linux binary whose platform the release publishes for is the only host that declares the four phases that policy may reach, ``, ``, `` and ``, and it declares them to canonical execution rather than through any contextual Api, middleware, repository lookup, ordinary `xmd run` profile or public syntax symbols; every other entrypoint states its provenance and no authority at all, so an npm, Bun, Deno-source or compiled Windows invocation has no phase to reach and stops at its own refusal before release lookup or any filesystem change. That host alone owns the private half: the exact `process.execPath` spelling it will replace and never a link it resolved, one non-blocking exclusive advisory lock on a stable sidecar beside that file, the bounded anonymous GitHub reads under a scope-bound abort signal, the downloaded bytes, the digest, the staged candidate it runs for its version, and one same-directory rename. Opaque identity is the boundary between the two halves — a release identity per admitted release, then one candidate advancing `downloaded → verified → committed` exactly once, with one installation attempt per invocation — so the document chooses among the releases it was shown and can name no other release, target, asset or destination, skip verification or replay a phase. Before the rename every failure and cancellation leaves the installed file byte-identical; after it the candidate is authoritative and no cleanup restores the old bytes | built on the #659 stack | | `` / `printErrors(fn)` | prints failures | built on main | | `` | stops authored work with the sentence its author wrote, raised where it is written. An ordinary overridable core default — never structural, never reserved, so a repository `Fail.md` is chosen ahead of it — with a closed schema of one required non-empty `message` and **self-closing only**: a paired spelling never enters its body, and `as` is refused by the body itself because there is nothing to bind. Every refusal reports the invocation and happens before the authored message, so a document that never reached its decision is never reported as having made one. A valid invocation is the ordinary failure of a function component: an `Error` carrying the exact authored message, positioned at the opening tag, rendering nothing and binding nothing. It carries no `printErrors()` declaration, which is what leaves recovery to an authored `` region under ordinary text-root modes; a value body's `throw` is not replaced there, so the authored failure settles the body ahead of missing-`` settlement. No authority, context, provider, resource, module state or durable operation of its own: replay of a completed root restores the recorded outcome without re-expanding the body | built on the #659 stack | | `` | binds one name in the current environment from exactly one source: the content it renders, or the exact value `value` names, bound by reference and never through the JSON boundary component props cross — the scanner resolves no JSON for that one prop, and expansion projects none. Which source it has is read from what the author wrote, before either one runs, so a construct naming both expands no child and evaluates no expression. It opens no scope, owns no resource, adds no middleware boundary and writes no journal record — replay reconstructs both sources through ordinary expansion | built on the #527 stack | @@ -3895,7 +4165,7 @@ Status is measured against main. | history fork | creates a new run from one compatible checkpoint and retained Workspace root, under a new immutable definition and normalized props | built on the #368 stack, Deno provider only | | workflow Agent session | a workflow document's `` runs under a profile the host attaches only for a live or partial run: an empty host-owned working directory instead of any Workspace, checkout or caller path, no MCP servers, an empty requested native tool set, and `deny-all` with a permission path that denies every native request and fails the turn that asked without reaching the public permission chain. Within a run a session is identified by the Agent/Session expansion identity the engine derived — the authored name is descriptive, so two sibling `` elements are two sessions — routed inside a placement bound to its element and good for one use, so a kept placement cannot be substituted for the next. The conversation is retained as a row in the run's own database with the provider, resolved agent command and policy fingerprint beside it as compatibility attributes. The order is placement, the backend's acceptance of the session's first turn, the provider's canonical tagged assertion, then the mapping commit — and only then is anything that turn produced exposed. A placement is inert: it creates no provider session and writes no row. Occupancy of a provider key is not an assertion, and a record held for a first turn nobody accepted asserts nothing at all; the pre-commit window reconciles only from exactly one canonical assertion, and a missing, conflicting, replaced or ambiguous assertion is one explicit refusal that starts no replacement. Deleting a run removes the row with the run and the provider-session directory beside it, and reports the categories. The profile selects ACP-only capability explicitly — no native-launch advertisement and no client-native attachment advertisement — rather than inheriting the provider package's ordinary-run sets by omission, and it supplies no machine session coordinator, construction-route store or executable observer: a workflow session belongs to a run, and the machine-wide account describes a different thing entirely | built on the #302 stack, with the explicit ACP-only selection from #561; the portable proof that an adapter honours an empty tool set is tracked by #496 and does not widen the ceiling | | generated-XMD admission | admits one Agent-generated fragment through the trusted-host seam: host policy is a `read` table and a `write` table of exact pinned identities, each carrying the authored forms it is admitted for, and an authored `allow` selects a canonical non-empty subset of the closed classes — omitted means `read`. The complete source is preflighted inside one `generated_xmd` durable effect before its first generated effect; only the pinned identity the selected classes hold for that name **and** that form executes; and the admitted source, class selection, selected root, every selected entry with its forms, the identity and form of each element named, and the normalized request policy are retained in that effect's own result — so a continuation restores the decision without reading the current candidate and expands only the retained source. The roots are an as-of-admission retained basis checked by membership — the run's own later root publications and an advanced retained current root pass, while a lost admission root or lost selected root refuses — and every non-root term is checked exactly, refusing a run whose classes, identities, forms or requests have moved. The admission and every nested generated effect are offered inline by the owning expansion in authored order, so a partial continuation restores each completed one without another live execution. Each admitted effect is retained by its own ordinary record, and a read's value is collected while a mutation's is not | built on the #369 stack, continuation basis amended by #589; core owns the mechanics and the workflow policy wrapper is internal | -| `` | the workflow host's component an authored document writes where an observation should happen. The host does not register it: it **declares** it to the execution through `ExecutionInstallation.components`, captured before any installation runs, and canonical execution calls its factory once for that attachment with the claimant it minted and registers what comes back. Registration provides availability only — a name a trusted document may write — and carries none of the authority. Its schema is closed on one required `source` string and one optional `allow` array selecting a non-empty duplicate-free subset of the closed effect classes `read` and `write` — omitted means `read` — and paired content is refused. It declares no `returns` and answers with a detached value — `{ observations: [{ name, value }], output }`, each admitted read's own returned value under the name the fragment invoked it by, in invocation order, with whatever the fragment rendered under `output` rather than instead of them, and the pinned identity that produced one left in the retained admission rather than copied here — so an admitted ``, which renders nothing at all, still reaches the document. An admitted mutation contributes no entry and no receipt, so a write-only fragment binds `{ observations: [], output: "" }`; `as` is valid for every selection and binds that same shape. An ordinary `as` captures that value by reference, and an authored `` renders it into the next ``: deciding how a value becomes text is the document's. Every ceiling comes from values the host captured at installation — the run's retained roots and its authoritative current root read from the run's own storage at invocation, as-of-admission provenance a continuation holds by membership so the run's own later publications and an advanced retained current root invalidate nothing, core's pinned self-closing `` read, the write table of core's paired ``, this package's lexical `` built from the definition the ordinary registration owns, and core's self-closing ``, and `` only when the captured request ceiling is non-empty — and no prop, binding, context or middleware return value supplies or widens one. `allow` selects among those tables and adds nothing to them; approval, when a workflow needs one, is authored control flow before the element. Its durable operation is named through that claimant, on the exact invocation the engine handed it and in that invocation's own frame — not from a context a document could rebind, a contextual Api answer, a definition, or a registry answer. Generated source never resolves through the registration: the evaluator consults only its own closed table of pinned identities. It is deliberately not wrapped in `printErrors`, so a refused fragment stops the authored loop rather than becoming text the next turn could read as a read that happened | built on the #302 stack, extended by #369 | +| `` | canonical core's own protected component, written where program text the document did not author should run. Core claims the name ahead of every host and author tier, so no registration, repository file, bundle member, declared Markdown component, import handler answer or second loaded copy replaces it; a handler may observe or refuse the import, and only canonical execution answers one. Protection settles which implementation runs and grants nothing: a host supplies the *ceiling* as one `ExecutionInstallation.evaluation` profile, captured by value before any installation runs, and an execution accepts one and refuses two. Its schema is closed on `text`, the workflow-only `source` alias, and an optional `allow` array selecting a non-empty duplicate-free subset of the closed effect classes `read` and `write` — omitted means `read`. The two input forms are disjoint: `text` states the program, paired content renders it, and an element stating both is refused rather than resolved by precedence. A paired producer renders under the narrowed syntax reference through an execution-owned one-shot projection that bypasses the public `content()`/`tryContent()` chain, and keeps its own operational authority while doing so. It declares no `returns` and answers with a detached value — `{ observations: [{ name, value }], output }`, each admitted read's own returned value under the name the fragment invoked it by, in invocation order, with whatever the fragment rendered under `output` rather than instead of them, and the pinned identity that produced one left in the retained admission rather than copied here — so an admitted ``, which renders nothing at all, still reaches the document. An admitted mutation contributes no entry and no receipt, so a write-only fragment binds `{ observations: [], output: "" }`; `as` is valid for every selection and binds that same shape. An ordinary `as` captures that value by reference, and an authored `` renders it into the next ``: deciding how a value becomes text is the document's. Every ceiling comes from values the host captured at installation — the run's retained roots and its authoritative current root read from the run's own storage per invocation, as-of-admission provenance a continuation holds by membership so the run's own later publications and an advanced retained current root invalidate nothing, core's self-closing `` read, the write table of core's paired ``, the workflow's `` and core's self-closing ``, and `` only when the captured request ceiling is non-empty — and no prop, binding, context or middleware return value supplies or widens one. An entry states what is behind a name in one of exactly two ways, and neither carries a function. A *capability* names an operation core supplies the body for: canonical capture reads the host's own operation off once, binds it behind a revocation the execution owns, and closes core's own body over it, so an admitted element reaches those operations and never `API.Files`, `API.Fetch` or `API.Env`. A *component answer* names an implementation the ordinary import chain resolves, and states only the identity a provider must have claimed for it; canonical execution resolves that name once — before the root import and before any document code, through the complete ordinary `Component.importComponent` chain and a private terminal that writes no record — and asks the identity owner about the exact final answer in one call. That call answers with the claim *and* core's own copy of what was claimed, taken when the claim was recorded: the check and the thing kept are one result, so nothing downstream reads the chain's object a second time and an answer whose members read differently on each read cannot pass a check with one reading and be sealed with another. The identity is then compared whole, and the copy — not the answer — is sealed behind the same revocation. One name states one identity, however many forms and tables hold it: two entries under one name are the two spellings of one component and share one lookup and one sealed implementation, while a second identity for that name, or a name held as both a capability and a component answer, refuses at capture rather than letting assembly order decide. Provider installation and occurrence authority are separate: installation receives a registrar that installs import middleware, and every invocation of that middleware receives a fresh request fixed to the exact name, position, provider installation, origin and resolution-window object it was asked under. Only that request may claim, and it states the answer plus key and revision without restating the name. Canonical execution closes the request synchronously when its handler returns, fails or is cancelled; an outer request remains live while it delegates and may claim a replacement after the inner handler returns. Enclosing resolution still closes in its own `finally` on success, fallback, failure or cancellation. A claim therefore requires the execution, request and exact current window to remain active, the request's fixed name to match that window, and that provider installation not to have stated a different answer in the window. Identification takes the expected window explicitly and accepts only a claim recorded for that exact object and name, so a stale request cannot answer a later same-name resolution or retag itself while another name is live. The installation is reusable: one provider may answer several admitted names and repeated resolutions through distinct requests. All of this state is held in ordinary private closures — no Context, shared symbol, public brand or module-global registry — and an unidentified middleware replacement remains a valid ordinary import answer outside fragment evaluation. A capability entry also states the exact version-1 identity strings it succeeds, which is the only thing a released untagged record reconciles against; a component answer states none. A capability-only profile performs no such lookup at all. Resolution happens at capture and never again: a fragment runs the sealed snapshot, a continuation resolves once more in its own capture and reconciles before any effect, and a provider still answering when a fragment resolves its admitted name is answering a generated import, which only canonical execution answers. A trusted layering control may enter one package bootstrap through an inherited layer and an execution-local one — documentation, registrations and the provider together — and the repeat is additive: the outer entry observes and delegates rather than claiming a second identity for an implementation that already states what it is, and a non-identical documentation overlap on one owner and component still refuses at the child collection boundary, before the root import and therefore before the provider is asked at all. `allow` selects among those tables and adds nothing to them; approval, when a workflow needs one, is authored control flow before the element. Its durable operation is named through that claimant, on the exact invocation the engine handed it and in that invocation's own frame — not from a context a document could rebind, a contextual Api answer, a definition, or a registry answer. Generated source resolves through none of that: the evaluator consults only its own closed table of pinned identities. It is deliberately not wrapped in `printErrors`, so a refused fragment stops the authored loop rather than becoming text the next turn could read as a read that happened | built on the #302 stack, extended by #369 | | generated mutation proposals | lets an Agent propose constrained executable changes that a separate admission then performs against the run's own Workspace | built on the #369 and #567 stacks, with directory creation added by #643: the standard Deno profile's write table is core's paired `File:write`, the paired `@executablemd/workflow/composition/dir-v2#Dir` and core's self-closing `File.Delete`, in that retained order and followed by any host extension. `allow={["write"]}` intentionally authorizes Dir's persistent recursive directory creation; its versioned identity makes every continuation retained under the former non-mutating Dir identity refuse before generated execution. Admitted mutations run as the ordinary components they are through the run's effect transactions, and the evaluator adds no mutation API or receipt. Approval is authored control flow before the write-enabled element. Local Git, Git-host, issue, process, execution, credential and external-write effects are outside the class | | Deno-local DOFS provider | owns one authoritative SQLite/DOFS connection per run path, captures arbitrary canonical retained roots, privately restores them, and atomically coordinates one Workspace mutation with its filtered Yield | built on the #365 stack; public document filesystem effects and the CLI lifecycle route to it on the #366 stack | | scoped Worker Shell | executes `just-bash` through the Workspace adapter inside a Deno Worker | containment and effect-transaction POCs complete (#351, #357); production integration unbuilt | diff --git a/deno.json b/deno.json index 7b0e6ff0b..923155a79 100644 --- a/deno.json +++ b/deno.json @@ -33,7 +33,7 @@ "@effectionx/fetch": "npm:@effectionx/fetch@0.2.1", "@effectionx/fs": "npm:@effectionx/fs@0.3.0", "@effectionx/middleware": "npm:@effectionx/middleware@0.1.1", - "@effectionx/node": "npm:@effectionx/node@0.2.4", + "@effectionx/node": "npm:@effectionx/node@0.2.5", "@effectionx/process": "npm:@effectionx/process@0.8.1", "@effectionx/scope-eval": "npm:@effectionx/scope-eval@0.1.3", "@effectionx/stream-helpers": "npm:@effectionx/stream-helpers@0.8.3", @@ -58,7 +58,8 @@ "verify:clean": "deno run --allow-all --node-modules-dir=none --cached-only --frozen scripts/verify-clean.ts", "deps": "deno run --allow-all scripts/deps.ts", "deps:target": "deno run --allow-all scripts/deps-target.ts", - "build": "deno task build:web && deno compile --node-modules-dir=none --cached-only --frozen --exclude-unused-npm --allow-all --include packages/code-review-agent --include packages/cli/src/documents --output dist/xmd packages/cli/src/compiled.ts", + "validate:docs": "deno run --allow-all scripts/validate-documentation.ts", + "build": "deno task validate:docs && deno task build:web && deno compile --node-modules-dir=none --cached-only --frozen --exclude-unused-npm --allow-all --include packages/code-review-agent --include packages/cli/src/documents --include packages/core/src/components/components.md --include packages/core/src/agent/components.md --include packages/cli/src/components.md --include packages/testing/src/components.md --include packages/web/src/components.md --include packages/workflow/src/composition/components.md --output dist/xmd packages/cli/src/compiled.ts", "build:web": "deno run --allow-all --node-modules-dir=none --cached-only --frozen scripts/preflight.ts scripts/build-web-client.ts", "gen:publish-workflow": "deno run --allow-all packages/cli/src/deno.ts run scripts/gen-publish-workflow.md", "bump": "deno run -A scripts/bump-version.ts", diff --git a/deno.lock b/deno.lock index a0c581cff..7772922ea 100644 --- a/deno.lock +++ b/deno.lock @@ -48,7 +48,7 @@ "npm:@effectionx/fetch@0.2.1": "0.2.1_effection@4.1.0", "npm:@effectionx/fs@0.3.0": "0.3.0_effection@4.1.0", "npm:@effectionx/middleware@0.1.1": "0.1.1", - "npm:@effectionx/node@0.2.4": "0.2.4_effection@4.1.0", + "npm:@effectionx/node@0.2.5": "0.2.5_effection@4.1.0", "npm:@effectionx/process@0.8.1": "0.8.1_effection@4.1.0", "npm:@effectionx/scope-eval@0.1.3": "0.1.3_effection@4.1.0", "npm:@effectionx/stream-helpers@0.8.3": "0.8.3_effection@4.1.0", @@ -548,11 +548,17 @@ "effection" ] }, + "@effectionx/node@0.2.5_effection@4.1.0": { + "integrity": "sha512-hL8mROda8Lx375MVS+Ubu86+yMht/I0wOZG5VR6Pel0XUA5ReObQDYvNS6ocW0cNFKnEmvlVLWGWbcjJ+VkVhA==", + "dependencies": [ + "effection" + ] + }, "@effectionx/process@0.8.1_effection@4.1.0": { "integrity": "sha512-xyXlFja0Ill80lQ3IYfksXtJkqVmWuUOogRn/qlHWCAGlZj+MGGF8gOFbyzk/3Kx4pj14riVGgF/cyT5XCzqDw==", "dependencies": [ "@effectionx/context-api", - "@effectionx/node", + "@effectionx/node@0.2.4_effection@4.1.0", "@effectionx/scope-eval", "cross-spawn", "ctrlc-windows", @@ -3966,7 +3972,7 @@ "npm:@effectionx/fetch@0.2.1", "npm:@effectionx/fs@0.3.0", "npm:@effectionx/middleware@0.1.1", - "npm:@effectionx/node@0.2.4", + "npm:@effectionx/node@0.2.5", "npm:@effectionx/process@0.8.1", "npm:@effectionx/scope-eval@0.1.3", "npm:@effectionx/stream-helpers@0.8.3", @@ -3997,7 +4003,7 @@ "npm:@effectionx/fetch@0.2.1", "npm:@effectionx/fs@0.3.0", "npm:@effectionx/middleware@0.1.1", - "npm:@effectionx/node@0.2.4", + "npm:@effectionx/node@0.2.5", "npm:@effectionx/process@0.8.1", "npm:@effectionx/scope-eval@0.1.3", "npm:@effectionx/stream-helpers@0.8.3", @@ -4081,7 +4087,7 @@ "npm:@effectionx/fetch@0.2.1", "npm:@effectionx/fs@0.3.0", "npm:@effectionx/middleware@0.1.1", - "npm:@effectionx/node@0.2.4", + "npm:@effectionx/node@0.2.5", "npm:@effectionx/process@0.8.1", "npm:@effectionx/scope-eval@0.1.3", "npm:@effectionx/stream-helpers@0.8.3", @@ -4117,7 +4123,7 @@ "npm:@effectionx/context-api@0.6.0", "npm:@effectionx/fetch@0.2.1", "npm:@effectionx/fs@0.3.0", - "npm:@effectionx/node@0.2.4", + "npm:@effectionx/node@0.2.5", "npm:@effectionx/process@0.8.1", "npm:effection@4.1.0" ] @@ -4126,7 +4132,7 @@ "packages/test-agent": { "dependencies": [ "npm:@agentclientprotocol/sdk@1.3.0", - "npm:@effectionx/node@0.2.4", + "npm:@effectionx/node@0.2.5", "npm:@effectionx/scope-eval@0.1.3", "npm:@effectionx/stream-helpers@0.8.3", "npm:acorn@^8.16.0", @@ -4136,7 +4142,7 @@ "packageJson": { "dependencies": [ "npm:@agentclientprotocol/sdk@1.3.0", - "npm:@effectionx/node@0.2.4", + "npm:@effectionx/node@0.2.5", "npm:@effectionx/scope-eval@0.1.3", "npm:@effectionx/stream-helpers@0.8.3", "npm:acorn@^8.16.0", @@ -4185,7 +4191,7 @@ "packageJson": { "dependencies": [ "npm:@effectionx/fs@0.3.0", - "npm:@effectionx/node@0.2.4", + "npm:@effectionx/node@0.2.5", "npm:@fontsource/montserrat@5.3.0", "npm:@fontsource/space-mono@5.3.0", "npm:@rjsf/core@6.7.1", diff --git a/package.json b/package.json index cd4385566..d17b1b1fe 100644 --- a/package.json +++ b/package.json @@ -30,7 +30,7 @@ "@effectionx/fetch": "0.2.1", "@effectionx/fs": "0.3.0", "@effectionx/middleware": "0.1.1", - "@effectionx/node": "0.2.4", + "@effectionx/node": "0.2.5", "@effectionx/process": "0.8.1", "@effectionx/scope-eval": "0.1.3", "@effectionx/stream-helpers": "0.8.3", diff --git a/packages/acp/tests/adapter-protocol.test.ts b/packages/acp/tests/adapter-protocol.test.ts index 461e539ee..e5cb3ce02 100644 --- a/packages/acp/tests/adapter-protocol.test.ts +++ b/packages/acp/tests/adapter-protocol.test.ts @@ -18,9 +18,18 @@ */ import { describe, it } from "@executablemd/test-support/bdd"; import { expect } from "@executablemd/test-support/expect"; -import { all, ensure, resource, spawn as effectionSpawn, until, withResolvers } from "effection"; +import { + all, + ensure, + resource, + scoped, + spawn as effectionSpawn, + until, + withResolvers, +} from "effection"; import type { Operation } from "effection"; import { rm } from "@effectionx/fs"; +import { Buffer } from "node:buffer"; import { spawn } from "node:child_process"; import type { ChildProcess } from "node:child_process"; import { mkdtemp } from "node:fs/promises"; @@ -36,6 +45,8 @@ const FAKE_CLAUDE = join(FIXTURES, "fake-claude-cli.cjs"); /** One ACP conversation with a spawned adapter. */ interface Adapter { + /** The adapter process, so a case can say what the resource still observes. */ + readonly child: ChildProcess; request(method: string, params: unknown): Operation>; /** * Every `session/update` notification the adapter sent, in arrival order. @@ -75,7 +86,7 @@ function useAdapter(provider: string, environment: Record): Oper let next = 1; let buffer = ""; - child.stdout?.on("data", (chunk: Buffer) => { + const onStdout = (chunk: Buffer): void => { buffer += chunk.toString("utf8"); let index = buffer.indexOf("\n"); while (index >= 0) { @@ -110,9 +121,20 @@ function useAdapter(provider: string, environment: Record): Oper child.stdin?.write(`${JSON.stringify({ jsonrpc: "2.0", id, result: {} })}\n`); } } + }; + + // Detached before the kill above, because destructors unwind in reverse: + // the adapter stops being read before the process it is reading is ended. + // Established before the subscription, because `yield* ensure(...)` is + // itself a suspension an owner can be halted at. + yield* ensure(() => { + child.stdout?.off("data", onStdout); }); + child.stdout?.on("data", onStdout); + yield* provide({ + child, updates, *request(method: string, params: unknown): Operation> { const id = next++; @@ -330,4 +352,33 @@ describe("Tier EA — the embedded adapters' prompt-response metadata", () => { expect(metaOf(first, "codex")).toEqual({ turnId: `turn:${sessionId}:1` }); expect(metaOf(second, "codex")).toEqual({ turnId: `turn:${sessionId}:2` }); }); + /** + * The adapter resource reads one child's stdout for as long as it holds it. + * The count is read after the resource has been torn down and before the + * event is replayed, because a handler removed by its own event would leave + * the same count behind as one the resource released. + */ + it("releases the adapter's output handler with the resource", function* () { + let child: ChildProcess | undefined; + let live = 0; + let before = 0; + + yield* scoped(function* () { + const adapter = yield* useAdapter("claude", {}); + child = adapter.child; + before = 0; + live = adapter.child.stdout?.listenerCount("data") ?? 0; + }); + + if (!child) { + throw new Error("the adapter never started"); + } + + expect(live).toBeGreaterThanOrEqual(before + 1); + expect(child.stdout?.listenerCount("data") ?? 0).toBe(live - 1); + + child.stdout?.emit("data", Buffer.from("after the adapter was torn down")); + + expect(child.stdout?.listenerCount("data") ?? 0).toBe(live - 1); + }); }); diff --git a/packages/acp/tests/fixtures/claude-native-launch-proof.ts b/packages/acp/tests/fixtures/claude-native-launch-proof.ts index 051835220..717991d94 100644 --- a/packages/acp/tests/fixtures/claude-native-launch-proof.ts +++ b/packages/acp/tests/fixtures/claude-native-launch-proof.ts @@ -354,30 +354,52 @@ function runChild( options.live.delete(running.pid); }); - child = spawnChild(command, args, { cwd: options.cwd, stdio: ["pipe", "pipe", "pipe"] }); - if (child.pid) { - options.live.add(child.pid); + const started = spawnChild(command, args, { + cwd: options.cwd, + stdio: ["pipe", "pipe", "pipe"], + }); + child = started; + if (started.pid) { + options.live.add(started.pid); } let stdout = ""; let stderr = ""; - child.stdout?.on("data", (chunk: Buffer) => { + + const onStdout = (chunk: Buffer): void => { stdout += chunk.toString(); - }); - child.stderr?.on("data", (chunk: Buffer) => { + }; + const onStderr = (chunk: Buffer): void => { stderr += chunk.toString(); - }); - child.once("error", (error: Error) => failed.reject(error)); - child.once("close", (code: number | null, signal: string | null) => { - if (child?.pid) { - options.live.delete(child.pid); + }; + const onError = (error: Error): void => failed.reject(error); + const onClose = (code: number | null, signal: string | null): void => { + if (started.pid) { + options.live.delete(started.pid); } settled.resolve({ code, signal, stdout, stderr }); + }; + + // Registered after the cleanup above and so torn down before it: the child + // stops being observed before it is signalled, and a race this arm loses + // leaves nothing attached to a process somebody else is still reading. + // Established before the subscriptions, because `yield* ensure(...)` is + // itself a suspension an owner can be halted at. + yield* ensure(() => { + started.stdout?.off("data", onStdout); + started.stderr?.off("data", onStderr); + started.off("error", onError); + started.off("close", onClose); }); + started.stdout?.on("data", onStdout); + started.stderr?.on("data", onStderr); + started.on("error", onError); + started.on("close", onClose); + if (options.input !== undefined) { - child.stdin?.write(options.input); + started.stdin?.write(options.input); } - child.stdin?.end(); + started.stdin?.end(); return yield* race([settled.operation, failed.operation]); })(); @@ -502,16 +524,17 @@ function ptyRun( options.live.delete(running.pid); }); - child = spawnChild("/usr/bin/script", ["-q", "/dev/null", command, ...args], { + const started = spawnChild("/usr/bin/script", ["-q", "/dev/null", command, ...args], { cwd: options.cwd, env: options.env, stdio: ["pipe", "pipe", "pipe"], }); - if (child.pid) { - options.live.add(child.pid); + child = started; + if (started.pid) { + options.live.add(started.pid); } - const react = (chunk: Buffer) => { + const react = (chunk: Buffer): void => { text += chunk.toString(); const waiter = pending; if (waiter && waiter.predicate(text.slice(consumed))) { @@ -520,17 +543,31 @@ function ptyRun( waiter.resolve(""); } }; - child.stdout?.on("data", react); - child.stderr?.on("data", react); - child.once("error", (error: Error) => failed.reject(error)); - child.once("close", (status: number | null) => { + const onError = (error: Error): void => failed.reject(error); + const onClose = (status: number | null): void => { code = status ?? -1; - if (child?.pid) { - options.live.delete(child.pid); + if (started.pid) { + options.live.delete(started.pid); } settled.resolve(); + }; + + // Registered after the interrupt cleanup above and so torn down before it: + // the terminal stops being read before the process holding it is signalled. + // Established before the subscriptions, because `yield* ensure(...)` is + // itself a suspension an owner can be halted at. + yield* ensure(() => { + started.stdout?.off("data", react); + started.stderr?.off("data", react); + started.off("error", onError); + started.off("close", onClose); }); + started.stdout?.on("data", react); + started.stderr?.on("data", react); + started.on("error", onError); + started.on("close", onClose); + const write = (bytes: string) => { // Everything already on screen belongs to the surface being answered, so // the next wait reads only what this write provoked. diff --git a/packages/acp/tests/fixtures/claude-native-to-acp-proof.ts b/packages/acp/tests/fixtures/claude-native-to-acp-proof.ts index 1012dd822..b218ecc1a 100644 --- a/packages/acp/tests/fixtures/claude-native-to-acp-proof.ts +++ b/packages/acp/tests/fixtures/claude-native-to-acp-proof.ts @@ -371,34 +371,53 @@ function runChild( options.live.delete(running.pid); }); - child = spawnChild(command, args, { + const started = spawnChild(command, args, { cwd: options.cwd, stdio: ["pipe", "pipe", "pipe"], ...(options.env === undefined ? {} : { env: options.env }), }); - if (child.pid) { - options.live.add(child.pid); + child = started; + if (started.pid) { + options.live.add(started.pid); } let stdout = ""; let stderr = ""; - child.stdout?.on("data", (chunk: Buffer) => { + + const onStdout = (chunk: Buffer): void => { stdout += chunk.toString(); - }); - child.stderr?.on("data", (chunk: Buffer) => { + }; + const onStderr = (chunk: Buffer): void => { stderr += chunk.toString(); - }); - child.once("error", (error: Error) => failed.reject(error)); - child.once("close", (code: number | null, signal: string | null) => { - if (child?.pid) { - options.live.delete(child.pid); + }; + const onError = (error: Error): void => failed.reject(error); + const onClose = (code: number | null, signal: string | null): void => { + if (started.pid) { + options.live.delete(started.pid); } settled.resolve({ code, signal, stdout, stderr }); + }; + + // Registered after the cleanup above and so torn down before it: the child + // stops being observed before it is signalled, and a race this arm loses + // leaves nothing attached to a process somebody else is still reading. + // Established before the subscriptions, because `yield* ensure(...)` is + // itself a suspension an owner can be halted at. + yield* ensure(() => { + started.stdout?.off("data", onStdout); + started.stderr?.off("data", onStderr); + started.off("error", onError); + started.off("close", onClose); }); + started.stdout?.on("data", onStdout); + started.stderr?.on("data", onStderr); + started.on("error", onError); + started.on("close", onClose); + if (options.input !== undefined) { - child.stdin?.write(options.input); + started.stdin?.write(options.input); } - child.stdin?.end(); + started.stdin?.end(); return yield* race([settled.operation, failed.operation]); })(); @@ -523,16 +542,17 @@ function ptyRun( options.live.delete(running.pid); }); - child = spawnChild("/usr/bin/script", ["-q", "/dev/null", command, ...args], { + const started = spawnChild("/usr/bin/script", ["-q", "/dev/null", command, ...args], { cwd: options.cwd, env: options.env, stdio: ["pipe", "pipe", "pipe"], }); - if (child.pid) { - options.live.add(child.pid); + child = started; + if (started.pid) { + options.live.add(started.pid); } - const react = (chunk: Buffer) => { + const react = (chunk: Buffer): void => { text += chunk.toString(); const waiter = pending; if (waiter && waiter.predicate(text.slice(consumed))) { @@ -541,17 +561,31 @@ function ptyRun( waiter.resolve(""); } }; - child.stdout?.on("data", react); - child.stderr?.on("data", react); - child.once("error", (error: Error) => failed.reject(error)); - child.once("close", (status: number | null) => { + const onError = (error: Error): void => failed.reject(error); + const onClose = (status: number | null): void => { code = status ?? -1; - if (child?.pid) { - options.live.delete(child.pid); + if (started.pid) { + options.live.delete(started.pid); } settled.resolve(); + }; + + // Registered after the interrupt cleanup above and so torn down before it: + // the terminal stops being read before the process holding it is signalled. + // Established before the subscriptions, because `yield* ensure(...)` is + // itself a suspension an owner can be halted at. + yield* ensure(() => { + started.stdout?.off("data", react); + started.stderr?.off("data", react); + started.off("error", onError); + started.off("close", onClose); }); + started.stdout?.on("data", react); + started.stderr?.on("data", react); + started.on("error", onError); + started.on("close", onClose); + const write = (bytes: string) => { // Everything already on screen belongs to the surface being answered, so // the next wait reads only what this write provoked. diff --git a/packages/cli/src/authorship-profile.ts b/packages/cli/src/authorship-profile.ts index cd45e03a6..f67d26496 100644 --- a/packages/cli/src/authorship-profile.ts +++ b/packages/cli/src/authorship-profile.ts @@ -45,7 +45,7 @@ import { useTerminalOutput, } from "@executablemd/core"; import type { AgentProviderOptions, Json } from "@executablemd/core"; -import type { DeclaredMarkdownComponent } from "@executablemd/core/host"; +import type { SyntaxSymbolsProvider, DeclaredMarkdownComponent } from "@executablemd/core/host"; import { executeInstalled, installInvocationAgentProvider } from "@executablemd/core/host"; import { createAcpxProvider } from "@executablemd/acp"; import type { AcpxProviderDependencies } from "@executablemd/acp"; @@ -163,11 +163,21 @@ export interface AuthorshipProfile { * The `` declaration this command runs under. * * Built by the command, from the packaged Component's bytes, before the adapter - * root is imported. It carries the sealed surface, the sealed verbosity, the - * catalog this invocation will build and the Agent context it settled — none of - * which is a prop the adapter could supply or a document could reach. + * root is imported. It carries the sealed surface, the sealed verbosity and the + * Agent context it settled — none of which is a prop the adapter could supply + * or a document could reach. */ declaration: DeclaredMarkdownComponent; + /** + * The symbols this authorship describes. + * + * The `run` profile's, because a Plan is a program a later `xmd run` executes: + * deriving them from this execution — which searches no repository and + * refuses almost every capability — would describe a vocabulary the approved + * program would not have. Captured with the rest of the installation, before + * any installed code, middleware or document code runs. + */ + symbols: SyntaxSymbolsProvider; } /** What building the constrained provider needs, and nothing more. */ @@ -226,7 +236,7 @@ export interface PlanAuthorshipPolicy { * * Not a description built beside the installation but the installation itself: * the same value registers the provider, installs the invocation options, and - * is handed to a trusted host as its observation. There is nothing for a report + * is handed to a trusted host as its reference. There is nothing for a report * to disagree with, because there is no second report. */ export interface PlanProviderAssembly { @@ -275,7 +285,7 @@ export function planAgentContext( defaultAgent: stack.defaultAgent, *installProvider(invocation: PlanAuthorshipInvocation): Operation { // One assembly, used for every installation and handed back as the - // observation. Nothing is reconstructed afterward, so a report cannot + // reference. Nothing is reconstructed afterward, so a report cannot // describe an arrangement other than the one installed. const installed: PlanProviderAssembly = { provider: "acpx", @@ -319,7 +329,7 @@ export interface AuthorshipFrame { readonly session: string; /** The exact authored label a trusted child host may address privately. */ readonly authoredSession?: string; - observe?(observation: PlanAuthorshipObservation): Operation; + observe?(reference: PlanAuthorshipObservation): Operation; installElicitation(): Operation; } @@ -456,6 +466,13 @@ export function* runPlanCommandDocument(profile: AuthorshipProfile): Operation` is registered at all. yield* Config.around({ verbose: () => verbose }, { at: "min" }); - // The repository-composition vocabulary, as ordinary shadowable defaults. - // Registering it installs no provider, discovers no repository, acquires no - // lock and reaches no network: what a name *does* is decided by whichever - // provider the command installed, and a runtime that installs none still - // resolves every one of these. - yield* registerComponents(COMPOSITION_REGISTRATIONS); + // The repository-composition vocabulary, as ordinary shadowable defaults, + // with the documentation that describes it. Bootstrapping it installs no + // provider, discovers no repository, acquires no lock and reaches no network: + // what a name *does* is decided by whichever provider the command installed, + // and a runtime that installs none still resolves every one of these. + yield* useCompositionComponents(); // Compose testing around the single core execution entrypoint: both // commands register the components (assertions work in regular documents, @@ -928,7 +939,7 @@ export function* installDocumentComponents(mode: DocumentMode, verbose: boolean) yield* installTestAgentComponents(); yield* installAgentComponents(); } else { - yield* registerComponents([VERBOSE_REGISTRATION]); + yield* useVerboseComponent(); yield* installTestingComponents({ verbose }); } @@ -1026,11 +1037,6 @@ function* runDocument( ? {} : { observeAuthorship: request.observeAuthorship }), installElicitation: request.installElicitation, - // Rendered when a `` first asks, not before: an ordinary run that - // writes none never builds a catalog it has no reader for. - *catalog() { - return renderSyntaxMarkdown(yield* syntaxCatalog(include)); - }, }); const plan = yield* planDeclaration({ @@ -1165,6 +1171,13 @@ function* runDocument( // and does not gain `` at its root — but the production run child // it can launch is the run profile, and gets it below. ...(mode.testing ? {} : { declarations: [plan] }), + // The ceiling a generated fragment runs under, stated only where the + // host that attached this execution stated none: a workflow attachment + // states its own Workspace-bound profile, and one execution offers one + // maximum authority. + ...(statesEvaluation(mode.installations) + ? {} + : { evaluation: ordinaryEvaluationProfile() }), }, // The declarations a nested execution may configure a child with, named // by the exact definitions this command installed. Recognizing one is @@ -2511,7 +2524,7 @@ function* dispatch( }, { ...(sessions === undefined ? {} : { sessions }), - catalog: syntaxCatalog, + symbols: syntaxSymbols, // The two facts about this process's own stderr that nothing further // in may go and read: whether it is a terminal, and whether it took // what it was handed. The approved Plan's sinks are stdout and @@ -2632,23 +2645,37 @@ function* dispatch( case "syntax": { // One inspection per invocation, then one complete document. A failure // writes nothing to stdout: a healthy subset printed as though it were - // the catalog would read as complete. + // the whole set of symbols would read as complete. let rendered: string; try { - const catalog = yield* syntaxCatalog(command.config.include); - rendered = command.config.json ? renderSyntaxJson(catalog) : renderSyntaxMarkdown(catalog); + const named = command.config.component; + if (named === undefined) { + // The compact list of symbols, unchanged: routine discovery output and + // every default Plan prompt read it, and long documentation would make + // both unnecessarily large. + const catalog = yield* syntaxSymbols(command.config.include); + rendered = command.config.json + ? renderSyntaxJson(catalog) + : renderSyntaxMarkdown(catalog); + } else { + // The same selection, index and renderer `` uses, so + // the command and the component cannot describe one component two + // ways. JSON stays the compact projection; it is the symbols' shape, + // and documentation is prose rather than a symbol member. + rendered = yield* renderSyntaxDocumentation(command.config.include, [named]); + } } catch (error) { console.error(describeError(error)); yield* exit(1); break; } - // Only the catalog goes through delivery today, because it is the one - // output this command writes in a single call and the only one already + // Only this command's rendering goes through delivery today, because it + // is the one output written in a single call and the only one already // past a pipe buffer. const written = yield* deliverWhole(rendered, process.stdout); if (!written.ok) { console.error( - `xmd syntax: stdout did not accept the whole catalog: ${describeError(written.error)}`, + `xmd syntax: stdout did not accept the whole output: ${describeError(written.error)}`, ); yield* exit(1); } @@ -2831,7 +2858,7 @@ export function* runXmd( const provisional = xmd.parse({ args: helpRequest.args }); const selected = provisional.ok ? provisional.value.config : undefined; // The two commands a `--timeout` bounds. `xmd plan`'s deadline encloses - // something different from a run's — the catalog, the assistant session, + // something different from a run's — the symbols, the assistant session, // every repair, the human review, provider teardown, final validation and the // artifact — and covers no later program, because it starts none. const planning = selected !== undefined && !selected.help && selected.name === "plan"; diff --git a/packages/cli/src/components.md b/packages/cli/src/components.md new file mode 100644 index 000000000..8507c0583 --- /dev/null +++ b/packages/cli/src/components.md @@ -0,0 +1,24 @@ +Long-form documentation for the components the `xmd` command registers. + +One component, and it exists because a document usually has two audiences: the +person running it, who wants the result, and the person debugging it, who wants +to know how it got there. + +## Verbose + +Expands its content only when run verbosity is on. + +```mdx + +Resolved {documents.length} documents from {include}. + +``` + +`--verbose` turns it on for the whole run, and a component may override +verbosity for its own content. When verbosity is off the content is not +expanded at all — so anything expensive inside it costs nothing on an ordinary +run, and this is a place to put detail rather than a place to hide it. + +`as` captures the rendered verbose text, or an **empty string** when verbosity +is off. That is what lets a document build a diagnostic once and use it in more +than one place without branching on the flag itself. diff --git a/packages/cli/src/documents/Plan.md b/packages/cli/src/documents/Plan.md index d30e0a84d..7dd182362 100644 --- a/packages/cli/src/documents/Plan.md +++ b/packages/cli/src/documents/Plan.md @@ -5,8 +5,8 @@ props: session: { type: string, minLength: 1 } additionalProperties: false description: >- - Create an XMD program from a prompt. `Ask for the user's age.` - emits the approved program source. + Generate program text from a prompt. `Ask for the user's age.` + emits the approved program text; `` runs it. as: Binds the exact approved program source and emits nothing. Omitted, that source is emitted where the component is written. context: The complete prompt, expanded once before authorship. --- @@ -71,6 +71,7 @@ word a person reads, for the same reason. Getting the available XMD components and constructs and setting up the planning session. + ## Say what this surface calls things @@ -159,7 +160,7 @@ out. Everything you may use is described below. Use nothing that is not here. -{inputs.syntax} +{syntax} Reply with the Plan source and nothing else. No enclosing code fence, no explanation before or after it. diff --git a/packages/cli/src/evaluation-profile.ts b/packages/cli/src/evaluation-profile.ts new file mode 100644 index 000000000..605c30975 --- /dev/null +++ b/packages/cli/src/evaluation-profile.ts @@ -0,0 +1,93 @@ +/** + * What `xmd run` lets a generated fragment do. + * + * `` is public and canonical core owns what it means, so this file + * does not decide the component — it decides the *ceiling*, which is the only + * part a host owns. It is stated here, at the assembly this command already + * does, before any document exists to ask for it. + * + * ## The two tables + * + * `read` is core's self-closing `` and nothing else. `write` is core's + * paired `…` and self-closing ``. A fragment run by + * `xmd run` therefore reaches the caller's own filesystem through the Files + * provider this command installed, and reaches nothing else at all: no network + * read, no process, no repository, no Git, no credential and no agent. + * + * `` is absent rather than present-and-bounded. An unbounded network + * read is a decision this command does not make on a document's behalf, and + * admitting the identity with an empty ceiling would be a different thing from + * not admitting it — the fragment would name a component it can never + * successfully call. + * + * ## No Workspace, and no alias + * + * An ordinary run evaluates against no Workspace. Its admitted effects address + * the Files provider its own execution installed, and there is no immutable + * root history for a continuation to be held to — which is a different + * statement from evaluating against an empty one, and the profile says so by + * omitting the access rather than answering it with nothing. + * + * It accepts `text` and refuses `source`: `xmd run` never shipped the earlier + * spelling, so there is no document written against it to keep working. + */ + +import { fileDeleteEntry, fileReadEntry, fileWriteEntry } from "@executablemd/core/host"; +import type { + ExecutionInstallation, + FragmentEvaluationInput, + FragmentFileAccess, +} from "@executablemd/core/host"; +import { cwd, hostFilesHandler } from "@executablemd/runtime"; + +/** + * The exact filesystem operations an admitted fragment performs. + * + * A handler of this command's own, constructed here rather than read from + * `API.Files` when a fragment runs. The document's provider and this one behave + * identically — both are `hostFilesHandler()` — and the difference is that + * nothing a document, a repository component or middleware installs is between + * a fragment and this instance. + * + * Only the five operations a fragment can name are copied across. The handler + * also offers globbing and temporary directories; an admitted fragment has + * neither, and not copying them is what makes that true. + */ +function ordinaryFiles(): FragmentFileAccess { + const handler = hostFilesHandler(); + return { + checkFilePath: (input) => handler.checkFilePath(input), + readTextFile: (input) => handler.readTextFile(input), + writeTextFile: (input) => handler.writeTextFile(input), + deleteFile: (input) => handler.deleteFile(input), + ensureDirectory: (input) => handler.ensureDirectory(input), + // The caller's working directory, which is what `xmd run` resolves every + // authored path against. Read when a fragment runs rather than frozen at + // assembly, so a fragment and the document that produced it resolve the + // same relative path to the same file. + workingDirectory: () => cwd(), + }; +} + +/** The ceiling `xmd run` and its run children state. */ +export function ordinaryEvaluationProfile(): FragmentEvaluationInput { + return { + read: [fileReadEntry()], + write: [fileWriteEntry(), fileDeleteEntry()], + files: ordinaryFiles(), + }; +} + +/** + * Whether the host that attached this execution already stated a ceiling. + * + * One execution offers one maximum authority, so a workflow attachment — which + * states its own Workspace-bound profile — is not also given the run profile's. + * Asked about the installations rather than resolved by order, because + * "whichever came last wins" is exactly how authority stops being auditable. + */ +export function statesEvaluation( + installations: readonly ExecutionInstallation[] | undefined, +): boolean { + return (installations ?? []).some((installation) => installation.evaluation !== undefined); +} diff --git a/packages/cli/src/plan-component.ts b/packages/cli/src/plan-component.ts index 31f4b12ff..cd2aa44f5 100644 --- a/packages/cli/src/plan-component.ts +++ b/packages/cli/src/plan-component.ts @@ -36,6 +36,11 @@ * So they resolve only while canonical core is expanding these exact bytes: * not from the caller's root, not from the Prompt the caller projected, not from * a sibling ``, and not from anything middleware can answer. + * + * `` is not among them. What a document may write here is a public + * question with a public answer, and canonical core owns both — so `Plan.md` + * writes the same component any other document writes, and the symbols the Agent + * is shown is the one an operator can print. */ import { createHash } from "node:crypto"; @@ -116,8 +121,8 @@ export type StructuralValidation = (candidate: string) => Operation` declaration has to be able to describe itself: the - * profile a Plan will run in contains ``, and the catalog the agent was - * shown says so. + * profile a Plan will run in contains ``, and the symbols the agent was + * shown say so. */ export function structuralValidation( includes: readonly string[], @@ -206,8 +211,6 @@ export interface PlanComponentAssembly { observeAuthorship?(observation: PlanAuthorshipObservation): Operation; /** Who answers the review question. */ installElicitation(): Operation; - /** The run profile's rendered vocabulary, as the first Agent turn receives it. */ - catalog(): Operation; /** * How this host decides whether one candidate is structurally a program. * @@ -223,22 +226,22 @@ export interface PlanComponentAssembly { const INPUTS_RETURNS = { type: "object", properties: { - syntax: { type: "string" }, session: { type: "string" }, surface: { type: "string" }, durable: { type: "boolean" }, authoredSession: { type: "string" }, }, - required: ["syntax", "session", "surface", "durable"], + required: ["session", "surface", "durable"], additionalProperties: false, }; /** - * What the frozen inputs are given: the caller's optional session name, and the + * What the frozen inputs are given: the caller's optional session name and the * prompt this invocation is about. * - * The prompt is here so that the first durable record of the invocation is - * about a question as well as a catalog. Only its digest is kept. + * Only the prompt's digest is retained here. The vocabulary the Agent is shown + * is public ``'s, with a durable record of its own, so each protocol + * can be read and reconciled independently. */ const INPUTS_PROPS = { type: "object", @@ -324,7 +327,7 @@ export function* planComponentDeclaration( ): Operation { const source = yield* readPackagedDocument(PLAN_DOCUMENT); // The admission validates against the profile a Plan will run in, and that - // profile now contains `` — the catalog the agent was shown says so. So + // profile now contains `` — the symbols the agent was shown say so. So // the declaration has to be able to describe itself, which is why it is // assigned back rather than rebuilt: a second copy of these bytes would be a // second Component identity. @@ -365,7 +368,7 @@ export function* planComponentDeclaration( * Inspection and validation answer about what a document may write. They mint no * execution, so there is no claimant to build a private capability from and no * frame to install — and none of that is describable anyway: a private name - * is not syntax a document may write, so a catalog listing one would describe an + * is not syntax a document may write, so symbols listing one would describe an * environment that does not exist. * * What they do report is the identity: the same name, the same origin, the same @@ -432,14 +435,15 @@ function* uninvocable(): Operation { /** * Freeze this invocation's authorship inputs, and retain them. * - * The catalog is an observation the first Agent turn is built from, so it is - * journaled: a continuation restores what the run actually showed the agent - * rather than rebuilding one from a working tree that has moved. The instruction - * identity beside it is what makes a continuation answerable at all: this is the - * first durable record of the invocation, so comparing it here refuses a Plan - * asked for different instructions before a directory, a provider, a turn, a - * review or an admission exists — the only place that can refuse without having - * already done some of the work it would be refusing. + * The vocabulary the Agent is shown is not here. `` is a public + * component canonical core owns, it retains its own observation, and `Plan.md` + * binds it directly — so the symbols and the question are two records that can + * be read and reconciled independently rather than one that has to be read + * whole. + * + * The instruction identity here makes a continuation answerable at all: + * comparing it refuses a Plan asked under another prompt before a directory, a + * provider, a turn, a review or an admission exists. * * The session placement is derived here too, from the durable identity canonical * execution minted for this exact expansion — which is what makes two `` @@ -469,7 +473,7 @@ function planInputs(assembly: PlanComponentAssembly): IdentityComponent { const instruction = sourceDigest(String(props.instruction)); const frozen = yield* durablePlanOperation(`plan:inputs:${id}`, function* () { - return { syntax: yield* assembly.catalog(), instruction }; + return { instruction }; }); // A history is input, so it is parsed rather than trusted. @@ -482,7 +486,6 @@ function planInputs(assembly: PlanComponentAssembly): IdentityComponent { } return { - syntax: retained.syntax, session, surface: assembly.surface, durable: durability(assembly, authored), @@ -692,14 +695,13 @@ function checkDraft(validate: StructuralValidation): IdentityComponent { /** What the frozen inputs retained, or nothing when the record is not one. */ interface RetainedInputs { - readonly syntax: string; readonly instruction: string; } /** * The frozen inputs a record holds, read as a closed protocol. * - * Exactly two members, both strings. A record missing one, carrying a member + * Exactly one string member. A record missing it, carrying a member * this version does not know, or holding one of the wrong type is a record this * version cannot read — not one to fill in a default for, because every default * here is a guess about what an earlier run actually asked. @@ -708,14 +710,14 @@ function readInputs(value: Json): RetainedInputs | undefined { if (typeof value !== "object" || value === null || Array.isArray(value)) { return undefined; } - const { syntax, instruction } = value; - if (Object.keys(value).length !== 2) { + const { instruction } = value; + if (Object.keys(value).length !== 1) { return undefined; } - if (typeof syntax !== "string" || typeof instruction !== "string") { + if (typeof instruction !== "string") { return undefined; } - return { syntax, instruction }; + return { instruction }; } /** diff --git a/packages/cli/src/plan.ts b/packages/cli/src/plan.ts index 8b0259f24..c91cd43a3 100644 --- a/packages/cli/src/plan.ts +++ b/packages/cli/src/plan.ts @@ -40,7 +40,7 @@ import { randomUUID } from "node:crypto"; import { resolve } from "node:path"; import process from "node:process"; -import type { SyntaxCatalog } from "@executablemd/core"; +import type { SyntaxSymbols } from "@executablemd/core"; import type { AcpxProviderDependencies } from "@executablemd/acp"; import { InMemoryStream } from "@executablemd/durable-streams"; import type { DurableStream } from "@executablemd/durable-streams"; @@ -98,7 +98,7 @@ export interface PlanDependencies { /** What the profile's provider is built on, beyond the host's own assembly. */ acp?: AcpxProviderDependencies; /** The run profile's complete structured vocabulary. */ - catalog(includes: readonly string[]): Operation; + symbols(includes: readonly string[]): Operation; /** Who answers the review question. */ installElicitation(): Operation; /** @@ -203,13 +203,6 @@ export function* runPlan(command: PlanCommand, deps: PlanDependencies): Operatio verbose: command.verbose, host, installElicitation: deps.installElicitation, - // Built when `` asks, which is what lets an authored phase - // announce the preparation before it happens. It is still sealed: the - // catalog the agent is shown is the one this command renders, and no prop - // on the thin adapter could supply another. - *catalog() { - return renderSyntaxMarkdown(yield* deps.catalog(command.include)); - }, validate, }); @@ -222,6 +215,13 @@ export function* runPlan(command: PlanCommand, deps: PlanDependencies): Operatio declaration, stream, progress: deps.progress, + // The vocabulary this authorship describes is the `run` profile's, not + // this execution's: a Plan is a program a later `xmd run` executes, so the + // symbols the Agent must be shown are the ones that run will have. Stated + // at the execution boundary and captured before any installed code — no + // prop on the thin adapter, and nothing the Component projects, could + // supply another. + symbols: () => deps.symbols(command.include), }); } catch (error) { console.error(describeError(error)); diff --git a/packages/cli/src/syntax.ts b/packages/cli/src/syntax.ts index 39915006f..3c5eb397d 100644 --- a/packages/cli/src/syntax.ts +++ b/packages/cli/src/syntax.ts @@ -2,327 +2,142 @@ * `xmd syntax` — everything a document may write here, described without * running any of it. * - * Two jobs, kept apart. The first is assembling the `run` host profile as - * *declarations*: the same arrays the runtime installers register, with none of + * Two jobs, kept apart. The first is entering the `run` profile's *declarative* + * bootstraps: the same calls the runtime installers delegate to, with none of * the middleware, providers, launchers or activation those installers also - * arrange. The second is rendering, and both renderers take the catalog as a + * arrange. The second is rendering, and both renderers take the symbols as a * value — neither performs discovery, and neither parses the other's output. * - * JSON is the canonical, lossless projection. Markdown is written for a person - * and says so when a schema is richer than a table can summarize honestly. + * Entering the bootstraps rather than splicing their registration arrays is + * what makes the documentation this command reads the profile's own: a package + * installs its registrations and its documentation in one call, so a command + * that has the components has the words that describe them. Splicing the arrays + * left the two halves to be kept in step by hand, and they were not. + * + * JSON is the canonical, lossless projection and belongs to this command. + * Markdown belongs to core, because a document that writes `` is shown + * the same symbols in the same words: two renderings that agreed only by hand + * would be one release away from telling an operator and an agent different + * things about one profile. */ import { planComponentDescription } from "./plan-component.ts"; import { scoped } from "effection"; import type { Operation } from "effection"; import { - AGENT_REGISTRATIONS, agentIdentityComponents, + capturedDocumentation, + documentationIndexFor, inspectSyntax, - registerComponents, -} from "@executablemd/core"; -import type { - CompleteComponentSyntaxEntry, - ComponentOrigin, - Json, - OriginOnlyComponentSyntaxEntry, - PropsSchema, - StructuralSyntaxEntry, - SyntaxCatalog, + renderSelectedDocumentation, + renderSyntaxMarkdown, + selectDocumented, + useAgentComponents, } from "@executablemd/core"; -import { TESTING_REGISTRATIONS } from "@executablemd/testing"; -import { WEB_REGISTRATIONS } from "@executablemd/web"; -import { VERBOSE_REGISTRATION } from "./verbose-component.ts"; -import { COMPOSITION_REGISTRATIONS } from "@executablemd/workflow"; +import type { SyntaxSymbols } from "@executablemd/core"; +import { useTestingComponents } from "@executablemd/testing"; +import { useWebComponents } from "@executablemd/web"; +import { useVerboseComponent } from "./verbose-component.ts"; +import { useCompositionComponents } from "@executablemd/workflow"; + +export { renderSyntaxMarkdown }; /** - * The catalog for the production `run` profile, in the contextual working + * The symbols for the production `run` profile, in the contextual working * directory. * - * The registrations are the ones `installTestingComponents()`, + * The declarations are the ones `installTestingComponents()`, * `installWebComponents()`, `installAgentComponents()` and the - * repository-composition installer register, read as values so this cannot - * drift from what a run installs. What those installers - * *also* do — testing activation and its execution middleware, the elicitation - * provider, the agent provider, the permission mode, the foreground launcher — - * is operational and belongs to a run, so none of it happens here. + * repository-composition installer each delegate to, entered here directly so + * this cannot drift from what a run installs. What those installers *also* do — + * testing activation and its execution middleware, the elicitation provider, + * the agent provider, the permission mode, the foreground launcher — is + * operational and belongs to a run, so none of it happens here. * * `` travels as a declaration for the same reason: its factory takes * an execution's claimant, and describing an environment mints no execution. * * The scope is bounded, and everything installed in it is declarative registry - * state. Leaving it removes the layer, and there is no process, agent, service, - * journal, file or authority left to clean up. + * state and documentation middleware. Leaving it removes the layer, and there + * is no process, agent, service, journal, file or authority left to clean up. */ -export function* syntaxCatalog(includes: readonly string[]): Operation { +export function* syntaxSymbols(includes: readonly string[]): Operation { return yield* scoped(function* () { yield* useRunProfileRegistry(); - return yield* inspectSyntax({ - includes, - components: agentIdentityComponents(), - // `` is part of the run profile, so a catalog that left it out would - // describe a vocabulary no run has. Described from the packaged bytes: - // inspection mints nothing, so it reports the Component's identity and - // contract without building the capabilities only a run can build. - declarations: [yield* planComponentDescription()], - }); + return yield* profileSymbols(includes); + }); +} + +/** The profile's symbols, inside a scope that has already bootstrapped it. */ +function* profileSymbols(includes: readonly string[]): Operation { + return yield* inspectSyntax({ + includes, + components: agentIdentityComponents(), + // `` is part of the run profile, so symbols that left it out would + // describe a vocabulary no run has. Described from the packaged bytes: + // inspection mints nothing, so it reports the Component's identity and + // contract without building the capabilities only a run can build. + declarations: [yield* planComponentDescription()], }); } /** - * The registrations the `run` profile installs, as registry state and nothing - * else. + * The declarations the `run` profile installs, and nothing else. * * Shared with `xmd plan`, which both describes this vocabulary to a generator - * and validates what comes back. Registering only here would make the catalog + * and validates what comes back. Bootstrapping only here would make the symbols * advertise `` while validation reported it unresolved — a document told * to use a component nobody would accept. + * + * Each call is one package's declarative bootstrap: its registrations and the + * documentation that describes them. None of them installs a provider, + * discovers an ambient repository, acquires a lock, spawns Git or reads a + * credential. */ export function* useRunProfileRegistry(): Operation { - yield* registerComponents([ - VERBOSE_REGISTRATION, - ...AGENT_REGISTRATIONS, - ...TESTING_REGISTRATIONS, - ...WEB_REGISTRATIONS, - // The repository-composition vocabulary. Registering it is all that happens - // here: catalog construction installs no provider, discovers no ambient - // repository, acquires no lock, spawns no Git and reads no credential. - ...COMPOSITION_REGISTRATIONS, - ]); + yield* useVerboseComponent(); + yield* useAgentComponents(); + yield* useTestingComponents(); + yield* useWebComponents(); + // The repository-composition vocabulary. + yield* useCompositionComponents(); } /** - * The catalog as JSON: two-space indent, one trailing newline. + * The symbols as JSON: two-space indent, one trailing newline. * - * Catalog construction owns member insertion order, category order and entry - * order, so the bytes are the same for the same environment. + * Construction owns member insertion order, category order and entry order, so + * the bytes are the same for the same environment. */ -export function renderSyntaxJson(catalog: SyntaxCatalog): string { - return `${JSON.stringify(catalog, null, 2)}\n`; -} - -/** The three category kinds, taken from the catalog rather than restated. */ -type CategoryKind = SyntaxCatalog["categories"][number]["kind"]; - -const HEADINGS: Record = { - structural: "## Built-in structural syntax", - "built-in": "## Built-in components", - "user-provided": "## User-provided components", -}; - -const EMPTY: Record = { - structural: "No structural constructs are reserved.", - "built-in": "No components are registered in this profile.", - "user-provided": "No components were found in the configured includes.", -}; - -export function renderSyntaxMarkdown(catalog: SyntaxCatalog): string { - const sections = catalog.categories.map((category) => { - const blocks: string[] = [HEADINGS[category.kind]]; - if (category.entries.length === 0) { - blocks.push(EMPTY[category.kind]); - return blocks.join("\n\n"); - } - for (const entry of category.entries) { - blocks.push(...renderEntry(entry)); - } - return blocks.join("\n\n"); - }); - return `${sections.join("\n\n")}\n`; -} - -function renderEntry( - entry: StructuralSyntaxEntry | CompleteComponentSyntaxEntry | OriginOnlyComponentSyntaxEntry, -): string[] { - if (entry.kind === "structural") { - return renderStructural(entry); - } - if (entry.inspectability === "origin-only") { - return renderOriginOnly(entry); - } - return renderComponent(entry); -} - -function heading(name: string): string { - return `### \`<${name}>\``; -} - -function renderStructural(entry: StructuralSyntaxEntry): string[] { - const blocks = [heading(entry.name), entry.description]; - blocks.push("**Syntax:**", fence("md", entry.syntax.join("\n"))); - blocks.push(...prose(entry)); - return blocks; -} - -function renderOriginOnly(entry: OriginOnlyComponentSyntaxEntry): string[] { - return [ - heading(entry.name), - "This component is a repository TypeScript module. Its contract lives on the module's " + - "exports, and reading it would import the module and run its top-level code — which " + - "describing an environment must not do. The module was not imported, so its props, " + - "captures, forms and return are unavailable here.", - `**Origin:** ${describeOrigin(entry.origin)}`, - ]; -} - -function renderComponent(entry: CompleteComponentSyntaxEntry): string[] { - const blocks = [heading(entry.name)]; - if (entry.description !== undefined) { - blocks.push(entry.description); - } - blocks.push(`**Forms:** ${entry.forms.map((form) => invocation(entry.name, form)).join(", ")}`); - blocks.push(...renderProps(entry.props)); - if (entry.captures.length > 0) { - blocks.push( - `**Captures:** ${entry.captures.map(code).join(", ")} — evaluated by the component ` + - "itself, so these props are deliberately absent from the schema above.", - ); - } - blocks.push(...prose(entry)); - blocks.push(...renderReturns(entry)); - blocks.push(`**Origin:** ${describeOrigin(entry.origin)}`); - return blocks; -} - -function prose(entry: { as?: string; context?: string }): string[] { - const blocks: string[] = []; - if (entry.as !== undefined) { - blocks.push(`**\`as\`:** ${entry.as}`); - } - if (entry.context !== undefined) { - blocks.push(`**Body context:** ${entry.context}`); - } - return blocks; -} - -function invocation(name: string, form: "self-closing" | "paired"): string { - return code(form === "self-closing" ? `<${name} />` : `<${name}>…`); -} - -function renderReturns(entry: CompleteComponentSyntaxEntry): string[] { - if (entry.returnMode === "text") { - return [ - "**Returns:** text — the markdown this component renders.", - fence("json", stringify(entry.returns)), - ]; - } - return [ - "**Returns:** a value — it renders nothing, and `as` binds what it returns.", - fence("json", stringify(entry.returns)), - ]; +export function renderSyntaxJson(symbols: SyntaxSymbols): string { + return `${JSON.stringify(symbols, null, 2)}\n`; } /** - * The props table, and the schema it summarizes. + * The selected components' metadata and long-form documentation. * - * The table is the readable half and the schema is the authoritative one. A - * table cannot carry `default`, `enum`, a combinator, a reference or a root - * constraint, so the schema is printed beside it rather than reduced into it, - * and a property the table cannot name a type for is labelled honestly instead - * of being given an invented one. - */ -function renderProps(props: PropsSchema): string[] { - const rows = propertyRows(props); - const blocks = ["#### Props"]; - if (rows.length === 0) { - blocks.push("This component declares no individual props."); - } else { - blocks.push( - ["| Prop | Type | Required | Description |", "| --- | --- | --- | --- |", ...rows].join("\n"), - ); - } - blocks.push(fence("json", stringify(props))); - return blocks; -} - -function propertyRows(props: PropsSchema): string[] { - const properties = props.properties; - if (typeof properties !== "object" || properties === null || Array.isArray(properties)) { - return []; - } - const required = new Set( - Array.isArray(props.required) - ? props.required.filter((name): name is string => typeof name === "string") - : [], - ); - const rows: string[] = []; - for (const [name, schema] of Object.entries(properties)) { - // Every cell is escaped on the way in, the prop name included: a schema - // property may be spelled with anything, and one pipe in a name would - // shift every column after it. - rows.push( - row([ - code(name), - summarizeType(schema), - required.has(name) ? "yes" : "no", - describeProp(schema), - ]), - ); - } - return rows; -} - -function row(cells: readonly string[]): string { - return `| ${cells.map(cell).join(" | ")} |`; -} - -/** - * The type column, or an honest refusal to reduce one. + * `xmd syntax Elicit` and `` are the same lookup: + * one selection, one index, one renderer. An operator reading a terminal and an + * agent reading a document are answering the same question, and two renderings + * that agreed only by hand would be one release away from disagreeing. * - * A plain `type` — one name or a union of them — summarizes faithfully. - * Anything else is a schema whose constraints do not fit a word, so the column - * says JSON Schema and the reader goes to the block below it. + * The symbols and the index come from *one* entry into the profile's + * bootstraps, inside this scope. Building them from two entries would let the + * command describe a component from one assembly and document it from another; + * building the index outside the scope would find no contribution at all, since + * a bootstrap's documentation belongs to the scope that entered it. + * + * Nothing here narrows execution, so every entry the symbols hold is available + * and each says so. */ -function summarizeType(schema: Json): string { - if (typeof schema !== "object" || schema === null || Array.isArray(schema)) { - return "JSON Schema"; - } - const type = schema.type; - if (typeof type === "string") { - return code(type); - } - if (Array.isArray(type) && type.every((member) => typeof member === "string")) { - // Unescaped: `row()` escapes every cell once, and escaping here as well - // would put a backslash in front of the backslash. - return type.map(code).join(" | "); - } - return "JSON Schema"; -} - -function describeProp(schema: Json): string { - if (typeof schema !== "object" || schema === null || Array.isArray(schema)) { - return ""; - } - const description = schema.description; - return typeof description === "string" ? description : ""; -} - -function describeOrigin(origin: ComponentOrigin): string { - if (origin.kind === "repository") { - return code(origin.path); - } - if (origin.kind === "registered") { - return `${code(origin.origin)} (${origin.reserved ? "reserved registration" : "registered default"})`; - } - if (origin.kind === "declared-markdown") { - return `${code(origin.origin)} (declared Markdown)`; - } - return `structural syntax (${code(origin.construct)})`; -} - -function code(text: string): string { - return `\`${text}\``; -} - -/** A table cell: pipes escaped, and line breaks folded so the row stays a row. */ -function cell(text: string): string { - return text.replace(/\|/g, "\\|").replace(/\r?\n/g, " ").trim(); -} - -function fence(language: string, body: string): string { - return ["```" + language, body, "```"].join("\n"); -} - -function stringify(value: Json): string { - return JSON.stringify(value, null, 2); +export function* renderSyntaxDocumentation( + includes: readonly string[], + names: readonly string[], +): Operation { + return yield* scoped(function* () { + yield* useRunProfileRegistry(); + const catalog = yield* profileSymbols(includes); + const index = documentationIndexFor(yield* capturedDocumentation()); + return renderSelectedDocumentation(selectDocumented(catalog, catalog, names, index)); + }); } diff --git a/packages/cli/src/testing-host.ts b/packages/cli/src/testing-host.ts index 526daacf5..4fe3a3144 100644 --- a/packages/cli/src/testing-host.ts +++ b/packages/cli/src/testing-host.ts @@ -58,6 +58,7 @@ import type { TestAgentChildConfiguration, } from "@executablemd/testing"; import { installDocumentComponents } from "./cli.ts"; +import { ordinaryEvaluationProfile } from "./evaluation-profile.ts"; import type { HostServiceInstaller } from "./cli.ts"; import type { RepositoryInstaller } from "./run-repositories.ts"; @@ -314,6 +315,11 @@ function* runProfileChild( // this child settled above, rather than taken from a declaration the // entrypoint built before this child's configuration had been read. installations.push({ + // The run profile's own evaluation ceiling. A `host="run"` child is an + // ordinary run whatever command is hosting it, so a child of `xmd test` — + // whose own document is a different profile — still evaluates a generated + // fragment under exactly what `xmd run` states. + evaluation: ordinaryEvaluationProfile(), declarations: [ yield* settings.planDeclaration({ context, diff --git a/packages/cli/src/verbose-component.ts b/packages/cli/src/verbose-component.ts index c9d056597..4c054c22e 100644 --- a/packages/cli/src/verbose-component.ts +++ b/packages/cli/src/verbose-component.ts @@ -14,8 +14,19 @@ * observed. */ -import { content, verbose } from "@executablemd/core"; -import type { ComponentRegistration, Json } from "@executablemd/core"; +import { + content, + contributeDocumentation, + packageDocumentation, + registerComponents, + verbose, +} from "@executablemd/core"; +import type { + ComponentRegistration, + DocumentationContribution, + DocumentationReader, + Json, +} from "@executablemd/core"; import type { Operation } from "effection"; export const VERBOSE_ORIGIN = "@executablemd/cli"; @@ -33,6 +44,30 @@ function* Verbose(_props: Record): Operation { return yield* content(); } +/** This command's long-form documentation, derived from what it registers. */ +export function* cliDocumentation( + read?: DocumentationReader, +): Operation { + return yield* packageDocumentation( + new URL("./components.md", import.meta.url), + { owner: VERBOSE_ORIGIN, asset: "packages/cli/src/components.md" }, + [VERBOSE_REGISTRATION.name], + read, + ); +} + +/** + * This command's own vocabulary, as declarations and nothing else. + * + * One registration and the documentation that describes it, installed together + * so a scope that has one has the other — the same call `xmd syntax` enters to + * describe the run profile. + */ +export function* useVerboseComponent(): Operation { + yield* registerComponents([VERBOSE_REGISTRATION]); + yield* contributeDocumentation(cliDocumentation); +} + /** The one declaration the run profile registers and `xmd syntax` describes. */ export const VERBOSE_REGISTRATION: ComponentRegistration = { name: "Verbose", diff --git a/packages/cli/src/workflow.ts b/packages/cli/src/workflow.ts index d68fb4752..a585232b1 100644 --- a/packages/cli/src/workflow.ts +++ b/packages/cli/src/workflow.ts @@ -1006,11 +1006,11 @@ export function runWorkflow( // reaches `node:sqlite` — which Node greets with an experimental warning on // standard error the moment it loads. A run that opens no workflow storage // should not be announcing that it might have. - // `evaluationComponents` comes through the same import, and for the same - // reason: `` is the workflow host's component, and a run that - // opens no workflow storage must not load the adapter that reaches - // `node:sqlite` — which Bun does not have at all. - const { createSuspensionController, evaluationComponents } = yield* until( + // `evaluationProfile` comes through the same import, and for the same + // reason: it is closed over this run's storage, and a run that opens no + // workflow storage must not load the adapter that reaches `node:sqlite` — + // which Bun does not have at all. + const { createSuspensionController, evaluationProfile } = yield* until( import("@executablemd/workflow/deno"), ); const suspension = createSuspensionController({ database }); @@ -1063,12 +1063,12 @@ export function runWorkflow( ...(source.value.components.length === 0 ? [] : [workflowBundleInstallation(source.value.components)]), - // `` names durable work after its own invocation, so this run - // declares it to the execution and canonical execution builds it from - // the claimant it minted for this attachment. Declared where the - // Workspace is attached — a completed replay restores its retained - // output and expands nothing, so it needs no component of its own. - ...(completed || replay ? [] : [{ components: evaluationComponents(database) }]), + // The ceiling a generated fragment runs under. `` itself is + // canonical core's, so what this run states is the profile and nothing + // about the component. Stated where the Workspace is attached — a + // completed replay restores its retained output and expands nothing, so + // it needs no ceiling of its own. + ...(completed || replay ? [] : [{ evaluation: yield* evaluationProfile(database) }]), ], around(operation: Operation): Operation { // A completed run replays its retained output and result. Attaching a diff --git a/packages/cli/tests/document-suites/syntax/Syntax.test.md b/packages/cli/tests/document-suites/syntax/Syntax.test.md index 6d9eccf60..9d7e65428 100644 --- a/packages/cli/tests/document-suites/syntax/Syntax.test.md +++ b/packages/cli/tests/document-suites/syntax/Syntax.test.md @@ -16,7 +16,7 @@ not detect one. ## Running the command Three invocations, once each: the default Markdown format, the same catalog as -version-1 JSON, and one more with two includes written in a deliberate order. +version-2 JSON, and one more with two includes written in a deliberate order. ```bash exec as="markdown" "$XMD_SYNTAX_BIN" syntax --include packages/cli/tests/document-suites/syntax/components diff --git a/packages/cli/tests/evaluate-component.test.ts b/packages/cli/tests/evaluate-component.test.ts new file mode 100644 index 000000000..6d092714d --- /dev/null +++ b/packages/cli/tests/evaluate-component.test.ts @@ -0,0 +1,156 @@ +/** + * Tier FE — `` under the ordinary `xmd run` profile. + * + * The core tier proves the component against a recorder. This one proves the + * *host*: that `xmd run` states a profile at all, that the profile it states is + * the Files-only one, and that a fragment run by it reaches the caller's own + * filesystem through operations this command captured rather than through + * whatever the document installed. + * + * It also fixes the spellings. `text` is the canonical one; `source` is the + * workflow's alone and is refused here; `program` was never released and is + * refused everywhere. + */ + +import { describe, it } from "@executablemd/test-support/bdd"; +import { expect } from "@executablemd/test-support/expect"; +import { runCli } from "@executablemd/test-support/launch"; +import { ensureDir, readTextFile, rm, writeTextFile } from "@effectionx/fs"; +import { ensure, scoped } from "effection"; +import type { Operation } from "effection"; +import { randomUUID } from "node:crypto"; +import { tmpdir } from "node:os"; +import { dirname, join } from "node:path"; +import { CORE_COMPONENT_NAMES } from "@executablemd/core"; +import { syntaxSymbols } from "../src/syntax.ts"; + +const NOTE = "the retained note\n"; + +function* useWorkspace( + files: Record, + body: (dir: string) => Operation, +): Operation { + const dir = join(tmpdir(), `xmd-fe-${randomUUID()}`); + return yield* scoped(function* () { + yield* ensure(() => rm(dir, { recursive: true, force: true })); + for (const [name, content] of Object.entries(files)) { + const path = join(dir, name); + yield* ensureDir(dirname(path)); + yield* writeTextFile(path, content); + } + return yield* body(dir); + }); +} + +describe("Tier FE — the ordinary run profile", () => { + it("FE21: `xmd run` states a profile, and a fragment reads the caller's files", function* () { + yield* useWorkspace( + { + "notes.md": NOTE, + "doc.md": + `\\n'} as="answer" />\n\n` + + `\n`, + }, + function* (dir) { + const result = yield* runCli(["run", join(dir, "doc.md")], { cwd: dir }).join(); + expect(result.code).toBe(0); + expect(result.stdout).toContain("the retained note"); + }, + ); + }); + + it("FE5: the ordinary profile admits writes and deletions, and nothing else", function* () { + yield* useWorkspace( + { + "doc.md": + `written by the fragment\\n'} ` + + `allow={["write"]} />\n\ndone\n`, + }, + function* (dir) { + const result = yield* runCli(["run", join(dir, "doc.md")], { cwd: dir }).join(); + expect(result.code).toBe(0); + expect(yield* readTextFile(join(dir, "made.md"))).toBe("written by the fragment"); + }, + ); + }); + + it("FE21: a fragment cannot reach an operation the ordinary profile withheld", function* () { + yield* useWorkspace( + { + "notes.md": NOTE, + "doc.md": `\\n'} />\n`, + }, + function* (dir) { + const result = yield* runCli(["run", join(dir, "doc.md")], { cwd: dir }).join(); + // `` is an ordinary component of this run and is not a name the + // fragment has: the ceiling is the profile's, not the document's. + expect(result.code).not.toBe(0); + expect(`${result.stdout}${result.stderr}`).toContain("did not admit"); + }, + ); + }); +}); + +describe("Tier FE — the spellings this host accepts", () => { + it("FE19: the ordinary profile refuses the workflow's `source` alias", function* () { + yield* useWorkspace( + { + "notes.md": NOTE, + "doc.md": `\\n'} />\n`, + }, + function* (dir) { + const result = yield* runCli(["run", join(dir, "doc.md")], { cwd: dir }).join(); + expect(result.code).not.toBe(0); + expect(`${result.stdout}${result.stderr}`).toContain("this host did not admit it"); + }, + ); + }); + + it("FE19: every profile refuses `program`, which was never released", function* () { + yield* useWorkspace( + { + "notes.md": NOTE, + "doc.md": `\\n'} />\n`, + }, + function* (dir) { + const result = yield* runCli(["run", join(dir, "doc.md")], { cwd: dir }).join(); + // The schema is closed, so an unreleased spelling is refused by prop + // validation rather than reaching the body. + expect(result.code).not.toBe(0); + expect(`${result.stdout}${result.stderr}`).toMatch(/program|additional/i); + }, + ); + }); +}); + +describe("Tier FE — what the source symbols report", () => { + it("FE28: source symbols carry `Evaluate` with a protected origin", function* () { + const symbols = yield* syntaxSymbols([]); + const evaluate = symbols.categories[1].entries.find((entry) => entry.name === "Evaluate"); + + expect(evaluate).not.toBe(undefined); + expect(evaluate?.origin).toEqual({ kind: "protected", origin: "@executablemd/core" }); + expect(evaluate?.sourceKind).toBe("protected"); + // The approved one-line description, exactly. + expect(evaluate?.description).toBe( + 'Evaluate program text. `` runs it.', + ); + // Both spellings, because the two input forms are two ways of stating one + // argument. + expect(evaluate?.forms).toEqual(["self-closing", "paired"]); + }); + + it("FE28: no host bootstrap is needed to make the name available", function* () { + // `syntaxSymbols([])` installs no host, no bundle and no declaration. A + // name that needed one to appear would be absent here and present under + // `xmd run`, which is exactly what a protected name is not. + const symbols = yield* syntaxSymbols([]); + const names = symbols.categories.flatMap((category) => + category.entries.map((entry) => entry.name), + ); + expect(names).toContain("Evaluate"); + // And it is core's own protected name rather than an ordinary registration + // this build happens to carry. + expect([...CORE_COMPONENT_NAMES]).not.toContain("Evaluate"); + }); +}); diff --git a/packages/cli/tests/evaluate-workflow.test.ts b/packages/cli/tests/evaluate-workflow.test.ts new file mode 100644 index 000000000..3c0d5fa37 --- /dev/null +++ b/packages/cli/tests/evaluate-workflow.test.ts @@ -0,0 +1,159 @@ +/** + * Tier FE — `` under the workflow host's own profile. + * + * Deno-only, and for the ordinary reason: this drives a real workflow run + * database, which opens `node:sqlite` — unflagged under Deno, behind + * `--experimental-sqlite` on Node 22, and absent from Bun. The exclusion is + * recorded in `scripts/runtime-test-exclusions.ts` against issue #713. + * + * What only this tier can show is that the two hosts differ in their *profile* + * and in nothing else. The component, the evaluator and the durable protocol + * are the same ones the ordinary run uses; what the workflow adds is a + * Workspace-bound ceiling, a basis that advances with the run, and the released + * `source` spelling — which the ordinary profile refuses and this one accepts + * without comment. + * + * It runs against the real attachment rather than the `xmd workflow` command + * line: a run started from the command boundary needs a git repository with a + * committed definition, and a row that silently degraded when that was + * unavailable would assert nothing. The command boundary itself is Tier WAL's. + */ + +import { describe, it } from "@executablemd/test-support/bdd"; +import { expect } from "@executablemd/test-support/expect"; +import { scoped } from "effection"; +import type { Operation } from "effection"; +import { collect, inlineSource } from "@executablemd/core"; +import { executeInstalled } from "@executablemd/core/host"; +import type { Json } from "@executablemd/durable-streams"; +import { API, useHostFiles } from "@executablemd/runtime"; +import type { WorkflowRunDatabase } from "@executablemd/workflow"; +import { evaluationProfile, withWorkflowWorkspace } from "@executablemd/workflow/deno"; +import { createRun, useStorageRoot, withStorage } from "./support/workflow-run.ts"; + +/** + * Run one document with this run's Workspace attached. + * + * The contextual working directory is pinned somewhere the run may not reach + * and a host Files provider is installed outside the attachment, so a read that + * fell through to the caller's filesystem would fail rather than quietly + * succeed. + */ +function runDocument( + database: WorkflowRunDatabase, + source: string, +): Operation<{ output?: Json; failure?: string }> { + return scoped(function* () { + yield* API.Env.around( + { + // deno-lint-ignore require-yield + *cwd(): Operation { + return "/nowhere-the-workflow-may-reach"; + }, + }, + { at: "min" }, + ); + yield* useHostFiles(); + try { + const output = yield* withWorkflowWorkspace( + database, + scoped(function* () { + return yield* collect( + yield* executeInstalled({ ...inlineSource(source), stream: database.journal }, [ + { evaluation: yield* evaluationProfile(database) }, + ]), + ); + }), + ); + return { output }; + } catch (error) { + return { failure: error instanceof Error ? error.message : String(error) }; + } + }); +} + +const NOTE = "the retained note"; + +describe("Tier FE — the workflow host's profile", () => { + it("FE19: `source` and `text` behave identically here, and neither warns", function* () { + const root = yield* useStorageRoot(); + yield* withStorage(root, function* () { + const aliased = yield* createRun(); + const alias = yield* runDocument( + aliased, + `${NOTE}\n\n` + + `\\n'} as="answer" />\n\n` + + `\n`, + ); + + expect(alias.failure).toBe(undefined); + const aliasText = String(alias.output); + expect(aliasText).toContain(NOTE); + // Silently: no deprecation sentence anywhere in the run's output. + expect(aliasText).not.toContain("earlier spelling"); + expect(aliasText).not.toContain("deprecated"); + + const canonical = yield* createRun({ runId: "canonical" }); + const direct = yield* runDocument( + canonical, + `${NOTE}\n\n` + + `\\n'} as="answer" />\n\n` + + `\n`, + ); + + expect(direct.failure).toBe(undefined); + // Identical behavior: the two spellings state the same argument. + expect(String(direct.output)).toBe(aliasText); + }); + }); + + it("FE19: `program` is a spelling no profile has", function* () { + const root = yield* useStorageRoot(); + yield* withStorage(root, function* () { + const database = yield* createRun(); + const attempt = yield* runDocument( + database, + `\\n'} />\n`, + ); + + expect(attempt.failure).not.toBe(undefined); + expect(String(attempt.failure)).toMatch(/program|additional/i); + }); + }); + + it("FE21: the workflow fragment reaches the run's Workspace, not the caller's disk", function* () { + const root = yield* useStorageRoot(); + yield* withStorage(root, function* () { + const database = yield* createRun(); + // The contextual working directory is `/nowhere-the-workflow-may-reach` + // and a host Files provider is installed outside the attachment. A read + // that fell through to either would fail rather than find this file. + const attempt = yield* runDocument( + database, + `${NOTE}\n\n` + + `\\n'} as="answer" />\n\n` + + `\n`, + ); + + expect(attempt.failure).toBe(undefined); + expect(String(attempt.output)).toContain(NOTE); + }); + }); + + it("FE20: a request outside the stated ceiling is refused, and none is admitted by default", function* () { + const root = yield* useStorageRoot(); + yield* withStorage(root, function* () { + const database = yield* createRun(); + // The production profile states no requests, so `` is not on the + // allowlist at all — a different thing from admitting it and refusing + // every request. + const attempt = yield* runDocument( + database, + `\\n'} />\n`, + ); + + expect(attempt.failure).not.toBe(undefined); + expect(String(attempt.failure)).toContain("did not admit"); + }); + }); +}); diff --git a/packages/cli/tests/fetch-cli.test.ts b/packages/cli/tests/fetch-cli.test.ts index afdf68ded..bc52eb8ec 100644 --- a/packages/cli/tests/fetch-cli.test.ts +++ b/packages/cli/tests/fetch-cli.test.ts @@ -1,5 +1,5 @@ /** - * Tier FE — what a host retains of a `` (spec §6.18). + * Tier FT — what a host retains of a `` (spec §6.18). * * Retention is the host's choice and nothing else's. The same document, the * same request, and the same rendered text produce a trace or no trace @@ -48,7 +48,23 @@ function useLoopback(): Operation { }); const listening = withResolvers(); - server.on("error", (error: Error) => listening.reject(error)); + // Removed with the resource rather than after the first error: a listening + // server outlives its bind, and a handler left behind would still be + // holding a rejected resolver when the next test binds its own. + // + // Established before the handler exists, because `yield* ensure(...)` is + // itself a suspension: an owner halted while it registers unwinds with no + // cleanup at all, so nothing may be attached until it has completed. + let onError: ((error: Error) => void) | undefined; + + yield* ensure(() => { + if (onError) { + server.off("error", onError); + } + }); + + onError = (error: Error) => listening.reject(error); + server.on("error", onError); server.listen(0, "127.0.0.1", () => listening.resolve()); yield* listening.operation; @@ -96,8 +112,8 @@ function document(origin: string): string { ].join("\n"); } -describe("Tier FE — diagnostic retention of a Fetch", () => { - it("FEC1: a run without --journal performs the request and keeps nothing", function* () { +describe("Tier FT — diagnostic retention of a Fetch", () => { + it("FTC1: a run without --journal performs the request and keeps nothing", function* () { const server = yield* useLoopback(); const dir = yield* useTempDirectory("xmd-fetch-cli-"); const path = join(dir, "doc.md"); @@ -114,7 +130,7 @@ describe("Tier FE — diagnostic retention of a Fetch", () => { expect(result.stdout).not.toContain("greeting"); }); - it("FEC2: a run with --journal retains the normalized request and the response", function* () { + it("FTC2: a run with --journal retains the normalized request and the response", function* () { const server = yield* useLoopback(); const dir = yield* useTempDirectory("xmd-fetch-cli-"); const path = join(dir, "doc.md"); diff --git a/packages/cli/tests/plan-cli.test.ts b/packages/cli/tests/plan-cli.test.ts index fb732f64b..47392c15f 100644 --- a/packages/cli/tests/plan-cli.test.ts +++ b/packages/cli/tests/plan-cli.test.ts @@ -174,7 +174,7 @@ function complaints(stderr: string): string { /** Every phase after a refusal, at zero. */ function untouched(harness: PlanHarness): Record { return { - catalogs: harness.catalogCalls.length, + catalogs: harness.symbolCalls.length, runtimes: harness.fake.created.length, started: harness.fake.started, turns: harness.fake.prompts.length, @@ -803,8 +803,9 @@ describe( "No Plan was returned. Nothing was output.", ); // Refused before the catalog, a directory, a provider, a turn or a - // review existed. The catalog is built by `` now, and this - // refusal happens before the command document starts at all. + // review existed. The catalog is observed by public `` inside + // the command document, and this refusal happens before that document + // starts at all. expect(untouched(harness)).toEqual({ catalogs: 0, runtimes: 0, @@ -1698,8 +1699,8 @@ describe( // observes is that Preparing reached the operator before the catalog // was read rather than merely that both happened. const before: string[][] = []; - const catalog = harness.deps.catalog; - harness.deps.catalog = function* (includes) { + const catalog = harness.deps.symbols; + harness.deps.symbols = function* (includes) { before.push(phasesOf(harness.progress.join(""))); return yield* catalog(includes); }; @@ -1707,7 +1708,7 @@ describe( const { value } = yield* delivered(() => runPlan(planning(dir), harness.deps)); expect(value).toBe(0); - expect(harness.catalogCalls).toEqual([[dir]]); + expect(harness.symbolCalls).toEqual([[dir]]); expect(before).toEqual([["Preparing the Plan"]]); }); }); diff --git a/packages/cli/tests/plan-command-document.test.ts b/packages/cli/tests/plan-command-document.test.ts index 7e1746fa3..eeee655ff 100644 --- a/packages/cli/tests/plan-command-document.test.ts +++ b/packages/cli/tests/plan-command-document.test.ts @@ -33,13 +33,23 @@ import { retainedSource, useNormalizedOutput, } from "@executablemd/core"; -import type { DocumentValidation, ElicitationRequest, Json } from "@executablemd/core"; +import type { + DocumentValidation, + ElicitationRequest, + Json, + SyntaxSymbols, +} from "@executablemd/core"; import { executeInstalled } from "@executablemd/core/host"; import { InMemoryStream } from "@executablemd/durable-streams"; import { PLAN_COMMAND_DOCUMENT, readPackagedDocument } from "../src/packaged-document.ts"; import { PLAN_COMMAND_IDENTITY } from "../src/authorship-profile.ts"; import type { PlanSurface } from "../src/plan-component.ts"; -import { AGENT, planDeclarationHarness, useWorkingDirectory } from "./support/plan-harness.ts"; +import { + AGENT, + CASE_CATALOG, + planDeclarationHarness, + useWorkingDirectory, +} from "./support/plan-harness.ts"; import type { ScriptedReview } from "./support/plan-harness.ts"; import type { ScriptedTurn } from "./support/fake-acp.ts"; @@ -132,10 +142,13 @@ function* runDocument(options: RunOptions = {}): Operation { session: SESSION, explicitSession: true, ...(options.verbose === undefined ? {} : { verbose: options.verbose }), + // The catalog is stated at the execution boundary now, so this records + // *when* the public `` occurrence observed it — which is what an + // ordering case about the authored Preparing phase is asking. // deno-lint-ignore require-yield - *catalog() { + *symbols(): Operation { events.push("catalog"); - return "## Built-in components\n\n### ``\n"; + return CASE_CATALOG; }, *validate(): Operation { const answer = validations.length > 1 ? validations.shift() : validations[0]; @@ -174,6 +187,7 @@ function* runDocument(options: RunOptions = {}): Operation { { components: agentIdentityComponents(), declarations: [harness.declaration], + symbols: harness.symbols, }, ], ); @@ -486,7 +500,13 @@ describe("the packaged plan command document", () => { secretDetection: true, props: { request: REQUEST, session: SESSION }, }, - [{ components: agentIdentityComponents(), declarations: [installed] }], + [ + { + components: agentIdentityComponents(), + declarations: [installed], + symbols: harness.symbols, + }, + ], ); // deno-lint-ignore require-yield yield* forEach(function* (chunk: string) { diff --git a/packages/cli/tests/plan-component.test.ts b/packages/cli/tests/plan-component.test.ts index 63e7cd44c..f864df990 100644 --- a/packages/cli/tests/plan-component.test.ts +++ b/packages/cli/tests/plan-component.test.ts @@ -30,9 +30,12 @@ import { retainedSource, useNormalizedOutput, } from "@executablemd/core"; -import type { Json } from "@executablemd/core"; -import { validateDocument } from "@executablemd/core"; -import { executeInstalled, sourceDigest } from "@executablemd/core/host"; +import type { Json, SyntaxSymbols } from "@executablemd/core"; +import { registerComponents, validateDocument } from "@executablemd/core"; +import { executeInstalled, fileReadEntry, sourceDigest } from "@executablemd/core/host"; +import type { FragmentEvaluationInput } from "@executablemd/core/host"; +import { recordedFiles } from "../../core/tests/support/fragment-files.ts"; +import type { RecordedFiles } from "../../core/tests/support/fragment-files.ts"; import { InMemoryStream } from "@executablemd/durable-streams"; import type { DurableEvent } from "@executablemd/durable-streams"; @@ -49,7 +52,8 @@ import { structuralValidation, } from "../src/plan-component.ts"; import type { StructuralValidation } from "../src/plan-component.ts"; -import { syntaxCatalog } from "../src/syntax.ts"; +import { syntaxSymbols } from "../src/syntax.ts"; +import { PLAN_DOCUMENT, readPackagedDocument } from "../src/packaged-document.ts"; const ROOT = "document.md"; @@ -102,6 +106,12 @@ function* runDocument(options: { validate?: StructuralValidation; session?: string; props?: Record; + /** + * The ceiling a generated fragment runs under, for the rows that write + * `` beside or around a ``. Absent for every other row, which + * is what makes "this execution offers no evaluation" the default. + */ + evaluation?: FragmentEvaluationInput; /** * Install the output middleware an ordinary `xmd run` installs. * @@ -153,6 +163,11 @@ function* runDocument(options: { { components: agentIdentityComponents(), declarations: [harness.declaration], + // Where the profile a document observes is settled now: the + // `` the packaged Plan writes is canonical core's public + // component, and what it answers with is this execution's. + symbols: harness.symbols, + ...(options.evaluation === undefined ? {} : { evaluation: options.evaluation }), }, ], ); @@ -228,6 +243,70 @@ describe("Tier PC — in an ordinary document", () => { }); }); + it("PC1b: the packaged Plan writes the public , and its catalog reaches the first turn once", function* () { + yield* useWorkingDirectory(function* () { + const source = yield* readPackagedDocument(PLAN_DOCUMENT); + // The public component, written the way any document writes it. + expect(source).toContain(''); + // And nothing declares a second one: the private closure is the five + // phases, and `Syntax` is not among them. + const declaration = yield* planComponentDescription(); + expect((declaration.privates ?? []).map((component) => component.name)).toEqual([ + "PlanInputs", + "PlanAuthorship", + "PlanProgress", + "CheckDraft", + "AdmitPlan", + ]); + + const run = yield* runDocument({ + source: ['Write a program.', "", "got: {approved}", ""].join( + "\n", + ), + reply: PLAN, + }); + + expect(run.failure).toBe(undefined); + // The retained catalog is what the first turn was built from, and it is + // there exactly once — a second copy would mean the Component both bound + // it and emitted it. + const first = run.harness.fake.prompts[0] ?? ""; + expect(first).toContain("### ``"); + expect(first.split("### ``").length - 1).toBe(1); + expect(run.harness.symbolCalls).toBe(1); + }); + }); + + it("PC1c: a catalog reference that fails reaches no session, turn, review or Plan", function* () { + yield* useWorkingDirectory(function* (dir) { + const harness = yield* planDeclarationHarness({ + surface: "component", + authorshipRoot: `${dir}-profile`, + // deno-lint-ignore require-yield + *symbols(): Operation { + throw new Error("the profile could not be described"); + }, + }); + const run = yield* runDocument({ + source: ['Write a program.', "", "got: {approved}", ""].join( + "\n", + ), + reply: PLAN, + reviews: [], + harness, + }); + + expect(run.failure).toContain("the profile could not be described"); + // Nothing downstream of the reference happened: no turn was taken, no + // review was asked, no draft was checked, and no Plan was bound. + expect(run.harness.fake.prompts).toEqual([]); + expect(run.harness.reviews).toEqual([]); + expect(run.harness.checked).toEqual([]); + expect(run.output).not.toContain("# Say hello"); + expect(run.emitted).not.toContain("# Say hello"); + }); + }); + it("PC2: the Prompt is not emitted, and no Plan source is printed", function* () { yield* useWorkingDirectory(function* () { const run = yield* runDocument({ @@ -308,19 +387,34 @@ describe("Tier PC — in an ordinary document", () => { it("PC6: the private capabilities resolve nowhere a document can write", function* () { yield* useWorkingDirectory(function* () { - for (const name of ["PlanInputs", "PlanAuthorship", "CheckDraft", "AdmitPlan"]) { + for (const name of [ + "PlanInputs", + "PlanAuthorship", + "PlanProgress", + "CheckDraft", + "AdmitPlan", + ]) { const run = yield* runDocument({ source: [`<${name} as="x" />`, ""].join("\n"), reviews: [], }); expect(run.failure).toContain(`Cannot resolve component: ${name}`); } + // The positive control for the same document shape: `` is not one + // of Plan's private names, it is the public component canonical core owns, + // so the identical invocation resolves and binds the catalog. + const open = yield* runDocument({ + source: ['', "{x}", ""].join("\n"), + reviews: [], + }); + expect(open.failure).toBeUndefined(); + expect(open.output).toContain("### ``"); }); }); it("PC7: the catalog advertises and none of its private names", function* () { yield* useWorkingDirectory(function* () { - const catalog = yield* syntaxCatalog([]); + const catalog = yield* syntaxSymbols([]); const builtIn = catalog.categories[1].entries; const plan = builtIn.find((entry) => entry.name === "Plan"); @@ -333,15 +427,25 @@ describe("Tier PC — in an ordinary document", () => { expect(Reflect.get(Object(plan?.origin), "origin")).toBe(PLAN_ORIGIN); // The description a document author reads is the packaged Component's own // frontmatter, so the asset and the entry describing it are one text. - expect(plan?.description).toContain("Create an XMD program from a prompt."); - expect(plan?.description).toContain("emits the approved program source."); + expect(plan?.description).toContain("Generate program text from a prompt."); + expect(plan?.description).toContain("emits the approved program text"); for (const category of catalog.categories) { const names = category.entries.map((entry) => entry.name); - for (const priv of ["PlanInputs", "PlanAuthorship", "CheckDraft", "AdmitPlan"]) { + for (const priv of [ + "PlanInputs", + "PlanAuthorship", + "PlanProgress", + "CheckDraft", + "AdmitPlan", + ]) { expect(names).not.toContain(priv); } } + // `` is not one of them any more. It is public, canonical core + // owns it, and the catalog an author reads says so — which is what stops + // the absence check above passing because the whole set went missing. + expect(builtIn.map((entry) => entry.name)).toContain("Syntax"); }); }); @@ -353,7 +457,7 @@ describe("Tier PC — in an ordinary document", () => { // unresolved, and never runs. yield* writeTextFile(join(dir, "CheckDraft.md"), "the repository file ran.\n"); - const catalog = yield* syntaxCatalog([dir]); + const catalog = yield* syntaxSymbols([dir]); for (const category of catalog.categories) { expect(category.entries.map((entry) => entry.name)).not.toContain("CheckDraft"); } @@ -421,11 +525,20 @@ describe("Tier PC — in an ordinary document", () => { // differently and a review nobody scripted. Neither is reached: the turn, // the check, the approval and the admission are all restored. const partial = yield* continuing(first); + let catalogs = 0; const two = yield* runDocument({ source, reply: "# A different Plan\n\nnot this one.\n", reviews: [], stream: partial, + harness: yield* planDeclarationHarness({ + surface: "component", + authorshipRoot: yield* authorshipRoot(), + *symbols() { + catalogs += 1; + throw new Error("a restored syntax snapshot was rebuilt"); + }, + }), }); expect(two.failure).toBe(undefined); @@ -433,6 +546,7 @@ describe("Tier PC — in an ordinary document", () => { expect(two.harness.fake.prompts).toEqual([]); expect(two.harness.reviews).toEqual([]); expect(two.harness.checked).toEqual([]); + expect(catalogs).toBe(0); }); }); @@ -978,7 +1092,7 @@ describe("Tier PC — in an ordinary document", () => { // a member of the wrong type. Each is refused with the same fixed // sentence, and none of them produces source or a binding. const cases: [string, (value: Json) => Json][] = [ - ["a member is missing", (value) => ({ syntax: Object(value).syntax })], + ["the member is missing", () => ({})], [ "a member this version does not know was added", (value) => ({ ...Object(value), extra: "surprise" }), @@ -999,6 +1113,41 @@ describe("Tier PC — in an ordinary document", () => { }); }); + it("PC27: the Plan's syntax record is core's closed record, and a hostile one produces nothing", function* () { + yield* useWorkingDirectory(function* () { + const approved = yield* approvedRun(); + // The record is canonical core's, not Plan's: the packaged Component + // writes the same public `` any document writes, so what a + // continuation restores is a `syntax_symbols` reference rather than + // anything this host retained. + const reference = (yield* approved.readAll()).find( + (event) => event.type === "yield" && event.description.type === "syntax_symbols", + ); + expect(reference?.type).toBe("yield"); + if (reference?.type !== "yield" || reference.result.status !== "ok") { + throw new Error("the approved run retained no syntax record"); + } + const value = Object(reference.result.value); + expect(Object.keys(value)).toEqual(["symbols"]); + expect(typeof value.symbols).toBe("string"); + + const cases: [string, (value: Json) => Json][] = [ + ["the member is missing", () => ({})], + ["an unknown member was added", (record) => ({ ...Object(record), extra: true })], + ["the member has the wrong type", () => ({ symbols: 7 })], + ]; + + for (const [, replace] of cases) { + const run = yield* continued(yield* tampered(approved, "syntax_symbols:", replace)); + expect(run.failure).toContain("retained text is not a record"); + expect(run.output).not.toContain("got:"); + expect(run.output).not.toContain("# Say hello"); + expect(run.harness.fake.prompts).toEqual([]); + expect(run.harness.reviews).toEqual([]); + } + }); + }); + it("PC26: a retained artifact this version cannot read produces nothing", function* () { yield* useWorkingDirectory(function* () { const cases: [string, (value: Json) => Json][] = [ @@ -1056,3 +1205,313 @@ describe("Tier PC — in an ordinary document", () => { }); }); }); + +/** + * Tier FE — `` beside ``. + * + * The two are complements and neither is the other. `` produces text and + * runs none of it; `` runs text and produces none of it. What these + * rows fix is the seam between them: which vocabulary each is told about, and + * that passing one to the other is an ordinary string binding rather than a + * type either of them shares. + */ +describe("Tier FE — Plan produces text, Evaluate runs it", () => { + /** A profile admitting one read, over a recorder these rows can count. */ + function profile(files: RecordedFiles): FragmentEvaluationInput { + return { read: [fileReadEntry()], files }; + } + + /** + * Two component names invented for these rows, and two markers that occur + * nowhere else. + * + * ``, `` and `` are all unusable as discriminators + * here: Plan's own fixed instructions name components, so a row asserting on + * one of them can pass on prose rather than on the catalog. These two exist + * only in this suite, so the marker appearing at all means the catalog + * described the entry. + */ + const READ_MARKER = "FE713_READ_MARKER"; + const WIDE_MARKER = "FE713_WIDE_MARKER"; + + /** The admitted read: available inside an evaluation and at the wide site. */ + const READ_COMPONENT = { + name: "FE713.Read", + origin: "test://fe713/read", + props: { + type: "object", + properties: { path: { type: "string" } }, + required: ["path"], + additionalProperties: false, + }, + description: `Reads one path. ${READ_MARKER}`, + }; + + /** Available at the ordinary authored site, and in no evaluation profile. */ + const WIDE_COMPONENT = { + name: "FE713.WideOnly", + origin: "test://fe713/wide", + props: { type: "object", properties: {}, additionalProperties: false }, + description: `Does nothing an evaluation may do. ${WIDE_MARKER}`, + }; + + /** + * The profile these rows evaluate under: the read entry, and never the wide + * one. + * + * The entry's identity is the fixture's own, so the catalog an agent is shown + * inside an evaluation names `test://fe713/read` and the admitted capability + * is core's file read behind it. + */ + function fixtureProfile(files: RecordedFiles): FragmentEvaluationInput { + return { + read: [ + { + ...fileReadEntry(), + name: READ_COMPONENT.name, + identity: { origin: READ_COMPONENT.origin, key: "FE713.Read", revision: "1" }, + description: READ_COMPONENT.description, + }, + ], + files, + }; + } + + /** Register both fixture components at the ordinary authored site. */ + function* useFixtureComponents(seen: string[]): Operation { + yield* registerComponents([ + { + ...READ_COMPONENT, + // deno-lint-ignore require-yield + *fn(): Operation { + return ""; + }, + }, + { + ...WIDE_COMPONENT, + // deno-lint-ignore require-yield + *fn(): Operation { + return ""; + }, + }, + { + // An ordinary producer-site component that records what `` + // rendered and emits nothing, so the catalog can be asserted on without + // it becoming part of the program. + name: "RememberSyntax", + origin: "test://fe713/remember", + props: { + type: "object", + properties: { text: { type: "string" } }, + required: ["text"], + additionalProperties: false, + }, + // deno-lint-ignore require-yield + *fn(props: Record): Operation { + seen.push(String(props.text)); + return ""; + }, + }, + ]); + } + + it("FE22: the approved bytes are inert text `` then runs", function* () { + yield* useWorkingDirectory(function* (dir) { + yield* writeTextFile(join(dir, "notes.md"), "the retained note"); + const files = recordedFiles({ "notes.md": "the retained note" }); + // What the agent approves is a program. `` hands it back as a + // string and performs none of it. + const PROGRAM = `\n`; + + const run = yield* runDocument({ + source: [ + 'Write a program.', + "", + '', + "", + "", + "", + ].join("\n"), + reply: PROGRAM, + evaluation: profile(files), + }); + + expect(run.failure).toBe(undefined); + // `` ran no fragment operation of its own. If it had evaluated what + // it approved, the recorder would show a read before `` ever + // reached it — and there would be two. + expect(files.performed).toEqual(["read notes.md"]); + // And `` did run it, so the row is not passing because nothing + // happened at all. + expect(run.output).toContain("the retained note"); + }); + }); + + it("FE22: an approved program is not run when no `` is written", function* () { + yield* useWorkingDirectory(function* () { + const files = recordedFiles({ "notes.md": "the retained note" }); + const run = yield* runDocument({ + source: ['Write a program.', "", "done", ""].join("\n"), + reply: `\n`, + evaluation: profile(files), + }); + + expect(run.failure).toBe(undefined); + expect(run.output).toContain("done"); + // The negative control for the row above: the same approved bytes, the + // same live profile, and nothing performed — because inertness is a + // property of `` rather than of whether a profile existed. + expect(files.performed).toEqual([]); + }); + }); + + it("FE8: a Plan inside `` is told the admitted vocabulary alone", function* () { + yield* useWorkingDirectory(function* () { + const files = recordedFiles({ "notes.md": "the retained note" }); + const seen: string[] = []; + + const run = yield* scoped(function* () { + yield* useFixtureComponents(seen); + return yield* runDocument({ + source: [ + '', + '', + "", + "Generate the requested fragment.", + "", + "", + ].join("\n"), + reply: `\n`, + evaluation: fixtureProfile(files), + }); + }); + + expect(run.failure).toBe(undefined); + // The public `` rendered inside the producer saw the narrowed + // reference. Its marker is the admitted entry's own description, which + // occurs nowhere in Plan's fixed prose. + expect(seen).toHaveLength(1); + const catalog = seen[0] ?? ""; + expect(catalog).toContain(READ_MARKER); + expect(catalog).toContain(READ_COMPONENT.origin); + // The wide-only component exists in this scope and is not in the profile, + // so its absence here is the narrowing rather than the fixture. + expect(catalog).not.toContain(WIDE_MARKER); + // And this is a narrowed catalog rather than the site's: the authored + // site has built-in components, and a fragment has none — FE9 asserts the + // other side of exactly this sentence. + expect(catalog).toContain("No components are registered in this profile."); + expect(catalog).not.toContain("### ``"); + + // Plan's own first turn was built from the same narrowed reference: the + // packaged Component writes `` too, and both must agree about + // what a fragment may contain. + const prompt = run.harness.fake.prompts[0] ?? ""; + expect(prompt.split(READ_MARKER).length - 1).toBe(1); + expect(prompt).not.toContain(WIDE_MARKER); + + // And the program the agent wrote against it ran, exactly once, through + // the captured operation. + expect(files.performed).toEqual(["read notes.md"]); + }); + }); + + it("FE8: a fragment naming the wide-only component refuses with no operation", function* () { + yield* useWorkingDirectory(function* () { + const files = recordedFiles({ "notes.md": "the retained note" }); + const seen: string[] = []; + + const run = yield* scoped(function* () { + yield* useFixtureComponents(seen); + return yield* runDocument({ + source: [ + '', + "Generate the requested fragment.", + "", + "", + ].join("\n"), + // The agent writes the component it was *not* told about. It resolves + // at the ordinary site and is not in the profile. + reply: `\n`, + evaluation: fixtureProfile(files), + }); + }); + + // The negative control for the row above: refused whole, before any + // effect, and the recorder shows the evaluation reached no operation. + expect(run.failure).not.toBe(undefined); + expect(String(run.failure)).toContain("did not admit"); + expect(files.performed).toEqual([]); + }); + }); + + it("FE9: a deferred Plan keeps its own site's wider vocabulary", function* () { + yield* useWorkingDirectory(function* () { + const files = recordedFiles({ "notes.md": "the retained note" }); + const seen: string[] = []; + + const run = yield* scoped(function* () { + yield* useFixtureComponents(seen); + return yield* runDocument({ + source: [ + '', + "", + 'Generate the requested fragment.', + '', + "", + ].join("\n"), + // Written against the wider catalog: the admitted read first, then + // the component only the authored site has. + reply: `\n\n`, + evaluation: fixtureProfile(files), + }); + }); + + // The Plan ran at the document's own site, so its catalog is the host + // profile's rather than an evaluation's. The discriminating fact against + // FE8, in the same rendered structure: this site *has* built-in + // components, and a narrowed one reports none. + expect(seen).toHaveLength(1); + const catalog = seen[0] ?? ""; + expect(catalog).toContain("## Built-in components"); + expect(catalog).toContain("### ``"); + expect(catalog).not.toContain("No components are registered in this profile."); + // And it is not the admitted vocabulary: the fragment's own entry is not + // something the authored site describes. + expect(catalog).not.toContain(READ_MARKER); + const prompt = run.harness.fake.prompts[0] ?? ""; + expect(prompt).toContain("### ``"); + expect(prompt).not.toContain(READ_MARKER); + + // The later, narrower `` refuses the whole fragment. The + // admitted read is written *first*, so a refusal that happened + // element-by-element would already have performed it — the empty + // recorder is what proves preflight decided the whole fragment. + expect(run.failure).not.toBe(undefined); + expect(String(run.failure)).toContain("did not admit"); + expect(files.performed).toEqual([]); + }); + }); + + it("FE9: a later narrower Evaluate refuses text the wider site allowed", function* () { + yield* useWorkingDirectory(function* () { + const files = recordedFiles({ "notes.md": "the retained note" }); + // The agent, told the document's own vocabulary, writes something a + // fragment may not contain. The narrower `` refuses it before + // any effect rather than running part of it. + const run = yield* runDocument({ + source: [ + 'Write a program.', + "", + "", + "", + ].join("\n"), + reply: `\n\n\n`, + evaluation: profile(files), + }); + + expect(run.failure).not.toBe(undefined); + expect(files.performed).toEqual([]); + }); + }); +}); diff --git a/packages/cli/tests/plan.test.ts b/packages/cli/tests/plan.test.ts index 8e7689dcb..d99822782 100644 --- a/packages/cli/tests/plan.test.ts +++ b/packages/cli/tests/plan.test.ts @@ -280,7 +280,7 @@ describe( expect(code).toBe(0); // Exactly one catalog, built with the invocation's own includes. - expect(harness.catalogCalls).toEqual([[dir]]); + expect(harness.symbolCalls).toEqual([[dir]]); // The turn is the shipped Markdown's, word for word: the sentences below // exist nowhere in TypeScript, so a host that wrote its own workflow could diff --git a/packages/cli/tests/run-composition.test.ts b/packages/cli/tests/run-composition.test.ts index a7da72545..0368e3131 100644 --- a/packages/cli/tests/run-composition.test.ts +++ b/packages/cli/tests/run-composition.test.ts @@ -27,7 +27,7 @@ import { exists, readdir, readTextFile, writeTextFile } from "@effectionx/fs"; import { useTempDirectory } from "@executablemd/test-support/temp"; import { join } from "node:path"; import { COMPOSITION_REGISTRATIONS } from "@executablemd/workflow"; -import { syntaxCatalog, useRunProfileRegistry } from "../src/syntax.ts"; +import { syntaxSymbols, useRunProfileRegistry } from "../src/syntax.ts"; import { DEFAULT_REPOSITORY_ROOT, unsupportedRepositories } from "../src/run-repositories.ts"; /** Every element an author can write that needs a repository provider. */ @@ -142,7 +142,7 @@ describe("ORC1 — describing the vocabulary reaches nothing", () => { }, { at: "min" }, ); - return yield* syntaxCatalog([]); + return yield* syntaxSymbols([]); }); // The whole vocabulary is described. @@ -183,7 +183,7 @@ describe("ORC2 — one language, described everywhere and operated somewhere", ( ); // And the catalog every runtime builds describes each of them completely. - const catalog = yield* scoped(() => syntaxCatalog([])); + const catalog = yield* scoped(() => syntaxSymbols([])); const builtIn = catalog.categories[1].entries; for (const name of COMPOSITION_NAMES) { const entry = builtIn.find((candidate) => candidate.name === name); diff --git a/packages/cli/tests/stdin-cli.test.ts b/packages/cli/tests/stdin-cli.test.ts index 4d9d27fc5..b1e7f630f 100644 --- a/packages/cli/tests/stdin-cli.test.ts +++ b/packages/cli/tests/stdin-cli.test.ts @@ -10,7 +10,10 @@ import { describe, it } from "@executablemd/test-support/bdd"; import { expect } from "@executablemd/test-support/expect"; -import { runCli } from "@executablemd/test-support/launch"; +import { runCli, runShell } from "@executablemd/test-support/launch"; +import { when } from "@effectionx/converge"; +import { Buffer } from "node:buffer"; +import type { ChildProcess } from "node:child_process"; import { createContext, Err, @@ -132,6 +135,17 @@ const CONFLICT_FIXTURE = { ].join("\n"), }; +/** What the shared launcher attaches to a child it owns, and to its pipes. */ +function attached(child: ChildProcess): number[] { + return [ + child.listenerCount("error"), + child.listenerCount("close"), + child.stdout?.listenerCount("data") ?? 0, + child.stderr?.listenerCount("data") ?? 0, + child.stdin?.listenerCount("error") ?? 0, + ]; +} + const STDIN_EFFECT = 'x\n\nSTDIN_MARKER\n'; describe( @@ -547,11 +561,25 @@ describe( "", ].join("\n"); + // The child this launcher owns, what its streams carried before the + // launcher attached anything, and what the run has captured from it. + let child: ChildProcess | undefined; + let before: number[] = []; + let captured: () => { stdout: string; stderr: string } = () => ({ + stdout: "", + stderr: "", + }); + const run = yield* spawn(function* () { yield* runCli(["run", "-", "--raw"], { cwd: dir, stdin: document, timeout: 60_000, + observeChild: (started, reader) => { + child = started; + before = attached(started); + captured = reader; + }, }).join(); }); @@ -564,10 +592,42 @@ describe( const [escaped, owned] = yield* waitForIds(idsPath); yield* ensure(() => endEscapedGroup(escaped)); + // Synchronized on the command having run, so every handler is attached + // and the child is still alive when the run is cancelled underneath it. + if (!child) { + throw new Error("the launcher never owned a child"); + } + const live = attached(child); + const seen = captured(); + + // At least what this owner attached, not exactly it: a runtime may hold + // handlers of its own on a child and its pipes, so the release below is + // measured against what was live rather than against the baseline. + live.forEach((count, index) => { + expect(count).toBeGreaterThanOrEqual(before[index] + 1); + }); + const at = Date.now(); yield* run.halt(); const elapsed = Date.now() - at; + // Cancelled while all five were live, and back to what the runtime had + // before the launcher attached anything. Read before the events are + // replayed, because a handler removed by its own event would leave the + // same counts behind as one the launcher released. + const released = live.map((count) => count - 1); + + expect(attached(child)).toEqual(released); + + child.stdout?.emit("data", Buffer.from("after the run was cancelled")); + child.stderr?.emit("data", Buffer.from("after the run was cancelled")); + child.emit("close", 0, null); + + // And nothing is still accumulating: the capture the run abandoned did + // not grow. + expect(captured()).toEqual(seen); + expect(attached(child)).toEqual(released); + // A signal that has been sent is not a process that is gone: teardown // returning as soon as it signalled would come back at once. Nor may it // wait forever on a child that is never going to answer SIGTERM — the @@ -580,6 +640,59 @@ describe( }); }); + /** + * Teardown's boundary is the child's `close`, not the signal it was sent + * and not an exit status the handle has recorded. This child traps the + * interrupt, writes on its way out, and only then leaves — so the marker + * can only be here if the capture was still attached while it stopped and + * the cancellation waited for the pipes to end. + */ + it("SI12b: what a cancelled child writes while it stops is still captured", function* () { + let child: ChildProcess | undefined; + let before: number[] = []; + let captured: () => { stdout: string; stderr: string } = () => ({ + stdout: "", + stderr: "", + }); + + const run = yield* spawn(function* () { + // A large trailing write, so what is asserted below is the whole of it + // rather than the first line to arrive. + yield* runShell("trap 'yes LATE | head -20000; exit 0' TERM; echo READY; sleep 20", { + stdin: "", + timeout: 60_000, + observeChild: (started, reader) => { + child = started; + before = attached(started); + captured = reader; + }, + }).join(); + }); + + // Synchronized on the child running, so the halt lands on a live process + // with every handler attached. + yield* when(function* () { + expect(captured().stdout).toContain("READY"); + }); + + if (!child) { + throw new Error("the launcher never owned a child"); + } + const live = attached(child); + + yield* run.halt(); + + live.forEach((count, index) => { + expect(count).toBeGreaterThanOrEqual(before[index] + 1); + }); + // All of it, written after the signal and before `close`: the capture was + // still attached while the child stopped, and teardown did not return + // until the pipes had ended. + expect(captured().stdout.split("LATE").length - 1).toBe(20_000); + // And released only once that had happened. + expect(attached(child)).toEqual(live.map((count) => count - 1)); + }); + it("SI9b: the stream adapter's listeners belong to the read's own scope", function* () { const stream = new PassThrough(); const read = yield* spawn(() => readInputStream(stream)); diff --git a/packages/cli/tests/stdout-delivery.test.ts b/packages/cli/tests/stdout-delivery.test.ts index b24b63b55..5c9183652 100644 --- a/packages/cli/tests/stdout-delivery.test.ts +++ b/packages/cli/tests/stdout-delivery.test.ts @@ -14,7 +14,7 @@ import { describe, it } from "@executablemd/test-support/bdd"; import { expect } from "@executablemd/test-support/expect"; -import { scoped, spawn, withResolvers } from "effection"; +import { ensure, scoped, spawn, withResolvers } from "effection"; import { EventEmitter } from "node:events"; import process from "node:process"; import { deliverWhole } from "../src/stdout-delivery.ts"; @@ -108,8 +108,13 @@ describe("Tier SDL — the listener lives for one delivery", () => { it("SDL2: the trailing event still finds the listener, which is gone after it", function* () { const sink = new RecordingSink(CALLBACK_THEN_EVENT); // A sentinel, so that detaching too early is this row's assertion rather - // than an unhandled `error` event thrown from a timer. - sink.on("error", () => {}); + // than an unhandled `error` event thrown from a timer. It belongs to the + // test, and comes off with it. + const sentinel = (): void => {}; + yield* ensure(() => { + sink.off("error", sentinel); + }); + sink.on("error", sentinel); const before = listeners(sink); const result = yield* deliverWhole("catalog", sink); @@ -170,7 +175,13 @@ describe("Tier SDL — the listener lives for one delivery", () => { it("SDL7: no finished delivery absorbs a later, unrelated failure", function* () { const sink = new RecordingSink(ACCEPTS); const seen: Error[] = []; - sink.on("error", (error: Error) => seen.push(error)); + const sentinel = (error: Error): void => { + seen.push(error); + }; + yield* ensure(() => { + sink.off("error", sentinel); + }); + sink.on("error", sentinel); yield* deliverWhole("catalog", sink); sink.emit("error", OTHER); diff --git a/packages/cli/tests/support/plan-harness.ts b/packages/cli/tests/support/plan-harness.ts index f3922f248..3435f2419 100644 --- a/packages/cli/tests/support/plan-harness.ts +++ b/packages/cli/tests/support/plan-harness.ts @@ -3,7 +3,7 @@ * * Every phase the command owns is driven in process: the ACPX runtime is the * scriptable fake, the review provider is a scripted `Elicitation` handler, the - * catalog and the execution are recorded, and the contextual working directory + * symbols and the execution are recorded, and the contextual working directory * is a temporary one. No live agent, browser, or network belongs in this * evidence. * @@ -13,7 +13,7 @@ */ import { Elicitation } from "@executablemd/core"; -import type { DocumentValidation, ElicitationRequest, SyntaxCatalog } from "@executablemd/core"; +import type { DocumentValidation, ElicitationRequest, SyntaxSymbols } from "@executablemd/core"; import { Err, Ok } from "effection"; import type { Operation, Result } from "effection"; import { ensure, scoped, useScope } from "effection"; @@ -24,12 +24,12 @@ import { join } from "node:path"; import { API, useHostFiles } from "@executablemd/runtime"; import { createEmbeddedAdapters } from "@executablemd/acp/embedded-adapters"; import type { EmbeddedAdapters } from "@executablemd/acp/embedded-adapters"; -import { syntaxCatalog } from "../../src/syntax.ts"; +import { syntaxSymbols } from "../../src/syntax.ts"; import { planComponentDeclaration } from "../../src/plan-component.ts"; import type { PlanSurface, StructuralValidation } from "../../src/plan-component.ts"; import { planAgentContext } from "../../src/authorship-profile.ts"; import type { AuthorshipStack } from "../../src/agent-stack.ts"; -import type { DeclaredMarkdownComponent } from "@executablemd/core/host"; +import type { SyntaxSymbolsProvider, DeclaredMarkdownComponent } from "@executablemd/core/host"; import type { PlanDependencies } from "../../src/plan.ts"; import { createFakeAcp, makeRegistry, makeStore } from "./fake-acp.ts"; import type { FakeAcp, FakeStore } from "./fake-acp.ts"; @@ -70,8 +70,8 @@ export interface ScriptedReview { export interface PlanHarness { fake: FakeAcp; - /** Every catalog request, by the includes it was made with. */ - catalogCalls: string[][]; + /** Every symbols request, by the includes it was made with. */ + symbolCalls: string[][]; /** Every review request a provider was asked, in order. */ reviews: ElicitationRequest[]; /** @@ -98,8 +98,8 @@ export function createPlanHarness(options: { * whose was whose. */ authorshipRoot: string; - /** Replace the catalog entirely, for a case about catalog failure. */ - catalog?: (includes: readonly string[]) => Operation; + /** Replace the symbols entirely, for a case about their failure. */ + symbols?: (includes: readonly string[]) => Operation; /** * The ACPX session store this invocation reads and writes. * @@ -122,7 +122,7 @@ export function createPlanHarness(options: { refuseProgress?: (chunk: string, index: number) => Operation; }): PlanHarness { const fake = createFakeAcp(); - const catalogCalls: string[][] = []; + const symbolCalls: string[][] = []; const reviews: ElicitationRequest[] = []; const answers: ScriptedReview[] = []; const progress: string[] = []; @@ -130,7 +130,7 @@ export function createPlanHarness(options: { const harness: PlanHarness = { fake, - catalogCalls, + symbolCalls, reviews, progress, script(review) { @@ -158,9 +158,9 @@ export function createPlanHarness(options: { sessionStore: options.store ?? makeStore(), agentRegistry: makeRegistry({ [AGENT]: `${AGENT}-cmd` }), }, - *catalog(includes) { - catalogCalls.push([...includes]); - return yield* (options.catalog ?? syntaxCatalog)(includes); + *symbols(includes) { + symbolCalls.push([...includes]); + return yield* (options.symbols ?? syntaxSymbols)(includes); }, authorshipRoot: options.authorshipRoot, *installElicitation() { @@ -291,6 +291,17 @@ export interface PlanDeclarationHarness { reviews: ElicitationRequest[]; /** The declaration to attach to an execution. */ declaration: DeclaredMarkdownComponent; + /** + * The symbols this case's execution describes. + * + * Attached to the execution rather than to the declaration, because that is + * where the profile a document is shown is now settled: `` is + * public, canonical core owns it, and what it answers with is the execution's + * own. + */ + symbols: SyntaxSymbolsProvider; + /** How many times that provider was asked. */ + symbolCalls: number; /** Review answers, taken in order. Running out is a test defect, not a case. */ script(review: ScriptedReview): void; } @@ -308,16 +319,14 @@ export function* planDeclarationHarness(options: { surface: PlanSurface; authorshipRoot: string; includes?: readonly string[]; - /** The catalog the first turn is built from. */ - syntax?: string; /** - * Build the catalog, in place of answering with {@link syntax}. + * The symbols this case's execution describes, in place of the default below. * - * A case that needs to know *when* the catalog was built supplies this, which - * is the only way to tell an authored phase that precedes the preparation from - * one that follows it. + * A case that needs to know *when* they were read supplies this, which is the + * only way to tell an authored phase that precedes the read from one that + * follows it. */ - catalog?: () => Operation; + symbols?: () => Operation; /** * How this case answers the one structural question the Component asks. * @@ -386,12 +395,6 @@ export function* planDeclarationHarness(options: { { at: "min" }, ); }, - *catalog() { - if (options.catalog !== undefined) { - return yield* options.catalog(); - } - return options.syntax ?? "## Built-in components\n\n### ``\n"; - }, // The deterministic seam standing where production's answer goes, recording // every candidate it was asked about — the draft check's and the // admission's alike, which is every time these bytes are decided on. @@ -402,13 +405,53 @@ export function* planDeclarationHarness(options: { }, }); - return { + const harness: PlanDeclarationHarness = { fake, checked, reviews, declaration, + symbolCalls: 0, + *symbols(): Operation { + harness.symbolCalls += 1; + if (options.symbols !== undefined) { + return yield* options.symbols(); + } + return CASE_CATALOG; + }, script(review) { answers.push(review); }, }; + return harness; } + +/** + * The vocabulary a case's execution describes, unless it states another. + * + * One entry, so the rendered catalog carries a marker a prompt assertion can + * look for without depending on the whole run profile being assembled. + */ +export const CASE_CATALOG: SyntaxSymbols = { + version: 2, + categories: [ + { kind: "structural", entries: [] }, + { + kind: "built-in", + entries: [ + { + kind: "component", + name: "File", + origin: { kind: "registered", origin: "@executablemd/core", reserved: false }, + sourceKind: "registered", + inspectability: "complete", + forms: ["self-closing", "paired"], + props: { type: "object", properties: {}, additionalProperties: false }, + captures: [], + returnMode: "text", + returns: { type: "string" }, + }, + ], + }, + { kind: "user-provided", entries: [] }, + ], +}; diff --git a/packages/cli/tests/support/pull-request-server.ts b/packages/cli/tests/support/pull-request-server.ts index 86e8157e4..af3a3c2c2 100644 --- a/packages/cli/tests/support/pull-request-server.ts +++ b/packages/cli/tests/support/pull-request-server.ts @@ -9,7 +9,8 @@ */ import { createServer, type IncomingMessage, type ServerResponse } from "node:http"; -import { ensure, type Operation, resource, until } from "effection"; +import { each, ensure, type Operation, resource, until, useScope } from "effection"; +import { fromReadable } from "@effectionx/node"; /** One request this server received, as an assertion reads it. */ export interface ServedRequest { @@ -63,9 +64,18 @@ export function usePullRequestServer(options: ServerOptions = {}): Operation { - incoming.resume(); - incoming.on("end", () => { + scope.run(function* () { + for (const _chunk of yield* each(fromReadable(incoming))) { + yield* each.next(); + } + const url = new URL(incoming.url ?? "/", "http://127.0.0.1"); const authorization = typeof incoming.headers["authorization"] === "string" diff --git a/packages/cli/tests/support/run-markdown-tier.ts b/packages/cli/tests/support/run-markdown-tier.ts index 123f847f2..b973aaa39 100644 --- a/packages/cli/tests/support/run-markdown-tier.ts +++ b/packages/cli/tests/support/run-markdown-tier.ts @@ -31,7 +31,6 @@ import { testHarnessInstallation, useTesting } from "@executablemd/testing"; import type { TestResult } from "@executablemd/testing"; import { cliBase, cliCommand, cliRuntime } from "@executablemd/test-support/launch"; import { planComponentDeclaration } from "../../src/plan-component.ts"; -import { renderSyntaxMarkdown, syntaxCatalog } from "../../src/syntax.ts"; import { testingExecutionHost } from "../../src/testing-host.ts"; import { useBunService } from "../../src/bun-service.ts"; import { useDenoService } from "../../src/deno-service.ts"; @@ -105,9 +104,6 @@ export function runMarkdownTier(document: string): Operation { ? {} : { observeAuthorship: request.observeAuthorship }), installElicitation: request.installElicitation, - *catalog(): Operation { - return renderSyntaxMarkdown(yield* syntaxCatalog(["components", "."])); - }, }), // This harness runs Markdown tiers, not repository work: a child that // asked for a checkout is told there is no provider. diff --git a/packages/cli/tests/syntax-cli.test.ts b/packages/cli/tests/syntax-cli.test.ts index d47fff3f1..03a493a60 100644 --- a/packages/cli/tests/syntax-cli.test.ts +++ b/packages/cli/tests/syntax-cli.test.ts @@ -6,7 +6,7 @@ * about assembly, not about argv. The grammar, failure and delivery rows shell * out, so exit status, stdout and stderr are the ones an operator sees. * - * Catalog behavior itself is Tier SY's; nothing here re-proves selection. + * Symbol behavior itself is Tier SY's; nothing here re-proves selection. */ import { describe, it } from "@executablemd/test-support/bdd"; @@ -21,8 +21,8 @@ import { platform, tmpdir } from "node:os"; import { dirname, join } from "node:path"; import { API } from "@executablemd/runtime"; import { CORE_COMPONENT_NAMES } from "@executablemd/core"; -import type { PropsSchema, SyntaxCatalog } from "@executablemd/core"; -import { renderSyntaxJson, renderSyntaxMarkdown, syntaxCatalog } from "../src/syntax.ts"; +import type { PropsSchema, SyntaxSymbols } from "@executablemd/core"; +import { renderSyntaxJson, renderSyntaxMarkdown, syntaxSymbols } from "../src/syntax.ts"; function* useWorkspace( files: Record, @@ -59,27 +59,27 @@ const WORKSPACE: Record = { "second/Only.md": "only in the second include\n", }; -function parseCatalog(text: string): SyntaxCatalog { +function parseSymbols(text: string): SyntaxSymbols { const parsed: unknown = JSON.parse(text); if (typeof parsed !== "object" || parsed === null) { - throw new Error("the catalog is not an object"); + throw new Error("the symbols are not an object"); } const version = Reflect.get(parsed, "version"); const categories = Reflect.get(parsed, "categories"); - if (version !== 1 || !Array.isArray(categories) || categories.length !== 3) { - throw new Error("the catalog is not the version-1 shape"); + if (version !== 2 || !Array.isArray(categories) || categories.length !== 3) { + throw new Error("the symbols are not the version-2 shape"); } return { version, categories: readCategories(categories) }; } -function readCategories(categories: unknown[]): SyntaxCatalog["categories"] { +function readCategories(categories: unknown[]): SyntaxSymbols["categories"] { const [structural, builtIn, userProvided] = categories.map(readCategory); if ( structural?.kind !== "structural" || builtIn?.kind !== "built-in" || userProvided?.kind !== "user-provided" ) { - throw new Error("the catalog categories are not the fixed tuple"); + throw new Error("the symbol categories are not the fixed tuple"); } return [ { kind: "structural", entries: structural.entries }, @@ -91,11 +91,11 @@ function readCategories(categories: unknown[]): SyntaxCatalog["categories"] { // deno-lint-ignore no-explicit-any function readCategory(category: unknown): { kind: unknown; entries: any[] } { if (typeof category !== "object" || category === null) { - throw new Error("a catalog category is not an object"); + throw new Error("a symbol category is not an object"); } const entries = Reflect.get(category, "entries"); if (!Array.isArray(entries)) { - throw new Error("a catalog category has no entries"); + throw new Error("a symbol category has no entries"); } return { kind: Reflect.get(category, "kind"), entries }; } @@ -105,9 +105,9 @@ function names(entries: readonly { name: string }[]): string[] { } /** One built-in entry carrying `props`, for a renderer row that supplies its own. */ -function catalogWith(props: PropsSchema): SyntaxCatalog { +function symbolsWith(props: PropsSchema): SyntaxSymbols { return { - version: 1, + version: 2, categories: [ { kind: "structural", entries: [] }, { @@ -151,7 +151,7 @@ const COMPOSITION_NAMES = [ describe("Tier SX — the run profile the command describes", () => { it("SX1: names core, Agent, testing and web defaults, and ", function* () { - const catalog = yield* syntaxCatalog([]); + const catalog = yield* syntaxSymbols([]); const builtIn = names(catalog.categories[1].entries); for (const name of CORE_COMPONENT_NAMES) { @@ -175,8 +175,37 @@ describe("Tier SX — the run profile the command describes", () => { expect(catalog.categories[2].entries).toEqual([]); }); + it("SX1b: describes once, as the component canonical core owns", function* () { + const catalog = yield* syntaxSymbols([]); + const everywhere = catalog.categories.flatMap((category) => + category.entries.filter((entry) => entry.name === "Syntax"), + ); + // Once, and in the built-in category: a name a document cannot take back is + // not user-provided, and two entries would mean two tiers answered. + expect(everywhere).toHaveLength(1); + expect(names(catalog.categories[1].entries)).toContain("Syntax"); + + const [entry] = everywhere; + if (entry === undefined || entry.kind !== "component" || entry.inspectability !== "complete") { + throw new Error("the symbols describe without a contract"); + } + expect(entry.origin).toEqual({ kind: "protected", origin: "@executablemd/core" }); + expect(entry.sourceKind).toBe("protected"); + expect(entry.forms).toEqual(["self-closing"]); + // One optional prop, closed: `names` selects documentation. + expect(Object.keys((entry.props.properties ?? {}) as object)).toEqual(["names"]); + expect(entry.props.additionalProperties).toBe(false); + expect(entry.captures).toEqual([]); + expect(entry.returnMode).toBe("text"); + expect(entry.description).toBe( + "Inspect available components and control-flow constructs. `` lists the " + + 'symbols available here; `` renders selected documentation.', + ); + expect(entry.as).toBe("Optional. Captures the rendered text instead of emitting it."); + }); + it("ORC1: names all thirteen repository-composition components, with contracts", function* () { - const catalog = yield* syntaxCatalog([]); + const catalog = yield* syntaxSymbols([]); const entries = catalog.categories[1].entries; const builtIn = names(entries); @@ -219,7 +248,7 @@ describe("Tier SX — the run profile the command describes", () => { ].join("\n"), }, function* (dir) { - const catalog = yield* syntaxCatalog([dir]); + const catalog = yield* syntaxSymbols([dir]); const provided = catalog.categories[2].entries.find((entry) => entry.name === "Worktree"); expect(provided).toBeDefined(); expect(names(catalog.categories[1].entries)).not.toContain("Worktree"); @@ -228,7 +257,7 @@ describe("Tier SX — the run profile the command describes", () => { }); it("SX2: documents every complete built-in in the profile", function* () { - const catalog = yield* syntaxCatalog([]); + const catalog = yield* syntaxSymbols([]); const undocumented = catalog.categories[1].entries.filter( (entry) => entry.description === undefined || entry.description.trim().length === 0, ); @@ -237,7 +266,7 @@ describe("Tier SX — the run profile the command describes", () => { }); it("SX2b: reports the testing contracts as they actually are", function* () { - const catalog = yield* syntaxCatalog([]); + const catalog = yield* syntaxSymbols([]); const entries = catalog.categories[1].entries; const throws = entries.find((entry) => entry.name === "AssertThrows"); @@ -267,7 +296,7 @@ describe("Tier SX — the run profile the command describes", () => { }); it("SX3: describes without minting an execution claimant", function* () { - const catalog = yield* syntaxCatalog([]); + const catalog = yield* syntaxSymbols([]); const session = catalog.categories[1].entries.find((entry) => entry.name === "Session"); // A registered default, described from the declaration a host makes: had @@ -284,7 +313,7 @@ describe("Tier SX — the run profile the command describes", () => { describe("Tier SX — the renderers take a value", () => { it("SX4: renders both formats without reaching the filesystem", function* () { - const catalog = yield* syntaxCatalog([]); + const catalog = yield* syntaxSymbols([]); const rendered = yield* scoped(function* () { yield* API.Fs.around({ @@ -317,7 +346,7 @@ describe("Tier SX — the renderers take a value", () => { it("SX4b: escapes a table cell that would otherwise shift the columns", function* () { const markdown = renderSyntaxMarkdown( - catalogWith({ + symbolsWith({ type: "object", properties: { "left|right": { type: "string", description: "a | in the description, and\na break" }, @@ -339,15 +368,15 @@ describe("Tier SX — the renderers take a value", () => { } }); - it("SX5: renders the same bytes twice from the same catalog", function* () { - const catalog = yield* syntaxCatalog([]); + it("SX5: renders the same bytes twice from the same symbols", function* () { + const catalog = yield* syntaxSymbols([]); expect(renderSyntaxMarkdown(catalog)).toBe(renderSyntaxMarkdown(catalog)); expect(renderSyntaxJson(catalog)).toBe(renderSyntaxJson(catalog)); }); it("SX6: renders the fixed category headings in order", function* () { - const markdown = renderSyntaxMarkdown(yield* syntaxCatalog([])); + const markdown = renderSyntaxMarkdown(yield* syntaxSymbols([])); const headings = [ "## Built-in structural syntax", "## Built-in components", @@ -367,7 +396,7 @@ describe("Tier SX — the command line", { sanitizeOps: false, sanitizeResources ["syntax", "--json", "--include", "first", "--include", "second"], { cwd }, ).expect(); - const forwardCatalog = parseCatalog(forward.stdout); + const forwardCatalog = parseSymbols(forward.stdout); expect(names(forwardCatalog.categories[2].entries)).toEqual(["Only", "Shared"]); const shared = forwardCatalog.categories[2].entries.find((one) => one.name === "Shared"); expect(shared?.origin).toEqual({ kind: "repository", path: "first/Shared.md" }); @@ -379,7 +408,7 @@ describe("Tier SX — the command line", { sanitizeOps: false, sanitizeResources ["syntax", "--json", "--include", "second", "--include", "first"], { cwd }, ).expect(); - const reversedShared = parseCatalog(reversed.stdout).categories[2].entries.find( + const reversedShared = parseSymbols(reversed.stdout).categories[2].entries.find( (one) => one.name === "Shared", ); expect(reversedShared?.origin).toEqual({ kind: "repository", path: "second/Shared.md" }); @@ -389,13 +418,46 @@ describe("Tier SX — the command line", { sanitizeOps: false, sanitizeResources it("SX8: falls back to components and . when no include is written", function* () { yield* useWorkspace(WORKSPACE, function* (cwd) { const { stdout } = yield* runCli(["syntax", "--json"], { cwd }).expect(); - const catalog = parseCatalog(stdout); + const catalog = parseSymbols(stdout); expect(names(catalog.categories[2].entries)).toContain("Default"); }); }); - it("SX9: reports an unusable include on stderr, exits 1, and prints no catalog", function* () { + it("SX8b: an ordinary run observes its own profile and includes, and agrees with the command", function* () { + yield* useWorkspace( + { + "components/Local.md": "---\ndescription: the first description.\n---\n\nlocal\n", + "catalog.md": "\n", + }, + function* (cwd) { + // The same site, asked two ways: the command that describes an + // environment, and a document running in it. A run that derived its + // catalog from anything but its own captured inputs would disagree. + const described = yield* runCli(["syntax"], { cwd }).expect(); + const observed = yield* runCli(["run", "catalog.md"], { cwd }).expect(); + expect(observed.stdout.trim()).toBe(described.stdout.trim()); + expect(observed.stdout).toContain("### ``"); + expect(observed.stdout).toContain("the first description."); + // And the run really is describing the run profile it has, not a + // reduced one: `` is declared to every ordinary run. + expect(observed.stdout).toContain("### ``"); + + // A fresh occurrence sees a moved environment. Nothing is cached across + // executions, and the catalog is the working tree's rather than the + // build's. + yield* writeTextFile( + join(cwd, "components/Local.md"), + "---\ndescription: the second description.\n---\n\nlocal\n", + ); + const again = yield* runCli(["run", "catalog.md"], { cwd }).expect(); + expect(again.stdout).toContain("the second description."); + expect(again.stdout).not.toContain("the first description."); + }, + ); + }); + + it("SX9: reports an unusable include on stderr, exits 1, and prints no symbols", function* () { yield* useWorkspace({ components: "not a directory\n" }, function* (cwd) { const { code, stdout, stderr } = yield* runCli(["syntax", "--include", "components"], { cwd, @@ -408,7 +470,56 @@ describe("Tier SX — the command line", { sanitizeOps: false, sanitizeResources }); }); - it("SX10: writes markdown by default and version-1 JSON with --json", function* () { + it("SX16: `xmd syntax Elicit` renders the same text the named component does", function* () { + const named = yield* runCli(["syntax", "Elicit", "--include", "."], { cwd: "." }).expect(); + + // Metadata, then documentation, then availability — the detailed renderer, + // not the compact catalog. + expect(named.stdout).toContain("### ``"); + expect(named.stdout).toContain("Asks a person a structured question"); + expect(named.stdout).toContain("**Available in this evaluation:** yes"); + // Only the one asked for: the compact catalog's other entries are absent. + expect(named.stdout).not.toContain("### ``"); + + // The compact form is untouched by the addition. + const compact = yield* runCli(["syntax", "--include", "."], { cwd: "." }).expect(); + expect(compact.stdout).toContain("## Built-in components"); + expect(compact.stdout).not.toContain("Asks a person a structured question"); + + // An unknown name refuses whole rather than printing a partial answer. + const unknown = yield* runCli(["syntax", "Nonexistent", "--include", "."], { + cwd: ".", + }).join(); + expect(unknown.code).not.toBe(0); + expect(unknown.stdout).toBe(""); + }); + + it("SX17: the command and the component read one index, for every package", function* () { + // `Prompt` is an Agent component: a different registration boundary from + // core's own file, and the one that exposed this. The command assembled the + // profile's contributions while a document's own named form fell back to a + // core-only index, so one product answered the same question two ways. + const command = yield* runCli(["syntax", "Prompt", "--include", "."], { cwd: "." }).expect(); + const document = yield* runCli( + ["run", "-e", '', "--include", "."], + { cwd: "." }, + ).expect(); + + // The whole rendered result, not a phrase from it. Both surfaces render the + // same text through the same renderer; they differ only in the trailing + // newline a rendered document ends with, which is the presentation boundary + // rather than the answer. Comparing substrings would pass just as happily + // if one surface silently dropped the documentation and kept the heading. + expect(document.stdout.trimEnd()).toBe(command.stdout.trimEnd()); + + // And it is a real answer rather than two matching empties. + expect(command.stdout).toContain("### ``"); + expect(command.stdout).toContain("Sends a prompt and renders the reply"); + expect(command.stdout).toContain("**Available in this evaluation:** yes"); + expect(command.stdout.length).toBeGreaterThan(400); + }); + + it("SX10: writes markdown by default and version-2 JSON with --json", function* () { yield* useWorkspace(WORKSPACE, function* (cwd) { const markdown = yield* runCli(["syntax", "--include", "first"], { cwd }).expect(); expect(markdown.stdout).toContain("## Built-in structural syntax"); @@ -416,8 +527,8 @@ describe("Tier SX — the command line", { sanitizeOps: false, sanitizeResources expect(markdown.stdout).toContain("from the first include."); const json = yield* runCli(["syntax", "--json", "--include", "first"], { cwd }).expect(); - const catalog = parseCatalog(json.stdout); - expect(catalog.version).toBe(1); + const catalog = parseSymbols(json.stdout); + expect(catalog.version).toBe(2); expect(names(catalog.categories[2].entries)).toEqual(["Shared"]); }); }); @@ -447,13 +558,13 @@ describe("Tier SX — the command line", { sanitizeOps: false, sanitizeResources expect(stderr).toBe(""); expect(code).toBe(0); - const catalog = parseCatalog(stdout); + const catalog = parseSymbols(stdout); expect(names(catalog.categories[2].entries)).toEqual(["Widget"]); }, ); }); - it("SX11: the catalog is inspection, and plan is the command that writes with it", function* () { + it("SX11: the symbols are inspection, and plan is the command that writes with them", function* () { const { stdout } = yield* runCli(["--help"]).expect(); expect(stdout).toContain("syntax"); @@ -497,8 +608,8 @@ describe( const { redirected, piped } = yield* deliveries(["--json"], cwd); expect(piped).toBe(redirected); - const catalog = parseCatalog(piped); - expect(catalog.version).toBe(1); + const catalog = parseSymbols(piped); + expect(catalog.version).toBe(2); expect(names(catalog.categories[2].entries)).toContain("ZBeyondTheBuffer"); expect(piped.lastIndexOf(`"ZBeyondTheBuffer"`)).toBeGreaterThan(PIPE_BUFFER); }); @@ -515,7 +626,7 @@ describe( expect(stdout.length).toBe(100); expect(stderr).toContain("xmd-exit=1"); - expect(stderr).toContain("stdout did not accept the whole catalog"); + expect(stderr).toContain("stdout did not accept the whole output"); // The broken pipe is reported, not raised: an unhandled write failure // ends the process with one of these instead. expect(stderr).not.toContain("Unhandled 'error' event"); diff --git a/packages/cli/tests/testing-execution-host.test.ts b/packages/cli/tests/testing-execution-host.test.ts index e56f8e3f5..4c8992804 100644 --- a/packages/cli/tests/testing-execution-host.test.ts +++ b/packages/cli/tests/testing-execution-host.test.ts @@ -322,6 +322,92 @@ describe("nested execution under the production run host", () => { expect(bare.stdout + bare.stderr).toContain("1 of 1 tests failed"); }); + /** + * A child is a root, so the vocabulary it observes is its own. + * + * `` describes the site it is written at, and a child's site is the + * child run profile with the child's includes — not the outer test's. The + * positive control is the same document run as the outer root: the name the + * child cannot see is one the outer include path really does supply, so the + * absence below is isolation rather than a component nobody has. + */ + it("gives a nested run child its own catalog rather than the outer root's", function* () { + const project = yield* useProject({ + "elsewhere/Greeting.md": doc("hello"), + "catalog.md": doc(""), + "README.md": doc( + '', + '\\n"} as="child">', + '', + "", + '', + "`/} />", + "", + "", + ), + }); + + // The child runs with no include of its own, so the outer command's + // `--include elsewhere` does not put `` in the child's catalog. + const nested = yield* runCli(["test", "README.md"], { cwd: project }).join(); + expect(nested.code).toBe(0); + + // The control: with that include configured, an ordinary root at the same + // site does observe the name, so the child's catalog above was narrower + // rather than empty. + const outer = yield* runCli(["run", "catalog.md", "--include", "elsewhere"], { + cwd: project, + }).join(); + expect(outer.code).toBe(0); + expect(outer.stdout).toContain("### ``"); + }); + + it("gives a nested run child the documentation for what it registers", function* () { + // `` is the web package's, which the run profile registers — so a + // child that is the run profile can run it, and must be able to explain it. + // Prose from `packages/web/src/components.md`, not the catalog description + // an entry already carries: a child whose index held core's contributions + // alone would render the entry and then say it is undocumented. + const project = yield* useProject({ + "elsewhere/Greeting.md": doc("hello"), + // Not `webform.md`: this filesystem is case-insensitive, so a document + // of that name is found as the repository component `WebForm` and + // shadows the registration — the fallback would then be correct, and the + // case would be measuring the wrong thing. + "lookup.md": doc(''), + "README.md": doc( + '', + '\\n"} as="child">', + '', + "", + '', + // Unique to the long-form documentation. + '', + // And never the fallback, which is what this regression is about. + "", + // The child's own isolation is unchanged: the outer include still does + // not reach it. + "`/} />", + "", + "", + ), + }); + + const nested = yield* runCli(["test", "README.md", "--include", "elsewhere"], { + cwd: project, + }).join(); + expect(nested.code).toBe(0); + + // The ordinary-run control: both profiles read the same owning package's + // documentation, so the child above is not a special case that happens to + // agree. + const ordinary = yield* runCli(["run", "lookup.md"], { cwd: project }).join(); + expect(ordinary.code).toBe(0); + expect(ordinary.stdout).toContain("### ``"); + expect(ordinary.stdout).toContain("anything awkward to type at a"); + expect(ordinary.stdout).not.toContain("No long-form documentation"); + }); + it("refuses outside a canonical ", function* () { const project = yield* useProject({ "child.md": doc("child"), @@ -658,10 +744,6 @@ describe("deterministic dependencies declared for a nested run", () => { ? {} : { observeAuthorship: request.observeAuthorship }), installElicitation: request.installElicitation, - // deno-lint-ignore require-yield - *catalog(): Operation { - return ""; - }, }), *observePlanAuthorship(observation): Operation { observed.resolve(observation); diff --git a/packages/cli/tests/verbose-component.test.ts b/packages/cli/tests/verbose-component.test.ts index 7e7e8ecc9..69ac5ac7a 100644 --- a/packages/cli/tests/verbose-component.test.ts +++ b/packages/cli/tests/verbose-component.test.ts @@ -1,12 +1,12 @@ import { describe, it } from "@executablemd/test-support/bdd"; import { expect } from "@executablemd/test-support/expect"; -import { syntaxCatalog } from "../src/syntax.ts"; +import { syntaxSymbols } from "../src/syntax.ts"; describe("Tier VB — ", () => { // Inspection reads the declaration's metadata. It invokes nothing, so no // verbosity — the command line's or a component's — takes part in it. it("VB5: the run syntax catalog describes it", function* () { - const catalog = yield* syntaxCatalog([]); + const catalog = yield* syntaxSymbols([]); const verbose = catalog.categories[1].entries.find((entry) => entry.name === "Verbose"); expect(verbose?.origin).toEqual({ diff --git a/packages/cli/tests/workflow-agent.test.ts b/packages/cli/tests/workflow-agent.test.ts index fed965e3e..f897b41ba 100644 --- a/packages/cli/tests/workflow-agent.test.ts +++ b/packages/cli/tests/workflow-agent.test.ts @@ -28,7 +28,7 @@ import { API, useHostFiles } from "@executablemd/runtime"; import { DatabaseSync } from "node:sqlite"; import type { WorkflowRunDatabase } from "@executablemd/workflow"; import { - evaluationComponents, + evaluationProfile, transactAgentPromptCheckpoints, withWorkflowWorkspace, workflowRunPath, @@ -256,15 +256,13 @@ function runFixture( stream: database.journal, ...(options.props === undefined ? {} : { props: options.props }), }, - // Both words this document writes name durable work after their - // own invocations, so the host declares them to the execution and - // canonical execution builds each from the claimant it minted. + // The Agent words name durable work after their own invocations, + // so the host declares them to the execution. `` is + // canonical core's: what the host states for it is the ceiling. [ { - components: [ - ...evaluationComponents(database, options.evaluation ?? {}), - ...agentIdentityComponents(), - ], + components: [...agentIdentityComponents()], + evaluation: yield* evaluationProfile(database, options.evaluation ?? {}), }, ], ), @@ -719,7 +717,10 @@ describe("Tier WAL — the workflow Agent observation loop", () => { expect(recorded).toHaveLength(1); const policy = JSON.stringify(recorded[0]); expect(policy).toContain("File:write"); - expect(policy).toContain("@executablemd/workflow/composition/dir-v2#Dir"); + // The identity is a structural record, so the origin, the key and the + // revision are read as themselves rather than as one assembled spelling. + expect(policy).toContain('"origin":"@executablemd/workflow/composition"'); + expect(policy).toContain('"key":"Dir","revision":"3"'); expect(policy).toContain('"allow":["write"]'); // And the change is in the run's own Workspace, where an ordinary read // beneath the fragment's own directory finds it. Anchored on the diff --git a/packages/cli/tests/workflow-fetch.test.ts b/packages/cli/tests/workflow-fetch.test.ts index 1780035b1..2218f0338 100644 --- a/packages/cli/tests/workflow-fetch.test.ts +++ b/packages/cli/tests/workflow-fetch.test.ts @@ -1,5 +1,5 @@ /** - * Tier FE — what a workflow run retains of a `` (spec §6.18). + * Tier FT — what a workflow run retains of a `` (spec §6.18). * * A workflow's journal is the run's own, and this is where the same event that * a diagnostic trace holds has to survive losing the host. The run is killed @@ -55,7 +55,23 @@ function useLoopback(): Operation { }); const listening = withResolvers(); - server.on("error", (error: Error) => listening.reject(error)); + // Removed with the resource rather than after the first error: a listening + // server outlives its bind, and a handler left behind would still be + // holding a rejected resolver when the next test binds its own. + // + // Established before the handler exists, because `yield* ensure(...)` is + // itself a suspension: an owner halted while it registers unwinds with no + // cleanup at all, so nothing may be attached until it has completed. + let onError: ((error: Error) => void) | undefined; + + yield* ensure(() => { + if (onError) { + server.off("error", onError); + } + }); + + onError = (error: Error) => listening.reject(error); + server.on("error", onError); server.listen(0, "127.0.0.1", () => listening.resolve()); yield* listening.operation; @@ -127,7 +143,7 @@ function useFixture(origin: string, body: (fixture: Fixture) => Operation) yield* git(fixture.repository, ["init", "-q", "--initial-branch=main", "."]); yield* git(fixture.repository, ["config", "user.email", "tier-fe@example.test"]); - yield* git(fixture.repository, ["config", "user.name", "Tier FE"]); + yield* git(fixture.repository, ["config", "user.name", "Tier FT"]); yield* git(fixture.repository, ["add", "-A"]); yield* git(fixture.repository, [ "-c", @@ -185,8 +201,8 @@ function* rootCloses(path: string): Operation { .length; } -describe("Tier FE — a workflow run's Fetch", () => { - it("FEW1: commits one response, and a resume restores it without asking again", function* () { +describe("Tier FT — a workflow run's Fetch", () => { + it("FTW1: commits one response, and a resume restores it without asking again", function* () { const server = yield* useLoopback(); yield* useFixture(server.origin, function* (fixture) { diff --git a/packages/cli/tests/workflow-installation.test.ts b/packages/cli/tests/workflow-installation.test.ts index 0263f6e7e..2ff961be5 100644 --- a/packages/cli/tests/workflow-installation.test.ts +++ b/packages/cli/tests/workflow-installation.test.ts @@ -287,28 +287,40 @@ describe("Tier WFI — what a run hands to canonical core", () => { ); expect(preparing.length).toEqual(1); expect(preparing[0]?.admissions?.length).toEqual(1); - // Every installation this run was given is one of two things, and neither - // is a second execution: one `executeInstalled()`, not one per phase. - // A run-contract installation carries its admission; the declaration of the - // components that name durable work carries components and nothing else. + // Every installation this run was given is one of three things, and none of + // them is a second execution: one `executeInstalled()`, not one per phase. + // A run-contract installation carries its admission; a bundle carries its + // own admission and no preparation; and the fragment-evaluation profile + // carries a ceiling and no admission at all, because stating what a + // generated fragment may do is not a claim about this run's history. + const profiles = (execution?.installations ?? []).filter( + (candidate) => candidate.evaluation !== undefined, + ); for (const candidate of execution?.installations ?? []) { - if (candidate.components === undefined) { - expect(candidate.admissions?.length).toEqual(1); + if (candidate.evaluation !== undefined) { + expect(candidate.admissions).toBe(undefined); + expect(candidate.prepare).toBe(undefined); + expect(candidate.components).toBe(undefined); continue; } - expect(candidate.admissions).toBe(undefined); - expect(candidate.prepare).toBe(undefined); + expect(candidate.admissions?.length).toEqual(1); } - // And there is exactly one declaration, naming ``: a live or - // partial run has that component because this run told the execution about - // it, and nothing else declares one. - const declared = (execution?.installations ?? []).filter( - (candidate) => candidate.components !== undefined, + + // The ceiling is stated exactly once, and it is a real one: a run that + // installed no profile, or an empty one, would leave `` with + // nothing to narrow and is the regression this asserts against rather than + // merely tolerating a third shape. + expect(profiles).toHaveLength(1); + expect((profiles[0]?.evaluation?.read ?? []).length).toBeGreaterThan(0); + + // And nothing declares ``. It is canonical core's own protected + // component, so a run states the profile it narrows from and never the + // component — a declaration here would be the workflow answering for a name + // core owns. + const declared = (execution?.installations ?? []).flatMap( + (candidate) => candidate.components ?? [], ); - expect(declared).toHaveLength(1); - expect((declared[0]?.components ?? []).map((component) => component.name)).toEqual([ - "Evaluate", - ]); + expect(declared.map((component) => component.name)).not.toContain("Evaluate"); }); it("WFI2: a completed run is given no Workspace to attach", function* () { diff --git a/packages/cli/tests/workflow-suspension.test.ts b/packages/cli/tests/workflow-suspension.test.ts index 2b1af9d61..3b57d8170 100644 --- a/packages/cli/tests/workflow-suspension.test.ts +++ b/packages/cli/tests/workflow-suspension.test.ts @@ -1542,8 +1542,14 @@ the retained note const suspended = retained(path); expect(suspended.status).toBe("suspended"); - // The admission carries the versioned identity. - expect(readRecords(path).some((record) => record.includes("dir-v2#Dir"))).toBe(true); + // The admission carries the versioned identity. It is a structural record + // rather than a spelling, and the version is the `revision` member — so + // that is what this reads, and what the refusal below moves. + expect( + readRecords(path).some( + (record) => record.includes(`"key":"Dir"`) && record.includes(`"revision":"3"`), + ), + ).toBe(true); // And the generated `` really executed before the run suspended: the // retained file effects hold an ensure for `/generated` ahead of the nested @@ -1618,12 +1624,12 @@ the retained note // The run becomes one admitted under the former identity. Nothing else // changes — same source, roots and selection — so the identity is the only // thing the resume can disagree about. - rewriteRecords(path, (record) => - record.replaceAll( - "@executablemd/workflow/composition/dir-v2#Dir", - "@executablemd/workflow/composition#Dir", - ), - ); + // + // The version is the `revision` member, and core's own entries state `2`, + // so `"revision":"3"` names the Dir entry and nothing else. Moving it back + // is exactly "admitted under the earlier revision", which is the grant the + // former placement-only Dir had. + rewriteRecords(path, (record) => record.replaceAll(`"revision":"3"`, `"revision":"2"`)); const beforeResume = counts(path); const rootsBeforeResume = workspaceRootState(path); @@ -1641,7 +1647,11 @@ the retained note expect(workspaceRootState(path)).toEqual(rootsBeforeResume); // Nothing after the wait reached the document either. expect(rendered).toEqual([]); - // And the refusal does not publish which identity moved. - expect(resumed.written.err.join("\n")).not.toContain("dir-v2"); + // And the refusal does not publish which identity moved. The identity is + // four members now, so this reads the two a refusal could carry: the origin + // that owns the implementation, and the member the rewrite actually moved. + const reported = resumed.written.err.join("\n"); + expect(reported).not.toContain("@executablemd/workflow/composition"); + expect(reported).not.toContain("revision"); }); }); diff --git a/packages/core/host.ts b/packages/core/host.ts index 3d1dabd02..a247c8565 100644 --- a/packages/core/host.ts +++ b/packages/core/host.ts @@ -68,6 +68,91 @@ */ export { executeInstalled } from "./src/execute.ts"; export type { ExecutionInstallation, JournalAdmission } from "./src/execute.ts"; +/** + * What a trusted host states about generated-fragment evaluation. + * + * Only the input a host writes, and only here, because this is the trusted + * surface `ExecutionInstallation` already lives on. What canonical execution + * captures from it — the bound operations and their revocation — is unexported: + * ordinary core publishes no getter for active authority and no way to install + * a provider. + */ +export type { + CapabilityEntry, + ComponentAnswerEntry, + FragmentEntry, + FragmentEvaluationInput, + FragmentForm, + FragmentIdentity, + FragmentWorkspaceAccess, +} from "./src/evaluation-profile.ts"; +/** + * How a trusted host supplies the implementation behind a `component-answer` + * entry. + * + * An installer, not a definition: canonical execution runs it during profile + * capture and hands it a registrar fixed to this installation's origin. The + * registrar composes import middleware, and every invocation of that middleware + * receives its own request — fixed to the name and position it was asked, and + * closed when that invocation ends — which is the only thing that states what + * the provider is returning. Canonical execution then decides whether that is + * what the profile admitted, and resolves the name itself through the ordinary + * import chain. + */ +export type { + ComponentAnswerHandler, + ComponentAnswerInstallation, + ComponentAnswerRegistrar, + ComponentAnswerRequest, +} from "./src/component-answers.ts"; +/** + * The private operations an admitted fragment performs, and the only ones it + * can reach. + * + * A host hands its own provider methods here; canonical capture reads each one + * off once and closes core's own fragment bodies over the bound result. There + * is no ordinary-core export for these, no getter for a live one, and no way to + * install one from a document — see `src/fragment-capabilities.ts`. + */ +export type { + FragmentCapability, + FragmentFetchAccess, + FragmentFileAccess, + FragmentPath, + FragmentWrite, +} from "./src/fragment-capabilities.ts"; +/** + * The response shape a fragment transport answers with, and the detaching a + * host needs to build one. + * + * The same record an authored `` retains, through the same code, so a + * fragment's observation and a document's are the same shape and a continuation + * restores either. + */ +export type { FetchResponseRecord } from "./src/fetch-response.ts"; +export { detachHeaders, detachStatus } from "./src/fetch-response.ts"; +/** + * Core's own entries, for the hosts that admit them. + * + * Constructors rather than a table a host assembles from `CORE_REGISTRY`, + * because each of them states a constraint the registry does not hold: which + * spelling of `` is being admitted, and that admitting `` requires + * the exact requests it may perform. + */ +export { + directoryEntry, + EvaluationProfileError, + fetchEntry, + fileDeleteEntry, + fileReadEntry, + fileWriteEntry, +} from "./src/evaluation-profile.ts"; +/** + * The symbols a host's profile describes, when they are not the ones the + * execution would derive from its own captured inputs — see + * `src/syntax-reference.ts`. + */ +export type { SyntaxSymbolsProvider } from "./src/syntax-reference.ts"; export type { DurablePreparation } from "./src/document-request.ts"; /** @@ -138,6 +223,7 @@ export type { GeneratedObservationValue, GeneratedRequest, GeneratedXmdRequest, + RetainedFragmentIdentity, } from "./src/generated-xmd.ts"; /** diff --git a/packages/core/mod.ts b/packages/core/mod.ts index d31eb6d15..3567808e5 100644 --- a/packages/core/mod.ts +++ b/packages/core/mod.ts @@ -210,9 +210,51 @@ export type { InspectSyntaxOptions, OriginOnlyComponentSyntaxEntry, StructuralSyntaxEntry, - SyntaxCatalog, + SyntaxSymbols, } from "./src/inspect.ts"; export { ComponentIncludeError } from "./src/components/candidates.ts"; +/** + * The catalog as Markdown, so `xmd syntax` and canonical `` print the + * same bytes for the same site rather than two renderings that agree by hand. + */ +export { renderSelectedDocumentation, renderSyntaxMarkdown } from "./src/syntax-markdown.ts"; +export type { SelectedEntry } from "./src/syntax-markdown.ts"; +/** + * The documentation index and the selection that reads it (#678). + * + * Exported because `xmd syntax Elicit` and `` must be the same + * lookup rather than two that agree by hand: the command reaches the index and + * the selection core's own component reaches. + */ +export { + agentDocumentation, + documentationIndexFor, + packageDocumentation, +} from "./src/component-documentation.ts"; +export type { + DocumentationContribution, + DocumentationReader, +} from "./src/component-documentation.ts"; +/** + * How a package's bootstrap contributes the documentation for what it registers. + * + * Exported because the contract is additive and contextual: a package installs + * its registrations and its documentation in one call, and canonical execution + * collects whatever the host bootstrapped. A host-maintained list of every + * package's documentation would be a second list beside the registrations, and + * two lists drift. + */ +export { + capturedDocumentation, + contributeDocumentation, + Documentation, +} from "./src/documentation-api.ts"; +export type { DocumentationApi } from "./src/documentation-api.ts"; +export { select as selectDocumented } from "./src/syntax-reference.ts"; +export { NO_DOCUMENTATION, UnknownComponentError } from "./src/documentation-index.ts"; +export type { DocumentationIndex } from "./src/documentation-index.ts"; +export { PROTECTED_COMPONENT_NAMES, ProtectedComponentError } from "./src/components/protected.ts"; +export { SYNTAX_COMPONENT } from "./src/components/Syntax.ts"; // Document validation — one supplied document read as authored program // structure, with nothing in it executed. export { @@ -274,6 +316,7 @@ export { AGENT_REGISTRATIONS, agentIdentityComponents, installAgentComponents, + useAgentComponents, } from "./src/agent/components.ts"; export type { AgentComponentsOptions } from "./src/agent/components.ts"; export { Agent } from "./src/agent/agent-api.ts"; diff --git a/packages/core/package.json b/packages/core/package.json index dd131db7c..8dadac530 100644 --- a/packages/core/package.json +++ b/packages/core/package.json @@ -13,7 +13,7 @@ "@effectionx/fetch": "0.2.1", "@effectionx/fs": "0.3.0", "@effectionx/middleware": "0.1.1", - "@effectionx/node": "0.2.4", + "@effectionx/node": "0.2.5", "@effectionx/process": "0.8.1", "@effectionx/scope-eval": "0.1.3", "@effectionx/stream-helpers": "0.8.3", diff --git a/packages/core/src/agent/components.md b/packages/core/src/agent/components.md new file mode 100644 index 000000000..e3f623277 --- /dev/null +++ b/packages/core/src/agent/components.md @@ -0,0 +1,127 @@ +Long-form documentation for the Agent components canonical core registers. + +These are the components a document uses to talk to a coding agent: which agent +and session to use, how to prompt it, how to answer the permission requests it +makes, and how to hand it the terminal. They register under +`@executablemd/core`, beside the components in `../components/components.md`, +and are documented here because this is the registration boundary they belong +to. + +Most of them are *regions*: they establish something for the content inside them +and restore what was there on the way out. That is what lets one document use two +agents, or grant broad permission for one narrow step without granting it +everywhere. + +## AgentProvider + +Sets the agent provider for its content. + +```mdx + +… + +``` + +Applies to everything inside. `defaultAgent` names the agent to use when a +`` or `` inside does not name one, and `timeout` bounds each +prompt rather than the region as a whole. + +An unknown provider fails **before** the content runs, so a document that names a +provider this host does not have stops rather than doing half its work and then +discovering it cannot finish. + +## Agent + +Chooses the agent for the prompts and launches inside it. + +```mdx + +Summarise the release notes. + +``` + +A region, like the provider above it: the content's prompts use this agent, and +what was in force outside is restored afterwards. Use it when one document needs +more than one agent — a fast one for a mechanical pass, a stronger one for the +judgement call. + +## Session + +Sets the default session for every prompt in its content. + +```mdx + +What changed since the last tag? +Which of those need release notes? + +``` + +Prompts in one session share the agent's context, so a later prompt can refer to +what an earlier one established. Without a session each prompt stands alone, +which is what you want for independent questions and not what you want for a +conversation. + +The session is durable: a run that resumes rejoins the session it was using +rather than starting a fresh one and losing the context the document built. + +## Session.Launch + +Launches a coding agent with prepared context, in the terminal. + +```mdx + +Here is the failing test and what I have tried. + +``` + +The content becomes the agent's starting context. The agent's own interface then +takes the terminal — this is the interactive agent, not a prompt-and-reply — and +the document continues when you are done with it. + +Reach for it when the work needs a person and an agent together, and for +prepared context that would be tedious to type. + +## Prompt + +Sends a prompt and renders the reply. + +```mdx +Summarise these release notes in three bullets. +``` + +The paired form sends its content. The reply is rendered where the element is +written, or bound with `as`. Props naming an agent, session or timeout override +the surrounding scope for this one prompt. + +A failed prompt renders what it got and the document continues, because a +partial answer is usually more useful than none. `throwOnError` stops the +document instead, for a prompt whose answer everything after it depends on. + +## ApproveAll + +Approves every permission request its content makes. + +```mdx + +Fix the failing test. + +``` + +A region, and deliberately a narrow one: it is how a document says *this step is +allowed to act without asking*, for exactly this step. Wrapping a whole document +in it grants far more than any single step needed, so wrap the step. + +## AskPermission + +Puts every permission request its content makes to you. + +```mdx + +Clean up the scratch directory. + +``` + +The opposite region to ``: each request is asked about rather than +granted. With no interactive terminal, or no valid answer, it **denies** — the +safe direction, so a document that runs unattended does not silently do what it +would have asked about. diff --git a/packages/core/src/agent/components.ts b/packages/core/src/agent/components.ts index 1ab783dfc..d4ba07632 100644 --- a/packages/core/src/agent/components.ts +++ b/packages/core/src/agent/components.ts @@ -25,6 +25,8 @@ import { Err, scoped, spawn, withResolvers } from "effection"; import type { Operation, Result } from "effection"; import { Execution } from "../execute.ts"; import { registerComponents } from "../components/registration.ts"; +import { contributeDocumentation } from "../documentation-api.ts"; +import { agentDocumentation } from "../component-documentation.ts"; import { CORE_ORIGIN } from "../components/registry.ts"; import { createReplayStream } from "../replay-stream.ts"; import { documented } from "../components/documentation.ts"; @@ -200,6 +202,19 @@ export const AGENT_REGISTRATIONS: readonly ComponentRegistration[] = [ }, ]; +/** + * The agent vocabulary, as declarations and nothing else. + * + * Registrations and the documentation that describes them, installed together + * so a scope that has one has the other. `xmd syntax` enters exactly this and + * stops: describing an environment installs no provider, no launcher and no + * completion policy. + */ +export function* useAgentComponents(): Operation { + yield* registerComponents(AGENT_REGISTRATIONS); + yield* contributeDocumentation(agentDocumentation); +} + export function* installAgentComponents(options?: AgentComponentsOptions): Operation { if (options?.defaultAgent !== undefined) { const defaultAgent = options.defaultAgent; @@ -210,7 +225,7 @@ export function* installAgentComponents(options?: AgentComponentsOptions): Opera yield* AgentInternal.around({ permissionMode: () => permissionMode }, { at: "min" }); } - yield* registerComponents(AGENT_REGISTRATIONS); + yield* useAgentComponents(); const rootProvider = options?.rootProvider; diff --git a/packages/core/src/component-answers.ts b/packages/core/src/component-answers.ts new file mode 100644 index 000000000..5232c9f84 --- /dev/null +++ b/packages/core/src/component-answers.ts @@ -0,0 +1,135 @@ +/** + * How a trusted host says what is behind a provider-backed fragment entry. + * + * A `component-answer` entry names something the ordinary import chain resolves + * rather than an operation core supplies a body for. The host does not hand + * over the implementation — it states which name, and which structural identity + * a provider must have claimed for that name. This module is the seam a + * provider uses to make that claim. + * + * ## Being asked and answering are two authorities + * + * A provider installs through a *registrar*, and receives no handle it can keep + * and state an answer with. The registrar is the right to be asked; each + * handler invocation then receives its own {@link ComponentAnswerRequest}, + * which is the right to answer *that* asking and nothing else. + * + * The split is the whole point. A stable handle a provider retains can only + * ever prove "some provider installed by this host" — it cannot prove which + * invocation is speaking, so a handler that has already returned, or one that + * lost, could state an answer to an import it is not deciding. A request proves + * it, because it is minted for one invocation, captures the resolution it was + * asked in by object identity, and is closed the moment that invocation ends. + * + * So `claim` takes no name. The name is fixed when the request is minted, from + * what the chain asked, and a handler cannot state an answer for a component it + * was never asked about. + * + * All of it is ordinary closures. There is no shared symbol, module registry or + * context name behind any of these objects, which is what lets a separately + * loaded copy of core hold one without any of those being a way in. + * + * ## Claiming is not authority + * + * Claiming identifies an answer. It authorizes nothing. A fragment still needs + * the host entry, an exact identity match, the right `allow` class, whole- + * fragment preflight and canonical generated import — all of them + * independently. Middleware that answers an import without claiming anything is + * an ordinary replacement and stays exactly as valid as it was; what it cannot + * do is enter an admitted fragment. + */ + +import type { Operation } from "effection"; + +import { Component } from "./component-api.ts"; +import type { + ComponentAnswerRequest, + ImportedDefinition, + ProviderInstallation, +} from "./components/import-authority.ts"; +import type { SourcePosition } from "./types.ts"; + +export type { ComponentAnswerRequest } from "./components/import-authority.ts"; + +/** + * One handler a provider registers, invoked once per import it observes. + * + * The shape ordinary `Component.importComponent` middleware already has, with + * the asked name and position arriving as a request rather than as loose + * arguments. Delegation is unchanged: `yield* next()` reaches the rest of the + * chain, and an outer handler may delegate, look at what came back, and then + * claim its own replacement before returning. + */ +export type ComponentAnswerHandler = ( + request: ComponentAnswerRequest, + next: (name?: string, position?: Readonly) => Operation, +) => Operation; + +/** + * What a provider installs through, in place of anything it could keep. + * + * Registering composes the middleware; canonical execution owns the request + * lifetime around each invocation of it, so a provider cannot hold the lease + * open past its own handler. + */ +export interface ComponentAnswerRegistrar { + /** + * Compose one handler around this execution's import chain. + * + * Spelled `around` because that is what the stable API a provider already + * knows spells it — `Component.around({ importComponent })` — and this + * registers the same middleware through the same composition, with the asked + * name and position arriving as a request instead of as loose arguments. One + * verb for one act keeps a host from having to learn a second vocabulary for + * the thing it was already doing. + */ + around(handler: ComponentAnswerHandler): Operation; +} + +/** + * One provider's installation, run during capture and before ordinary installs. + * + * `origin` is the provider's name for itself and becomes the origin on every + * identity its requests state. `install` registers whatever import middleware + * answers for its names. + */ +export interface ComponentAnswerInstallation { + readonly origin: string; + install(registrar: ComponentAnswerRegistrar): Operation; +} + +/** + * The registrar canonical execution hands one installation. + * + * Each registered handler is wrapped so that one request is minted for the + * invocation and closed in a `finally` when it returns, throws or is cancelled. + * The close is synchronous on purpose: a lease that needed a suspension point + * to end would still be open across one, which is exactly the window a settled + * handler must not have. + */ +export function componentAnswerRegistrar( + installation: ProviderInstallation, +): ComponentAnswerRegistrar { + return { + *around(handler: ComponentAnswerHandler): Operation { + yield* Component.around({ + *importComponent([name, position], next) { + const asked = installation.open(name, position); + try { + return yield* handler( + asked.request, + (forName?: string, forPosition?: Readonly) => + next(forName ?? name, forPosition ?? position), + ); + } finally { + // Synchronous, and in `finally`, so the lease ends the same way the + // invocation did — returned, threw, or was cancelled. Nothing + // yields here: a close that needed a suspension point would leave + // the request open across one. + asked.close(); + } + }, + }); + }, + }; +} diff --git a/packages/core/src/component-documentation.ts b/packages/core/src/component-documentation.ts new file mode 100644 index 000000000..f8d564cf6 --- /dev/null +++ b/packages/core/src/component-documentation.ts @@ -0,0 +1,229 @@ +/** + * The documentation canonical core ships for the components it owns. + * + * The bytes live in `src/components/components.md`, beside the registration + * boundary they document, and are located from this module's own URL. Never + * from the working directory and never through `--include`: a documentation set + * that moved with the caller's directory would describe a different product + * depending on where somebody stood, and a repository file could answer for it. + * + * Each build keeps the asset beside its module — `deno compile --include` + * embeds it at the same relative path, the npm build copies it into the emitted + * tree, and JSR publishes the source file — so the one lookup below is correct + * in all four, and a build that forgets the asset fails loudly on first use + * rather than quietly serving a product with no documentation. + */ + +import { readTextFile } from "@effectionx/fs"; +import type { Operation } from "effection"; + +import { buildDocumentationIndex } from "./documentation-index.ts"; +import type { DocumentationIndex, DocumentationSource } from "./documentation-index.ts"; +import { CORE_ORIGIN, CORE_REGISTRY } from "./components/registry.ts"; +import { PROTECTED_COMPONENT_NAMES } from "./components/protected.ts"; +import { AGENT_REGISTRATIONS, agentIdentityComponents } from "./agent/components.ts"; + +/** Where core's own documentation lives, as a URL beside this module. */ +export function componentDocumentationUrl(): URL { + return new URL("./components/components.md", import.meta.url); +} + +/** + * Where the Agent registrations' documentation lives. + * + * Beside `agent/components.ts`, which is the boundary that registers them. They + * carry core's origin, so they are core's components for the join; what makes + * them a separate file is that they are a separate registration boundary, and + * documentation belongs beside the code it documents. + */ +export function agentDocumentationUrl(): URL { + return new URL("./agent/components.md", import.meta.url); +} + +/** + * The Agent registrations' contribution, for a host that installs them. + * + * Offered rather than assumed: a run that registers no Agent components has no + * Agent components to document, and demanding their documentation would refuse + * an index for a profile that is complete without them. + */ +export function* agentDocumentation( + read: DocumentationReader = packagedAssetReader, +): Operation { + return { + source: yield* readPackagedDocumentation( + agentDocumentationUrl(), + { owner: CORE_ORIGIN, asset: "packages/core/src/agent/components.md" }, + read, + ), + supplies: agentComponentNames(), + }; +} + +/** + * Every component the Agent registration boundary supplies, by name. + * + * Built when asked rather than at module scope. The Agent bootstrap imports this + * module for its own contribution, so a module-scoped set built from + * `AGENT_REGISTRATIONS` would read that array before its own module finished + * evaluating, depending on which of the two was loaded first. + */ +function agentComponentNames(): ReadonlySet { + return new Set([ + ...AGENT_REGISTRATIONS.map((registration) => registration.name), + ...agentIdentityComponents().map((component) => component.name), + ]); +} + +/** Core's documentation source, read from the package rather than the caller. */ +export function* readCoreDocumentation( + read: DocumentationReader = packagedAssetReader, +): Operation { + const url = componentDocumentationUrl(); + try { + return { + owner: CORE_ORIGIN, + asset: "packages/core/src/components/components.md", + // The direct Effection filesystem, not `API.Fs` and not the document + // facing `Files` authority. Both of those are middleware a running + // document can compose around: a repository component, an eval block or + // an installed handler could answer the read and decide what the product's + // own documentation says. This is the engine reading an immutable asset + // out of its own package, so it goes to the filesystem directly, at a URL + // derived from this module — package-relative whatever the working + // directory and search path are. + text: yield* read(url), + }; + } catch (error) { + throw new Error( + `the packaged component documentation is missing from this build (looked in ${url.href})`, + { cause: error }, + ); + } +} + +/** + * A contribution built from the registrations it documents. + * + * The set is derived from the same declarations the package installs, so a + * component added to a boundary demands documentation without anyone having to + * remember to list it here. That is the whole point of deriving it: a + * hand-maintained second list is exactly the thing that goes stale. + */ +export function* packageDocumentation( + url: URL, + named: { owner: string; asset: string }, + supplies: Iterable, + read: DocumentationReader = packagedAssetReader, +): Operation { + return { + source: yield* readPackagedDocumentation(url, named, read), + supplies: new Set(supplies), + }; +} + +/** + * How one execution reads its packaged assets. + * + * Carried by value from where the execution is built, never held in module + * scope. A module-level reader would be one variable shared by every execution + * in the process: two runs in one process would read through each other's, and + * a test that substituted one would change what an unrelated execution is told + * the product says. It is also not a provider, a Context, an installation field + * or a hook — nothing a document, a component or an installed package can reach + * names it at all. + */ +export type DocumentationReader = (url: URL) => Operation; + +/** The reader every ordinary execution uses. */ +export const packagedAssetReader: DocumentationReader = readTextFile; + +/** One packaged documentation asset, read the same guarded way. */ +export function* readPackagedDocumentation( + url: URL, + named: { owner: string; asset: string }, + read: DocumentationReader = packagedAssetReader, +): Operation { + try { + return { ...named, text: yield* read(url) }; + } catch (error) { + throw new Error( + `the packaged component documentation ${named.asset} is missing from this build ` + + `(looked in ${url.href})`, + { cause: error }, + ); + } +} + +/** + * The index every documentation reader shares. + * + * Validated against what the *package* supplies, not against whichever catalog + * is in scope. Those are different sets and conflating them is a real bug: a + * narrowing evaluation boundary carries a catalog holding a handful of admitted + * components, and validating core's own documentation against that would report + * `Elicit` as a component core does not supply. What a heading has to name is a + * component this build actually ships; which of them a given site can select is + * a separate question the selection answers. + */ +export function documentationIndexFor( + /** + * What the packages bootstrapped in this execution supply, core's own + * included. + * + * Collected through the `Documentation` Api at the trusted boundary, before + * any document code exists — core is the terminal, and the Agent, CLI, + * testing, web and workflow bootstraps each append their own file and the set + * of components it must cover. Nothing is added here: a second core entry + * appended by this function would be a source no bootstrap accounted for, and + * the collector's duplicate refusal would never see it. + */ + all: readonly DocumentationContribution[], +): DocumentationIndex { + // Merged per owner, not replaced. One package can have several registration + // boundaries — core registers its own components and its Agent components + // from two files — and keying by owner alone would let the second boundary's + // set hide the first's, so every component in the file that lost would look + // like documentation for something the package does not supply. + const supplied = new Map>(); + for (const one of all) { + const held = supplied.get(one.source.owner) ?? new Set(); + for (const name of one.supplies) { + held.add(name); + } + supplied.set(one.source.owner, held); + } + return buildDocumentationIndex( + all.map((one) => one.source), + (owner) => supplied.get(owner) ?? new Set(), + ); +} + +/** One package's documentation, and the components it must account for. */ +export interface DocumentationContribution { + readonly source: DocumentationSource; + /** + * Every public component this package supplies. + * + * Both halves of the check: a heading outside this set is documentation for + * something the package does not have, and a member of it with no heading is + * a component shipped without documentation. + */ + readonly supplies: ReadonlySet; +} + +/** + * Every component canonical core supplies, by name. + * + * Its registrations and the protected tier together — the two ways core puts a + * component into an execution — read from the same declarations execution reads, + * so this cannot drift from what the package actually ships. + * + * Deliberately not `CORE_COMPONENT_NAMES`, which is the registrations alone. + * Documentation has to account for the protected tier as well, and two sets + * under one name would eventually be used for each other's question. + */ +export const CORE_DOCUMENTED_NAMES: ReadonlySet = new Set([ + ...CORE_REGISTRY.keys(), + ...PROTECTED_COMPONENT_NAMES, +]); diff --git a/packages/core/src/components/Evaluate.ts b/packages/core/src/components/Evaluate.ts new file mode 100644 index 000000000..438bf0858 --- /dev/null +++ b/packages/core/src/components/Evaluate.ts @@ -0,0 +1,479 @@ +/** + * `` — running program text a document did not author. + * + * A generating agent proposes a fragment of Executable Markdown; this is where + * that fragment is admitted and performed. It is public, so any author may + * write it, and canonical core owns what it means. + * + * ## Public, protected, and not authority + * + * Protection settles *which implementation runs* and nothing else. Every + * ceiling this component draws on was stated by a trusted host at the + * installation boundary, before a document existed to ask for one, and + * canonical execution captured it by value there + * (`evaluation-profile.ts`). An execution whose host stated no profile has no + * evaluation at all, and the refusal happens at the invocation rather than at + * installation: a document that never writes `` is not asking for a + * ceiling. + * + * So writing `` grants nothing. It reaches the profile this run's + * host stated, narrowed by `allow`, and never widens it: no prop, binding, + * interpolation, context, contextual API answer, middleware return value, + * registration or generated name adds an identity, a root, a destination or a + * request to what the host already installed. `allow` names an effect *class* + * and selects between two tables that already exist; omitting it asks for + * `read`. + * + * ## Two disjoint input forms + * + * Self-closing takes the program as `text`: a value the document already holds, + * bound from a component that produced it. + * + * ```markdown + * + * ``` + * + * Paired makes the content the producer, and what it renders is the program. + * + * ```markdown + * + * Read the changelog and report the version. + * + * ``` + * + * They are disjoint because they answer the same question twice: a paired + * element that also carried `text` would leave which fragment ran a matter of + * precedence, and precedence is not something an author should have to know to + * predict what executed. + * + * The producer renders under the *narrowed* syntax reference — the vocabulary + * this evaluation admitted, not the vocabulary of the site the element was + * written at. An agent asked to write a fragment is told what a fragment may + * contain, which is the only description of it that is true. + * + * ## What it answers with + * + * Each admitted observation's own value in invocation order, with whatever the + * fragment rendered kept beside them under `output`. A value rather than a + * serialization: how a value becomes text is the document's decision, and it + * has `` to make it with. An admitted write puts nothing here — what it + * did is retained by its own ordinary durable effect, which is the + * authoritative account of it. + * + * It is deliberately not wrapped in `printErrors`. A refused or failed + * evaluation must stop the authored loop unless the document put a recovery + * boundary around it: returning a refusal as observation text would leave the + * agent reasoning from a read that never happened. + */ + +import type { Operation } from "effection"; + +import { getExpansion } from "../expansion.ts"; +import { NO_PROFILE, REVOKED } from "../evaluation-profile.ts"; +import type { CapturedEntry, CapturedProfile } from "../evaluation-profile.ts"; +import { evaluateGeneratedXmd } from "../generated-xmd.ts"; +import type { + GeneratedEffectClass, + GeneratedMutation, + GeneratedObservation, + GeneratedObservationResult, + GeneratedXmdRequest, + RetainedFragmentIdentity, +} from "../generated-xmd.ts"; +import { ComponentInvocationError, invocationForm } from "../invocation-identity.ts"; +import type { + ComponentInvocation, + IdentityClaimant, + ProtectedBody, + ProtectedSite, +} from "../invocation-identity.ts"; +import { admittedSymbols } from "../syntax-admitted.ts"; +import type { SyntaxReference } from "../syntax-reference.ts"; +import type { ProtectedComponent } from "./protected.ts"; +import { CORE_ORIGIN } from "./registry.ts"; +import { documented } from "./documentation.ts"; +import type { FunctionComponentDefinition, Json, PropsSchema } from "../types.ts"; + +/** The public name canonical core claims for the evaluation component. */ +export const EVALUATE_COMPONENT = "Evaluate"; + +/** The classes this component accepts, and the order a selection canonicalizes to. */ +const EFFECT_CLASSES: readonly GeneratedEffectClass[] = ["read", "write"]; + +/** + * The whole schema: the program, the selection, and nothing else accepted. + * + * `allow` narrows the host's already-installed profile and can do nothing else. + * It names no identity, no root, no destination and no request — it says which + * of the tables this host captured before the document existed a fragment may + * draw on, and omitting it asks for `read`. + */ +export const props: PropsSchema = { + type: "object", + properties: { + text: { + type: "string", + description: + "The program to evaluate, for the self-closing form. Written instead of content, " + + "never beside it.", + }, + source: { + type: "string", + description: + "The earlier spelling of `text`, accepted only by a host that admitted it. " + + "New documents write `text`.", + }, + allow: { + type: "array", + minItems: 1, + uniqueItems: true, + items: { enum: [...EFFECT_CLASSES] }, + description: + "Optional. Which effect classes the program may draw on, narrowing what this host " + + "already installed. Omitted asks for `read`.", + }, + }, + additionalProperties: false, +}; + +const UNISSUED_REFUSAL = + " is invoked by canonical core; this is not an invocation the engine issued."; + +const BOTH_FORMS = + " takes the program either as `text` or as its content, and never as both: an " + + "element that states it twice leaves which program ran a question of precedence."; + +const NO_PROGRAM = + " requires the program as a string. An element that states no program " + + "and renders no content has nothing to evaluate."; + +const PAIRED_TEXT = + " renders its content to produce the program, so a paired element does not also " + + "carry `text`."; + +const NO_ALIAS = + " is the earlier spelling, and this host did not admit it. Write the " + + "program as `text`."; + +const ALLOW_SHAPE = " takes `allow` as a non-empty array of effect classes."; + +const ALLOW_UNKNOWN = " admits the effect classes `read` and `write`, and nothing else."; + +const ALLOW_REPEATED = " takes each effect class in `allow` once."; + +const NO_WRITE_TABLE = + ' asks for mutation, and this host installed no write table. A ' + + "selection reaches what the host already stated; it never adds to it."; + +const NO_READ_TABLE = + ' asks for observation, and this host installed no read table.'; + +const NO_PROJECTION = + " renders its content to produce the program, and this invocation has no content to " + + "render."; + +/** + * The declaration canonical core selects for ``. + * + * Both forms, because the two input spellings are two ways of stating the same + * argument. No `returns`, so the value binds by reference under `as`, + * unchecked: rewriting it on the way to the document would change what the + * fragment observed. + */ +export const EVALUATE_PROTECTED: ProtectedComponent = { + name: EVALUATE_COMPONENT, + origin: CORE_ORIGIN, + props, + forms: ["self-closing", "paired"], + ...documented({ + description: 'Evaluate program text. `` runs it.', + as: "Optional. Captures the observations and rendered output instead of emitting them.", + context: null, + }), + build: (claim: IdentityClaimant) => evaluate(claim), +}; + +function evaluate(claim: IdentityClaimant): ProtectedBody { + return function* runEvaluate( + elementProps: Record, + invocation: ComponentInvocation, + site: ProtectedSite, + ): Operation { + // The engine's own account of how the element was written, not a method on + // the object this was handed: a wrapper can mint an object carrying + // `hasContent`, and it cannot mint an issuance. + const form = invocationForm(invocation); + if (form === undefined) { + throw new ComponentInvocationError(UNISSUED_REFUSAL); + } + + // Every refusal a selection or a spelling can produce happens before the + // durable name is claimed and before any program exists, so an element this + // host cannot answer for leaves no admission record and performs no effect. + const profile = site.evaluation; + if (profile === undefined) { + throw new ComponentInvocationError(NO_PROFILE); + } + const allow = requestedClasses(elementProps.allow) ?? ["read"]; + const stated = statedText(elementProps, profile, form); + const entries = selectedTables(profile, allow); + + const id = yield* claim(invocation); + + // Narrowed from the tables the selection resolved to, so the vocabulary a + // producer is told about is the vocabulary the fragment is admitted for — + // and reported as availability against the enclosing reference, which keeps + // the authoring documentation the site already had. + const narrowed = narrow(site.syntax, entries); + const source = stated === undefined ? yield* project(site, narrowed) : stated; + + // Read after the producer has rendered, and exactly once per occurrence: a + // producer may itself commit mutations, and the basis this admission is + // made under is the run's basis at the moment the program exists. + if (!profile.live()) { + throw new ComponentInvocationError(REVOKED); + } + const basis = profile.workspace === undefined ? undefined : yield* profile.workspace.snapshot(); + + // Where the authored element was written, as diagnostic journal data beside + // the admission. A generated fragment's own elements are scanned from a + // dynamic string and carry no authored position of their own. + const expansion = yield* getExpansion(); + + const request: GeneratedXmdRequest = { + id, + source, + allow, + observations: entries.observations, + ...(entries.mutations.length === 0 ? {} : { mutations: entries.mutations }), + ...(basis === undefined ? {} : { workspaceRoots: basis.roots, selectedRoot: basis.current }), + ...(expansion.position === undefined ? {} : { position: expansion.position }), + }; + // Every path in this fragment resolves against the directory the run is in + // now, and the cursor is restored however the evaluation ends — so a + // fragment produced inside another fragment's producer leaves the outer one + // where it was, and a failed one leaves nothing behind. + const leave = yield* profile.enterFragment(); + try { + return answer(yield* evaluateGeneratedXmd(request)); + } finally { + leave(); + } + }; +} + +/** + * The program the element stated, or nothing when its content will produce one. + * + * The two forms are disjoint and the check is symmetric: a paired element + * carrying `text` and a self-closing element carrying none are the same + * mistake seen from either side, and both are refused rather than resolved by + * precedence. + */ +function statedText( + elementProps: Record, + profile: CapturedProfile, + form: "self-closing" | "paired", +): string | undefined { + const text = elementProps.text; + const alias = elementProps.source; + if (alias !== undefined && !profile.deprecatedSourceAlias) { + throw new ComponentInvocationError(NO_ALIAS); + } + if (text !== undefined && alias !== undefined) { + throw new ComponentInvocationError(BOTH_FORMS); + } + const stated = text ?? alias; + if (form === "paired") { + if (stated !== undefined) { + throw new ComponentInvocationError(PAIRED_TEXT); + } + return undefined; + } + if (typeof stated !== "string") { + throw new ComponentInvocationError(NO_PROGRAM); + } + return stated; +} + +/** + * The classes this element asked for, canonicalized. + * + * Parsed rather than read. The declared schema already refuses everything but a + * non-empty duplicate-free subset, and what this adds is the canonical order + * the admission retains — two documents asking for the same two classes are + * asking for the same thing, so authored order takes no part in the policy a + * continuation is held to. It is a second gate rather than a redundant one: a + * protected body is handed a props object rather than a promise that one was + * validated. + */ +function requestedClasses(value: Json | undefined): readonly GeneratedEffectClass[] | undefined { + if (value === undefined) { + return undefined; + } + if (!Array.isArray(value) || value.length === 0) { + throw new ComponentInvocationError(ALLOW_SHAPE); + } + const selected = new Set(); + for (const entry of value) { + const effect = EFFECT_CLASSES.find((known) => known === entry); + if (effect === undefined) { + throw new ComponentInvocationError(ALLOW_UNKNOWN); + } + if (selected.has(effect)) { + throw new ComponentInvocationError(ALLOW_REPEATED); + } + selected.add(effect); + } + return EFFECT_CLASSES.filter((effect) => selected.has(effect)); +} + +/** The pinned tables one selection resolves to. */ +interface SelectedTables { + readonly observations: readonly GeneratedObservation[]; + readonly mutations: readonly GeneratedMutation[]; + /** Both tables together, for describing the vocabulary the fragment has. */ + readonly admitted: readonly CapturedEntry[]; +} + +/** + * The host's captured tables, as the pinned identities the evaluator admits. + * + * A class the host installed nothing for is refused here — before the durable + * name is claimed and before a program exists — rather than reaching the + * evaluator as an empty table. The identity a run retains is built from the + * three structural parts the host stated, so a reader comparing two admissions + * can say which part moved. + * + * The definition carries the *captured* schema rather than its own, so a host + * that edits the schema on the definition it handed over after installation + * does not change what a fragment's props are validated against. + */ +function selectedTables( + profile: CapturedProfile, + allow: readonly GeneratedEffectClass[], +): SelectedTables { + const observations: GeneratedObservation[] = []; + const mutations: GeneratedMutation[] = []; + const admitted: CapturedEntry[] = []; + if (allow.includes("read")) { + if (profile.read.length === 0) { + throw new ComponentInvocationError(NO_READ_TABLE); + } + for (const entry of profile.read) { + observations.push({ + name: entry.name, + identity: pinned(entry), + definition: entry.definition, + ...(entry.dispatch === undefined ? {} : { dispatch: entry.dispatch }), + // An entry admitted for one spelling admits that one; an entry admitted + // for both is the form-insensitive component it has always been. + ...(entry.forms.length === 1 && entry.forms[0] === "self-closing" + ? { selfClosing: true } + : {}), + // The version-1 strings this entry stated it succeeds, carried through + // so a continuation admitted under one of them reconciles against this + // entry. Nothing this run writes holds them. + ...(entry.legacy === undefined ? {} : { legacy: entry.legacy }), + ...(entry.requests === undefined ? {} : { requests: entry.requests.map(asRequest) }), + }); + admitted.push(entry); + } + } + if (allow.includes("write")) { + if (profile.write.length === 0) { + throw new ComponentInvocationError(NO_WRITE_TABLE); + } + for (const entry of profile.write) { + mutations.push({ + name: entry.name, + identity: pinned(entry), + definition: entry.definition, + ...(entry.dispatch === undefined ? {} : { dispatch: entry.dispatch }), + ...(entry.legacy === undefined ? {} : { legacy: entry.legacy }), + form: entry.forms.length === 2 ? "either" : (entry.forms[0] ?? "self-closing"), + }); + admitted.push(entry); + } + } + return { observations, mutations, admitted }; +} + +/** + * The identity a run retains for one captured entry. + * + * Structural rather than assembled: the four terms travel as themselves, so a + * continuation compares them one at a time and a reader looking at two + * admissions can say which of them moved. Nothing here is derived from the + * implementation — an implementation is not an identity — and the kind travels + * with the rest because an operation core supplies the body for and an answer + * the import chain resolved are different grants under the same three names. + */ +function pinned(entry: CapturedEntry): RetainedFragmentIdentity { + const { origin, key, revision } = entry.identity; + return entry.kind === "component-answer" + ? { kind: "component-answer", origin, key, revision } + : { kind: "capability", origin, key, revision }; +} + +/** One captured ceiling, as the request record the evaluator compares against. */ +function asRequest(request: { + url: string; + method: string; + headers: Record; + timeout?: number; +}): Record { + return { + url: request.url, + method: request.method, + headers: { ...request.headers }, + ...(request.timeout === undefined ? {} : { timeout: `${request.timeout}ms` }), + }; +} + +/** + * The reference a producer renders under, or nothing when the site carries none. + * + * Narrowing is the reference's own operation rather than something assembled + * here, because what an author may *read about* is not what this evaluation + * admits: the enclosing documentation index stays exactly what it was, and only + * the availability of each entry is replaced. + */ +function narrow( + enclosing: SyntaxReference | undefined, + tables: SelectedTables, +): SyntaxReference | undefined { + return enclosing?.available(admittedSymbols(tables.admitted)); +} + +/** + * Render the content once, under the narrowed reference. + * + * The projection is canonical expansion's own, delivered directly to this body + * and reachable from nowhere else: it is not a context, not a registration, and + * not published through `ActiveProjection`, so nothing that runs inside the + * producer can render this element's content a second time or under a different + * vocabulary. + */ +function* project(site: ProtectedSite, narrowed: SyntaxReference | undefined): Operation { + if (site.projectContent === undefined || narrowed === undefined) { + throw new ComponentInvocationError(NO_PROJECTION); + } + return yield* site.projectContent(narrowed); +} + +/** + * What the document reads back: a detached value, not text. + * + * Copied out of the evaluator's own result rather than handed on, so the object + * a document binds shares nothing with the evaluation that produced it. + */ +function answer(result: GeneratedObservationResult): Json { + return { + observations: result.observations.map((observation) => ({ + name: observation.name, + value: observation.value, + })), + output: result.output, + }; +} diff --git a/packages/core/src/components/Syntax.ts b/packages/core/src/components/Syntax.ts new file mode 100644 index 000000000..66754864d --- /dev/null +++ b/packages/core/src/components/Syntax.ts @@ -0,0 +1,235 @@ +/** + * `` — what a document may write, written into the document. + * + * An author asking "which components do I have here?" and an agent being told + * what to write are the same question, and `xmd syntax` already answers it from + * outside. This is the same answer from inside: the symbols for the site the + * element was written at, as the Markdown that command prints. + * + * ## Why canonical core owns it + * + * The symbols describe the vocabulary an execution actually has. A repository + * `Syntax.md`, a bundled `Syntax`, a registration, an import handler or a second + * loaded copy answering for the name would each describe a vocabulary the run + * does not have — to whoever is reading, and to whichever agent is being told + * what to write next. So the name is claimed by the canonical protected tier, + * ahead of every host and author tier, and the definition canonical core + * selected is what runs. + * + * Protection is about the *answer*, not about power. The component receives one + * reference that renders symbol text and nothing else: no definitions, no + * import witness, no invocation capability, no policy table, no provider and no + * registration handle. Naming a component in the symbols is not permission to + * run it. + * + * ## What one occurrence does + * + * It claims the occurrence identity this execution minted, renders once, and + * retains exactly what it rendered. A continuation reads that record and hands + * the same text back without consulting the filesystem, the registry, the + * bundle, the host or the lexical reference again — so an agent resuming + * authorship is shown the vocabulary the run actually showed it, not one + * rebuilt from a tree that has moved since. + */ + +import { createDurableOperation, StaleInputError } from "@executablemd/durable-streams"; +import type { Json as DurableJson, Workflow } from "@executablemd/durable-streams"; +import type { Operation } from "effection"; + +import { getExpansion } from "../expansion.ts"; +import { ComponentInvocationError, invocationForm } from "../invocation-identity.ts"; +import type { + ComponentInvocation, + IdentityClaimant, + ProtectedBody, + ProtectedSite, +} from "../invocation-identity.ts"; +import { sourceDescription } from "../source-position.ts"; +import type { ProtectedComponent } from "./protected.ts"; +import { CORE_ORIGIN } from "./registry.ts"; +import { documented } from "./documentation.ts"; +import type { Json, PropsSchema, SourcePosition } from "../types.ts"; + +/** The public name canonical core claims for the syntax component. */ +export const SYNTAX_COMPONENT = "Syntax"; + +/** The durable effect one occurrence records. */ +const SYNTAX_SYMBOLS = "syntax_symbols"; + +/** + * One optional prop, closed. + * + * The site decides what the symbols say; `names` decides only whether the + * occurrence renders the list or the selected documentation. Any other prop is + * refused before the body runs, which is what keeps a spelling nobody supports + * from quietly rendering every symbol anyway. + */ +export const props: PropsSchema = { + type: "object", + properties: { + names: { + type: "array", + items: { type: "string" }, + minItems: 1, + uniqueItems: true, + description: + "Optional. Render these components' metadata and long-form documentation " + + "instead of the list of available symbols. Entries render once each, in symbol order.", + }, + }, + additionalProperties: false, +}; + +const PAIRED_REFUSAL = + " renders the available symbols and reads no content, so it is written self-closing."; + +const NAMES_REFUSAL = + " takes a non-empty list of component names, each a string."; + +const DUPLICATE_REFUSAL = + " takes each component name once: an entry renders once however " + + "many times it is asked for."; + +const UNISSUED_REFUSAL = + " is invoked by canonical core; this is not an invocation the engine issued."; + +const NO_REFERENCE_REFUSAL = + " has no symbols to read here: this expansion carries no syntax reference, so " + + "nothing established what a document may write at this site."; + +const UNREADABLE_RECORD = + "the retained text is not a record this version can read, so no symbols were " + + "produced."; + +/** + * The declaration canonical core selects for ``. + * + * Self-closing only, one optional prop, and no `returns` — which is what makes + * it a text component: it emits through the current presentation middleware, + * and `as` captures the same text through the engine's ordinary capture and + * emits nothing. + */ +export const SYNTAX_PROTECTED: ProtectedComponent = { + name: SYNTAX_COMPONENT, + origin: CORE_ORIGIN, + props, + forms: ["self-closing"], + ...documented({ + description: + "Inspect available components and control-flow constructs. `` lists the " + + 'symbols available here; `` renders selected documentation.', + as: "Optional. Captures the rendered text instead of emitting it.", + context: null, + }), + build: (claim: IdentityClaimant) => syntax(claim), +}; + +function syntax(claim: IdentityClaimant): ProtectedBody { + return function* renderSyntax( + props: Record, + invocation: ComponentInvocation, + site: ProtectedSite, + ): Operation { + const reference = site.syntax; + // Read off the issuance the engine holds rather than off a method the + // caller could have written, and answered before anything is claimed or + // rendered: a paired spelling is a document asking for something this + // component does not have, not symbols to go and build. + const form = invocationForm(invocation); + if (form === undefined) { + throw new ComponentInvocationError(UNISSUED_REFUSAL); + } + if (form === "paired") { + throw new ComponentInvocationError(PAIRED_REFUSAL); + } + // Read before anything is claimed or rendered, so a list this component + // cannot answer for refuses with no durable record and no partial text. + // The schema has already rejected an empty list, a duplicate and a + // non-string member; what is left is whether the value is the array shape + // this reads, because a protected body is handed props rather than trusting + // that somebody validated them. + const names = requestedNames(props.names); + const id = yield* claim(invocation); + if (reference === undefined) { + throw new Error(NO_REFERENCE_REFUSAL); + } + const expansion = yield* getExpansion(); + return yield* persistSymbols(id, expansion.position, () => + names === undefined ? reference.symbols() : reference.documentation(names), + ); + }; +} + +/** + * The names this occurrence asked to document, or nothing for the bare form. + * + * The declared schema is the first gate and rejects an empty list, a duplicate + * and a non-string member before the body is entered. This is the second, and it + * exists because a body is handed a props object rather than a promise that one + * was checked: a value that is not the shape this reads is refused here rather + * than becoming an empty selection that renders every symbol. + */ +function requestedNames(value: Json | undefined): readonly string[] | undefined { + if (value === undefined) { + return undefined; + } + if (!Array.isArray(value) || value.length === 0) { + throw new ComponentInvocationError(NAMES_REFUSAL); + } + const names: string[] = []; + for (const member of value) { + if (typeof member !== "string" || member.length === 0) { + throw new ComponentInvocationError(NAMES_REFUSAL); + } + if (names.includes(member)) { + throw new ComponentInvocationError(DUPLICATE_REFUSAL); + } + names.push(member); + } + return names; +} + +function* persistSymbols( + id: string, + position: Readonly | undefined, + live: () => Operation, +): Workflow { + const stored = yield createDurableOperation( + { + type: SYNTAX_SYMBOLS, + name: `${SYNTAX_SYMBOLS}:${id}`, + ...sourceDescription(position), + }, + function* (): Operation { + return { symbols: yield* live() }; + }, + ); + const symbols = readSymbols(stored); + if (symbols === undefined) { + // A record this version cannot read is the journal no longer describing + // this run, not a component that failed: it travels as the stale input it + // is, rather than becoming an error segment a printing boundary could turn + // into text and carry on past. + throw new StaleInputError(UNREADABLE_RECORD); + } + return symbols; +} + +/** + * The text a record holds, read as a closed protocol. + * + * Exactly one member, a string. A record missing it, carrying a member this + * version does not know, or holding one of the wrong type is a record this + * version cannot read — not one to fill a default in for, because every default + * here is a guess about what an earlier run actually showed somebody. + */ +function readSymbols(value: unknown): string | undefined { + if (typeof value !== "object" || value === null || Array.isArray(value)) { + return undefined; + } + const symbols = Reflect.get(value, "symbols"); + if (Object.keys(value).length !== 1 || typeof symbols !== "string") { + return undefined; + } + return symbols; +} diff --git a/packages/core/src/components/bundle.ts b/packages/core/src/components/bundle.ts index 51f477006..14b5bbb67 100644 --- a/packages/core/src/components/bundle.ts +++ b/packages/core/src/components/bundle.ts @@ -37,6 +37,7 @@ */ import type { ImportRefusal, ImportTier } from "./import-authority.ts"; +import { PROTECTED_COMPONENT_NAMES, protectedNameRefusal } from "./protected.ts"; import { CORE_COMPONENT_NAMES } from "./registry.ts"; import { isComponentName } from "./registration.ts"; import { RESERVED_STRUCTURAL } from "../structural.ts"; @@ -114,6 +115,11 @@ export class WorkflowImportAuthority implements ImportTier { return this.#components.get(name); } + /** Every name this bundle supplies, for a catalog describing the run. */ + names(): Iterable { + return this.#components.keys(); + } + claims(name: string): boolean { return this.#components.has(name); } @@ -175,6 +181,11 @@ export function installedBundle( "owns rather than a component.", ); } + if (PROTECTED_COMPONENT_NAMES.has(name)) { + throw new WorkflowBundleError( + `a workflow component bundle ${protectedNameRefusal(name, "declare")}.`, + ); + } if (CORE_COMPONENT_NAMES.has(name)) { throw new WorkflowBundleError( `a workflow component bundle declared "${name}", which is a component the engine ` + diff --git a/packages/core/src/components/components.md b/packages/core/src/components/components.md new file mode 100644 index 000000000..c942e7bfa --- /dev/null +++ b/packages/core/src/components/components.md @@ -0,0 +1,343 @@ +Long-form documentation for the components canonical core owns. + +Each level-two heading below is the exact name of one component. The compact +list of symbols — `xmd syntax`, or a bare `` — names every component +with its forms, props and one-line description. This file holds the part that +does not belong in a list: when to reach for a component, what it does at run +time, and what it will refuse. + +Every component this package supplies has a section here. A build in which one +does not refuses rather than serving a reference with a silent hole in it: a +reader cannot tell "nobody has written this yet" from "this component has +nothing to say". The no-documentation sentence is for *custom* components, which +no package governs. + +## Syntax + +Inspects the components and control-flow constructs available where the element +is written. + +```mdx + +``` + +The bare form lists the symbols available here: every name a document may write +at that site, with its forms, props and description. It is the same text +`xmd syntax` prints, built by the same code, so an operator reading a terminal +and an agent reading a document are never told different things about one +profile. + +```mdx + +``` + +The named form renders the selected components' metadata followed by the +long-form documentation on this page. Use it when something needs to know how to +use a few specific components rather than what exists — a prompt that has to +explain `` does not need the other seventy entries. Entries render once +each, in symbol order, whatever order they were asked for in. + +`as` captures the rendered text instead of emitting it, in either form: + +```mdx + +``` + +### What the symbols describe + +The site, not the product. They reflect the host profile the execution is running +under, its working directory and includes, the workflow bundle or declared +components it is closed over, and any narrowing a trusted evaluation boundary +applied. Two sites in one document can therefore answer differently, and that is +the point: the answer is what *this* element may write. + +Inside an evaluation that narrows what may execute, the bare form reports the +narrowed vocabulary, while the named form still explains components from the +enclosing authoring symbols and states for each whether it is available in the +current evaluation. Reference material and execution authority are different +questions, and conflating them would either hide documentation an author needs +or imply an authority they do not have. + +### What it refuses + +An empty `names` list, a duplicate name, a member that is not a string, and a +name no symbol entry matches are each refused before anything is read, so a +refusal produces no partial text and no retained result. A paired spelling +and any prop other than `names` and `as` are refused the same way. + +### What it does not do + +Seeing a component named here is not permission to run it. The symbols and +this documentation are text; what a name means is still resolution's decision, +and what may run is still the execution's. + +## Evaluate + +Runs program text this document did not author. + +```mdx + +``` + +An agent proposes a fragment of Executable Markdown; this is where that fragment +is admitted and performed. The self-closing form takes the program as `text` — a +value the document already holds. The paired form makes the content the +producer, and what it renders is the program: + +```mdx + + Read the changelog and report the version. + +``` + +The two are disjoint. An element that states `text` *and* renders content is +stating the program twice, and is refused rather than resolved by precedence. + +It returns each admitted observation's own value, in the order the fragment +invoked them, with whatever the fragment rendered kept beside them under +`output`. That is a value rather than text: render it where you want text, with +``. + +### `allow` narrows; it never grants + +Writing `` grants nothing. Every ceiling was stated by the host that +assembled this run, before any document existed — which components a fragment +may name, which spelling of each, and the exact requests any of them may +perform. `allow` names an effect *class* and selects among the tables that +already exist: `read` observes, `write` mutates. Omitting it asks for `read`. A +class the host installed nothing for is refused before the program is read. + +An execution whose host stated no evaluation profile has no evaluation at all, +and says so when the element is written rather than at startup. + +### What a fragment may contain + +Much less than a document. The evaluator admits the host's pinned components and +plain text, and refuses the whole fragment — before its first effect — for an +executable code block, an expression prop, an interpolated binding, an `as` +binding, any structural construct, a component the host did not admit, a +spelling the host did not admit that component for, or a request outside the +stated ceiling. + +A producer written as content is told exactly this: it renders under the +narrowed symbols, so `` inside it lists what a fragment may write +rather than what the surrounding document may write. + +### Resuming + +One occurrence is one durable admission. A continuation restores the decision +that was made rather than making it again, and refuses if the run now offers +different text, or states ceilings the admission was not granted under. + +## Elicit + +Asks a person a structured question and returns their answer. + +```mdx + +Which release should ship first? + +``` + +The content is the request shown to the person. `schema` is a JSON Schema the +answer is validated against, so what comes back is the shape the document said +it needed rather than free text a later step has to interpret. The answer binds +through `as`. + +The question is asked once and retained. A run that resumes after the answer was +given restores it rather than asking again, which is what makes a document with +an elicitation in it safe to interrupt. + +How the question reaches a person is the host's: a terminal prompts, and another +host may route it somewhere else entirely. The document states what it needs to +know, not how to ask. + +## File + +Reads or writes a file, relative to the working directory. + +```mdx + + + +The content to write. + +``` + +The self-closing form reads the file and renders its content. The paired form +writes its content to the path. Both are ordinary durable effects: a write that +already happened is not repeated on a continuation, and a read restores what it +read rather than re-reading a file that has since changed. + +A read of a path that does not exist fails. The write form creates the file and +the directories above it as needed. + +## File.Delete + +Deletes a file, relative to the working directory. + +```mdx + +``` + +Self-closing, and it renders nothing. Deleting a path that does not exist +succeeds: the component's promise is that the file is gone afterwards, not that +it was there first, which is what makes it safe to write in a cleanup step that +may run more than once. + +Like ``, it is an ordinary durable effect — a deletion that already +happened is not repeated on a continuation. + +## TempDir + +Runs work in a temporary working directory. + +```mdx + +working notes + +``` + +The paired form expands its content with the temporary directory as the working +directory, so a `` or a command inside writes there rather than in the +directory the run started in. The self-closing form renders the path instead, +which is what you want when something outside the region needs to know where it +is: + +```mdx + +``` + +Use it to keep intermediate work out of the user's tree, and to make a document +that writes files safe to run from anywhere. + +## Fail + +Stops authored work with an actionable failure. + +```mdx + +``` + +Raises the message where it is written. It is the authored counterpart of an +error a component raises on its own: the document has decided that what it found +is not something it can proceed from, and says so in its own words rather than +letting a later step fail obscurely. + +The message is the whole point — write what a reader would need in order to act, +not that something went wrong. + +## Fetch + +Reads over HTTP. + +```mdx + +``` + +Only GET and HEAD are currently supported. Without `as`, a non-2xx status fails +the document. With `as`, the response binds instead — status included — which is +what makes a status something to branch on rather than an error: + +```mdx + + + +Nothing published yet. + +``` + +The request is journaled, so a continuation restores what the first run received +rather than asking the network again. + +## Glob + +Lists the files matching a pattern, relative to the working directory. + +```mdx + +``` + +`include` is a list of patterns, so one element can gather several shapes of +path in one pass. `as` is required: the component's result is the list of +matched paths, and there is no useful text to render. + +The list is sorted, so a document that iterates it produces the same output for +the same tree. Directories and symbolic links are never results — only files. + +## CodeBlock + +Shows arbitrary text as a fenced Markdown code block. + +```mdx + +``` + +Self-closing: the text is the `value` prop rather than content. Use it when a +value is going into a document that will be read as Markdown and must not be +interpreted as Markdown — a fragment containing backticks, a diff, or anything +an agent might otherwise read as instructions. + +It chooses a fence the value cannot break out of, so a value that itself +contains fences is still shown rather than escaping into the surrounding +document. `as` captures the exact fenced Markdown instead of emitting it. + +## Json + +Renders a value as JSON text. + +```mdx + +``` + +Writes the JSON where the element is written, or binds it with `as`. The +counterpart of ``: this turns a value into text, that turns text into a +value. + +## Parse + +Parses JSON text against a schema, and errors on invalid content. + +```mdx +{raw} +``` + +The content is the JSON text. `as` is required, because the parsed value is the +result. Invalid content — malformed JSON, or JSON the schema rejects — fails the +document, which is what you want when there is nothing sensible to do without +the value. + +Use `` instead when the document should decide what to do about +invalid input. + +## SafeParse + +Parses JSON text against a schema, and returns a result object instead of +failing. + +```mdx +{raw} +``` + +The bound result is either the validated value or the issues that rejected it, +so the document can branch on which it got. Reach for this when invalid input is +an expected case — reading something a person typed, or a response from a +service that may be having a bad day — rather than a reason to stop. + +## Test + +Declares a test case. + +```mdx + +… + +``` + +It runs under `xmd test`, or inside a `` region. Elsewhere it is +skipped, so a document carrying its own tests stays runnable as an ordinary +document — the tests are inert until something asks for them. + +A failing command or assertion inside the case fails that case rather than the +whole run, which is what lets one run report every failure rather than only the +first. diff --git a/packages/core/src/components/declared-markdown.ts b/packages/core/src/components/declared-markdown.ts index 12a202554..ff03b71de 100644 --- a/packages/core/src/components/declared-markdown.ts +++ b/packages/core/src/components/declared-markdown.ts @@ -46,6 +46,7 @@ import { parseMarkdownDefinition } from "../definition.ts"; import { formsRefusal } from "../invocation-identity.ts"; import type { IdentityComponent } from "../invocation-identity.ts"; import { RESERVED_STRUCTURAL } from "../structural.ts"; +import { PROTECTED_COMPONENT_NAMES, protectedNameRefusal } from "./protected.ts"; import { CanonicalImports, retain } from "./import-authority.ts"; import type { ImportedDefinition, ImportRefusal, ImportTier } from "./import-authority.ts"; import { admitDeclaration, isComponentName } from "./registration.ts"; @@ -178,6 +179,9 @@ export function* admitDeclaredMarkdown( "engine owns rather than a component.", ); } + if (PROTECTED_COMPONENT_NAMES.has(name)) { + throw refuse(`a host ${protectedNameRefusal(name, "declare as Markdown")}.`); + } if (origin.length === 0) { throw refuse( `the declared Markdown component "${name}" needs an origin naming where it came from.`, diff --git a/packages/core/src/components/import-authority.ts b/packages/core/src/components/import-authority.ts index b747d600f..a1d98837e 100644 --- a/packages/core/src/components/import-authority.ts +++ b/packages/core/src/components/import-authority.ts @@ -18,10 +18,16 @@ * nothing a handler still holds decides what is *invoked*. */ -import type { ComponentDefinition, FunctionComponentDefinition } from "../types.ts"; -import type { FormSelections, InvocationIdentities } from "../invocation-identity.ts"; +import type { ComponentDefinition, FunctionComponentDefinition, SourcePosition } from "../types.ts"; +import type { + FormSelections, + InvocationIdentities, + ProtectedBodies, +} from "../invocation-identity.ts"; import type { DeclaredImports, PrivateClosure } from "./declared-markdown.ts"; import type { ExactSource } from "../output/exact-source.ts"; +import type { SyntaxReference } from "../syntax-reference.ts"; +import type { CapturedProfile } from "../evaluation-profile.ts"; /** A definition an import may answer with. */ export type ImportedDefinition = ComponentDefinition | FunctionComponentDefinition; @@ -101,6 +107,46 @@ export interface ExpansionAuthority { * a component or middleware can name reaches it. */ readonly forms?: FormSelections; + /** + * What a document may write at the site being expanded. + * + * The execution builds one at its root from the selection inputs it captured, + * and hands it here by value like everything else on this object — not through + * a Context, because a context resolves by name and a name is not a secret, so + * a document could build one and answer for the vocabulary it is shown. + * + * It is lexical. A trusted canonical evaluation boundary that has already + * admitted the exact vocabulary a subtree may write replaces this member for + * that subtree, and leaving the subtree restores the enclosing one. Nothing + * else changes it: an ordinary component's body, the content a caller + * projected and an imported definition each carry what the site carried. + */ + readonly syntax?: SyntaxReference; + /** + * The maximum authority a generated fragment may be evaluated under. + * + * Stated by the trusted host at the installation boundary, before the root + * import and before any document, component or middleware code exists, and + * handed here by value like everything else on this object. It is on the + * private authority rather than in a context for the reason the rest are, and + * one more: `` is a *public* component, so any author may write it, + * and what keeps that from being a capability is that the ceiling it narrows + * from was settled by somebody the document cannot reach. + * + * Absent for a host that offers no evaluation. That is not an unrestricted + * evaluation — it is no evaluation, and `` refuses. + */ + readonly evaluation?: CapturedProfile; + /** + * The bodies this execution will enter for the components canonical core + * protects. + * + * Held by the execution and handed here by value, like the identity domains + * beside it. It is what makes a protected implementation reachable at all: an + * implementation another loaded copy built is in that copy's table, and one + * kept past this execution's teardown reaches a table that is gone. + */ + readonly protectedBodies?: ProtectedBodies; } /** Why an answer is not the one canonical execution produced for this name. */ @@ -220,8 +266,413 @@ function read(inspect: () => T): T | undefined { * authorized because it *is* the object the terminal minted, not because it * resembles one. */ +/** + * A provider's stable statement about the implementation it supplied. + * + * `origin` is canonical execution's to fix, not the provider's: each provider + * installation carries its origin into every request it opens, so one provider + * cannot state an identity under another's name. + */ +export interface AnswerIdentity { + readonly origin: string; + readonly key: string; + readonly revision: string; +} + +/** What a provider states, minus the part it does not get to choose. */ +export interface ClaimedIdentity { + readonly key: string; + readonly revision: string; +} + +/** + * One identified answer: what a provider stated, and what core kept of it. + * + * The two travel together because a caller needs both and must not obtain them + * separately. The definition is core's own copy, taken when the claim was + * recorded; it is what the caller keeps, and it is why nothing downstream ever + * reads the object the public chain returned a second time. + */ +export interface IdentifiedAnswer { + readonly identity: AnswerIdentity; + readonly definition: ImportedDefinition; +} + +/** One claim this owner recorded, with core's own copy of what was claimed. */ +interface Claim { + readonly name: string; + readonly identity: AnswerIdentity; + /** Which provider installation stated it, so a second cannot overwrite. */ + readonly installation: object; + /** + * The exact resolution this was an answer to. + * + * The window *object*, not a number describing one. Provenance is a question + * about which import a statement answered, and a caller asking it presents + * the window it is holding — so the comparison is between two references to + * one thing rather than between a record and whatever the owner's mutable + * current state happens to say. A number would have to be trusted against + * that mutable state; an object cannot be forged into being the one the + * caller opened. + */ + readonly window: ResolutionWindow; + readonly canonical: ImportedDefinition | undefined; +} + +/** A provider request used after its execution ended. */ +export const REVOKED_ANSWER_AUTHORITY = + "the execution that installed this answer provider has ended, so nothing it states identifies " + + "an implementation here"; + +/** A claim stated outside the handler invocation it would have been an answer to. */ +export const SETTLED_CLAIM = + "this resolution has settled, so a claim stated now identifies nothing. An answer is " + + "identified by the handler invocation the import asked, while that invocation is still " + + "deciding; a handler that has returned, or one recording after the fact, is not supplying it."; + +/** A second, different statement from one installation about one resolution. */ +export const SPENT_OPPORTUNITY = + "this provider already stated which implementation answers this import, and one import is one " + + "implementation. The next resolution offers a fresh request; this one is decided."; + +/** + * One open resolution, and the only way to end it. + * + * Handed to canonical execution rather than published: the window belongs to + * the import that opened it, and closing somebody else's would settle a + * decision still being made. It is also the value a caller presents to + * `identify`, so what proves an answer belongs to this import is holding the + * object rather than describing it. + */ +export interface ResolutionWindow { + /** The component this resolution is deciding. */ + readonly name: string; + /** Which resolution this is, for a reader following two of them. */ + readonly occurrence: number; + /** Stop admitting claims for this resolution. Idempotent. */ + close(): void; +} + +/** + * What one handler invocation may state about the import it was asked. + * + * Minted per invocation, and closed the moment that invocation returns, throws + * or is cancelled. This is the authority a provider actually claims through: + * the installation gives a provider the right to *be asked*, and the request + * gives it the right to answer this one asking. Separating them is what makes a + * claim provable — a stable installation handle can only say "some provider", + * while a request says "this handler, deciding this import, right now". + * + * There is no `name` parameter on `claim`. The name is fixed when the request + * is minted, from what the chain asked, so a handler cannot state an answer for + * a component it was not asked about. + */ +export interface ComponentAnswerRequest { + /** The component this invocation was asked to resolve. */ + readonly name: string; + /** Where the element that asked was written, when the chain knew. */ + readonly position?: Readonly; + /** State which implementation answers this import, as this provider. */ + claim(answer: ImportedDefinition, identity: ClaimedIdentity): ImportedDefinition; +} + +/** One minted request, and the caller's own handle for ending it. */ +export interface OpenAnswerRequest { + readonly request: ComponentAnswerRequest; + /** + * End it, synchronously. + * + * Called from the `finally` of the invocation that opened it, so it runs + * whether the handler returned, threw or was cancelled. Nothing yields here: + * a lease that needed a suspension point to close would still be open across + * one. + */ + close(): void; +} + +/** + * One provider installation's private authority. + * + * Holding this is the right to be asked, fixed to one origin. It is not the + * right to answer: every statement goes through a request this mints for one + * invocation, so a retained installation handle can open a *new* request but + * cannot revive a settled one or reach another provider's. + */ +export interface ProviderInstallation { + /** Begin one handler invocation's request for one asked name. */ + open(name: string, position?: Readonly): OpenAnswerRequest; +} + +/** An identity a provider stated in a shape this execution cannot record. */ +export class AnswerIdentityError extends Error { + override name = "AnswerIdentityError"; +} + +/** The retained spelling of one identity, for a reader. */ +export function identityRecord(identity: AnswerIdentity): string { + return `${identity.origin}#${identity.key}@${identity.revision}`; +} + export class CanonicalImports { readonly #issued = new WeakMap(); + /** + * The identities providers stated, in the same owner that holds issuance. + * + * One owner rather than two registries: retention, exact-object lookup, the + * `stillDescribes` comparison and the lifecycle are one question asked about + * one table, and a parallel WeakMap would be a second place for an answer to + * be authorized from. + */ + readonly #claims = new WeakMap(); + /** + * Whether this owner identifies anything yet. + * + * Starts inactive. Canonical execution registers teardown, then activates, + * then creates provider installations — so a failure between construction + * and activation cannot leave live answer authority with no teardown behind + * it. + */ + #active = false; + /** + * The one resolution a claim may be stated during, when one is open. + * + * A provider installation outlives every resolution it takes part in, while a + * fresh request belongs to one handler invocation. Canonical execution asks + * the chain for one name, reads the answer, and closes the occurrence. A + * claim arriving outside that window is a losing or delayed handler recording + * into a decision already made, which is exactly what an admission must not + * acquire afterwards. + * + * So the window carries both terms. The occurrence keeps a claim from + * belonging to a resolution other than the live one, and the name keeps a + * handler settled for one name from recording under it while a different + * name is being resolved. + */ + #window: ResolutionWindow | undefined; + /** How many resolutions this owner has opened, so each one is its own. */ + #occurrences = 0; + /** + * The resolution each installation last stated an answer for. + * + * Secondary. What proves a statement belongs to an import is the request's + * captured window, not this; this only keeps one provider from naming two + * different implementations for one import, where only one of them could be + * what it resolved to. + * + * Keyed by the installation's own frozen token and holding the window object, + * so an installation stays reusable: it answers several admitted names and + * the same name resolved more than once, because each of those is a different + * window. Spending the installation itself would break a valid multi-name + * provider, which the contract does not ask a host to split up. + */ + readonly #spent = new WeakMap(); + + /** Begin identifying. Called after teardown is registered. */ + activate(): void { + this.#active = true; + } + + /** Stop. Called at teardown, on completion, failure or cancellation. */ + revoke(): void { + this.#active = false; + this.#window = undefined; + } + + get identifying(): boolean { + return this.#active; + } + + /** + * Open the claim window for one resolution of one name. + * + * Canonical execution calls this immediately before asking the ordinary chain + * and closes it in a `finally`, so the window ends the same way whether the + * resolution answered, fell back, failed or was cancelled. Nothing else opens + * one: there is no path from a document, a component or a provider to this + * method. + */ + beginResolution(name: string): ResolutionWindow { + if (!this.#active) { + throw new AnswerIdentityError(REVOKED_ANSWER_AUTHORITY); + } + this.#occurrences += 1; + const occurrence = this.#occurrences; + const owner = this; + const window: ResolutionWindow = { + name, + occurrence, + close(): void { + // Only this window is closed. A nested resolution that already replaced + // it has its own close, and clearing another one here would settle a + // decision still being made. Compared by identity, so no bookkeeping + // value has to be trusted to say which window this is. + if (owner.#window === window) { + owner.#window = undefined; + } + }, + }; + this.#window = window; + return window; + } + + /** + * One provider installation's authority, fixed to that origin. + * + * Holding this is the right to be *asked*. It states nothing on its own: + * every answer goes through a request this mints for one handler invocation, + * so a retained installation handle can be asked again — which a multi-name + * provider needs — and can never revive a settled request or reach another + * provider's. + * + * Ordinary closures throughout. A separately loaded copy of core receives + * this object and can answer with it; what it cannot do is state an origin + * canonical execution did not give it, or reach the table any other way — + * there is no shared symbol, module registry or context name behind this. + */ + provider(origin: string): ProviderInstallation { + const installation = Object.freeze({}); + const owner = this; + return { + open(name: string, position?: Readonly): OpenAnswerRequest { + // Captured by identity, here, at the moment this invocation begins. A + // request minted while resolution N is open answers resolution N or + // nothing: it holds the object, so it cannot be made to describe + // whichever window is open later. + const opened = owner.#window; + let live = true; + const request: ComponentAnswerRequest = Object.freeze({ + name, + // Copied, like everything else a provider is shown. The scanner's + // position is the engine's own mutable object, and handing it over + // would let a provider edit what a *later* reader of that element + // sees — a diagnostic seam turned into a write. + ...(position === undefined ? {} : { position: capturePosition(position) }), + claim(answer: ImportedDefinition, stated: ClaimedIdentity): ImportedDefinition { + return owner.#record( + installation, + origin, + { name, window: opened, live: () => live }, + answer, + stated, + ); + }, + }); + return Object.freeze({ + request, + close(): void { + live = false; + }, + }); + }, + }; + } + + #record( + installation: object, + origin: string, + asked: { name: string; window: ResolutionWindow | undefined; live: () => boolean }, + answer: ImportedDefinition, + stated: ClaimedIdentity, + ): ImportedDefinition { + if (!this.#active) { + throw new AnswerIdentityError(REVOKED_ANSWER_AUTHORITY); + } + // Four questions, and each of them is about the invocation rather than + // about the provider: is this handler still deciding; is the resolution it + // was asked in the one still open, by identity; and is the name it was + // asked the name that resolution is deciding. A stable installation handle + // answers none of them, which is why it does not claim. + const open = asked.window; + if (!asked.live() || open === undefined || open !== this.#window || open.name !== asked.name) { + throw new AnswerIdentityError(SETTLED_CLAIM); + } + const identity = complete(origin, stated); + const held = + typeof answer === "object" && answer !== null ? this.#claims.get(answer) : undefined; + if (held !== undefined) { + // Restating exactly what is already there is what a provider installed + // twice does, and it is not a conflict. Anything else is two providers + // disagreeing about one object, and the first statement stands: a later + // claim that overwrote it would let a second provider rename the first's + // implementation. + if ( + held.installation !== installation || + held.window !== open || + held.name !== asked.name || + held.identity.origin !== identity.origin || + held.identity.key !== identity.key || + held.identity.revision !== identity.revision + ) { + throw new AnswerIdentityError( + "this answer already carries an identity, and a second claim does not replace it. One " + + "implementation states what it is once.", + ); + } + return answer; + } + // Secondary, and about the provider rather than the invocation: naming a + // *different* implementation for an import this provider already answered + // is two answers where only one could be what it resolved to. + if (this.#spent.get(installation) === open) { + throw new AnswerIdentityError(SPENT_OPPORTUNITY); + } + // Copied on the way in, so a later edit of the claimed object is visible as + // the change it is. + this.#claims.set(answer, { + name: asked.name, + identity, + installation, + window: open, + canonical: retain(answer), + }); + // Spent for this window and no other: the next import is a different + // window, which is what lets one installation answer several admitted + // names and the same name resolved twice. + this.#spent.set(installation, open); + return answer; + } + + /** + * The identity stated for this exact answer, and core's own copy of it. + * + * Read after the whole public chain has returned, so what is asked about is + * the final answer rather than an intermediate one. Nothing here refuses: an + * unidentified answer is an ordinary answer, and whether that is enough is + * the caller's question. + * + * Both halves come back together, and the copy is the one taken when the + * claim was recorded rather than one made now. Answering with the identity + * alone would leave the caller holding the chain's object and needing to copy + * it itself — one more read of a value the chain controls, after the read + * this one checked. An alternating proxy or an accessor that answers twice + * would pass the check and hand the second answer to the copy. So the check + * and the thing kept are one result of one call, and the object that + * travelled through the chain is never read again. + */ + identify(resolution: ResolutionWindow, answer: unknown): IdentifiedAnswer | undefined { + if (!this.#active || typeof answer !== "object" || answer === null) { + return undefined; + } + const claim = this.#claims.get(answer); + if (claim === undefined) { + return undefined; + } + // The caller presents the resolution it opened, and the claim has to be an + // answer to *that* one, by object identity and under the name it decides. + // Nothing here consults the owner's current window: provenance read out of + // mutable state would be a claim about whenever the question was asked + // rather than about which import the statement answered. + if (claim.window !== resolution || claim.name !== resolution.name) { + return undefined; + } + // A claimed object the chain went on to edit is not the thing that was + // claimed. Reading it runs whatever it is made of, and a value that refuses + // to be compared has failed the comparison. + if (claim.canonical === undefined || !stillDescribes(claim.canonical, answer)) { + return undefined; + } + return Object.freeze({ identity: claim.identity, definition: claim.canonical }); + } /** * Record that canonical execution produced this answer for this name, and @@ -267,6 +718,50 @@ export class CanonicalImports { } } +/** + * One authored position, copied and frozen for a provider to read. + * + * The scanner's object belongs to the engine and is read again after any + * handler has seen it, so it crosses this boundary by value like every other + * thing a provider is shown. Four members, written out: a spread would carry + * whatever a future member turned out to be, and this is a surface a host's + * code holds. + */ +function capturePosition(position: Readonly): Readonly { + return Object.freeze({ + ...(position.path === undefined ? {} : { path: position.path }), + offset: position.offset, + line: position.line, + column: position.column, + }); +} + +/** + * The identity as this owner records it, or the reason it cannot. + * + * Three non-empty strings, checked here rather than trusted: what a + * continuation is compared against must be a value a reader can look at and say + * which part moved, and a partial identity would compare equal to a different + * partial one. `origin` is canonical execution's, so only the two the provider + * states are read off its object. + */ +function complete(origin: string, stated: ClaimedIdentity): AnswerIdentity { + const { key, revision } = stated; + if ( + typeof key !== "string" || + key.length === 0 || + typeof revision !== "string" || + revision.length === 0 + ) { + throw new AnswerIdentityError( + "an identity states a non-empty key and revision. A continuation is compared against both " + + "and the origin canonical execution fixed, so a partial one would compare equal to a " + + "different partial one.", + ); + } + return Object.freeze({ origin, key, revision }); +} + /** * How one closed tier words a refusal of an answer it did not produce. * @@ -302,11 +797,19 @@ export interface ImportTier { * an answer decides only how a refusal reads, never whether one is authorized. */ export class ExecutionImports implements ImportAuthority { - readonly #imports = new CanonicalImports(); + readonly #imports: CanonicalImports; readonly #tiers: readonly ImportTier[]; - constructor(tiers: readonly ImportTier[]) { + /** + * The owner is handed in rather than made here, because it outlives this + * object at both ends. Canonical execution constructs it before any + * installation runs — inactive, with its teardown already registered — so a + * provider can state identities during profile capture, long before the tiers + * this authority is built from exist. + */ + constructor(tiers: readonly ImportTier[], imports: CanonicalImports) { this.#tiers = tiers; + this.#imports = imports; } /** Record that canonical execution produced this answer for this name. */ diff --git a/packages/core/src/components/protected.ts b/packages/core/src/components/protected.ts new file mode 100644 index 000000000..f2132aaea --- /dev/null +++ b/packages/core/src/components/protected.ts @@ -0,0 +1,145 @@ +/** + * The names canonical core answers for, ahead of every other tier. + * + * Resolution already had a protected tier — a reserved registration, or exact + * Markdown a host declared — but both of those are *a host's* claim, made by + * whoever assembled the run. This tier is the engine's own, and it sits above + * them: a protected name means the same thing in every execution, whichever host + * built it, whichever package registered what, and whatever a repository holds. + * + * There are two components in it. `` describes the vocabulary of the + * site it is written at, and a description of a run's vocabulary that anything + * in the run could answer for is a description of nothing — the value of the + * answer is exactly that nobody but core produced it. `` runs program + * text the document did not author, under a ceiling a trusted host stated + * before any document existed; a component in the run answering for that name + * would decide what an unaudited fragment is allowed to do. + * + * ## What protection is, and is not + * + * Protection settles *which definition runs*. A repository `Syntax.md`, a + * bundled `Syntax`, an ordinary or reserved registration, a declared Markdown + * component and a definition from a second loaded copy of core can none of them + * win selection here. `Component.importComponent` middleware composes around the + * import exactly as it composes around any other: it may observe it, delegate it + * and refuse it by throwing, and what it cannot do is answer one — the answer is + * verified at the call site against core's own retained copy, and core's copy is + * what is invoked. + * + * Protection is not authority to do anything. A protected implementation is + * handed the site canonical execution built for it and nothing else: the + * lexical syntax reference, the evaluation profile a trusted host stated at the + * installation boundary, and a one-shot projection of its own content. No + * component definitions, no import witness, no invocation capability, no + * provider and no registration handle — and no way to widen what the profile + * already says, because the profile is a captured value rather than a handle to + * the host's own objects. + * + * ## Why it is not a registration + * + * A registration is an answer a registry gives for a name, and a registry is + * something a nested scope layers over, a host installs into, and a handler can + * keep a record from and hand back somewhere else. A name that must mean one + * thing cannot be decided by any of that, so the tier is the resolver's own + * table and the implementation reaches the call site through the execution that + * built it. + */ + +import { EVALUATE_PROTECTED } from "./Evaluate.ts"; +import { SYNTAX_PROTECTED } from "./Syntax.ts"; +import type { ImportRefusal, ImportTier } from "./import-authority.ts"; +import type { ComponentDocumentation } from "./documentation.ts"; +import type { ProtectedDeclaration } from "../invocation-identity.ts"; +import type { ComponentOrigin } from "../types.ts"; + +/** + * One component canonical core claims the name of. + * + * The same contract an identity component declares, minus the choice of origin: + * a protected component is core's, so it reports core's origin. Its body is + * built once per execution, with the claimant that execution minted, and the + * implementation the execution wraps it in is the only thing that name resolves + * to there. + */ +export interface ProtectedComponent extends ProtectedDeclaration, ComponentDocumentation {} + +/** Every name the engine itself claims. */ +export const PROTECTED_COMPONENTS: readonly ProtectedComponent[] = Object.freeze([ + EVALUATE_PROTECTED, + SYNTAX_PROTECTED, +]); + +const BY_NAME: ReadonlyMap = new Map( + PROTECTED_COMPONENTS.map((component) => [component.name, component]), +); + +/** The names above, as a set, for the admissions that must refuse them. */ +export const PROTECTED_COMPONENT_NAMES: ReadonlySet = new Set(BY_NAME.keys()); + +/** What canonical core answers for this name, or nothing when it answers none. */ +export function protectedComponent(name: string): ProtectedComponent | undefined { + return BY_NAME.get(name); +} + +/** + * The origin a protected component reports. + * + * Its own kind, because reusing `registered` with `reserved: true` said + * something untrue. A reserved registration is a *host* installing a component + * under a name it wants kept, so it can be absent from another execution, + * replaced by a different host, or refused when two hosts claim it. A protected + * component is core's own declaration: present in every execution, supplied by + * no registry, and unable to be registered at all. Reporting one as the other + * told a reader the name was a host's to take. + */ +export function protectedOrigin( + component: ProtectedComponent, +): Extract { + return { kind: "protected", origin: component.origin }; +} + +/** A protected name a host, a bundle or a registration tried to claim. */ +export class ProtectedComponentError extends Error { + override name = "ProtectedComponentError"; +} + +/** What a name that canonical core owns refuses a second claim with. */ +export function protectedNameRefusal(name: string, claim: string): string { + return ( + `cannot ${claim} "${name}": canonical core owns that name, so what it means is the same in ` + + "every execution and nothing else answers for it" + ); +} + +/** The fixed diagnostic each verification failure produces. */ +const REFUSED: Record = { + unissued: + "Component.importComponent middleware answered an import of a component canonical core " + + "owns with a definition canonical execution did not produce. A handler may observe, " + + "delegate or refuse the import; only canonical execution answers one.", + "another-name": + "Component.importComponent middleware answered an import of a component canonical core " + + "owns with the definition canonical execution produced for another component.", + changed: + "Component.importComponent middleware changed the definition canonical execution produced " + + "for a component canonical core owns before it was invoked.", +}; + +/** + * The tier a protected import is verified through. + * + * It closes the names it claims and nothing else, exactly as a declaration does: + * claiming `Syntax` says nothing about what any other name in the execution may + * resolve to, and every other import stays the open one it has always been. + */ +export class ProtectedImports implements ImportTier { + claims(name: string): boolean { + return PROTECTED_COMPONENT_NAMES.has(name); + } + + readonly closesExecution = false; + + refuse(refusal: ImportRefusal): Error { + return new ProtectedComponentError(REFUSED[refusal]); + } +} diff --git a/packages/core/src/components/registration.ts b/packages/core/src/components/registration.ts index 0f29b787f..9670f2cb6 100644 --- a/packages/core/src/components/registration.ts +++ b/packages/core/src/components/registration.ts @@ -17,6 +17,7 @@ import type { Context, Operation } from "effection"; import { Component } from "../component-api.ts"; import { updateOwn } from "../scope-local.ts"; import { RESERVED_STRUCTURAL } from "../structural.ts"; +import { PROTECTED_COMPONENT_NAMES, protectedNameRefusal } from "./protected.ts"; import { formsRefusal } from "../invocation-identity.ts"; import { documentationOf } from "./documentation.ts"; import type { ComponentDocumentation } from "./documentation.ts"; @@ -135,6 +136,9 @@ function assertUsableName(name: string): void { `cannot register "${name}": it is structural syntax the engine owns, not a component`, ); } + if (PROTECTED_COMPONENT_NAMES.has(name)) { + throw new ComponentRegistrationError(protectedNameRefusal(name, "register")); + } if (!isComponentName(name)) { throw new ComponentRegistrationError( `cannot register "${name}": a component name is capitalized, and each ` + diff --git a/packages/core/src/components/select.ts b/packages/core/src/components/select.ts index e6970712c..1a80a372b 100644 --- a/packages/core/src/components/select.ts +++ b/packages/core/src/components/select.ts @@ -5,13 +5,16 @@ * disagree about which tier won: * * 1. structural syntax the engine owns; - * 2. a host claiming the name — a reserved registration, or exact Markdown this + * 2. a component canonical core protects. The engine's own claim rather than a + * host's, so a protected name means the same thing in every execution, + * whichever host assembled it and whatever a repository holds; + * 3. a host claiming the name — a reserved registration, or exact Markdown this * environment declares. Two claims on one name are refused where they are * installed, so this tier never has to choose between them; - * 3. the workflow component bundle this execution is closed over; - * 4. a repository-local file; - * 5. a registered default, including core's own components; - * 6. nothing, which is the unresolved printed error. + * 4. the workflow component bundle this execution is closed over; + * 5. a repository-local file; + * 6. a registered default, including core's own components; + * 7. nothing, which is the unresolved printed error. * * The bundle tier exists only while a trusted host installed one, and a * workflow execution searches no repository directories at all — so what a @@ -29,6 +32,7 @@ import type { Operation } from "effection"; import type { WorkflowImportAuthority } from "./bundle.ts"; import type { DeclaredMarkdownCatalog } from "./declared-markdown.ts"; import { mergeRegistry } from "./registration.ts"; +import { protectedComponent, protectedOrigin } from "./protected.ts"; import { CORE_REGISTRY } from "./registry.ts"; import { RESERVED_STRUCTURAL } from "../structural.ts"; import type { ComponentOrigin, ComponentRegistry, ComponentSelection } from "../types.ts"; @@ -129,6 +133,14 @@ export function* selectComponent( return { kind: "structural", construct: name }; } + // Above every host and author tier, and unconditional: the table is core's + // own, so no option a caller passes — or leaves out — can put a repository + // file, a bundle member or a registration in front of it. + const owned = protectedComponent(name); + if (owned !== undefined) { + return { kind: "protected", component: owned, origin: protectedOrigin(owned) }; + } + if (entry?.reserved) { return { kind: "registered", diff --git a/packages/core/src/definition.ts b/packages/core/src/definition.ts index b88af713a..cef192914 100644 --- a/packages/core/src/definition.ts +++ b/packages/core/src/definition.ts @@ -36,7 +36,7 @@ interface ParsedSource { * gray-matter normalization surprise into a loud error instead of silently * wrong source positions. */ -function parseSource(path: string, content: string): ParsedSource { +export function parseSource(path: string, content: string): ParsedSource { const parsed = matter(content); const baseOffset = content.length - parsed.content.length; if (content.slice(baseOffset) !== parsed.content) { diff --git a/packages/core/src/document-validation.ts b/packages/core/src/document-validation.ts index 5ec432ffc..15350db98 100644 --- a/packages/core/src/document-validation.ts +++ b/packages/core/src/document-validation.ts @@ -879,6 +879,25 @@ class ValidationState { return; } + if (selected.kind === "protected") { + // Checked from the declaration alone: what the name means is core's, and + // validation neither runs an execution nor calls a factory to learn it. + draft.origin = selected.origin; + yield* this.#checkContract( + segment, + context, + draft, + { + props: selected.component.props, + captures: selected.component.captures ?? [], + forms: selected.component.forms ?? BOTH_FORMS, + hasReturns: selected.component.returns !== undefined, + }, + capture, + ); + return; + } + if (selected.kind === "registered") { draft.origin = selected.origin; yield* this.#checkContract( diff --git a/packages/core/src/documentation-api.ts b/packages/core/src/documentation-api.ts new file mode 100644 index 000000000..49fa6235b --- /dev/null +++ b/packages/core/src/documentation-api.ts @@ -0,0 +1,248 @@ +/** + * How a package contributes the documentation for the components it registers. + * + * Documentation composes with the components it describes. A package's + * bootstrap installs its registrations and its documentation together, through + * this one namespaced Api, so a host that bootstraps a package gets both by + * invoking one thing. The alternative — a host-maintained list of every + * package's documentation, kept beside a host-maintained list of every + * package's registrations — is two lists that drift, and they did: a nested run + * registered `` and then reported it undocumented, because one list + * had been updated and the other had not. + * + * ## The terminal is core's own + * + * `contributions()` answers with core's own documentation and nothing else. A + * package wraps it, delegates, and appends its own: + * + * ```ts + * yield* Documentation.around({ + * *contributions([read], next) { + * return [...(yield* next(read)), yield* webDocumentation(read)]; + * }, + * }); + * ``` + * + * Composition is why order cannot choose a winner: every wrapper delegates, so + * every contribution reaches the collector, and two contributions for one + * component refuse there rather than the later one silently replacing the + * earlier. + * + * ## What an execution reads + * + * Canonical execution asks *once*, after trusted host bootstrap and before the + * root import or any document code, and snapshots the answer by value. So + * middleware a document or a component installs later composes into a chain + * nothing reads: what `` renders is what the host assembled, + * not what the document arranged afterwards. Two executions assembled in + * separate scopes see their own, because a scope is what an Api answer belongs + * to. + * + * The execution's own asset reader travels as the argument rather than being + * read from module scope, so every package's asset is read through the reader + * belonging to the execution that asked, and two executions in one process + * cannot read through each other's. + */ + +import { type Api, createApi } from "@effectionx/context-api"; +import type { Operation } from "effection"; + +import { + CORE_DOCUMENTED_NAMES, + packagedAssetReader, + readCoreDocumentation, +} from "./component-documentation.ts"; +import type { DocumentationContribution, DocumentationReader } from "./component-documentation.ts"; +import { DocumentationIndexError } from "./documentation-index.ts"; +import { snapshotContributions } from "./syntax-reference.ts"; + +/** What this Api answers. */ +export interface DocumentationApi { + /** + * Every bootstrapped package's documentation, in bootstrap order. + * + * Order is not authority: it decides how the list reads, and nothing else. + * Two contributions naming one component of one package refuse wherever they + * sit in it. + */ + contributions(read: DocumentationReader): Operation; +} + +/** + * The namespace a package's bootstrap reaches. + * + * Stable and namespaced so a separately loaded copy of a package composes here + * too: what makes two copies agree is the Api's name, not a shared module + * instance. It carries documentation and nothing else — no definitions, no + * import witnesses, no registration handle — so composing with it grants a + * package no authority it did not already have. + */ +export const Documentation: Api = createApi("Documentation", { + /** + * Canonical core's own documentation, as the terminal. + * + * Every chain ends here, so core's components are documented in an execution + * that bootstrapped no other package at all. + */ + *contributions(read: DocumentationReader): Operation { + return [{ source: yield* readCoreDocumentation(read), supplies: CORE_DOCUMENTED_NAMES }]; + }, +}); + +/** + * Add this package's documentation to whatever the enclosing scope contributes. + * + * The one call a package's bootstrap makes, beside registering its components. + * It delegates first and appends after, so nothing it composes over is lost and + * no bootstrap can answer for a package that is not its own. + * + * The contribution is read when the collector asks, not when this is called, + * and through the reader the collector supplies: a bootstrap installed in one + * execution's scope reads that execution's assets. + * + * It appends, and nothing more. Whether a repeated bootstrap is one statement + * made twice or two bootstraps disagreeing is {@link capturedDocumentation}'s + * question, because this helper is a convenience and the Api it wraps is + * public: a package may compose `Documentation.around(...)` directly, and + * correctness cannot depend on which of the two it chose. Deduplicating here as + * well would put the rule in two places, and the copy that ran second would be + * the one nobody tested. + */ +export function* contributeDocumentation( + contribute: (read: DocumentationReader) => Operation, +): Operation { + yield* Documentation.around({ + *contributions([read], next): Operation { + return [...(yield* next(read)), yield* contribute(read)]; + }, + }); +} + +/** + * Whether two contributions are the same value. + * + * All four of what a contribution *is*: the owning package, the asset identity, + * the exact documentation text, and the set of components it accounts for. The + * text is not redundant with the asset — two bootstraps can name one path and + * read different bytes, from a stale build tree or a substituted reader — and + * coalescing those would pick a winner silently, which is the whole thing this + * boundary exists to prevent. + * + * Compared by value, not by identity: each bootstrap builds a fresh object, and + * a `Set` has no order, so two contributions listing the same names in different + * orders are the same statement. + */ +function identical(one: DocumentationContribution, other: DocumentationContribution): boolean { + return ( + one.source.owner === other.source.owner && + one.source.asset === other.source.asset && + one.source.text === other.source.text && + sameNames(one.supplies, other.supplies) + ); +} + +/** Two name sets holding the same names, whatever order they were built in. */ +function sameNames(one: ReadonlySet, other: ReadonlySet): boolean { + return one.size === other.size && [...one].every((name) => other.has(name)); +} + +/** + * What this execution's packages contributed, captured by value. + * + * Asked once, where canonical execution is assembled: after the trusted host's + * bootstrap and before the root import, so what a document later installs + * composes into a chain nothing reads. Snapshotted field by field for the same + * reason the installation boundary snapshots anything — the objects belong to + * whoever built them, and their `Set`s and strings can move afterwards. + * + * **This boundary decides what a duplicate is**, not the helper that most + * bootstraps happen to use. `Documentation` is public: a package may compose + * `Documentation.around(...)` itself and hand back two value-identical + * contributions directly, and that assembly is exactly as valid as one built + * with {@link contributeDocumentation}. Classifying in the helper instead would + * make correctness depend on which spelling a package chose. + * + * So: snapshot first, then fold, then classify. A contribution is four + * values — owning package, asset identity, exact documentation text, and the set + * of components it accounts for. Every completely identical contribution folds + * to one, because a repetition says exactly what the first said and there is no + * winner to pick; that is what lets one package's declarative vocabulary be + * entered at more than one layer, as an ordinary run's bootstrap and a workflow + * attachment both do, without losing either layer's registrations. + * + * What is left after folding are contributions that *disagree*: they overlap on + * one owning package and one component name while differing in asset, text or + * the set accounted for. Those refuse, wherever they sat in the chain — a later + * one silently winning would make what a document is told about a component + * depend on the order its host bootstrapped packages in. Two *different owners* + * documenting a same-spelled component is not a conflict, because documentation + * joins by name and origin, and neither is one owner accounting for disjoint + * sets from two files. + * + * Collection is where a disagreement is caught, because it is the only boundary + * every execution passes through. Deferring it to the named form's index would + * let a document that writes bare ``, or no `` at all, run to + * completion on an assembly nobody validated. + */ +export function* capturedDocumentation( + read: DocumentationReader = packagedAssetReader, +): Operation { + // Snapshotted *before* anything is compared, so what is classified is what + // will be kept. A contribution is a caller's object until this line — its + // strings and its `Set` can move afterwards — and comparing the live objects + // would decide identity against values the snapshot might not preserve. + const captured = snapshotContributions(yield* Documentation.operations.contributions(read)); + + // Every completely identical contribution folds to one, whichever way it + // reached the chain: the convenience helper and a direct + // `Documentation.around(...)` are the same to this boundary, which is the + // point of deciding here rather than in the helper. + const folded: DocumentationContribution[] = []; + for (const one of captured) { + if (!folded.some((kept) => identical(kept, one))) { + folded.push(one); + } + } + + // Keyed by owning package *and* component name, because that pair is what + // documentation joins on. Two packages may document same-spelled components, + // and one package may account for disjoint sets from two files; neither is a + // question anyone has to answer, so neither is a conflict. + const seen = new Map(); + for (const one of folded) { + for (const name of one.supplies) { + const owner = one.source.owner; + const first = seen.get(`${owner} ${name}`); + if (first !== undefined) { + throw new DocumentationIndexError( + `${owner} contributes documentation for ${name} twice, and the two ` + + `contributions are not the same: ${difference(first, one)}. One component of ` + + "one package has one documentation value, whichever order the bootstraps " + + "supplied it — so there is no winner to pick.", + ); + } + seen.set(`${owner} ${name}`, one); + } + } + return Object.freeze(folded); +} + +/** + * What two conflicting contributions disagree about, for the refusal to name. + * + * An identical pair never reaches this — `contributeDocumentation()` coalesces + * those — so at least one of the three differs, and saying which is the + * difference between a diagnosis and a complaint. + */ +function difference(first: DocumentationContribution, second: DocumentationContribution): string { + if (first.source.asset !== second.source.asset) { + return `they name different assets, ${first.source.asset} and ${second.source.asset}`; + } + if (first.source.text !== second.source.text) { + return `both name ${first.source.asset}, but its text differs between them`; + } + return ( + `both name ${first.source.asset} with the same text, but they account for ` + + "different components" + ); +} diff --git a/packages/core/src/documentation-index.ts b/packages/core/src/documentation-index.ts new file mode 100644 index 000000000..a217ab480 --- /dev/null +++ b/packages/core/src/documentation-index.ts @@ -0,0 +1,311 @@ +/** + * The long-form documentation a package ships beside the components it owns. + * + * `xmd syntax` and bare `` answer *what may I write here* — a compact + * catalog of names, forms and one-line descriptions. Neither answers *how do I + * use this one*, and an agent handed the compact catalog has to guess. So a + * package that registers components also ships their documentation, and this + * module turns those files into one validated index that ``, + * `xmd syntax Elicit` and the release reference all read (#678). + * + * ## Beside the components, not beside the website + * + * A registration bundle keeps `components.md` beside its own registration + * boundary, and the bytes are located from the owning module's URL — never from + * the working directory and never through `--include`. A documentation set that + * moved with the caller's directory would describe a different product depending + * on where somebody stood, and a repository file could answer for it. + * + * ## The shape, and why it validates + * + * A level-two heading is an exact public component name. Everything before the + * first one documents the bundle. Level-three and deeper headings belong to the + * component whose section they are in, so a component's own documentation can + * have structure without ending its section. + * + * Four things refuse the whole index rather than producing a partial one: + * + * - a **missing** section — a component the package supplies and does not + * document. A reader cannot tell "nobody has written this yet" from "this + * component has nothing to say", so a first-party package with a hole in its + * reference is not a valid build; + * - a **duplicate** heading, in one file or across two, because then a component + * has two documentations and nothing says which is current; + * - an **unknown** heading, because it is documentation for something this + * boundary does not register — a rename that updated one side, usually; + * - a heading that is **not a component name at all**, which is a file that has + * drifted from this format into ordinary prose. + * + * The no-documentation sentence `` renders is therefore *not* for a + * first-party component. It is for a **custom** component — a repository file, a + * bundle member, a host's declared Markdown — which no package governs and which + * takes its documentation from its own document's body when it has any. + * + * ## The join is name *and* origin + * + * Documentation is attached by both together. A repository `Elicit.md` is a + * different component from the built-in `Elicit` however it is spelled, and + * handing it the built-in's prose would describe behaviour the author's own file + * does not have. + */ + +import { isComponentName } from "./components/registration.ts"; +import { parseSource } from "./definition.ts"; +import type { ComponentOrigin } from "./types.ts"; + +/** One package's documentation for the components it registers. */ +export interface DocumentationSource { + /** + * The package this file documents, as its components report it — + * `@executablemd/core`, for the components canonical core owns. + * + * Half of the join, and a package rather than one origin *value* because a + * single registration boundary supplies components of more than one origin + * kind: canonical core owns `Syntax` in the protected tier and registers + * `Elicit` and `File` beside it, and all three are documented in one file. + * + * What this deliberately cannot match is an origin that names no package: a + * repository path, a workflow blob, a host's declared Markdown. A repository + * `Elicit.md` is a different component that happens to share a name, and + * handing it the built-in's prose would describe behaviour the author's own + * file does not have. + */ + readonly owner: string; + /** Where the bytes came from, so a refusal names a file somebody can open. */ + readonly asset: string; + readonly text: string; +} + +/** + * The package an origin names, or nothing when it names none. + * + * Only a registration and a protected component come from a package. A + * repository file, a bundled blob and declared Markdown are all *this run's*, + * however they are spelled, so no package-owned documentation is theirs. + */ +export function owningPackage(origin: ComponentOrigin): string | undefined { + if (origin.kind === "registered" || origin.kind === "protected") { + return origin.origin; + } + return undefined; +} + +/** A documentation set this version will not build an index from. */ +export class DocumentationIndexError extends Error { + override name = "DocumentationIndexError"; +} + +/** + * A name selected for documentation that this site has no component for. + * + * Its own error because it is the author's mistake rather than the build's: a + * misspelling, or a component that is not on this profile. It refuses the whole + * lookup, so nothing is rendered and nothing is retained. + */ +export class UnknownComponentError extends Error { + override name = "UnknownComponentError"; +} + +/** The long-form documentation one component has, if it has any. */ +export interface DocumentationIndex { + /** The documentation for exactly this component, or nothing when it has none. */ + documentationFor(name: string, origin: ComponentOrigin): string | undefined; + /** What the bundle at this origin says about itself, if anything. */ + bundleDocumentation(origin: ComponentOrigin): string | undefined; +} + +/** + * A Markdown component's own documentation, taken from its own document. + * + * A repository component, a bundled one and a host's declared Markdown are all + * *this run's* rather than a package's, so no `components.md` documents them. + * Their long-form documentation, when they have any, is the prose in their own + * file — which is where an author writing one would put it, and the only place + * that stays correct when the file changes. + * + * Read from the source the selection already holds, so this loads nothing: a + * repository component's bytes were read to describe it, and a bundled or + * declared one's were admitted before the run began. + */ +export function markdownDocumentation(path: string, source: string): string | undefined { + // The canonical splitter, not a delimiter search of this module's own: a + // document whose body repeats `---` is ordinary, and a second implementation + // of the rule is one release from disagreeing with the one that decides what + // actually runs. + const body = parseSource(path, source).content.trim(); + return body.length === 0 ? undefined : body; +} + +/** What a component with no authored documentation renders instead of prose. */ +export const NO_DOCUMENTATION = "No long-form documentation is available for this component."; + +/** A level-two heading, captured without its marker. */ +const HEADING = /^##\s+(.+?)\s*$/; +/** Any ATX heading, so a deeper one can be told from a section boundary. */ +const ANY_HEADING = /^(#{1,6})\s+/; +/** + * A fence line: its delimiter run, and whatever follows on the line. + * + * Three rules decide whether a line ends the block it is in, and getting any of + * them wrong reads an example as documentation. The closing run must be the same + * character, at *least* as long as the opener — so a four-backtick example may + * contain a three-backtick block — and it must carry nothing after it but + * whitespace. An *opening* fence may carry an info string (` ```mdx `); a + * closing one may not, so a same-length delimiter followed by text is still + * inside the example. + */ +const FENCE = /^ {0,3}(`{3,}|~{3,})(.*)$/; + +/** One source, parsed into the bundle's own prose and a section per component. */ +interface ParsedSource { + readonly bundle: string; + readonly sections: ReadonlyMap; +} + +/** + * Parse one `components.md`. + * + * Fences are tracked because a documentation file is mostly examples, and an + * example that writes `## Heading` inside a fenced block is showing Markdown + * rather than starting a section. Reading it as a section would silently move + * every following component's prose into the wrong entry. + */ +export function parseDocumentationSource(source: DocumentationSource): ParsedSource { + const lines = source.text.split(/\r?\n/); + const sections = new Map(); + const bundle: string[] = []; + let current: string[] = bundle; + /** The fence currently open, as the exact marker that opened it. */ + let fence: string | undefined; + + for (const line of lines) { + const fenced = FENCE.exec(line); + if (fenced !== null) { + const marker = fenced[1] ?? ""; + const trailing = fenced[2] ?? ""; + if (fence === undefined) { + // Opening: the info string is allowed and ignored. + fence = marker; + } else if ( + marker[0] === fence[0] && + marker.length >= fence.length && + trailing.trim().length === 0 + ) { + fence = undefined; + } + current.push(line); + continue; + } + if (fence !== undefined) { + current.push(line); + continue; + } + const heading = HEADING.exec(line); + // A level-two heading opens a section; `###` and deeper stay in the one + // they are written in, which is what lets a component's documentation have + // headings of its own. + const depth = ANY_HEADING.exec(line)?.[1]?.length; + if (heading === null || depth !== 2) { + current.push(line); + continue; + } + const name = heading[1] ?? ""; + // The canonical grammar rather than a second copy of it, so a dotted name + // like `File.Delete` or `PullRequest.Reviews` can be documented and looked + // up. A private regex here would have quietly excluded every namespaced + // component in the product. + if (!isComponentName(name)) { + throw new DocumentationIndexError( + `${source.asset} has the level-two heading "${name}", which is not a component name. ` + + "Every level-two heading in a component documentation file names one component.", + ); + } + if (sections.has(name)) { + throw new DocumentationIndexError( + `${source.asset} documents ${name} twice, so nothing says which section is current.`, + ); + } + current = []; + sections.set(name, current); + } + + return { + bundle: joined(bundle), + sections: new Map([...sections].map(([name, body]) => [name, joined(body)])), + }; +} + +/** A section's lines as one string, with the blank edges trimmed off. */ +function joined(lines: readonly string[]): string { + return lines.join("\n").trim(); +} + +/** + * Build the index every documentation reader shares. + * + * `known` is what each origin actually registers, so a heading naming something + * else is caught here rather than becoming an entry nothing can ever select. It + * is the catalog's own answer, which is what keeps the index and the catalog + * from disagreeing about which components exist. + */ +export function buildDocumentationIndex( + sources: readonly DocumentationSource[], + known: (owner: string) => ReadonlySet, +): DocumentationIndex { + const documentation = new Map>(); + const bundles = new Map(); + + for (const source of sources) { + const parsed = parseDocumentationSource(source); + const registered = known(source.owner); + const held = documentation.get(source.owner) ?? new Map(); + for (const [name, body] of parsed.sections) { + if (!registered.has(name)) { + throw new DocumentationIndexError( + `${source.asset} documents ${name}, which ${source.owner} does not supply. ` + + "Documentation for a component nothing there declares can never be selected.", + ); + } + // Across sources as well as within one: two files documenting one + // component of one package is the same ambiguity as one file doing it. + if (held.has(name)) { + throw new DocumentationIndexError( + `${name} is documented twice for ${source.owner}, so nothing says which is current.`, + ); + } + held.set(name, body); + } + documentation.set(source.owner, held); + if (parsed.bundle.length > 0) { + bundles.set(source.owner, parsed.bundle); + } + } + + // Exact coverage, not merely no surprises. A first-party package documents + // every public component it supplies: a member with no section is a component + // shipped without documentation, and letting it fall back to the sentence + // would make the product's own reference silently incomplete — the reader + // cannot tell "nobody has written this yet" from "this component has nothing + // to say". The fallback is for custom components, which no package governs. + for (const [owner, held] of documentation) { + const missing = [...known(owner)].filter((name) => !held.has(name)).sort(); + if (missing.length > 0) { + throw new DocumentationIndexError( + `${owner} supplies ${missing.length === 1 ? "a component" : "components"} with no ` + + `documentation: ${missing.join(", ")}. Every public component a first-party ` + + "package supplies has exactly one documentation section.", + ); + } + } + + return { + documentationFor(name: string, origin: ComponentOrigin): string | undefined { + const owner = owningPackage(origin); + return owner === undefined ? undefined : documentation.get(owner)?.get(name); + }, + bundleDocumentation(origin: ComponentOrigin): string | undefined { + const owner = owningPackage(origin); + return owner === undefined ? undefined : bundles.get(owner); + }, + }; +} diff --git a/packages/core/src/evaluation-profile.ts b/packages/core/src/evaluation-profile.ts new file mode 100644 index 000000000..7e45a1486 --- /dev/null +++ b/packages/core/src/evaluation-profile.ts @@ -0,0 +1,1106 @@ +/** + * What a trusted host allows a generated fragment to do, stated before any + * document code exists. + * + * `` is a public component: any author may write it, and canonical + * core owns what it means. That settles *which implementation runs* and nothing + * else — protection grants no authority. Canonical `` uses only + * authority already captured in this profile, selected by `allow`. So the + * ceiling lives here, in a value the host hands canonical execution at the + * installation boundary, before a document exists to ask for it. + * + * ## Host input and what execution captures are two shapes + * + * A host states {@link FragmentEvaluationInput}: entry tables, an optional + * Workspace snapshot operation, and whether it accepts the released `source` + * spelling. Canonical execution turns that into a captured profile of its own + * — every value deep-copied and frozen, every live operation read and bound + * once — and keeps the captured one. A host that mutates its own arrays, + * headers or tables after installation changes nothing about the execution, + * because the execution is no longer reading them. + * + * Only the input type is exported, and only from the trusted `core/host` + * surface, because that is where `ExecutionInstallation` already is. The + * captured profile, its bound operations and its revocation are unexported: + * ordinary core publishes no getter for active authority and no way to install + * a provider. + * + * ## One per execution + * + * Two profiles would be two answers to "what may a fragment do here", and + * choosing between them by installation order would make authority depend on + * assembly. So an execution accepts one and refuses two — even two identical + * ones, because a host that stated it twice has an assembly nobody validated. + * A host that stated none offers no evaluation at all, which `` + * refuses with when it is invoked rather than at installation: a document that + * never writes `` is not asking for a ceiling. + */ + +import type { Operation } from "effection"; + +import { CORE_ORIGIN } from "./components/registry.ts"; +import { + CAPABILITY_FORMS, + capabilityDefinition, + capabilityProps, + captureCapabilities, + FragmentCapabilityError, + REVOKED_CAPABILITY, +} from "./fragment-capabilities.ts"; +import type { + CapturedCapabilities, + FragmentCapability, + FragmentFetchAccess, + FragmentFileAccess, +} from "./fragment-capabilities.ts"; +import { isFormDispatcher } from "./invocation-identity.ts"; +import type { ComponentInvocation } from "./invocation-identity.ts"; +import type { FetchRequest } from "./fetch-request.ts"; +import { normalizeFetchRequest, requestRecord } from "./fetch-request.ts"; +import { CORE_REVISION } from "./generated-xmd.ts"; +import type { GeneratedRequest } from "./generated-xmd.ts"; +import type { FunctionComponent, FunctionComponentDefinition, Json, PropsSchema } from "./types.ts"; + +/** The two forms an element is actually written in. */ +export type FragmentForm = "self-closing" | "paired"; + +/** + * What a run retains about which implementation an entry runs. + * + * Three structural parts rather than one opaque string, because a continuation + * compares them and a reader has to be able to say which one moved. Never + * derived from a function: an implementation is not an identity, and + * serializing or inspecting one would make the retained policy depend on how a + * host happened to write its code. + */ +export interface FragmentIdentity { + /** The package or host that owns this implementation. */ + readonly origin: string; + /** Which component of that origin, including the form when forms differ. */ + readonly key: string; + /** Which version of it. A changed revision is a changed grant. */ + readonly revision: string; +} + +/** + * One component a generated fragment may name, as the host states it. + * + * A closed union of the two ways a host can say what is behind a name, and it + * is closed on purpose: a third shape would be a third answer to "where does + * this implementation come from", which is exactly the question an admission + * has to settle once. + * + * Neither arm carries a function. A capability names an operation canonical + * core supplies the body for; a component answer names something the ordinary + * import chain resolves, which canonical capture does once — before any + * document code — and then holds by value. What is journaled is the structural + * identity, the forms and the limits. + */ +export type FragmentEntry = CapabilityEntry | ComponentAnswerEntry; + +/** An entry core supplies the body for, bound to the host's own operations. */ +export interface CapabilityEntry { + readonly kind: "capability"; + /** The name a fragment writes. */ + readonly name: string; + readonly identity: FragmentIdentity; + /** + * The exact spellings this entry is admitted for. + * + * A component whose two spellings do different things is two entries under + * one name — `` reads and `…` writes — so the form + * travels with the identity and preflight chooses between them. + */ + readonly forms: readonly FragmentForm[]; + /** The contract this entry's props are validated against. */ + readonly props: PropsSchema; + /** + * What this entry does, for the agent being told it may write the name. + * + * The host's, because the entry is: canonical core supplies the operation + * behind a capability, and only the host knows what admitting it under this + * name means here. A narrowed catalog without it would name a component and + * say nothing about it, which is the one thing the catalog exists to avoid. + */ + readonly description?: string; + /** + * Which captured operation this entry runs. + * + * Not a definition, because a definition is where a provider lookup would go. + * A host names the capability and hands its private operations to the + * profile; canonical capture reads each one off exactly once and closes + * core's own body over the bound result. A host therefore chooses *what a + * fragment may do* and never *how it reaches it* + * (`fragment-capabilities.ts`). + */ + readonly capability: FragmentCapability; + /** + * The exact version-1 identity strings this entry is the successor of. + * + * Version-1 records retained one opaque string, chosen by whoever built the + * pinned entry: core wrote `@executablemd/core#File:read`, the workflow host + * wrote `@executablemd/workflow/composition/dir-v2#Dir`, and a host passing + * its own observation wrote whatever it liked. There is no rule that recovers + * four structural terms from one of those, and inferring one would be this + * module deciding on a host's behalf that two grants are the same. + * + * So it is stated rather than derived, by the party that owns the entry. + * Listing a string here is an assertion that this entry authorizes no more + * than the entry that string named — a continuation admitted under it resumes + * against this one. An entry that lists none is a new grant, and every + * version-1 record naming it refuses. + * + * Only a capability arm has this. The component-answer arm did not exist when + * version-1 records were written, so no such string ever described one. + */ + readonly legacy?: readonly string[]; + /** + * The exact requests this entry may perform, for an entry that performs any. + * + * Stated per entry rather than per profile, because a ceiling belongs to the + * identity it bounds: two entries performing requests are two ceilings, and + * flattening them would let one entry's limit admit another's request. + */ + readonly requests?: readonly GeneratedRequest[]; +} + +/** + * An entry whose implementation the ordinary import chain answers for. + * + * The host states *what it expects* — the name, the structural identity a + * provider must claim, and the forms — and states no implementation at all. + * There is no definition, function, resolver, provider operation or context + * handle on this arm to hold one. + * + * Canonical capture resolves the name once, before the root import and before + * any document code, through the complete ordinary `Component.importComponent` + * chain; reads the identity the provider claimed on that exact final answer; + * compares it with what this entry expects; retains the answer defensively; and + * seals the result. Props and the callable definition are derived from that + * answer rather than restated here, so a host cannot describe a contract the + * implementation does not have. + * + * Resolution happens at capture and never again. A fragment invokes the sealed + * snapshot, and a continuation resolves once more in its own capture and + * reconciles before any effect. Calling a retained resolver later would see + * document-time middleware, which is the whole thing this avoids. + * + * This arm states no version-1 alias, and has nowhere to put one: version-1 + * records predate it, so no string an older run committed ever described an + * implementation the import chain answered for. + */ +export interface ComponentAnswerEntry { + readonly kind: "component-answer"; + readonly name: string; + /** What a provider must have claimed for this name, exactly. */ + readonly identity: FragmentIdentity; + readonly forms: readonly FragmentForm[]; + readonly description?: string; +} + +/** + * How a host answers what its Workspace currently retains. + * + * An operation rather than a value, because a run's own progress legitimately + * advances it: every committed mutation retains another immutable root. A + * basis captured once at installation would be stale by the second evaluation + * in one run, so canonical `` asks this per invocation instead. + * + * An ordinary host has none — it evaluates against no Workspace at all, which + * is a different statement from evaluating against an empty one. + */ +export interface FragmentWorkspaceAccess { + snapshot(): Operation<{ readonly roots: readonly string[]; readonly current: string }>; +} + +/** One host's complete statement of what a fragment may do. */ +export interface FragmentEvaluationInput { + /** + * The entries the `read` class resolves to. + * + * For both standard profiles this is exactly the self-closing ``. + * `` joins it only where the host also states the exact requests it + * may perform. + */ + readonly read: readonly FragmentEntry[]; + /** + * The entries the `write` class resolves to, when this host offers the class. + * + * For both standard profiles this is exactly paired ``, paired `` + * and self-closing ``. It does not include the read table: + * `write` selects mutation and nothing else. + */ + readonly write?: readonly FragmentEntry[]; + /** How this host answers for its Workspace, when it evaluates against one. */ + readonly workspace?: FragmentWorkspaceAccess; + /** + * The exact filesystem operations the admitted entries run. + * + * Required by every file and directory entry, and read off this object once + * at capture. A profile that admits `` without stating them is a + * profile that admitted an operation it cannot perform, and is refused rather + * than falling through to whatever provider a document installed. + */ + readonly files?: FragmentFileAccess; + /** The exact transport an admitted `` performs its request through. */ + readonly fetch?: FragmentFetchAccess; + /** + * The effective Fetch timeout this host resolved, in milliseconds. + * + * Resolved once, by the host, when it builds the profile. Reading it from + * context during preflight would let the ceiling a fragment is held to depend + * on where the reading happened. + */ + readonly fetchTimeout?: number; + /** + * Whether this host accepts the released `source` spelling beside `text`. + * + * The workflow profile does, silently, because documents were written against + * it before `text` existed. The ordinary profile does not: it never shipped + * that spelling. + */ + readonly deprecatedSourceAlias?: boolean; +} + +/** + * Core's ``, admitted to observe and not to write. + * + * `` reads when it has no content and writes when it has some, so one + * unconstrained entry would admit a write. The form travels with the identity + * and preflight decides between the two — before the first effect, rather than + * inside a body that has already been entered. + * + * An admitted read reaches the `readTextFile` the host handed the profile and + * nothing else. Under a workflow run that is the transaction-bound operation, + * so the read still crosses the run's own transaction; what it no longer does + * is resolve a provider through the contextual API at the moment it runs. + */ +export function fileReadEntry(): CapabilityEntry { + return coreEntry("File", "File:read", "file:read"); +} + +/** Core's `…`, admitted to write and not to read. */ +export function fileWriteEntry(): CapabilityEntry { + return coreEntry("File", "File:write", "file:write"); +} + +/** + * Core's ``, in the one form it has. + * + * One name, one identity: unlike ``, whose two spellings do different + * things, this answers the self-closing form alone. Stating the form is what + * puts the decision in preflight, before the fragment's first effect — a paired + * spelling costs an earlier admitted element nothing. + */ +export function fileDeleteEntry(): CapabilityEntry { + return coreEntry("File.Delete", "File.Delete", "file:delete"); +} + +/** + * A directory an admitted fragment may create, under the name the host gives it. + * + * The name is the host's because the component is: the workflow calls it + * ``, and an ordinary run admits no such thing. What core supplies is the + * operation — `ensureDirectory`, from the profile — so the body a fragment + * reaches is not the ordinary registration and cannot be composed around. + */ +export function directoryEntry( + identity: FragmentIdentity, + name: string, + /** + * The version-1 strings this host's own directory entry succeeds. + * + * The host's to state, because the identity is: core never wrote one of + * these, so it has nothing to assert about which older grant this entry is + * the same as. + */ + legacy?: readonly string[], +): CapabilityEntry { + return { + kind: "capability", + name, + identity, + forms: [...CAPABILITY_FORMS["directory:ensure"]], + capability: "directory:ensure", + props: capabilityProps("directory:ensure"), + description: CORE_DESCRIPTIONS["directory:ensure"], + ...(legacy === undefined ? {} : { legacy }), + }; +} + +/** + * Core's ``, bounded to exactly these requests. + * + * The ceiling is not optional and this constructor does not decide it: an + * unbounded network read is a different grant from an admitted one, and a host + * that states no request admits `` not at all rather than admitting it + * and refusing everything it asks for. + */ +export function fetchEntry(requests: readonly GeneratedRequest[]): CapabilityEntry { + return { ...coreEntry("Fetch", "Fetch", "fetch"), requests }; +} + +/** What core's own entries tell an agent they do. */ +const CORE_DESCRIPTIONS: Readonly> = Object.freeze({ + "file:read": "Read one file and render its text. Written self-closing.", + "file:write": "Write what it renders to one file. Written with content.", + "file:delete": "Remove one file. Written self-closing.", + "directory:ensure": + "Make one directory exist, and resolve the paths inside it against it. Written with content.", + fetch: "Perform one admitted HTTP read. Written self-closing.", +}); + +/** + * The exact strings each of core's own entries was retained as under version 1. + * + * Enumerated literally rather than assembled from the origin and the key. + * There was never a rule that produced these — a released build wrote each one + * where it built the pinned entry — so a template here would be this module + * inventing the convention the whole design says does not exist, and a later + * change to `CORE_ORIGIN` would silently rewrite what an old journal is + * compared against. Core states them because core owns these entries and is the + * party that can assert the current one authorizes no more than the old did. + */ +const CORE_LEGACY: Readonly> = Object.freeze({ + "file:read": Object.freeze(["@executablemd/core#File:read"]), + "file:write": Object.freeze(["@executablemd/core#File:write"]), + "file:delete": Object.freeze(["@executablemd/core#File.Delete"]), + // Core never pinned a directory entry under version 1; the workflow host did, + // and states its own alias. + "directory:ensure": Object.freeze([]), + fetch: Object.freeze(["@executablemd/core#Fetch"]), +}); + +function coreEntry(name: string, key: string, capability: FragmentCapability): CapabilityEntry { + const legacy = CORE_LEGACY[capability]; + return { + kind: "capability", + name, + identity: { origin: CORE_ORIGIN, key, revision: CORE_REVISION }, + forms: [...CAPABILITY_FORMS[capability]], + capability, + props: capabilityProps(capability), + description: CORE_DESCRIPTIONS[capability], + ...(legacy.length === 0 ? {} : { legacy }), + }; +} + +/** + * One provider-backed name, already resolved and reconciled. + * + * Produced by canonical execution's capture step before this module is asked to + * seal anything: the chain has answered, the provider's claim has been read off + * that exact final object, it has been compared with what the host entry + * expects, and the answer has been retained defensively. What arrives here is + * the settled result, so nothing in this module resolves, looks up, or holds a + * resolver it could call later. + */ +export interface ResolvedAnswer { + /** Core's own retained copy, which is what a fragment invokes. */ + readonly definition: FunctionComponentDefinition; +} + +/** Every provider-backed name this capture resolved, by name. */ +export type ResolvedAnswers = ReadonlyMap; + +/** + * One captured entry: frozen structural data beside one sealed implementation. + * + * `kind` survives capture because the retained record keeps it: a continuation + * comparing identities has to know whether the thing behind a name was core's + * own operation or a provider's answer, and the two are different grants. + */ +export interface CapturedEntry { + readonly name: string; + readonly identity: FragmentIdentity; + readonly forms: readonly FragmentForm[]; + readonly props: PropsSchema; + readonly kind: "capability" | "component-answer"; + /** Which operation this runs, for a capability entry. */ + readonly capability?: FragmentCapability; + /** What the admitted vocabulary says this entry does, when the host said. */ + readonly description?: string; + /** + * What a fragment invokes. + * + * For a capability, core's own body closed over the operations this capture + * bound. For a component answer, core's retained copy of the exact final + * answer the import chain gave — never the object the chain was holding, and + * never something re-resolved later. + */ + readonly definition: FunctionComponentDefinition; + /** + * The form authority underneath this entry's implementation, when the sealed + * definition wraps one. + * + * A component answer runs behind a lifetime guard canonical capture built, so + * the function on the definition is core's rather than the provider's. A + * provider whose own answer dispatches on the authored form would otherwise + * lose that dispatch, because what a selection records as the form authority + * is read off the definition it was handed. So the authority travels + * explicitly, and a capability — whose definition is core's own dispatcher — + * states none. + */ + readonly dispatch?: unknown; + /** + * The version-1 identity strings this entry succeeds, copied by value. + * + * Read off the host's array once at capture and frozen, like everything else + * here: a host that appends to its own list afterwards is appending to an + * object nothing is looking at, so it cannot widen which retained records + * reconcile to this entry from inside its own `install()`. + */ + readonly legacy?: readonly string[]; + /** This entry's own ceiling, normalized once and canonically ordered. */ + readonly requests?: readonly FetchRequest[]; +} + +/** What canonical execution keeps, and what canonical `` reads. */ +export interface CapturedProfile { + readonly read: readonly CapturedEntry[]; + readonly write: readonly CapturedEntry[]; + readonly workspace?: FragmentWorkspaceAccess; + readonly deprecatedSourceAlias: boolean; + /** + * Begin one fragment, and answer with how to end it. + * + * Reads the host's working directory once per fragment rather than once per + * element, and restores whatever was current afterwards — so a fragment + * produced inside another fragment's producer leaves the outer one where it + * was. + */ + readonly enterFragment: () => Operation<() => void>; + /** Whether this profile's bound operations are still usable. */ + readonly live: () => boolean; + /** + * End them. + * + * Called by canonical execution at teardown, and reachable from nowhere a + * document can put code: an operation a fragment or a handler retained past + * the execution refuses rather than acting on a filesystem the run no longer + * holds a transaction for. + */ + readonly revoke: () => void; +} + +/** What an execution that was offered no evaluation profile refuses with. */ +export const NO_PROFILE = + " has no evaluation profile here: this host stated none, so nothing established " + + "what a generated fragment may do. A profile is the host's own statement of the maximum " + + "authority an evaluation has, and `allow` only narrows it."; + +/** What an execution offered two evaluation profiles refuses with. */ +export const TWO_PROFILES = + "two installations stated the evaluation profile this execution offers. One execution offers " + + "one maximum authority, so what a generated fragment may do is never a question of assembly " + + "order."; + +/** What a profile whose execution has ended refuses with. */ +export const REVOKED = + "the evaluation profile this fragment was admitted under belongs to an execution that has " + + "ended."; + +/** A profile a host stated in a shape canonical execution cannot capture. */ +export class EvaluationProfileError extends Error { + override name = "EvaluationProfileError"; +} + +/** + * One entry this execution has copied, before it knows what is behind a + * provider-backed name. + * + * Everything a host stated is already settled here — the identity, the forms, + * the schema for a capability, the ceiling. What is missing is exactly the part + * a host does not state: a component answer's implementation and the contract + * that comes with it. + */ +interface PreparedEntry { + readonly name: string; + readonly identity: FragmentIdentity; + readonly forms: readonly FragmentForm[]; + readonly kind: "capability" | "component-answer"; + readonly description?: string; + readonly props?: PropsSchema; + readonly capability?: FragmentCapability; + readonly definition?: FunctionComponentDefinition; + readonly legacy?: readonly string[]; + readonly requests?: readonly FetchRequest[]; +} + +/** + * One host's profile, copied and bound, waiting only for its provider answers. + * + * Preparation is where a host stops being consulted: the tables are copied, the + * identities and forms are frozen, the ceilings are normalized, and every live + * operation is read off the host's objects once and bound behind this + * execution's revocation. A host that mutates its own arrays, headers or tables + * afterwards changes nothing. + * + * What preparation cannot do is decide what a provider-backed name resolves to: + * that takes the complete ordinary import chain, which does not exist until the + * providers have installed. So it publishes the names that need resolving and + * seals afterwards. + */ +export interface PreparedProfile { + /** + * The provider-backed names this profile admits, with what it expects of each. + * + * Empty for a capability-only profile, which performs no component-chain + * lookup at all — a host that offers no provider answer pays for none. + */ + readonly answered: readonly { readonly name: string; readonly identity: FragmentIdentity }[]; + /** Whether the operations this preparation bound are still usable. */ + readonly live: () => boolean; + /** End them. Registered by canonical execution before any installation runs. */ + readonly revoke: () => void; + /** The completed profile, from the answers this execution resolved. */ + seal(answers: ResolvedAnswers): Operation; +} + +/** + * Copy and bind one host's profile, before any installation runs. + * + * Everything structural is copied and frozen here, so a host mutating its own + * arrays, schemas, headers or tables afterwards changes nothing this execution + * does. What is not copied is the implementation itself: a function is the one + * thing a profile must keep by reference, and it is kept behind a revocation + * this execution owns rather than handed onward. + */ +export function* prepareEvaluationProfile( + input: FragmentEvaluationInput, +): Operation { + // Every live operation is read off the host's objects here, once, before a + // single installation has run. What comes back is bound and revocable, and + // the host's own objects are never consulted again. + const capabilities = captureCapabilities({ + ...(input.files === undefined ? {} : { files: input.files }), + ...(input.fetch === undefined ? {} : { fetch: input.fetch }), + }); + // One definition per *name*, built from every capability that name holds + // across both tables. `` observing and `…` writing are + // two entries under one name, and the evaluator resolves an import by name — + // so they arrive as one definition whose dispatch separates the two + // spellings, exactly as the ordinary `` does. + const built = buildDefinitions(input, capabilities); + const read = yield* prepareEntries(input.read ?? [], input.fetchTimeout, built); + const write = yield* prepareEntries(input.write ?? [], input.fetchTimeout, built); + if (read.length === 0 && write.length === 0) { + throw new EvaluationProfileError( + "an evaluation profile states no component at all. A host offering evaluation states what " + + "a fragment may do; one that offers none states no profile.", + ); + } + // One lookup per distinct name, however many entries and tables hold it: a + // name resolves to one implementation, and asking twice would be two chances + // for the chain to answer differently. + const answered = answeredNames([...read, ...write]); + const workspace = input.workspace === undefined ? undefined : bindWorkspace(input.workspace); + const deprecatedSourceAlias = input.deprecatedSourceAlias === true; + return Object.freeze({ + answered: Object.freeze( + [...answered.entries()].map(([name, identity]) => Object.freeze({ name, identity })), + ), + live: () => capabilities.live(), + revoke: () => { + capabilities.revoke(); + }, + // deno-lint-ignore require-yield + *seal(answers: ResolvedAnswers): Operation { + // One sealed implementation per name, built before either table is + // sealed. A name that holds two entries — the self-closing spelling in + // `read` and the paired one in `write` — is one component seen from two + // sides, so both entries carry the same object: two guards over one + // answer would be two lifetimes for one implementation, and which of + // them a fragment reached would depend on which table admitted it. + const sealed = sealAnswers(answered, answers, capabilities); + return Object.freeze({ + read: sealEntries(read, sealed), + write: sealEntries(write, sealed), + ...(workspace === undefined ? {} : { workspace }), + deprecatedSourceAlias, + enterFragment: () => capabilities.enterFragment(), + live: () => capabilities.live(), + revoke: () => { + capabilities.revoke(); + }, + }); + }, + }); +} + +/** + * The workspace access, read once and bound. + * + * Bound rather than kept as the host's object, so replacing the method on that + * object after installation reaches nothing. + */ +function bindWorkspace(access: FragmentWorkspaceAccess): FragmentWorkspaceAccess { + const snapshot = access.snapshot.bind(access); + return Object.freeze({ + *snapshot() { + const answered = yield* snapshot(); + // Copied on the way out too: what the host answered with is its own + // object, and the basis this invocation is admitted under must not move + // afterwards. + return Object.freeze({ + roots: Object.freeze([...answered.roots]), + current: answered.current, + }); + }, + }); +} + +/** + * One definition per admitted name, built before any entry is captured. + * + * Built here rather than per entry because a name's definition depends on + * *every* capability that name holds: a profile admitting `` to read + * and `…` to write is describing one component with two spellings, + * and which body a spelling reaches is dispatch's decision rather than the + * table's. + * + * A ceiling belongs to the entry that states it, so the requests handed to a + * name's definition are that name's own — never a flattened set, which would + * let one entry's limit admit another's request. + */ +function buildDefinitions( + input: FragmentEvaluationInput, + capabilities: CapturedCapabilities, +): Map { + const admitted = new Map< + string, + { "self-closing"?: FragmentCapability; paired?: FragmentCapability } + >(); + const ceilings = new Map(); + for (const entry of [...(input.read ?? []), ...(input.write ?? [])]) { + // A component answer has no capability behind it: canonical capture + // resolves its implementation through the ordinary import chain and derives + // the definition from that answer, so there is nothing for this to build. + if (entry.kind !== "capability") { + continue; + } + const held = admitted.get(entry.name) ?? {}; + for (const form of CAPABILITY_FORMS[entry.capability]) { + held[form] = entry.capability; + } + admitted.set(entry.name, held); + if (entry.requests !== undefined) { + ceilings.set(entry.name, entry.requests); + } + } + const built = new Map(); + for (const [name, forms] of admitted) { + built.set( + name, + capabilityDefinition(name, forms, capabilities, normalizedCeiling(ceilings.get(name), input)), + ); + } + return built; +} + +/** + * One name's ceiling, normalized on the same terms the retained policy is. + * + * The component performs the admitted request rather than the props, so the + * value it holds has to be the one preflight compared against — normalized + * once, against the host's own resolved timeout, rather than read again from a + * context wherever the performance happens. + */ +function normalizedCeiling( + requests: readonly GeneratedRequest[] | undefined, + input: FragmentEvaluationInput, +): readonly FetchRequest[] { + if (requests === undefined) { + return []; + } + return requests.map((request) => normalizeFetchRequest({ ...request }, input.fetchTimeout)); +} + +function* prepareEntries( + entries: readonly FragmentEntry[], + fetchTimeout: number | undefined, + built: Map, +): Operation { + const prepared: PreparedEntry[] = []; + for (const entry of entries) { + prepared.push(yield* prepareEntry(entry, fetchTimeout, built)); + } + return Object.freeze(prepared); +} + +function* prepareEntry( + entry: FragmentEntry, + fetchTimeout: number | undefined, + built: Map, +): Operation { + const forms = canonicalForms(entry.forms); + if (forms.length === 0) { + throw new EvaluationProfileError( + `an evaluation profile admitted "${entry.name}" for no authored form.`, + ); + } + const identity = captureIdentity(entry.identity, entry.name); + const described = + typeof entry.description === "string" && entry.description.length > 0 + ? { description: entry.description } + : {}; + + if (entry.kind === "component-answer") { + // Nothing about the implementation is settled here. What this entry states + // is which name, and which identity a provider must have claimed for it; + // the chain answers later, and props come from that answer rather than from + // the host, so a host cannot describe a contract the implementation lacks. + return Object.freeze({ + name: entry.name, + identity, + forms, + kind: "component-answer" as const, + ...described, + }); + } + + const requests = + entry.requests === undefined + ? undefined + : yield* captureRequests(entry.requests, fetchTimeout, entry.name); + // Detached, so a host that edits its schema afterwards does not change what a + // fragment's props are validated against. + const props = detach(entry.props); + // The definition every entry under this name shares, built before any entry + // was prepared. + const definition = built.get(entry.name); + if (definition === undefined) { + throw new EvaluationProfileError( + "an evaluation profile admitted a name with no operation behind it.", + ); + } + const legacy = captureLegacy(entry.legacy, entry.name); + return Object.freeze({ + name: entry.name, + identity, + forms, + props, + kind: "capability" as const, + capability: entry.capability, + ...described, + definition, + ...(legacy === undefined ? {} : { legacy }), + ...(requests === undefined ? {} : { requests }), + }); +} + +/** + * The version-1 aliases this entry states, copied and checked. + * + * Copied because the host's array is the host's, and checked because an alias + * is a comparison term: an empty string would reconcile against nothing + * usefully, and a duplicate would say one thing twice. A host stating an empty + * list has stated no alias, which is the same as stating none. + */ +function captureLegacy( + legacy: readonly string[] | undefined, + name: string, +): readonly string[] | undefined { + if (legacy === undefined) { + return undefined; + } + const held = new Set(); + for (const alias of legacy) { + if (typeof alias !== "string" || alias.length === 0) { + throw new EvaluationProfileError( + `an evaluation profile stated an empty version-1 identity for "${name}". An alias is a ` + + "string an older run actually retained, and there is no such record holding nothing.", + ); + } + held.add(alias); + } + return held.size === 0 ? undefined : Object.freeze([...held]); +} + +/** + * The provider-backed names this profile admits, with the one identity each of + * them states. + * + * A name is one component, so it has one implementation and one identity. Two + * entries under one name are the two spellings of that component — the + * self-closing one admitted to observe and the paired one to mutate — and + * stating a second identity for the second spelling would be stating that the + * name means two things, with which of them a fragment reached decided by which + * table admitted it. That is not a narrower grant than the host wrote down; it + * is an ambiguous one, so it refuses at capture rather than resolving by + * position. + * + * A name a host holds as both a capability and a component answer is the same + * ambiguity with the sharper edge: canonical core would supply one body and the + * import chain the other, which are different grants under one spelling. + */ +function answeredNames(prepared: readonly PreparedEntry[]): ReadonlyMap { + const answered = new Map(); + const kinds = new Map(); + for (const entry of prepared) { + const held = kinds.get(entry.name); + if (held !== undefined && held !== entry.kind) { + throw new EvaluationProfileError( + `an evaluation profile admitted "${entry.name}" both as an operation core supplies the ` + + "body for and as an implementation the import chain answers for. One name is one " + + "component, and those are different grants.", + ); + } + kinds.set(entry.name, entry.kind); + if (entry.kind !== "component-answer") { + continue; + } + const stated = answered.get(entry.name); + if (stated === undefined) { + answered.set(entry.name, entry.identity); + continue; + } + if ( + stated.origin !== entry.identity.origin || + stated.key !== entry.identity.key || + stated.revision !== entry.identity.revision + ) { + throw new EvaluationProfileError( + `an evaluation profile admitted "${entry.name}" as ${spelling(stated)} and as ` + + `${spelling(entry.identity)}. One name states one identity, however many forms and ` + + "tables hold it.", + ); + } + } + return answered; +} + +/** One identity as a reader sees it in a refusal. */ +function spelling(identity: FragmentIdentity): string { + return `${identity.origin}#${identity.key}@${identity.revision}`; +} + +/** One provider-backed name's sealed implementation, shared by every entry. */ +interface SealedAnswer { + readonly props: PropsSchema; + readonly definition: FunctionComponentDefinition; + readonly dispatch?: unknown; +} + +/** + * One sealed implementation per provider-backed name, built once per capture. + * + * Built here rather than per entry because a name has one implementation: the + * lifetime guard, the detached schema and the form authority are properties of + * that implementation, and building them twice would give one component two + * bodies whose only difference was which table asked for it. + */ +function sealAnswers( + answered: ReadonlyMap, + answers: ResolvedAnswers, + capabilities: CapturedCapabilities, +): ReadonlyMap { + const sealed = new Map(); + for (const name of answered.keys()) { + // Resolved once, before any document code, through the complete ordinary + // import chain — and already reconciled against what this entry expects. + const resolved = answers.get(name); + if (resolved === undefined) { + throw new EvaluationProfileError( + `an evaluation profile admitted "${name}" as a component answer, and this execution ` + + "resolved none for it.", + ); + } + const answer = resolved.definition; + const inner = answer.fn; + sealed.set( + name, + Object.freeze({ + props: detach(answer.props), + definition: Object.freeze({ ...answer, fn: bounded(inner, capabilities) }), + // The provider's own dispatcher, when its answer has one. The + // definition above runs behind core's lifetime guard, so what a + // selection would read off it is core's function rather than the + // provider's — and a form authority read off the wrong function selects + // no body at all. + ...(isFormDispatcher(inner) ? { dispatch: inner } : {}), + }), + ); + } + return sealed; +} + +function sealEntries( + prepared: readonly PreparedEntry[], + sealed: ReadonlyMap, +): readonly CapturedEntry[] { + return Object.freeze(prepared.map((entry) => sealEntry(entry, sealed))); +} + +function sealEntry(entry: PreparedEntry, sealed: ReadonlyMap): CapturedEntry { + const described = entry.description === undefined ? {} : { description: entry.description }; + if (entry.kind === "component-answer") { + const answer = sealed.get(entry.name); + if (answer === undefined) { + throw new EvaluationProfileError( + `an evaluation profile admitted "${entry.name}" as a component answer, and this ` + + "execution resolved none for it.", + ); + } + return Object.freeze({ + name: entry.name, + identity: entry.identity, + forms: entry.forms, + props: answer.props, + kind: "component-answer" as const, + ...described, + definition: answer.definition, + ...(answer.dispatch === undefined ? {} : { dispatch: answer.dispatch }), + }); + } + const definition = entry.definition; + const props = entry.props; + if (definition === undefined || props === undefined) { + throw new EvaluationProfileError( + "an evaluation profile admitted a name with no operation behind it.", + ); + } + return Object.freeze({ + name: entry.name, + identity: entry.identity, + forms: entry.forms, + props, + kind: "capability" as const, + ...(entry.capability === undefined ? {} : { capability: entry.capability }), + ...described, + definition, + ...(entry.legacy === undefined ? {} : { legacy: entry.legacy }), + ...(entry.requests === undefined ? {} : { requests: entry.requests }), + }); +} + +/** + * One provider's implementation, held to this execution's lifetime. + * + * A capability reaches the host's operations through bindings this capture + * already revokes, so a capability body retained past teardown refuses on its + * own. A provider's answer reaches whatever the provider closed over, which this + * execution never saw and cannot revoke — so the *entry* carries the lifetime + * instead, and a definition somebody kept refuses rather than running against a + * run that is over. + * + * The guard is what canonical capture builds the entry's definition around, not + * a wrapper placed over one afterwards: this function is the implementation the + * entry has ever had, so nothing about which body a form selects moves. + */ +function bounded( + inner: FunctionComponentDefinition["fn"], + capabilities: CapturedCapabilities, +): FunctionComponent { + if (typeof inner !== "function") { + throw new EvaluationProfileError( + "an evaluation profile admitted a component answer with no invocable implementation.", + ); + } + const implementation = inner; + return function* held( + props: Record, + invocation: ComponentInvocation, + ): Operation { + if (!capabilities.live()) { + throw new FragmentCapabilityError(REVOKED_CAPABILITY); + } + return yield* implementation(props, invocation); + }; +} + +function captureIdentity(identity: FragmentIdentity, name: string): FragmentIdentity { + const { origin, key, revision } = identity; + if ( + typeof origin !== "string" || + origin.length === 0 || + typeof key !== "string" || + key.length === 0 || + typeof revision !== "string" || + revision.length === 0 + ) { + throw new EvaluationProfileError( + `an evaluation profile admitted "${name}" without a complete identity. An entry states the ` + + "origin, key and revision a continuation is compared against.", + ); + } + return Object.freeze({ origin, key, revision }); +} + +/** The forms an entry is admitted for: a non-empty set in canonical order. */ +const FORMS: readonly FragmentForm[] = ["self-closing", "paired"]; + +function canonicalForms(forms: readonly FragmentForm[]): readonly FragmentForm[] { + const held = new Set(forms); + return Object.freeze(FORMS.filter((form) => held.has(form))); +} + +/** + * One entry's ceiling, normalized once and canonically ordered. + * + * Normalized here rather than at preflight so the comparison a continuation + * makes is one normalization rather than two readings that could disagree, and + * ordered so two hosts stating the same requests in different orders state the + * same ceiling. The timeout is the host's resolved value: reading it from + * context later would make the ceiling depend on where it was read. + */ +function* captureRequests( + requests: readonly GeneratedRequest[], + fetchTimeout: number | undefined, + name: string, +): Operation { + if (requests.length === 0) { + throw new EvaluationProfileError( + `an evaluation profile admitted "${name}" as a request entry with no request it may perform.`, + ); + } + const normalized: FetchRequest[] = []; + for (const request of requests) { + // Normalized against the host's own resolved bound rather than a context + // read: a ceiling compared across a suspension cannot depend on where the + // comparison happened. + const prepared = normalizeFetchRequest({ ...request }, fetchTimeout); + normalized.push( + Object.freeze({ + url: prepared.url, + method: prepared.method, + headers: Object.freeze({ ...prepared.headers }), + ...(prepared.timeout === undefined ? {} : { timeout: prepared.timeout }), + }), + ); + } + // Deduplicated and ordered by the record a continuation compares, so one + // ceiling stated twice is one ceiling. + const seen = new Map(); + for (const request of normalized) { + seen.set(JSON.stringify(requestRecord(request)), request); + } + return Object.freeze( + [...seen.entries()] + .sort(([one], [other]) => (one < other ? -1 : one > other ? 1 : 0)) + .map(([, request]) => request), + ); +} + +/** A frozen deep copy of one JSON value the host stated. */ +function detach(value: T): T { + return freeze(structuredClone(value)); +} + +function freeze(value: T): T { + if (Array.isArray(value)) { + for (const member of value) { + freeze(member); + } + Object.freeze(value); + return value; + } + if (typeof value === "object" && value !== null) { + for (const member of Object.values(value)) { + freeze(member); + } + Object.freeze(value); + return value; + } + return value; +} diff --git a/packages/core/src/execute.ts b/packages/core/src/execute.ts index 7e516dfd4..90d72defa 100644 --- a/packages/core/src/execute.ts +++ b/packages/core/src/execute.ts @@ -137,8 +137,36 @@ import { parseFormDeclaration, } from "./invocation-identity.ts"; import type { IdentityComponent } from "./invocation-identity.ts"; -import { ExecutionImports } from "./components/import-authority.ts"; -import type { ExpansionAuthority, ImportTier } from "./components/import-authority.ts"; +import { + CanonicalImports, + ExecutionImports, + identityRecord, +} from "./components/import-authority.ts"; +import type { + AnswerIdentity, + ExpansionAuthority, + ImportTier, +} from "./components/import-authority.ts"; +import { PROTECTED_COMPONENTS, ProtectedImports } from "./components/protected.ts"; +import { rootSyntaxReference } from "./syntax-reference.ts"; +import { capturedDocumentation } from "./documentation-api.ts"; +import { + EvaluationProfileError, + prepareEvaluationProfile, + TWO_PROFILES, +} from "./evaluation-profile.ts"; +import type { + FragmentEvaluationInput, + FragmentIdentity, + PreparedProfile, + ResolvedAnswer, + ResolvedAnswers, +} from "./evaluation-profile.ts"; +import { componentAnswerRegistrar } from "./component-answers.ts"; +import type { ComponentAnswerInstallation } from "./component-answers.ts"; +import { packagedAssetReader } from "./component-documentation.ts"; +import type { DocumentationContribution, DocumentationReader } from "./component-documentation.ts"; +import type { SyntaxSymbolsProvider } from "./syntax-reference.ts"; import type { WorkflowComponentBundle, WorkflowImportAuthority } from "./components/bundle.ts"; import type { CodeBlockContext, CodeBlockResult, EvalEnv } from "./types.ts"; import { readRootSource, rootSourcePath } from "./root-source.ts"; @@ -247,7 +275,16 @@ type DurableSelection = * the same way the live run did — by being inside the same declaration's body * when it asks. */ - | { kind: "declared-private"; origin: string }; + | { kind: "declared-private"; origin: string } + /** + * A component canonical core protects. + * + * Nothing but the kind is retained. The name is already the record's identity, + * and what the name resolves to is core's own table rather than anything a + * host, a registry or a checkout supplies — so a replay reconstructs it the + * way the live run did, by asking the copy of core that is running. + */ + | { kind: "protected" }; /** * What a recorded import decided, read as a closed protocol. @@ -282,6 +319,13 @@ function readDurableSelection(value: unknown): DurableSelection | undefined { return { kind: "declared-private", origin }; } + if (kind === "protected") { + if (members !== 1) { + return undefined; + } + return { kind: "protected" }; + } + if (kind === "declared-markdown") { const origin = record["origin"]; const digest = record["digest"]; @@ -394,119 +438,142 @@ function targetFailureRecord(failure: DocumentTargetFailure): TargetFailureRecor }; } -function* durableImportComponent( +/** + * Everything one import needs to decide a name and build what it decided. + * + * One value rather than eight parameters, because two callers pass it: the + * ordinary durable terminal below, and canonical execution's own private + * resolution of the provider-backed names an evaluation profile admits. The two + * differ in whether the decision is journaled, and in nothing else — which is + * the property this shape exists to make true rather than to promise. + */ +interface ImportInputs { + readonly searchPaths: string[]; + readonly registry: ComponentRegistry; + readonly bundle: WorkflowImportAuthority | undefined; + readonly declared: DeclaredImports | undefined; + readonly guarded: ReadonlyMap; +} + +/** + * What resolving this name decides, before anything is built from it. + * + * The root import reads its own source here, so the journal holds the root's + * identity and its text: a replay restores both without reading anything, + * whether the source was a file or supplied. The selector resolves here too, + * against the text this decision is about to record, so the exact target the run + * executed is part of the record rather than something a later read has to + * rediscover. + */ +function* selectImport( name: string, root: RootDocumentSource | undefined, - searchPaths: string[], - registry: ComponentRegistry, - position: Readonly | undefined, - bundle: WorkflowImportAuthority | undefined, - declared: DeclaredImports | undefined, -): Workflow { - // Taken before the durable operation and outside it, because the offer is - // canonical core's own and a replay has to reach this the same way the live - // run did: the element asking is inside the declaration's body, or it is not. - const claimed = name === "__root__" ? undefined : declared?.claim(name); - const recorded = yield createDurableOperation( - // The root import is the run's own entry rather than an authored element, - // so it carries no source however it was reached. - { type: "import_component", name, ...(root ? {} : sourceDescription(position)) }, - function* (): Operation { - if (name === "__root__" && root) { - // Inside the durable operation, so the journal holds the root's identity - // and its text: a replay restores both without reading anything, whether - // the source was a file or supplied. - // - // The selector resolves here too, against the text this operation is - // about to record, so the exact target the run executed is part of the - // record rather than something a later read has to rediscover. Only the - // exact target is recorded — a glob describes what the caller asked - // for, not what ran. - const path = rootSourcePath(root); - const content = yield* readRootSource(root); - if (root.target === undefined) { - return { kind: "repository", path, content }; - } - const resolved = resolveDocumentTarget(path, content, root.target); - if (resolved.ok) { - return { kind: "repository", path, content, target: resolved.value }; - } - const failure = asDocumentTargetError(resolved.error); - if (failure === undefined) { - throw resolved.error; - } - return { - kind: "target-failure", - path, - content, - failure: targetFailureRecord(failure.data), - }; - } - - if (claimed !== undefined) { - return { kind: "declared-private", origin: claimed.origin }; - } + inputs: ImportInputs, + /** The private offer this element carries, taken before the decision. */ + claimed: PrivateOffer, +): Operation { + if (name === "__root__" && root) { + const path = rootSourcePath(root); + const content = yield* readRootSource(root); + if (root.target === undefined) { + return { kind: "repository", path, content }; + } + const resolved = resolveDocumentTarget(path, content, root.target); + if (resolved.ok) { + return { kind: "repository", path, content, target: resolved.value }; + } + const failure = asDocumentTargetError(resolved.error); + if (failure === undefined) { + throw resolved.error; + } + return { + kind: "target-failure", + path, + content, + failure: targetFailureRecord(failure.data), + }; + } - const selected = yield* selectComponent(name, { - includes: searchPaths, - registry, - ...(bundle === undefined ? {} : { workflow: bundle }), - ...(declared === undefined ? {} : { declared: declared.catalog }), - }); + if (claimed !== undefined) { + return { kind: "declared-private", origin: claimed.origin }; + } - switch (selected.kind) { - case "repository": - return { - kind: "repository", - path: selected.path, - content: yield* readTextFile(selected.path), - }; - case "workflow": - // The exact pinned source, already in hand: the bundle was read from - // the definition's own commit before this run existed, so recording it - // reads nothing and a replay reconstructs it without resolving a name. - return { - kind: "workflow", - path: selected.path, - sourceHash: selected.sourceHash, - content: selected.content, - }; - case "declared-markdown": - // The exact declared bytes, already in hand: they were admitted - // before this run imported a root, so recording them reads nothing - // and a replay reconstructs the component without resolving a name. - return { - kind: "declared-markdown", - origin: selected.origin, - digest: selected.digest, - content: selected.source, - // Recorded only when it holds, so an ordinary declaration's record - // is exactly what it always was. - ...(selected.exact ? { exact: true } : {}), - }; - case "registered": - return { - kind: "registered", - origin: selected.origin.kind === "registered" ? selected.origin.origin : "", - reserved: selected.origin.kind === "registered" && selected.origin.reserved, - }; - case "structural": - throw new Error( - `${name} is structural syntax the engine owns, so it never resolves a component`, - ); - case "unresolved": - throw new Error(unresolvedMessage(name, selected.searched)); - } - }, - ); + const { searchPaths, registry, bundle, declared } = inputs; + const selected = yield* selectComponent(name, { + includes: searchPaths, + registry, + ...(bundle === undefined ? {} : { workflow: bundle }), + ...(declared === undefined ? {} : { declared: declared.catalog }), + }); - // Parsed rather than asserted: a replay hands back whatever the journal holds, - // and a history somebody else wrote is not a `DurableSelection` because it - // type-checked on the way in. - const selection = readDurableSelection(recorded); - if (selection === undefined) { - throw new Error(name === "__root__" ? UNREADABLE_ROOT_RECORD : UNREADABLE_IMPORT_RECORD); + switch (selected.kind) { + case "protected": + // Nothing about the answer is recorded: what this name means is core's + // own, so a replay asks the copy of core that is running rather than + // restoring an origin a registry would have to still hold. + return { kind: "protected" }; + case "repository": + return { + kind: "repository", + path: selected.path, + content: yield* readTextFile(selected.path), + }; + case "workflow": + // The exact pinned source, already in hand: the bundle was read from the + // definition's own commit before this run existed, so recording it reads + // nothing and a replay reconstructs it without resolving a name. + return { + kind: "workflow", + path: selected.path, + sourceHash: selected.sourceHash, + content: selected.content, + }; + case "declared-markdown": + // The exact declared bytes, already in hand: they were admitted before + // this run imported a root, so recording them reads nothing and a replay + // reconstructs the component without resolving a name. + return { + kind: "declared-markdown", + origin: selected.origin, + digest: selected.digest, + content: selected.source, + // Recorded only when it holds, so an ordinary declaration's record is + // exactly what it always was. + ...(selected.exact ? { exact: true } : {}), + }; + case "registered": + return { + kind: "registered", + origin: selected.origin.kind === "registered" ? selected.origin.origin : "", + reserved: selected.origin.kind === "registered" && selected.origin.reserved, + }; + case "structural": + throw new Error( + `${name} is structural syntax the engine owns, so it never resolves a component`, + ); + case "unresolved": + throw new Error(unresolvedMessage(name, selected.searched)); } +} + +/** What a declaration offered this element, when the element is inside one. */ +type PrivateOffer = ReturnType | undefined; + +/** + * The definition one decided selection produces. + * + * A plain `Operation`, so both callers reach the identical code: the durable + * terminal runs it behind one `ephemeral()`, which is what it always was — every + * step here is transparent to the journal — and canonical execution's private + * capture runs it directly, before a durable run exists to be transparent to. + */ +function* materializeImport( + name: string, + selection: DurableSelection, + inputs: ImportInputs, + claimed: PrivateOffer, +): Operation { + const { registry, declared, guarded } = inputs; // Rebuilt here rather than carried out of the durable operation, so a replayed // failed selection and a live one raise the same error with the same fields. @@ -519,6 +586,21 @@ function* durableImportComponent( throw documentTargetError(failure); } + if (selection.kind === "protected") { + // The implementation this execution built from the claimant it minted for + // this component. Not a registry lookup and not a file: a protected name is + // canonical core's answer, and an execution that built none for it has no + // protected implementation to run. + const own = guarded.get(name); + if (own === undefined) { + throw new Error( + `Component ${name} was recorded as a component canonical core owns, and this execution ` + + "built no implementation for it.", + ); + } + return own; + } + if (selection.kind === "registered") { // The function was never journaled. Find the implementation the recorded // origin names in the registry this run has; refusing when it is gone is @@ -574,7 +656,7 @@ function* durableImportComponent( // Reconstructed from the record's own source. Selection already decided // this name, and a bundled component is Markdown by construction, so // nothing here reads a file or imports a module. - return yield* ephemeral(parseMarkdownDefinition(name, selection.path, selection.content)); + return yield* parseMarkdownDefinition(name, selection.path, selection.content); } const { path, content, target } = selection; @@ -585,9 +667,9 @@ function* durableImportComponent( // that chose this path stats there too, so a component that rebinds `cwd` // for its content — `` — does not change which components that // content can resolve, or leave a selected path unloadable. - const currentDir = yield* ephemeral(processCwd()); + const currentDir = yield* processCwd(); const absolutePath = path.startsWith("/") ? path : `${currentDir}/${path}`; - const mod = yield* ephemeral(until(import(`file://${absolutePath}`))); + const mod = yield* until(import(`file://${absolutePath}`)); if (typeof mod !== "object" || mod === null) { throw new Error(`Function component "${name}" at ${path} did not load a module`); } @@ -597,9 +679,9 @@ function* durableImportComponent( // this copy of core built — the copy performing the execution — so a // component loaded from disk beside its own copy of core is still // authenticated against the invocation this engine minted. - const declared = parseFormDeclaration("form" in mod ? mod.form : undefined); + const declaration = parseFormDeclaration("form" in mod ? mod.form : undefined); const defaultExport = "default" in mod ? mod.default : undefined; - if (declared === undefined && !isFunctionComponent(defaultExport)) { + if (declaration === undefined && !isFunctionComponent(defaultExport)) { throw new Error( `Function component "${name}" at ${path} must have a default export that is a generator function`, ); @@ -610,7 +692,7 @@ function* durableImportComponent( propsExport === undefined ? { type: "object", properties: {}, additionalProperties: false } : parseJsonObject(propsExport); - yield* ephemeral(compilePropsSchema(props)); + yield* compilePropsSchema(props); const definition: FunctionComponentDefinition = { kind: "function", @@ -618,13 +700,13 @@ function* durableImportComponent( props, // Read off the same declaration the dispatcher is built from, so what the // component accepts and what it says it accepts come from one value. - ...(declared === undefined ? {} : { forms: declaredForms(declared) }), - fn: declared === undefined ? defaultExport : formDispatcher(declared), + ...(declaration === undefined ? {} : { forms: declaredForms(declaration) }), + fn: declaration === undefined ? defaultExport : formDispatcher(declaration), }; if ("returns" in mod && mod.returns !== undefined) { const returns = parseReturnsDeclaration(mod.returns); - yield* ephemeral(compileReturnsSchema(returns)); + yield* compileReturnsSchema(returns); definition.returns = returns; } @@ -636,9 +718,169 @@ function* durableImportComponent( // the same section from the same text the first run recorded, whatever the // file on disk says now. if (target !== undefined) { - return (yield* ephemeral(parseRootMarkdownDefinition(name, path, content, target))).definition; + return (yield* parseRootMarkdownDefinition(name, path, content, target)).definition; + } + return yield* parseMarkdownDefinition(name, path, content); +} + +/** + * Resolve one name the way an execution's own capture does: through the same + * decision and the same construction, and into no journal at all. + * + * Canonical execution reaches this for the provider-backed names an evaluation + * profile admits, before the root import and before any document code. What it + * skips is the durable record — a capture is not authored work, and an + * `import_component` written for it would be an effect a continuation would then + * have to account for. + */ +function* resolveImportUnrecorded( + name: string, + inputs: ImportInputs, +): Operation { + const claimed = inputs.declared?.claim(name); + const selection = yield* selectImport(name, undefined, inputs, claimed); + return yield* materializeImport(name, selection, inputs, claimed); +} + +/** What a provider-backed name that nothing identified refuses with. */ +const UNIDENTIFIED_ANSWER = + "an evaluation profile admits a component answer, and the implementation the import chain " + + "answered with carries no identity from the provider that installed it. A fragment runs an " + + "answer a continuation can be compared against, or it runs none."; + +/** + * What a provider-backed name whose answer is somebody else's refuses with. + * + * Both identities are spelled the one way a reader is ever shown one, so a + * diagnostic and a claim describe the same thing the same way. Neither is a + * secret: an origin, a key and a revision are what a host published in order to + * admit the name, and naming them is what makes this actionable. + */ +function differentAnswer(name: string, expected: FragmentIdentity, stated: AnswerIdentity): string { + return ( + `an evaluation profile admits <${name} /> as ${identityRecord(expected)}, and the import ` + + `chain answered with ${identityRecord(stated)}. An admitted identity is the exact ` + + "implementation a fragment may run." + ); +} + +/** + * Resolve every provider-backed name this profile admits, once, before the root + * import and before any document code exists. + * + * Eager on purpose. A profile that resolved a name lazily would resolve it + * through whatever middleware the running document had arranged by then, which + * is exactly the substitution an admitted identity exists to close. So the + * lookup happens here, through the complete ordinary `Component.importComponent` + * chain — every provider handler composes around it — and terminates in a + * private canonical resolver that writes no `import_component` record: a capture + * is not authored work, and a durable record for it would be an effect a + * continuation then had to account for. + * + * What comes back is asked about once. The provider's claim is read off the + * *exact final answer* rather than off anything the chain held in between, and + * that one call answers with both the identity and core's own copy of what was + * claimed — taken when the claim was recorded, before the chain could see it + * again. So the value this run keeps is the value the check was made about: + * there is no second read of the chain's object for an alternating proxy or a + * computed member to answer differently. + * + * The identity is then compared whole with what the host entry expects, and the + * copy — not the answer — is what the profile seals. + */ +function* resolveComponentAnswers( + prepared: PreparedProfile, + imports: CanonicalImports, + inputs: ImportInputs, +): Operation { + const answers = new Map(); + if (prepared.answered.length === 0) { + // A capability-only profile performs no component-chain lookup at all. + return answers; } - return yield* ephemeral(parseMarkdownDefinition(name, path, content)); + yield* scoped(function* () { + yield* Component.around( + { + *importComponent([name], _next) { + return yield* resolveImportUnrecorded(name, inputs); + }, + }, + { at: "min" }, + ); + for (const { name, identity } of prepared.answered) { + // One resolution, one claim window. It opens before the chain is asked + // and closes however this iteration leaves — answered, refused, or + // cancelled partway through — so a handler that lost this decision, or + // one still holding its request while the *next* name resolves, states + // nothing into a decision that is already made. + // + // One call, one result. The claim and core's copy of what was claimed + // come back together, and the object the chain returned is not read + // again — so nothing this run keeps was decided by a second read. + const resolution = imports.beginResolution(name); + let identified; + try { + // The resolution this run opened is handed to identification rather + // than looked up: what makes an answer this import's is the exact + // window object, so provenance is never read out of whichever window + // happens to be current when the question is asked. + identified = imports.identify(resolution, yield* importComponent(name)); + } finally { + resolution.close(); + } + if (identified === undefined) { + throw new EvaluationProfileError(UNIDENTIFIED_ANSWER); + } + const { identity: stated, definition: own } = identified; + if ( + stated.origin !== identity.origin || + stated.key !== identity.key || + stated.revision !== identity.revision + ) { + throw new EvaluationProfileError(differentAnswer(name, identity, stated)); + } + // An answer this execution could not keep a copy of never reaches here: + // identification refuses it above, because there would be nothing to hand + // back. What is left is an answer that copied fine and is not something a + // fragment can invoke. + if (own.kind !== "function" || typeof own.fn !== "function") { + throw new EvaluationProfileError( + `an evaluation profile admits <${name} />, and the import chain answered with a ` + + "definition that has no implementation to invoke. An admitted entry runs something.", + ); + } + answers.set(name, { definition: own }); + } + }); + return answers; +} + +function* durableImportComponent( + name: string, + root: RootDocumentSource | undefined, + position: Readonly | undefined, + inputs: ImportInputs, +): Workflow { + // Taken before the durable operation and outside it, because the offer is + // canonical core's own and a replay has to reach this the same way the live + // run did: the element asking is inside the declaration's body, or it is not. + const claimed = name === "__root__" ? undefined : inputs.declared?.claim(name); + const recorded = yield createDurableOperation( + // The root import is the run's own entry rather than an authored element, + // so it carries no source however it was reached. + { type: "import_component", name, ...(root ? {} : sourceDescription(position)) }, + () => selectImport(name, root, inputs, claimed), + ); + + // Parsed rather than asserted: a replay hands back whatever the journal holds, + // and a history somebody else wrote is not a `DurableSelection` because it + // type-checked on the way in. + const selection = readDurableSelection(recorded); + if (selection === undefined) { + throw new Error(name === "__root__" ? UNREADABLE_ROOT_RECORD : UNREADABLE_IMPORT_RECORD); + } + + return yield* ephemeral(materializeImport(name, selection, inputs, claimed)); } function isFunctionComponent(value: unknown): value is FunctionComponent { @@ -835,7 +1077,7 @@ function readRootSelection(value: unknown): RootImportRecord { } // Same standard for a failure: the recorded selector must fail against the // recorded content in exactly the way the record claims. That verifies the - // catalog and the matches too, which no amount of shape checking could. + // symbols and the matches too, which no amount of shape checking could. const rederived = findTarget(outline, failure.selector); if (rederived.ok) { return MALFORMED; @@ -1857,7 +2099,15 @@ function* documentWorkflow( (function* (): Operation { const imported = yield* importComponent("__root__"); const imports = authority.imports; - return imports === undefined ? imported : imports.authorize("__root__", imported); + // Asked only when a tier actually closes this name, exactly as an + // ordinary import is. The authority used to be absent altogether for a + // run with no bundle and no declarations, so this could authorize + // unconditionally; the protected tier is present in *every* execution, so + // an unguarded call now refuses the root of every ordinary run — nothing + // claims `__root__` unless a bundle closes the execution. + return imports?.closes("__root__") === true + ? imports.authorize("__root__", imported) + : imported; })(), ); @@ -2110,6 +2360,37 @@ function* executeDocument( bundles: readonly WorkflowComponentBundle[] = [], identityComponents: readonly IdentityComponent[] = [], declarations: readonly DeclaredMarkdownComponent[] = [], + providers: readonly SyntaxSymbolsProvider[] = [], + /** + * The documentation each bootstrapped package contributed. + * + * Carried by value from the collection boundary, like the symbols provider + * beside it, so the index a document's own `` reads is the + * index the profile actually assembled. + */ + documentation: readonly DocumentationContribution[] = [], + /** This execution's packaged-asset reader, carried by value from the caller. */ + readAsset: DocumentationReader = packagedAssetReader, + /** + * The one owner of every answer this execution produces or a provider + * identifies. + * + * Built before any installation ran, so a provider could state identities + * during profile capture; handed here so the import authority this execution + * imports through is the same table those claims went into. + */ + canonicalImports: CanonicalImports = new CanonicalImports(), + /** + * The one evaluation profile this host stated, copied and bound, when it + * stated one. + * + * Carried by value like the symbols provider beside it. It is sealed below, + * once the provider-backed names it admits have been resolved, and the sealed + * result is handed to core's own expansion on the private authority rather + * than through any context: it is the ceiling `` narrows from, and a + * document that could reach it could raise it. + */ + prepared?: PreparedProfile, ): Operation { const { stream, @@ -2226,6 +2507,10 @@ function* executeDocument( const identity = installIdentities( identityComponents, admittedDeclarations.flatMap((declaration) => [...declaration.privates]), + // Core's own, minted the same way and registered nowhere. Every + // execution has them, whatever its host declared, which is what makes a + // protected name mean one thing everywhere. + PROTECTED_COMPONENTS, ); yield* registerComponents(identity.registrations); identity.activate(); @@ -2263,16 +2548,61 @@ function* executeDocument( if (declaredImports !== undefined) { tiers.push(declaredImports); } - const imports = tiers.length === 0 ? undefined : new ExecutionImports(tiers); + // Last, because a tier that claims a name answers for it and the earlier + // ones claim names of their own; a bundled execution still words every + // other refusal exactly as it always did. Present in every execution, + // because a protected name is closed in every execution. + tiers.push(new ProtectedImports()); + const imports = new ExecutionImports(tiers, canonicalImports); + + // The one place a provider-backed profile name is resolved, and the last + // thing that happens before the authority exists: the registry, the + // bundle, the declarations, the protected table and the identity domains + // are all established, and the root import has not been asked for. So the + // chain a capture resolves through is the ordinary one, and no document + // code has run to arrange it. + const evaluation = + prepared === undefined + ? undefined + : yield* prepared.seal( + yield* resolveComponentAnswers(prepared, canonicalImports, { + searchPaths: includes, + registry: startingRegistry, + bundle, + declared: declaredImports, + guarded: identity.protected, + }), + ); + const authority: ExpansionAuthority = { - ...(imports === undefined ? {} : { imports }), + imports, ...(declaredImports === undefined ? {} : { declared: declaredImports }), identities: identity.identities, + protectedBodies: identity.protectedBodies, forms, // Created here, held here, and reclaimed with this execution. Nothing a // document, a component, middleware or a separately loaded copy can // name reaches this object. exact: createExactSource(), + // Built from what this execution captured before any installation, + // middleware or document code ran, and asked only when an occurrence + // renders: a run whose document never writes `` enumerates + // nothing. + syntax: rootSyntaxReference( + { + includes, + registry: startingRegistry, + components: identityComponents, + declarations, + ...(bundle === undefined ? {} : { workflow: bundle }), + }, + providers[0], + documentation, + ), + // The ceiling a generated fragment is evaluated under, when this host + // offers evaluation at all. Absent is a host that offers none, and + // `` refuses on that rather than inventing one. + ...(evaluation === undefined ? {} : { evaluation }), }; // Install the document's runtime Component providers before durableRun @@ -2287,11 +2617,14 @@ function* executeDocument( const definition = yield* durableImportComponent( name, name === "__root__" ? root : undefined, - includes, - registered, position, - bundle, - declaredImports, + { + searchPaths: includes, + registry: registered, + bundle, + declared: declaredImports, + guarded: identity.protected, + }, ); // Canonical selection, recorded where it is made. This is the only // thing that puts an invocation in one of this execution's identity @@ -2546,6 +2879,53 @@ export interface ExecutionInstallation { * component that can name a durable operation after its invocation. */ readonly components?: readonly IdentityComponent[]; + /** + * The symbols this host's profile describes, when its profile is not the one + * the execution itself would derive. + * + * Captured by value alongside the admissions, before any installation runs, + * for the reason the rest are: what a document is shown is settled before + * anything can observe or replace it. Omitted is the ordinary case — canonical + * core derives the symbols from the selection inputs this execution captured, + * which is what makes an ordinary run's reference the run's own. + * + * `xmd plan` states one, because the Plan being written is a program a later + * `xmd run` executes: the vocabulary the agent must be shown is that profile's + * rather than the authorship execution's. One execution accepts one. + * + * Documentation does not travel here. It is additive — a profile installing + * four packages has four boundaries — so each package's bootstrap contributes + * its own through the `Documentation` Api, and this execution collects them + * once at the same boundary. + */ + readonly symbols?: SyntaxSymbolsProvider; + /** + * The maximum authority a generated fragment may be evaluated under here. + * + * Captured by value alongside the rest, before any installation runs, for a + * reason the others share and this one sharpens: `` is a public + * component, so any author may write it, and what stops that from being a + * capability is that the ceiling was stated by the host before a document + * existed. Nothing a running document reaches names this — not a context, + * whose name is not a secret; not a registry, which a nested scope layers + * over; not a prop, which an author writes. + * + * Omitted is the ordinary case for a host that offers no evaluation at all: + * `` then refuses before the producer runs, rather than evaluating + * under a ceiling nobody stated. One execution accepts one; two are refused + * rather than ordered. + */ + readonly evaluation?: FragmentEvaluationInput; + /** + * The providers behind this installation's `component-answer` entries. + * + * Run during profile capture, before this installation's ordinary + * `install()`, and handed a registrar fixed to each provider installation. + * Every registered handler receives a fresh request carrying that + * installation's origin. Captured by value with the rest: what a provider may + * identify is settled before any document code exists. + */ + readonly componentAnswers?: readonly ComponentAnswerInstallation[]; install?(): Operation; } @@ -2630,6 +3010,13 @@ export const Execution: Api = createApi("Execution", function* runInvocation( options: ExecuteOptions, installations: readonly ExecutionInstallation[], + /** + * How this execution reads its packaged documentation assets. + * + * Defaulted to the real filesystem reader and carried by value from here into + * the syntax reference, so it belongs to this execution alone. + */ + readAsset: DocumentationReader = packagedAssetReader, observed?: () => void, ): Operation { const ready = withResolvers(); @@ -2662,7 +3049,7 @@ function* runInvocation( let published = false; try { yield* scoped(function* () { - const execution = yield* invoke(options, installations); + const execution = yield* invoke(options, installations, readAsset); published = true; ready.resolve(execution); state.document = yield* execution; @@ -2848,6 +3235,8 @@ function detachedSchema(schema: Sche function* invoke( options: ExecuteOptions, installations: readonly ExecutionInstallation[], + /** This execution's packaged-asset reader, carried by value from its caller. */ + readAsset: DocumentationReader = packagedAssetReader, ): Operation { const admissions = Object.freeze( installations.flatMap((installation) => [...(installation.admissions ?? [])]), @@ -2937,12 +3326,105 @@ function* invoke( ), ); - for (const installation of installations) { - if (installation.install) { - yield* installation.install(); + // Read once and frozen with the rest, and before any installation runs: which + // profile a document is shown is settled before anything can replace it. Two + // are refused rather than ordered — symbols chosen by installation order + // would make what an agent is told to write depend on assembly order. + const providers = Object.freeze( + installations.flatMap((installation) => { + const provider = installation.symbols; + return provider === undefined ? [] : [provider]; + }), + ); + if (providers.length > 1) { + throw new Error( + "two installations stated the symbols this execution describes. One execution describes " + + "one vocabulary, so which profile a document is shown is never a question of order.", + ); + } + + // What each installation will actually be asked to do, read once and in + // order, before the first of them runs. Both members are read here for the + // reason everything else on the installation is: a property that answered + // differently the second time would let a host be tested for one provider and + // install another, and a provider installer added from inside an earlier + // `install()` would be one nothing captured. + const assembly = Object.freeze( + installations.map((installation) => + Object.freeze({ + providers: Object.freeze( + [...(installation.componentAnswers ?? [])].map((provider) => + Object.freeze({ origin: provider.origin, install: provider.install.bind(provider) }), + ), + ), + install: installation.install?.bind(installation), + }), + ), + ); + + // The maximum authority a generated fragment may be evaluated under here, + // read on the same terms and for a stronger reason: two profiles would be two + // answers to what a fragment may *do*, and choosing between them by + // installation order would make authority depend on assembly. A host that + // stated none offers no evaluation at all, which `` refuses with + // rather than inventing a ceiling for. + const stated = installations.flatMap((installation) => { + const profile = installation.evaluation; + return profile === undefined ? [] : [profile]; + }); + if (stated.length > 1) { + throw new Error(TWO_PROFILES); + } + // Copied and bound here, before any `install()` runs, so a host that mutates + // its own tables, schemas or headers from inside one changes nothing this + // execution does. What it cannot settle yet is what a provider-backed name + // resolves to: that takes the complete ordinary import chain, and the + // providers have not installed. + const prepared = stated[0] === undefined ? undefined : yield* prepareEvaluationProfile(stated[0]); + if (prepared !== undefined) { + // Registered before the first `install()`, so the operations this profile + // bound are revoked however the execution ends — returning, failing, or + // being halted partway through assembly. A fragment body, a handler or a + // retained callback that reaches one afterwards refuses rather than acting + // on a filesystem the run no longer holds a transaction for. + yield* ensure(function* () { + prepared.revoke(); + }); + } + + // The one owner of every answer this execution produces or a provider + // identifies. Constructed inactive, and activated only after its teardown is + // registered: a claim recorded in between would be a claim with nothing behind + // it to revoke. + const canonicalImports = new CanonicalImports(); + yield* ensure(function* () { + canonicalImports.revoke(); + }); + canonicalImports.activate(); + + // Each provider installer first, then that installation's ordinary `install`, + // in the order the installations were captured. A provider registers + // middleware through its registrar, and each invocation states what it is + // returning through a fresh request fixed to the provider's origin. + for (const { providers, install } of assembly) { + for (const { origin, install: installProvider } of providers) { + yield* installProvider(componentAnswerRegistrar(canonicalImports.provider(origin))); + } + if (install !== undefined) { + yield* install(); } } + // The documentation this execution's packages contributed, collected once + // here and snapshotted by value. + // + // Asked *after* the trusted host's bootstrap, because that is where a package + // installs its registrations and its documentation together, and *before* the + // root import and every element below it, because what a document is told + // about a component must not be something the document arranged. Middleware + // installed later composes into a chain nothing reads again. + const documentation = yield* capturedDocumentation(readAsset); + const issued = issueExecution(options); // The terminal for this invocation and no other. @@ -2975,6 +3457,11 @@ function* invoke( bundles, identityComponents, declarations, + providers, + documentation, + readAsset, + canonicalImports, + prepared, ); } @@ -3012,7 +3499,7 @@ export function executeObserved( ): Operation { // The callback is read here, once, and passed on as a value. What the caller // does to its own record afterwards is its own business. - return runInvocation(options, [...installations], observers.observed); + return runInvocation(options, [...installations], packagedAssetReader, observers.observed); } /** @@ -3030,6 +3517,28 @@ export function executeInstalled( return runInvocation(options, [...installations]); } +/** + * One execution whose packaged-asset reads go through `read`. + * + * Core's own evidence seam, exported from this module and from neither `mod.ts` + * nor `host.ts`, so it is not part of the package's surface. It *builds a new + * execution* around the reader rather than changing anything an existing one + * holds — so importing it from a repository component, an installed package or + * a document's own code cannot reach the current execution's reader, and two + * executions in one process are unaffected by each other. + * + * It exists because cancelling *inside documentation-index construction* is a + * different claim from cancelling inside symbol discovery, and there is no + * other point in that operation a test can stand at. + */ +export function executeReadingAssetsWith( + options: ExecuteOptions, + installations: readonly ExecutionInstallation[], + readAsset: DocumentationReader, +): Operation { + return runInvocation(options, [...installations], readAsset); +} + /** * Apply every additive completion policy, in registration order. * diff --git a/packages/core/src/expand.ts b/packages/core/src/expand.ts index ffe05f103..8f5b5b631 100644 --- a/packages/core/src/expand.ts +++ b/packages/core/src/expand.ts @@ -105,6 +105,8 @@ import { elementFrame, elementSite, extendPath, publishExpansion, snapshot } fro import type { ExpansionFrame } from "./expansion.ts"; import { isPrivateImplementation, issueInvocation } from "./invocation-identity.ts"; import type { IdentityDomain } from "./invocation-identity.ts"; +import { protectedContentLease } from "./protected-content.ts"; +import type { SyntaxReference } from "./syntax-reference.ts"; import { withInvocation } from "./invocation.ts"; import type { Invocation } from "./invocation.ts"; import { ActiveProjection } from "./projection.ts"; @@ -3091,7 +3093,7 @@ function* expandFunctionComponent( !selfClosing, dispatcher, ); - const handle = createProjectionHandle({ + const projectionState: ProjectionState = { invocation, projecting: issued.projecting, enclosing, @@ -3114,7 +3116,13 @@ function* expandFunctionComponent( ownPath: path, checkedFailures, authority, - }); + }; + // Only the ordinary handle is published. The richer projection a + // protected body may reach stays a closure at the dispatch below, so + // nothing that can read `ActiveProjection` — another loaded copy, a + // repository component, a registration, a declaration, middleware, a + // context — obtains or influences it. + const handle = createProjectionHandle(projectionState); invocation.evalScope.scope.set(ActiveProjection, handle); yield* ActiveLoop.set(undefined); @@ -3221,6 +3229,57 @@ function* expandFunctionComponent( if (TestHarnessComponentDefinition.own(definition.fn)) { return yield* definition.fn.invoke(validatedProps, binding); } + // A component canonical core protects reads one lexical fact — the + // syntax reference for this site — and the fact changes as expansion + // descends, so it cannot be closed over when the implementation is + // built. It is delivered here instead, by the copy of core performing + // the expansion, from the authority it is already holding. + const guarded = authority?.protectedBodies?.body(definition.fn); + if (guarded !== undefined) { + // What to project, under which authority and in which scope stays + // here, where those things already are. How many times it may + // answer, and for how long, is the lease's + // (`protected-content.ts`). + const lease = selfClosing + ? undefined + : protectedContentLease(name, function* (syntax: SyntaxReference) { + // The same state the ordinary handle was built from, with only + // the reference replaced: the producer keeps its own imports, + // declarations, closure, identities, bindings, providers, + // directory, forms, exact-source tracking and error mode, and + // sees a narrower vocabulary. This is a second handle over one + // content scope, not a second expansion. + const narrowed = createProjectionHandle({ + ...projectionState, + authority: { ...authority, syntax }, + }); + const outcome = yield* narrowed.tryProject({ + kind: "slot", + name: undefined, + mode: siteErrorMode, + }); + if (outcome.failure !== undefined) { + throw outcome.failure; + } + const errors = errorSegments(outcome.segments); + if (errors.length > 0) { + throw new ContentExpansionFailure(errors, undefined, outcome.segments); + } + return renderSegments(outcome.segments); + }); + try { + return yield* guarded(validatedProps, issued.invocation, { + syntax: authority?.syntax, + evaluation: authority?.evaluation, + projectContent: lease?.project, + }); + } finally { + // Closed in the same breath the issuance is: a projector a body + // kept authorizes nothing once that body has finished. + lease?.close(); + issued.close(); + } + } // Ended in the same breath the body is: an issuance a wrapper kept // from a finished element authorizes nothing when it is routed here. try { diff --git a/packages/core/src/fetch-request.ts b/packages/core/src/fetch-request.ts index 0583cbcd1..2e381ac01 100644 --- a/packages/core/src/fetch-request.ts +++ b/packages/core/src/fetch-request.ts @@ -184,6 +184,22 @@ function admitHeaders(value: Json | undefined): Record { * bound at all. */ export function* prepareFetchRequest(props: Record): Operation { + return normalizeFetchRequest(props, yield* timeoutFetch); +} + +/** + * The same normalization against a timeout the caller already resolved. + * + * A generated fragment's ceiling is compared across a suspension, so the bound + * it carries cannot come from a context read at whichever point the comparison + * happens. A trusted host resolves its effective timeout once when it builds + * its evaluation profile, and preflight and execution both normalize against + * that value rather than reading the context again. + */ +export function normalizeFetchRequest( + props: Record, + effective: number | undefined, +): FetchRequest { const url = admitUrl(props.url); const method = admitMethod(props.method); const headers = admitHeaders(props.headers); @@ -194,7 +210,7 @@ export function* prepareFetchRequest(props: Record): Operation ("timeout" prop)'); return { diff --git a/packages/core/src/fragment-capabilities.ts b/packages/core/src/fragment-capabilities.ts new file mode 100644 index 000000000..ce988177b --- /dev/null +++ b/packages/core/src/fragment-capabilities.ts @@ -0,0 +1,552 @@ +/** + * The operations an admitted fragment actually performs, and nothing else it + * could reach. + * + * An admitted `` used to be core's ordinary `` component, which + * resolves its provider through `API.Files` at the moment it runs. That made + * the authority a generated fragment exercises a property of whatever the + * document, a repository component or middleware had installed by then — so a + * fragment admitted for "read a file" reached whichever Files provider was + * nearest, and a handler composed around one could observe, rewrite or answer + * the call. + * + * The operations here are the alternative. A trusted host hands its private + * provider to `captureEvaluationProfile()`, which reads each method off it + * exactly once and closes a fragment-facing component over the bound result. + * What runs inside a fragment therefore reaches: + * + * - only the methods this file names — read, path check, write, delete and + * directory creation, plus one Fetch — and not the rest of the host's + * provider, not `API.Files`, not `API.Fetch`, and not `API.Env`; + * - only the working directory the host stated when it built the profile; and + * - only for as long as the execution that captured them is alive. + * + * ## Why the components are core's own + * + * A host cannot supply the body, because a body is where a provider lookup + * would go. These are written once, here, against the bound operations, and a + * host chooses only *which* of them a fragment may name and under what + * identity. That is also why the schemas match the ordinary components': a + * fragment written against `` is the same text an author would + * write, and only the authority behind it differs. + * + * ## What is deliberately absent + * + * No glob, no temporary directory, no environment, no process, no elicitation + * and no agent. Those are operations the ordinary components have and an + * admitted fragment does not, and leaving them out here is what makes that + * true rather than a claim about what a host will remember not to admit. + */ + +import type { Operation, Result } from "effection"; + +import type { FunctionComponentDefinition, Json, PropsSchema } from "./types.ts"; +import { content } from "./component-api.ts"; +import { ContentError, ProjectedContentError } from "./errors.ts"; +import { getExpansion } from "./expansion.ts"; +import { persistFetch } from "./fetch-journal.ts"; +import { parseResponseRecord } from "./fetch-response.ts"; +import type { FetchResponseRecord } from "./fetch-response.ts"; +import { formDispatcher } from "./invocation-identity.ts"; + +/** A refusal an admitted fragment's own operation produced. */ +export class FragmentCapabilityError extends Error { + override name = "FragmentCapabilityError"; +} + +/** What a fragment operation reached after its execution had ended. */ +export const REVOKED_CAPABILITY = + "an admitted fragment reached an operation belonging to an execution that has ended."; + +/** One path an operation is about to act on, resolved against the stated root. */ +export interface FragmentPath { + readonly cwd: string; + readonly path: string; +} + +/** One write an operation is about to perform. */ +export interface FragmentWrite extends FragmentPath { + readonly content: string; +} + +/** + * The exact filesystem operations a fragment may perform. + * + * A strict subset of what a host's own provider offers, listed rather than + * derived: a fragment provider built by widening `FilesHandler` would gain + * every operation a later release adds to it, which is the opposite of what a + * ceiling is for. + * + * `Result` rather than a throw, matching the provider contract the host already + * implements, so a host passes its own methods without adapting them. + */ +export interface FragmentFileAccess { + /** Whether this path is admissible at all. No filesystem access. */ + checkFilePath(input: FragmentPath): Operation>; + readTextFile(input: FragmentPath): Operation>; + writeTextFile(input: FragmentWrite): Operation>; + deleteFile(input: FragmentPath): Operation>; + ensureDirectory(input: FragmentPath): Operation>; + /** + * The directory every path above resolves against. + * + * An operation rather than a value, because a workflow run's authoritative + * root advances as the run commits, and a fragment admitted later addresses + * the root the run is on rather than the one it started from. + */ + workingDirectory(): Operation; +} + +/** + * The exact HTTP read a fragment may perform. + * + * It answers with the same record the ordinary `` retains — status, + * headers and body — because that is what a document binds and what a + * continuation restores. A transport that answered with the body alone would + * make a fragment's observation a different shape from an authored one, and a + * document that branches on a 404 could not. + */ +export interface FragmentFetchAccess { + fetch(request: { + readonly url: string; + readonly method: string; + readonly headers: Record; + readonly timeout?: number; + }): Operation; +} + +/** + * Where a fragment's relative paths resolve, while it is running. + * + * A directory component scopes its content — `` makes the + * `out.md` inside it mean `nested/out.md` — and the ordinary component does that + * by installing a working directory on the contextual environment. A fragment + * cannot: the contextual environment is exactly what a document, a repository + * component or middleware can answer, and a fragment whose destination could be + * moved from outside would be a fragment admitted for one path and performing + * another. + * + * So the cursor is the evaluation's own. It is created per capture, reset at the + * start of each fragment, and saved and restored around a directory's content by + * the body that scoped it. Nothing outside this module holds a reference. + */ +interface DirectoryCursor { + current: string; +} + +/** Every live operation one captured profile holds, read once and revocable. */ +export interface CapturedCapabilities { + readonly files?: FragmentFileAccess; + readonly fetch?: FragmentFetchAccess; + /** Where this evaluation's relative paths currently resolve. Module-private. */ + readonly cursor: DirectoryCursor; + /** + * Begin one fragment, and answer with how to end it. + * + * Reads the host's working directory once, here, rather than once per + * element: every path in one fragment resolves against the directory the run + * was in when that fragment was admitted. The returned operation restores + * whatever was current before, so a fragment produced inside another + * fragment's producer leaves the outer one where it was. + */ + enterFragment(): Operation<() => void>; + /** Whether the execution that captured these is still running. */ + live(): boolean; + /** Called at execution teardown. Every bound operation refuses afterwards. */ + revoke(): void; +} + +/** + * Read each method off the host's objects once, and hand back bound copies. + * + * Bound, so replacing a method on the object the host passed after installation + * reaches nothing. Behind a revocation this execution owns, so an operation + * retained past teardown refuses rather than acting on a filesystem the run no + * longer has a transaction for. + */ +export function captureCapabilities(input: { + readonly files?: FragmentFileAccess; + readonly fetch?: FragmentFetchAccess; +}): CapturedCapabilities { + let alive = true; + const live = () => alive; + const guard = (operation: (argument: A) => Operation) => + function* (argument: A): Operation { + if (!alive) { + throw new FragmentCapabilityError(REVOKED_CAPABILITY); + } + return yield* operation(argument); + }; + + const files = input.files; + const fetching = input.fetch; + const directory = files?.workingDirectory.bind(files); + const cursor: DirectoryCursor = { current: "" }; + return Object.freeze({ + cursor, + *enterFragment(): Operation<() => void> { + if (!alive) { + throw new FragmentCapabilityError(REVOKED_CAPABILITY); + } + const enclosing = cursor.current; + cursor.current = directory === undefined ? "" : yield* directory(); + return () => { + cursor.current = enclosing; + }; + }, + ...(files === undefined + ? {} + : { + files: Object.freeze({ + checkFilePath: guard(files.checkFilePath.bind(files)), + readTextFile: guard(files.readTextFile.bind(files)), + writeTextFile: guard(files.writeTextFile.bind(files)), + deleteFile: guard(files.deleteFile.bind(files)), + ensureDirectory: guard(files.ensureDirectory.bind(files)), + *workingDirectory(): Operation { + if (!alive || directory === undefined) { + throw new FragmentCapabilityError(REVOKED_CAPABILITY); + } + return yield* directory(); + }, + }), + }), + ...(fetching === undefined + ? {} + : { fetch: Object.freeze({ fetch: guard(fetching.fetch.bind(fetching)) }) }), + live, + revoke: () => { + alive = false; + }, + }); +} + +/** The one path prop the file components take, matching the ordinary schema. */ +const PATH_PROPS: PropsSchema = { + type: "object", + properties: { path: { type: "string" } }, + required: ["path"], + additionalProperties: false, +}; + +/** The props the fetch component takes, matching the ordinary schema. */ +const FETCH_PROPS: PropsSchema = { + type: "object", + properties: { + url: { type: "string" }, + method: { type: "string" }, + headers: { type: "object" }, + timeout: { type: "string" }, + }, + required: ["url"], + additionalProperties: false, +}; + +/** The schema each capability's component declares. */ +export function capabilityProps(capability: FragmentCapability): PropsSchema { + return capability === "fetch" ? FETCH_PROPS : PATH_PROPS; +} + +/** Which captured operation one admitted entry runs. */ +export type FragmentCapability = + | "file:read" + | "file:write" + | "file:delete" + | "directory:ensure" + | "fetch"; + +/** The authored forms each capability is written in. */ +export const CAPABILITY_FORMS: Readonly< + Record +> = Object.freeze({ + "file:read": ["self-closing"], + "file:write": ["paired"], + "file:delete": ["self-closing"], + "directory:ensure": ["paired"], + fetch: ["self-closing"], +}); + +/** + * The component one capability runs, closed over the bound operations. + * + * Built once per capture rather than per invocation, so every element in every + * fragment of one execution runs the same definition — which is also what lets + * two entries under one name reach one definition, as `` and + * `…` do. + */ +export function capabilityDefinition( + name: string, + admitted: { + readonly "self-closing"?: FragmentCapability; + readonly paired?: FragmentCapability; + }, + capabilities: CapturedCapabilities, + requests: readonly { + readonly url: string; + readonly method: string; + readonly headers: Record; + readonly timeout?: number; + }[] = [], +): FunctionComponentDefinition { + const selfClosing = admitted["self-closing"]; + const paired = admitted.paired; + // The schema is the union of what each spelling takes, which for every pair + // core supplies is one schema anyway: `` and `…` both + // take one `path`. + const props = capabilityProps(selfClosing ?? paired ?? "file:read"); + + const read = + selfClosing === undefined ? undefined : body(name, selfClosing, capabilities, requests); + const write = paired === undefined ? undefined : body(name, paired, capabilities, requests); + + // One name, one definition — even when it holds two identities in two + // classes. `` observes and `…` writes, and the evaluator + // resolves an import by name, so the two must arrive as one definition whose + // *dispatch* separates them. Two definitions would be one name with two + // answers, which it refuses; one undispatched body would let a read reach the + // write path. + if (read !== undefined && write !== undefined) { + return { + kind: "function", + name, + props, + forms: ["self-closing", "paired"], + fn: formDispatcher({ forms: "both", "self-closing": read, paired: write }), + }; + } + if (read !== undefined) { + return { + kind: "function", + name, + props, + forms: ["self-closing"], + fn: formDispatcher({ + forms: "self-closing", + fn: read, + refuse: () => + new FragmentCapabilityError( + `<${name} /> reads and renders no content of its own, so it is written self-closing.`, + ), + }), + }; + } + if (write !== undefined) { + return { + kind: "function", + name, + props, + forms: ["paired"], + fn: formDispatcher({ + forms: "paired", + fn: write, + refuse: () => + new FragmentCapabilityError( + `<${name} /> writes what it renders, so it is written with content rather than ` + + "self-closing.", + ), + }), + }; + } + throw new FragmentCapabilityError(`an evaluation profile admitted "${name}" for no form.`); +} + +/** The body one capability runs, closed over the operations this capture bound. */ +function body( + name: string, + capability: FragmentCapability, + capabilities: CapturedCapabilities, + requests: readonly { + readonly url: string; + readonly method: string; + readonly headers: Record; + readonly timeout?: number; + }[], +) { + if (capability === "fetch") { + return fetchBody(capabilities, requests); + } + const files = capabilities.files; + if (files === undefined) { + throw new FragmentCapabilityError( + `an evaluation profile admitted "${name}" without stating the filesystem operations it runs.`, + ); + } + const cursor = capabilities.cursor; + if (capability === "file:read") { + return readBody(files, cursor); + } + if (capability === "file:delete") { + return deleteBody(files, cursor); + } + return capability === "file:write" ? writeBody(files, cursor) : ensureBody(files, cursor); +} + +function readBody(files: FragmentFileAccess, cursor: DirectoryCursor) { + return function* read(props: Record): Operation { + const requested = String(props.path); + const text = yield* files.readTextFile({ cwd: cursor.current, path: requested }); + if (!text.ok) { + throw new FragmentCapabilityError(refusal(requested, "read")); + } + return text.value; + }; +} + +function deleteBody(files: FragmentFileAccess, cursor: DirectoryCursor) { + return function* remove(props: Record): Operation { + const requested = String(props.path); + const removed = yield* files.deleteFile({ cwd: cursor.current, path: requested }); + if (!removed.ok) { + throw new FragmentCapabilityError(refusal(requested, "delete")); + } + return ""; + }; +} + +function writeBody(files: FragmentFileAccess, cursor: DirectoryCursor) { + return function* write(props: Record): Operation { + const requested = String(props.path); + const cwd = cursor.current; + // The path is admitted before the content is produced, so a fragment whose + // destination is refused renders nothing at all. + const admitted = yield* files.checkFilePath({ cwd, path: requested }); + if (!admitted.ok) { + throw new FragmentCapabilityError(refusal(requested, "write")); + } + const text = yield* rendered(requested); + // Resolved against the directory this element was written in, captured + // before its content ran: a directory the content itself scoped must not + // move where this write lands. + const written = yield* files.writeTextFile({ cwd, path: requested, content: text }); + if (!written.ok) { + throw new FragmentCapabilityError(refusal(requested, "write")); + } + return ""; + }; +} + +function ensureBody(files: FragmentFileAccess, cursor: DirectoryCursor) { + return function* ensure(props: Record): Operation { + const requested = String(props.path); + const enclosing = cursor.current; + // The directory exists before its content runs: content that ran in a + // directory the ensure was going to refuse would be content run somewhere + // nobody chose. + const made = yield* files.ensureDirectory({ cwd: enclosing, path: requested }); + if (!made.ok) { + throw new FragmentCapabilityError(refusal(requested, "create")); + } + // And it scopes what it renders, which is what makes `` + // inside it mean this directory's `out.md`. Scoped through the evaluation's + // own cursor rather than the contextual environment, so nothing outside the + // fragment can move where its content writes. + cursor.current = resolve(enclosing, requested); + try { + return yield* rendered(requested); + } finally { + cursor.current = enclosing; + } + }; +} + +/** + * One authored path against the directory it was written in. + * + * An absolute path is used as written — a workflow run's logical root is one — + * and anything else is joined, which is what nesting means. + */ +function resolve(enclosing: string, path: string): string { + if (path.startsWith("/")) { + return path; + } + const base = enclosing.endsWith("/") ? enclosing.slice(0, -1) : enclosing; + return `${base}/${path}`; +} + +function fetchBody( + capabilities: CapturedCapabilities, + requests: readonly { + readonly url: string; + readonly method: string; + readonly headers: Record; + readonly timeout?: number; + }[], +) { + return function* perform(props: Record): Operation { + const fetching = capabilities.fetch; + if (fetching === undefined) { + throw new FragmentCapabilityError( + "an admitted fragment asked for a request, and this host stated no transport for one.", + ); + } + // Preflight already compared this element against the ceiling, from the + // scan, before the fragment's first effect. This finds the admitted request + // it matched and performs *that* — never the props, which would let a + // header the ceiling normalized away travel anyway. + const url = String(props.url); + const admitted = requests.find((request) => request.url === url); + if (admitted === undefined) { + throw new FragmentCapabilityError( + "an admitted fragment asked for a request this host did not admit.", + ); + } + // Journaled on exactly the terms an authored `` is, through the same + // module: one element is one durable request, and a continuation restores + // what came back rather than asking again. + const expansion = yield* getExpansion(); + const retained = yield* persistFetch( + { + id: expansion.id, + ...(expansion.position === undefined ? {} : { position: expansion.position }), + }, + admitted, + () => fetching.fetch(admitted), + ); + // The status rule is the authored one: a fragment binds nothing, so a + // response it cannot branch on must not be handed back as though it were + // an answer. + const response = parseResponseRecord(retained); + if (response.status < 200 || response.status > 299) { + throw new FragmentCapabilityError( + `an admitted fragment's request answered with status ${response.status}.`, + ); + } + return retained; + }; +} + +/** + * A refusal that names the path and the direction, and nothing else. + * + * The provider's own reason is deliberately dropped. A generated fragment's + * failure is serialized into the journal with its message, and a host provider's + * diagnostic can carry an absolute path, a root identifier or a transaction + * detail that the run has no business publishing on behalf of text nobody + * audited. + */ +function refusal(path: string, verb: string): string { + return `an admitted fragment could not ${verb} ${JSON.stringify(path)}.`; +} + +/** + * The rendered children, or a failure instead of a partial write. + * + * `content()` is a failure boundary, and for a component that *writes* what it + * rendered, embedding a printed error in the destination would be worse than + * useless. So this recovers from the boundary and fails the invocation: nothing + * reaches the provider, and the target keeps whatever it already held. + */ +function* rendered(path: string): Operation { + try { + return yield* content(); + } catch (error) { + if (error instanceof ContentError || error instanceof ProjectedContentError) { + throw new FragmentCapabilityError( + `an admitted fragment's content for ${JSON.stringify(path)} failed to render, so nothing ` + + "was written.", + { cause: error }, + ); + } + throw error; + } +} diff --git a/packages/core/src/generated-xmd.ts b/packages/core/src/generated-xmd.ts index ebd5c3fc1..f3d442cf3 100644 --- a/packages/core/src/generated-xmd.ts +++ b/packages/core/src/generated-xmd.ts @@ -96,6 +96,15 @@ * effects themselves are retained by their own durable records — `fetch` for * ``, the Workspace file effect for `` — so a replay restores both * without asking anyone anything a second time. + * + * An identity in that record is a closed tagged structural record: what kind of + * thing is behind the name, the origin that owns it, the component key, and the + * revision. Four terms rather than one spelling, so a reader comparing two + * admissions can say which of them moved and no two hosts have to agree on a + * separator. Version 1 — the untagged #369 record — wrote one opaque string + * instead, and stays both readable *and* resumable: a version-1 string is + * reconciled against a current capability identity under the spelling that + * build used. Nothing this build writes uses that spelling. */ import { createDurableOperation } from "@executablemd/durable-streams"; @@ -111,7 +120,7 @@ import { isComponentName } from "./components/registration.ts"; import { CORE_ORIGIN, CORE_REGISTRY } from "./components/registry.ts"; import { createBlockCounter, expandSegmentsWithin } from "./expand.ts"; import { extendPath } from "./expansion.ts"; -import { parseRequestRecord, prepareFetchRequest, requestRecord } from "./fetch-request.ts"; +import { prepareFetchRequest, requestRecord } from "./fetch-request.ts"; import { timeoutFetch } from "@executablemd/runtime"; import type { FetchRequest } from "./fetch-request.ts"; import { isJsonObject, parseJson } from "./json.ts"; @@ -184,6 +193,20 @@ const CEILING = const UNREADABLE = "the retained generated-XMD admission record cannot be read as one."; +/** + * What a resumed run is refused with when the text it now offers is not the + * text that was admitted. + * + * The admission is a decision about one exact fragment. Expanding the retained + * copy while the caller holds a different one would run something nobody + * admitted *this* run for — the earlier behavior, which silently preferred the + * retained text and let a changed candidate pass unnoticed. Naming neither + * fragment, because both are generated text. + */ +const STALE_TEXT = + "a generated fragment was admitted for text this run no longer offers. A retained admission " + + "resumes only for the exact text it was made about."; + /** * What an admitted invocation is refused with when its form is not the one its * identity was admitted for. @@ -249,6 +272,132 @@ export type GeneratedRequest = Record; */ export type GeneratedEffectClass = "read" | "write"; +/** + * Which implementation one admitted entry runs, as the run retains it. + * + * A closed tagged record rather than a string, and structural rather than + * assembled, because this is what a continuation compares. Four terms, each + * with one meaning: a reader looking at two admissions can say *which* of them + * moved, and no host has to agree with another about how to spell a separator. + * + * `kind` is part of the identity rather than beside it. An operation canonical + * core supplies the body for and an implementation the ordinary import chain + * answered with are different grants even under one origin, key and revision — + * the first cannot be composed around and the second was resolved through + * middleware — so a record that confused them would compare a fragment's + * authority equal to authority it never had. + * + * Never derived from a function. An implementation is not an identity, and + * serializing or inspecting one would make the retained policy depend on how a + * host happened to write its code. Functions never enter the journal. + */ +export type RetainedFragmentIdentity = + | { + readonly kind: "capability"; + readonly origin: string; + readonly key: string; + readonly revision: string; + } + | { + readonly kind: "component-answer"; + readonly origin: string; + readonly key: string; + readonly revision: string; + }; + +/** The two ways a host can say what is behind an admitted name. */ +const IDENTITY_KINDS: readonly RetainedFragmentIdentity["kind"][] = [ + "capability", + "component-answer", +]; + +/** + * What a record retains for one entry: this version's identity, or version 1's. + * + * Version 1 retained one opaque string. It is kept as the string it is rather + * than parsed into a shape it never had, and reconciled against a version-2 + * identity under the spelling the build that wrote it used — so an older record + * stays readable *and* stays a grant, instead of being readable and useless. + */ +type RetainedIdentity = RetainedFragmentIdentity | LegacyIdentity; + +/** One version-1 identity: the exact string an older build wrote. */ +interface LegacyIdentity { + readonly legacy: string; +} + +/** + * One identity as a map key, for the run's own tables. + * + * Never retained and never compared: what a continuation is held to is the + * record's four members, read one at a time. This exists because a `Map` needs + * a primitive, and it carries the kind so two grants that differ only in how + * they were resolved do not share a ceiling. + * + * The encoding is JSON rather than the four terms joined by a separator, + * because a joined key is only as unique as the separator is illegal. A host + * may state an origin or a key holding any character at all — a URL with a + * space in it, a component key with a `#` — and two identities that differ only + * in where the separator falls would key one entry's request ceiling under + * another's. JSON escapes what it encodes, so distinct terms encode distinctly. + */ +function identityKey(identity: RetainedFragmentIdentity): string { + return JSON.stringify([identity.kind, identity.origin, identity.key, identity.revision]); +} + +/** + * Whether two version-2 identities describe the same implementation. + * + * Member by member, kind included: an operation canonical core supplies the + * body for and an implementation the ordinary import chain answered with are + * different grants even under one origin, key and revision. + */ +function sameStructural(one: RetainedFragmentIdentity, other: RetainedFragmentIdentity): boolean { + return ( + one.kind === other.kind && + one.origin === other.origin && + one.key === other.key && + one.revision === other.revision + ); +} + +/** + * Whether the entry this run states is the one a retained record was admitted + * under. + * + * Two version-2 records compare structurally, and two version-1 strings compare + * as the strings they are. + * + * A version-1 string against a version-2 entry is the case that cannot be + * decided by looking at either one. Version 1 retained one opaque value chosen + * by whoever built the pinned entry — `@executablemd/core#File:read` from core, + * `@executablemd/workflow/composition/dir-v2#Dir` from the workflow host, and + * whatever a host passed to `pinnedComponent` from anyone else. No rule + * recovers four structural terms from one of those, and a rule that appeared to + * would only be reading the shapes core happens to use today back onto records + * core did not write. + * + * So it is not inferred. The *current* entry states which version-1 strings it + * succeeds, and a retained string reconciles only against that stated list. + * Listing one is an assertion by the party that owns the entry; an entry that + * lists none refuses every version-1 record naming it, which is the safe + * direction. New records state the structural identity and never a string. + */ +function sameIdentity(retained: RetainedIdentity, current: RetainedEntry): boolean { + const here = current.identity; + if ("legacy" in retained) { + if ("legacy" in here) { + return retained.legacy === here.legacy; + } + // Only a capability reconciles. The component-answer arm did not exist when + // version-1 records were written, so no such string ever described one — + // and the kind is checked here rather than left to the profile, because + // this is the comparison a continuation is actually held to. + return here.kind === "capability" && current.legacy?.includes(retained.legacy) === true; + } + return "legacy" in here ? false : sameStructural(retained, here); +} + /** * The authored forms one pinned identity runs in. * @@ -280,8 +429,27 @@ function authoredForms(form: GeneratedComponentForm): readonly AuthoredForm[] { */ export interface GeneratedObservation { readonly name: string; - readonly identity: string; + readonly identity: RetainedFragmentIdentity; readonly definition: FunctionComponentDefinition; + /** + * The form authority underneath this entry's implementation, when the + * definition wraps one. + * + * A host whose admitted definition is core's own guard around somebody else's + * implementation still has to say which function decides the authored form, + * because a selection reads that off the definition it was handed. Absent is + * the ordinary case: the definition's own `fn` is the authority. + */ + readonly dispatch?: unknown; + /** + * The exact version-1 identity strings this entry states it succeeds. + * + * Stated by whoever owns the entry rather than derived from the identity + * above, because version 1 retained one opaque value that nothing recovers + * four structural terms from. A retained version-1 record reconciles against + * this list and against nothing else. + */ + readonly legacy?: readonly string[]; /** * The exact requests this observation may perform, when it performs HTTP * reads at all. Present only on the pinned `` identity. @@ -300,6 +468,50 @@ export interface GeneratedObservation { readonly selfClosing?: boolean; } +/** + * The revision core's own entries state, wherever they are admitted from. + * + * One number for all of them, bumped whenever what any of these entries + * authorizes changes, so a continuation admitted under an earlier revision is + * refused rather than silently granted the newer authority. + * + * One constant rather than one per table. The pinned constructors here and the + * evaluation profile's entries describe the same operations, and a reader + * comparing two admissions has to be able to trust that they say so — two + * numbers that had to be moved together would eventually not be. + * + * Revision 2 is where the authority behind these entries changed: an admitted + * element used to invoke the ordinary component and resolve `API.Files` or + * `API.Fetch` wherever it happened to run, and now it invokes a body closed over + * the operations the host handed the profile. + */ +export const CORE_REVISION = "2"; + +/** One of core's own pinned identities, under core's origin and revision. */ +function coreIdentity(key: string): RetainedFragmentIdentity { + return { kind: "capability", origin: CORE_ORIGIN, key, revision: CORE_REVISION }; +} + +/** + * The exact strings core's own entries were retained as under version 1. + * + * Written out one by one, and deliberately not assembled from the origin and + * the key. There was never a rule producing these — `pinnedFetch` wrote its + * string, `pinnedFileRead` wrote its own, and a host writing a fourth was under + * no obligation to resemble either — so a function that built them would be + * inventing the rule this module exists to say does not exist. Enumerating them + * is also what makes the set reviewable: adding an alias is adding a line, and + * every line is an assertion that the current entry authorizes no more than the + * one that string named. + * + * Frozen and read by value at each call site. Nothing derives one, and a fifth + * released alias would be a fifth literal here rather than a broader pattern. + */ +const CORE_V1_FETCH = Object.freeze(["@executablemd/core#Fetch"]); +const CORE_V1_FILE_READ = Object.freeze(["@executablemd/core#File:read"]); +const CORE_V1_FILE_WRITE = Object.freeze(["@executablemd/core#File:write"]); +const CORE_V1_FILE_DELETE = Object.freeze(["@executablemd/core#File.Delete"]); + /** * The pinned core `` identity, bounded to exactly these requests. * @@ -321,7 +533,8 @@ export function pinnedFetch(requests: readonly GeneratedRequest[]): GeneratedObs } return { name: "Fetch", - identity: `${CORE_ORIGIN}#Fetch`, + identity: coreIdentity("Fetch"), + legacy: CORE_V1_FETCH, definition, requests: [...requests], }; @@ -348,7 +561,8 @@ export function pinnedFileRead(): GeneratedObservation { } return { name: "File", - identity: `${CORE_ORIGIN}#File:read`, + identity: coreIdentity("File:read"), + legacy: CORE_V1_FILE_READ, definition, selfClosing: true, }; @@ -360,10 +574,18 @@ export function pinnedFileRead(): GeneratedObservation { */ export function pinnedComponent( name: string, - identity: string, + identity: RetainedFragmentIdentity, definition: FunctionComponentDefinition, + /** + * The version-1 strings this host's own entry succeeds. + * + * The host's to state, because under version 1 this constructor took whatever + * string the host chose and retained it verbatim. Core has nothing to assert + * about which of those an entry is the successor of. + */ + legacy?: readonly string[], ): GeneratedObservation { - return { name, identity, definition }; + return { name, identity, definition, ...(legacy === undefined ? {} : { legacy }) }; } /** @@ -383,8 +605,12 @@ export function pinnedComponent( */ export interface GeneratedMutation { readonly name: string; - readonly identity: string; + readonly identity: RetainedFragmentIdentity; readonly definition: FunctionComponentDefinition; + /** The form authority underneath this entry's implementation, when it wraps one. */ + readonly dispatch?: unknown; + /** The exact version-1 identity strings this entry states it succeeds. */ + readonly legacy?: readonly string[]; readonly form: GeneratedComponentForm; } @@ -404,7 +630,8 @@ export function pinnedFileWrite(): GeneratedMutation { } return { name: "File", - identity: `${CORE_ORIGIN}#File:write`, + identity: coreIdentity("File:write"), + legacy: CORE_V1_FILE_WRITE, definition, form: "paired", }; @@ -436,7 +663,8 @@ export function pinnedFileDelete(): GeneratedMutation { } return { name: "File.Delete", - identity: `${CORE_ORIGIN}#File.Delete`, + identity: coreIdentity("File.Delete"), + legacy: CORE_V1_FILE_DELETE, definition, form: "self-closing", }; @@ -448,11 +676,13 @@ export function pinnedFileDelete(): GeneratedMutation { */ export function pinnedMutation( name: string, - identity: string, + identity: RetainedFragmentIdentity, definition: FunctionComponentDefinition, form: GeneratedComponentForm, + /** The version-1 strings this host's own entry succeeds, as the host states them. */ + legacy?: readonly string[], ): GeneratedMutation { - return { name, identity, definition, form }; + return { name, identity, definition, form, ...(legacy === undefined ? {} : { legacy }) }; } /** @@ -495,10 +725,18 @@ export interface GeneratedXmdRequest { readonly id: string; /** The candidate source, exactly as it was generated. */ readonly source: string; - /** The retained Workspace roots the host is willing to expose. */ - readonly workspaceRoots: readonly string[]; - /** The one root admitted effects address. */ - readonly selectedRoot: string; + /** + * The retained Workspace roots the host is willing to expose. + * + * Absent for a host that evaluates against no Workspace. An ordinary run is + * one: its admitted effects address the Files provider its own execution + * installed, and there is no immutable root history for a continuation to be + * held to. A workflow run states both, and a continuation is then held to + * that basis by membership. + */ + readonly workspaceRoots?: readonly string[]; + /** The one root admitted effects address, for a host that has one. */ + readonly selectedRoot?: string; /** The pinned observation identities the `read` class resolves to. */ readonly observations: readonly GeneratedObservation[]; /** The pinned mutation identities the `write` class resolves to. */ @@ -525,24 +763,37 @@ export interface GeneratedXmdRequest { */ interface Entry { readonly name: string; - readonly identity: string; + readonly identity: RetainedFragmentIdentity; readonly definition: FunctionComponentDefinition; + /** The form authority under this entry's definition, when a host stated one. */ + readonly dispatch?: unknown; readonly forms: readonly AuthoredForm[]; readonly effect: GeneratedEffectClass; + /** The version-1 identity strings this entry states it succeeds. */ + readonly legacy?: readonly string[]; readonly requests?: readonly GeneratedRequest[]; } /** The pinned identity of one admitted entry, as the run retains it. */ interface RetainedEntry { readonly name: string; - readonly identity: string; + readonly identity: RetainedIdentity; readonly forms: readonly AuthoredForm[]; + /** + * The version-1 strings this entry succeeds, on the *current* side only. + * + * Never read from a record and never written to one: a journal holds what was + * admitted, and which older grants a current entry is willing to answer for + * is a statement this run's host makes now. Reading it from the record would + * let a retained value nominate its own successor. + */ + readonly legacy?: readonly string[]; } /** One element the fragment actually named, as the run retains it. */ interface RetainedInvocation { readonly name: string; - readonly identity: string; + readonly identity: RetainedIdentity; readonly form: AuthoredForm; } @@ -557,8 +808,15 @@ interface RetainedInvocation { */ interface Policy { readonly allow: readonly GeneratedEffectClass[]; - readonly roots: readonly string[]; - readonly selectedRoot: string; + /** + * The Workspace basis, or nothing for a host that evaluates against none. + * + * The distinction is itself a ceiling. A run admitted with no Workspace and + * one admitted against a Workspace were granted different things, so a + * continuation that acquired or lost one is asking for a different grant + * rather than restating the same one. + */ + readonly workspace?: { readonly roots: readonly string[]; readonly selectedRoot: string }; readonly allowed: readonly RetainedEntry[]; readonly requests: readonly FetchRequest[]; } @@ -707,8 +965,11 @@ class GeneratedImportAuthority implements ImportAuthority { // it is the form authority. Remembered by name so `authorize` can record it // against core's own copy — the object expansion actually invokes — because // a trusted collection wrapper takes no part in deciding which - // form-specific body runs. - this.#dispatchers.set(name, implementation); + // form-specific body runs. A host that admitted a definition of its own + // states that authority explicitly, because an entry whose implementation + // is core's guard around somebody else's would otherwise offer the guard, + // and a form authority read off the guard selects no body at all. + this.#dispatchers.set(name, entry.dispatch ?? implementation); return this.#imports.issue(name, admitted); } @@ -786,10 +1047,12 @@ function selectedEntries( name: observation.name, identity: observation.identity, definition: observation.definition, + ...(observation.dispatch === undefined ? {} : { dispatch: observation.dispatch }), // An entry the host constrained to its self-closing spelling admits // that one; an unconstrained one admits both, as it always has. forms: observation.selfClosing === true ? ["self-closing"] : AUTHORED_FORMS, effect: "read", + ...(observation.legacy === undefined ? {} : { legacy: observation.legacy }), ...(observation.requests === undefined ? {} : { requests: observation.requests }), }); } @@ -806,8 +1069,10 @@ function selectedEntries( name: mutation.name, identity: mutation.identity, definition: mutation.definition, + ...(mutation.dispatch === undefined ? {} : { dispatch: mutation.dispatch }), forms: authoredForms(mutation.form), effect: "write", + ...(mutation.legacy === undefined ? {} : { legacy: mutation.legacy }), }); } } @@ -914,7 +1179,7 @@ function* normalizedCeilings(entries: readonly Entry[]): Operation root.length === 0)) { + throw new GeneratedXmdError("a generated-XMD host stated a Workspace basis retaining no root."); + } + if (new Set(roots).size !== roots.length) { + throw new GeneratedXmdError("a generated-XMD host stated one retained Workspace root twice."); + } + if (!roots.includes(selectedRoot)) { + throw new GeneratedXmdError( + "a generated-XMD host selected a Workspace root it does not retain.", + ); + } + return { roots, selectedRoot }; +} + +/** + * The policy as journal data, in the closed version-2 shape. + * + * Version 1 is the untagged #369 record, whose `roots` and `selectedRoot` sat + * at the top level and were mandatory. Version 2 tags itself and carries the + * Workspace basis as one optional member, because an ordinary host has none — + * and telling the two apart matters: a reader that treated a missing basis as + * an empty one would compare a Workspace-less admission equal to a Workspace + * admission that had lost every root. + */ function policyRecord(policy: Policy): JsonObject { return { + version: RECORD_VERSION, allow: [...policy.allow], - roots: [...policy.roots], - selectedRoot: policy.selectedRoot, + ...(policy.workspace === undefined + ? {} + : { + workspace: { + roots: [...policy.workspace.roots], + selectedRoot: policy.workspace.selectedRoot, + }, + }), allowed: policy.allowed.map((entry) => ({ name: entry.name, - identity: entry.identity, + identity: retainedIdentityRecord(entry.identity), forms: [...entry.forms], })), requests: policy.requests.map(requestRecord), }; } +/** + * One identity as journal data. + * + * Total over both retained shapes, so a policy read back is written back as the + * shape it was read as. What this run *states* is always the version-2 record — + * the version-1 arm exists because a record read from an older journal is still + * a policy, not because this build writes one. + * + * Distinct from the one-line spelling a *diagnostic* uses + * (`components/import-authority.ts`): that one is for a reader, and this one is + * what a continuation compares, so they are named apart rather than allowed to + * drift into each other. + */ +function retainedIdentityRecord(identity: RetainedIdentity): Json { + if ("legacy" in identity) { + return identity.legacy; + } + return { + kind: identity.kind, + origin: identity.origin, + key: identity.key, + revision: identity.revision, + }; +} + /** * The policy a record holds, parsed rather than trusted. * @@ -963,94 +1316,254 @@ function policyRecord(policy: Policy): JsonObject { * because it happens to have the right keys, and a policy this version cannot * read is refused rather than treated as matching. */ +/** + * Whether an object carries exactly these members and nothing else. + * + * Every retained shape below is closed, which is a stronger claim than "the + * members it needs are present and well-typed". A record carrying an extra + * member was written by something this build does not know the rules of, and + * reading the members it recognizes would be admitting a grant on terms it + * never saw. Destructuring alone cannot say that, because a destructure is + * blind to what it did not name. + */ +function exactly( + value: JsonObject, + required: readonly string[], + optional: readonly string[] = [], +): boolean { + if (!required.every((member) => Object.hasOwn(value, member))) { + return false; + } + const known = new Set([...required, ...optional]); + return Object.keys(value).every((member) => known.has(member)); +} + +/** + * The policy a record holds, in whichever closed shape its version defines. + * + * The two versions are exact and disjoint. Version 1 is the untagged #369 + * record: no version member, the Workspace basis as two mandatory top-level + * ones. Version 2 tags itself, carries the basis as one optional member, and + * has no legacy root fields at all — so a record mixing the two, or tagging + * itself with a version this build does not have, is refused rather than read + * as whichever it most resembles. + */ function readPolicy(value: Json): Policy | undefined { if (!isJsonObject(value)) { return undefined; } - const { allow, roots, selectedRoot, allowed, requests } = value; - if (!Array.isArray(roots) || typeof selectedRoot !== "string") { + const tagged = Object.hasOwn(value, "version"); + if (tagged) { + if (value.version !== RECORD_VERSION) { + return undefined; + } + if (!exactly(value, ["version", "allow", "allowed", "requests"], ["workspace"])) { + return undefined; + } + const workspace = Object.hasOwn(value, "workspace") + ? readWorkspace(value.workspace) + : undefined; + if (workspace === MALFORMED) { + return undefined; + } + return readPolicyTerms(value, workspace, 2); + } + if (!exactly(value, ["allow", "roots", "selectedRoot", "allowed", "requests"])) { return undefined; } - if (!Array.isArray(allow) || !Array.isArray(allowed) || !Array.isArray(requests)) { + // A version-1 record always evaluated against a Workspace, so its basis is + // mandatory and reads through the same validation a version-2 basis does. + const workspace = readWorkspace({ roots: value.roots, selectedRoot: value.selectedRoot }); + if (workspace === MALFORMED || workspace === undefined) { return undefined; } - const classes = readClasses(allow); - if (classes === undefined) { + return readPolicyTerms(value, workspace, 1); +} + +/** The terms both versions share, read closed. */ +function readPolicyTerms( + value: JsonObject, + workspace: { roots: readonly string[]; selectedRoot: string } | undefined, + version: 1 | 2, +): Policy | undefined { + const classes = readClasses(value.allow); + const identities = readAllowed(value.allowed, version); + const requests = readRequests(value.requests); + if (classes === undefined || identities === undefined || requests === undefined) { return undefined; } - const retainedRoots: string[] = []; - for (const root of roots) { - if (typeof root !== "string") { - return undefined; - } - retainedRoots.push(root); + return { + allow: classes, + ...(workspace === undefined ? {} : { workspace }), + allowed: identities, + requests, + }; +} + +/** A record this version cannot read, told apart from one that holds nothing. */ +const MALFORMED = Symbol("malformed"); + +/** + * One Workspace basis, validated rather than merely well-typed. + * + * A basis with no roots, a repeated root, or a selected root the run does not + * retain is not a stricter grant than one without those faults — it is not a + * grant at all, because there is nothing coherent for a continuation to be held + * to. So it is refused here, before the admission, rather than compared later + * against a live basis and appearing to hold. + */ +function readWorkspace( + value: Json | undefined, +): { roots: readonly string[]; selectedRoot: string } | undefined | typeof MALFORMED { + if (!isJsonObject(value) || !exactly(value, ["roots", "selectedRoot"])) { + return MALFORMED; + } + const roots = readRoots(value.roots); + const { selectedRoot } = value; + if (roots === undefined || typeof selectedRoot !== "string") { + return MALFORMED; } - const identities = readAllowed(allowed); - if (identities === undefined) { + return roots.includes(selectedRoot) ? { roots, selectedRoot } : MALFORMED; +} + +/** The retained roots, which are a non-empty set rather than a list. */ +function readRoots(value: Json | undefined): readonly string[] | undefined { + if (!Array.isArray(value) || value.length === 0) { return undefined; } - const retainedRequests: FetchRequest[] = []; - for (const request of requests) { - const parsed = readRequest(request); - if (parsed === undefined) { + const roots: string[] = []; + for (const root of value) { + if (typeof root !== "string" || root.length === 0 || roots.includes(root)) { return undefined; } - retainedRequests.push(parsed); + roots.push(root); } - return { - allow: classes, - roots: retainedRoots, - selectedRoot, - allowed: identities, - requests: retainedRequests, - }; + return roots; } -/** One retained request, or nothing when this version cannot read it. */ -function readRequest(value: Json): FetchRequest | undefined { - try { - return parseRequestRecord(value); - } catch { +/** + * The retained requests, read closed. + * + * `parseRequestRecord()` is the wrong reader here: it ignores members it does + * not know and treats a malformed `timeout` as an absent one, so a retained + * ceiling could compare equal to a live one it does not describe. + */ +function readRequests(value: Json): FetchRequest[] | undefined { + if (!Array.isArray(value)) { return undefined; } + const requests: FetchRequest[] = []; + for (const entry of value) { + if (!isJsonObject(entry) || !exactly(entry, ["url", "method", "headers"], ["timeout"])) { + return undefined; + } + const { url, method, headers, timeout } = entry; + if (typeof url !== "string" || typeof method !== "string" || !isJsonObject(headers)) { + return undefined; + } + if (Object.hasOwn(entry, "timeout") && typeof timeout !== "number") { + return undefined; + } + const named: Record = {}; + for (const [header, value] of Object.entries(headers)) { + if (typeof value !== "string") { + return undefined; + } + named[header] = value; + } + requests.push({ + url, + method, + headers: named, + ...(typeof timeout === "number" ? { timeout } : {}), + }); + } + return requests; } -function readClasses(value: readonly Json[]): GeneratedEffectClass[] | undefined { +/** The selected classes, which are a non-empty set in canonical order. */ +function readClasses(value: Json): GeneratedEffectClass[] | undefined { + if (!Array.isArray(value) || value.length === 0) { + return undefined; + } const classes: GeneratedEffectClass[] = []; for (const effect of value) { const parsed = EFFECT_CLASSES.find((known) => known === effect); - if (parsed === undefined) { + if (parsed === undefined || classes.includes(parsed)) { return undefined; } classes.push(parsed); } - return classes; + return sameStrings( + classes, + EFFECT_CLASSES.filter((known) => classes.includes(known)), + ) + ? classes + : undefined; } +/** The forms one entry is admitted for: a non-empty set, canonically ordered. */ function readForms(value: Json): AuthoredForm[] | undefined { - if (!Array.isArray(value)) { + if (!Array.isArray(value) || value.length === 0) { return undefined; } const forms: AuthoredForm[] = []; for (const form of value) { const parsed = AUTHORED_FORMS.find((known) => known === form); - if (parsed === undefined) { + if (parsed === undefined || forms.includes(parsed)) { return undefined; } forms.push(parsed); } - return forms; + return sameStrings( + forms, + AUTHORED_FORMS.filter((known) => forms.includes(known)), + ) + ? forms + : undefined; } -function readAllowed(value: readonly Json[]): RetainedEntry[] | undefined { +/** + * One retained identity, in whichever shape its version defines. + * + * The two are exact and disjoint. Version 1 is a string and nothing else. + * Version 2 is the closed tagged record and nothing else: a string, a missing + * member, an extra member, an unknown kind and a non-string member are each + * refused rather than read as whichever shape the value most resembles. + */ +function readIdentity(value: Json | undefined, version: 1 | 2): RetainedIdentity | undefined { + if (version === 1) { + return typeof value === "string" ? { legacy: value } : undefined; + } + if (!isJsonObject(value) || !exactly(value, ["kind", "origin", "key", "revision"])) { + return undefined; + } + const { origin, key, revision } = value; + const kind = IDENTITY_KINDS.find((known) => known === value.kind); + if ( + kind === undefined || + typeof origin !== "string" || + typeof key !== "string" || + typeof revision !== "string" + ) { + return undefined; + } + return { kind, origin, key, revision }; +} + +function readAllowed(value: Json, version: 1 | 2): RetainedEntry[] | undefined { + if (!Array.isArray(value)) { + return undefined; + } const identities: RetainedEntry[] = []; for (const entry of value) { - if (!isJsonObject(entry)) { + if (!isJsonObject(entry) || !exactly(entry, ["name", "identity", "forms"])) { return undefined; } - const { name, identity } = entry; + const { name } = entry; + const identity = readIdentity(entry.identity, version); const forms = readForms(entry.forms); - if (typeof name !== "string" || typeof identity !== "string" || forms === undefined) { + if (typeof name !== "string" || identity === undefined || forms === undefined) { return undefined; } identities.push({ name, identity, forms }); @@ -1058,15 +1571,19 @@ function readAllowed(value: readonly Json[]): RetainedEntry[] | undefined { return identities; } -function readNamed(value: readonly Json[]): RetainedInvocation[] | undefined { +function readNamed(value: Json, version: 1 | 2): RetainedInvocation[] | undefined { + if (!Array.isArray(value)) { + return undefined; + } const named: RetainedInvocation[] = []; for (const entry of value) { - if (!isJsonObject(entry)) { + if (!isJsonObject(entry) || !exactly(entry, ["name", "identity", "form"])) { return undefined; } - const { name, identity } = entry; + const { name } = entry; + const identity = readIdentity(entry.identity, version); const form = AUTHORED_FORMS.find((known) => known === entry.form); - if (typeof name !== "string" || typeof identity !== "string" || form === undefined) { + if (typeof name !== "string" || identity === undefined || form === undefined) { return undefined; } named.push({ name, identity, form }); @@ -1098,11 +1615,7 @@ function readNamed(value: readonly Json[]): RetainedInvocation[] | undefined { * the *current* policy would wave through. */ function policyHolds(retained: Policy, current: Policy): boolean { - const held = new Set(current.roots); - if (!retained.roots.every((root) => held.has(root))) { - return false; - } - if (!held.has(retained.selectedRoot) || !held.has(current.selectedRoot)) { + if (!workspaceHolds(retained.workspace, current.workspace)) { return false; } if (!sameStrings(retained.allow, current.allow)) { @@ -1116,7 +1629,9 @@ function policyHolds(retained: Policy, current: Policy): boolean { return ( here === undefined || here.name !== entry.name || - here.identity !== entry.identity || + // The retained side is what was granted; the current side is the entry + // asking to answer for it, and only that side states version-1 aliases. + !sameIdentity(entry.identity, here) || !sameStrings(here.forms, entry.forms) ); }); @@ -1132,6 +1647,32 @@ function policyHolds(retained: Policy, current: Policy): boolean { }); } +/** + * Whether a resumed run still holds the Workspace basis it was admitted over. + * + * Absent on both sides is a host that evaluates against no Workspace, and two + * of those hold each other. Present on one side only is a run that acquired or + * lost a Workspace between the admission and the resume, which is a different + * grant rather than the same one restated. + * + * Present on both compares by membership, because the set legitimately grows: + * every committed mutation retains another immutable root and advances the + * authoritative current one. So each admission root and the admission's + * selected root must still be retained, and the root the run now stands on must + * be a retained one — while additional roots change nothing this admission was + * granted under. + */ +function workspaceHolds(retained: Policy["workspace"], current: Policy["workspace"]): boolean { + if (retained === undefined || current === undefined) { + return retained === current; + } + const held = new Set(current.roots); + if (!retained.roots.every((root) => held.has(root))) { + return false; + } + return held.has(retained.selectedRoot) && held.has(current.selectedRoot); +} + /** One element the fragment named, and the entry preflight selected for it. */ interface Planned extends RetainedInvocation { readonly entry: Entry; @@ -1224,7 +1765,7 @@ function* walk( throw new Refusal(form === "paired" ? "content" : "form"); } if (entry.requests !== undefined) { - const ceiling = ceilings.get(entry.identity) ?? []; + const ceiling = ceilings.get(identityKey(entry.identity)) ?? []; const candidate = yield* admitCandidateRequest(segment.props); if (!ceiling.some((allowed) => sameRequest(allowed, candidate))) { throw new Refusal("request"); @@ -1243,6 +1784,17 @@ function* walk( const GENERATED_XMD = "generated_xmd"; +/** + * The record shape new executions write. + * + * Version 1 is the untagged #369 record and stays readable: a run suspended + * before this build resumes under exactly the ceilings it was admitted with. + * Version 2 tags itself, which is what lets an untagged record be recognized as + * the older shape rather than guessed at, and carries the Workspace basis as + * one optional member because an ordinary host has none. + */ +const RECORD_VERSION = 2; + /** * What the durable admission records for this source. * @@ -1260,18 +1812,23 @@ function* admitSource( try { const { named } = yield* preflight(source, table, ceilings); return parseJson({ + version: RECORD_VERSION, decision: "admitted", source, named: named.map((entry) => ({ name: entry.name, - identity: entry.identity, + identity: retainedIdentityRecord(entry.identity), form: entry.form, })), policy: policyRecord(policy), }); } catch (error) { if (error instanceof Refusal) { - return parseJson({ decision: "refused", construct: error.construct }); + return parseJson({ + version: RECORD_VERSION, + decision: "refused", + construct: error.construct, + }); } throw error; } @@ -1314,8 +1871,19 @@ function readAdmission(value: Json): RetainedAdmission | undefined { if (!isJsonObject(value)) { return undefined; } + // Tagged at the result level as well as inside the policy, and the two must + // agree: a record whose result claims one version and whose policy claims + // another describes no shape this build has. + const tagged = Object.hasOwn(value, "version"); + if (tagged && value.version !== RECORD_VERSION) { + return undefined; + } + const version = tagged ? ["version"] : []; const { decision } = value; if (decision === "refused") { + if (!exactly(value, [...version, "decision", "construct"])) { + return undefined; + } const { construct } = value; return typeof construct === "string" && isConstruct(construct) ? { decision, construct } @@ -1324,15 +1892,24 @@ function readAdmission(value: Json): RetainedAdmission | undefined { if (decision !== "admitted") { return undefined; } - const { source, named, policy } = value; - if (typeof source !== "string" || !Array.isArray(named) || policy === undefined) { + if (!exactly(value, [...version, "decision", "source", "named", "policy"])) { + return undefined; + } + const { source } = value; + if (typeof source !== "string") { return undefined; } - const invocations = readNamed(named); - const retained = readPolicy(policy); + const invocations = readNamed(value.named, tagged ? 2 : 1); + const retained = readPolicy(value.policy); if (invocations === undefined || retained === undefined) { return undefined; } + // The policy's own version has to be the one the result claimed. A tagged + // result holding an untagged policy, or the reverse, is two shapes at once. + const policyTagged = isJsonObject(value.policy) && Object.hasOwn(value.policy, "version"); + if (policyTagged !== tagged) { + return undefined; + } return { decision, source, named: invocations, policy: retained }; } @@ -1439,9 +2016,16 @@ export function* evaluateGeneratedXmd( if (!policyHolds(decided.policy, policy)) { throw new GeneratedXmdError(CEILING); } + // And for the exact text, on the same terms as the ceilings. An admission is + // a decision about one fragment; a caller now holding a different one is + // asking for a decision that was never made, so it refuses here rather than + // quietly expanding the retained copy in its place. + if (decided.source !== request.source) { + throw new GeneratedXmdError(STALE_TEXT); + } - // The retained source is what expands, so a continuation runs the fragment - // this run admitted rather than whatever a later caller happens to hold. + // The retained source is what expands, so a continuation runs exactly the + // bytes this run admitted rather than a caller's copy of them. const restored = yield* preflight(decided.source, table, ceilings); return yield* expand(request.id, restored.segments, restored.named); } diff --git a/packages/core/src/inspect.ts b/packages/core/src/inspect.ts index f2dca639b..8757e3c7b 100644 --- a/packages/core/src/inspect.ts +++ b/packages/core/src/inspect.ts @@ -3,6 +3,7 @@ import { readTextFile } from "@executablemd/runtime"; import type { ComponentOrigin, + ComponentRegistry, ComponentSelection, InvocationForm, PropsSchema, @@ -21,6 +22,8 @@ import { declaredRegistry } from "./components/declared-registry.ts"; import { admitDeclaredMarkdown, declaredCatalog } from "./components/declared-markdown.ts"; import type { DeclaredMarkdownComponent } from "./components/declared-markdown.ts"; import { repositoryCandidateNames } from "./components/candidates.ts"; +import { PROTECTED_COMPONENT_NAMES } from "./components/protected.ts"; +import type { WorkflowImportAuthority } from "./components/bundle.ts"; import { documentationOf } from "./components/documentation.ts"; import type { ComponentDocumentation } from "./components/documentation.ts"; import { STRUCTURAL_DECLARATIONS } from "./structural.ts"; @@ -155,6 +158,19 @@ export type ComponentInfo = props: PropsSchema; returns?: ReturnsSchema; } & DescribedContract) + /** + * A component canonical core claims the name of. + * + * Its own `kind` for the same reason its origin has one: a caller asking what + * `Syntax` is needs to learn that no registry supplies it and none can, which + * `registered` said the opposite of. + */ + | ({ + kind: "protected"; + origin: ComponentOrigin; + props: PropsSchema; + returns?: ReturnsSchema; + } & DescribedContract) | ({ kind: "markdown"; origin: ComponentOrigin; @@ -167,7 +183,7 @@ export type ComponentInfo = /** * What a fully describable component reports beyond its schemas. * - * The same values the catalog carries, built by the same code, so describing + * The same values the symbols carry, built by the same code, so describing * one name and describing the whole environment cannot disagree. `returns` * above stays the *declared* schema — absent in text mode — while `returnMode` * is what tells the two apart. @@ -205,6 +221,16 @@ export function* inspectComponent(options: InspectComponentOptions): Operation; - readonly sourceKind: "registered" | "markdown" | "declared-markdown"; + /** + * What kind of thing supplied the contract above. + * + * `protected` and `workflow-markdown` are version 2's additions. Both were + * previously folded into a neighbour — `registered` and `markdown` — which + * made a symbols reader unable to tell core's own component from a host's + * registration, or a pinned bundle member from a file on disk. + */ + readonly sourceKind: + | "registered" + | "protected" + | "markdown" + | "workflow-markdown" + | "declared-markdown"; readonly inspectability: "complete"; readonly forms: readonly ("self-closing" | "paired")[]; readonly props: PropsSchema; @@ -364,6 +411,28 @@ export interface OriginOnlyComponentSyntaxEntry { export interface InspectSyntaxOptions { /** Where to look, matching the search path execution uses. */ readonly includes?: readonly string[]; + /** + * The registrations to describe, when the caller holds them. + * + * Omitted reads whatever the calling scope installed, which is what `xmd + * syntax` wants — it assembles the profile it is describing and then asks. An + * execution passes the registry it *captured* instead: what a document may + * write is decided by the registrations the execution started with, and a + * nested `registerComponents()` somewhere inside it does not change the + * environment the run was assembled as. + */ + readonly registry?: ComponentRegistry; + /** + * The component bundle the execution being described is closed over. + * + * Omitted for an inspection: `xmd syntax` installs no bundle, so it describes + * a document rather than a run. An execution that has one passes it, because + * leaving it out would describe a workflow root as having none of the + * components its own pinned tree supplies. Nothing here imports or executes a + * bundle member: the pinned source is already in hand, and describing it + * parses the same bytes execution would. + */ + readonly workflow?: WorkflowImportAuthority; /** * Identity components the host would declare to an execution, with the same * meaning `ExecuteOptions.components` gives them — admissibility included. @@ -379,7 +448,7 @@ export interface InspectSyntaxOptions { * The exact Markdown a host would declare to an execution, with the same * meaning `ExecutionInstallation.declarations` gives it — admissibility * included. A private declaration contributes no entry: it is not a name a - * document can write, so a catalog that listed it would describe syntax that + * document can write, so symbols that listed it would describe syntax that * does not exist. */ readonly declarations?: readonly DeclaredMarkdownComponent[]; @@ -400,8 +469,9 @@ export interface InspectSyntaxOptions { * is for execution. This adds the names to ask about and the shape of the * answer. */ -export function* inspectSyntax(options: InspectSyntaxOptions): Operation { +export function* inspectSyntax(options: InspectSyntaxOptions): Operation { const includes = options.includes ?? DEFAULT_INCLUDES; + const bundled = options.workflow; const declared = options.components ?? []; // The whole declaration set is admitted before anything is built from it, on // exactly the terms ordinary execution admits it on. A set an execution would @@ -411,7 +481,10 @@ export function* inspectSyntax(options: InspectSyntaxOptions): Operation { + if (selected.kind === "protected") { + const { component, origin } = selected; + // Described from the declaration alone. The factory is never called: it + // takes an execution's claimant, and describing an environment mints no + // execution and no claimant to give it. + return complete(name, origin, "protected", { + forms: component.forms ?? BOTH_FORMS, + props: component.props, + captures: component.captures ?? [], + returns: component.returns, + documentation: documentationOf(component), + }); + } + if (selected.kind === "registered") { const { definition, origin } = selected; if (origin.kind === "structural") { @@ -540,10 +642,32 @@ function* componentEntry( }); } + if (selected.kind === "workflow") { + // The pinned bytes, already in hand: the bundle was read from the + // definition's own commit before this execution existed, so describing one + // reads no file, imports no module and runs nothing. Reported at the + // canonical path it holds inside that commit *and* by the blob's own object + // id, because the path alone would read as a repository candidate — a + // mutable file that happens to sit there now — and this is the exact blob + // the run was defined against. + const definition = yield* parseMarkdownDefinition(name, selected.path, selected.content); + return complete( + name, + { kind: "workflow", path: selected.path, sourceHash: selected.sourceHash }, + "workflow-markdown", + { + forms: BOTH_FORMS, + props: definition.props, + captures: [], + returns: definition.returns, + documentation: documentationOf(definition.meta), + }, + ); + } + if (selected.kind !== "repository") { - // A bundled or unresolved name describes no environment a document writes - // in: inspection installs no bundle, and a name nothing supplies is exactly - // the absence the catalog reports by leaving it out. + // An unresolved name is exactly the absence the symbols report by leaving + // it out. return undefined; } @@ -586,7 +710,7 @@ interface CompleteContract { function complete( name: string, origin: Exclude, - sourceKind: "registered" | "markdown" | "declared-markdown", + sourceKind: CompleteComponentSyntaxEntry["sourceKind"], contract: CompleteContract, ): CompleteComponentSyntaxEntry { return { diff --git a/packages/core/src/invocation-identity.ts b/packages/core/src/invocation-identity.ts index 259e71e6c..624b5ce96 100644 --- a/packages/core/src/invocation-identity.ts +++ b/packages/core/src/invocation-identity.ts @@ -52,6 +52,8 @@ import type { Operation, Scope } from "effection"; import { printErrors, printsErrors } from "./component-failures.ts"; import { documentationOf } from "./components/documentation.ts"; import type { ComponentDocumentation } from "./components/documentation.ts"; +import type { SyntaxReference } from "./syntax-reference.ts"; +import type { CapturedProfile } from "./evaluation-profile.ts"; import type { FunctionComponent, FunctionComponentDefinition, @@ -151,6 +153,144 @@ export interface IdentityDomain { readonly component: string; } +/** + * What the body of a component canonical core protects receives. + * + * A protected component is selected by canonical core ahead of every host or + * author tier, so its name is settled before anything a document, a package or + * middleware can reach. What is *not* settled by the name is the lexical fact it + * reads — the syntax reference in scope where the element was written — and + * that changes as expansion descends, so it cannot be closed over when the + * implementation is built. It is delivered here instead, by the copy of core + * performing the expansion, from the authority that copy is already holding. + * + * The observation is `undefined` where an expansion carries none. The body + * refuses rather than inventing symbols: a component that answered without one + * would be describing an environment nothing established. + */ +export type ProtectedBody = ( + props: Record, + invocation: ComponentInvocation, + site: ProtectedSite, +) => Operation; + +/** + * Render this element's own paired content under a syntax reference of the + * body's choosing. + * + * The one facility canonical protected dispatch offers that ordinary expansion + * does not, and it exists for one reason: a boundary that narrows what a + * subtree may write has to narrow it *while the subtree renders*, and the + * projection handle is built before the body runs. So the body is handed a way + * to project its own region again under a different reference. + * + * It projects exactly the paired region, returns exactly what that region + * rendered under ordinary content-failure semantics, and reuses the invocation's + * own projection machinery and content scope — it is not a second expansion. + * Nothing public reaches it: it is not on `Component`, not on the invocation, + * not on the reference, not in a context, and not exported. It is one-shot and + * bound to this invocation, so a body that kept it cannot project a second time + * or act after the body closed. + */ +export type ProjectProtectedContent = (syntax: SyntaxReference) => Operation; + +/** + * The lexical facts one protected body is handed, beside its props. + * + * All three change as expansion descends or belong to this invocation alone, so + * none of them can be closed over when the implementation is built. They are + * delivered here instead, by the copy of core performing the expansion, from + * the private authority it is already holding. + */ +export interface ProtectedSite { + /** + * The syntax reference in scope where the element was written. + * + * `undefined` where an expansion carries none. A body refuses rather than + * inventing symbols: a component that answered without one would be + * describing an environment nothing established. + */ + readonly syntax: SyntaxReference | undefined; + /** + * The maximum authority a generated fragment may be evaluated under here. + * + * `undefined` where the host offers no evaluation, which is not an + * unrestricted one. + */ + readonly evaluation: CapturedProfile | undefined; + /** How this body renders its own paired content, when it has any. */ + readonly projectContent: ProjectProtectedContent | undefined; +} + +/** + * The bodies one execution will enter, keyed by the exact function it built. + * + * Execution-owned and reclaimed with the execution, like the domains beside it, + * and handed to core's own expansion by value. Two things follow. An + * implementation another loaded copy built — which is ordinary, because a + * component can be loaded from disk beside its own copy — is in that copy's + * table rather than this one, so it has no body here. And an implementation kept + * past this execution's teardown reaches nothing, because the table went with + * the execution. + */ +export interface ProtectedBodies { + /** The body canonical expansion may enter for this exact implementation. */ + body(fn: unknown): ProtectedBody | undefined; +} + +interface ProtectedInstallation extends ProtectedBodies { + /** + * Build one implementation and keep its real body here. + * + * What comes back is inert: it is what registration-shaped machinery compares + * by identity and what a form selection records, and calling it through any + * route but canonical expansion refuses rather than observing anything. + */ + implementation( + name: string, + build: (claim: IdentityClaimant) => ProtectedBody, + claim: IdentityClaimant, + ): FunctionComponent; +} + +function createProtectedBodies(): ProtectedInstallation { + const bodies = new WeakMap(); + return { + implementation(name, build, claim): FunctionComponent { + // deno-lint-ignore require-yield + function* unreachable(): Operation { + throw new ComponentInvocationError( + `<${name} /> is invoked by canonical core, so an implementation reached any other way ` + + "observes nothing", + ); + } + bodies.set(unreachable, build(claim)); + return unreachable; + }, + body(fn): ProtectedBody | undefined { + return typeof fn === "function" ? bodies.get(fn) : undefined; + }, + }; +} + +/** + * A component canonical core claims the name of, as this module builds one. + * + * The same declaration a host's identity component makes, except that the + * factory hands over a body rather than an implementation: what a protected + * component may be invoked through is the execution's to decide, not the + * declaration's. + */ +export interface ProtectedDeclaration { + readonly name: string; + readonly props: PropsSchema; + readonly returns?: ReturnsSchema; + readonly captures?: readonly string[]; + readonly forms?: readonly InvocationForm[]; + readonly origin: string; + build(claim: IdentityClaimant): ProtectedBody; +} + /** * What one execution knows about the components it gave identity to. * @@ -388,7 +528,7 @@ export function isFormDispatcher(fn: unknown): boolean { * The only ways a forms declaration may be written. * * A closed list rather than a set membership test, because the *order* is part - * of the declaration: one canonical spelling per meaning means a catalog can be + * of the declaration: one canonical spelling per meaning means two entries can be * compared without normalizing, and a reader never has to wonder whether * `["paired", "self-closing"]` said something different. */ @@ -706,7 +846,9 @@ function mintDomain(component: string): Minted { * this one function so the two cannot come to disagree about what a host may * declare. */ -export function assertDistinctIdentityNames(components: readonly IdentityComponent[]): void { +export function assertDistinctIdentityNames( + components: readonly { readonly name: string }[], +): void { const seen = new Set(); for (const component of components) { if (seen.has(component.name)) { @@ -798,6 +940,17 @@ export interface IdentityInstallation { * the declaration that carries it. */ readonly privates: ReadonlyMap; + /** + * The implementations canonical core's own protected tier resolves, by name. + * + * Minted exactly like the rest — one domain, one claimant, revoked with the + * execution — and then registered nowhere, because a registry is precisely + * what must not decide a protected name. Canonical resolution answers for + * these from its own table (`components/protected.ts`). + */ + readonly protected: ReadonlyMap; + /** The bodies canonical expansion may enter for those implementations. */ + readonly protectedBodies: ProtectedBodies; /** Called once the registrations have been validated and committed. */ activate(): void; } @@ -814,17 +967,45 @@ export interface IdentityInstallation { export function installIdentities( components: readonly IdentityComponent[], privateComponents: readonly IdentityComponent[] = [], + protectedComponents: readonly ProtectedDeclaration[] = [], ): IdentityInstallation { // Before any factory: a set nobody can register is a set nobody may build // implementations from either, and a duplicate that reached a factory would - // have minted a claimant for a domain that is about to be discarded. The two + // have minted a claimant for a domain that is about to be discarded. The three // sets are checked together because they mint into one table of domains, so a - // private name that shadowed a registered one would take its domain. - assertDistinctIdentityNames([...components, ...privateComponents]); + // private name that shadowed a registered one would take its domain — and a + // host that declared a protected name reaches this only after admission has + // already refused it, so a duplicate here is core's own mistake. + assertDistinctIdentityNames([...components, ...privateComponents, ...protectedComponents]); const minted = new Map(); const registrations: IdentityRegistration[] = []; const privates = new Map(); + const guarded = new Map(); + const protectedBodies = createProtectedBodies(); + for (const component of protectedComponents) { + const domain = mintDomain(component.name); + minted.set(component.name, domain); + const implementation = protectedBodies.implementation( + component.name, + component.build, + domain.claim, + ); + domain.implementation = implementation; + // Not marked private: a protected implementation is resolved by canonical + // core's own tier under its public name, so refusing it wherever an answer + // becomes something the engine invokes would refuse the component itself. + guarded.set(component.name, { + kind: "function", + name: component.name, + props: component.props, + ...(component.returns === undefined ? {} : { returns: component.returns }), + ...(component.captures === undefined ? {} : { captures: component.captures }), + ...(component.forms === undefined ? {} : { forms: component.forms }), + ...documentationOf(component), + fn: implementation, + }); + } for (const component of privateComponents) { const domain = mintDomain(component.name); minted.set(component.name, domain); @@ -915,6 +1096,8 @@ export function installIdentities( }, registrations, privates, + protected: guarded, + protectedBodies, activate: () => { for (const domain of minted.values()) { domain.activate(); diff --git a/packages/core/src/protected-content.ts b/packages/core/src/protected-content.ts new file mode 100644 index 000000000..0535f3079 --- /dev/null +++ b/packages/core/src/protected-content.ts @@ -0,0 +1,107 @@ +/** + * The lifetime of a protected body's one projection of its own content. + * + * Canonical dispatch hands a protected body a callback that renders that + * element's content once. The callback is authority — it projects the + * document's own children under a reference the body chose — so how long it + * lives and how many times it answers are not details. This module owns exactly + * that question, and nothing else: what to project, under which authority, in + * which scope and with which error mode all stay in `expand.ts`, which already + * holds them. What arrives here is one already-bound operation. + * + * ## Why it is its own module + * + * Because the state machine is the part worth testing directly. Every guard + * below is unreachable from a document — only canonical `` consumes a + * projector, and it consumes one once — so a black-box test cannot make a + * second call, cannot retain a callback past a body, and cannot race two. A + * test that could would need a protected component of its own, which is a hole + * in the tier this exists to protect. Splitting the machine out is what lets the + * guards be proved without opening one. + * + * It is package-internal on purpose. `expand.ts` imports it by relative path and + * so does its own unit test; it appears in no barrel, no `host.ts`, no package + * export, no Context, no `Component` api and no test-support surface. There is + * nothing here for a document, a repository component, middleware or a second + * loaded copy to reach. + * + * ## Three states, not two flags + * + * `open` → `spent` when a projection is consumed, and either → `closed` when + * the body returns. Two booleans would admit a fourth combination nobody means, + * and would leave "already projected" and "the body has finished" reporting the + * same thing when they are different facts about different mistakes. + * + * Consumption happens *before* the underlying operation can suspend. A machine + * that marked itself spent on completion would let two calls overlap while the + * first was still running, which is the concurrent case rather than a + * theoretical one: a body that spawns is ordinary Effection. + */ + +import type { Operation } from "effection"; + +import { ComponentInvocationError } from "./invocation-identity.ts"; +import type { ProjectProtectedContent } from "./invocation-identity.ts"; +import type { SyntaxReference } from "./syntax-reference.ts"; + +/** What a protected body holds, and what canonical dispatch closes. */ +export interface ProtectedContentLease { + /** The callback the body may call once, while it is running. */ + readonly project: ProjectProtectedContent; + /** + * End it. + * + * Called from the same `finally` that closes the invocation's issued + * authority, so a callback a body retained — in a closure, on an object it + * returned, in something it spawned — refuses rather than projecting content + * of an element that has finished. + */ + close(): void; +} + +/** Where one lease is in its life. */ +type LeaseState = "open" | "spent" | "closed"; + +/** + * One lease over one already-bound projection. + * + * `perform` is called at most once, and only from the `open` state. Everything + * this adds is the refusal: the operation itself is `expand.ts`'s, closed over + * the projection state, the authority and the scope that element actually has. + */ +export function protectedContentLease( + name: string, + perform: ProjectProtectedContent, +): ProtectedContentLease { + let state: LeaseState = "open"; + return { + *project(syntax: SyntaxReference): Operation { + // Read and moved before anything can suspend. A second call that arrived + // while the first was still running would otherwise find `open` and + // project the same content again, under a reference the first call did + // not choose. + if (state !== "open") { + throw new ComponentInvocationError(refusal(name, state)); + } + state = "spent"; + return yield* perform(syntax); + }, + close(): void { + state = "closed"; + }, + }; +} + +/** + * What a refused call says, chosen by which mistake it was. + * + * A second call inside a running body and a call from a callback that outlived + * the body are different errors with different fixes, and reporting them + * identically would tell an author to look in the wrong place. + */ +function refusal(name: string, state: LeaseState): string { + return state === "spent" + ? `<${name}> renders its own content once, and this invocation has already rendered it.` + : `<${name}> renders its own content while its body is running, and this invocation has ` + + "finished."; +} diff --git a/packages/core/src/syntax-admitted.ts b/packages/core/src/syntax-admitted.ts new file mode 100644 index 000000000..c2dcf0027 --- /dev/null +++ b/packages/core/src/syntax-admitted.ts @@ -0,0 +1,75 @@ +/** + * The vocabulary one admitted evaluation has, as symbols. + * + * An agent asked to write a fragment is told what a fragment may contain. That + * is not the vocabulary of the site the `` element was written at: + * the evaluator admits the host's pinned identities and nothing else — no + * executable code block, no expression prop, no interpolation, no `as` binding, + * and none of the structural constructs an ordinary document has. + * + * So the symbols built here say exactly that. The structural and built-in + * categories are empty, because a fragment writes neither, and the components + * are the entries the selection resolved to. Anything else would describe a + * vocabulary the fragment does not have, and an agent told it had `` + * would produce a fragment the evaluator refuses whole. + * + * What an author may *read about* is a separate question, and this does not + * answer it: `SyntaxReference.available()` keeps the enclosing documentation + * index and replaces only what may run. + */ + +import type { CapturedEntry } from "./evaluation-profile.ts"; +import type { CompleteComponentSyntaxEntry, SyntaxSymbols } from "./inspect.ts"; +import type { ReturnsSchema } from "./types.ts"; + +/** The default return contract of a component that declares none. */ +const UNDECLARED: ReturnsSchema = { type: "string" }; + +/** + * The symbols an evaluation admitted, in the order the tables state them. + * + * The identity's origin is what each entry reports it came from, because that + * is what the host stated and what a continuation is compared against. Reported + * as a registration rather than as core's own protected tier even for core's + * pinned ``: inside a fragment it is an identity the *host* admitted, and + * a fragment cannot re-register or shadow anything at all. + */ +export function admittedSymbols(entries: readonly CapturedEntry[]): SyntaxSymbols { + return { + version: 2, + categories: [ + { kind: "structural", entries: [] }, + { kind: "built-in", entries: [] }, + { kind: "user-provided", entries: entries.map(describe) }, + ], + }; +} + +function describe(entry: CapturedEntry): CompleteComponentSyntaxEntry { + const definition = entry.definition; + return { + kind: "component", + name: entry.name, + origin: { kind: "registered", origin: entry.identity.origin, reserved: false }, + sourceKind: "registered", + inspectability: "complete", + // The forms the *host admitted this entry for*, which is narrower than the + // forms the implementation accepts whenever one name holds two identities: + // `` observes and `…` writes, and a fragment told it + // had both would be told it could write under a read selection. + forms: [...entry.forms], + props: entry.props, + captures: definition.captures === undefined ? [] : [...definition.captures], + returnMode: definition.returns === undefined ? "text" : "value", + returns: definition.returns ?? UNDECLARED, + // The host's own prose about the admitted entry, not the capability body's. + // Core supplies the operation; only the host knows what admitting it under + // this name means here, and a catalog that named a component and said + // nothing about it would tell an agent a name and no more. + ...(entry.description === undefined ? {} : { description: entry.description }), + // `as` and `context` are deliberately absent whatever the implementation + // declares: a fragment may bind nothing, and the evaluator refuses an `as` + // before the first effect. Describing them would document a spelling that + // refuses the whole fragment. + }; +} diff --git a/packages/core/src/syntax-markdown.ts b/packages/core/src/syntax-markdown.ts new file mode 100644 index 000000000..3d11437e0 --- /dev/null +++ b/packages/core/src/syntax-markdown.ts @@ -0,0 +1,303 @@ +/** + * The symbols as Markdown a person reads. + * + * One renderer, in core, because two things print it: `xmd syntax`, which + * describes an environment without running it, and canonical ``, which + * hands the same text to a document that is running. Rendering here is what + * makes those two answers the same bytes for the same site — a renderer the CLI + * owned could only be reached by the CLI, and a component in core would have + * needed a second one. + * + * It takes the symbols as a value. It discovers nothing, reads no filesystem, + * resolves no name and parses no other projection's output, so what it prints is + * exactly what construction decided. + * + * Markdown is written for a person: where a schema carries more than a table can + * summarize honestly, the table says so and the schema is printed beside it. + */ + +import type { + CompleteComponentSyntaxEntry, + OriginOnlyComponentSyntaxEntry, + StructuralSyntaxEntry, + SyntaxSymbols, +} from "./inspect.ts"; +import { NO_DOCUMENTATION } from "./documentation-index.ts"; +import type { ComponentOrigin, Json, PropsSchema } from "./types.ts"; + +/** The three category kinds, taken from the symbols rather than restated. */ +type CategoryKind = SyntaxSymbols["categories"][number]["kind"]; + +const HEADINGS: Record = { + structural: "## Built-in structural syntax", + "built-in": "## Built-in components", + "user-provided": "## User-provided components", +}; + +const EMPTY: Record = { + structural: "No structural constructs are reserved.", + "built-in": "No components are registered in this profile.", + "user-provided": "No components were found in the configured includes.", +}; + +export function renderSyntaxMarkdown(symbols: SyntaxSymbols): string { + const sections = symbols.categories.map((category) => { + const blocks: string[] = [HEADINGS[category.kind]]; + if (category.entries.length === 0) { + blocks.push(EMPTY[category.kind]); + return blocks.join("\n\n"); + } + for (const entry of category.entries) { + blocks.push(...renderEntry(entry)); + } + return blocks.join("\n\n"); + }); + return `${sections.join("\n\n")}\n`; +} + +/** One symbol entry, as the named form selects it. */ +export interface SelectedEntry { + readonly entry: + | StructuralSyntaxEntry + | CompleteComponentSyntaxEntry + | OriginOnlyComponentSyntaxEntry; + /** The long-form documentation this entry has, if it has any. */ + readonly documentation: string | undefined; + /** + * Whether the current evaluation can actually run this component. + * + * Stated rather than implied, because the named form reads from the enclosing + * authoring symbols: inside a narrowed evaluation it can explain a component + * the evaluation may not execute, and a reader shown documentation with no + * word about availability would reasonably assume they had both. + */ + readonly available: boolean; +} + +/** + * The selected entries, each with its metadata and its long-form documentation. + * + * What `` and `xmd syntax Elicit` both render — one renderer, + * so the component and the command cannot describe one component two ways. + */ +export function renderSelectedDocumentation(selected: readonly SelectedEntry[]): string { + const sections = selected.map((one) => { + const blocks = renderEntry(one.entry); + blocks.push(`**Available in this evaluation:** ${one.available ? "yes" : "no"}`); + blocks.push(one.documentation ?? NO_DOCUMENTATION); + return blocks.join("\n\n"); + }); + return `${sections.join("\n\n")}\n`; +} + +function renderEntry( + entry: StructuralSyntaxEntry | CompleteComponentSyntaxEntry | OriginOnlyComponentSyntaxEntry, +): string[] { + if (entry.kind === "structural") { + return renderStructural(entry); + } + if (entry.inspectability === "origin-only") { + return renderOriginOnly(entry); + } + return renderComponent(entry); +} + +function heading(name: string): string { + return `### \`<${name}>\``; +} + +function renderStructural(entry: StructuralSyntaxEntry): string[] { + const blocks = [heading(entry.name), entry.description]; + blocks.push("**Syntax:**", fence("md", entry.syntax.join("\n"))); + blocks.push(...prose(entry)); + return blocks; +} + +function renderOriginOnly(entry: OriginOnlyComponentSyntaxEntry): string[] { + return [ + heading(entry.name), + "This component is a repository TypeScript module. Its contract lives on the module's " + + "exports, and reading it would import the module and run its top-level code — which " + + "describing an environment must not do. The module was not imported, so its props, " + + "captures, forms and return are unavailable here.", + `**Origin:** ${describeOrigin(entry.origin)}`, + ]; +} + +function renderComponent(entry: CompleteComponentSyntaxEntry): string[] { + const blocks = [heading(entry.name)]; + if (entry.description !== undefined) { + blocks.push(entry.description); + } + blocks.push(`**Forms:** ${entry.forms.map((form) => invocation(entry.name, form)).join(", ")}`); + blocks.push(...renderProps(entry.props)); + if (entry.captures.length > 0) { + blocks.push( + `**Captures:** ${entry.captures.map(code).join(", ")} — evaluated by the component ` + + "itself, so these props are deliberately absent from the schema above.", + ); + } + blocks.push(...prose(entry)); + blocks.push(...renderReturns(entry)); + blocks.push(`**Origin:** ${describeOrigin(entry.origin)}`); + return blocks; +} + +function prose(entry: { as?: string; context?: string }): string[] { + const blocks: string[] = []; + if (entry.as !== undefined) { + blocks.push(`**\`as\`:** ${entry.as}`); + } + if (entry.context !== undefined) { + blocks.push(`**Body context:** ${entry.context}`); + } + return blocks; +} + +function invocation(name: string, form: "self-closing" | "paired"): string { + return code(form === "self-closing" ? `<${name} />` : `<${name}>…`); +} + +function renderReturns(entry: CompleteComponentSyntaxEntry): string[] { + if (entry.returnMode === "text") { + return [ + "**Returns:** text — the markdown this component renders.", + fence("json", stringify(entry.returns)), + ]; + } + return [ + "**Returns:** a value — it renders nothing, and `as` binds what it returns.", + fence("json", stringify(entry.returns)), + ]; +} + +/** + * The props table, and the schema it summarizes. + * + * The table is the readable half and the schema is the authoritative one. A + * table cannot carry `default`, `enum`, a combinator, a reference or a root + * constraint, so the schema is printed beside it rather than reduced into it, + * and a property the table cannot name a type for is labelled honestly instead + * of being given an invented one. + */ +function renderProps(props: PropsSchema): string[] { + const rows = propertyRows(props); + const blocks = ["#### Props"]; + if (rows.length === 0) { + blocks.push("This component declares no individual props."); + } else { + blocks.push( + ["| Prop | Type | Required | Description |", "| --- | --- | --- | --- |", ...rows].join("\n"), + ); + } + blocks.push(fence("json", stringify(props))); + return blocks; +} + +function propertyRows(props: PropsSchema): string[] { + const properties = props.properties; + if (typeof properties !== "object" || properties === null || Array.isArray(properties)) { + return []; + } + const required = new Set( + Array.isArray(props.required) + ? props.required.filter((name): name is string => typeof name === "string") + : [], + ); + const rows: string[] = []; + for (const [name, schema] of Object.entries(properties)) { + // Every cell is escaped on the way in, the prop name included: a schema + // property may be spelled with anything, and one pipe in a name would + // shift every column after it. + rows.push( + row([ + code(name), + summarizeType(schema), + required.has(name) ? "yes" : "no", + describeProp(schema), + ]), + ); + } + return rows; +} + +function row(cells: readonly string[]): string { + return `| ${cells.map(cell).join(" | ")} |`; +} + +/** + * The type column, or an honest refusal to reduce one. + * + * A plain `type` — one name or a union of them — summarizes faithfully. + * Anything else is a schema whose constraints do not fit a word, so the column + * says JSON Schema and the reader goes to the block below it. + */ +function summarizeType(schema: Json): string { + if (typeof schema !== "object" || schema === null || Array.isArray(schema)) { + return "JSON Schema"; + } + const type = schema.type; + if (typeof type === "string") { + return code(type); + } + if (Array.isArray(type) && type.every((member) => typeof member === "string")) { + // Unescaped: `row()` escapes every cell once, and escaping here as well + // would put a backslash in front of the backslash. + return type.map(code).join(" | "); + } + return "JSON Schema"; +} + +function describeProp(schema: Json): string { + if (typeof schema !== "object" || schema === null || Array.isArray(schema)) { + return ""; + } + const description = schema.description; + return typeof description === "string" ? description : ""; +} + +function describeOrigin(origin: ComponentOrigin): string { + if (origin.kind === "repository") { + return code(origin.path); + } + if (origin.kind === "registered") { + return `${code(origin.origin)} (${origin.reserved ? "reserved registration" : "registered default"})`; + } + if (origin.kind === "protected") { + // Not "reserved registration": a reader deciding whether they can supply + // this name themselves gets the opposite answer from the two phrases. + return `${code(origin.origin)} (protected component)`; + } + if (origin.kind === "workflow") { + // The object id as well as the path, so this cannot be read as a file the + // reader could edit. Abbreviated the way a commit is: enough to compare, + // short enough to sit in a table cell. + return `${code(origin.path)} (workflow bundle, ${code(abbreviate(origin.sourceHash))})`; + } + if (origin.kind === "declared-markdown") { + return `${code(origin.origin)} (declared Markdown)`; + } + return `structural syntax (${code(origin.construct)})`; +} + +/** A blob id, shortened for a table cell but left whole when it is already short. */ +function abbreviate(sourceHash: string): string { + return sourceHash.length > 12 ? sourceHash.slice(0, 12) : sourceHash; +} + +function code(text: string): string { + return `\`${text}\``; +} + +/** A table cell: pipes escaped, and line breaks folded so the row stays a row. */ +function cell(text: string): string { + return text.replace(/\|/g, "\\|").replace(/\r?\n/g, " ").trim(); +} + +function fence(language: string, body: string): string { + return ["```" + language, body, "```"].join("\n"); +} + +function stringify(value: Json): string { + return JSON.stringify(value, null, 2); +} diff --git a/packages/core/src/syntax-reference.ts b/packages/core/src/syntax-reference.ts new file mode 100644 index 000000000..3729a4218 --- /dev/null +++ b/packages/core/src/syntax-reference.ts @@ -0,0 +1,363 @@ +/** + * What a document may write here, as one thing an execution carries. + * + * `xmd syntax` answers that question for an environment nobody is running. + * Canonical `` answers it for the site an element was actually written + * at, and the two have to be the same answer — symbols an agent is shown and + * symbols an operator prints describe one vocabulary or they describe none. + * + * So there is one construction and one renderer, and this module is where an + * execution keeps its own use of them. The reference is built from the + * selection inputs the execution captured before any installation, middleware or + * document code ran: the includes it resolves against, the registry it started + * with, the identity components and exact Markdown its host declared, and the + * component bundle it is closed over when it has one. Nothing is read from a + * context, a registry answer, or anything a document can reach. + * + * A trusted host may state the symbols for its own profile instead. `xmd plan` + * does: a Plan is written to be run by `xmd run`, so the vocabulary the agent + * must be shown is the run profile's rather than the authorship execution's. + * That provider is captured with the rest of the installation, before any + * installed code exists, and one execution accepts one — two are refused rather + * than ordered, because ordering them would make which profile a document is + * shown depend on installation order. + * + * The reference carries no authority at all. It answers with text. Seeing a + * component named in the symbols neither registers it, resolves it, nor + * authorizes it: what a name means is still `selectComponent()`'s decision, and + * what may run is still the execution's. + */ + +import type { Operation } from "effection"; + +import { inspectSyntax } from "./inspect.ts"; +import type { SyntaxSymbols } from "./inspect.ts"; +import { renderSelectedDocumentation, renderSyntaxMarkdown } from "./syntax-markdown.ts"; +import type { SelectedEntry } from "./syntax-markdown.ts"; +import { documentationIndexFor } from "./component-documentation.ts"; +import type { DocumentationContribution } from "./component-documentation.ts"; +import type { DocumentationIndex } from "./documentation-index.ts"; +import { UnknownComponentError } from "./documentation-index.ts"; +import type { WorkflowImportAuthority } from "./components/bundle.ts"; +import type { DeclaredMarkdownComponent } from "./components/declared-markdown.ts"; +import type { IdentityComponent } from "./invocation-identity.ts"; +import type { ComponentOrigin, ComponentRegistry } from "./types.ts"; + +/** + * The symbols in scope for the segments being expanded. + * + * Held by the execution and handed to core's own expansion by value, beside the + * import authority and the identity domains. It is not a Context: a context + * resolves by name, and a name is not a secret, so a document could build one + * and answer for the vocabulary it is shown. + */ +export interface SyntaxReference { + /** The symbols this site describes, rendered as Markdown. */ + symbols(): Operation; + /** + * The selected components' metadata and long-form documentation. + * + * Two inputs, not one, and this is the reason the reference is an object + * rather than a string. *What may I write here* and *what may I read about* + * are different questions, and a narrowing evaluation boundary answers them + * differently on purpose: the vocabulary it admits is smaller than the + * vocabulary an author is entitled to understand. + * + * So selection reads the **enclosing authoring symbols**, which is why a + * nested Plan can be told how `` works even where it may not run one, + * and each rendered entry states whether it is available in the current + * evaluation. Collapsing the two would either hide reference material an + * author needs or imply an authority they do not have. + */ + documentation(names: readonly string[]): Operation; + /** + * The reference for a subtree that may execute less than this site. + * + * The narrowing seam, and it belongs here rather than in the evaluator + * because everything it needs is already here. A canonical evaluation + * boundary that has admitted a vocabulary hands it over; what comes back + * reports that vocabulary from `symbols()` and keeps *this* reference's + * authoring symbols and documentation index for `documentation()`. + * + * Deriving it any other way would mean the evaluator recovering the raw + * contributions and rebuilding an index — which is both a hole (that list is + * execution-private for a reason) and a way for the two indexes to drift. + * Narrowing what may run is not narrowing what may be read about, and the + * reference is the thing that already knows both. + */ + available(symbols: SyntaxSymbols): SyntaxReference; +} + +/** + * A trusted host's statement of the symbols its profile describes. + * + * Captured by value with the rest of the installation, before any installed + * code, middleware or document code runs. It returns the symbols and core + * renders them, so a host cannot make its profile print differently from the + * way `xmd syntax` prints the same symbols. + */ +export type SyntaxSymbolsProvider = () => Operation; + +/** The selection inputs an execution captured, as symbol construction reads them. */ +export interface CapturedSymbolInputs { + readonly includes: readonly string[]; + /** The registrations this execution started with, captured before it ran. */ + readonly registry: ComponentRegistry; + readonly components: readonly IdentityComponent[]; + readonly declarations: readonly DeclaredMarkdownComponent[]; + /** The bundle this execution is closed over, when a trusted host installed one. */ + readonly workflow?: WorkflowImportAuthority; +} + +/** + * The reference one execution's root carries. + * + * Nothing is built until an occurrence asks. An execution whose document never + * writes `` enumerates no includes, parses no component and reads no + * frontmatter, so carrying the reference costs a run that does not use it + * nothing at all. + * + * Each ask builds afresh. Two authored occurrences are two references, which + * is what makes an occurrence's retained symbols its own rather than a copy of + * whichever one ran first. + */ +export function rootSyntaxReference( + inputs: CapturedSymbolInputs, + provider: SyntaxSymbolsProvider | undefined, + /** + * The documentation the installed packages contribute. + * + * Handed in rather than assumed, so a document's own named lookup reads the + * index its profile assembled. Defaulting to none is what made an Agent + * component answer with documentation on the command line and with the + * fallback sentence inside a document. + */ + contributions: readonly DocumentationContribution[] = [], +): SyntaxReference { + function* current(): Operation { + return provider === undefined ? yield* derived(inputs) : yield* provider(); + } + // Snapshotted once, here, so the contributions a reference reads are the + // ones the collection boundary captured rather than whatever the caller's + // objects hold by the time a document asks. + const captured = snapshotContributions(contributions); + // No admission at a root: nothing has narrowed what may execute, so the one + // set of symbols this resolves is both what a document may write and what it + // may read about. + return referencing(current, undefined, captured); +} + +/** + * One reference over authoring symbols and executable ones. + * + * `authoring` is what named lookup reads and `admitted` is what may run. At a + * root they are the same operation; a narrowed reference keeps the authoring + * symbols and replaces the admitted ones, which is the whole of the seam. + */ +function referencing( + /** The authoring symbols: what may be read about here. */ + authoring: () => Operation, + /** + * What may *execute* here, when a boundary has narrowed it. + * + * Absent at a root, where the two are the same symbols — and must be the same + * *value*. Resolving twice would call the trusted symbols provider twice + * for one occurrence, and the environment could move between the two calls: + * an entry's metadata would then come from different symbols than the + * availability reported beside it. + */ + admitted: SyntaxSymbols | undefined, + contributions: readonly DocumentationContribution[], +): SyntaxReference { + return { + *symbols(): Operation { + // A narrowed reference reports its admission and asks the enclosing + // symbols for nothing — the bare form is about what runs. + return renderSyntaxMarkdown(admitted ?? (yield* authoring())); + }, + *documentation(names: readonly string[]): Operation { + // One resolution, both decisions. + const readable = yield* authoring(); + const runnable = admitted ?? readable; + const index = documentationIndexFor(contributions); + return renderSelectedDocumentation(select(readable, runnable, names, index)); + }, + available(next: SyntaxSymbols): SyntaxReference { + // The enclosing authoring symbols and the enclosing contributions, + // unchanged. Only what may execute is replaced, so a nested author keeps + // the documentation they had and every entry reports its availability + // against the admission. + return referencing(authoring, next, contributions); + }, + }; +} + +/** + * One symbol entry's identity: its name and its complete origin. + * + * Every member of the origin participates, not just its kind — a workflow blob + * differs from another by `sourceHash`, a declared component by `digest`, two + * registrations by their package and whether either is reserved. Comparing any + * less would let a component that merely resembles the admitted one report + * itself as admitted. + */ +function identityOf(entry: { name: string; origin: ComponentOrigin }): string { + const origin = entry.origin; + const parts: readonly string[] = + origin.kind === "structural" + ? [origin.construct] + : origin.kind === "repository" + ? [origin.path] + : origin.kind === "registered" + ? [origin.origin, String(origin.reserved)] + : origin.kind === "protected" + ? [origin.origin] + : origin.kind === "workflow" + ? [origin.path, origin.sourceHash] + : [origin.origin, origin.digest]; + // Length-prefixed, so no member's content can spell a separator and make two + // different identities collide. + return [entry.name, origin.kind, ...parts].map((part) => `${part.length}:${part}`).join(""); +} + +/** + * The selected entries, in symbol order, with their documentation and + * availability. + * + * `authoring` is what selection reads; `admitted` is what the current + * evaluation may actually run. At a root they are the same object. Under a + * narrowing boundary they are not, and the difference is what each entry's + * availability reports. + */ +export function select( + authoring: SyntaxSymbols, + admitted: SyntaxSymbols, + names: readonly string[], + index: DocumentationIndex, +): SelectedEntry[] { + const requested = new Set(names); + // Keyed by identity, not by name. A name is a spelling, and the whole point of + // the two inputs is that the enclosing symbols may hold a *different* + // component under the same one: an authoring entry for the built-in `Elicit` + // beside an admitted repository `Elicit.md` is two components. Reporting the + // authoring entry as available because something called `Elicit` can run + // would tell an author they may execute the thing they were just shown. + const runnable = new Set( + admitted.categories.flatMap((category) => category.entries.map(identityOf)), + ); + const selected: SelectedEntry[] = []; + // Walked in symbol order rather than request order, so two documents asking + // for the same components in different orders render the same text — which is + // what makes one occurrence's retained result comparable with another's. + for (const category of authoring.categories) { + for (const entry of category.entries) { + // Components only. `names` is a component lookup under the current + // contract, so a structural construct is not a thing this can select — + // and skipping it here leaves the name in `requested`, which refuses + // below rather than silently rendering nothing for it. + if (!requested.has(entry.name) || entry.kind === "structural") { + continue; + } + requested.delete(entry.name); + selected.push({ + entry, + documentation: index.documentationFor(entry.name, entry.origin), + available: runnable.has(identityOf(entry)), + }); + } + } + // Whatever is left named nothing this site has. Refused whole rather than + // rendered partially: a reader handed three of the four components they asked + // about has no way to tell which request went unanswered. + if (requested.size > 0) { + throw new UnknownComponentError( + ` was asked to document ${[...requested].sort().join(", ")}, which ` + + `${requested.size === 1 ? "is not a component" : "are not components"} available here.`, + ); + } + return selected; +} + +function* derived(inputs: CapturedSymbolInputs): Operation { + return yield* inspectSyntax({ + includes: inputs.includes, + registry: inputs.registry, + components: inputs.components, + declarations: inputs.declarations, + ...(inputs.workflow === undefined ? {} : { workflow: inputs.workflow }), + }); +} + +/** + * A defensive copy of what a caller handed the installation boundary. + * + * Field by field, and the name set materialized into one this module owns. A + * contribution is a caller's object: the array can be reordered, the source + * replaced, the `Set` added to after capture, and an iterable can answer + * differently the second time it is walked. Retaining any of those would make + * what a document is told about the product depend on what its host did + * afterwards. + */ +export function snapshotContributions( + contributions: readonly DocumentationContribution[], +): readonly DocumentationContribution[] { + return Object.freeze( + [...contributions].map((one) => + Object.freeze({ + source: Object.freeze({ + owner: String(one.source.owner), + asset: String(one.source.asset), + text: String(one.source.text), + }), + supplies: new Set([...one.supplies].map((name) => String(name))), + }), + ), + ); +} + +/** + * A reference over symbols a trusted boundary already decided. + * + * The narrowing seam. A canonical evaluation boundary that has already admitted + * the exact vocabulary a subtree may write installs the corresponding symbols + * for that subtree, and the enclosing reference is restored on leaving it. It + * adds nothing: the symbols handed here are the admission's, so an entry that is + * not in the admission cannot be in the reference. + */ +export function syntaxReference( + admitted: SyntaxSymbols, + /** + * The authoring symbols this boundary is nested in. + * + * Where the two inputs come apart. `admitted` is what may *execute* here, and + * this is what may be *read about* — the vocabulary of the site the evaluation + * was written at. Omitted, the two are the same, which is the ordinary case + * for a boundary that narrows nothing. + * + * A narrowing boundary passes both, and named selection then explains a + * component this evaluation cannot run while saying so on the entry. Dropping + * the enclosing symbols instead would leave a nested author unable to look up + * the very components they are being asked to write about. + */ + authoring: SyntaxSymbols = admitted, + /** + * The enclosing execution's documentation contributions, carried across the + * seam. + * + * Narrowing what may *execute* does not narrow what an author may read about: + * the enclosing authoring documentation travels in with the enclosing + * symbols, so a nested author keeps the reference material they had. #713 + * installs the executable symbols; this is the index that goes with it. + */ + contributions: readonly DocumentationContribution[] = [], +): SyntaxReference { + const captured = snapshotContributions(contributions); + return referencing( + // deno-lint-ignore require-yield + function* () { + return authoring; + }, + admitted, + captured, + ); +} diff --git a/packages/core/src/types.ts b/packages/core/src/types.ts index 3dc9214ed..b5014901a 100644 --- a/packages/core/src/types.ts +++ b/packages/core/src/types.ts @@ -9,6 +9,7 @@ import type { Operation, Result } from "effection"; import type { Json as DurableJson } from "@executablemd/durable-streams"; import type { TestHarnessComponentDefinition } from "./test-harness.ts"; import type { ComponentInvocation, InvocationForm } from "./invocation-identity.ts"; +import type { ProtectedComponent } from "./components/protected.ts"; export type Json = DurableJson; @@ -327,6 +328,29 @@ export type ComponentOrigin = | { kind: "structural"; construct: string } | { kind: "repository"; path: string } | { kind: "registered"; origin: string; reserved: boolean } + /** + * A component canonical core claims the name of, ahead of every host and + * author tier (`components/protected.ts`). + * + * Its own kind rather than a reserved registration, because it is not one: a + * reserved registration is a host installing something under a name this + * execution happens to protect, and it can be absent, replaced or refused at + * registration. This is core's own declaration, present in every execution, + * and no registry supplies it. Reporting it as a registration told a reader + * the name could be re-registered, which is exactly what it cannot be. + */ + | { kind: "protected"; origin: string } + /** + * A component a workflow definition is closed over, at the canonical + * repository-relative path the blob holds inside the pinned commit. + * + * Distinct from `repository` because the two answer differently to the only + * question a reader has about them: a repository candidate is whatever that + * path holds now, and a bundle member is the exact blob `sourceHash` names, + * fixed when the run was defined. Reporting a bundled component as a + * repository one said a mutable path decided it. + */ + | { kind: "workflow"; path: string; sourceHash: string } /** * Exact Markdown a trusted host declared to this environment. It names the * first-party asset the bytes came from, never a path a repository could @@ -343,6 +367,17 @@ export type ComponentOrigin = */ export type ComponentSelection = | { kind: "structural"; construct: string } + /** + * A component canonical core claims the name of, ahead of every host or author + * tier. The declaration is core's own; the implementation belongs to whichever + * execution is running, so selection reports the contract and the execution + * supplies what it built (`components/protected.ts`). + */ + | { + kind: "protected"; + component: ProtectedComponent; + origin: Extract; + } | { kind: "registered"; definition: FunctionComponentDefinition; origin: ComponentOrigin } | { kind: "repository"; path: string } /** diff --git a/packages/core/tests/answer-identity.test.ts b/packages/core/tests/answer-identity.test.ts new file mode 100644 index 000000000..cac655445 --- /dev/null +++ b/packages/core/tests/answer-identity.test.ts @@ -0,0 +1,582 @@ +/** + * Tier CIV — what a stated identity is bound to, and when it stops meaning + * anything. + * + * A generated fragment's admission is a decision about which implementation may + * run. A function carries no identity a run can retain — comparing one compares + * how somebody wrote their code — so a provider states one. Everything here is + * about the ways a stated identity could be weaker than it looks: outliving the + * execution that minted it, outliving the *invocation* that made it, surviving + * the answer being replaced or edited, or being overwritten by a second + * provider. + * + * The shape under test is the split between two authorities. A provider + * installation is the right to be asked, and it is deliberately reusable: one + * installation answers several admitted names and the same name resolved more + * than once. A request is the right to answer one asking, and it is closed the + * moment that handler invocation ends. Only a request claims, which is what + * makes a statement provable — an installation handle can prove "some provider + * this host installed" and nothing about which invocation is speaking. + * + * The rows read what `identify()` answers rather than what a refusal says: a + * claim that reported correctly while still identifying a substituted object + * would satisfy an error-shape assertion and none of these. + * + * The owner is `CanonicalImports`, which already holds issuance and retention. + * One owner asks one question about one table; a second registry would be a + * second place an answer could be authorized from. + */ + +import { describe, it } from "@executablemd/test-support/bdd"; +import { expect } from "@executablemd/test-support/expect"; +import type { Operation } from "effection"; + +import { + AnswerIdentityError, + CanonicalImports, + identityRecord, +} from "../src/components/import-authority.ts"; +import type { + AnswerIdentity, + ComponentAnswerRequest, + ImportedDefinition, + ProviderInstallation, + ResolutionWindow, +} from "../src/components/import-authority.ts"; +import type { Json } from "../src/types.ts"; + +const ORIGIN = "test://provider"; +const IDENTITY: AnswerIdentity = { origin: ORIGIN, key: "Open", revision: "1" }; +const OPEN = { key: "Open", revision: "1" } as const; +const OTHER = { key: "Other", revision: "1" } as const; + +/** One owner, activated the way canonical execution activates it. */ +function owner(): CanonicalImports { + const imports = new CanonicalImports(); + imports.activate(); + return imports; +} + +/** + * Run one resolution of `name`, the way canonical execution runs one. + * + * The window is handed to the body, because that is the value identification + * takes: a caller proves which import it is asking about by holding the object + * it opened. + */ +function resolving( + imports: CanonicalImports, + name: string, + work: (resolution: ResolutionWindow) => T, +): T { + const resolution = imports.beginResolution(name); + try { + return work(resolution); + } finally { + resolution.close(); + } +} + +/** + * Run one handler invocation of this installation, asked for `name`. + * + * The request is closed when the body returns, which is what the registrar does + * around a real handler. A row that keeps the request is keeping exactly what a + * provider could keep. + */ +function asking( + installation: ProviderInstallation, + name: string, + work: (request: ComponentAnswerRequest) => T, +): T { + const asked = installation.open(name); + try { + return work(asked.request); + } finally { + asked.close(); + } +} + +/** One answer a provider might return, fresh each time. */ +function answer(name = "Open"): ImportedDefinition { + return { + kind: "function", + name, + props: { type: "object", properties: {}, additionalProperties: false }, + // deno-lint-ignore require-yield + *fn(): Operation { + return ""; + }, + }; +} + +/** The implementation on a definition, whichever arm of the union it is. */ +function implementationOf(definition: ImportedDefinition | undefined): unknown { + return definition === undefined ? undefined : (definition as { fn?: unknown }).fn; +} + +/** What a call refused with, or `undefined` when it did not refuse. */ +function refusalOf(attempt: () => unknown): unknown { + try { + attempt(); + return undefined; + } catch (error) { + return error; + } +} + +describe("Tier CIV — an identity belongs to one answer of one import", () => { + it("CIV23: an identity is stated on the exact answer and read back", function* () { + const imports = owner(); + const provider = imports.provider(ORIGIN); + + const { resolution, supplied } = resolving(imports, "Open", (resolution) => ({ + resolution, + supplied: asking(provider, "Open", (request) => request.claim(answer(), OPEN)), + })); + + expect(imports.identify(resolution, supplied)?.identity).toEqual(IDENTITY); + expect(identityRecord(IDENTITY)).toBe("test://provider#Open@1"); + }); + + it("CIV24: identification answers with the claim-time copy, not the answer", function* () { + const imports = owner(); + const provider = imports.provider(ORIGIN); + const supplied = answer(); + + const resolution = resolving(imports, "Open", (resolution) => { + asking(provider, "Open", (request) => request.claim(supplied, OPEN)); + return resolution; + }); + + // One call answers both halves. The definition is core's own copy, taken + // when the claim was recorded — so a caller that keeps what identification + // gave it has kept the object the check was made about, and never has to + // read the chain's object a second time to obtain one. + const identified = imports.identify(resolution, supplied); + expect(identified?.identity).toEqual(IDENTITY); + expect(identified?.definition).not.toBe(supplied); + expect(identified?.definition.name).toBe("Open"); + // The implementation crosses by reference, because a function is not + // copyable and is the one thing a fragment must invoke as itself. + expect(implementationOf(identified?.definition)).toBe(implementationOf(supplied)); + }); + + it("CIV24: an alternating answer cannot launder a copy through a second read", function* () { + const imports = owner(); + const provider = imports.provider(ORIGIN); + const honest = answer(); + const reads: string[] = []; + // The check/use gap, planted. Reading a member through `[[Get]]` alternates + // between what was claimed and a substitution; comparing descriptors, which + // is how the claim is checked, does not run this at all. So a caller that + // checked the descriptors and then *read* the object again to keep a copy + // would keep the substitution, and this row is what says nothing does. + const alternating = new Proxy(honest, { + get(target, key, receiver) { + if (key !== "name") { + return Reflect.get(target, key, receiver); + } + reads.push(key); + return reads.length % 2 === 1 ? "Open" : "Substituted"; + }, + }) as ImportedDefinition; + + const resolution = resolving(imports, "Open", (resolution) => { + asking(provider, "Open", (request) => request.claim(alternating, OPEN)); + return resolution; + }); + const identified = imports.identify(resolution, alternating); + + // The claim itself was recorded from the first reading, and identification + // answers with that recording. + expect(identified?.identity).toEqual(IDENTITY); + expect(identified?.definition.name).toBe("Open"); + // The plant is live rather than inert: the very next read of the object the + // chain returned says something else, which is what a second read would + // have kept. + expect((alternating as { name: string }).name).toBe("Substituted"); + expect(identified?.definition.name).toBe("Open"); + }); + + it("CIV23: restating exactly the same claim is idempotent", function* () { + const imports = owner(); + const provider = imports.provider(ORIGIN); + const supplied = answer(); + + const resolution = resolving(imports, "Open", (resolution) => { + asking(provider, "Open", (request) => { + request.claim(supplied, OPEN); + // A provider installed twice states the same thing twice. That is not + // two providers disagreeing, and it is not a conflict. + expect(refusalOf(() => request.claim(supplied, OPEN))).toBe(undefined); + }); + return resolution; + }); + expect(imports.identify(resolution, supplied)?.identity).toEqual(IDENTITY); + }); + + it("CIV23: a competing claim refuses and never overwrites the first", function* () { + const imports = owner(); + const first = imports.provider(ORIGIN); + const second = imports.provider("test://other"); + const held = answer(); + + const resolution = resolving(imports, "Open", (resolution) => { + asking(first, "Open", (request) => request.claim(held, OPEN)); + // Another installation, asked in the same live resolution, cannot rename + // what the first stated. An overwrite would let a second provider take + // the first's implementation. + asking(second, "Open", (request) => { + expect(refusalOf(() => request.claim(held, OPEN))).toBeInstanceOf(AnswerIdentityError); + }); + // And the first installation's *next* invocation cannot restate it + // differently either. + asking(first, "Open", (request) => { + expect( + refusalOf(() => request.claim(held, { key: "Other", revision: "1" })), + ).toBeInstanceOf(AnswerIdentityError); + expect(refusalOf(() => request.claim(held, { key: "Open", revision: "2" }))).toBeInstanceOf( + AnswerIdentityError, + ); + }); + return resolution; + }); + + expect(imports.identify(resolution, held)?.identity).toEqual(IDENTITY); + }); + + it("CIV25: a request whose handler has returned states nothing", function* () { + const imports = owner(); + const provider = imports.provider(ORIGIN); + const late = answer(); + let stale: ComponentAnswerRequest | undefined; + + // The losing-handler case, inside one live resolution. The handler keeps + // the request it was given and returns; the resolution is still open, and + // the execution is still very much alive. What has ended is this + // invocation, and an invocation that has returned is not supplying an + // answer. + const resolution = resolving(imports, "Open", (resolution) => { + asking(provider, "Open", (request) => { + stale = request; + }); + expect(refusalOf(() => stale?.claim(late, OPEN))).toBeInstanceOf(AnswerIdentityError); + // The positive control in the same still-open window: a fresh invocation + // of the same installation claims, so the refusal above is about the + // handler lease rather than about the window or the provider. + asking(provider, "Open", (request) => request.claim(answer(), OPEN)); + return resolution; + }); + + expect(imports.identify(resolution, late)).toBe(undefined); + expect(imports.identifying).toBe(true); + }); + + it("CIV25: a stale request cannot answer the next resolution of its own name", function* () { + const imports = owner(); + const provider = imports.provider(ORIGIN); + const fresh = answer(); + const requests: ComponentAnswerRequest[] = []; + let stale: ComponentAnswerRequest | undefined; + + // Resolution N of `Open`, whose handler keeps its request. + resolving(imports, "Open", () => { + asking(provider, "Open", (request) => { + requests.push(request); + stale = request; + }); + }); + + // Resolution N+1 of the same name. The stale request names the right + // component and belongs to an import that is over. + const { resolution, claimed } = resolving(imports, "Open", (resolution) => { + expect(refusalOf(() => stale?.claim(fresh, OPEN))).toBeInstanceOf(AnswerIdentityError); + // The same installation is asked again and answers this resolution: the + // installation is reusable, the request is not. + return { + resolution, + claimed: asking(provider, "Open", (request) => { + requests.push(request); + return request.claim(answer(), OPEN); + }), + }; + }); + + expect(requests).toHaveLength(2); + expect(requests[0]).not.toBe(requests[1]); + expect(imports.identify(resolution, fresh)).toBe(undefined); + expect(imports.identify(resolution, claimed)?.identity).toEqual(IDENTITY); + }); + + it("CIV25: a stale request cannot retag while another name is being decided", function* () { + const imports = owner(); + const provider = imports.provider(ORIGIN); + const substitute = answer("Other"); + let stale: ComponentAnswerRequest | undefined; + + resolving(imports, "Open", () => { + asking(provider, "Open", (request) => { + stale = request; + }); + }); + + // `claim` takes no name, so the stale request cannot even ask about the + // name being decided: it is fixed to `Open`, which this resolution is not. + const { resolution, claimed } = resolving(imports, "Other", (resolution) => { + expect(stale?.name).toBe("Open"); + expect(refusalOf(() => stale?.claim(substitute, OTHER))).toBeInstanceOf(AnswerIdentityError); + return { + resolution, + claimed: asking(provider, "Other", (request) => request.claim(answer("Other"), OTHER)), + }; + }); + + expect(imports.identify(resolution, substitute)).toBe(undefined); + expect(imports.identify(resolution, claimed)?.identity.key).toBe("Other"); + }); + + it("CIV25: one installation answers two names through two distinct requests", function* () { + const imports = owner(); + // One installation owns one origin and may answer more than one admitted + // name. Spending the installation on its first answer would break exactly + // this, so what settles is the request rather than the provider. + const provider = imports.provider(ORIGIN); + const seen: ComponentAnswerRequest[] = []; + + const open = resolving(imports, "Open", (resolution) => ({ + resolution, + claimed: asking(provider, "Open", (request) => { + seen.push(request); + return request.claim(answer(), OPEN); + }), + })); + const other = resolving(imports, "Other", (resolution) => ({ + resolution, + claimed: asking(provider, "Other", (request) => { + seen.push(request); + return request.claim(answer("Other"), OTHER); + }), + })); + + // Two invocations, two requests, each fixed to what it was asked. + expect(seen).toHaveLength(2); + expect(seen[0]).not.toBe(seen[1]); + expect(seen.map((request) => request.name)).toEqual(["Open", "Other"]); + expect(imports.identify(open.resolution, open.claimed)?.identity.key).toBe("Open"); + expect(imports.identify(other.resolution, other.claimed)?.identity.key).toBe("Other"); + }); + + it("CIV25: identification takes the window it was asked about", function* () { + const imports = owner(); + const provider = imports.provider(ORIGIN); + const requests: ComponentAnswerRequest[] = []; + + const first = resolving(imports, "Open", (resolution) => ({ + resolution, + claimed: asking(provider, "Open", (request) => { + requests.push(request); + return request.claim(answer(), OPEN); + }), + })); + const second = resolving(imports, "Open", (resolution) => ({ + resolution, + claimed: asking(provider, "Open", (request) => { + requests.push(request); + return request.claim(answer(), OPEN); + }), + })); + + // Each answer belongs to the import it answered, and to no other. Nothing + // here reads whichever window is current: the caller presents the one it + // opened, which is why the answer does not change with when it is asked. + expect(requests).toHaveLength(2); + expect(requests[0]).not.toBe(requests[1]); + expect(imports.identify(first.resolution, first.claimed)?.identity).toEqual(IDENTITY); + expect(imports.identify(second.resolution, second.claimed)?.identity).toEqual(IDENTITY); + expect(imports.identify(first.resolution, second.claimed)).toBe(undefined); + expect(imports.identify(second.resolution, first.claimed)).toBe(undefined); + }); + + it("CIV25: a request shows the asked position by value", function* () { + const imports = owner(); + const provider = imports.provider(ORIGIN); + // The engine's own object, which it reads again after any handler has seen + // it. A provider is shown a copy, like everything else it is shown. + const scanned = { path: "doc.md", offset: 12, line: 3, column: 5 }; + + resolving(imports, "Open", () => { + const asked = provider.open("Open", scanned); + try { + const shown = asked.request.position; + expect(shown).toEqual(scanned); + expect(shown).not.toBe(scanned); + // Editing what the handler was given reaches nothing, and the request + // itself cannot be re-pointed at another element. + expect(() => { + (shown as { line: number }).line = 99; + }).toThrow(); + expect(() => { + (asked.request as { name: string }).name = "Other"; + }).toThrow(); + } finally { + asked.close(); + } + // And the engine's own object is untouched by having been shown. + expect(scanned).toEqual({ path: "doc.md", offset: 12, line: 3, column: 5 }); + }); + }); + + it("CIV23: a different object carries no claim, however alike", function* () { + const imports = owner(); + const provider = imports.provider(ORIGIN); + + const { resolution, claimed } = resolving(imports, "Open", (resolution) => ({ + resolution, + claimed: asking(provider, "Open", (request) => request.claim(answer(), OPEN)), + })); + + expect(imports.identify(resolution, claimed)?.identity).toEqual(IDENTITY); + // The outer-replacement case: a handler further out returns its own object, + // so an intermediate claim does not travel with the name. + expect(imports.identify(resolution, answer())).toBe(undefined); + // A copy of the claimed object is a different object too. + expect(imports.identify(resolution, { ...claimed })).toBe(undefined); + }); + + it("CIV23: editing the claimed answer invalidates the claim", function* () { + const imports = owner(); + const provider = imports.provider(ORIGIN); + + const { resolution, claimed } = resolving(imports, "Open", (resolution) => ({ + resolution, + claimed: asking(provider, "Open", (request) => request.claim(answer(), OPEN)), + })); + expect(imports.identify(resolution, claimed)?.identity).toEqual(IDENTITY); + + // The same object, edited after the claim by a handler further out. What + // was claimed is no longer what is there. + (claimed as { name: string }).name = "Substituted"; + + expect(imports.identify(resolution, claimed)).toBe(undefined); + }); + + it("CIV25: an installation retained past its execution opens nothing", function* () { + const imports = owner(); + const provider = imports.provider(ORIGIN); + const { resolution, before } = resolving(imports, "Open", (resolution) => ({ + resolution, + before: asking(provider, "Open", (request) => request.claim(answer(), OPEN)), + })); + expect(imports.identify(resolution, before)?.identity).toEqual(IDENTITY); + + imports.revoke(); + + // The installation is the object a provider kept. The window it would need + // cannot be opened, and a request minted from it states nothing. + expect(refusalOf(() => imports.beginResolution("Open"))).toBeInstanceOf(AnswerIdentityError); + expect( + refusalOf(() => asking(provider, "Open", (request) => request.claim(answer(), OPEN))), + ).toBeInstanceOf(AnswerIdentityError); + // And what it stated while the execution was live identifies nothing now: + // an admission may not be reconciled against a run that is over. + expect(imports.identify(resolution, before)).toBe(undefined); + expect(imports.identifying).toBe(false); + }); + + it("CIV25: an owner starts inactive, so a claim before activation refuses", function* () { + const imports = new CanonicalImports(); + // Canonical execution registers teardown, then activates, then installs + // providers. A claim that landed before activation would be a claim with no + // teardown behind it. + expect(imports.identifying).toBe(false); + expect(refusalOf(() => imports.beginResolution("Open"))).toBeInstanceOf(AnswerIdentityError); + expect( + refusalOf(() => + asking(imports.provider(ORIGIN), "Open", (request) => request.claim(answer(), OPEN)), + ), + ).toBeInstanceOf(AnswerIdentityError); + }); + + it("CIV25: overlapping executions are isolated", function* () { + const first = owner(); + const second = owner(); + const shared = answer(); + + const one = resolving(first, "Open", (resolution) => { + asking(first.provider(ORIGIN), "Open", (request) => request.claim(shared, OPEN)); + return resolution; + }); + // Live at the same time, and each answers only for what it recorded. + const two = resolving(second, "Open", (resolution) => { + expect(second.identify(resolution, shared)).toBe(undefined); + asking(second.provider(ORIGIN), "Open", (request) => + request.claim(shared, { key: "Open", revision: "2" }), + ); + return resolution; + }); + + expect(first.identify(one, shared)?.identity).toEqual(IDENTITY); + expect(second.identify(two, shared)?.identity).toEqual({ + origin: ORIGIN, + key: "Open", + revision: "2", + }); + + // The positive control: tearing one down leaves the other working. + first.revoke(); + expect(first.identify(one, shared)).toBe(undefined); + expect(second.identify(two, shared)?.identity.revision).toBe("2"); + }); + + it("CIV23: a provider cannot state an origin canonical execution did not give it", function* () { + const imports = owner(); + // The installation carries the origin; the provider states only key and + // revision. There is no member on the claim call to put another origin in. + const { resolution, claimed } = resolving(imports, "Open", (resolution) => ({ + resolution, + claimed: asking(imports.provider("test://assigned"), "Open", (request) => + request.claim(answer(), OPEN), + ), + })); + + expect(imports.identify(resolution, claimed)?.identity.origin).toBe("test://assigned"); + }); + + it("CIV23: a partial identity is refused rather than recorded", function* () { + const imports = owner(); + const provider = imports.provider(ORIGIN); + const attempts = [ + { key: "", revision: "1" }, + { key: "Open", revision: "" }, + ]; + + for (const attempt of attempts) { + const supplied = answer(); + const resolution = resolving(imports, "Open", (resolution) => { + asking(provider, "Open", (request) => { + expect(refusalOf(() => request.claim(supplied, attempt))).toBeInstanceOf( + AnswerIdentityError, + ); + }); + return resolution; + }); + // Refused rather than partially recorded: a half identity would compare + // equal to a different half identity. + expect(imports.identify(resolution, supplied)).toBe(undefined); + } + }); + + it("CIV23: an answer nobody claimed identifies nothing", function* () { + const imports = owner(); + // The ordinary case, and the reason this is not authority: an unidentified + // answer is a perfectly good answer. What it cannot be is the thing a + // fragment runs, because a continuation would have nothing to compare. + resolving(imports, "Open", (resolution) => { + expect(imports.identify(resolution, answer())).toBe(undefined); + expect(imports.identify(resolution, undefined)).toBe(undefined); + expect(imports.identify(resolution, null)).toBe(undefined); + expect(imports.identify(resolution, "Open")).toBe(undefined); + }); + }); +}); diff --git a/packages/core/tests/documentation-index.test.ts b/packages/core/tests/documentation-index.test.ts new file mode 100644 index 000000000..6bbc830e9 --- /dev/null +++ b/packages/core/tests/documentation-index.test.ts @@ -0,0 +1,196 @@ +/** + * Tier SYN — the documentation index (#678). + * + * The index is what ``, `xmd syntax Elicit` and the release + * reference all read, so a set that parses wrongly is wrong in three places at + * once. Everything here is about it refusing rather than producing a partial + * answer: a documentation set that has drifted from the components it documents + * is a build problem, and the moment to say so is the build. + */ + +import { describe, it } from "@executablemd/test-support/bdd"; +import { expect } from "@executablemd/test-support/expect"; + +import { + buildDocumentationIndex, + DocumentationIndexError, + owningPackage, + parseDocumentationSource, +} from "../src/documentation-index.ts"; +import type { DocumentationSource } from "../src/documentation-index.ts"; +import type { ComponentOrigin } from "../src/types.ts"; + +const OWNER = "@executablemd/test"; + +/** One documentation file, as a package ships it. */ +function source(text: string): DocumentationSource { + return { owner: OWNER, asset: "packages/test/src/components/components.md", text }; +} + +/** What that package supplies, as the index validates headings against. */ +function supplies(...names: readonly string[]): (owner: string) => ReadonlySet { + return (asked) => (asked === OWNER ? new Set(names) : new Set()); +} + +const REGISTERED: ComponentOrigin = { kind: "registered", origin: OWNER, reserved: false }; +const PROTECTED: ComponentOrigin = { kind: "protected", origin: OWNER }; +const REPOSITORY: ComponentOrigin = { kind: "repository", path: "components/Alpha.md" }; + +describe("Tier SYN — parsing one documentation file", () => { + it("SYN32: reads the bundle's own prose, then a section per component", function* () { + const parsed = parseDocumentationSource( + source( + [ + "What this bundle is for.", + "", + "## Alpha", + "", + "About Alpha.", + "", + "### A detail of Alpha", + "", + "Still Alpha.", + "", + "## Beta", + "", + "About Beta.", + "", + ].join("\n"), + ), + ); + + expect(parsed.bundle).toBe("What this bundle is for."); + // A level-three heading stays in the section it is written in, so a + // component's own documentation can have structure. + expect(parsed.sections.get("Alpha")).toContain("### A detail of Alpha"); + expect(parsed.sections.get("Alpha")).toContain("Still Alpha."); + expect(parsed.sections.get("Beta")).toBe("About Beta."); + expect([...parsed.sections.keys()]).toEqual(["Alpha", "Beta"]); + }); + + it("SYN33: reads a heading inside a fence as the example it is", function* () { + const parsed = parseDocumentationSource( + source( + ["## Alpha", "", "Write a heading like this:", "", "```md", "## Beta", "```", ""].join( + "\n", + ), + ), + ); + + // One section, not two: the fenced `## Beta` is Markdown being shown, and + // reading it as a section would move everything after it into the wrong + // component. + expect([...parsed.sections.keys()]).toEqual(["Alpha"]); + expect(parsed.sections.get("Alpha")).toContain("## Beta"); + }); + + it("SYN33b: keeps a shorter fence inside a longer one as example text", function* () { + const parsed = parseDocumentationSource( + source( + [ + "## Alpha", + "", + "How to write a fenced example:", + "", + "````md", + "```mdx", + "", + "```", + "", + "## Beta", + "````", + "", + "Still Alpha.", + "", + ].join("\n"), + ), + ); + + // The inner three-backtick run neither opens nor closes anything: the outer + // four-backtick fence is still open, so `## Beta` inside it is the example + // it is being shown as. Closing on the character alone would have ended the + // example early and read the rest as a second component. + expect([...parsed.sections.keys()]).toEqual(["Alpha"]); + expect(parsed.sections.get("Alpha")).toContain("## Beta"); + expect(parsed.sections.get("Alpha")).toContain("Still Alpha."); + }); + + it("SYN33c: accepts a dotted component name as a heading", function* () { + const parsed = parseDocumentationSource(source("## File.Delete\n\nAbout File.Delete.\n")); + // The canonical grammar, so every namespaced component in the product can + // be documented and looked up. + expect(parsed.sections.get("File.Delete")).toBe("About File.Delete."); + }); + + it("SYN34: refuses a duplicate section and a heading that is not a name", function* () { + expect(() => + parseDocumentationSource(source(["## Alpha", "one", "", "## Alpha", "two", ""].join("\n"))), + ).toThrow(DocumentationIndexError); + + expect(() => + parseDocumentationSource(source(["## Getting started", "prose", ""].join("\n"))), + ).toThrow(DocumentationIndexError); + }); +}); + +describe("Tier SYN — building the index", () => { + it("SYN35: refuses a heading naming something the package does not supply", function* () { + expect(() => + buildDocumentationIndex([source("## Gamma\n\nAbout Gamma.\n")], supplies("Alpha")), + ).toThrow(DocumentationIndexError); + }); + + it("SYN36: refuses one component documented in two files", function* () { + const first = source("## Alpha\n\nOne.\n"); + const second = { ...first, asset: "packages/test/src/other/components.md" }; + expect(() => buildDocumentationIndex([first, second], supplies("Alpha"))).toThrow( + DocumentationIndexError, + ); + }); + + it("SYN37: attaches documentation by name and owning package together", function* () { + const index = buildDocumentationIndex( + [source("Bundle prose.\n\n## Alpha\n\nAbout Alpha.\n")], + supplies("Alpha"), + ); + + // The package's own components, however core puts them into an execution. + expect(index.documentationFor("Alpha", REGISTERED)).toBe("About Alpha."); + expect(index.documentationFor("Alpha", PROTECTED)).toBe("About Alpha."); + expect(index.bundleDocumentation(REGISTERED)).toBe("Bundle prose."); + + // A repository component that happens to share the name is a different + // component, and gets none of it: its origin names no package at all. + expect(index.documentationFor("Alpha", REPOSITORY)).toBeUndefined(); + expect(owningPackage(REPOSITORY)).toBeUndefined(); + + // Neither does a component this package does not document. + expect(index.documentationFor("Beta", REGISTERED)).toBeUndefined(); + }); + + it("SYN38: refuses a package that supplies a component it does not document", function* () { + // The negative control for coverage. A first-party package documents every + // public component it supplies, so a missing section refuses the whole + // index rather than letting that component fall back to the sentence — + // which would leave the product's own reference silently incomplete, with + // no way for a reader to tell an undocumented component from one that has + // nothing to say. + expect(() => + buildDocumentationIndex([source("## Alpha\n\nAbout Alpha.\n")], supplies("Alpha", "Beta")), + ).toThrow(DocumentationIndexError); + + // Deleting any one built-in's documentation is the same failure, which is + // what makes this a live check on the shipped files rather than a rule + // nothing enforces. + expect(() => + buildDocumentationIndex([source("Bundle prose only.\n")], supplies("Alpha")), + ).toThrow(DocumentationIndexError); + + // The positive control: exact coverage builds. + const complete = buildDocumentationIndex( + [source("## Alpha\n\nAbout Alpha.\n\n## Beta\n\nAbout Beta.\n")], + supplies("Alpha", "Beta"), + ); + expect(complete.documentationFor("Beta", REGISTERED)).toBe("About Beta."); + }); +}); diff --git a/packages/core/tests/evaluate-component.test.ts b/packages/core/tests/evaluate-component.test.ts new file mode 100644 index 000000000..ef89d7526 --- /dev/null +++ b/packages/core/tests/evaluate-component.test.ts @@ -0,0 +1,1389 @@ +/** + * Tier FE — canonical `` against a host profile. + * + * `` is public: any author may write it. What keeps that from being a + * capability is that everything it can reach was stated by a trusted host at the + * installation boundary, before a document existed — and that `allow` selects + * among those tables rather than adding to them. + * + * Every row drives the real component through a real execution against a real + * captured profile. The filesystem operations are a recorder rather than a disk, + * and it is never installed as a provider: a read that appears in its log went + * through the captured operation, because there is no other way to reach it. + */ + +import { describe, it } from "@executablemd/test-support/bdd"; +import { expect } from "@executablemd/test-support/expect"; +import { InMemoryStream } from "@executablemd/durable-streams"; +import type { DurableEvent, Json } from "@executablemd/durable-streams"; +import { ensure, Ok, scoped, spawn, suspend, withResolvers } from "effection"; +import type { Operation, Result } from "effection"; +import { API } from "@executablemd/runtime"; + +import { collect } from "../src/collect.ts"; +import { Component, content } from "../src/component-api.ts"; +import { executeInstalled } from "../host.ts"; +import { directoryEntry, fileDeleteEntry, fileReadEntry, fileWriteEntry } from "../host.ts"; +import type { ExecutionInstallation, FragmentEvaluationInput } from "../host.ts"; +import { registerComponents } from "../src/components/registration.ts"; +import { retainedSource } from "../src/root-source.ts"; +import { recordedFiles } from "./support/fragment-files.ts"; +import type { RecordedFiles } from "./support/fragment-files.ts"; +import { answerProvider, implementation } from "./support/answer-provider.ts"; +import type { Implementation, ProviderOptions } from "./support/answer-provider.ts"; + +const ROOT_PATH = "evaluate.md"; + +const NOTE = "the retained note\n"; + +/** A read-only profile over one recorder. */ +function reading(files: RecordedFiles): ExecutionInstallation { + return { evaluation: { read: [fileReadEntry()], files } }; +} + +/** A profile offering both classes over one recorder. */ +function both(files: RecordedFiles): ExecutionInstallation { + return { + evaluation: { + read: [fileReadEntry()], + write: [ + fileWriteEntry(), + directoryEntry({ origin: "test://host", key: "Dir", revision: "1" }, "Dir"), + fileDeleteEntry(), + ], + files, + }, + }; +} + +function run( + source: string, + installations: readonly ExecutionInstallation[], + stream: InMemoryStream = new InMemoryStream(), +): Operation { + return scoped(function* () { + return yield* collect( + yield* executeInstalled({ ...retainedSource(ROOT_PATH, source), stream }, [...installations]), + ); + }); +} + +/** What one execution refused with, as a string. */ +function* refusal(operation: Operation): Operation { + try { + yield* operation; + } catch (error) { + return error instanceof Error ? error.message : String(error); + } + throw new Error("expected the operation to be refused"); +} + +/** Every generated-XMD admission a run recorded. */ +function admissions(events: readonly DurableEvent[]): DurableEvent[] { + return events.filter( + (event) => event.type === "yield" && event.description.type === "generated_xmd", + ); +} + +describe("Tier FE — a program the document holds", () => { + it("FE1: `text` runs, and its observations bind by name and order", function* () { + const files = recordedFiles({ "notes.md": NOTE, "other.md": "the other note\n" }); + const output = yield* run( + `\\n\\n\\n'} as="answer" />` + + `\n\n\n`, + [reading(files)], + ); + + const rendered = String(output); + expect(rendered).toContain("the retained note"); + expect(rendered).toContain("the other note"); + // Through the captured operations, in the order the fragment wrote them. + expect(files.performed).toEqual(["read notes.md", "read other.md"]); + }); + + it("FE1: an element with no program and no content is refused", function* () { + expect(yield* refusal(run(`\n`, [reading(recordedFiles())]))).toContain( + "requires the program as a string", + ); + }); + + it("FE1: `text` beside content is refused rather than resolved by precedence", function* () { + expect( + yield* refusal( + run(`\ncontent\n\n`, [ + reading(recordedFiles()), + ]), + ), + ).toContain("does not also carry `text`"); + }); + + it("FE1: the released `source` spelling is refused where the host did not admit it", function* () { + const files = recordedFiles({ "notes.md": NOTE }); + expect( + yield* refusal(run(`'} />\n`, [reading(files)])), + ).toContain("this host did not admit it"); + // Refused before the program was read, so nothing was performed. + expect(files.performed).toEqual([]); + }); + + it("FE1: a host that admitted the alias accepts it, silently", function* () { + const files = recordedFiles({ "notes.md": NOTE }); + const output = yield* run( + `\\n'} as="answer" />\n\n\n`, + [{ evaluation: { read: [fileReadEntry()], files, deprecatedSourceAlias: true } }], + ); + + expect(String(output)).toContain("the retained note"); + expect(String(output)).not.toContain("earlier spelling"); + }); +}); + +describe("Tier FE — a fragment reaches the captured operations and nothing else", () => { + it("FE13: a Files provider installed around the document is not what a fragment reads", function* () { + const files = recordedFiles({ "notes.md": NOTE }); + const reachedDocumentProvider: string[] = []; + + const output = yield* scoped(function* () { + // A provider installed exactly where a document, a repository component or + // a middleware package would install one — nearer than anything the host + // set up, and answering every read. + yield* API.Files.around( + { + // deno-lint-ignore require-yield + *readTextFile([input]): Operation { + reachedDocumentProvider.push(String(input.path)); + throw new Error("the document's provider answered"); + }, + }, + { at: "min" }, + ); + return yield* run( + `\\n'} as="answer" />\n\n\n`, + [reading(files)], + ); + }); + + // The fragment read through the captured operation. The provider the + // document installed was never consulted — it would have thrown. + expect(String(output)).toContain("the retained note"); + expect(files.performed).toEqual(["read notes.md"]); + expect(reachedDocumentProvider).toEqual([]); + }); + + it("FE13: a fragment cannot glob, however the document's provider is arranged", function* () { + const files = recordedFiles({ "notes.md": NOTE }); + const failed = yield* refusal( + run(`\\n'} />\n`, [reading(files)]), + ); + + // Not "globbing failed" — `` is not a name this fragment has at all. + expect(failed).toContain("did not admit"); + expect(files.performed).toEqual([]); + }); + + it("FE13: an operation retained past its execution refuses rather than acting", function* () { + const files = recordedFiles({ "notes.md": NOTE }); + yield* run(`\\n'} />\n`, [reading(files)]); + const during = [...files.performed]; + + // The execution has ended. Whatever a fragment body, a handler or a + // retained callback still holds is bound to operations this execution + // revoked at teardown. + expect(during).toEqual(["read notes.md"]); + expect(files.performed).toEqual(during); + }); +}); + +describe("Tier FE — the paired form produces its own program", () => { + it("FE2: the exact bytes the producer rendered are the admitted source", function* () { + // Not "contains" and not "after trimming": the retained admission is a + // decision about one exact fragment, and a continuation is held to those + // bytes. A projection that trimmed, re-indented or normalised newlines + // would make the retained text disagree with what the producer wrote. + const PROGRAM = `\n\n\n\n`; + const files = recordedFiles({ "notes.md": NOTE, "other.md": "the other note\n" }); + const stream = new InMemoryStream(); + yield* scoped(function* () { + yield* registerComponents([ + { + name: "Producer", + origin: "test://producer", + props: { type: "object", properties: {}, additionalProperties: false }, + // deno-lint-ignore require-yield + *fn(): Operation { + return PROGRAM; + }, + }, + ]); + return yield* run(`\n\n\n`, [reading(files)], stream); + }); + + const recorded = admissions(yield* stream.readAll()); + expect(recorded).toHaveLength(1); + const event = recorded[0]; + const value = + event?.type === "yield" && event.result.status === "ok" ? event.result.value : undefined; + const source = + typeof value === "object" && value !== null && !Array.isArray(value) + ? value.source + : undefined; + // The content region is `\n\n` — the newlines the document put + // around the element are part of what `` was given to render, so + // the admitted source is those bytes with the producer's own in place of + // the element. Byte-exact in both directions: nothing was trimmed, and + // nothing the document did not write was added. + expect(source).toBe(`\n${PROGRAM}\n`); + expect(files.performed).toEqual(["read notes.md", "read other.md"]); + }); + + it("FE2: content renders once, and what it rendered is the program", function* () { + const files = recordedFiles({ "notes.md": NOTE }); + const rendered: string[] = []; + const output = yield* scoped(function* () { + yield* registerComponents([ + { + name: "Producer", + origin: "test://producer", + props: { type: "object", properties: {}, additionalProperties: false }, + // deno-lint-ignore require-yield + *fn(): Operation { + rendered.push("produced"); + return `\n`; + }, + }, + ]); + return yield* run( + `\n\n\n\n\n`, + [reading(files)], + ); + }); + + expect(rendered).toEqual(["produced"]); + expect(String(output)).toContain("the retained note"); + expect(files.performed).toEqual(["read notes.md"]); + }); + + it("FE2: a producer keeps its own authority while the fragment does not", function* () { + // The producer's content runs at the document's site, so its `` is + // the ordinary component and reaches the document's own provider. The + // program it renders reaches only the captured operations. Both are counted + // here, separately: narrowing what a fragment may *name* must not narrow + // what the producer may *do* to write it. + const files = recordedFiles({ "notes.md": NOTE }); + const documentReads: string[] = []; + + const output = yield* scoped(function* () { + yield* API.Files.around( + { + // deno-lint-ignore require-yield + *readTextFile([input]): Operation> { + documentReads.push(String(input.path)); + return Ok(`\n`); + }, + }, + { at: "min" }, + ); + return yield* run( + `\n\n\n\n` + + `\n`, + [reading(files)], + ); + }); + + expect(documentReads).toEqual(["producer-input.md"]); + expect(files.performed).toEqual(["read notes.md"]); + expect(String(output)).toContain("the retained note"); + }); + + it("FE2: a producer that fails stops the evaluation before any fragment work", function* () { + const stream = new InMemoryStream(); + const files = recordedFiles(); + const failed = yield* refusal( + scoped(function* () { + yield* registerComponents([ + { + name: "Broken", + origin: "test://producer", + props: { type: "object", properties: {}, additionalProperties: false }, + // deno-lint-ignore require-yield + *fn(): Operation { + throw new Error("the producer refused"); + }, + }, + ]); + return yield* run(`\n\n\n`, [reading(files)], stream); + }), + ); + + expect(failed).toContain("the producer refused"); + expect(admissions(yield* stream.readAll())).toHaveLength(0); + expect(files.performed).toEqual([]); + }); + + it("FE17: cancelling inside the producer waits for its cleanup and admits nothing", function* () { + const stream = new InMemoryStream(); + const files = recordedFiles({ "notes.md": NOTE }); + const cleanup: string[] = []; + const reached = withResolvers(); + + yield* scoped(function* () { + const running = yield* spawn(() => + scoped(function* () { + yield* registerComponents([ + { + name: "Slow", + origin: "test://producer", + props: { type: "object", properties: {}, additionalProperties: false }, + *fn(): Operation { + // Registered before the barrier, so cancellation cannot arrive + // between entering the body and owning the cleanup. + yield* ensure(function* () { + cleanup.push("producer cleanup"); + }); + reached.resolve(); + yield* suspend(); + return `\n`; + }, + }, + ]); + return yield* run(`\n\n\n`, [reading(files)], stream); + }), + ); + // The producer has actually entered, so this is cancellation of work in + // flight rather than of work that never started. + yield* reached.operation; + yield* running.halt(); + }); + + // Halt waited for the producer's own cleanup before returning. + expect(cleanup).toEqual(["producer cleanup"]); + // And nothing was decided: no admission, and no fragment operation. A run + // that recorded an admission here would resume believing a decision was + // made about a program that never finished being written. + expect(admissions(yield* stream.readAll())).toHaveLength(0); + expect(files.performed).toEqual([]); + }); + + it("FE17: cancelling after fragment work starts records no terminal result", function* () { + // The other half of FE17. That row cancels while the *producer* is running, + // so it proves the projector side and nothing about the evaluation. This one + // cancels after the admission committed and an admitted operation is + // actually in flight, which is the only state where a fragment effect is + // interrupted rather than never started. + const stream = new InMemoryStream(); + const reached = withResolvers(); + const files = recordedFiles( + { "notes.md": NOTE, "second.md": "the second note\n" }, + { + *hold(path) { + if (path === "notes.md") { + reached.resolve(); + yield* suspend(); + } + }, + }, + ); + + yield* scoped(function* () { + const running = yield* spawn(() => + run( + `\\n\\n\\n'} />\n`, + [reading(files)], + stream, + ), + ); + yield* reached.operation; + yield* running.halt(); + }); + + const events = yield* stream.readAll(); + // The admission committed — it is the decision, and it precedes the effects + // it authorized. + expect(admissions(events)).toHaveLength(1); + // The first admitted read entered and never answered; the second never + // started. A run that recorded a terminal result here would resume + // believing the fragment finished. + expect(files.performed).toEqual(["read notes.md"]); + expect(events.some((event) => event.type === "close" && event.coroutineId === "root")).toBe( + false, + ); + }); + + it("FE17: cancellation before the producer enters is the negative control", function* () { + const stream = new InMemoryStream(); + const files = recordedFiles({ "notes.md": NOTE }); + const entered: string[] = []; + + yield* scoped(function* () { + const running = yield* spawn(() => + scoped(function* () { + yield* registerComponents([ + { + name: "Never", + origin: "test://producer", + props: { type: "object", properties: {}, additionalProperties: false }, + // deno-lint-ignore require-yield + *fn(): Operation { + entered.push("entered"); + return `\n`; + }, + }, + ]); + return yield* run(`\n\n\n`, [reading(files)], stream); + }), + ); + // Halted without waiting for any signal that work began. + yield* running.halt(); + }); + + // Whether the producer entered at all is not what this row fixes — what it + // fixes is that halting early records no admission either, so the row above + // is about *cancelling live work* rather than about halting in general. + expect(admissions(yield* stream.readAll())).toHaveLength(0); + expect(files.performed).toEqual([]); + expect(entered.length).toBeLessThanOrEqual(1); + }); + + it("FE3: the producer is told the narrowed vocabulary, not the site's", function* () { + const files = recordedFiles({ "notes.md": NOTE }); + const told: string[] = []; + yield* scoped(function* () { + yield* registerComponents([ + { + name: "Peek", + origin: "test://peek", + props: { + type: "object", + properties: { text: { type: "string" } }, + required: ["text"], + additionalProperties: false, + }, + // deno-lint-ignore require-yield + *fn(props: Record): Operation { + told.push(String(props.text)); + return ""; + }, + }, + { + name: "Program", + origin: "test://producer", + props: { type: "object", properties: {}, additionalProperties: false }, + // deno-lint-ignore require-yield + *fn(): Operation { + return `\n`; + }, + }, + ]); + return yield* run( + `\n\n\n\n\n\n\n`, + [reading(files)], + ); + }); + + expect(told).toHaveLength(1); + const vocabulary = told[0] ?? ""; + // The admitted entry is described, and only in the form it was admitted + // for. Nothing else is: an agent told it had ``, `` or the + // producer's own `` would write a fragment the evaluator refuses + // whole, before its first effect. + expect(vocabulary).toContain("File"); + expect(vocabulary).not.toContain("Loop"); + expect(vocabulary).not.toContain("Evaluate"); + expect(vocabulary).not.toContain("Program"); + expect(vocabulary).not.toContain("Glob"); + }); + + it("FE3: hostile content middleware does not reach the producer projection", function* () { + const files = recordedFiles({ "notes.md": NOTE }); + const intercepted: string[] = []; + const output = yield* scoped(function* () { + // Every public way a handler can reach an element's content, answered + // rather than delegated. A handler that could substitute at any of them + // would decide what program ran. + yield* Component.around({ + // deno-lint-ignore require-yield + *content(_args, _next) { + intercepted.push("content"); + return `\n`; + }, + // deno-lint-ignore require-yield + *tryContent(_args, _next) { + intercepted.push("tryContent"); + return { text: `\n`, segments: [], failure: undefined }; + }, + // deno-lint-ignore require-yield + *hasContent(_args, _next) { + return false; + }, + }); + yield* registerComponents([ + { + name: "Producer", + origin: "test://producer", + props: { type: "object", properties: {}, additionalProperties: false }, + // deno-lint-ignore require-yield + *fn(): Operation { + return `\n`; + }, + }, + { + // The positive control, in the same run and under the same handlers: + // an ordinary component reading its content through the public chain. + // If this one is not intercepted either, the handlers are not live and + // the assertion below would pass for the wrong reason. + name: "Ordinary", + origin: "test://ordinary", + props: { type: "object", properties: {}, additionalProperties: false }, + *fn(): Operation { + return yield* content(); + }, + }, + ]); + return yield* run( + `\n\n\n\nignored\n\n` + + `\n`, + [reading(files)], + ); + }); + + // The handlers are live: the ordinary component's content was answered by + // them, and its element rendered the substitution rather than "ignored". + expect(intercepted).toContain("content"); + // The projection is not: the program is what the producer rendered, and the + // substituted path was never read. + expect(String(output)).toContain("the retained note"); + expect(files.performed).toEqual(["read notes.md"]); + expect(files.performed).not.toContain("read substituted.md"); + }); +}); + +describe("Tier FE — allow selects, and never adds", () => { + it("FE4: omitting `allow` admits exactly the read table", function* () { + const files = recordedFiles(); + const failed = yield* refusal( + run(`x\\n'} />\n`, [both(files)]), + ); + + expect(failed).toContain("admitted only in its self-closing form"); + expect(files.performed).toEqual([]); + }); + + it("FE4: a class this host installed nothing for is refused before any program", function* () { + const stream = new InMemoryStream(); + const files = recordedFiles(); + const failed = yield* refusal( + run( + `\\n'} allow={["write"]} />\n`, + [reading(files)], + stream, + ), + ); + + expect(failed).toContain("installed no write table"); + expect(admissions(yield* stream.readAll())).toHaveLength(0); + expect(files.performed).toEqual([]); + }); + + it("FE4: an admitted write reaches the captured write operations and binds nothing", function* () { + const files = recordedFiles(); + const output = yield* run( + `\\n\\nmade\\n\\n\\n'} ` + + `allow={["write"]} as="answer" />\n\n\n`, + [both(files)], + ); + + expect(files.performed).toEqual([ + "ensure nested", + "check nested/out.md", + "write nested/out.md", + ]); + expect(files.entries.get("nested/out.md")).toBe("made"); + // A mutation contributes no observation. + expect(String(output)).toContain('"observations": []'); + }); + + it("FE4: an admitted deletion runs, and reads are not admitted with it", function* () { + const files = recordedFiles({ "gone.md": "x" }); + yield* run(`\\n'} allow={["write"]} />\n`, [ + both(files), + ]); + expect(files.performed).toEqual(["delete gone.md"]); + expect(files.entries.has("gone.md")).toBe(false); + + const second = recordedFiles({ "notes.md": NOTE }); + expect( + yield* refusal( + run(`\\n'} allow={["write"]} />\n`, [ + both(second), + ]), + ), + ).toContain("admitted only in its paired form"); + expect(second.performed).toEqual([]); + }); + + it("FE4: a selection this component does not have is refused", function* () { + expect( + yield* refusal( + run(`\n`, [ + reading(recordedFiles()), + ]), + ), + ).toMatch(/allow|enum/i); + }); +}); + +describe("Tier FE — an execution with no ceiling has no evaluation", () => { + it("FE5: a host that stated no profile refuses at the element, not at startup", function* () { + expect(String(yield* run(`nothing asked for\n`, []))).toContain("nothing asked for"); + + expect(yield* refusal(run(`\n`, []))).toContain( + "this host stated none", + ); + }); + + it("FE5: two installations stating a profile is refused before anything installs", function* () { + const files = recordedFiles(); + expect(yield* refusal(run(`nothing asked for\n`, [reading(files), reading(files)]))).toContain( + "one maximum authority", + ); + }); + + it("FE5: a profile admitting a file entry without operations is refused", function* () { + expect( + yield* refusal(run(`nothing asked for\n`, [{ evaluation: { read: [fileReadEntry()] } }])), + ).toContain("without stating the filesystem operations"); + }); +}); + +describe("Tier FE — what the element itself may say", () => { + it("FE6: a fragment carrying a construct the evaluator does not admit refuses", function* () { + const refused: Array<[string, string]> = [ + ["an executable code block", "```ts exec\\nconsole.log(1)\\n```\\n"], + ["an expression prop", `\\n`], + ["an interpolated binding", `\\n{binding}\\n`], + ["an `as` binding", `\\n`], + ["a structural construct", `\\n\\n\\n`], + ]; + const outcomes: Array<[string, string[]]> = []; + for (const [what, fragment] of refused) { + const files = recordedFiles({ "a.md": NOTE }); + yield* refusal(run(`\n`, [reading(files)])); + outcomes.push([what, files.performed]); + } + // Refused whole, before the first effect: every one of them performed + // nothing at all, and the label says which row would have. + expect(outcomes).toEqual(refused.map(([what]) => [what, []])); + }); + + it("FE7: `as` captures the result and emits nothing; without it nothing is emitted either", function* () { + const captured = recordedFiles({ "notes.md": NOTE }); + const bound = yield* run( + `\\n'} as="answer" />\n\nbetween\n`, + [reading(captured)], + ); + // The value went to the binding, so the element emitted nothing of its own. + expect(String(bound)).toContain("between"); + expect(String(bound)).not.toContain("the retained note"); + + const loose = recordedFiles({ "notes.md": NOTE }); + const unbound = yield* run(`\\n'} />\n\nbetween\n`, [ + reading(loose), + ]); + // And an unbound occurrence emits nothing either: the result is a value, + // and a value has nowhere to render. The read still happened. + expect(String(unbound)).toContain("between"); + expect(String(unbound)).not.toContain("the retained note"); + expect(loose.performed).toEqual(["read notes.md"]); + }); +}); + +describe("Tier FE — one occurrence, one durable decision", () => { + it("FE16: two occurrences do not consume one another's records", function* () { + const files = recordedFiles({ "one.md": "first\n", "two.md": "second\n" }); + const stream = new InMemoryStream(); + yield* run( + `\\n'} />\n\n` + + `\\n'} />\n`, + [reading(files)], + stream, + ); + + const recorded = admissions(yield* stream.readAll()); + expect(recorded).toHaveLength(2); + // Two durable names. One shared name would make the second occurrence + // replay the first's admitted fragment. + const names = recorded.map((event) => (event.type === "yield" ? event.description.name : "")); + expect(new Set(names).size).toBe(2); + expect(files.performed).toEqual(["read one.md", "read two.md"]); + }); + + it("FE10: a continuation resumes the exact admitted text", function* () { + const first = recordedFiles({ "notes.md": NOTE }); + const stream = new InMemoryStream(); + const source = `\\n'} as="answer" />\n`; + yield* run(source, [reading(first)], stream); + const complete = yield* stream.readAll(); + + // Truncated to the admission itself: the decision committed, and the read + // it authorized had not. That is the only state in which a resumed run + // still has fragment work left to do. + const admitted = complete.findIndex( + (event) => event.type === "yield" && event.description.type === "generated_xmd", + ); + expect(admitted).toBeGreaterThanOrEqual(0); + const partial = complete.slice(0, admitted + 1); + + const second = recordedFiles({ "notes.md": NOTE }); + yield* run(source, [reading(second)], new InMemoryStream(partial)); + // The retained admission was restored rather than made again, and the read + // it authorized ran on this attempt. + expect(second.performed).toEqual(["read notes.md"]); + }); + + it("FE10: a continuation offering different text refuses before any effect", function* () { + // The document is byte-identical across both attempts, so the occurrence + // identity — and therefore the durable name — is the same. What differs is + // the program the producer rendered, which is exactly the case a retained + // admission has to refuse: a decision was made about one fragment, and the + // run is now holding another. + const DOCUMENT = `\n\n\n`; + + function producing(fragment: string) { + return function* (): Operation { + yield* registerComponents([ + { + name: "Program", + origin: "test://producer", + props: { type: "object", properties: {}, additionalProperties: false }, + // deno-lint-ignore require-yield + *fn(): Operation { + return fragment; + }, + }, + ]); + }; + } + + const first = recordedFiles({ "notes.md": NOTE }); + const stream = new InMemoryStream(); + yield* scoped(function* () { + yield* producing(`\n`)(); + yield* run(DOCUMENT, [reading(first)], stream); + }); + const complete = yield* stream.readAll(); + const admitted = complete.findIndex( + (event) => event.type === "yield" && event.description.type === "generated_xmd", + ); + expect(admitted).toBeGreaterThanOrEqual(0); + const partial = complete.slice(0, admitted + 1); + + const second = recordedFiles({ "notes.md": NOTE, "other.md": "elsewhere\n" }); + const failed = yield* refusal( + scoped(function* () { + yield* producing(`\n`)(); + return yield* run(DOCUMENT, [reading(second)], new InMemoryStream(partial)); + }), + ); + + expect(failed).toContain("the exact text it was made about"); + // Nothing was performed: neither the retained fragment nor the offered one. + expect(second.performed).toEqual([]); + }); + + it("FE10: a continuation whose ceiling moved refuses before any effect", function* () { + const first = recordedFiles({ "notes.md": NOTE }); + const stream = new InMemoryStream(); + const source = `\\n'} />\n`; + yield* run(source, [reading(first)], stream); + const complete = yield* stream.readAll(); + const admitted = complete.findIndex( + (event) => event.type === "yield" && event.description.type === "generated_xmd", + ); + const partial = complete.slice(0, admitted + 1); + + // The same fragment, admitted under a table that now states another + // identity. A grant is the exact set it was made under. + const second = recordedFiles({ "notes.md": NOTE }); + const widened: ExecutionInstallation = { + evaluation: { + read: [ + { + ...fileReadEntry(), + identity: { origin: "@executablemd/core", key: "File:read", revision: "99" }, + }, + ], + files: second, + }, + }; + const failed = yield* refusal(run(source, [widened], new InMemoryStream(partial))); + + expect(failed).toContain("no longer states"); + expect(second.performed).toEqual([]); + }); + + it("FE11: a completed evaluation replays without performing anything again", function* () { + const first = recordedFiles({ "notes.md": NOTE }); + const stream = new InMemoryStream(); + const source = `\\n'} as="answer" />\n\n\n`; + const original = yield* run(source, [reading(first)], stream); + expect(first.performed).toEqual(["read notes.md"]); + + // The whole history, terminal close included. A second run over it restores + // the admission *and* the observation it authorized. + const complete = yield* stream.readAll(); + const second = recordedFiles({ "notes.md": "a note this run must not read\n" }); + const replayed = yield* run(source, [reading(second)], new InMemoryStream(complete)); + + // Nothing was performed again — the recorder's log is empty, and the file + // it holds now says something else, so a re-read would be visible in the + // output rather than merely in the count. + expect(second.performed).toEqual([]); + expect(String(replayed)).toContain("the retained note"); + expect(String(replayed)).not.toContain("must not read"); + expect(String(replayed)).toBe(String(original)); + }); + + it("FE12: a retained record this version cannot read fails closed", function* () { + const files = recordedFiles({ "notes.md": NOTE }); + const stream = new InMemoryStream(); + const source = `\\n'} />\n`; + yield* run(source, [reading(files)], stream); + const complete = yield* stream.readAll(); + + // The admission's own result, replaced with a shape no version wrote. A + // record that cannot be read is not a decision to guess at. + const hostile = complete.map((event) => { + if (event.type !== "yield" || event.description.type !== "generated_xmd") { + return event; + } + return { ...event, result: { status: "ok" as const, value: { decision: "maybe" } } }; + }); + const admitted = hostile.findIndex( + (event) => event.type === "yield" && event.description.type === "generated_xmd", + ); + + const second = recordedFiles({ "notes.md": NOTE }); + const failed = yield* refusal( + run(source, [reading(second)], new InMemoryStream(hostile.slice(0, admitted + 1))), + ); + + expect(failed.length).toBeGreaterThan(0); + expect(second.performed).toEqual([]); + }); +}); + +describe("Tier FE — protection settles which implementation runs, and grants nothing", () => { + it("FE26: protection adds no class and no identity to what `allow` selects", function* () { + // A profile with no write table. `` being canonical core's does + // not add one, and no spelling of `allow` conjures one. + const files = recordedFiles(); + expect( + yield* refusal( + run(`x\\n'} allow={["write"]} />\n`, [ + reading(files), + ]), + ), + ).toContain("installed no write table"); + expect(files.performed).toEqual([]); + }); + + it("FE23: a name in the enclosing symbols is still not admitted", function* () { + // `` is an ordinary core component and appears in the document's own + // vocabulary. Naming it in a fragment reaches the fragment's table, which + // does not hold it — symbols text registers, resolves and authorizes + // nothing. + const files = recordedFiles({ "notes.md": NOTE }); + const failed = yield* refusal( + run(`\\n'} />\n`, [reading(files)]), + ); + expect(failed).toContain("did not admit"); + expect(files.performed).toEqual([]); + }); + + it("FE29: nothing a document controls reads, replaces or widens the profile", function* () { + const files = recordedFiles({ "notes.md": NOTE }); + const output = yield* scoped(function* () { + // A component that binds every composable channel it has before the + // `` beneath it runs. + yield* registerComponents([ + { + name: "Forge", + origin: "test://forge", + props: { type: "object", properties: {}, additionalProperties: false }, + *fn(): Operation { + yield* API.Files.around( + { + // deno-lint-ignore require-yield + *readTextFile([input]): Operation> { + return Ok(`forged ${String(input.path)}`); + }, + }, + { at: "min" }, + ); + return yield* content(); + }, + }, + ]); + return yield* run( + `\n\\n'} as="answer" />\n\n` + + `\n\n`, + [reading(files)], + ); + }); + + // The fragment read the captured operation's file, not the forged one. + expect(String(output)).toContain("the retained note"); + expect(String(output)).not.toContain("forged"); + expect(files.performed).toEqual(["read notes.md"]); + }); +}); + +/** + * Tier FE14 — an implementation the ordinary import chain answered for. + * + * A `component-answer` entry is the one arm of a profile where the host does + * not supply the body. It states a name and the exact structural identity a + * provider must have claimed, and canonical execution resolves that name once — + * before the root import, through the complete ordinary chain — reads the claim + * off the exact final answer, compares it whole, and seals what it retained. + * + * So the rows here are about the two ways that could be weaker than it looks: + * an answer nothing identified, and an answer that stopped being the thing that + * was identified. Each refusal names what did *not* happen — no admission, no + * body, no fragment effect — because a refusal that arrived after A ran would + * satisfy an error-shape assertion and none of these. + */ +describe("Tier FE14 — the chain answers, and the answer is held to its identity", () => { + const OPEN = `\\n'} as="answer" />\n\n\n`; + + /** The entry a host states for a provider-backed name. */ + function admits( + identity: { origin?: string; key?: string; revision?: string } = {}, + files: RecordedFiles = recordedFiles(), + ): FragmentEvaluationInput { + return { + read: [ + { + kind: "component-answer", + name: "Open", + identity: { + origin: identity.origin ?? "test://provider", + key: identity.key ?? "Open", + revision: identity.revision ?? "1", + }, + forms: ["self-closing"], + }, + ], + files, + }; + } + + /** One installation admitting `` and backing it with one provider. */ + function backed( + answer: Implementation, + options: ProviderOptions = {}, + identity: { origin?: string; key?: string; revision?: string } = {}, + ): ExecutionInstallation { + return { + evaluation: admits(identity), + componentAnswers: [answerProvider("Open", answer.definition, options)], + }; + } + + /** The index of the admission record, or -1 when the run made none. */ + function admittedAt(events: readonly DurableEvent[]): number { + return events.findIndex( + (event) => event.type === "yield" && event.description.type === "generated_xmd", + ); + } + + /** The identity one admission retained for the name it admitted. */ + function retainedIdentity(event: DurableEvent): Json { + const result = event.type === "yield" ? event.result : undefined; + if (result === undefined || result.status !== "ok" || !isRecord(result.value)) { + throw new Error("the admission recorded no result"); + } + const policy = result.value.policy; + if (!isRecord(policy) || !Array.isArray(policy.allowed)) { + throw new Error("the admission recorded no policy"); + } + const entry = policy.allowed[0]; + if (!isRecord(entry)) { + throw new Error("the admission admitted nothing"); + } + return entry.identity ?? null; + } + + function isRecord(value: Json | undefined): value is Record { + return typeof value === "object" && value !== null && !Array.isArray(value); + } + + it("FE14: the final chain answer is the implementation a fragment runs", function* () { + const A = implementation("Open", "A ran"); + // Nothing registers `Open`, no file supplies it, and the provider answers + // without delegating — so the chain's final answer is this implementation + // or the import fails. There is no second place it could have come from. + const output = yield* run(OPEN, [backed(A)]); + + expect(String(output)).toContain("A ran"); + expect(A.invoked).toEqual(["A ran"]); + }); + + it("FE14: a fresh admission records the exact structural identity", function* () { + const A = implementation("Open", "A ran"); + const stream = new InMemoryStream(); + yield* run(OPEN, [backed(A)], stream); + + const events = yield* stream.readAll(); + const admission = events[admittedAt(events)]; + if (admission === undefined) { + throw new Error("the run recorded no admission"); + } + // The four members as themselves, and the kind among them: an answer the + // chain resolved is a different grant from an operation core supplies the + // body for, even under the same origin, key and revision. + expect(retainedIdentity(admission)).toEqual({ + kind: "component-answer", + origin: "test://provider", + key: "Open", + revision: "1", + }); + expect(A.invoked).toEqual(["A ran"]); + }); + + it("FE14: an inner claim and an outer unclaimed answer refuse before any body", function* () { + const A = implementation("Open", "A ran"); + const B = implementation("Open", "B ran"); + const files = recordedFiles({ "notes.md": NOTE }); + const stream = new InMemoryStream(); + + const failed = yield* refusal( + scoped(function* () { + // A handler further out returns its own object. The inner provider's + // claim was about the object it returned, and this is not that object — + // so nothing identifies the answer the chain finally gave. + yield* Component.around({ + *importComponent([name], next) { + const definition = yield* next(name); + return name === "Open" ? B.definition : definition; + }, + }); + return yield* run(OPEN, [{ ...backed(A), evaluation: admits({}, files) }], stream); + }), + ); + + expect(failed).toContain("carries no identity"); + // Ahead of everything: no admission, neither body, no fragment effect. + expect(admissions(yield* stream.readAll())).toHaveLength(0); + expect(A.invoked).toEqual([]); + expect(B.invoked).toEqual([]); + expect(files.performed).toEqual([]); + }); + + it("FE14: an inner claimed A and an outer claimed B compose, and only B runs", function* () { + const A = implementation("Open", "A ran"); + const B = implementation("Open", "B ran"); + const delegated: unknown[] = []; + const outerRequests: string[] = []; + + // The positive control for the row above, and a real chain rather than one + // provider. Two providers answer for one name: the one installed first + // composes outermost, delegates — so the inner one genuinely answers and + // claims A — and then returns its own claimed B. The difference from the + // refusal above is only that the identity travels with the object the chain + // finally gives back, which is what makes the substitution honest. + const output = yield* run(OPEN, [ + { + evaluation: admits({ key: "Outer" }), + componentAnswers: [ + answerProvider("Open", B.definition, { + key: "Outer", + delegatesFirst: true, + delegated, + whileResolving(request) { + outerRequests.push(request.name); + }, + }), + answerProvider("Open", A.definition, { key: "Inner" }), + ], + }, + ]); + + // The inner half really answered: what delegation returned is the exact + // object the inner provider claimed. So A was resolvable, and the row below + // it is about which of two live answers runs rather than about one. + expect(delegated[0]).toBe(A.definition); + expect(outerRequests).toEqual(["Open"]); + expect(String(output)).toContain("B ran"); + expect(B.invoked).toEqual(["B ran"]); + expect(A.invoked).toEqual([]); + }); + + it("FE14: an answer whose contract reads differently each time cannot substitute", function* () { + const reads: string[] = []; + // The claimed contract accepts one prop; the substitution accepts none. + const A = implementation("Open", "A ran", { + type: "object", + properties: { flag: { type: "string" } }, + additionalProperties: false, + }); + + // The check/use gap, planted. Reading the answer's schema alternates, and + // comparing own descriptors — which is how the claim is checked — does not + // run the trap. So the reading the capture *keeps* decides what a fragment + // is validated against: the claim-time schema admits `flag`, and the + // substitution refuses it. + const output = yield* run(`\\n'} as="answer" />\n`, [ + { + evaluation: admits(), + componentAnswers: [ + answerProvider("Open", A.definition, { + alternating: { + substitute: { type: "object", properties: {}, additionalProperties: false }, + reads, + }, + }), + ], + }, + ]); + + expect(String(output)).not.toContain("Error"); + expect(A.invoked).toEqual(["A ran"]); + // The plant is live rather than inert: the object the chain returned was + // read, and it does answer differently on the next read. + expect(reads.length).toBeGreaterThan(0); + }); + + it("FE14: a provider still answering when a fragment resolves is refused there", function* () { + const A = implementation("Open", "A ran"); + const B = implementation("Open", "B ran"); + const stream = new InMemoryStream(); + + // A provider settles at the capture: the profile sealed what it retained, + // and a fragment runs that snapshot. One that keeps answering is answering + // a *generated* import, which only canonical execution answers — so the + // witness refuses it rather than letting a live provider substitute into an + // admitted fragment. + const failed = yield* refusal( + run( + OPEN, + [ + { + evaluation: admits(), + componentAnswers: [answerProvider("Open", A.definition, { keepsAnswering: true })], + }, + ], + stream, + ), + ); + + expect(failed).toContain("canonical execution did not produce"); + expect(B.invoked).toEqual([]); + // The admission committed — the refusal is at the import, not the ceiling — + // and no body ran. + expect(A.invoked).toEqual([]); + }); + + it("FE14: a provider still claiming when a fragment resolves is refused earlier", function* () { + const A = implementation("Open", "A ran"); + const stream = new InMemoryStream(); + + // The same substitution, one step earlier. The resolution this provider + // answered settled during the capture, so there is no window left to state + // an identity into — the claim refuses before the witness is consulted, and + // an answer nothing could identify never reaches the fragment. + const failed = yield* refusal( + run( + OPEN, + [ + { + evaluation: admits(), + componentAnswers: [answerProvider("Open", A.definition, { reclaimsLater: true })], + }, + ], + stream, + ), + ); + + expect(failed).toContain("this resolution has settled"); + expect(A.invoked).toEqual([]); + }); + + it("FE14: an unidentified, a copied and a mutated answer each refuse", function* () { + const cases: readonly (readonly [string, ProviderOptions])[] = [ + ["nothing identified it", { unclaimed: true }], + ["the chain returned a copy of what was claimed", { copied: true }], + ["the claimed object was edited afterwards", { mutated: true }], + ]; + + for (const [what, options] of cases) { + const A = implementation("Open", "A ran"); + const files = recordedFiles({ "notes.md": NOTE }); + const stream = new InMemoryStream(); + const failed = yield* refusal( + run( + OPEN, + [ + { + evaluation: admits({}, files), + componentAnswers: [answerProvider("Open", A.definition, options)], + }, + ], + stream, + ), + ); + + expect([what, failed.includes("carries no identity")]).toEqual([what, true]); + // Before the root import, so there is no admission and no body — and the + // document's own effects never started either. + expect([what, admissions(yield* stream.readAll()).length]).toEqual([what, 0]); + expect([what, A.invoked]).toEqual([what, []]); + expect([what, files.performed]).toEqual([what, []]); + } + }); + + it("FE14: an answer under another identity refuses before any body", function* () { + const A = implementation("Open", "A ran"); + const files = recordedFiles(); + const stream = new InMemoryStream(); + + const failed = yield* refusal( + run( + OPEN, + [ + { + evaluation: admits({ revision: "1" }, files), + // The provider claims honestly — for a revision this host did not + // admit. An admitted identity is the exact implementation, and a + // changed revision is a changed grant. + componentAnswers: [answerProvider("Open", A.definition, { revision: "2" })], + }, + ], + stream, + ), + ); + + expect(failed).toContain("the exact implementation"); + expect(admissions(yield* stream.readAll())).toHaveLength(0); + expect(A.invoked).toEqual([]); + }); + + it("FE14: a continuation whose outer provider now answers B runs neither", function* () { + const A = implementation("Open", "A ran"); + const stream = new InMemoryStream(); + yield* run(OPEN, [backed(A)], stream); + const complete = yield* stream.readAll(); + const admitted = admittedAt(complete); + expect(admitted).toBeGreaterThanOrEqual(0); + // Truncated to the admission itself: the decision committed and the body it + // authorized had not, which is the only state with fragment work left. + const partial = complete.slice(0, admitted + 1); + + // The resumed run supplies both implementations, live. The inner provider + // answers the unchanged A under the identity the admission was made over; + // the outer one delegates to it and then answers with its own claimed B, + // under the identity this host now admits. So capture reconciles and seals + // B — and then the *retained* policy refuses, because the admission was + // made over `Open@1` and this run states `Other@1` behind the same name. + const resumedA = implementation("Open", "A ran"); + const B = implementation("Open", "B ran"); + const delegated: unknown[] = []; + const failed = yield* refusal( + run( + OPEN, + [ + { + evaluation: admits({ key: "Other" }), + componentAnswers: [ + answerProvider("Open", B.definition, { + key: "Other", + delegatesFirst: true, + delegated, + }), + answerProvider("Open", resumedA.definition, { key: "Open" }), + ], + }, + ], + new InMemoryStream(partial), + ), + ); + + expect(failed).toContain("admitted under"); + // Both were genuinely reachable in this run — the unchanged A answered the + // chain, which is what makes its silence a fact rather than an absence — + // and neither body ran. + expect(delegated[0]).toBe(resumedA.definition); + expect(resumedA.invoked).toEqual([]); + expect(B.invoked).toEqual([]); + }); + + it("FE14: an unchanged continuation resumes and invokes A once", function* () { + const A = implementation("Open", "A ran"); + const stream = new InMemoryStream(); + yield* run(OPEN, [backed(A)], stream); + const complete = yield* stream.readAll(); + const partial = complete.slice(0, admittedAt(complete) + 1); + + const resumed = implementation("Open", "A ran"); + const output = yield* run(OPEN, [backed(resumed)], new InMemoryStream(partial)); + + // The retained admission was restored rather than made again, and the body + // it authorized ran on this attempt — once. + expect(String(output)).toContain("A ran"); + expect(resumed.invoked).toEqual(["A ran"]); + }); + + it("FE14: two occurrences share one capture lookup and reach no chain again", function* () { + const A = implementation("Open", "A ran"); + const B = implementation("Open", "B ran"); + const lookups: string[] = []; + const stream = new InMemoryStream(); + + const asked: string[] = []; + const output = yield* run( + `\\n'} as="one" />\n\n\n\n` + + `\\n'} as="two" />\n\n\n`, + [ + { + evaluation: admits(), + componentAnswers: [answerProvider("Open", A.definition, { lookups, asked })], + }, + ], + stream, + ); + + // Answered exactly once, eagerly, during capture — and the two fragments + // between them asked twice more, which the provider observed and delegated. + // So neither fragment resolved this name through the provider; both ran the + // snapshot the capture sealed. + expect(lookups).toEqual(["Open"]); + expect(asked).toEqual(["Open", "Open", "Open"]); + expect(admissions(yield* stream.readAll())).toHaveLength(2); + expect(A.invoked).toEqual(["A ran", "A ran"]); + expect(B.invoked).toEqual([]); + expect(String(output)).not.toContain("B ran"); + }); + + it("FE14: a capability-only profile resolves no name at all", function* () { + const lookups: string[] = []; + const files = recordedFiles({ "notes.md": NOTE }); + + // The negative control for the eager lookup: a host that admits no + // component answer pays for no component-chain resolution, and a document + // that never writes `` still installs the provider. + const output = yield* run( + `\\n'} as="answer" />\n\n\n`, + [ + { + evaluation: { read: [fileReadEntry()], files }, + componentAnswers: [ + answerProvider("Open", implementation("Open", "A ran").definition, { lookups }), + ], + }, + ], + ); + + expect(lookups).toEqual([]); + expect(String(output)).toContain("the retained note"); + }); +}); diff --git a/packages/core/tests/evaluate-loaded-copy.test.ts b/packages/core/tests/evaluate-loaded-copy.test.ts new file mode 100644 index 000000000..ff31c7884 --- /dev/null +++ b/packages/core/tests/evaluate-loaded-copy.test.ts @@ -0,0 +1,334 @@ +/** + * Tier FE — who answers for the name `Evaluate`, and what answering grants. + * + * Two separate claims, and they are proved separately here because one is + * routinely mistaken for the other. + * + * *Ownership* is that canonical core decides which implementation runs. A + * repository file, a registration, a declared Markdown component, an import + * handler's substituted definition and a second loaded copy of core each fail + * to replace it (FE24), including a replacement written specifically to ignore + * `allow` (FE25). + * + * *Authority* is separate and is not conferred by ownership. Protection settles + * which body runs and adds nothing to what that body may do: a profile with no + * write table still cannot write, and a name in the enclosing symbols is still + * not admitted (FE26). A component that is replaceable at an ordinary authored + * site stays replaceable there, and that replaceability still does not let it + * into a fragment (FE27). + */ + +import { describe, it } from "@executablemd/test-support/bdd"; +import { expect } from "@executablemd/test-support/expect"; +import { InMemoryStream } from "@executablemd/durable-streams"; +import type { DurableEvent, Json } from "@executablemd/durable-streams"; +import { Ok, scoped } from "effection"; +import type { Operation, Result } from "effection"; +import { API, fileWriteSuccess } from "@executablemd/runtime"; + +import { collect } from "../src/collect.ts"; +import { Component } from "../src/component-api.ts"; +import { executeInstalled } from "../host.ts"; +import { fileDeleteEntry, fileReadEntry, fileWriteEntry } from "../host.ts"; +import type { ExecutionInstallation } from "../host.ts"; +import { registerComponents } from "../src/components/registration.ts"; +import { retainedSource } from "../src/root-source.ts"; +import { recordedFiles } from "./support/fragment-files.ts"; +import type { RecordedFiles } from "./support/fragment-files.ts"; + +const ROOT_PATH = "evaluate.md"; +const NOTE = "the retained note\n"; + +/** A read-only profile: no write table at all. */ +function reading(files: RecordedFiles): ExecutionInstallation { + return { evaluation: { read: [fileReadEntry()], files } }; +} + +/** A profile offering both classes. */ +function both(files: RecordedFiles): ExecutionInstallation { + return { + evaluation: { + read: [fileReadEntry()], + write: [fileWriteEntry(), fileDeleteEntry()], + files, + }, + }; +} + +function run( + source: string, + installations: readonly ExecutionInstallation[], + stream: InMemoryStream = new InMemoryStream(), +): Operation { + return scoped(function* () { + return yield* collect( + yield* executeInstalled({ ...retainedSource(ROOT_PATH, source), stream }, [...installations]), + ); + }); +} + +function* refusal(operation: Operation): Operation { + try { + yield* operation; + } catch (error) { + return error instanceof Error ? error.message : String(error); + } + throw new Error("expected the operation to be refused"); +} + +function admissions(events: readonly DurableEvent[]): DurableEvent[] { + return events.filter( + (event) => event.type === "yield" && event.description.type === "generated_xmd", + ); +} + +const READS = + `\\n'} as="answer" />\n\n` + `\n`; + +describe("Tier FE — nothing replaces the protected implementation", () => { + it("FE24: the name cannot be registered at all", function* () { + const reached: string[] = []; + const failed = yield* refusal( + scoped(function* () { + yield* registerComponents([ + { + name: "Evaluate", + origin: "test://replacement", + props: { type: "object", properties: {}, additionalProperties: true }, + // deno-lint-ignore require-yield + *fn(): Operation { + reached.push("registration"); + return "the replacement answered"; + }, + }, + ]); + }), + ); + + // Refused where the claim is made, rather than resolved against later: a + // registry that accepted the name and lost selection would still be a + // registry that had it to hand back somewhere else. + expect(failed).toContain("canonical core owns that name"); + expect(reached).toEqual([]); + }); + + it("FE24: a live registration of the name does not exist to be selected", function* () { + // The positive half of the row above: with the claim refused, an ordinary + // document still resolves `` to canonical core's implementation + // and the fragment runs. + const files = recordedFiles({ "notes.md": NOTE }); + const output = yield* run(READS, [reading(files)]); + expect(String(output)).toContain("the retained note"); + expect(files.performed).toEqual(["read notes.md"]); + }); + + it("FE24: a middleware answer is refused, and honest delegation is not", function* () { + const files = recordedFiles({ "notes.md": NOTE }); + const substituted = yield* refusal( + scoped(function* () { + yield* Component.around({ + *importComponent([name], next) { + const definition = yield* next(name); + if (name !== "Evaluate" || definition.kind !== "function") { + return definition; + } + // A wrapper that forwards everything and changes nothing. Refused + // anyway: only canonical execution answers a protected import. + return { ...definition }; + }, + }); + return yield* run(READS, [reading(files)]); + }), + ); + expect(substituted).toContain("canonical core owns"); + expect(files.performed).toEqual([]); + + // The same handler, delegating honestly, is still supported. + const observed: string[] = []; + const clean = recordedFiles({ "notes.md": NOTE }); + const output = yield* scoped(function* () { + yield* Component.around({ + *importComponent([name], next) { + const definition = yield* next(name); + observed.push(name); + return definition; + }, + }); + return yield* run(READS, [reading(clean)]); + }); + expect(observed).toContain("Evaluate"); + expect(String(output)).toContain("the retained note"); + }); + + it("FE24: a deliberate middleware refusal stays a refusal", function* () { + const files = recordedFiles({ "notes.md": NOTE }); + const failed = yield* refusal( + scoped(function* () { + yield* Component.around({ + *importComponent([name], next) { + if (name === "Evaluate") { + throw new Error("this environment does not offer Evaluate"); + } + return yield* next(name); + }, + }); + return yield* run(READS, [reading(files)]); + }), + ); + + // A handler may refuse an import. What it may not do is answer one. + expect(failed).toContain("does not offer Evaluate"); + expect(files.performed).toEqual([]); + }); + + it("FE25: a replacement written to ignore `allow` is never invoked", function* () { + // The replacement's body would write through the widest ambient authority + // it can reach, under a read-only selection. It never runs, and canonical + // `` refuses the write before any provider call. + const files = recordedFiles(); + const ambientWrites: string[] = []; + const reached: string[] = []; + + const failed = yield* refusal( + scoped(function* () { + yield* API.Files.around( + { + // deno-lint-ignore require-yield + *writeTextFile([input]) { + ambientWrites.push(String(input.path)); + return Ok(fileWriteSuccess("host-committed")); + }, + }, + { at: "min" }, + ); + // Registration is refused outright, so the only channel left for a + // replacement is a handler answering the import. This one answers with + // a body that writes through the widest ambient authority it can reach. + yield* Component.around({ + *importComponent([name], next) { + const definition = yield* next(name); + if (name !== "Evaluate" || definition.kind !== "function") { + return definition; + } + return { + ...definition, + *fn(): Operation { + reached.push("replacement"); + yield* API.Files.operations.writeTextFile({ + cwd: "/", + path: "escaped.md", + content: "written by the replacement", + }); + return ""; + }, + }; + }, + }); + return yield* run( + `x\\n'} allow={["write"]} />\n`, + [reading(files)], + ); + }), + ); + + // The replacement was never entered and nothing was written anywhere: the + // refusal happens at the import boundary, before the body could ignore + // `allow`. + expect(reached).toEqual([]); + expect(ambientWrites).toEqual([]); + expect(failed).toContain("canonical core owns"); + expect(files.performed).toEqual([]); + }); +}); + +describe("Tier FE — ownership is not authority", () => { + it("FE26: owning the name adds no class and no identity", function* () { + const files = recordedFiles(); + // A host offering only reads. Canonical `` cannot add a write + // table, and no spelling of `allow` conjures one. + expect( + yield* refusal( + run(`x\\n'} allow={["write"]} />\n`, [ + reading(files), + ]), + ), + ).toContain("installed no write table"); + + // And with a write table, the read table is still not part of `write`: + // selecting a class reaches that class's own entries and no others. + const mixed = recordedFiles({ "notes.md": NOTE }); + expect( + yield* refusal( + run(`\\n'} allow={["write"]} />\n`, [both(mixed)]), + ), + ).toContain("admitted only in its paired form"); + expect(mixed.performed).toEqual([]); + }); + + it("FE27: a replaceable ordinary component still wins at an authored site", function* () { + const files = recordedFiles({ "notes.md": NOTE }); + const stream = new InMemoryStream(); + const output = yield* scoped(function* () { + // `` is an ordinary registration, so a repository or host component + // of that name replaces it at an authored site. That is unchanged. + yield* registerComponents([ + { + name: "File", + origin: "test://repository", + props: { + type: "object", + properties: { path: { type: "string" } }, + required: ["path"], + additionalProperties: false, + }, + // deno-lint-ignore require-yield + *fn(): Operation { + return "the repository component answered"; + }, + }, + ]); + return yield* run(`\n\n${READS}`, [reading(files)], stream); + }); + + const rendered = String(output); + // The authored site got the replacement... + expect(rendered).toContain("the repository component answered"); + // ...and the fragment did not: it reached the pinned identity the profile + // admitted, through the captured operation. + expect(rendered).toContain("the retained note"); + expect(files.performed).toEqual(["read notes.md"]); + expect(admissions(yield* stream.readAll())).toHaveLength(1); + }); + + it("FE27: a repository component cannot enter a fragment under an admitted name", function* () { + const files = recordedFiles({ "notes.md": NOTE }); + const reached: string[] = []; + const output = yield* scoped(function* () { + yield* registerComponents([ + { + name: "File", + origin: "test://repository", + props: { + type: "object", + properties: { path: { type: "string" } }, + required: ["path"], + additionalProperties: false, + }, + // deno-lint-ignore require-yield + *fn(): Operation { + reached.push("repository"); + return "replaced"; + }, + }, + ]); + return yield* run(READS, [reading(files)]); + }); + + // The name is admitted, the registration is live, and the fragment still + // ran the pinned identity: an evaluator that resolved through the registry + // would have let a repository file into an admitted fragment. + expect(reached).toEqual([]); + expect(String(output)).toContain("the retained note"); + expect(files.performed).toEqual(["read notes.md"]); + }); +}); diff --git a/packages/core/tests/evaluate-provider-lifetime.test.ts b/packages/core/tests/evaluate-provider-lifetime.test.ts new file mode 100644 index 000000000..627ffa127 --- /dev/null +++ b/packages/core/tests/evaluate-provider-lifetime.test.ts @@ -0,0 +1,641 @@ +/** + * Tier FE15 — what a provider's answer is bound to, at execution scale. + * + * `answer-identity.test.ts` holds the unit half: an identity belongs to one + * object in one execution, and a request that outlived its invocation states + * nothing. These rows are the other half — the same guarantees driven through + * a real execution, where the provider is installed by a trusted host, the + * resolution happens during capture before the root import, and the sealed + * answer is what a fragment runs. + * + * The failure this tier exists to prevent is a provider that keeps working + * after the run that admitted it. A resolution suspended when the execution is + * cancelled, a request retained past teardown, a losing answer arriving late, + * and a captured body invoked afterwards are four shapes of the same thing, and + * each of them here is asked about state the run actually left behind rather + * than about the wording of a refusal. + */ + +import { describe, it } from "@executablemd/test-support/bdd"; +import { expect } from "@executablemd/test-support/expect"; +import { InMemoryStream } from "@executablemd/durable-streams"; +import type { DurableEvent, Json } from "@executablemd/durable-streams"; +import { ensure, scoped, spawn, suspend, withResolvers } from "effection"; +import type { Operation } from "effection"; + +import { collect } from "../src/collect.ts"; +import { Component } from "../src/component-api.ts"; +import { executeInstalled } from "../host.ts"; +import type { + ComponentAnswerRegistrar, + ComponentAnswerRequest, + ExecutionInstallation, + FragmentEvaluationInput, +} from "../host.ts"; +import { registerComponents } from "../src/components/registration.ts"; +import { prepareEvaluationProfile } from "../src/evaluation-profile.ts"; +import type { ComponentAnswerEntry, ResolvedAnswer } from "../src/evaluation-profile.ts"; +import { REVOKED_CAPABILITY } from "../src/fragment-capabilities.ts"; +import { retainedSource } from "../src/root-source.ts"; +import { recordedFiles } from "./support/fragment-files.ts"; +import { answerProvider, implementation } from "./support/answer-provider.ts"; +import type { Implementation } from "./support/answer-provider.ts"; +import type { ComponentInvocation } from "../src/invocation-identity.ts"; + +const ROOT_PATH = "evaluate.md"; +const OPEN = `\\n'} as="answer" />\n\n\n`; + +/** What a provider states for the one name these rows admit. */ +const IDENTITY = { key: "Open", revision: "1" } as const; + +/** One provider-backed entry, under the identity this suite's providers claim. */ +function answerEntry(name: string): ComponentAnswerEntry { + return { + kind: "component-answer", + name, + identity: { origin: "test://provider", key: name, revision: "1" }, + forms: ["self-closing"], + }; +} + +/** The entry a host states for a provider-backed name. */ +function admits(): FragmentEvaluationInput { + return { read: [answerEntry("Open")], files: recordedFiles() }; +} + +/** + * The first sentence of what one call refused with. + * + * A row comparing whole refusal prose would be comparing wording; what these + * rows are about is *which* refusal happened — the settled resolution or the + * ended execution — so they read the clause that says so. + */ +function refused(attempt: () => unknown): string { + try { + attempt(); + } catch (error) { + const message = error instanceof Error ? error.message : String(error); + return message.split(",")[0]?.trim() ?? message; + } + throw new Error("expected the claim to be refused"); +} + +function run( + source: string, + installations: readonly ExecutionInstallation[], + stream: InMemoryStream = new InMemoryStream(), +): Operation { + return scoped(function* () { + return yield* collect( + yield* executeInstalled({ ...retainedSource(ROOT_PATH, source), stream }, [...installations]), + ); + }); +} + +/** What a retained request refused with when it was used, as a string. */ +function refusalFrom(request: ComponentAnswerRequest | undefined): string { + if (request === undefined) { + throw new Error("the provider was never asked"); + } + try { + request.claim(implementation("Open", "late").definition, { key: "Open", revision: "1" }); + } catch (error) { + return error instanceof Error ? error.message : String(error); + } + throw new Error("expected the retained request to refuse"); +} + +/** What one execution refused with, as a string. */ +function* refusal(operation: Operation): Operation { + try { + yield* operation; + } catch (error) { + return error instanceof Error ? error.message : String(error); + } + throw new Error("expected the operation to be refused"); +} + +/** Every generated-XMD admission a run recorded. */ +function admissions(events: readonly DurableEvent[]): DurableEvent[] { + return events.filter( + (event) => event.type === "yield" && event.description.type === "generated_xmd", + ); +} + +/** Whether a run got as far as importing its root document. */ +function importedRoot(events: readonly DurableEvent[]): boolean { + return events.some( + (event) => + event.type === "yield" && + event.description.type === "import_component" && + event.description.name === "__root__", + ); +} + +describe("Tier FE15 — a provider's answer belongs to the execution that captured it", () => { + it("FE15: cancelling a suspended resolution waits for cleanup and revokes the request", function* () { + const A = implementation("Open", "A ran"); + const stream = new InMemoryStream(); + const cleanup: string[] = []; + const reached = withResolvers(); + const retained: ComponentAnswerRequest[] = []; + + yield* scoped(function* () { + const running = yield* spawn(() => + run( + OPEN, + [ + { + evaluation: admits(), + componentAnswers: [ + { + origin: "test://provider", + *install(registrar: ComponentAnswerRegistrar): Operation { + yield* registrar.around(function* (request, next) { + if (request.name !== "Open") { + return yield* next(); + } + retained.push(request); + // Registered before the barrier, so cancellation cannot + // arrive between entering the handler and owning the + // cleanup. + yield* ensure(function* () { + cleanup.push("resolution cleanup"); + }); + reached.resolve(); + yield* suspend(); + return request.claim(A.definition, { key: "Open", revision: "1" }); + }); + }, + }, + ], + }, + ], + stream, + ), + ); + // The resolution has actually entered, so this cancels work in flight + // rather than work that never started. + yield* reached.operation; + yield* running.halt(); + }); + + // Halt waited for the resolution's own cleanup before returning. + expect(cleanup).toEqual(["resolution cleanup"]); + // And the capture never completed, so nothing downstream of it happened: + // no profile was sealed, the root was never imported, no admission was + // decided, and the answer's body never ran. + const events = yield* stream.readAll(); + expect(importedRoot(events)).toBe(false); + expect(admissions(events)).toHaveLength(0); + expect(A.invoked).toEqual([]); + // Cancellation is a teardown like any other: the request this handler was + // handed belongs to an execution that is over. + expect(refusalFrom(retained[0])).toContain("has ended"); + }); + + it("FE15: a request retained past a successful execution states nothing afterwards", function* () { + const A = implementation("Open", "A ran"); + const retained: ComponentAnswerRequest[] = []; + + // The provider keeps the request its handler received. + const output = yield* run(OPEN, [ + { + evaluation: admits(), + componentAnswers: [answerProvider("Open", A.definition, { retain: retained })], + }, + ]); + expect(String(output)).toContain("A ran"); + + // The execution has ended. The request is the object the provider kept, + // and it refuses rather than recording into a run that is over. + expect(refusalFrom(retained[0])).toContain("has ended"); + }); + + it("FE15: a failed execution revokes the request it minted", function* () { + const A = implementation("Open", "A ran"); + const retained: ComponentAnswerRequest[] = []; + + // The third way a run ends. The fragment names something the profile never + // admitted, so the run fails after capture succeeded — and teardown is + // registered before the first installation, so it runs anyway. + const failed = yield* refusal( + run(`\\n'} />\n`, [ + { + evaluation: admits(), + componentAnswers: [answerProvider("Open", A.definition, { retain: retained })], + }, + ]), + ); + + expect(failed).toContain("did not admit"); + expect(A.invoked).toEqual([]); + expect(refusalFrom(retained[0])).toContain("has ended"); + }); + + it("FE15: a losing handler's request states nothing into the execution still running", function* () { + const A = implementation("Open", "A ran"); + const B = implementation("Open", "B ran"); + const late: ComponentAnswerRequest[] = []; + const attempted: string[] = []; + + // Two providers under one name. The first is installed outermost, is + // genuinely asked, keeps the request it was handed and delegates without + // claiming — which is what losing looks like. The second answers, and the + // capture seals what it claimed. + // + // `` runs from the document, *after* the capture and while this + // execution is still live — the window a post-run row cannot reach. So what + // it proves is not that a revoked handle is inert; it is that a live losing + // one changes nothing either. + const output = yield* scoped(function* () { + yield* registerComponents([ + { + name: "Meddle", + origin: "test://meddler", + props: { type: "object", properties: {}, additionalProperties: false }, + // deno-lint-ignore require-yield + *fn(): Operation { + const held = late[0]; + if (held === undefined) { + throw new Error("the losing provider was never asked"); + } + // The resolution of `Open` settled during capture, before the root + // import, and this handler returned before that. The execution is + // still very much alive — so what refuses is the invocation, not + // the teardown. + attempted.push(refused(() => held.claim(A.definition, IDENTITY))); + // The same for the winner's own implementation: there is no live + // request to retag it through either. + attempted.push(refused(() => held.claim(B.definition, IDENTITY))); + return "meddled"; + }, + }, + ]); + return yield* run( + `\n\n\\n'} as="answer" />\n\n\n`, + [ + { + evaluation: admits(), + componentAnswers: [ + { + origin: "test://losing", + *install(registrar: ComponentAnswerRegistrar): Operation { + yield* registrar.around(function* (request, next) { + if (request.name === "Open") { + late.push(request); + } + return yield* next(); + }); + }, + }, + answerProvider("Open", B.definition), + ], + }, + ], + ); + }); + + // The meddling really happened, while the run was live, and neither attempt + // was admitted. + expect(attempted).toEqual(["this resolution has settled", "this resolution has settled"]); + // And the fragment ran what the capture sealed. + expect(String(output)).toContain("B ran"); + expect(B.invoked).toEqual(["B ran"]); + expect(A.invoked).toEqual([]); + }); + + it("FE15: a losing handler cannot claim once it has returned, mid-resolution", function* () { + const A = implementation("Open", "A ran"); + const B = implementation("Open", "B ran"); + const inner: ComponentAnswerRequest[] = []; + const attempted: string[] = []; + const delegated: unknown[] = []; + let outerSettled = false; + let innerSettled = false; + + // The frozen mid-resolution race. The inner provider is asked, keeps its + // request and returns without claiming; the outer provider is *still + // running* the same resolution, and invokes the inner one's request from + // inside its own handler. The window is open, the name is right, the + // execution is live — and the invocation that would have been speaking is + // over. + const output = yield* run(OPEN, [ + { + evaluation: admits(), + componentAnswers: [ + { + // The origin the profile admits: this is the provider whose claim + // has to be the one the capture reconciles. + origin: "test://provider", + *install(registrar: ComponentAnswerRegistrar): Operation { + yield* registrar.around(function* (request, next) { + // Settles at the capture, like any provider: a fragment's own + // import is canonical execution's to answer. + if (request.name !== "Open" || outerSettled) { + return yield* next(); + } + outerSettled = true; + // Delegating first is what an outer replacement does, and it is + // what closes the inner handler. + delegated.push(yield* next()); + const held = inner[0]; + if (held === undefined) { + throw new Error("the inner provider was never asked"); + } + attempted.push(refused(() => held.claim(A.definition, IDENTITY))); + // And the outer handler's own request is still live, so it may + // claim its replacement before returning. + return request.claim(B.definition, { key: "Open", revision: "1" }); + }); + }, + }, + { + origin: "test://inner", + *install(registrar: ComponentAnswerRegistrar): Operation { + yield* registrar.around(function* (request, next) { + if (request.name !== "Open" || innerSettled) { + return yield* next(); + } + innerSettled = true; + // Answers, keeps its request, and claims nothing — so it loses + // the decision the outer handler goes on to make. + inner.push(request); + return A.definition; + }); + }, + }, + ], + }, + ]); + + expect(delegated).toHaveLength(1); + expect(attempted).toEqual(["this resolution has settled"]); + // The outer replacement is what ran, which is the positive half: claiming + // after delegating has to keep working. + expect(String(output)).toContain("B ran"); + expect(B.invoked).toEqual(["B ran"]); + expect(A.invoked).toEqual([]); + }); + + it("FE15: a request from one settled resolution cannot restate that name", function* () { + const A = implementation("Open", "A ran"); + const B = implementation("Other", "B ran"); + const late = implementation("Open", "late ran"); + const stale: ComponentAnswerRequest[] = []; + const attempted: string[] = []; + const carried: unknown[] = []; + + // Two provider-backed names, so the capture opens two resolutions in order. + // The stale request is the one that *answered* the first, used from inside + // the second resolution's live handler. + const output = yield* run( + `\\n\\n'} as="answer" />\n\n\n`, + [ + { + evaluation: { + read: [answerEntry("Open"), answerEntry("Other")], + files: recordedFiles(), + }, + componentAnswers: [ + answerProvider("Open", A.definition, { retain: stale }), + answerProvider("Other", B.definition, { + key: "Other", + whileResolving: () => { + const held = stale[0]; + if (held === undefined) { + throw new Error("the first provider was never asked"); + } + // The request is fixed to the name it was asked, so it cannot + // even address the name being decided — and its own invocation + // is over. + expect(held.name).toBe("Open"); + attempted.push(refused(() => held.claim(late.definition, IDENTITY))); + carried.push(A.definition); + }, + }), + ], + }, + ], + ); + + expect(attempted).toEqual(["this resolution has settled"]); + expect(carried).toEqual([A.definition]); + // Both names ran what their own resolution sealed — which is also the + // positive control that the second provider's own claim, made in the same + // window as the refusal, was admitted. + expect(String(output)).toContain("A ran"); + expect(String(output)).toContain("B ran"); + expect(A.invoked).toEqual(["A ran"]); + expect(B.invoked).toEqual(["B ran"]); + // The substitution reached no fragment. + expect(late.invoked).toEqual([]); + }); + + it("FE15: one provider installation answers two admitted names", function* () { + const A = implementation("Open", "A ran"); + const B = implementation("Other", "B ran"); + const answered: string[] = []; + let installations = 0; + + // The positive control the request contract exists to keep working. One + // installation owns one origin and registers *one* handler for both + // admitted names; the capture opens a resolution per name, and that handler + // is invoked once per resolution with a distinct request each time. An + // installation spent on its first answer would fail here, on the second + // name. + const requests: ComponentAnswerRequest[] = []; + const output = yield* run( + `\\n\\n'} as="answer" />\n\n\n`, + [ + { + evaluation: { + read: [answerEntry("Open"), answerEntry("Other")], + files: recordedFiles(), + }, + componentAnswers: [ + { + origin: "test://provider", + *install(registrar: ComponentAnswerRegistrar): Operation { + installations += 1; + const supplied: Record = { Open: A, Other: B }; + yield* registrar.around(function* (request, next) { + const held = supplied[request.name]; + if (held === undefined || answered.includes(request.name)) { + return yield* next(); + } + answered.push(request.name); + requests.push(request); + return request.claim(held.definition, { + key: request.name, + revision: "1", + }); + }); + }, + }, + ], + }, + ], + ); + + // One installation, one registered handler, two distinct requests — each + // fixed to the name its own invocation was asked. + expect(installations).toBe(1); + expect(answered).toEqual(["Open", "Other"]); + expect(requests).toHaveLength(2); + expect(requests[0]).not.toBe(requests[1]); + expect(requests.map((request) => request.name)).toEqual(["Open", "Other"]); + expect(String(output)).toContain("A ran"); + expect(String(output)).toContain("B ran"); + expect(A.invoked).toEqual(["A ran"]); + expect(B.invoked).toEqual(["B ran"]); + }); + + it("FE15: two executions live at once each run the answer it captured", function* () { + const B = implementation("Open", "B ran"); + const entered = withResolvers(); + const release = withResolvers(); + const first: string[] = []; + + // The first execution's admitted body stops inside itself, so the second + // one below overlaps it rather than following it. Two live executions, two + // owners, two sealed profiles. + const held = { + kind: "function" as const, + name: "Open", + props: { type: "object", properties: {}, additionalProperties: false }, + *fn(): Operation { + first.push("A ran"); + entered.resolve(); + yield* release.operation; + return "A ran"; + }, + }; + + yield* scoped(function* () { + const running = yield* spawn(() => + run(OPEN, [{ evaluation: admits(), componentAnswers: [answerProvider("Open", held)] }]), + ); + yield* entered.operation; + + // Started and finished while the first is suspended inside its fragment. + const second = yield* run(OPEN, [ + { evaluation: admits(), componentAnswers: [answerProvider("Open", B.definition)] }, + ]); + expect(String(second)).toContain("B ran"); + // The overlapping run did not reach into the first: its answer ran once, + // and the first is still holding its own. + expect(B.invoked).toEqual(["B ran"]); + expect(first).toEqual(["A ran"]); + + release.resolve(); + expect(String(yield* running)).toContain("A ran"); + }); + + // Each finished with what it captured, and neither ran the other's. + expect(first).toEqual(["A ran"]); + expect(B.invoked).toEqual(["B ran"]); + }); + + it("FE15: a captured provider body cannot run after the execution ends", function* () { + const A = implementation("Open", "A ran"); + + // The internal seam: this is the two-stage capture canonical execution + // performs, driven directly, because there is no public way to reach a + // sealed profile — which is itself the point. A row that had to add a + // getter to observe this would be proving something about the getter. + const prepared = yield* prepareEvaluationProfile(admits()); + expect(prepared.answered).toEqual([ + { name: "Open", identity: { origin: "test://provider", key: "Open", revision: "1" } }, + ]); + + const answers = new Map([["Open", { definition: A.definition }]]); + const profile = yield* prepared.seal(answers); + const sealed = profile.read[0]?.definition.fn; + if (typeof sealed !== "function") { + throw new Error("the profile sealed no invocable entry"); + } + + // While the execution is live, the sealed definition runs. + const invocation: ComponentInvocation = { hasContent: () => false }; + expect(yield* sealed({}, invocation)).toBe("A ran"); + expect(A.invoked).toEqual(["A ran"]); + + // Teardown. Whatever a provider, a handler or a retained callback still + // holds is bound to an execution that has ended. + profile.revoke(); + + let refused: unknown; + try { + yield* sealed({}, invocation); + } catch (error) { + refused = error; + } + expect(String(refused)).toContain(REVOKED_CAPABILITY); + // The body itself never entered a second time. + expect(A.invoked).toEqual(["A ran"]); + }); + + it("FE15: a later execution without the provider sees nothing the first left", function* () { + const A = implementation("Open", "A ran"); + const first = yield* run(OPEN, [ + { evaluation: admits(), componentAnswers: [answerProvider("Open", A.definition)] }, + ]); + expect(String(first)).toContain("A ran"); + + // The same document and the same profile, with no provider installed. The + // first run's answer is not somewhere the second can find it: a claim leaves + // no registry, no context and no module-level table behind it, so the name + // resolves to nothing at all rather than to what the first run admitted. + const failed = yield* refusal(run(OPEN, [{ evaluation: admits() }])); + expect(failed).toContain("Cannot resolve component: Open"); + expect(A.invoked).toEqual(["A ran"]); + }); + + it("FE15: cancelling an admitted body waits for its cleanup and records no result", function* () { + const stream = new InMemoryStream(); + const cleanup: string[] = []; + const reached = withResolvers(); + const invoked: string[] = []; + + const slow = { + kind: "function" as const, + name: "Open", + props: { type: "object", properties: {}, additionalProperties: false }, + *fn(): Operation { + yield* ensure(function* () { + cleanup.push("body cleanup"); + }); + invoked.push("entered"); + reached.resolve(); + yield* suspend(); + return "A ran"; + }, + }; + + yield* scoped(function* () { + const running = yield* spawn(() => + run( + OPEN, + [{ evaluation: admits(), componentAnswers: [answerProvider("Open", slow)] }], + stream, + ), + ); + yield* reached.operation; + yield* running.halt(); + }); + + // Halt waited for the admitted body's own cleanup. + expect(invoked).toEqual(["entered"]); + expect(cleanup).toEqual(["body cleanup"]); + + const events = yield* stream.readAll(); + // The admission committed — it is the decision, and it precedes what it + // authorized. What did not happen is a terminal: a run that recorded one + // here would resume believing the fragment finished. + expect(admissions(events)).toHaveLength(1); + expect(events.some((event) => event.type === "close" && event.coroutineId === "root")).toBe( + false, + ); + }); +}); diff --git a/packages/core/tests/evaluation-profile.test.ts b/packages/core/tests/evaluation-profile.test.ts new file mode 100644 index 000000000..41d93a486 --- /dev/null +++ b/packages/core/tests/evaluation-profile.test.ts @@ -0,0 +1,439 @@ +/** + * Tier EP — what canonical execution captures from a host's evaluation profile. + * + * `` is public, so any author may write it. What keeps that from + * being a capability is that the ceiling it narrows from was stated by a + * trusted host before a document existed — and that the execution stopped + * reading the host's objects the moment it captured them. + * + * Every row here is about that second half. A host that edits its own tables, + * schemas, headers or roots after installation is editing objects nothing is + * looking at, and a profile that kept a reference instead of a copy would let + * it move the ceiling from inside its own `install()`. + */ + +import { describe, it } from "@executablemd/test-support/bdd"; +import { expect } from "@executablemd/test-support/expect"; +import type { Operation } from "effection"; + +import { prepareEvaluationProfile } from "../src/evaluation-profile.ts"; +import type { + CapabilityEntry, + CapturedProfile, + ComponentAnswerEntry, + FragmentEvaluationInput, + ResolvedAnswer, + ResolvedAnswers, +} from "../src/evaluation-profile.ts"; +import type { Json } from "../src/types.ts"; +import { recordedFiles } from "./support/fragment-files.ts"; + +function entry(overrides: Partial = {}): CapabilityEntry { + const name = overrides.name ?? "File"; + return { + kind: "capability", + name, + identity: { origin: "test://host", key: "File:read", revision: "1" }, + forms: ["self-closing"], + props: { type: "object", properties: {}, additionalProperties: false }, + // The capability follows the name unless a row states otherwise: what these + // rows are about is how a ceiling is captured, not which body runs. + capability: name === "Fetch" ? "fetch" : "file:read", + ...overrides, + }; +} + +/** + * Capture a profile, with the operations every file entry needs supplied. + * + * A host that admits `` states the operations it runs, and a profile + * that does not is refused — EP17 covers that on its own. Every other row is + * about what capture does with a profile a host *can* state, so they all supply + * them and none of them restates the fact. + */ +function* capture(overrides: Partial = {}): Operation { + // Preparation copies and binds; sealing settles the provider-backed names. + // A capability-only profile resolves none, which is why every row here seals + // against no answers at all. + const prepared = yield* prepareEvaluationProfile({ + read: [entry()], + files: recordedFiles(), + ...overrides, + }); + return yield* prepared.seal(new Map()); +} + +/** What capturing this profile refused with, as a string. */ +function* refusal(overrides: Partial): Operation { + try { + yield* capture(overrides); + } catch (error) { + return error instanceof Error ? error.message : String(error); + } + throw new Error("expected the profile to be refused"); +} + +// deno-lint-ignore require-yield +function* emptyBasis(): Operation<{ roots: readonly string[]; current: string }> { + return { roots: [], current: "" }; +} + +describe("Tier EP — a captured profile stops reading the host's objects", () => { + it("EP1: a table the host mutates after capture does not change the profile", function* () { + const read: CapabilityEntry[] = [entry()]; + const captured = yield* capture({ read }); + + read.push(entry({ name: "Added", identity: { origin: "t", key: "Added", revision: "1" } })); + read.length = 0; + + expect(captured.read).toHaveLength(1); + expect(captured.read[0]?.name).toBe("File"); + }); + + it("EP2: a schema the host edits after capture does not change what validates", function* () { + const props: Record = { + type: "object", + properties: { path: { type: "string" } }, + additionalProperties: false, + }; + const captured = yield* capture({ read: [entry({ props })] }); + + props.additionalProperties = true; + const properties = props.properties; + if (typeof properties === "object" && properties !== null && !Array.isArray(properties)) { + properties.widened = { type: "string" }; + } + + expect(captured.read[0]?.props.additionalProperties).toBe(false); + const held = captured.read[0]?.props.properties; + const names = + typeof held === "object" && held !== null && !Array.isArray(held) ? Object.keys(held) : []; + expect(names).toEqual(["path"]); + }); + + it("EP3: an identity the host edits after capture does not change what is compared", function* () { + const identity = { origin: "test://host", key: "Probe", revision: "1" }; + const captured = yield* capture({ read: [entry({ identity })] }); + + identity.revision = "2"; + + expect(captured.read[0]?.identity.revision).toBe("1"); + }); + + it("EP4: request headers and lists the host mutates do not widen the ceiling", function* () { + const headers: Record = { accept: "text/plain" }; + const requests: Record[] = [{ url: "https://api.example.test/one", headers }]; + const captured = yield* capture({ + read: [entry({ name: "Fetch", requests })], + fetchTimeout: 1000, + }); + + headers.authorization = "Bearer widened"; + requests.push({ url: "https://api.example.test/two" }); + + const ceiling = captured.read[0]?.requests ?? []; + expect(ceiling).toHaveLength(1); + expect(Object.keys(ceiling[0]?.headers ?? {})).toEqual(["accept"]); + expect(ceiling[0]?.url).toBe("https://api.example.test/one"); + }); + + it("EP5: the host's resolved timeout is the ceiling, read once", function* () { + const captured = yield* capture({ + read: [entry({ name: "Fetch", requests: [{ url: "https://api.example.test/one" }] })], + fetchTimeout: 2500, + }); + + // The host resolved it when it built the profile. Nothing here reads a + // context, so where preflight later happens cannot change the ceiling. + expect(captured.read[0]?.requests?.[0]?.timeout).toBe(2500); + }); + + it("EP6: a request stating its own timeout outranks the host default", function* () { + const captured = yield* capture({ + read: [ + entry({ + name: "Fetch", + // The prop grammar, which is a duration rather than milliseconds. + requests: [{ url: "https://api.example.test/one", timeout: "100ms" }], + }), + ], + fetchTimeout: 2500, + }); + + expect(captured.read[0]?.requests?.[0]?.timeout).toBe(100); + }); + + it("EP7: one ceiling stated twice is one ceiling", function* () { + const captured = yield* capture({ + read: [ + entry({ + name: "Fetch", + requests: [ + { url: "https://api.example.test/one" }, + { url: "https://api.example.test/one" }, + ], + }), + ], + fetchTimeout: 1000, + }); + + expect(captured.read[0]?.requests).toHaveLength(1); + }); + + it("EP8: ceilings are canonically ordered, so two statements of one set match", function* () { + const one = yield* capture({ + read: [ + entry({ + name: "Fetch", + requests: [{ url: "https://api.example.test/a" }, { url: "https://api.example.test/b" }], + }), + ], + fetchTimeout: 1000, + }); + const other = yield* capture({ + read: [ + entry({ + name: "Fetch", + requests: [{ url: "https://api.example.test/b" }, { url: "https://api.example.test/a" }], + }), + ], + fetchTimeout: 1000, + }); + + expect(one.read[0]?.requests?.map((request) => request.url)).toEqual( + other.read[0]?.requests?.map((request) => request.url), + ); + }); + + it("EP9: each entry keeps its own ceiling rather than a flattened one", function* () { + const captured = yield* capture({ + read: [ + entry({ name: "First", requests: [{ url: "https://api.example.test/first" }] }), + entry({ + name: "Second", + identity: { origin: "test://host", key: "Second", revision: "1" }, + requests: [{ url: "https://api.example.test/second" }], + }), + ], + fetchTimeout: 1000, + }); + + // Flattening them would let the first entry's limit admit the second's + // request: a ceiling belongs to the identity it bounds. + expect(captured.read[0]?.requests?.map((request) => request.url)).toEqual([ + "https://api.example.test/first", + ]); + expect(captured.read[1]?.requests?.map((request) => request.url)).toEqual([ + "https://api.example.test/second", + ]); + }); + + it("EP10: a Workspace answer the host mutates afterwards does not move the basis", function* () { + const roots = ["workspace://one"]; + const captured = yield* capture({ + read: [entry()], + workspace: { + // deno-lint-ignore require-yield + *snapshot() { + return { roots, current: "workspace://one" }; + }, + }, + }); + + const basis = yield* captured.workspace?.snapshot() ?? emptyBasis(); + roots.push("workspace://two"); + + expect(basis.roots).toEqual(["workspace://one"]); + }); + + it("EP11: replacing the host's snapshot method after capture reaches nothing", function* () { + const access = { + // deno-lint-ignore require-yield + *snapshot(): Operation<{ roots: readonly string[]; current: string }> { + return { roots: ["workspace://honest"], current: "workspace://honest" }; + }, + }; + const captured = yield* capture({ read: [entry()], workspace: access }); + + // deno-lint-ignore require-yield + access.snapshot = function* () { + return { roots: ["workspace://substituted"], current: "workspace://substituted" }; + }; + + const basis = yield* captured.workspace?.snapshot() ?? emptyBasis(); + expect(basis.roots).toEqual(["workspace://honest"]); + }); +}); + +describe("Tier EP — a profile a host cannot state", () => { + it("EP12: an entry with no complete identity refuses", function* () { + const attempts: Partial[] = [ + { identity: { origin: "", key: "Probe", revision: "1" } }, + { identity: { origin: "test://host", key: "", revision: "1" } }, + { identity: { origin: "test://host", key: "Probe", revision: "" } }, + ]; + for (const attempt of attempts) { + expect(yield* refusal({ read: [entry(attempt)] })).toContain("complete identity"); + } + }); + + it("EP13: an entry admitted for no form refuses", function* () { + expect(yield* refusal({ read: [entry({ forms: [] })] })).toContain("no authored form"); + }); + + it("EP14: a request entry with no request it may perform refuses", function* () { + expect(yield* refusal({ read: [entry({ name: "Fetch", requests: [] })] })).toContain( + "no request it may perform", + ); + }); + + it("EP15: a profile stating no component at all refuses", function* () { + expect(yield* refusal({ read: [] })).toContain("no component at all"); + }); + + it("EP17: a file entry with no operations behind it refuses", function* () { + // A host that admits `` and states no operations has admitted + // something it cannot perform. Refused at capture — rather than admitted and + // left to fall through to whichever provider a document installed, which is + // exactly the reach this profile exists to remove. + expect( + yield* refusal({ + read: [entry()], + files: undefined, + }), + ).toContain("without stating the filesystem operations"); + }); + + it("EP16: forms are canonically ordered, so two statements of one pair match", function* () { + const one = yield* capture({ + read: [entry({ forms: ["paired", "self-closing"] })], + }); + const other = yield* capture({ + read: [entry({ forms: ["self-closing", "paired"] })], + }); + + expect(one.read[0]?.forms).toEqual(other.read[0]?.forms); + expect(one.read[0]?.forms).toEqual(["self-closing", "paired"]); + }); +}); + +/** + * Tier EP — one name, one implementation. + * + * A provider-backed name is resolved once through the ordinary import chain and + * sealed. So what a profile may say about a name is settled here rather than at + * the lookup: two entries under one name are the two spellings of one + * component, and a second identity for the second spelling would make which + * implementation a fragment reached depend on which table admitted it. + */ +describe("Tier EP — a provider-backed name states one identity", () => { + const OPEN: ComponentAnswerEntry = { + kind: "component-answer", + name: "Open", + identity: { origin: "test://provider", key: "Open", revision: "1" }, + forms: ["self-closing"], + }; + + /** One implementation, as canonical execution hands sealing its answer. */ + function answered(): ResolvedAnswers { + return new Map([ + [ + "Open", + { + definition: { + kind: "function", + name: "Open", + props: { type: "object", properties: {}, additionalProperties: false }, + // deno-lint-ignore require-yield + *fn(): Operation { + return "opened"; + }, + }, + }, + ], + ]); + } + + /** Prepare a profile holding these entries, without sealing it. */ + function prepare(input: Partial) { + return prepareEvaluationProfile({ read: [entry()], files: recordedFiles(), ...input }); + } + + /** What preparing this profile refused with. */ + function* refused(input: Partial): Operation { + try { + yield* prepare(input); + } catch (error) { + return error instanceof Error ? error.message : String(error); + } + throw new Error("expected the profile to be refused"); + } + + it("EP18: one name across disjoint forms and tables is one lookup and one implementation", function* () { + // The self-closing spelling admitted to observe and the paired one to + // mutate: two entries, two forms, two tables, one component. The host says + // the same identity for both, because there is only one thing behind the + // name. + const prepared = yield* prepare({ + read: [OPEN], + write: [{ ...OPEN, forms: ["paired"] }], + }); + + // Asked for once, however many entries hold it: a second lookup would be a + // second chance for the chain to answer differently. + expect(prepared.answered).toEqual([ + { name: "Open", identity: { origin: "test://provider", key: "Open", revision: "1" } }, + ]); + + const profile = yield* prepared.seal(answered()); + const observing = profile.read[0]; + const mutating = profile.write[0]; + expect(observing?.forms).toEqual(["self-closing"]); + expect(mutating?.forms).toEqual(["paired"]); + // One sealed implementation, shared. Two guards over one answer would be + // two lifetimes for one implementation, and which of them a fragment + // reached would depend on which table admitted it. + expect(observing?.definition).toBe(mutating?.definition); + expect(observing?.props).toBe(mutating?.props); + }); + + it("EP19: a second identity for one name refuses, naming both", function* () { + const cases: readonly (readonly [string, Partial])[] = [ + [ + "within one table", + { read: [OPEN, { ...OPEN, identity: { ...OPEN.identity, revision: "2" } }] }, + ], + [ + "across the two tables", + { + read: [OPEN], + write: [{ ...OPEN, forms: ["paired"], identity: { ...OPEN.identity, key: "Other" } }], + }, + ], + [ + "under another origin", + { read: [OPEN, { ...OPEN, identity: { ...OPEN.identity, origin: "test://other" } }] }, + ], + ]; + + for (const [where, input] of cases) { + const failed = yield* refused(input); + // Refused rather than resolved by position: a last-stated identity + // winning would be a ceiling decided by the order a host assembled its + // tables in. + expect([where, failed.includes("One name states one identity")]).toEqual([where, true]); + expect([where, failed.includes("test://provider#Open@1")]).toEqual([where, true]); + } + }); + + it("EP20: one name held as both a capability and an answer refuses", function* () { + // The same ambiguity with the sharper edge: canonical core would supply one + // body and the import chain the other, which are different grants under one + // spelling. + const failed = yield* refused({ + read: [entry({ name: "Open", forms: ["self-closing"] }), { ...OPEN, forms: ["paired"] }], + }); + + expect(failed).toContain("different grants"); + }); +}); diff --git a/packages/core/tests/fetch-component.test.ts b/packages/core/tests/fetch-component.test.ts index d9609b47d..4c8a2e910 100644 --- a/packages/core/tests/fetch-component.test.ts +++ b/packages/core/tests/fetch-component.test.ts @@ -1,5 +1,5 @@ /** - * Tier FE — `` (spec §6.18). + * Tier FT — `` (spec §6.18). * * The component is a boundary between three things that must not blur: what a * document is allowed to ask for, who performs the request, and what is kept @@ -215,7 +215,7 @@ function partial(events: DurableEvent[]): InMemoryStream { ); } -describe("Tier FE — what a request has to be before it is sent", () => { +describe("Tier FT — what a request has to be before it is sent", () => { beforeAll(() => useTempFileCompiler()); const REFUSED: Array<[string, string]> = [ @@ -239,7 +239,7 @@ describe("Tier FE — what a request has to be before it is sent", () => { ]; for (const [what, element] of REFUSED) { - it(`FE1: refuses ${what} before any request`, function* () { + it(`FT1: refuses ${what} before any request`, function* () { const workspace = yield* useWorkspace(); const probe = yield* useAnswer({ status: 200 }); @@ -251,7 +251,7 @@ describe("Tier FE — what a request has to be before it is sent", () => { }); } - it("FE2: defaults to GET and sends exactly the normalized request", function* () { + it("FT2: defaults to GET and sends exactly the normalized request", function* () { const workspace = yield* useWorkspace(); const probe = yield* useAnswer({ status: 200, body: "ok" }); @@ -268,7 +268,7 @@ describe("Tier FE — what a request has to be before it is sent", () => { ]); }); - it("FE3: accepts HEAD and sends it verbatim", function* () { + it("FT3: accepts HEAD and sends it verbatim", function* () { const workspace = yield* useWorkspace(); const probe = yield* useAnswer({ status: 200 }); @@ -277,7 +277,7 @@ describe("Tier FE — what a request has to be before it is sent", () => { expect(probe.performed[0]?.init?.method).toBe("HEAD"); }); - it("FE4: orders header names lexicographically and leaves values alone", function* () { + it("FT4: orders header names lexicographically and leaves values alone", function* () { const workspace = yield* useWorkspace(); const probe = yield* useAnswer({ status: 200 }); @@ -296,7 +296,7 @@ describe("Tier FE — what a request has to be before it is sent", () => { }); }); -describe("Tier FE — what bounds the request", () => { +describe("Tier FT — what bounds the request", () => { beforeAll(() => useTempFileCompiler()); const ACCEPTED: Array<[string, number]> = [ @@ -307,7 +307,7 @@ describe("Tier FE — what bounds the request", () => { ]; for (const [spelling, ms] of ACCEPTED) { - it(`FE5: sends ${spelling} as ${ms}ms`, function* () { + it(`FT5: sends ${spelling} as ${ms}ms`, function* () { const workspace = yield* useWorkspace(); const probe = yield* useAnswer({ status: 200 }); @@ -323,7 +323,7 @@ describe("Tier FE — what bounds the request", () => { const REJECTED = ["", " ", "0", "0s", "-1", "1.5s", "Infinity", "NaN", "abc", "5x", "1e3"]; for (const spelling of REJECTED) { - it(`FE6: refuses the timeout ${JSON.stringify(spelling)} before any request`, function* () { + it(`FT6: refuses the timeout ${JSON.stringify(spelling)} before any request`, function* () { const workspace = yield* useWorkspace(); const probe = yield* useAnswer({ status: 200 }); @@ -337,7 +337,7 @@ describe("Tier FE — what bounds the request", () => { }); } - it("FE7: an explicit timeout outranks the contextual Fetch default", function* () { + it("FT7: an explicit timeout outranks the contextual Fetch default", function* () { const workspace = yield* useWorkspace(); const probe = yield* scoped(function* () { @@ -350,7 +350,7 @@ describe("Tier FE — what bounds the request", () => { expect(probe.performed[0]?.init?.timeout).toBe(1_000); }); - it("FE8: the contextual Fetch default is what an unbounded element carries", function* () { + it("FT8: the contextual Fetch default is what an unbounded element carries", function* () { const workspace = yield* useWorkspace(); const probe = yield* scoped(function* () { @@ -363,7 +363,7 @@ describe("Tier FE — what bounds the request", () => { expect(probe.performed[0]?.init?.timeout).toBe(9_000); }); - it("FE9: no bound at all is an absent field rather than a number", function* () { + it("FT9: no bound at all is an absent field rather than a number", function* () { const workspace = yield* useWorkspace(); const probe = yield* useAnswer({ status: 200 }); @@ -379,7 +379,7 @@ describe("Tier FE — what bounds the request", () => { }); }); -describe("Tier FE — the response a document keeps", () => { +describe("Tier FT — the response a document keeps", () => { beforeAll(() => useTempFileCompiler()); const MIXED: Answer = { @@ -403,7 +403,7 @@ describe("Tier FE — the response a document keeps", () => { "BINDING {shape}", ); - it("FE10: binds one canonical JSON value", function* () { + it("FT10: binds one canonical JSON value", function* () { const workspace = yield* useWorkspace(); yield* useAnswer(MIXED); @@ -415,7 +415,7 @@ describe("Tier FE — the response a document keeps", () => { ); }); - it("FE11: retains the same value it bound", function* () { + it("FT11: retains the same value it bound", function* () { const workspace = yield* useWorkspace(); yield* useAnswer(MIXED); @@ -430,7 +430,7 @@ describe("Tier FE — the response a document keeps", () => { }); }); - it("FE12: a provider that rewrites its headers afterwards changes nothing", function* () { + it("FT12: a provider that rewrites its headers afterwards changes nothing", function* () { const workspace = yield* useWorkspace(); yield* useAnswer({ ...MIXED, mutatesAfterRead: true }); @@ -441,7 +441,7 @@ describe("Tier FE — the response a document keeps", () => { expect(JSON.stringify(result.events)).not.toContain("x-rewritten"); }); - it("FE13: refuses a provider that cannot enumerate its headers", function* () { + it("FT13: refuses a provider that cannot enumerate its headers", function* () { const workspace = yield* useWorkspace(); yield* useAnswer({ status: 200, opaqueHeaders: true, body: "ignored" }); @@ -451,7 +451,7 @@ describe("Tier FE — the response a document keeps", () => { expect(committed(result.events)).toHaveLength(0); }); - it("FE14: GET reads the body exactly once; HEAD does not read it at all", function* () { + it("FT14: GET reads the body exactly once; HEAD does not read it at all", function* () { const workspace = yield* useWorkspace(); const reading = yield* scoped(function* () { @@ -484,10 +484,10 @@ describe("Tier FE — the response a document keeps", () => { }); }); -describe("Tier FE — what a status means", () => { +describe("Tier FT — what a status means", () => { beforeAll(() => useTempFileCompiler()); - it("FE15: a captured non-2xx binds the same shape as a captured 2xx", function* () { + it("FT15: a captured non-2xx binds the same shape as a captured 2xx", function* () { const workspace = yield* useWorkspace(); yield* useAnswer({ status: 404, headers: [["content-type", "text/plain"]], body: "missing" }); @@ -512,7 +512,7 @@ describe("Tier FE — what a status means", () => { expect(result.output).toContain("AFTER"); }); - it("FE16: an uncaptured 2xx succeeds and renders nothing of its own", function* () { + it("FT16: an uncaptured 2xx succeeds and renders nothing of its own", function* () { const workspace = yield* useWorkspace(); yield* useAnswer({ status: 204, headers: [["x-marker", "hidden"]], body: "not rendered" }); @@ -525,7 +525,7 @@ describe("Tier FE — what a status means", () => { expect(result.output).not.toContain("204"); }); - it("FE17: an uncaptured non-2xx records the response and stops the document", function* () { + it("FT17: an uncaptured non-2xx records the response and stops the document", function* () { const workspace = yield* useWorkspace(); const marker = join(workspace, "ran.txt"); yield* useAnswer({ status: 500, headers: [["x-mark", "kept"]], body: "server error" }); @@ -552,7 +552,7 @@ describe("Tier FE — what a status means", () => { }); }); -describe("Tier FE — the binding-mode seam", () => { +describe("Tier FT — the binding-mode seam", () => { /** * The engine's answer to "will what I return be captured?" and nothing else. * Driven through `expandSegments` rather than a document, because what is @@ -584,7 +584,7 @@ describe("Tier FE — the binding-mode seam", () => { return { Report: definition }; } - it("FE18: reports true only where `as` was written", function* () { + it("FT18: reports true only where `as` was written", function* () { const reported: string[] = []; const segments: Segment[] = yield* expandAll( '\n\n', @@ -600,7 +600,7 @@ describe("Tier FE — the binding-mode seam", () => { expect(rendered).not.toContain("[sibling:true]"); }); - it("FE19: a nested invocation answers for itself, not for its caller", function* () { + it("FT19: a nested invocation answers for itself, not for its caller", function* () { const reported: string[] = []; yield* expandAll( '\n', @@ -613,7 +613,7 @@ describe("Tier FE — the binding-mode seam", () => { expect(reported).toEqual(["nested:false", "nested:true"]); }); - it("FE20: two invocations that are live at once answer independently", function* () { + it("FT20: two invocations that are live at once answer independently", function* () { const reported: string[] = []; const arrived: string[] = []; const both = withResolvers(); @@ -649,10 +649,10 @@ describe("Tier FE — the binding-mode seam", () => { }); }); -describe("Tier FE — a failure never becomes data", () => { +describe("Tier FT — a failure never becomes data", () => { beforeAll(() => useTempFileCompiler()); - it("FE21: a transport failure binds nothing and records no result", function* () { + it("FT21: a transport failure binds nothing and records no result", function* () { const workspace = yield* useWorkspace(); yield* API.Fetch.around( { @@ -671,7 +671,7 @@ describe("Tier FE — a failure never becomes data", () => { expect(committed(result.events)).toHaveLength(0); }); - it("FE22: a body-read failure binds nothing, even though a status arrived", function* () { + it("FT22: a body-read failure binds nothing, even though a status arrived", function* () { const workspace = yield* useWorkspace(); yield* useAnswer({ status: 200, headers: [["x-mark", "seen"]], bodyFails: true }); @@ -684,7 +684,7 @@ describe("Tier FE — a failure never becomes data", () => { expect(committed(result.events)).toHaveLength(0); }); - it("FE23: a timeout binds nothing", function* () { + it("FT23: a timeout binds nothing", function* () { const workspace = yield* useWorkspace(); yield* API.Fetch.around( { @@ -706,7 +706,7 @@ describe("Tier FE — a failure never becomes data", () => { expect(committed(result.events)).toHaveLength(0); }); - it("FE24: cancelling the owner tears the provider down in both phases", function* () { + it("FT24: cancelling the owner tears the provider down in both phases", function* () { for (const phase of ["request", "body"]) { const workspace = yield* useWorkspace(); const timeline: string[] = []; @@ -758,7 +758,7 @@ describe("Tier FE — a failure never becomes data", () => { }); }); -describe("Tier FE — who is allowed to perform the request", () => { +describe("Tier FT — who is allowed to perform the request", () => { beforeAll(() => useTempFileCompiler()); /** A trusted host that refuses one destination and performs nothing. */ @@ -775,7 +775,7 @@ describe("Tier FE — who is allowed to perform the request", () => { ); } - it("FE25: ordinary middleware may observe and delegate, and still cannot widen", function* () { + it("FT25: ordinary middleware may observe and delegate, and still cannot widen", function* () { const workspace = yield* useWorkspace(); const performed: string[] = []; const observed: string[] = []; @@ -796,7 +796,7 @@ describe("Tier FE — who is allowed to perform the request", () => { expect(committed(result.events)).toHaveLength(0); }); - it("FE26: a synthetic answer is substitution, not a request the host performed", function* () { + it("FT26: a synthetic answer is substitution, not a request the host performed", function* () { const workspace = yield* useWorkspace(); const performed: string[] = []; @@ -830,7 +830,7 @@ describe("Tier FE — who is allowed to perform the request", () => { expect(performed).toEqual([]); }); - it("FE27: eval's own fetch crosses the same ceiling", function* () { + it("FT27: eval's own fetch crosses the same ceiling", function* () { const workspace = yield* useWorkspace(); const performed: string[] = []; @@ -851,7 +851,7 @@ describe("Tier FE — who is allowed to perform the request", () => { expect(result.failure?.message).toContain("does not reach that destination"); }); - it("FE28: a repository Fetch shadows core and acquires no authority from the name", function* () { + it("FT28: a repository Fetch shadows core and acquires no authority from the name", function* () { const workspace = yield* useWorkspace(); const components = join(workspace, "components"); yield* ensureDir(components); @@ -902,7 +902,7 @@ describe("Tier FE — who is allowed to perform the request", () => { }); }); -describe("Tier FE — what a later run finds", () => { +describe("Tier FT — what a later run finds", () => { beforeAll(() => useTempFileCompiler()); const REPLAYED = doc( @@ -915,7 +915,7 @@ describe("Tier FE — what a later run finds", () => { "BINDING {shape}", ); - it("FE29: a partial replay restores the response and performs no second request", function* () { + it("FT29: a partial replay restores the response and performs no second request", function* () { const workspace = yield* useWorkspace(); const probe = yield* useAnswer({ status: 200, @@ -937,7 +937,7 @@ describe("Tier FE — what a later run finds", () => { expect(restored).toEqual(committed); }); - it("FE30: the committed event names the expansion and where it was written", function* () { + it("FT30: the committed event names the expansion and where it was written", function* () { const workspace = yield* useWorkspace(); yield* useAnswer({ status: 200, body: "" }); @@ -954,7 +954,7 @@ describe("Tier FE — what a later run finds", () => { }); }); - it("FE31: an interruption before the commit leaves no record, and one retry commits one", function* () { + it("FT31: an interruption before the commit leaves no record, and one retry commits one", function* () { const workspace = yield* useWorkspace(); const probe = yield* useAnswer({ status: 200, body: "once" }); const blocked = new InMemoryStream(); @@ -996,7 +996,7 @@ describe("Tier FE — what a later run finds", () => { }); }); -describe("Tier FE — the secret gate covers the whole event", () => { +describe("Tier FT — the secret gate covers the whole event", () => { beforeAll(() => useTempFileCompiler()); const ALPHABET = "abcdefghijklmnopqrstuvwxyz0123456789"; @@ -1016,7 +1016,7 @@ describe("Tier FE — the secret gate covers the whole event", () => { }); } - it("FE32: the scanner sees the request and the response in one event", function* () { + it("FT32: the scanner sees the request and the response in one event", function* () { const workspace = yield* useWorkspace(); const scanned: string[] = []; @@ -1111,7 +1111,7 @@ describe("Tier FE — the secret gate covers the whole event", () => { return `${outcome.run.output}${outcome.run.failure?.message ?? ""}`; } - it("FE33: a credential in the request refuses the append and binds nothing", function* () { + it("FT33: a credential in the request refuses the append and binds nothing", function* () { const outcome = yield* refused( ``, { status: 200, body: "" }, @@ -1123,7 +1123,7 @@ describe("Tier FE — the secret gate covers the whole event", () => { expect(outcome.run.output).not.toContain(CANARY); }); - it("FE34: a credential in the response refuses the append and binds nothing", function* () { + it("FT34: a credential in the response refuses the append and binds nothing", function* () { const outcome = yield* refused(``, { status: 200, body: `token ${CANARY}`, @@ -1140,10 +1140,10 @@ describe("Tier FE — the secret gate covers the whole event", () => { } }); -describe("Tier FE — invocations stay independent", () => { +describe("Tier FT — invocations stay independent", () => { beforeAll(() => useTempFileCompiler()); - it("FE35: cancelling one request leaves the other's response untouched", function* () { + it("FT35: cancelling one request leaves the other's response untouched", function* () { const first = yield* useWorkspace(); const second = yield* useWorkspace(); const timeline: string[] = []; diff --git a/packages/core/tests/generated-xmd.test.ts b/packages/core/tests/generated-xmd.test.ts index 05463afc4..84c22f469 100644 --- a/packages/core/tests/generated-xmd.test.ts +++ b/packages/core/tests/generated-xmd.test.ts @@ -68,8 +68,9 @@ import type { GeneratedObservationResult, GeneratedObservationValue, GeneratedXmdRequest, + RetainedFragmentIdentity, } from "../host.ts"; -import type { FunctionComponentDefinition, Json } from "../src/types.ts"; +import type { FunctionComponentDefinition, Json, JsonObject } from "../src/types.ts"; const ROOT_PATH = "workflows/agent.md"; const ROOT_SOURCE = "The host ran a generated fragment.\n"; @@ -79,6 +80,17 @@ const ROOTS = ["workspace://primary", "workspace://secondary"]; /** A root the run retains after the admission — its own legitimate progress. */ const ADVANCED = "workspace://advanced"; +/** + * One host-stated identity, in the shape a run retains. + * + * `component-answer`, because everything a test admits here is an + * implementation the host holds rather than an operation core supplies the body + * for — which is the distinction the kind exists to keep. + */ +function hostIdentity(origin: string, key: string, revision = "1"): RetainedFragmentIdentity { + return { kind: "component-answer", origin, key, revision }; +} + /** The one host observation component the tests admit beside ``. */ const PROBE: FunctionComponentDefinition = { kind: "function", @@ -91,7 +103,7 @@ const PROBE: FunctionComponentDefinition = { }; function probe(): GeneratedObservation { - return pinnedComponent("Probe", "test://probe", PROBE); + return pinnedComponent("Probe", hostIdentity("test://probe", "Probe"), PROBE); } function useWorkspace(): Operation { @@ -304,23 +316,34 @@ describe("Tier GX — the trusted-host seam", () => { throw new Error("the run recorded no generated-XMD admission"); } expect(admission.description.name).toBe("generated:turn-1"); + // The version-2 shape: tagged, and carrying the Workspace basis as one + // member rather than two top-level ones, because a host may have none. + // The identity is the closed structural record rather than a spelling: four + // members a reader can compare one at a time, so an admission that moved + // says which part moved. + const PROBE_IDENTITY = { + kind: "component-answer", + origin: "test://probe", + key: "Probe", + revision: "1", + }; expect(admission.description.input).toMatchObject({ - roots: ROOTS, - selectedRoot: ROOTS[0], - allowed: [{ name: "Probe", identity: "test://probe" }], + version: 2, + workspace: { roots: ROOTS, selectedRoot: ROOTS[0] }, + allowed: [{ name: "Probe", identity: PROBE_IDENTITY }], }); expect(admission.result).toMatchObject({ status: "ok", value: { decision: "admitted", source: "\n", - named: [{ name: "Probe", identity: "test://probe" }], + named: [{ name: "Probe", identity: PROBE_IDENTITY }], // Retained in the result as well as the input, because durable replay // matches an effect by type and name and never compares a description. policy: { - roots: ROOTS, - selectedRoot: ROOTS[0], - allowed: [{ name: "Probe", identity: "test://probe" }], + version: 2, + workspace: { roots: ROOTS, selectedRoot: ROOTS[0] }, + allowed: [{ name: "Probe", identity: PROBE_IDENTITY }], requests: [], }, }, @@ -666,6 +689,98 @@ describe("Tier GX — the request a generated fragment may perform", () => { it("GX14: admitting Fetch without a ceiling is refused before any evaluation", function* () { expect(() => pinnedFetch([])).toThrow(); }); + + /** + * Two identities that differ only in where a space falls. + * + * A ceiling belongs to the identity that states it, and the run keys its own + * ceiling table by that identity. A key built by joining the four terms with + * a separator is only as unique as the separator is illegal — and nothing + * makes a space illegal in an origin or a component key. These two are the + * smallest pair that collides under a space-joined key and not under the + * terms themselves, so a table that shared one entry's limit with another + * fails here rather than somewhere a host would have to notice. + */ + const SPACED_ONE: RetainedFragmentIdentity = { + kind: "capability", + origin: "test://host a", + key: "b", + revision: "1", + }; + const SPACED_OTHER: RetainedFragmentIdentity = { + kind: "capability", + origin: "test://host", + key: "a b", + revision: "1", + }; + + /** One admitted read that states a ceiling and records what it was asked. */ + function bounded( + name: string, + identity: RetainedFragmentIdentity, + url: string, + performed: string[], + ): GeneratedObservation { + return { + name, + identity, + requests: [{ url }], + definition: { + kind: "function", + name, + props: { + type: "object", + properties: { url: { type: "string" } }, + required: ["url"], + additionalProperties: false, + }, + // deno-lint-ignore require-yield + *fn(props: Record): Operation { + performed.push(`${name} ${String(props.url)}`); + return name; + }, + }, + }; + } + + it("GX14b: two identities that collide under a joined key keep their own ceilings", function* () { + const performed: string[] = []; + const attempt = yield* evaluate( + request(`\n\n\n`, [ + bounded("One", SPACED_ONE, URL_ONE, performed), + bounded("Other", SPACED_OTHER, URL_TWO, performed), + ]), + ); + + // Each element performed the request its own entry stated. A shared key + // would have left one of the two holding the other's limit, and one of + // these two admitted requests would have been refused. + expect(attempt.failure).toBe(undefined); + expect(performed).toEqual([`One ${URL_ONE}`, `Other ${URL_TWO}`]); + // And the retained policy holds both limits rather than one of them twice. + const policy = recordedPolicy(admittedFragments(attempt.events)[0]); + expect(isRecord(policy) ? policy.requests : undefined).toEqual([ + { url: URL_ONE, method: "GET", headers: {} }, + { url: URL_TWO, method: "GET", headers: {} }, + ]); + }); + + it("GX14c: one entry's ceiling never admits the other's request", function* () { + const performed: string[] = []; + const attempt = yield* evaluate( + request(`\n`, [ + bounded("One", SPACED_ONE, URL_ONE, performed), + bounded("Other", SPACED_OTHER, URL_TWO, performed), + ]), + ); + + // The negative half of the pair above. `` may perform exactly what + // its own entry stated, and the other entry's limit is not its own however + // the two identities are spelled. + expect(attempt.failure).toContain("did not admit"); + expect(admittedFragments(attempt.events)).toHaveLength(0); + expect(performed).toEqual([]); + }); }); describe("Tier GX — what the run keeps", () => { @@ -810,7 +925,7 @@ describe("Tier GX — nested generated effects belong to the owning expansion", function durableWrite(executed: string[]): GeneratedMutation { return pinnedMutation( "Write", - "test://durable-write", + hostIdentity("test://durable-write", "Write"), { kind: "function", name: "Write", @@ -1027,6 +1142,67 @@ describe("Tier GX — a malformed generated request reports its class, not itsel }); } + /** Every incoherent Workspace basis a host can state, and what is wrong. */ + const INCOHERENT: readonly (readonly [string, Partial])[] = [ + ["roots without a selected root", { workspaceRoots: ROOTS, selectedRoot: undefined }], + ["a selected root without roots", { workspaceRoots: undefined, selectedRoot: ROOTS[0] }], + ["no retained root at all", { workspaceRoots: [], selectedRoot: "" }], + ["an empty retained root", { workspaceRoots: [""], selectedRoot: "" }], + ["one retained root twice", { workspaceRoots: [ROOTS[0], ROOTS[0]], selectedRoot: ROOTS[0] }], + [ + "a selected root it does not retain", + { + workspaceRoots: ROOTS, + selectedRoot: "workspace://never-retained", + }, + ], + ]; + + /** The probe under its admitted identity, counting live invocations. */ + function countingProbe(performed: string[]): GeneratedObservation { + return pinnedComponent("Probe", hostIdentity("test://probe", "Probe"), { + kind: "function", + name: "Probe", + props: { type: "object", properties: {}, additionalProperties: false }, + // deno-lint-ignore require-yield + *fn(): Operation { + performed.push("probed"); + return "probed"; + }, + }); + } + + for (const [what, basis] of INCOHERENT) { + it(`GX26b: a host stating ${what} fails before anything is appended`, function* () { + const performed: string[] = []; + const attempt = yield* evaluate({ + ...request("\n", [countingProbe(performed)]), + ...basis, + }); + + // The host's own values, so this is the host's own error — and it happens + // before the candidate is read, so nothing of it is retained. + expect([what, attempt.failure !== undefined]).toEqual([what, true]); + expect([what, admissions(attempt.events).length]).toEqual([what, 0]); + expect([what, performed]).toEqual([what, []]); + }); + } + + it("GX26c: a host stating no Workspace basis at all evaluates", function* () { + // The positive control the six above need, and the ordinary-run shape: a + // host with no Workspace is not a host with a broken one. + const { + workspaceRoots: _roots, + selectedRoot: _selected, + ...withoutBasis + } = request("\n", [probe()]); + + const attempt = yield* evaluate(withoutBasis); + + expect(attempt.failure).toBe(undefined); + expect(attempt.output).toContain("probed"); + }); + it("GX26: a malformed host ceiling fails as the host's own error, before anything is appended", function* () { const transport = yield* useTransport(() => ({ status: 200, body: "body" })); @@ -1062,7 +1238,7 @@ describe("Tier GX — a resumed run is held to the ceilings it was admitted unde * than inferring it from rendered text alone. */ function countedProbe(performed: string[]): GeneratedObservation { - return pinnedComponent("Probe", "test://probe", { + return pinnedComponent("Probe", hostIdentity("test://probe", "Probe"), { kind: "function", name: "Probe", props: { type: "object", properties: {}, additionalProperties: false }, @@ -1074,6 +1250,51 @@ describe("Tier GX — a resumed run is held to the ceilings it was admitted unde }); } + /** + * The identity this run states for the host observation a version-1 record + * named. + * + * A capability, because that is the only arm that existed when untagged + * records were written: the component-answer arm came with this issue, so no + * version-1 string ever described one. + */ + const V1_IDENTITY: RetainedFragmentIdentity = { + kind: "capability", + origin: "test://probe", + key: "Probe", + revision: "1", + }; + + /** + * The exact string a version-1 journal holds for it. + * + * Arbitrary, and that is the point. Under `bb2c1c49` this string was whatever + * the host passed to `pinnedComponent`, retained verbatim — there was no + * spelling rule at all, so nothing recovers `V1_IDENTITY` from it. The + * current entry states it as an alias, which is the only thing that connects + * the two. + */ + const V1_SPELLING = "urn:host:probe/v1"; + + /** The same counting probe, stating the version-1 identity it succeeds. */ + function versionOneProbe(performed: string[]): GeneratedObservation { + return pinnedComponent( + "Probe", + V1_IDENTITY, + { + kind: "function", + name: "Probe", + props: { type: "object", properties: {}, additionalProperties: false }, + // deno-lint-ignore require-yield + *fn(): Operation { + performed.push("probed"); + return "probed"; + }, + }, + [V1_SPELLING], + ); + } + /** * A history left by a run interrupted during preparation: the admission * committed, and nothing after it did. @@ -1137,12 +1358,817 @@ describe("Tier GX — a resumed run is held to the ceilings it was admitted unde return yield* blocked.readAll(); } + /** + * The untagged #369 policy, written out rather than derived. + * + * A run suspended before this build wrote exactly this: no version member, + * and the Workspace basis as two mandatory top-level ones. Deriving it by + * rewriting a record this build produced would prove only that the rewrite + * and the reader agree; a literal is what an older journal actually holds. + */ + const V1_POLICY: JsonObject = Object.freeze({ + allow: ["read"], + roots: [...ROOTS], + selectedRoot: ROOTS[0], + allowed: [{ name: "Probe", identity: V1_SPELLING, forms: ["self-closing", "paired"] }], + requests: [], + }); + + /** One frozen version-1 admission, as an older build committed it. */ + function versionOneAdmission(source: string): DurableEvent { + return { + type: "yield", + coroutineId: "root", + description: { type: "generated_xmd", name: "generated:turn-1", input: V1_POLICY }, + result: { + status: "ok", + value: { + decision: "admitted", + source, + named: [{ name: "Probe", identity: V1_SPELLING, form: "self-closing" }], + policy: V1_POLICY, + }, + }, + }; + } + + /** One frozen version-1 refusal, in the shape that build wrote refusals. */ + function versionOneRefusal(): DurableEvent { + return { + type: "yield", + coroutineId: "root", + description: { type: "generated_xmd", name: "generated:turn-1", input: V1_POLICY }, + result: { status: "ok", value: { decision: "refused", construct: "block" } }, + }; + } + + /** + * A history whose admission is that literal version-1 record. + * + * The events around it are a real run's, because what is under test is the + * record rather than the journal's framing. + */ + function withVersionOne( + events: readonly DurableEvent[], + admission: DurableEvent, + ): DurableEvent[] { + return events.map((event) => + event.type === "yield" && event.description.type === "generated_xmd" ? admission : event, + ); + } + + /** + * The literal version-1 admission a `bb2c1c49` workflow run committed for the + * standard read table. + * + * Written out from what that build actually emitted rather than derived: + * `pinnedFileRead()` retained `@executablemd/core#File:read` — one opaque + * value, no revision in it, no separator convention to read back out — beside + * the untagged policy shape with its two top-level Workspace members. + */ + const V1_STANDARD_POLICY: JsonObject = Object.freeze({ + allow: ["read"], + roots: [...ROOTS], + selectedRoot: ROOTS[0], + allowed: [{ name: "File", identity: "@executablemd/core#File:read", forms: ["self-closing"] }], + requests: [], + }); + + /** That admission, as the event an older journal holds. */ + function versionOneStandard(source: string): DurableEvent { + return { + type: "yield", + coroutineId: "root", + description: { type: "generated_xmd", name: "generated:turn-1", input: V1_STANDARD_POLICY }, + result: { + status: "ok", + value: { + decision: "admitted", + source, + named: [{ name: "File", identity: "@executablemd/core#File:read", form: "self-closing" }], + policy: V1_STANDARD_POLICY, + }, + }, + }; + } + + /** + * One literal version-1 admission, in the untagged shape and holding whatever + * a released build retained for the entry the case is about. + */ + function versionOneRecord( + source: string, + allow: readonly string[], + allowed: readonly JsonObject[], + requests: readonly JsonObject[] = [], + ): DurableEvent { + const policy: JsonObject = { + allow: [...allow], + roots: [...ROOTS], + selectedRoot: ROOTS[0], + allowed: allowed.map((entry) => ({ ...entry })), + requests: requests.map((request) => ({ ...request })), + }; + return { + type: "yield", + coroutineId: "root", + description: { type: "generated_xmd", name: "generated:turn-1", input: policy }, + result: { + status: "ok", + value: { + decision: "admitted", + source, + named: allowed.map((entry) => ({ + name: entry.name, + identity: entry.identity, + form: Array.isArray(entry.forms) ? entry.forms[0] : "self-closing", + })), + policy, + }, + }, + }; + } + + /** + * The workflow host's own ``, at the identity and alias it states. + * + * Built here rather than imported, because what is under test is the seam a + * host reaches: `pinnedMutation` takes the structural identity and the exact + * version-1 string, and this is the string a released workflow run retained. + */ + function dirEntry(executed: string[]): GeneratedMutation { + return pinnedMutation( + "Dir", + { + kind: "capability", + origin: "@executablemd/workflow/composition", + key: "Dir", + revision: "3", + }, + { + kind: "function", + name: "Dir", + props: { + type: "object", + properties: { path: { type: "string" } }, + required: ["path"], + additionalProperties: false, + }, + // deno-lint-ignore require-yield + *fn(props: Record): Operation { + executed.push(`dir:${String(props.path)}`); + return ""; + }, + }, + "paired", + ["@executablemd/workflow/composition/dir-v2#Dir"], + ); + } + + /** + * Every version-1 identity a released build actually retained, and the entry + * that states it as the one it succeeds. + * + * One row per alias rather than one row for the mechanism, because each of + * these is a separate literal assertion: the alias is written out at the + * entry, and a journal holding that exact string has to reconcile to it. + * `` is here too — it is the workflow host's string, not core's, and the + * host-supplied path has to work for the same reason core's does. + */ + const RELEASED_ALIASES: readonly (readonly [ + string, + string, + () => GeneratedXmdRequest, + readonly string[], + readonly JsonObject[], + readonly JsonObject[], + ])[] = [ + [ + "core's Fetch", + `\n`, + () => request(`\n`, [pinnedFetch([ADMITTED_REQUEST])]), + ["read"], + [ + { + name: "Fetch", + identity: "@executablemd/core#Fetch", + forms: ["self-closing", "paired"], + }, + ], + [{ url: URL_ONE, method: "GET", headers: {} }], + ], + [ + "core's read-only File", + `\n`, + () => request(`\n`, [pinnedFileRead()]), + ["read"], + [{ name: "File", identity: "@executablemd/core#File:read", forms: ["self-closing"] }], + [], + ], + [ + "core's paired File", + `from the fragment\n`, + () => + selecting(`from the fragment\n`, [], { + allow: ["write"], + mutations: [pinnedFileWrite()], + }), + ["write"], + [{ name: "File", identity: "@executablemd/core#File:write", forms: ["paired"] }], + [], + ], + [ + "core's File.Delete", + `\n`, + () => + selecting(`\n`, [], { + allow: ["write"], + mutations: [pinnedFileDelete()], + }), + ["write"], + [ + { + name: "File.Delete", + identity: "@executablemd/core#File.Delete", + forms: ["self-closing"], + }, + ], + [], + ], + ]; + + for (const [what, source, candidate, allow, allowed, requests] of RELEASED_ALIASES) { + it(`FE18/GX21z: a version-1 record naming ${what} resumes against its stated alias`, function* () { + const root = yield* useWorkspace(); + yield* writeTextFile(join(root, "notes.md"), "the retained note\n"); + const transport = yield* useTransport(() => ({ status: 200, body: "body" })); + + const first = yield* scoped(function* () { + yield* useWorkspaceFiles(root); + return yield* evaluate(candidate()); + }); + expect([what, first.failure]).toEqual([what, undefined]); + + // The same run, resumed against the record a released build wrote for + // this entry: the opaque string, no revision, the untagged policy shape. + const again = yield* scoped(function* () { + yield* useWorkspaceFiles(root); + return yield* evaluate(candidate(), { + stream: new InMemoryStream( + withVersionOne( + yield* partial(first.events).readAll(), + versionOneRecord(source, allow, allowed, requests), + ), + ), + }); + }); + + expect([what, again.failure]).toEqual([what, undefined]); + // Restored rather than decided a second time. + expect([what, admissions(again.events).length]).toEqual([what, 1]); + expect(transport.performed.length).toBeGreaterThanOrEqual(0); + }); + } + + it("FE18/GX21z: a version-1 record naming the workflow Dir resumes against its stated alias", function* () { + // The host-supplied half of the same contract. `@executablemd/workflow/ + // composition/dir-v2#Dir` is the workflow run's string, stated by the + // workflow entry rather than by core — and a journal holding it reconciles + // for exactly the same reason core's do. + const executed: string[] = []; + const candidate = () => + selecting(`held\n`, [], { + allow: ["write"], + mutations: [dirEntry(executed)], + }); + + const first = yield* evaluate(candidate()); + expect(first.failure).toBe(undefined); + expect(executed).toEqual(["dir:generated"]); + + const again = yield* evaluate(candidate(), { + stream: new InMemoryStream( + withVersionOne( + yield* partial(first.events).readAll(), + versionOneRecord( + `held\n`, + ["write"], + [ + { + name: "Dir", + identity: "@executablemd/workflow/composition/dir-v2#Dir", + forms: ["paired"], + }, + ], + ), + ), + ), + }); + + expect(again.failure).toBe(undefined); + expect(admissions(again.events)).toHaveLength(1); + }); + + it("FE18/GX21z: the standard core admission a released build wrote still resumes", function* () { + // The record is `bb2c1c49`'s own: core's read-only `` retained as the + // string that build committed, under the untagged policy shape. What lets + // it resume is not a spelling rule — there is none to find — but that + // `pinnedFileRead()` states that exact string as the version-1 identity it + // succeeds. + const root = yield* useWorkspace(); + yield* writeTextFile(join(root, "notes.md"), "the retained note\n"); + + const first = yield* scoped(function* () { + yield* useWorkspaceFiles(root); + return yield* evaluate(request(`\n`, [pinnedFileRead()])); + }); + expect(first.failure).toBe(undefined); + + const again = yield* scoped(function* () { + const reads = yield* useWorkspaceFiles(root); + const evaluated = yield* evaluate(request(`\n`, [pinnedFileRead()]), { + stream: new InMemoryStream( + withVersionOne( + yield* partial(first.events).readAll(), + versionOneStandard(`\n`), + ), + ), + }); + return { evaluated, reads: [...reads.performed] }; + }); + + expect(again.evaluated.failure).toBe(undefined); + // Restored rather than decided a second time: one record, carried over from + // the run that wrote it in the older shape. + expect(admissions(again.evaluated.events)).toHaveLength(1); + }); + + it("FE18/GX21z: an entry that states no alias refuses the record that named it", function* () { + // The same released record, against a run whose `` entry does not + // state that string. Reconciliation is an assertion the entry makes, so an + // entry that makes none is a new grant — and the safe answer to "is this + // the thing the record was admitted under" is the one that refuses. + const root = yield* useWorkspace(); + yield* writeTextFile(join(root, "notes.md"), "the retained note\n"); + const first = yield* scoped(function* () { + yield* useWorkspaceFiles(root); + return yield* evaluate(request(`\n`, [pinnedFileRead()])); + }); + + const silent: GeneratedObservation = { ...pinnedFileRead(), legacy: undefined }; + const again = yield* scoped(function* () { + const reads = yield* useWorkspaceFiles(root); + const evaluated = yield* evaluate(request(`\n`, [silent]), { + stream: new InMemoryStream( + withVersionOne( + yield* duringPreparation(first.events).readAll(), + versionOneStandard(`\n`), + ), + ), + }); + return { evaluated, reads: [...reads.performed] }; + }); + + expect(again.evaluated.failure).toContain("admitted under"); + // Ahead of the first generated effect: the refusal is the ceiling's, and + // the admitted read never reached the provider. + expect(again.reads).toEqual([]); + }); + + it("FE18/GX21z: an arbitrary version-1 host identity resumes against its stated alias", function* () { + const first = yield* evaluate(request("\n", [versionOneProbe([])])); + expect(first.output).toContain("probed"); + + // The other half of what version 1 could hold: whatever string the host + // passed to `pinnedComponent`, retained verbatim. `urn:host:probe/v1` + // resembles no structural identity at all, which is why nothing but the + // entry's own stated alias could connect it to what this run admits. + // + // This is also the version-1 positive control for the `GX21y` table below: + // every row there is this record with one thing wrong, so without this each + // of them could be refusing because an untagged record never resumes at all. + const performed: string[] = []; + const again = yield* evaluate(request("\n", [versionOneProbe(performed)]), { + stream: new InMemoryStream( + withVersionOne(yield* partial(first.events).readAll(), versionOneAdmission("\n")), + ), + }); + + expect(again.failure).toBe(undefined); + expect(performed).toEqual(["probed"]); + expect(admissions(again.events)).toHaveLength(1); + }); + + it("FE18/GX21z: a version-1 string no entry lists as its own refuses", function* () { + const first = yield* evaluate(request("\n", [versionOneProbe([])])); + + // Reconciliation is a stated alias, not a waiver. This run's entry lists a + // different string, so the retained one names an entry it does not have. + // + // The probe still counts, so this says the component was not reached rather + // than that no counting probe was admitted. + const performed: string[] = []; + const moved: GeneratedObservation = { + ...versionOneProbe(performed), + legacy: ["urn:host:probe/v0"], + }; + const again = yield* evaluate(request("\n", [moved]), { + stream: new InMemoryStream( + withVersionOne( + yield* duringPreparation(first.events).readAll(), + versionOneAdmission("\n"), + ), + ), + }); + + expect(again.failure).toContain("admitted under"); + expect(performed).toEqual([]); + }); + + it("FE18/GX21z: a version-1 string never describes a component answer", function* () { + const first = yield* evaluate(request("\n", [versionOneProbe([])])); + + // The component-answer arm did not exist when untagged records were + // written, so no version-1 string ever described one. Reading this string as + // though it might would compare a fragment's authority equal to authority it + // never had — even though the origin, the key and the revision all match. + const performed: string[] = []; + const answered: GeneratedObservation = { + ...versionOneProbe(performed), + identity: hostIdentity("test://probe", "Probe"), + }; + const again = yield* evaluate(request("\n", [answered]), { + stream: new InMemoryStream( + withVersionOne( + yield* duringPreparation(first.events).readAll(), + versionOneAdmission("\n"), + ), + ), + }); + + expect(again.failure).toContain("admitted under"); + expect(performed).toEqual([]); + }); + + it("FE18/GX21x: a literal version-1 refusal replays as the refusal it recorded", function* () { + const first = yield* evaluate(request("\n", [probe()])); + + const performed: string[] = []; + const again = yield* evaluate(request("\n", [countedProbe(performed)]), { + stream: new InMemoryStream( + withVersionOne(yield* duringPreparation(first.events).readAll(), versionOneRefusal()), + ), + }); + + // The refusal shape had no version member either, and restoring it means + // refusing again rather than re-deciding a fragment an older build declined. + expect(again.failure).toContain("executable code block"); + expect(performed).toEqual([]); + }); + + /** + * One admission whose policy and result carry whatever members a case wants. + * + * Built as a literal rather than by rewriting a real record, so a case states + * the exact shape it is claiming this build must refuse. + */ + function forgedAdmission(policy: JsonObject, extra: JsonObject = {}): DurableEvent { + return { + type: "yield", + coroutineId: "root", + description: { type: "generated_xmd", name: "generated:turn-1", input: policy }, + result: { + status: "ok", + value: { + decision: "admitted", + source: "\n", + named: [{ name: "Probe", identity: "test://probe", form: "self-closing" }], + policy, + ...extra, + }, + }, + }; + } + + /** + * One version-2 admission whose identities are whatever a case wants. + * + * Version 2 is where the identity is the closed tagged record, so these are + * the cases that say what "closed" means: a value that is not that record is + * refused rather than read as whichever shape it most resembles. + */ + function forgedVersionTwo(identity: Json, namedIdentity: Json = identity): DurableEvent { + const policy: JsonObject = { + version: 2, + allow: ["read"], + workspace: { roots: [...ROOTS], selectedRoot: ROOTS[0] }, + allowed: [{ name: "Probe", identity, forms: ["self-closing", "paired"] }], + requests: [], + }; + return { + type: "yield", + coroutineId: "root", + description: { type: "generated_xmd", name: "generated:turn-1", input: policy }, + result: { + status: "ok", + value: { + version: 2, + decision: "admitted", + source: "\n", + named: [{ name: "Probe", identity: namedIdentity, form: "self-closing" }], + policy, + }, + }, + }; + } + + /** The identity this run actually states for ``, as journal data. */ + const PROBE_RECORD: JsonObject = { + kind: "component-answer", + origin: "test://probe", + key: "Probe", + revision: "1", + }; + + /** + * Every version-2 identity shape this build refuses. + * + * Separate from the version-1 table below because the two need different + * resumes: these records name `PROBE_RECORD`, so the run has to state that + * same component-answer identity, or every row would refuse on the kind + * rather than on what it says it is about. Their positive control is + * `forgedVersionTwo(PROBE_RECORD)`, which resumes under exactly that probe. + */ + const HOSTILE_V2: readonly (readonly [string, DurableEvent])[] = [ + ["a version-2 identity written as a version-1 string", forgedVersionTwo("test://probe")], + [ + "a version-2 identity with no kind", + forgedVersionTwo({ + origin: "test://probe", + key: "Probe", + revision: "1", + }), + ], + [ + "a version-2 identity naming a kind this build does not have", + forgedVersionTwo({ + ...PROBE_RECORD, + kind: "registration", + }), + ], + [ + "a version-2 identity carrying an extra member", + forgedVersionTwo({ + ...PROBE_RECORD, + widened: true, + }), + ], + [ + "a version-2 identity whose revision is not a string", + forgedVersionTwo({ + ...PROBE_RECORD, + revision: 1, + }), + ], + ["a version-2 identity that is an array", forgedVersionTwo(["test://probe", "Probe", "1"])], + ["a version-2 identity that is null", forgedVersionTwo(null)], + // A closed record is closed in both directions: an extra member is refused + // above, and each missing one is refused here. `kind` has its own row, so + // these are the three that carry what the implementation actually is. + [ + "a version-2 identity with no origin", + forgedVersionTwo({ kind: "component-answer", key: "Probe", revision: "1" }), + ], + [ + "a version-2 identity with no key", + forgedVersionTwo({ kind: "component-answer", origin: "test://probe", revision: "1" }), + ], + [ + "a version-2 identity with no revision", + forgedVersionTwo({ kind: "component-answer", origin: "test://probe", key: "Probe" }), + ], + // The kind is compared, not merely parsed. Everything else about this record + // is what this run states, and `capability` is a kind this build has — so a + // row that refuses here is refusing on the kind alone, which is the whole + // reason the kind travels inside the identity rather than beside it. + [ + "a version-2 identity whose kind is the other one", + forgedVersionTwo({ ...PROBE_RECORD, kind: "capability" }), + ], + // One record, two shapes. The policy and the named invocation are read by + // different readers, and a record that satisfied one while contradicting the + // other would be a grant assembled out of two different versions. + // + // The first of these is the row that carries its own weight alone. A + // retained `named` list is *parsed* and never compared — the policy is what + // a continuation is held to — so the version strictness of that parse is the + // only thing standing behind it. Relaxing version 2 to accept a version-1 + // string reddens this row and neither of the others, because those are + // caught by the comparison as well. + [ + "a version-2 policy beside a version-1 named identity", + forgedVersionTwo(PROBE_RECORD, V1_SPELLING), + ], + [ + "a version-1 named identity beside a version-2 policy", + forgedVersionTwo(V1_SPELLING, PROBE_RECORD), + ], + ]; + + /** Every retained shape this build refuses, and what is wrong with each. */ + const HOSTILE: readonly (readonly [string, DurableEvent])[] = [ + ["a version this build does not have", forgedAdmission({ ...V1_POLICY, version: 99 })], + [ + "a version-1 policy carrying a version-2 workspace", + forgedAdmission({ + ...V1_POLICY, + workspace: { roots: [...ROOTS], selectedRoot: ROOTS[0] }, + }), + ], + [ + "a version-2 policy still carrying legacy roots", + forgedAdmission({ + version: 2, + allow: ["read"], + roots: [...ROOTS], + selectedRoot: ROOTS[0], + allowed: V1_POLICY.allowed, + requests: [], + }), + ], + [ + "a policy carrying a member this build does not know", + forgedAdmission({ + ...V1_POLICY, + widened: true, + }), + ], + ["an empty class selection", forgedAdmission({ ...V1_POLICY, allow: [] })], + ["one class selected twice", forgedAdmission({ ...V1_POLICY, allow: ["read", "read"] })], + ["classes out of canonical order", forgedAdmission({ ...V1_POLICY, allow: ["write", "read"] })], + [ + "an allowed entry carrying an extra member", + forgedAdmission({ + ...V1_POLICY, + allowed: [ + { + name: "Probe", + identity: "test://probe", + forms: ["self-closing"], + widened: true, + }, + ], + }), + ], + [ + "an allowed entry with no forms", + forgedAdmission({ + ...V1_POLICY, + allowed: [{ name: "Probe", identity: "test://probe", forms: [] }], + }), + ], + ["a repeated retained root", forgedAdmission({ ...V1_POLICY, roots: [ROOTS[0], ROOTS[0]] })], + ["no retained roots at all", forgedAdmission({ ...V1_POLICY, roots: [], selectedRoot: "" })], + [ + "a selected root the basis does not hold", + forgedAdmission({ + ...V1_POLICY, + selectedRoot: "workspace://never-retained", + }), + ], + [ + "a request carrying a member this build does not know", + forgedAdmission({ + ...V1_POLICY, + requests: [{ url: URL_ONE, method: "GET", headers: {}, widened: true }], + }), + ], + [ + "a request whose timeout is not a number", + forgedAdmission({ + ...V1_POLICY, + requests: [{ url: URL_ONE, method: "GET", headers: {}, timeout: "soon" }], + }), + ], + [ + "a result carrying a member this build does not know", + forgedAdmission(V1_POLICY, { + widened: true, + }), + ], + [ + "a tagged result holding an untagged policy", + { + type: "yield", + coroutineId: "root", + description: { type: "generated_xmd", name: "generated:turn-1", input: V1_POLICY }, + result: { + status: "ok", + value: { + version: 2, + decision: "admitted", + source: "\n", + named: [{ name: "Probe", identity: "test://probe", form: "self-closing" }], + policy: V1_POLICY, + }, + }, + }, + ], + [ + "a named invocation carrying an extra member", + { + type: "yield", + coroutineId: "root", + description: { type: "generated_xmd", name: "generated:turn-1", input: V1_POLICY }, + result: { + status: "ok", + value: { + decision: "admitted", + source: "\n", + named: [ + { + name: "Probe", + identity: "test://probe", + form: "self-closing", + widened: true, + }, + ], + policy: V1_POLICY, + }, + }, + }, + ], + ]; + + for (const [what, admission] of HOSTILE_V2) { + it(`FE18/GX21y: a retained record with ${what} refuses before any effect`, function* () { + // Resumed under the component-answer probe these records name, so a row + // refuses on the defect it is about rather than on a kind that never + // matched. The control below resumes on the unmodified record. + const first = yield* evaluate(request("\n", [probe()])); + + const performed: string[] = []; + const again = yield* evaluate(request("\n", [countedProbe(performed)]), { + stream: new InMemoryStream( + withVersionOne(yield* duringPreparation(first.events).readAll(), admission), + ), + }); + + expect([what, again.failure !== undefined]).toEqual([what, true]); + expect([what, performed]).toEqual([what, []]); + }); + } + + for (const [what, admission] of HOSTILE) { + it(`FE18/GX21y: a retained record with ${what} refuses before any effect`, function* () { + // The version-1 probe, because these rows are `V1_POLICY` with one thing + // wrong: the base has to be a record that *would* resume, or each row + // passes because an untagged record never resumes rather than because of + // the defect it names. `GX21z` is that control. + const first = yield* evaluate(request("\n", [versionOneProbe([])])); + + const performed: string[] = []; + const again = yield* evaluate(request("\n", [versionOneProbe(performed)]), { + stream: new InMemoryStream( + withVersionOne(yield* duringPreparation(first.events).readAll(), admission), + ), + }); + + // A record this build cannot read whole is not a grant. Reading the + // members it recognizes would admit one on terms it never saw. + expect([what, again.failure !== undefined]).toEqual([what, true]); + expect([what, performed]).toEqual([what, []]); + }); + } + + it("FE18/GX21y: the same forged record with the exact identity resumes", function* () { + const first = yield* evaluate(request("\n", [probe()])); + + // The positive control for the structural-identity rows above. Every one of + // them is this record with one part of the identity moved, so this is what + // proves each refuses for its own reason rather than because a forged + // version-2 admission never resumes at all. + const performed: string[] = []; + const again = yield* evaluate(request("\n", [countedProbe(performed)]), { + stream: new InMemoryStream( + withVersionOne(yield* partial(first.events).readAll(), forgedVersionTwo(PROBE_RECORD)), + ), + }); + + expect(again.failure).toBe(undefined); + expect(performed).toEqual(["probed"]); + // Restored rather than decided a second time. + expect(admissions(again.events)).toHaveLength(1); + }); + it("GX21: a changed identity behind the same name refuses before invoking it", function* () { const first = yield* evaluate(request("\n", [probe()])); expect(first.output).toContain("probed"); const again = yield* evaluate( - request("\n", [pinnedComponent("Probe", "test://other", OTHER)]), + request("\n", [ + pinnedComponent("Probe", hostIdentity("test://other", "Probe"), OTHER), + ]), { stream: duringPreparation(first.events) }, ); @@ -1158,7 +2184,9 @@ describe("Tier GX — a resumed run is held to the ceilings it was admitted unde const first = yield* evaluate(request("\n", [probe()])); const again = yield* evaluate( - request("\n", [pinnedComponent("Probe", "test://other", OTHER)]), + request("\n", [ + pinnedComponent("Probe", hostIdentity("test://other", "Probe"), OTHER), + ]), { stream: partial(first.events) }, ); @@ -1301,29 +2329,56 @@ describe("Tier GX — a resumed run is held to the ceilings it was admitted unde expect(observations(again.events)).toHaveLength(1); }); - it("GX24: a changed current source does not change what replay expands", function* () { + it("GX24: a changed current source refuses before it expands anything", function* () { const first = yield* evaluate(request("\n", [probe()])); + const performed: string[] = []; const again = yield* evaluate( - request("\n\nan extra sentence the first run never had.\n", [probe()]), - { stream: partial(first.events) }, + request("\n\nan extra sentence the first run never had.\n", [ + countedProbe(performed), + ]), + { stream: duringPreparation(first.events) }, ); - expect(again.failure).toBe(undefined); - expect(again.output).toBe(first.output); - expect(again.output).not.toContain("an extra sentence"); + // An admission is a decision about one exact fragment. A caller now holding + // a different one is asking for a decision nobody made. + expect(again.failure).toContain("no longer offers"); + // Naming neither fragment: both are generated text. + expect(again.failure).not.toContain("an extra sentence"); + expect(performed).toEqual([]); + expect(String(again.output ?? "")).not.toContain("an extra sentence"); }); - it("GX24b: an unsafe current source does not stop replay of the retained one", function* () { + it("GX24b: an unsafe current source refuses rather than replaying the retained one", function* () { const first = yield* evaluate(request("\n", [probe()])); + const performed: string[] = []; const again = yield* evaluate( - request("\n\n```bash exec\nprintf ran\n```\n", [probe()]), - { stream: partial(first.events) }, + request("\n\n```bash exec\nprintf unsafe-block-executed\n```\n", [ + countedProbe(performed), + ]), + { stream: duringPreparation(first.events) }, ); + expect(again.failure).toContain("no longer offers"); + expect(performed).toEqual([]); + // Neither the block nor its text reaches the journal: the refusal happens + // before the fragment is walked, so the candidate is never retained. + expect(persisted(again.events)).not.toContain("unsafe-block-executed"); + }); + + it("GX24c: the unchanged text resumes from the retained copy", function* () { + const first = yield* evaluate(request("\n", [probe()])); + + // The positive control the two refusals above need: identical text still + // resumes, so what refuses them is the change rather than the comparison. + const again = yield* evaluate(request("\n", [probe()]), { + stream: partial(first.events), + }); + expect(again.failure).toBe(undefined); expect(again.output).toBe(first.output); + expect(admissions(again.events)).toHaveLength(1); }); }); @@ -1366,7 +2421,14 @@ describe("Tier GX — the secret gate covers what is retained", () => { if (admission === undefined || observation === undefined) { throw new Error("the gate was not given both events"); } - for (const marker of ["url-marker", "workspace://primary", `${"@executablemd/core"}#Fetch`]) { + // The identity travels as its four members, so the gate sees the origin and + // the key as themselves rather than as one assembled spelling. + for (const marker of [ + "url-marker", + "workspace://primary", + "@executablemd/core", + '"key":"Fetch"', + ]) { expect(admission).toContain(marker); } expect(observation).toContain("response-body-marker"); @@ -1479,7 +2541,9 @@ describe("Tier WGAC — the pinned read-only File", () => { it("WGAC1: the read identity is not the unconstrained File identity", function* () { // A retained admission resumes only under the identity it was granted with, // and the comparison is on this string. - expect(pinnedFileRead().identity).not.toBe("@executablemd/core#File"); + // Not the unconstrained ``: the read form is a key of its own, so a + // run that later admitted the unconstrained one states a different identity. + expect(pinnedFileRead().identity.key).toBe("File:read"); expect(pinnedFileRead().selfClosing).toBe(true); }); @@ -1489,7 +2553,12 @@ describe("Tier WGAC — the pinned read-only File", () => { expect(deletion.name).toBe("File.Delete"); // What a retained admission is compared against, and what a continuation // that selected the write table is held to. - expect(deletion.identity).toBe("@executablemd/core#File.Delete"); + expect(deletion.identity).toEqual({ + kind: "capability", + origin: "@executablemd/core", + key: "File.Delete", + revision: "2", + }); // One name, one identity: the component answers the self-closing spelling // and refuses the paired one, so this states what the identity is rather // than narrowing it — and stating it is what decides a paired spelling in @@ -1531,8 +2600,10 @@ describe("Tier WGAC — the pinned read-only File", () => { admitted?.type === "yield" && admitted.result.status === "ok" ? admitted.result.value : undefined; - expect(JSON.stringify(named)).toContain(pinnedFileRead().identity); - expect(JSON.stringify(named)).toContain(pinnedFetch([ADMITTED_REQUEST]).identity); + expect(JSON.stringify(named)).toContain(JSON.stringify(pinnedFileRead().identity)); + expect(JSON.stringify(named)).toContain( + JSON.stringify(pinnedFetch([ADMITTED_REQUEST]).identity), + ); expect(values[0]?.value).toBe("the retained note\n"); const response = values[1]?.value; expect(isRecord(response)).toBe(true); @@ -1635,7 +2706,7 @@ const NEST: FunctionComponentDefinition = { }; function nest(form: GeneratedComponentForm = "paired"): GeneratedMutation { - return pinnedMutation("Nest", "test://nest", NEST, form); + return pinnedMutation("Nest", hostIdentity("test://nest", "Nest"), NEST, form); } /** One candidate, with the classes and the write table a run states for it. */ @@ -1708,8 +2779,12 @@ describe("Tier GXC — a selection is not a grant", () => { allow: ["read", "write"], // The read table first, the write table second, host order inside each. allowed: [ - { name: "Probe", identity: "test://probe", forms: ["self-closing", "paired"] }, - { name: "Nest", identity: "test://nest", forms: ["paired"] }, + { + name: "Probe", + identity: hostIdentity("test://probe", "Probe"), + forms: ["self-closing", "paired"], + }, + { name: "Nest", identity: hostIdentity("test://nest", "Nest"), forms: ["paired"] }, ], }); }); @@ -1766,7 +2841,9 @@ describe("Tier GXC — a selection is not a grant", () => { const attempt = yield* evaluate( selecting("\n", [probe()], { allow: ["read", "write"], - mutations: [pinnedMutation("Probe", "test://probe-write", NEST, "paired")], + mutations: [ + pinnedMutation("Probe", hostIdentity("test://probe-write", "Probe"), NEST, "paired"), + ], }), ); @@ -2053,7 +3130,9 @@ describe("Tier GXC — a resumed run is held to its classes and forms", () => { "a replaced write identity", { allow: ["write"], - mutations: [pinnedMutation("File", "test://other-write", NEST, "paired")], + mutations: [ + pinnedMutation("File", hostIdentity("test://other-write", "File"), NEST, "paired"), + ], }, ], ["an added write identity", { allow: ["write"], mutations: [pinnedFileWrite(), nest()] }], @@ -2114,7 +3193,9 @@ describe("Tier GXC — a resumed run is held to its classes and forms", () => { const again = yield* evaluate( selecting("\n", [probe()], { allow: ["read"], - mutations: [pinnedMutation("Nest", "test://replaced", NEST, "paired")], + mutations: [ + pinnedMutation("Nest", hostIdentity("test://replaced", "Nest"), NEST, "paired"), + ], }), { stream: duringPreparation(first.events) }, ); @@ -2153,7 +3234,7 @@ describe("Tier GXC — the authored form survives the public content chain", () /** A read component that reports what the chain answers, so a lie is visible. */ function says(): GeneratedObservation { - return pinnedComponent("Says", "test://says", { + return pinnedComponent("Says", hostIdentity("test://says", "Says"), { kind: "function", name: "Says", props: { type: "object", properties: {}, additionalProperties: false }, @@ -2201,7 +3282,7 @@ describe("Tier GXC — the authored form survives the public content chain", () ]); // And the admission still names the identity and form it was granted for. expect(recordedNames(admittedFragments(attempt.evaluated.events)[0])).toEqual([ - { name: "Says", identity: "test://says", form: "self-closing" }, + { name: "Says", identity: hostIdentity("test://says", "Says"), form: "self-closing" }, { name: "File", identity: pinnedFileRead().identity, form: "self-closing" }, ]); }); @@ -2242,7 +3323,7 @@ describe("Tier GXC — the authored form survives the public content chain", () expect(attempt.files).toEqual(["write:proposed.md"]); expect(yield* readTextFile(join(root, "proposed.md"))).toBe("the fragment wrote this"); expect(recordedNames(admittedFragments(attempt.evaluated.events)[0])).toEqual([ - { name: "Says", identity: "test://says", form: "self-closing" }, + { name: "Says", identity: hostIdentity("test://says", "Says"), form: "self-closing" }, { name: "File", identity: pinnedFileWrite().identity, form: "paired" }, ]); }); diff --git a/packages/core/tests/protected-content.test.ts b/packages/core/tests/protected-content.test.ts new file mode 100644 index 000000000..ffed8de1a --- /dev/null +++ b/packages/core/tests/protected-content.test.ts @@ -0,0 +1,213 @@ +/** + * Tier PCL — the lifetime of a protected body's one content projection. + * + * `PCL` rather than `PC`, which is the Plan component suite's. + * + * These guards are unreachable from a document. Only canonical `` + * consumes a projector and it consumes one once, so a black-box test cannot + * make a second call, cannot retain a callback past a body, and cannot race + * two. A test that could would need a protected component of its own, which is + * a hole in the tier the projector exists inside. So the state machine is a + * module and this is its unit test. + * + * Every row counts how many times the *underlying* operation ran, because that + * is the fact that matters: a lease that refused with the right error while + * still projecting would satisfy an error-shape assertion and none of these. + * The rows assert the error class and the timing rather than the diagnostic + * prose, which is free to improve. + */ + +import { describe, it } from "@executablemd/test-support/bdd"; +import { expect } from "@executablemd/test-support/expect"; +import { sleep, spawn, suspend, withResolvers } from "effection"; +import type { Operation } from "effection"; + +import { ComponentInvocationError } from "../src/invocation-identity.ts"; +import { protectedContentLease } from "../src/protected-content.ts"; +import type { SyntaxReference } from "../src/syntax-reference.ts"; + +/** + * A structurally complete reference that answers nothing interesting. + * + * The lease never reads it — it forwards whatever it was handed — so what + * matters is that this *is* a `SyntaxReference` rather than an object asserted + * into the position of one. A cast here would let the lease's parameter type + * drift without any row noticing. + */ +const REFERENCE: SyntaxReference = { + // deno-lint-ignore require-yield + *symbols(): Operation { + return ""; + }, + // deno-lint-ignore require-yield + *documentation(): Operation { + return ""; + }, + available(): SyntaxReference { + return REFERENCE; + }, +}; + +const PROJECTED = "the exact projected bytes"; + +/** One underlying operation, with a call count and an optional hold. */ +function counted(options: { hold?: () => Operation; fail?: string } = {}) { + const calls: string[] = []; + return { + calls, + *perform(): Operation { + calls.push("performed"); + if (options.hold !== undefined) { + yield* options.hold(); + } + if (options.fail !== undefined) { + throw new Error(options.fail); + } + return PROJECTED; + }, + }; +} + +/** What one call refused with, or `undefined` when it did not refuse. */ +function* refusalOf(operation: Operation): Operation { + try { + yield* operation; + return undefined; + } catch (error) { + return error; + } +} + +describe("Tier PCL — one projection, then nothing", () => { + it("PCL1: the first call projects exactly, the second refuses, and one ran", function* () { + const underlying = counted(); + const lease = protectedContentLease("Evaluate", underlying.perform); + + expect(yield* lease.project(REFERENCE)).toBe(PROJECTED); + + const second = yield* refusalOf(lease.project(REFERENCE)); + expect(second).toBeInstanceOf(ComponentInvocationError); + // The refusal did not merely report: it did not project. + expect(underlying.calls).toHaveLength(1); + }); + + it("PCL2: a closed lease refuses, and the operation is never constructed", function* () { + const underlying = counted(); + const lease = protectedContentLease("Evaluate", underlying.perform); + + // Built while open, closed before it is interpreted. An operation is inert + // until something runs it, so this is the shape a body that returned an + // unstarted operation would leave behind. + const pending = lease.project(REFERENCE); + lease.close(); + + const refused = yield* refusalOf(pending); + expect(refused).toBeInstanceOf(ComponentInvocationError); + expect(underlying.calls).toHaveLength(0); + }); + + it("PCL3: a retained callback called after close refuses", function* () { + const underlying = counted(); + const lease = protectedContentLease("Evaluate", underlying.perform); + // Exactly what a body keeping the callback in a closure, on a returned + // object, or in something it spawned would hold. + const retained = lease.project; + + lease.close(); + + const refused = yield* refusalOf(retained(REFERENCE)); + expect(refused).toBeInstanceOf(ComponentInvocationError); + expect(underlying.calls).toHaveLength(0); + }); + + it("PCL4: a second call while the first is suspended refuses", function* () { + const reached = withResolvers(); + const release = withResolvers(); + const underlying = counted({ + *hold() { + reached.resolve(); + yield* release.operation; + }, + }); + const lease = protectedContentLease("Evaluate", underlying.perform); + + const first = yield* spawn(() => lease.project(REFERENCE)); + // The first call is inside the underlying operation and has not returned. + yield* reached.operation; + + const concurrent = yield* refusalOf(lease.project(REFERENCE)); + expect(concurrent).toBeInstanceOf(ComponentInvocationError); + // Consumption happened before the first call could suspend, so the + // concurrent one found a spent lease rather than an open one. + expect(underlying.calls).toHaveLength(1); + + release.resolve(); + expect(yield* first).toBe(PROJECTED); + expect(underlying.calls).toHaveLength(1); + }); + + it("PCL5: a failed first call leaves the lease spent", function* () { + const underlying = counted({ fail: "the producer refused" }); + const lease = protectedContentLease("Evaluate", underlying.perform); + + const failure = yield* refusalOf(lease.project(REFERENCE)); + expect(failure).toBeInstanceOf(Error); + expect(String(failure)).toContain("the producer refused"); + + // A retry is not a second chance: the content was projected, and what + // failed was the projection. Re-running it would render the document's + // children twice. + const retry = yield* refusalOf(lease.project(REFERENCE)); + expect(retry).toBeInstanceOf(ComponentInvocationError); + expect(underlying.calls).toHaveLength(1); + }); + + it("PCL6: a cancelled first call leaves the lease spent", function* () { + const reached = withResolvers(); + const underlying = counted({ + *hold() { + reached.resolve(); + yield* suspend(); + }, + }); + const lease = protectedContentLease("Evaluate", underlying.perform); + + const first = yield* spawn(() => lease.project(REFERENCE)); + yield* reached.operation; + yield* first.halt(); + + const retry = yield* refusalOf(lease.project(REFERENCE)); + expect(retry).toBeInstanceOf(ComponentInvocationError); + expect(underlying.calls).toHaveLength(1); + }); + + it("PCL7: two leases are independent", function* () { + const one = counted(); + const other = counted(); + const first = protectedContentLease("Evaluate", one.perform); + const second = protectedContentLease("Evaluate", other.perform); + + expect(yield* first.project(REFERENCE)).toBe(PROJECTED); + // Spending one does not spend the other. Module-scoped state shared between + // leases — a file-level flag, a registry keyed by name — would refuse here. + expect(yield* second.project(REFERENCE)).toBe(PROJECTED); + + expect(yield* refusalOf(first.project(REFERENCE))).toBeInstanceOf(ComponentInvocationError); + expect(yield* refusalOf(second.project(REFERENCE))).toBeInstanceOf(ComponentInvocationError); + expect(one.calls).toHaveLength(1); + expect(other.calls).toHaveLength(1); + }); + + it("PCL8: closing twice, and closing after spending, stay refusals", function* () { + const underlying = counted(); + const lease = protectedContentLease("Evaluate", underlying.perform); + + expect(yield* lease.project(REFERENCE)).toBe(PROJECTED); + lease.close(); + lease.close(); + yield* sleep(0); + + expect(yield* refusalOf(lease.project(REFERENCE))).toBeInstanceOf(ComponentInvocationError); + expect(underlying.calls).toHaveLength(1); + }); +}); diff --git a/packages/core/tests/support/answer-provider.ts b/packages/core/tests/support/answer-provider.ts new file mode 100644 index 000000000..7e4b11088 --- /dev/null +++ b/packages/core/tests/support/answer-provider.ts @@ -0,0 +1,244 @@ +/** + * A trusted host's provider for one `component-answer` entry. + * + * The shape a real provider has: it composes `Component.importComponent` + * middleware for its own name, and states an identity for the exact object it + * returns using the request canonical execution minted for that handler + * invocation. It holds no definition the profile could read, because the + * profile arm it backs has nowhere to put one. + * + * Every deviation a row needs is a member here rather than a second provider, + * so what one case changes about the honest one is visible in the case. + */ + +import type { Operation } from "effection"; + +import type { + ComponentAnswerInstallation, + ComponentAnswerRegistrar, + ComponentAnswerRequest, +} from "../../host.ts"; +import type { FunctionComponentDefinition, Json, PropsSchema } from "../../src/types.ts"; + +/** The contract an implementation states when a row does not care what it is. */ +const NO_PROPS: PropsSchema = { type: "object", properties: {}, additionalProperties: false }; + +/** One implementation a provider can answer with, and a log of what ran. */ +export interface Implementation { + readonly definition: FunctionComponentDefinition; + readonly invoked: string[]; +} + +/** One named implementation that records each time it is entered. */ +export function implementation( + name: string, + label: string, + props: PropsSchema = NO_PROPS, +): Implementation { + const invoked: string[] = []; + return { + invoked, + definition: { + kind: "function", + name, + props, + // deno-lint-ignore require-yield + *fn(): Operation { + invoked.push(label); + return label; + }, + }, + }; +} + +/** How one row wants its provider to differ from the honest one. */ +export interface ProviderOptions { + /** The provider's own name for itself; the origin every claim carries. */ + readonly origin?: string; + /** The key and revision this provider states. */ + readonly key?: string; + readonly revision?: string; + /** Every lookup this provider *answered*, in order. */ + readonly lookups?: string[]; + /** Every lookup this provider was asked at all, answered or delegated. */ + readonly asked?: string[]; + /** Return the answer without stating an identity for it at all. */ + readonly unclaimed?: boolean; + /** State the identity, then answer with a copy of the claimed object. */ + readonly copied?: boolean; + /** State the identity, then edit the claimed object. */ + readonly mutated?: boolean; + /** + * Keep answering after the capture, rather than settling. + * + * The refusal case: a generated import is answered by canonical execution, + * so a provider still answering when a fragment resolves its admitted name is + * substituting into the fragment and is refused there. + */ + readonly keepsAnswering?: boolean; + /** + * Keep *claiming* after the capture, rather than settling. + * + * The other half of the same refusal, one step earlier: the resolution this + * provider answered has settled, so there is no open window to state an + * identity into and the claim refuses before the witness is ever consulted. + */ + readonly reclaimsLater?: boolean; + /** + * Delegate first, and answer with this provider's own claimed object. + * + * What an *outer* provider does. The one installed first composes outermost, + * so it sees the chain's answer before deciding, and returning its own + * claimed object is an honest substitution: the identity travels with the + * object the chain finally gives back. + */ + readonly delegatesFirst?: boolean; + /** + * Delegate and answer with whatever came back, claiming nothing. + * + * What a *repeated* bootstrap does. A package entered through an inherited + * layer and again through a local one composes twice, and the outer entry + * cannot claim a second identity for the object the inner one already + * claimed — one implementation states what it is once. So it observes and + * passes the answer through, which is what keeps the layering additive. + */ + readonly observesOnly?: boolean; + /** What delegating answered with, for a row that has to prove it happened. */ + readonly delegated?: unknown[]; + /** + * Run this inside the live resolution, just before claiming. + * + * The one place a row can reach that is *during* an open claim window for + * this name — which is what a race between two resolutions has to be observed + * from, since there is no other way to be inside one. + */ + readonly whileResolving?: (request: ComponentAnswerRequest) => void; + /** + * Answer with a value whose contract reads differently each time it is read. + * + * The check/use gap at execution scale. `[[Get]]` alternates between the + * claimed schema and a substitution; comparing own descriptors, which is how + * the claim is checked, does not run the trap. So a capture that checked the + * answer and then *read it again* to take its copy would seal the + * substitution — and a fragment would be validated against a contract the + * provider never claimed. + */ + readonly alternating?: { readonly substitute: PropsSchema; readonly reads: string[] }; + /** Where this provider leaves each request its handler received, for a row to keep. */ + readonly retain?: ComponentAnswerRequest[]; +} + +/** + * One answer whose `props` alternates between what was claimed and something + * else, read by read. + * + * Only `[[Get]]` is trapped, because that is what taking a copy of a definition + * uses and what comparing own descriptors does not. The schema is the member to + * move, because it is load-bearing: what a fragment's element is validated + * against is the schema on the definition the profile sealed. + */ +function alternatingAnswer( + answer: FunctionComponentDefinition, + substitute: PropsSchema, + reads: string[], +): FunctionComponentDefinition { + return new Proxy(answer, { + get(target, key, receiver) { + if (key !== "props") { + return Reflect.get(target, key, receiver); + } + reads.push(String(key)); + return reads.length % 2 === 1 ? target.props : substitute; + }, + }); +} + +/** + * One honest provider, with whatever one row changed about it. + * + * It answers the capture and settles. That is what a provider is *for*: the + * profile resolves its name once, before the root import, and seals what it + * retained — so a fragment runs the sealed snapshot and the provider has + * nothing left to supply. It still observes every later import and delegates, + * which is the most a handler may do to a generated one. + * + * The middleware answers without delegating while it is answering, which is the + * case that matters: nothing is registered under this name and no file supplies + * it, so the chain's final answer at capture is this provider's implementation + * or the import fails. There is no second place it could come from. + */ +export function answerProvider( + name: string, + answer: FunctionComponentDefinition, + options: ProviderOptions = {}, +): ComponentAnswerInstallation { + const origin = options.origin ?? "test://provider"; + const key = options.key ?? name; + const revision = options.revision ?? "1"; + let settled = false; + return { + origin, + *install(registrar: ComponentAnswerRegistrar): Operation { + yield* registrar.around(function* (request, next) { + const asked = request.name; + if (asked !== name) { + return yield* next(); + } + // Retained *per invocation*, which is what a row keeping a stale one + // is keeping: the installation is never handed over, so there is no + // stable object for a provider to hold instead. + options.retain?.push(request); + options.asked?.push(asked); + if (settled) { + if (options.reclaimsLater) { + // Still trying to *identify* an answer after its resolution + // settled. There is no window open for this name, so the claim + // itself is what refuses. + return request.claim(answer, { key, revision }); + } + if (options.keepsAnswering) { + // Still answering, without claiming. Whatever this returns is not + // the object canonical execution issued for the import that asked, + // so the witness is what refuses. + return answer; + } + return yield* next(); + } + settled = true; + options.lookups?.push(asked); + if (options.observesOnly) { + const answered = yield* next(); + options.delegated?.push(answered); + return answered; + } + if (options.delegatesFirst) { + // An outer provider sees the chain's answer before deciding. What it + // does with it is a row's business; recording it is what lets a row + // say the inner half genuinely answered — and claiming *after* + // delegating is what an outer replacement has to be able to do. + options.delegated?.push(yield* next()); + } + options.whileResolving?.(request); + if (options.unclaimed) { + return answer; + } + const supplied = + options.alternating === undefined + ? answer + : alternatingAnswer(answer, options.alternating.substitute, options.alternating.reads); + const claimed = request.claim(supplied, { key, revision }); + if (options.copied) { + // The outer-replacement case: a handler further out returns its own + // object, so the claim does not travel with the name. + return { ...claimed }; + } + if (options.mutated) { + // The same object, edited after the claim. What was claimed is no + // longer what is there. + Object.assign(claimed, { name: `${name}Substituted` }); + } + return claimed; + }); + }, + }; +} diff --git a/packages/core/tests/support/fragment-files.ts b/packages/core/tests/support/fragment-files.ts new file mode 100644 index 000000000..4e7ec9b0f --- /dev/null +++ b/packages/core/tests/support/fragment-files.ts @@ -0,0 +1,77 @@ +/** + * A recording stand-in for the private filesystem operations a host hands an + * evaluation profile. + * + * Deliberately *not* a Files provider. It is never installed into `API.Files`, + * never composed around, and reachable only through the profile it is passed + * to — which is the point: a row that reads a file through this one has proved + * the read went through the captured operation, because there is no other way + * to reach this object. + */ + +import type { Operation, Result } from "effection"; +import { Err, Ok } from "effection"; + +import type { FragmentFileAccess, FragmentPath, FragmentWrite } from "../../host.ts"; + +/** Every operation one fragment performed, in order, as `verb path`. */ +export interface RecordedFiles extends FragmentFileAccess { + readonly performed: string[]; + readonly entries: Map; +} + +/** + * One recorder, seeded with whatever files a case says already exist. + * + * `workingDirectory` answers a fixed logical root, so a row asserting on a + * recorded path is asserting on what the fragment asked for rather than on + * wherever the test happened to run. + */ +export function recordedFiles( + seed: Record = {}, + options: { readonly hold?: (path: string) => Operation } = {}, +): RecordedFiles { + const entries = new Map(Object.entries(seed)); + const performed: string[] = []; + return { + performed, + entries, + // deno-lint-ignore require-yield + *checkFilePath(input: FragmentPath): Operation> { + performed.push(`check ${input.path}`); + return input.path.startsWith("..") ? Err(new Error("outside")) : Ok(undefined); + }, + *readTextFile(input: FragmentPath): Operation> { + performed.push(`read ${input.path}`); + // A row about cancelling work already inside the fragment holds here: + // the operation has begun and has not answered, which is the only state + // where an admitted effect is in flight. + if (options.hold !== undefined) { + yield* options.hold(input.path); + } + const held = entries.get(input.path); + return held === undefined ? Err(new Error("absent")) : Ok(held); + }, + // deno-lint-ignore require-yield + *writeTextFile(input: FragmentWrite): Operation> { + performed.push(`write ${input.path}`); + entries.set(input.path, input.content); + return Ok(undefined); + }, + // deno-lint-ignore require-yield + *deleteFile(input: FragmentPath): Operation> { + performed.push(`delete ${input.path}`); + entries.delete(input.path); + return Ok(undefined); + }, + // deno-lint-ignore require-yield + *ensureDirectory(input: FragmentPath): Operation> { + performed.push(`ensure ${input.path}`); + return Ok(undefined); + }, + // deno-lint-ignore require-yield + *workingDirectory(): Operation { + return "/workspace"; + }, + }; +} diff --git a/packages/core/tests/syntax-catalog.test.ts b/packages/core/tests/syntax-catalog.test.ts index 711d02ed6..8339704bb 100644 --- a/packages/core/tests/syntax-catalog.test.ts +++ b/packages/core/tests/syntax-catalog.test.ts @@ -30,6 +30,7 @@ import { agentIdentityComponents, CORE_COMPONENT_NAMES, inspectSyntax, + PROTECTED_COMPONENT_NAMES, registerComponents, RESERVED_STRUCTURAL, STRUCTURAL_DECLARATIONS, @@ -38,7 +39,7 @@ import type { CompleteComponentSyntaxEntry, OriginOnlyComponentSyntaxEntry, StructuralSyntaxEntry, - SyntaxCatalog, + SyntaxSymbols, } from "../mod.ts"; import type { IdentityComponent } from "../host.ts"; import type { InvocationForm } from "../mod.ts"; @@ -187,23 +188,23 @@ function catalogFor( tree: Tree, includes: readonly string[], enumeration: Enumeration = {}, -): Operation { +): Operation { return scoped(function* () { yield* useTree(tree, enumeration); return yield* inspectSyntax({ includes }); }); } -function structural(catalog: SyntaxCatalog): readonly StructuralSyntaxEntry[] { +function structural(catalog: SyntaxSymbols): readonly StructuralSyntaxEntry[] { return catalog.categories[0].entries; } -function builtIn(catalog: SyntaxCatalog): readonly CompleteComponentSyntaxEntry[] { +function builtIn(catalog: SyntaxSymbols): readonly CompleteComponentSyntaxEntry[] { return catalog.categories[1].entries; } function userProvided( - catalog: SyntaxCatalog, + catalog: SyntaxSymbols, ): readonly (CompleteComponentSyntaxEntry | OriginOnlyComponentSyntaxEntry)[] { return catalog.categories[2].entries; } @@ -236,10 +237,10 @@ const DOCUMENTED = [ ].join("\n"); describe("Tier SY: the versioned shape", () => { - it("SY1: reports version 1 and the three categories in a fixed order", function* () { + it("SY1: reports version 2 and the three categories in a fixed order", function* () { const catalog = yield* catalogFor({ components: { kind: "directory" } }, ["components"]); - expect(catalog.version).toBe(1); + expect(catalog.version).toBe(2); expect(catalog.categories.map((category) => category.kind)).toEqual([ "structural", "built-in", @@ -320,7 +321,7 @@ describe("Tier SY: structural vocabulary", () => { const catalog = yield* catalogFor({}, []); const entries = structural(catalog); - expect(catalog.version).toBe(1); + expect(catalog.version).toBe(2); expect(find(entries, "Switch")).toEqual({ kind: "structural", name: "Switch", @@ -561,7 +562,12 @@ describe("Tier SY: selection decides", () => { it("SY10: falls back to a registration when no repository file supplies a name", function* () { const catalog = yield* catalogFor({ components: { kind: "directory" } }, ["components"]); - expect(names(builtIn(catalog)).sort()).toEqual([...CORE_COMPONENT_NAMES].sort()); + // Core's overridable defaults, plus the names canonical core protects: both + // are built-in to a reader, and the set is pinned exactly so a name arriving + // in either list has to be written down here. + expect(names(builtIn(catalog)).sort()).toEqual( + [...CORE_COMPONENT_NAMES, ...PROTECTED_COMPONENT_NAMES].sort(), + ); expect(userProvided(catalog)).toEqual([]); }); @@ -1147,7 +1153,7 @@ describe("Tier SY: inspection is observation, never authority", () => { }, }); - let catalog: SyntaxCatalog | undefined; + let catalog: SyntaxSymbols | undefined; const failure = yield* raised( scoped(function* () { yield* useTree({}); diff --git a/packages/core/tests/syntax-component.test.ts b/packages/core/tests/syntax-component.test.ts new file mode 100644 index 000000000..27822d29b --- /dev/null +++ b/packages/core/tests/syntax-component.test.ts @@ -0,0 +1,2236 @@ +/** + * Tier SYN — ``, the component canonical core owns. + * + * What a document may write here is a public question, and this is the public + * answer: the symbols for the site the element was written at, in the words + * `xmd syntax` prints. Three things follow, and every case here is about one of + * them. + * + * **The name is the engine's.** A repository `Syntax.md`, a bundled `Syntax`, an + * ordinary or reserved registration, a host declaration, import middleware and a + * definition from a second loaded copy can none of them answer for it. Symbols + * anything in the run could answer for describe nothing. + * + * **The answer is the site's.** The reference is built from the selection + * inputs the execution captured before any installation, middleware or document + * code ran, and it travels lexically on canonical core's own expansion + * authority — not through a context, where a name is not a secret. + * + * **One occurrence renders once.** It claims the identity this execution + * minted, records exactly `{ symbols }`, and a continuation hands that back + * without consulting the filesystem, the registry, the bundle or the host again. + * + * Protection is about the answer, not about power: the component receives one + * reference that renders symbol text and nothing else, and naming a component in + * the symbols is not permission to run it. + */ + +import { describe, it } from "@executablemd/test-support/bdd"; +import { expect } from "@executablemd/test-support/expect"; +import { ensure, scoped, sleep, spawn, suspend, until, withResolvers } from "effection"; +import type { Operation } from "effection"; +import { InMemoryStream } from "@executablemd/durable-streams"; +import type { DurableEvent, Json } from "@executablemd/durable-streams"; +import { mkdtemp, realpath } from "node:fs/promises"; +import { rm, writeTextFile } from "@effectionx/fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { API, useHostFiles } from "@executablemd/runtime"; + +import { Component, content } from "../src/component-api.ts"; +import { collect } from "../src/collect.ts"; +import { execute } from "../src/execute.ts"; +import { executeInstalled, sourceDigest } from "../host.ts"; +import type { + ComponentAnswerRequest, + DeclaredMarkdownComponent, + ExecutionInstallation, + FragmentEvaluationInput, +} from "../host.ts"; +import { inspectComponent, inspectSyntax } from "../src/inspect.ts"; +import { validateDocumentStructure } from "../src/document-validation.ts"; +import { registerComponents } from "../src/components/registration.ts"; +import type { ComponentRegistration } from "../src/components/registration.ts"; +import { selectComponent } from "../src/components/select.ts"; +import { installedBundle } from "../src/components/bundle.ts"; +import { retainedSource } from "../src/root-source.ts"; +import { renderSyntaxMarkdown } from "../src/syntax-markdown.ts"; +import { syntaxReference, rootSyntaxReference } from "../src/syntax-reference.ts"; +import type { SyntaxReference } from "../src/syntax-reference.ts"; +import { + capturedDocumentation, + contributeDocumentation, + Documentation, +} from "../src/documentation-api.ts"; +import { executeReadingAssetsWith } from "../src/execute.ts"; +import type { DocumentationContribution } from "../src/component-documentation.ts"; +import { SYNTAX_COMPONENT } from "../src/components/Syntax.ts"; +import type { ImportedDefinition } from "../src/components/import-authority.ts"; +import type { ComponentOrigin, FunctionComponent, SyntaxSymbols } from "../mod.ts"; +import { answerProvider, implementation } from "./support/answer-provider.ts"; + +/** An origin a *component* symbol entry can carry — everything but structural. */ +type NamedOrigin = Exclude; + +const ROOT_PATH = "documents/root.md"; + +/** The approved description, spelled here so a change to it fails a test. */ +const DESCRIPTION = + "Inspect available components and control-flow constructs. `` lists the " + + 'symbols available here; `` renders selected documentation.'; + +/** Symbols with one built-in entry per name, for a case that needs a marker. */ +function symbolsOf(...names: readonly string[]): SyntaxSymbols { + return { + version: 2, + categories: [ + { kind: "structural", entries: [] }, + { + kind: "built-in", + entries: names.map((name) => ({ + kind: "component" as const, + name, + origin: { kind: "registered" as const, origin: "@executablemd/test", reserved: false }, + sourceKind: "registered" as const, + inspectability: "complete" as const, + forms: ["self-closing" as const], + props: { type: "object", properties: {}, additionalProperties: false }, + captures: [], + returnMode: "text" as const, + returns: { type: "string" }, + })), + }, + { kind: "user-provided", entries: [] }, + ], + }; +} + +/** A host that states the symbols its profile describes, and counts the asks. */ +function stating(symbols: SyntaxSymbols, calls: { count: number } = { count: 0 }) { + const installation: ExecutionInstallation = { + // deno-lint-ignore require-yield + *symbols(): Operation { + calls.count += 1; + return symbols; + }, + }; + return { installation, calls }; +} + +/** + * A package of this suite's own, contributing documentation for ``. + * + * A name core does not ship, so a case about contribution is not also a case + * about colliding with core's real documentation. + */ +function useMarkerDocumentation( + /** + * Whichever of the four values a case wants to vary. + * + * All four are what a contribution *is*, so a case about equality has to be + * able to change exactly one and leave the rest alone. A fresh object is built + * on every ask, so nothing here can pass by object identity. + */ + vary: { + owner?: string; + asset?: string; + text?: string; + supplies?: readonly string[]; + } = {}, +): Operation { + // deno-lint-ignore require-yield + return contributeDocumentation(function* () { + return { + source: { + owner: vary.owner ?? "@executablemd/test", + asset: vary.asset ?? "packages/test/src/components.md", + text: vary.text ?? "## Marker\n\nMARKER PROSE.\n", + }, + supplies: new Set(vary.supplies ?? ["Marker"]), + }; + }); +} + +/** Prose for a contribution accounting for two components at once. */ +const PAIR_PROSE = "## Marker\n\nMARKER PROSE.\n\n## Other\n\nOTHER PROSE.\n"; + +/** + * A collected list as its values, for comparing two collections. + * + * Object identity says nothing here — every ask builds fresh objects — and a + * `Set` has no order, so the names are sorted. What is left is exactly the four + * values equality is defined over. + */ +function describeContributions( + contributions: readonly DocumentationContribution[], +): readonly string[] { + return contributions.map((one) => + [one.source.owner, one.source.asset, one.source.text, [...one.supplies].sort().join(",")].join( + " | ", + ), + ); +} + +/** A trivial registered component, for proving a layer kept its registrations. */ +function layeredComponent(which: string): ComponentRegistration { + return { + name: which === "outer" ? "Outer" : "Inner", + origin: "@executablemd/test", + props: { type: "object", properties: {}, additionalProperties: false }, + // deno-lint-ignore require-yield + *fn(): Operation { + return `${which} component ran`; + }, + }; +} + +/** Run one root, with whatever installations the case supplies. */ +function run( + source: string, + installations: readonly ExecutionInstallation[] = [], + stream: InMemoryStream = new InMemoryStream(), + includes: readonly string[] = [], +): Operation { + return scoped(function* () { + return yield* collect( + yield* executeInstalled( + { ...retainedSource(ROOT_PATH, source), stream, includes: [...includes] }, + [...installations], + ), + ); + }); +} + +/** What one execution refused with, as a string. */ +function* refusal(operation: Operation): Operation { + try { + yield* operation; + } catch (error) { + return error instanceof Error ? error.message : String(error); + } + throw new Error("expected the operation to be refused"); +} + +/** Every retained syntax record, in order. */ +function syntaxReads(events: readonly DurableEvent[]): DurableEvent[] { + return events.filter( + (event) => event.type === "yield" && event.description.type === "syntax_symbols", + ); +} + +/** + * Only the syntaxReads that succeeded. + * + * A refusal still records the attempt and its failure, which is how a journal + * says what happened. What must not exist is a *successful* record: that is the + * thing a continuation would restore and hand back as the symbols. + */ +function retained(events: readonly DurableEvent[]): DurableEvent[] { + return syntaxReads(events).filter( + (event) => event.type === "yield" && event.result.status === "ok", + ); +} + +/** A continuation stream: everything one run recorded but its terminals. */ +function* continuing(stream: InMemoryStream): Operation { + const partial = new InMemoryStream(); + for (const event of yield* stream.readAll()) { + if (event.type === "close") { + continue; + } + yield* partial.append(event); + } + return partial; +} + +/** The same history with one retained reference replaced. */ +function* tampered( + stream: InMemoryStream, + replace: (value: Json) => Json, +): Operation { + const partial = new InMemoryStream(); + for (const event of yield* stream.readAll()) { + if (event.type === "close") { + continue; + } + if ( + event.type === "yield" && + event.description.type === "syntax_symbols" && + event.result.status === "ok" + ) { + yield* partial.append({ + ...event, + result: { status: "ok", value: replace(event.result.value ?? null) }, + }); + continue; + } + yield* partial.append(event); + } + return partial; +} + +/** Symbols holding one component entry of exactly this identity. */ +function symbolsNamed(name: string, origin: NamedOrigin): SyntaxSymbols { + return { + version: 2, + categories: [ + { kind: "structural", entries: [] }, + { + kind: "built-in", + entries: [ + { + kind: "component" as const, + name, + origin, + sourceKind: "registered" as const, + inspectability: "complete" as const, + forms: ["self-closing" as const], + props: { type: "object", properties: {}, additionalProperties: false }, + captures: [], + returnMode: "text" as const, + returns: { type: "string" }, + }, + ], + }, + { kind: "user-provided", entries: [] }, + ], + }; +} + +/** + * The reference an ordinary root carries. + * + * Built the way an execution builds it — from captured selection inputs, with + * no host contribution — so a case about narrowing is about the object the + * product actually hands to expansion. + */ +function* rootObservation(): Operation { + return rootSyntaxReference( + { includes: [], registry: new Map(), components: [], declarations: [] }, + undefined, + // What canonical execution hands it: whatever the scope bootstrapped, + // terminating in core's own. Passing nothing here would leave the reference + // with no index at all, and a case about narrowing would then be reading a + // fallback sentence rather than real documentation. + yield* capturedDocumentation(), + ); +} + +/** A working directory of this case's own, torn down on the way out. */ +function useWorkingDirectory(body: (dir: string) => Operation): Operation { + return scoped(function* () { + const made = yield* until(mkdtemp(join(tmpdir(), "xmd-syntax-"))); + const dir = yield* until(realpath(made)); + yield* ensure(() => rm(dir, { recursive: true, force: true })); + yield* API.Env.around({ + // deno-lint-ignore require-yield + *cwd() { + return dir; + }, + }); + yield* useHostFiles(); + return yield* body(dir); + }); +} + +describe("Tier SYN — what one occurrence answers", () => { + it("SYN1: the bare form renders the symbols once, and `as` binds the same text", function* () { + const { installation } = stating(symbolsOf("Marker")); + const bare = yield* run("\n", [installation]); + expect(String(bare)).toContain("### ``"); + // Once, not twice: one occurrence is one rendering. + expect(String(bare).split("### ``").length - 1).toBe(1); + + const captured = yield* run('\nbound:{symbols}\n', [installation]); + // The same text, and the occurrence itself emitted nothing — what is in the + // document is the binding this case interpolated, not a second copy. + expect(String(captured)).toContain("bound:"); + expect(String(captured)).toContain("### ``"); + expect(String(captured).indexOf("### ``")).toBeGreaterThan( + String(captured).indexOf("bound:"), + ); + expect(String(captured).split("### ``").length - 1).toBe(1); + }); + + it("SYN2: it renders exactly what the shared Markdown renderer produces", function* () { + const symbols = symbolsOf("Marker", "Other"); + const { installation } = stating(symbols); + const bare = yield* run('{symbols}', [installation]); + // The same function `xmd syntax` renders with, not a second one that agrees + // today: an invocation and the command cannot describe one profile in two + // sets of words. + expect(String(bare)).toBe(renderSyntaxMarkdown(symbols)); + }); + + it("SYN3: a paired spelling and an authored prop refuse before any reference", function* () { + const paired = stating(symbolsOf("Marker")); + expect(yield* refusal(run("content\n", [paired.installation]))).toContain( + "written self-closing", + ); + expect(paired.calls.count).toBe(0); + + const propped = stating(symbolsOf("Marker")); + expect(yield* refusal(run('\n', [propped.installation]))).toContain( + "mode", + ); + expect(propped.calls.count).toBe(0); + + // The positive control for the same host: the accepted spelling reads. + const accepted = stating(symbolsOf("Marker")); + expect(String(yield* run("\n", [accepted.installation]))).toContain("Marker"); + expect(accepted.calls.count).toBe(1); + }); + + it("SYN4: one occurrence reads once, two read independently, a binding reads neither again", function* () { + const one = stating(symbolsOf("Marker")); + yield* run('{symbols}{symbols}{symbols}', [one.installation]); + expect(one.calls.count).toBe(1); + + const two = stating(symbolsOf("Marker")); + yield* run("\n\n", [two.installation]); + expect(two.calls.count).toBe(2); + + // Two identities, so two records rather than one record read twice. + const stream = new InMemoryStream(); + yield* run("\n\n", [stating(symbolsOf("Marker")).installation], stream); + expect(syntaxReads(yield* stream.readAll()).length).toBe(2); + }); +}); + +describe("Tier SYN — the named form", () => { + it("SYN29: renders the selected entries' metadata and documentation, and captures it", function* () { + const named = String(yield* run('\n')); + + // Both selected, each once, with metadata and long-form documentation. + expect(named).toContain("### ``"); + expect(named).toContain("### ``"); + expect(named).toContain("Asks a person a structured question"); + expect(named).toContain("Reads or writes a file"); + // Symbol order, not request order: `Elicit` precedes `File` alphabetically + // and the request asked for them the other way round. + expect(named.indexOf("### ``")).toBeLessThan(named.indexOf("### ``")); + // Nothing but the selection: the rest of the symbols are not here. + expect(named).not.toContain("### ``"); + + // `as` binds the same text and emits nothing of it. + const captured = String( + yield* run(['', "{reference}", ""].join("\n")), + ); + const bare = String(yield* run('\n')); + expect(captured.trim()).toBe(bare.trim()); + }); + + it("SYN30: states availability, and says so when documentation is absent", function* () { + const named = String(yield* run('\n')); + // At a root nothing has narrowed execution, so a selected entry is + // available by construction. + expect(named).toContain("**Available in this evaluation:** yes"); + + // A component core supplies but has not documented yet renders its + // metadata and says the documentation is missing, rather than refusing. + // A custom component nothing documents: a repository file, which no + // first-party package governs, so the fallback is the honest answer rather + // than a hole in the product's own reference. + yield* useWorkingDirectory(function* (dir) { + yield* writeTextFile(join(dir, "Homegrown.md"), "a component of my own\n"); + const undocumented = String( + yield* run('\n', [], undefined, [dir]), + ); + expect(undocumented).toContain("### ``"); + expect(undocumented).toContain("No long-form documentation is available for this component."); + }); + + // A structural construct is not a component, and `names` is a component + // lookup: it refuses rather than rendering one. + const structural = yield* refusal(run('\n')); + expect(structural).toContain("If"); + }); + + it("SYN40: documentation survives planted filesystem middleware", function* () { + const asked: string[] = []; + const canonical = String(yield* run('\n')); + expect(canonical).toContain("Asks a person a structured question"); + + // A repository component that wraps the named form, with `API.Fs` middleware + // planted around it. If package documentation were read through the + // filesystem Api a document can compose — or through the `Files` authority — + // this would answer for the product's own reference, and an agent could be + // handed instructions the product never wrote. + yield* useWorkingDirectory(function* (dir) { + yield* writeTextFile( + join(dir, "Wrapper.md"), + ['', ""].join("\n"), + ); + const output = String( + yield* scoped(function* () { + yield* API.Fs.around({ + *readTextFile([path], next) { + asked.push(String(path)); + if (String(path).endsWith("components.md")) { + return "## Elicit\n\nSUBSTITUTED BY A DOCUMENT.\n"; + } + return yield* next(path); + }, + }); + return yield* collect( + yield* executeInstalled( + { + ...retainedSource(ROOT_PATH, "\n"), + stream: new InMemoryStream(), + includes: [dir], + }, + [], + ), + ); + }), + ); + + expect(output).toContain("Asks a person a structured question"); + expect(output).not.toContain("SUBSTITUTED BY A DOCUMENT"); + // And the read never went through that Api at all, which is why. + expect(asked.some((path) => path.endsWith("components.md"))).toBe(false); + }); + }); + + it("SYN46: cancelling documentation collection tears down and commits nothing", function* () { + const torn: string[] = []; + const stream = new InMemoryStream(); + + // Suspended inside *documentation collection*, which is where the packaged + // asset is read: once, at the execution's own boundary, after the trusted + // host bootstrapped and before the root import. So this is the window + // between an execution having begun and any element of its document having + // run, and what it rules out is a teardown that leaves the read hanging or + // a partial record behind. The occurrence-level window is SYN22's. + // + // The reader belongs to *this* execution, handed to it at construction. + // Nothing module-scoped: a second execution in this process reads through + // its own, which SYN48 below is about. + const suspending = function* (): Operation { + torn.push("entered"); + yield* ensure(() => { + torn.push("torn down"); + }); + yield* suspend(); + return "## Elicit\n\nunreachable\n"; + }; + + yield* scoped(function* () { + const task = yield* spawn(function* () { + return yield* collect( + yield* executeReadingAssetsWith( + { + ...retainedSource(ROOT_PATH, '\n'), + stream, + includes: [], + }, + [], + suspending, + ), + ); + }); + // Let collection get inside the read before cancelling it. + yield* sleep(20); + yield* task.halt(); + }); + + const events = yield* stream.readAll(); + // Reached the read, then tore it down — in that order. Cancelling before + // the read was entered would leave `entered` absent, which is the vacuous + // pass this ordering rules out. + expect(torn).toEqual(["entered", "torn down"]); + // And nothing was committed at all. The document never expanded, so no + // occurrence claimed an identity and no durable operation opened: there is + // no record for a continuation to restore, successful or otherwise. + expect(syntaxReads(events)).toHaveLength(0); + expect(retained(events)).toHaveLength(0); + }); + + it("SYN25g: collection snapshots a contribution by value", function* () { + // Everything a bootstrap still holds after its contribution is collected: + // the source object, its text, and the name set. Collection snapshots field + // by field, which is exactly the window this closes — a shallow copy of the + // array would serve whatever these say by the time a document asks. + // + // A package of its own, so this is about capture rather than about + // colliding with core's real documentation of the same name. + const supplies = new Set(["Marker"]); + const source = { + owner: "@executablemd/test", + asset: "packages/test/src/components.md", + text: "## Marker\n\nTHE CAPTURED PROSE.\n", + }; + + const captured = yield* scoped(function* () { + // deno-lint-ignore require-yield + yield* contributeDocumentation(function* () { + return { source, supplies }; + }); + const collected = yield* capturedDocumentation(); + // Rewritten *after* the collector returned, which is the whole window: a + // reference built from this snapshot must not see any of it. + source.text = "## Marker\n\nSUBSTITUTED AFTER COLLECTION.\n"; + source.owner = "@executablemd/impostor"; + supplies.add("Substituted"); + supplies.delete("Marker"); + return collected; + }); + + const mine = captured.find((one) => one.source.asset.startsWith("packages/test/")); + if (mine === undefined) { + throw new Error("the collector did not take this bootstrap's contribution"); + } + expect(mine.source.text).toContain("THE CAPTURED PROSE."); + expect(mine.source.text).not.toContain("SUBSTITUTED AFTER COLLECTION"); + expect(mine.source.owner).toBe("@executablemd/test"); + expect([...mine.supplies]).toEqual(["Marker"]); + + // And the snapshot renders that way through the reference an execution + // builds from it, rather than only reading that way as a value. + const rendered = yield* syntaxReference( + symbolsOf("Marker"), + symbolsOf("Marker"), + captured, + ).documentation(["Marker"]); + expect(rendered).toContain("THE CAPTURED PROSE."); + expect(rendered).not.toContain("SUBSTITUTED AFTER COLLECTION"); + }); + + it("SYN25h: documentation arrives with the bootstrap that registers, or not at all", function* () { + // The whole point of one call: registrations and documentation arrive + // together or not at all. Two lists is what let a nested run register + // `` and then report it undocumented — a component it can run, + // described as undocumented. + const { installation: marker } = stating(symbolsOf("Marker")); + const without = String(yield* scoped(() => run('\n', [marker]))); + // The component is there — the profile states it — and the prose is not. + expect(without).toContain("### ``"); + expect(without).toContain("No long-form documentation is available"); + expect(without).not.toContain("MARKER PROSE."); + + // Entered, the same site answers with the prose instead — *and* core's own + // documentation is still there beside it. A wrapper that returned its own + // contribution instead of appending to what it composed over would pass the + // first assertion and lose the terminal, which is the whole reason the + // chain delegates. + // Core's own ``, at core's own identity, beside this suite's + // ``: the index joins on name *and* origin, so an `Elicit` entry + // carrying this suite's origin would find no core documentation whether the + // terminal survived the chain or not. + const elicit = symbolsNamed("Elicit", { + kind: "registered", + origin: "@executablemd/core", + reserved: false, + }); + const marked = symbolsOf("Marker"); + const pair: SyntaxSymbols = { + version: 2, + categories: [ + { kind: "structural", entries: [] }, + { + kind: "built-in", + entries: [...marked.categories[1].entries, ...elicit.categories[1].entries], + }, + { kind: "user-provided", entries: [] }, + ], + }; + const both = String( + yield* scoped(function* () { + yield* useMarkerDocumentation(); + return yield* run('\n', [ + stating(pair).installation, + ]); + }), + ); + expect(both).toContain("MARKER PROSE."); + expect(both).toContain("Asks a person a structured question"); + expect(both).not.toContain("No long-form documentation is available"); + }); + + it("SYN25k: middleware a running document installs reaches nothing", function* () { + // The ordering half of the contract. Collection happens after the trusted + // host's bootstrap and *before* the root import, so a component that + // composes around the Api while the document is running composes into a + // chain nothing reads again. Otherwise a document could describe a + // component to the next agent however it liked. + const { installation: marker } = stating(symbolsOf("Marker")); + const planted: ExecutionInstallation = { + *install(): Operation { + yield* registerComponents([ + { + name: "Plant", + origin: "@executablemd/test", + props: { type: "object", properties: {}, additionalProperties: false }, + *fn(): Operation { + yield* contributeDocumentation( + // deno-lint-ignore require-yield + function* () { + return { + source: { + owner: "@executablemd/test", + asset: "packages/test/src/planted.md", + text: "## Marker\n\nPLANTED BY THE DOCUMENT.\n", + }, + supplies: new Set(["Marker"]), + }; + }, + ); + // The occurrence renders *inside* this scope, which is the only + // arrangement that tests anything: a sibling element would find + // this middleware already gone and pass however late collection + // happened. + return `planted\n\n${yield* content()}`; + }, + }, + ]); + }, + }; + + const output = String( + yield* scoped(function* () { + yield* useMarkerDocumentation(); + return yield* run('\n\n\n', [marker, planted]); + }), + ); + // The component ran, so the plant is not being reported absent by accident. + expect(output).toContain("planted"); + // And what the occurrence renders is what the host bootstrapped. Had the + // document's contribution been read, this would either say so or refuse as + // a duplicate — either way, not this. + expect(output).toContain("MARKER PROSE."); + expect(output).not.toContain("PLANTED BY THE DOCUMENT"); + }); + + it("SYN25i: two contributions for one component refuse, whichever order", function* () { + // Order decides how the list reads and nothing else. A later contribution + // silently winning would make what a document is told about a component + // depend on the order its host happened to bootstrap packages in. + const { installation: marker } = stating(symbolsOf("Marker")); + const orders: string[] = []; + for (const [first, second] of [ + ["packages/one/components.md", "packages/two/components.md"], + ["packages/two/components.md", "packages/one/components.md"], + ]) { + orders.push( + yield* refusal( + scoped(function* () { + yield* useMarkerDocumentation({ asset: first }); + yield* useMarkerDocumentation({ asset: second }); + return yield* run('\n', [marker]); + }), + ), + ); + } + for (const refused of orders) { + expect(refused).toContain("they name different assets"); + } + // Both orders refuse, and each names the pair it saw rather than one fixed + // winner: a refusal that reported the same asset either way would be + // consistent with a chain that had picked a winner and then complained. + expect(orders[0]).not.toBe(orders[1]); + }); + + it("SYN25l.1: a value-identical repeated bootstrap contributes nothing new", function* () { + // One package's declarative vocabulary is deliberately entered at more than + // one layer — the repository-composition set by an ordinary run's bootstrap + // and again inside a workflow attachment, because either may be the only + // one — and the inner scope descends from the outer, so both wrappers sit + // in one chain. Refusing the repeat turns valid layering into a failure + // while there is nothing ambiguous to resolve. + // + // Every ask builds a fresh object and the second names its components in + // the opposite order, so what passes here is value equality: object + // identity and set insertion order are both ruled out. + const { installation: marker } = stating(symbolsOf("Marker", "Other")); + const twice = (source: string): Operation => + scoped(function* () { + yield* useMarkerDocumentation({ supplies: ["Marker", "Other"], text: PAIR_PROSE }); + yield* useMarkerDocumentation({ supplies: ["Other", "Marker"], text: PAIR_PROSE }); + return yield* run(source, [marker]); + }); + + // Named, bare, and a document that writes no `` at all. Collection + // happens for the execution rather than for an occurrence, so a repeat that + // broke anything would break all three. + expect(String(yield* twice('\n'))).toContain("MARKER PROSE."); + expect(String(yield* twice("\n"))).toContain("### ``"); + expect(String(yield* twice("nothing here\n"))).toContain("nothing here"); + + // And one bootstrap produces the same documentation as two, which is what + // "contributes nothing new" means: not merely that the repeat is tolerated, + // but that it leaves the captured value unchanged. + const captured = (repeats: number): Operation => + scoped(function* () { + for (let entered = 0; entered < repeats; entered += 1) { + yield* useMarkerDocumentation({ supplies: ["Marker", "Other"], text: PAIR_PROSE }); + } + return yield* capturedDocumentation(); + }); + expect(describeContributions(yield* captured(2))).toEqual( + describeContributions(yield* captured(1)), + ); + }); + + it("SYN25l.2: a changed asset, text or component set refuses, in either order", function* () { + // The three ways two contributions can overlap on one owner and component + // while disagreeing. Each refuses, and each refuses whichever order the + // bootstraps ran in — so no order picks a winner. + const { installation: marker } = stating(symbolsOf("Marker")); + const conflicts: [string, Parameters[0]][] = [ + ["a changed asset", { asset: "packages/other/src/components.md" }], + ["changed text", { text: "## Marker\n\nSUBSTITUTED PROSE.\n" }], + ["an overlapping but different set", { supplies: ["Marker", "Extra"] }], + ]; + + for (const [what, changed] of conflicts) { + for (const reversed of [false, true]) { + const refused = yield* refusal( + scoped(function* () { + yield* useMarkerDocumentation(reversed ? changed : {}); + yield* useMarkerDocumentation(reversed ? {} : changed); + // No `` at all: the refusal is the execution's, before its + // root ran, not something an occurrence discovered. + return yield* run("nothing here\n", [marker]); + }), + ); + expect([ + what, + reversed, + refused.includes("contributes documentation for Marker twice"), + ]).toEqual([what, reversed, true]); + } + } + }); + + it("SYN25l.3: a distinct owner and a disjoint set are not conflicts", function* () { + // Documentation joins by component name *and* origin, so two packages may + // each document a component spelled `Marker`: neither answers for the + // other's, and there is no question for anyone to arbitrate. One owner + // accounting for disjoint sets from two files is the same story. + // + // These are the controls that keep SYN25l.2 from passing for the wrong + // reason — a rule that refused any repeated *name* would fail here. + const { installation: marker } = stating(symbolsOf("Marker")); + const rendered = String( + yield* scoped(function* () { + yield* useMarkerDocumentation(); + yield* useMarkerDocumentation({ + owner: "@executablemd/other", + asset: "packages/other/src/components.md", + text: "## Marker\n\nSOMEBODY ELSE'S MARKER.\n", + }); + yield* useMarkerDocumentation({ + asset: "packages/test/src/more.md", + text: "## Separate\n\nA DISJOINT SET.\n", + supplies: ["Separate"], + }); + return yield* run('\n', [marker]); + }), + ); + // The execution ran, and the entry took its own owner's prose — not the + // other package's, which documents a different component of the same name. + expect(rendered).toContain("MARKER PROSE."); + expect(rendered).not.toContain("SOMEBODY ELSE'S MARKER"); + }); + + it("SYN25l.4: overlapping executions each keep their own contributions", function* () { + // Collection is execution-scoped, so a conflict in one assembly is not a + // fact about another running beside it. The first scope is assembled with a + // genuine conflict and refuses; the second, live at the same time, is + // unaffected and renders its own. + const { installation: marker } = stating(symbolsOf("Marker")); + // Spawned in the case's own scope rather than a nested one, which would + // close and halt it: the point is that the two are live at the same time. + const conflicted = yield* spawn(() => + refusal( + scoped(function* () { + yield* useMarkerDocumentation(); + yield* useMarkerDocumentation({ text: "## Marker\n\nCONFLICTING.\n" }); + return yield* run("nothing here\n", [marker]); + }), + ), + ); + + const beside = String( + yield* scoped(function* () { + yield* useMarkerDocumentation(); + return yield* run('\n', [marker]); + }), + ); + expect(beside).toContain("MARKER PROSE."); + expect(beside).not.toContain("CONFLICTING"); + expect(yield* conflicted).toContain("contributes documentation for Marker twice"); + }); + + it("SYN25l.6: duplicate classification belongs to collection, not to the helper", function* () { + // `Documentation` is public, so a package may compose `around(...)` itself + // and hand back two value-identical contributions directly. That assembly + // is exactly as valid as one built with `contributeDocumentation()`, and + // this proves collection says so — nothing below calls the helper. + // + // A rule that lived in the helper would pass every other row in this tier + // and fail here, which is the point of the row. + const { installation: marker } = stating(symbolsOf("Marker", "Other")); + const directly = (vary: Parameters[0] = {}): Operation => + Documentation.around({ + *contributions([read], next): Operation { + // A fresh object on every ask, so nothing here can pass by identity. + return [ + ...(yield* next(read)), + { + source: { + owner: vary.owner ?? "@executablemd/test", + asset: vary.asset ?? "packages/test/src/components.md", + text: vary.text ?? PAIR_PROSE, + }, + supplies: new Set(vary.supplies ?? ["Marker", "Other"]), + }, + ]; + }, + }); + + const twice = (source: string): Operation => + scoped(function* () { + yield* directly(); + // The same four values, with the names listed the other way round. + yield* directly({ supplies: ["Other", "Marker"] }); + return yield* run(source, [marker]); + }); + + expect(String(yield* twice('\n'))).toContain("MARKER PROSE."); + expect(String(yield* twice("\n"))).toContain("### ``"); + expect(String(yield* twice("nothing here\n"))).toContain("nothing here"); + + // And canonical capture holds one contribution, not two. + const captured = yield* scoped(function* () { + yield* directly(); + yield* directly({ supplies: ["Other", "Marker"] }); + return yield* capturedDocumentation(); + }); + expect( + captured.filter((one) => one.source.asset === "packages/test/src/components.md"), + ).toHaveLength(1); + + // The same three disagreements, reached the same direct way, still refuse + // before the root — in either order, and with no `` to discover it. + const conflicts: [string, Parameters[0]][] = [ + ["a changed asset", { asset: "packages/other/src/components.md" }], + ["changed text", { text: `${PAIR_PROSE}\nSUBSTITUTED.\n` }], + ["an overlapping but different set", { supplies: ["Marker", "Extra"] }], + ]; + for (const [what, changed] of conflicts) { + for (const reversed of [false, true]) { + const refused = yield* refusal( + scoped(function* () { + yield* directly(reversed ? changed : {}); + yield* directly(reversed ? {} : changed); + return yield* run("nothing here\n", [marker]); + }), + ); + expect([ + what, + reversed, + refused.includes("contributes documentation for Marker twice"), + ]).toEqual([what, reversed, true]); + } + } + }); + + it("SYN25l.5: layered trusted bootstraps keep their registrations and documentation", function* () { + // The shape a nested run and an `` host both take: an outer + // trusted layer enters a package's bootstrap, an inner one enters the same + // package again, and the inner scope descends from the outer. Both halves + // have to survive — the registrations *and* the documentation — because + // losing either is how the two-list design failed in the first place. + const outerThenInner = yield* scoped(function* () { + yield* useMarkerDocumentation(); + yield* registerComponents([layeredComponent("outer")]); + return yield* scoped(function* () { + // The inner layer enters the same package bootstrap, and adds a + // registration of its own. + yield* useMarkerDocumentation(); + yield* registerComponents([layeredComponent("inner")]); + return yield* run('\n\n\n\n\n', [ + stating(symbolsOf("Marker")).installation, + ]); + }); + }); + + const rendered = String(outerThenInner); + // Both registrations resolve: entering the bootstrap twice took nothing + // away from the enclosing layer. + expect(rendered).toContain("outer component ran"); + expect(rendered).toContain("inner component ran"); + // And the documentation is there exactly once, from one coalesced value. + expect(rendered).toContain("MARKER PROSE."); + expect(rendered.match(/MARKER PROSE\./g)).toHaveLength(1); + }); + + /** + * FE30 — one package bootstrap entered through an inherited layer and a + * local one, where the bootstrap includes a component-answer provider. + * + * SYN25l.5's harness, asked the question `` adds to it: a trusted + * layering control's bootstrap is not only documentation and registrations. + * It also installs the provider that answers for a name the host's evaluation + * profile admits, and *that* is the part `allow` decides about. + * + * The layering is the same shape as SYN25l.5 — the inner scope descends from + * the outer, and both enter the same bootstrap. What the repeat means for a + * provider is that the outer entry observes and delegates rather than + * claiming again: one implementation states what it is once, so an additive + * layering has exactly one claim behind the name. + * + * These rows reuse #765's collector rather than adding one. The + * documentation halves here are the same asks SYN25l.1/.2/.5 make, and what + * is new is only the provider and the profile beside them. + */ + const PROBE_LABEL = "probe ran"; + + /** The provider-backed entry a layered host admits, at the identity it admits. */ + function layeredEntry(revision = "1"): FragmentEvaluationInput { + return { + read: [ + { + kind: "component-answer", + name: "Open", + identity: { origin: "test://layer", key: "Open", revision }, + forms: ["self-closing"], + }, + ], + }; + } + + /** The registration the inner layer adds beside the bootstrap's own. */ + function probeRegistration(probed: string[]): ComponentRegistration { + return { + name: "Probe", + origin: "@executablemd/test", + props: { type: "object", properties: {}, additionalProperties: false }, + // deno-lint-ignore require-yield + *fn(): Operation { + probed.push("probed"); + return PROBE_LABEL; + }, + }; + } + + it("FE30: an inherited and a local provider bootstrap keep every half", function* () { + const probed: string[] = []; + const answer = implementation("Open", "the layered answer ran"); + const inheritedLookups: string[] = []; + const localLookups: string[] = []; + const observed: unknown[] = []; + const retained: ComponentAnswerRequest[] = []; + + // The inherited layer's entry into the bootstrap: documentation, a + // registration, and the package's provider — which observes and delegates, + // because the local entry below is the one that states the identity. + const inherited: ExecutionInstallation = { + componentAnswers: [ + answerProvider("Open", answer.definition, { + origin: "test://layer", + lookups: inheritedLookups, + observesOnly: true, + delegated: observed, + }), + ], + }; + + const rendered = String( + yield* scoped(function* () { + yield* useMarkerDocumentation(); + yield* registerComponents([layeredComponent("outer")]); + return yield* scoped(function* () { + // The inner layer enters the same package bootstrap again, and adds + // its own registration and its own entry into the same provider. + yield* useMarkerDocumentation(); + yield* registerComponents([layeredComponent("inner"), probeRegistration(probed)]); + const local: ExecutionInstallation = { + evaluation: layeredEntry(), + componentAnswers: [ + answerProvider("Open", answer.definition, { + origin: "test://layer", + lookups: localLookups, + retain: retained, + }), + ], + }; + return yield* run( + '\n\n\n\n\n\n\n\n' + + `\\n'} allow={["read"]} as="answer" />\n\n` + + "\n", + [stating(symbolsOf("Marker")).installation, inherited, local], + ); + }); + }), + ); + + // The bootstrap survived being entered twice. Both registrations resolve, + // the ordinary `` runs, and the named documentation is there once + // — SYN25l.5's evidence, unchanged. + expect(rendered).toContain("outer component ran"); + expect(rendered).toContain("inner component ran"); + expect(rendered).toContain(PROBE_LABEL); + expect(rendered.match(/MARKER PROSE\./g)).toHaveLength(1); + expect(probed).toEqual(["probed"]); + + // And the provider half survived too. Both layers were in the chain — the + // inherited one observed the lookup and passed the answer through — and the + // locally claimed identity is what the capture admitted and the fragment + // ran. + expect(inheritedLookups).toEqual(["Open"]); + expect(localLookups).toEqual(["Open"]); + expect(observed[0]).toBe(answer.definition); + expect(rendered).toContain("the layered answer ran"); + expect(answer.invoked).toEqual(["the layered answer ran"]); + + // Teardown is the execution's, not the layer's: the request the local + // bootstrap's handler was given states nothing once the child run is over, + // and the enclosing layers are still standing. + const held = retained[0]; + if (held === undefined) { + throw new Error("the local provider was never asked"); + } + let refused: unknown; + try { + held.claim(implementation("Open", "late").definition, { key: "Open", revision: "1" }); + } catch (error) { + refused = error; + } + expect(String(refused)).toContain("has ended"); + }); + + it("FE30: the bootstrap's own names are still not admitted ones", function* () { + // The negative half. `` is registered by the inner layer and its + // package is documented, and the provider's answer is admitted — and none + // of the first two is authority. The profile admits exactly ``, and + // the fragment naming `` is refused before the component is + // reached. + const probed: string[] = []; + const answer = implementation("Open", "the layered answer ran"); + + const failed = yield* refusal( + scoped(function* () { + yield* useMarkerDocumentation(); + yield* registerComponents([layeredComponent("outer")]); + return yield* scoped(function* () { + yield* useMarkerDocumentation(); + yield* registerComponents([layeredComponent("inner"), probeRegistration(probed)]); + return yield* run(`\\n'} allow={["read"]} />\n`, [ + stating(symbolsOf("Marker")).installation, + { + evaluation: layeredEntry(), + componentAnswers: [ + answerProvider("Open", answer.definition, { origin: "test://layer" }), + ], + }, + ]); + }); + }), + ); + + expect(failed).toContain("did not admit"); + // Before the component: the registration a layer kept is still not an + // admission, and neither the registered name nor the admitted one ran. + expect(probed).toEqual([]); + expect(answer.invoked).toEqual([]); + }); + + it("FE30: a provider identity the layered host did not admit refuses", function* () { + // The other refusal the provider half adds. The layering is honest and the + // bootstrap's documentation and registrations are intact; what moved is the + // revision the local provider claims. An admitted identity is the exact + // implementation, so this refuses at capture — before the root import, and + // therefore before any registration resolves or any fragment work happens. + const probed: string[] = []; + const answer = implementation("Open", "the layered answer ran"); + + const failed = yield* refusal( + scoped(function* () { + yield* useMarkerDocumentation(); + yield* registerComponents([layeredComponent("outer")]); + return yield* scoped(function* () { + yield* useMarkerDocumentation(); + yield* registerComponents([layeredComponent("inner"), probeRegistration(probed)]); + return yield* run(`\n\n\\n'} allow={["read"]} />\n`, [ + stating(symbolsOf("Marker")).installation, + { + evaluation: layeredEntry("1"), + componentAnswers: [ + answerProvider("Open", answer.definition, { + origin: "test://layer", + revision: "2", + }), + ], + }, + ]); + }); + }), + ); + + expect(failed).toContain("the exact implementation"); + expect(answer.invoked).toEqual([]); + expect(probed).toEqual([]); + }); + + it("FE30: a non-identical overlap refuses before the child root or any fragment work", function* () { + // #765's own conflict, in this harness. One owner and one overlapping + // component documented differently by the two layers is a bootstrap nobody + // validated, and it refuses at the child collection boundary — which is + // before the root import, so the provider is never asked and the fragment + // never exists. + const probed: string[] = []; + const answer = implementation("Open", "the layered answer ran"); + const conflicts: [string, Parameters[0]][] = [ + ["a changed asset", { asset: "packages/other/src/components.md" }], + ["changed text", { text: "## Marker\n\nSUBSTITUTED PROSE.\n" }], + ["an overlapping but different set", { supplies: ["Marker", "Extra"] }], + ]; + + for (const [what, changed] of conflicts) { + for (const reversed of [false, true]) { + const lookups: string[] = []; + const failed = yield* refusal( + scoped(function* () { + yield* useMarkerDocumentation(reversed ? changed : {}); + yield* registerComponents([layeredComponent("outer")]); + return yield* scoped(function* () { + yield* useMarkerDocumentation(reversed ? {} : changed); + yield* registerComponents([layeredComponent("inner"), probeRegistration(probed)]); + return yield* run( + `\n\n\\n'} allow={["read"]} />\n`, + [ + stating(symbolsOf("Marker")).installation, + { + evaluation: layeredEntry(), + componentAnswers: [ + answerProvider("Open", answer.definition, { + origin: "test://layer", + lookups, + }), + ], + }, + ], + ); + }); + }), + ); + + expect([what, reversed, failed.includes("Marker twice")]).toEqual([what, reversed, true]); + // Before the child root: the provider was never asked, so no answer was + // resolved, no profile was sealed and no fragment ran. + expect([what, reversed, lookups]).toEqual([what, reversed, []]); + } + } + expect(answer.invoked).toEqual([]); + expect(probed).toEqual([]); + }); + + it("SYN25j: two scopes each read their own contributions", function* () { + // A contribution belongs to the scope that installed it, because that is + // what an Api answer belongs to. Two executions assembled in sibling scopes + // must not read through each other's. + const { installation: marker } = stating(symbolsOf("Marker")); + const inside = String( + yield* scoped(function* () { + yield* useMarkerDocumentation(); + return yield* run('\n', [marker]); + }), + ); + expect(inside).toContain("MARKER PROSE."); + + // The sibling scope installed nothing, so it has nothing — and the first + // scope's contribution did not outlive it. + const outside = String(yield* scoped(() => run('\n', [marker]))); + expect(outside).toContain("No long-form documentation is available"); + expect(outside).not.toContain("MARKER PROSE."); + }); + + it("SYN48: an ordinary reference is unaffected by another execution's suspended one", function* () { + // Two executions overlapping in one process. One is stopped inside its own + // documentation collection; the other is ordinary and must read canonical + // documentation and finish on its own. + // + // This is what a module-scoped reader gets wrong: one variable shared by + // every execution means the suspended one's substitution is what the + // ordinary one reads, and it would either hang on the same suspend or + // render the substituted prose. Against the module-global implementation at + // 9d92bcbe this fails. + const entered = withResolvers(); + const stream = new InMemoryStream(); + + const ordinary = yield* scoped(function* () { + const held = yield* spawn(function* () { + return yield* collect( + yield* executeReadingAssetsWith( + { + ...retainedSource(ROOT_PATH, '\n'), + stream, + includes: [], + }, + [], + function* (): Operation { + entered.resolve(); + yield* suspend(); + return "## Elicit\n\nSUBSTITUTED BY THE OTHER EXECUTION.\n"; + }, + ), + ); + }); + + // Only once the first execution is genuinely inside its own index work. + yield* entered.operation; + + // A second, ordinary execution — no reader of its own, so it uses the + // real one. + const rendered = String(yield* run('\n')); + yield* held.halt(); + return rendered; + }); + + expect(ordinary).toContain("Asks a person a structured question"); + expect(ordinary).not.toContain("SUBSTITUTED BY THE OTHER EXECUTION"); + }); + + it("SYN49: a named root occurrence resolves its symbols exactly once", function* () { + // A contribution that *changes* between calls, so a second resolution is + // not merely wasteful but visible: an entry's metadata would come from one + // symbols and the availability beside it from another. + const calls = { count: 0 }; + const moving: ExecutionInstallation = { + // deno-lint-ignore require-yield + *symbols(): Operation { + calls.count += 1; + return symbolsOf(`Marker${calls.count}`); + }, + }; + + const stream = new InMemoryStream(); + const first = String(yield* run('\n', [moving], stream)); + + // Once — not once for selection and again for availability. + expect(calls.count).toBe(1); + // And both decisions came from that one value: the entry is rendered, and + // it is available. A second resolution would have produced `Marker2`, + // leaving `Marker1` unselectable or unavailable. + expect(first).toContain("### ``"); + expect(first).toContain("**Available in this evaluation:** yes"); + + // Two occurrences still read independently: this is one read per + // occurrence, not one per execution. + calls.count = 0; + const both = String( + yield* run( + ['', "", '', ""].join("\n"), + [moving], + ), + ); + expect(calls.count).toBe(2); + expect(both).toContain("### ``"); + expect(both).toContain("### ``"); + + // A continuation restores the retained text without asking again. + calls.count = 0; + const resumed = String( + yield* run('\n', [moving], yield* continuing(stream)), + ); + expect(resumed.trim()).toBe(first.trim()); + expect(calls.count).toBe(0); + }); + + it("SYN39: retains the named text, and a continuation restores it whole", function* () { + const stream = new InMemoryStream(); + const first = String(yield* run('\n', [], stream)); + expect(first).toContain("Asks a person a structured question"); + + // Exactly what was rendered, not the compact list: the record is the + // occurrence's final text whichever form produced it. + const records = retained(yield* stream.readAll()); + expect(records).toHaveLength(1); + const record = records[0]; + const value = + record?.type === "yield" && record.result.status === "ok" ? record.result.value : undefined; + expect(Object.keys(value as object)).toEqual(["symbols"]); + // The component's own return, which the document then renders — so the two + // differ by the trailing newline presentation adds, and nothing else. + expect(String((value as { symbols: string }).symbols).trim()).toBe(first.trim()); + + // A continuation hands the same text back. The documentation asset is not + // reread and the symbols are not rebuilt: what an agent was shown is what it + // is shown again. + const resumed = String( + yield* run('\n', [], yield* continuing(stream)), + ); + expect(resumed).toBe(first); + + // And a record this version cannot read refuses rather than inventing one. + const corrupted = yield* tampered(stream, () => ({ symbols: "x", extra: 1 })); + const refused = yield* refusal(run('\n', [], corrupted)); + expect(refused).toContain("not a record this version can read"); + }); + + it("SYN31: refuses an unusable list before reading anything", function* () { + const stream = new InMemoryStream(); + const unknown = yield* refusal(run('\n', [], stream)); + expect(unknown).toContain("Nonexistent"); + // No successful record: the attempt and its failure are journaled, as any + // effect's are, but there is nothing for a continuation to restore and hand + // back as the symbols. + expect(retained(yield* stream.readAll())).toHaveLength(0); + + for (const written of [ + "", + '', + "", + '', + '', + ]) { + const each = new InMemoryStream(); + const message = yield* refusal(run(`${written}\n`, [], each)); + expect([written, message.length > 0]).toEqual([written, true]); + expect([written, retained(yield* each.readAll()).length]).toEqual([written, 0]); + } + }); +}); + +describe("Tier SYN — the name canonical core owns", () => { + it("SYN5: a repository Syntax.md, Syntax.ts and directory candidate never win", function* () { + yield* useWorkingDirectory(function* (dir) { + yield* writeTextFile(join(dir, "Syntax.md"), "repository symbols\n"); + yield* writeTextFile(join(dir, "Nearby.md"), "a nearby repository component\n"); + const { installation } = stating(symbolsOf("Marker")); + + const output = String( + yield* run("\n\n", [installation], undefined, [dir]), + ); + // The protected component answered, and the repository file did not. + expect(output).toContain("### ``"); + expect(output).not.toContain("repository symbols"); + // The positive control: repository discovery is active in this very run, + // so the absence above is protection rather than a search that never ran. + expect(output).toContain("a nearby repository component"); + }); + }); + + it("SYN6: selection reports the protected tier ahead of every other", function* () { + yield* useWorkingDirectory(function* (dir) { + yield* writeTextFile(join(dir, "Syntax.md"), "repository symbols\n"); + const selected = yield* selectComponent(SYNTAX_COMPONENT, { includes: [dir] }); + expect(selected.kind).toBe("protected"); + // Its own origin kind. Not a reserved registration: that is a host + // installing something under a name, which can be absent, replaced or + // refused, and none of those is true of a name core owns. + expect(selected.kind === "protected" ? selected.origin : undefined).toEqual({ + kind: "protected", + origin: "@executablemd/core", + }); + }); + }); + + it("SYN7: an ordinary and a reserved registration named Syntax are both refused", function* () { + const refused = yield* refusal( + scoped(function* () { + yield* registerComponents([ + { + name: "Syntax", + origin: "@executablemd/test", + props: { type: "object", properties: {}, additionalProperties: false }, + // deno-lint-ignore require-yield + *fn(): Operation { + return "replaced"; + }, + }, + ]); + }), + ); + expect(refused).toContain("canonical core owns that name"); + + const reservedRefusal = yield* refusal( + scoped(function* () { + yield* registerComponents([ + { + name: "Syntax", + origin: "@executablemd/test", + reserved: true, + props: { type: "object", properties: {}, additionalProperties: false }, + // deno-lint-ignore require-yield + *fn(): Operation { + return "replaced"; + }, + }, + ]); + }), + ); + expect(reservedRefusal).toContain("canonical core owns that name"); + }); + + it("SYN8: the refused batch registers nothing, and an adjacent registration still works", function* () { + const good = { + name: "Adjacent", + origin: "@executablemd/test", + props: { type: "object", properties: {}, additionalProperties: false }, + // deno-lint-ignore require-yield + *fn(): Operation { + return "adjacent ran"; + }, + }; + // The batch is refused whole: `Adjacent` is beside the refused name, and + // nothing of it survives. + yield* refusal( + registerComponents([ + good, + { + name: "Syntax", + origin: "@executablemd/test", + props: { type: "object", properties: {}, additionalProperties: false }, + // deno-lint-ignore require-yield + *fn(): Operation { + return "replaced"; + }, + }, + ]), + ); + const after = yield* selectComponent("Adjacent", { + includes: [], + registry: yield* Component.operations.registry, + }); + expect(after.kind).toBe("unresolved"); + + // The positive control: registration is available in this very scope. + yield* registerComponents([good]); + expect( + (yield* selectComponent("Adjacent", { + includes: [], + registry: yield* Component.operations.registry, + })).kind, + ).toBe("registered"); + }); + + it("SYN9: a host that declares Markdown called Syntax is refused before the root import", function* () { + const source = "declared symbols\n"; + const declaration: DeclaredMarkdownComponent = { + name: "Syntax", + origin: "@executablemd/test/Syntax.md", + source, + digest: sourceDigest(source), + }; + const stream = new InMemoryStream(); + expect( + yield* refusal(run("\n", [{ declarations: [declaration] }], stream)), + ).toContain("canonical core owns that name"); + // Before the root import: nothing was imported and nothing was read. + const events = yield* stream.readAll(); + expect(events.filter((event) => event.type === "yield").length).toBe(0); + + // The positive control: an adjacent declaration under another name is + // admitted and runs, so the refusal is about the name. + const adjacent: DeclaredMarkdownComponent = { + name: "Policy", + origin: "@executablemd/test/Policy.md", + source, + digest: sourceDigest(source), + }; + expect(String(yield* run("\n", [{ declarations: [adjacent] }]))).toContain( + "declared symbols", + ); + }); + + it("SYN10: a workflow bundle member called Syntax is refused before the root import", function* () { + const bundled = { + name: "Syntax", + path: "components/Syntax.md", + sourceHash: "0".repeat(40), + content: "bundled symbols\n", + }; + const adjacent = { + name: "Bundled", + path: "components/Bundled.md", + sourceHash: "1".repeat(40), + content: "a bundled component\n", + }; + expect( + yield* refusal(run("\n", [{ bundle: { components: [bundled, adjacent] } }])), + ).toContain("canonical core owns that name"); + + // The positive control: the same bundle without the protected name installs + // and its member runs. + expect(String(yield* run("\n", [{ bundle: { components: [adjacent] } }]))).toContain( + "a bundled component", + ); + }); +}); + +describe("Tier SYN — what the chain may and may not do", () => { + /** A handler that answers `Syntax` with whatever `answer` produces. */ + function answering( + answer: (real: ImportedDefinition) => ImportedDefinition, + ): ExecutionInstallation { + return { + *install() { + yield* Component.around( + { + *importComponent([name, position], next) { + if (name !== SYNTAX_COMPONENT) { + return yield* next(name, position); + } + return answer(yield* next(name, position)); + }, + }, + { at: "max" }, + ); + }, + }; + } + + it("SYN11: ordinary delegation reaches canonical Syntax", function* () { + const seen: string[] = []; + const watching: ExecutionInstallation = { + *install() { + yield* Component.around( + { + *importComponent([name, position], next) { + seen.push(name); + return yield* next(name, position); + }, + }, + { at: "max" }, + ); + }, + }; + const { installation } = stating(symbolsOf("Marker")); + expect(String(yield* run("\n", [installation, watching]))).toContain("Marker"); + // The handler observed the import it could not answer. + expect(seen).toContain(SYNTAX_COMPONENT); + }); + + it("SYN12: a handler that answers, substitutes, mutates or copies runs no replacement", function* () { + const replacement: FunctionComponent = function* () { + return "replaced symbols"; + }; + const cases: [string, (real: ImportedDefinition) => ImportedDefinition][] = [ + [ + "answers without delegating", + () => ({ + kind: "function", + name: SYNTAX_COMPONENT, + props: { type: "object", properties: {}, additionalProperties: false }, + fn: replacement, + }), + ], + [ + "substitutes a copy that describes the same contract", + (real) => ({ ...Object(real), fn: replacement }), + ], + [ + "mutates what canonical execution produced", + (real) => { + Reflect.set(Object(real), "fn", replacement); + return real; + }, + ], + ]; + + for (const [, answer] of cases) { + const { installation, calls } = stating(symbolsOf("Marker")); + const refused = yield* refusal(run("\n", [installation, answering(answer)])); + expect(refused).toContain("canonical core owns"); + // Refused before the body: no symbols were read for the replacement. + expect(calls.count).toBe(0); + } + }); + + it("SYN13: a handler that redirects the name, or delegates twice, answers nothing", function* () { + const redirecting: ExecutionInstallation = { + *install() { + yield* Component.around( + { + *importComponent([name, position], next) { + // The answer canonical execution produced for another name. + return name === SYNTAX_COMPONENT + ? yield* next("Other", position) + : yield* next(name, position); + }, + }, + { at: "max" }, + ); + }, + }; + expect( + yield* refusal(run("\n", [stating(symbolsOf("Marker")).installation, redirecting])), + ).toBeTruthy(); + + const twice: ExecutionInstallation = { + *install() { + yield* Component.around( + { + *importComponent([name, position], next) { + if (name !== SYNTAX_COMPONENT) { + return yield* next(name, position); + } + yield* next(name, position); + return yield* next(name, position); + }, + }, + { at: "max" }, + ); + }, + }; + // Two canonical selections in one frame yield no domain, so the occurrence + // can name no durable operation and the invocation refuses. + expect( + yield* refusal(run("\n", [stating(symbolsOf("Marker")).installation, twice])), + ).toBeTruthy(); + }); + + it("SYN14: a deliberate middleware refusal stays a refusal", function* () { + const refusing: ExecutionInstallation = { + *install() { + yield* Component.around( + { + *importComponent([name, position], next) { + if (name === SYNTAX_COMPONENT) { + throw new Error("this host refuses the symbols"); + } + return yield* next(name, position); + }, + }, + { at: "max" }, + ); + }, + }; + const { installation, calls } = stating(symbolsOf("Marker")); + expect(yield* refusal(run("\n", [installation, refusing]))).toContain( + "this host refuses the symbols", + ); + expect(calls.count).toBe(0); + }); + + it("SYN15: a document-authored context and a look-alike reference change nothing", function* () { + // Nothing a document writes reaches the reference: it is not addressed by + // name. The strongest thing an authored document can do is register and + // bind, and the symbols are unchanged by both. + const { installation } = stating(symbolsOf("Marker")); + const source = [ + '', + '', + "{read}", + "", + ].join("\n"); + const output = String(yield* run(source, [installation])); + expect(output).toContain("### ``"); + expect(output).not.toContain("planted symbols"); + }); +}); + +describe("Tier SYN — the site the symbols describe", () => { + it("SYN16: the derived symbols report this execution's own includes and registry", function* () { + yield* useWorkingDirectory(function* (dir) { + yield* writeTextFile(join(dir, "Local.md"), "a local component\n"); + // No host contribution: canonical core derives the symbols from the + // selection inputs this execution captured. + const output = String(yield* run("\n", [], undefined, [dir])); + expect(output).toContain("### ``"); + // And it describes itself, once, with the approved description. + expect(output).toContain("### ``"); + expect(output).toContain(DESCRIPTION); + // Its own provenance, not a registration's. A reader deciding whether + // they could supply this name themselves gets the opposite answer from + // the two phrases, so the symbols must not print the other one. + expect(output).toContain("`@executablemd/core` (protected component)"); + expect(output).not.toContain("reserved registration"); + }); + }); + + it("SYN27: the symbols report a protected component as protected, not registered", function* () { + const { installation } = stating(symbolsOf("Marker")); + const symbols = yield* scoped(function* () { + yield* executeInstalled( + { + ...retainedSource(ROOT_PATH, "\n"), + stream: new InMemoryStream(), + includes: [], + }, + [installation], + ); + return yield* inspectSyntax({ includes: [] }); + }); + expect(symbols.version).toBe(2); + + // Built-in: the second category, where a reader indexes for it. + const entry = symbols.categories[1].entries.find((candidate) => candidate.name === "Syntax"); + if (entry === undefined) { + throw new Error("expected the symbols to describe "); + } + // The structured origin, which is what a machine reader switches on. + expect(entry.origin).toEqual({ kind: "protected", origin: "@executablemd/core" }); + expect(entry.sourceKind).toBe("protected"); + + // And `inspectComponent` agrees, so one name and the whole environment + // cannot describe the same component two ways. + const info = yield* scoped(function* () { + return yield* inspectComponent({ name: "Syntax", includes: [] }); + }); + if (info.kind !== "protected") { + throw new Error(`expected a protected component, got ${info.kind}`); + } + expect(info.origin).toEqual({ kind: "protected", origin: "@executablemd/core" }); + }); + + it("SYN28: a bundled component is reported as pinned, not as a repository file", function* () { + const sourceHash = "1".repeat(40); + const bundle = { + components: [ + { + name: "Bundled", + path: "components/Bundled.md", + sourceHash, + content: "a bundled component\n", + }, + ], + }; + const output = String(yield* run("\n", [{ bundle }])); + + // The path alone would read as a file the reader could edit; the blob id is + // what says this is the exact source the run was defined against. + expect(output).toContain("`components/Bundled.md` (workflow bundle, `111111111111`)"); + + const symbols = yield* scoped(function* () { + const registry = yield* Component.operations.registry; + const workflow = installedBundle([bundle], registry); + if (workflow === undefined) { + throw new Error("expected the bundle to install"); + } + return yield* inspectSyntax({ includes: [], workflow }); + }); + // User-provided: the third category. + const entry = symbols.categories[2].entries.find((candidate) => candidate.name === "Bundled"); + if (entry === undefined || entry.inspectability !== "complete") { + throw new Error("expected the symbols to describe completely"); + } + expect(entry.origin).toEqual({ + kind: "workflow", + path: "components/Bundled.md", + sourceHash, + }); + expect(entry.sourceKind).toBe("workflow-markdown"); + }); + + it("SYN17: a workflow root reports its own bundle without running a member", function* () { + const entered: string[] = []; + const bundle = { + components: [ + { + name: "Bundled", + path: "components/Bundled.md", + sourceHash: "1".repeat(40), + content: "a bundled component\n", + }, + ], + }; + const stream = new InMemoryStream(); + const output = String(yield* run("\n", [{ bundle }], stream)); + expect(output).toContain("### ``"); + // Described, not run: nothing imported or expanded the member. + expect(entered).toEqual([]); + const imported = (yield* stream.readAll()).filter( + (event) => event.type === "yield" && event.description.type === "import_component", + ); + expect( + imported.some((event) => event.type === "yield" && event.description.name === "Bundled"), + ).toBe(false); + }); + + it("SYN18: a declared Markdown component's own body reports the site it inherited", function* () { + const source = ['', "policy sees {symbols}", ""].join("\n"); + const declaration: DeclaredMarkdownComponent = { + name: "Policy", + origin: "@executablemd/test/Policy.md", + source, + digest: sourceDigest(source), + }; + const { installation } = stating(symbolsOf("Marker")); + const output = String( + yield* run("\n", [installation, { declarations: [declaration] }]), + ); + expect(output).toContain("policy sees"); + expect(output).toContain("### ``"); + }); +}); + +describe("Tier SYN — the record one occurrence keeps", () => { + it("SYN19: the retained payload is closed on exactly { symbols }", function* () { + const stream = new InMemoryStream(); + yield* run("\n", [stating(symbolsOf("Marker")).installation], stream); + const [reference] = syntaxReads(yield* stream.readAll()); + if (reference?.type !== "yield" || reference.result.status !== "ok") { + throw new Error("the run retained no syntax record"); + } + const value = Object(reference.result.value); + expect(Object.keys(value)).toEqual(["symbols"]); + expect(typeof value.symbols).toBe("string"); + }); + + it("SYN20: a continuation restores the symbols after the environment moves, and asks nothing", function* () { + const first = new InMemoryStream(); + const before = String( + yield* run("\n", [stating(symbolsOf("Before")).installation], first), + ); + expect(before).toContain("### ``"); + + // The environment moved: the host now states a different profile, and the + // contribution refuses to answer at all. + const moved: ExecutionInstallation = { + // deno-lint-ignore require-yield + *symbols(): Operation { + throw new Error("the continuation rebuilt the symbols"); + }, + }; + const continued = String(yield* run("\n", [moved], yield* continuing(first))); + expect(continued).toContain("### ``"); + expect(continued).not.toContain("### ``"); + + // A fresh execution sees the moved environment, which is what shows the + // restoration above was retention rather than the reference being inert. + expect( + String(yield* run("\n", [stating(symbolsOf("After")).installation])), + ).toContain("### ``"); + }); + + it("SYN21: a missing, extra or wrong-typed retained payload refuses before output or binding", function* () { + const cases: [string, (value: Json) => Json][] = [ + ["the member is missing", () => ({})], + ["an unknown member was added", (value) => ({ ...Object(value), extra: true })], + ["the member has the wrong type", () => ({ symbols: 7 })], + ]; + for (const [, replace] of cases) { + const first = new InMemoryStream(); + yield* run("\n", [stating(symbolsOf("Marker")).installation], first); + const hostile = yield* tampered(first, replace); + const refused = yield* refusal( + run( + 'bound:{symbols}', + [stating(symbolsOf("Marker")).installation], + hostile, + ), + ); + expect(refused).toContain("is not a record this version can read"); + } + }); + + it("SYN22: a cancelled reference tears down and commits no record", function* () { + const teardown: string[] = []; + const stream = new InMemoryStream(); + const hanging: ExecutionInstallation = { + *symbols(): Operation { + yield* ensure(function* () { + teardown.push("released"); + }); + yield* suspend(); + throw new Error("unreachable"); + }, + }; + + yield* scoped(function* () { + const running = yield* spawn(function* () { + yield* run("\n", [hanging], stream); + }); + // Long enough for the reference to be entered and suspended. + yield* sleep(20); + yield* running.halt(); + }); + + // The structured teardown ran, and nothing successful was committed. + expect(teardown).toEqual(["released"]); + const committed = syntaxReads(yield* stream.readAll()).filter( + (event) => event.type === "yield" && event.result.status === "ok", + ); + expect(committed).toEqual([]); + }); +}); + +describe("Tier SYN — reference is never authority", () => { + it("SYN23: symbols naming a component neither register nor resolve it", function* () { + // The strongest form: the trusted host itself states symbols naming a + // component nothing supplies. + const { installation } = stating(symbolsOf("Phantom")); + const output = String(yield* run("\n", [installation])); + expect(output).toContain("### ``"); + + // It is still a name nothing answers for. + expect(yield* refusal(run("\n", [installation]))).toContain( + "Cannot resolve component: Phantom", + ); + expect((yield* selectComponent("Phantom", { includes: [] })).kind).toBe("unresolved"); + }); + + it("SYN24: the component is described identically by inspection and by validation", function* () { + const symbols = yield* inspectSyntax({ includes: [] }); + const entry = symbols.categories[1].entries.find((candidate) => candidate.name === "Syntax"); + expect(entry).toBeDefined(); + expect(entry?.description).toBe(DESCRIPTION); + expect(entry?.forms).toEqual(["self-closing"]); + expect(entry?.returnMode).toBe("text"); + // One optional prop, closed: `names` selects documentation, and anything + // else is refused before an reference. + expect(entry?.props).toEqual({ + type: "object", + properties: { + names: { + type: "array", + items: { type: "string" }, + minItems: 1, + uniqueItems: true, + description: + "Optional. Render these components' metadata and long-form documentation " + + "instead of the list of available symbols. Entries render once each, in symbol order.", + }, + }, + additionalProperties: false, + }); + expect(entry?.origin).toEqual({ kind: "protected", origin: "@executablemd/core" }); + // Exactly one entry, in exactly one category. + const everywhere = symbols.categories.flatMap((category) => + category.entries.filter((candidate) => candidate.name === "Syntax"), + ); + expect(everywhere.length).toBe(1); + + const described = yield* inspectComponent({ name: "Syntax", includes: [] }); + expect(described.kind).toBe("protected"); + + // Validation reads the same declaration, so a paired spelling is invalid + // before anything runs and the self-closing one is valid. + const bad = yield* validateDocumentStructure({ + ...retainedSource("", "content\n"), + includes: [], + }); + expect(bad.diagnostics.some((issue) => issue.code === "invocation-form-invalid")).toBe(true); + const good = yield* validateDocumentStructure({ + ...retainedSource("", "\n"), + includes: [], + }); + expect(good.diagnostics).toEqual([]); + }); + + /** + * The seam a trusted evaluation boundary narrows through. + * + * `` admits an exact vocabulary before it expands a generated + * fragment, and the reference it installs for that subtree is that + * admission's own symbols — it cannot add an entry the admission does not + * hold, because it is handed the symbols rather than asked to build one. + * Installing it for an evaluation subtree is #713's; that the reference is + * the symbols and nothing more is this. + */ + it("SYN25b: a narrowed reference answers with exactly the symbols it was given", function* () { + const narrowed = symbolsOf("Admitted"); + const reference = syntaxReference(narrowed); + expect(yield* reference.symbols()).toBe(renderSyntaxMarkdown(narrowed)); + // Nothing of the enclosing site leaks into it: a name the wider profile has + // is absent, because the symbols it was handed do not hold one. + expect(yield* reference.symbols()).not.toContain("### ``"); + }); + + /** + * The seam #713 installs through, proved without an ``. + * + * A narrowing boundary hands the reference two sets of symbols: what may execute + * in the subtree, and the enclosing authoring symbols selection reads from. + * Everything below is about them being genuinely two. + */ + it("SYN25c: a narrowed reference documents the enclosing site and marks availability", function* () { + const enclosing = symbolsOf("Admitted", "Withheld"); + const narrowed = symbolsOf("Admitted"); + const reference = syntaxReference(narrowed, enclosing); + + // What may execute here is the narrowed set, and the bare form reports + // exactly that. + const available = yield* reference.symbols(); + expect(available).toContain("### ``"); + expect(available).not.toContain("### ``"); + + // Reference material comes from the enclosing symbols, so a component this + // subtree may not run can still be explained — and the entry says so + // rather than leaving a reader to assume they have both. + const documented = yield* reference.documentation(["Withheld"]); + expect(documented).toContain("### ``"); + expect(documented).toContain("**Available in this evaluation:** no"); + + // And one that is admitted reports the other answer, so the field is + // discriminating rather than a constant. + const admitted = yield* reference.documentation(["Admitted"]); + expect(admitted).toContain("**Available in this evaluation:** yes"); + + // A boundary that narrows nothing has one set, and everything in it is + // available — the ordinary case. + const open = syntaxReference(enclosing); + expect(yield* open.documentation(["Withheld"])).toContain( + "**Available in this evaluation:** yes", + ); + }); + + it("SYN25d: availability compares the whole identity, not the spelling", function* () { + /** Symbols holding a single entry of exactly this identity. */ + const holding = (origin: NamedOrigin): SyntaxSymbols => ({ + version: 2, + categories: [ + { kind: "structural", entries: [] }, + { + kind: "built-in", + entries: [ + { + kind: "component" as const, + name: "Elicit", + origin, + sourceKind: "registered" as const, + inspectability: "complete" as const, + forms: ["self-closing" as const], + props: { type: "object", properties: {}, additionalProperties: false }, + captures: [], + returnMode: "text" as const, + returns: { type: "string" }, + }, + ], + }, + { kind: "user-provided", entries: [] }, + ], + }); + + const authored: NamedOrigin = { + kind: "registered", + origin: "@executablemd/core", + reserved: false, + }; + + // Each of these is a *different component* that happens to be spelled + // `Elicit`. Reporting the authoring entry as available because something of + // that name can run would tell an author they may execute what they were + // just shown. + const impostors: Record = { + "another registered origin": { + kind: "registered", + origin: "@someone/else", + reserved: false, + }, + "a reserved registration of the same origin": { + kind: "registered", + origin: "@executablemd/core", + reserved: true, + }, + "a repository file": { kind: "repository", path: "components/Elicit.md" }, + "a bundled blob": { + kind: "workflow", + path: "components/Elicit.md", + sourceHash: "a".repeat(40), + }, + "declared Markdown": { + kind: "declared-markdown", + origin: "@executablemd/core", + digest: "b".repeat(64), + }, + }; + + for (const [what, origin] of Object.entries(impostors)) { + const nested = syntaxReference(holding(origin), holding(authored)); + const rendered = yield* nested.documentation(["Elicit"]); + expect([what, rendered.includes("**Available in this evaluation:** no")]).toEqual([ + what, + true, + ]); + } + + // Two more of the same kind, differing only in the member that identifies + // them: a different blob under one path, and a different digest under one + // origin. + const bundled: NamedOrigin = { + kind: "workflow", + path: "components/Elicit.md", + sourceHash: "a".repeat(40), + }; + const moved: NamedOrigin = { ...bundled, sourceHash: "c".repeat(40) }; + expect( + yield* syntaxReference(holding(moved), holding(bundled)).documentation(["Elicit"]), + ).toContain("**Available in this evaluation:** no"); + + // The positive control: one exact identity, admitted. + expect( + yield* syntaxReference(holding(authored), holding(authored)).documentation(["Elicit"]), + ).toContain("**Available in this evaluation:** yes"); + }); + + it("SYN25e: a narrowed reference is derived from the enclosing one", function* () { + // The seam as an evaluator actually meets it: it holds the enclosing + // reference and an admitted set of symbols, and nothing else. No raw + // contribution list, no second index — which is the point, because that + // list is execution-private and rebuilding an index from it is how two + // indexes drift apart. + const enclosing = yield* rootObservation(); + const admitted = symbolsOf("Admitted"); + const narrowed = enclosing.available(admitted); + + // What may run is the admission. + const executable = yield* narrowed.symbols(); + expect(executable).toContain("### ``"); + expect(executable).not.toContain("### ``"); + + // What may be read about is still the enclosing site's, with the enclosing + // index behind it — so a real component's real documentation survives. + const documented = yield* narrowed.documentation(["Elicit"]); + expect(documented).toContain("### ``"); + expect(documented).toContain("Asks a person a structured question"); + expect(documented).toContain("**Available in this evaluation:** no"); + + // And the enclosing reference is unchanged by having been narrowed. + expect(yield* enclosing.symbols()).toContain("### ``"); + expect(yield* enclosing.documentation(["Elicit"])).toContain( + "**Available in this evaluation:** yes", + ); + }); + + it("SYN25f: contributions are captured, not held by reference", function* () { + const supplies = new Set(["Alpha"]); + const source = { + owner: "@executablemd/mutable", + asset: "packages/mutable/src/components.md", + text: "## Alpha\n\nThe captured documentation.\n", + }; + const contribution = { source, supplies }; + const reference = syntaxReference( + symbolsNamed("Alpha", { + kind: "registered", + origin: "@executablemd/mutable", + reserved: false, + }), + undefined, + [contribution], + ); + + // Everything a caller still holds, changed after capture. + source.text = "## Alpha\n\nSUBSTITUTED AFTER CAPTURE.\n"; + source.owner = "@executablemd/other"; + supplies.add("Beta"); + supplies.delete("Alpha"); + + const rendered = yield* reference.documentation(["Alpha"]); + expect(rendered).toContain("The captured documentation."); + expect(rendered).not.toContain("SUBSTITUTED AFTER CAPTURE"); + }); + + it("SYN25: an execution that carries no reference refuses rather than inventing one", function* () { + // `execute()` driven directly still carries one, so the case that has none + // is an expansion driven outside an execution — which is what a component + // reaching for symbols with nothing established would meet. + const output = String( + yield* collect( + yield* execute({ + ...retainedSource(ROOT_PATH, "\n"), + stream: new InMemoryStream(), + includes: [], + }), + ), + ); + // An ordinary `execute()` derives its own, so this is the positive control + // that the derived path needs no host at all. + expect(output).toContain("### ``"); + }); +}); diff --git a/packages/core/tests/syntax-loaded-copy.test.ts b/packages/core/tests/syntax-loaded-copy.test.ts new file mode 100644 index 000000000..a93e059f5 --- /dev/null +++ b/packages/core/tests/syntax-loaded-copy.test.ts @@ -0,0 +1,222 @@ +/** + * Tier SYN — a protected implementation from a second loaded copy answers for + * nothing. + * + * A component can be loaded from disk beside its own copy of core: that is what + * `--include` does, and what a middleware package holding its own copy is. So + * "another loaded copy cannot answer for ``" is a claim about ordinary + * arrangements rather than a hypothetical, and it has two halves. + * + * **The answer is refused.** Canonical execution issues a witness for the + * definition it produced and verifies it where the component is invoked, keyed + * by the object itself. A definition another copy built is not that object, + * whatever it looks like. + * + * **And a body is unreachable anyway.** The table a protected body lives in + * belongs to the execution that built the implementation, inside the copy that + * built it, so this execution holds no body for a function another copy's + * installation created. That half is what makes the refusal above a boundary + * rather than a single check. + * + * The separate copy is built with `deno bundle`, which is Deno's, so this file + * runs under Deno alone and is registered in the runtime exclusions. What it is + * about — the witness comparison and the private body table — is proved under + * all three runtimes by `syntax-component.test.ts` against handler-built + * answers; what is only provable here is that a *real* second copy is one of + * those answers. + */ + +import { describe, it } from "@executablemd/test-support/bdd"; +import { expect } from "@executablemd/test-support/expect"; +import { ensure, resource, scoped, until } from "effection"; +import type { Operation } from "effection"; +import { rm } from "@effectionx/fs"; +import { exec } from "@effectionx/process"; +import { InMemoryStream } from "@executablemd/durable-streams"; +import { mkdtemp } from "node:fs/promises"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { fileURLToPath } from "node:url"; +import process from "node:process"; + +import { Component } from "../src/component-api.ts"; +import { collect } from "../src/collect.ts"; +import { executeInstalled } from "../host.ts"; +import type { ExecutionInstallation } from "../host.ts"; +import { retainedSource } from "../src/root-source.ts"; +import { SYNTAX_COMPONENT, props as syntaxProps } from "../src/components/Syntax.ts"; +import type { FunctionComponentDefinition } from "../src/types.ts"; + +const PROTECTION_MODULE = fileURLToPath(new URL("../src/invocation-identity.ts", import.meta.url)); +const REPOSITORY = fileURLToPath(new URL("../../../", import.meta.url)); + +/** What the bundled copy exposes: its own installation, with its own tables. */ +interface LoadedCopy { + installIdentities( + components: readonly unknown[], + privateComponents: readonly unknown[], + protectedComponents: readonly unknown[], + ): { + protected: ReadonlyMap; + protectedBodies: { body(fn: unknown): unknown }; + activate(): void; + }; +} + +function isLoadedCopy(value: unknown): value is LoadedCopy { + return ( + typeof value === "object" && + value !== null && + typeof Reflect.get(value, "installIdentities") === "function" + ); +} + +/** + * `packages/core/src/invocation-identity.ts`, bundled and evaluated as its own + * module. + * + * The bundle is what makes the copy separate: importing the source path again + * would resolve to the module this test already holds, and share the private + * body table with it. The declaration handed to it below is this test's, because + * the component module itself does not bundle; what is genuinely the second + * copy's is what decides — the implementation wrapper it built and the table its + * body lives in. + */ +function useSeparateCopy(): Operation { + return resource(function* (provide) { + const directory = yield* until(mkdtemp(join(tmpdir(), "sl-syntax-"))); + yield* ensure(() => rm(directory, { recursive: true, force: true })); + const bundle = join(directory, "protection.js"); + + // `process.execPath` under Deno is the deno binary, so the driver stays + // typed against node:process rather than a runtime global. + const built = yield* exec(process.execPath, { + arguments: [ + "bundle", + "--frozen", + "--node-modules-dir=none", + PROTECTION_MODULE, + "--output", + bundle, + ], + cwd: REPOSITORY, + }).join(); + if (built.code !== 0) { + throw new Error(`could not bundle the identity module:\n${built.stdout}${built.stderr}`); + } + + const loaded: unknown = yield* until(import(`file://${bundle}`)); + if (!isLoadedCopy(loaded)) { + throw new Error("the bundled copy does not expose the protection surface"); + } + yield* provide(loaded); + }); +} + +/** + * A handler that delegates the protected import and then answers with something + * else. + * + * Delegating first is the strongest form: this is a handler that saw canonical + * execution's own answer, not one that skipped the chain. + */ +function answering(definition: unknown): ExecutionInstallation { + return { + *install() { + yield* Component.around( + { + *importComponent([name, position], next) { + if (name !== SYNTAX_COMPONENT) { + return yield* next(name, position); + } + yield* next(name, position); + return definition as FunctionComponentDefinition; + }, + }, + { at: "max" }, + ); + }, + }; +} + +function runRoot(installations: readonly ExecutionInstallation[]): Operation { + return scoped(function* () { + return yield* collect( + yield* executeInstalled( + { + ...retainedSource("documents/root.md", "\n"), + stream: new InMemoryStream(), + includes: [], + }, + [...installations], + ), + ); + }); +} + +function* refusal(operation: Operation): Operation { + try { + yield* operation; + } catch (error) { + return error instanceof Error ? error.message : String(error); + } + throw new Error("expected the operation to be refused"); +} + +describe("Tier SYN — a separately loaded protected implementation", () => { + it("SYN26: an implementation another copy built answers for nothing here", function* () { + const copy = yield* useSeparateCopy(); + + // An installation the other copy performed: it minted the domain, built the + // implementation and kept the body in its own table. Nothing here is a fake + // — this is that copy's real protected-component path. + const installed = copy.installIdentities( + [], + [], + [ + { + name: SYNTAX_COMPONENT, + origin: "@executablemd/core", + props: syntaxProps, + forms: ["self-closing"], + // deno-lint-ignore require-yield + build: () => + // deno-lint-ignore require-yield + function* (): Operation { + return "a foreign catalog"; + }, + }, + ], + ); + installed.activate(); + const foreign = installed.protected.get(SYNTAX_COMPONENT); + if (foreign === undefined) { + throw new Error("the bundled copy built no protected implementation"); + } + // The premise, stated as a fact rather than assumed: that copy holds a body + // for its own implementation. + expect(installed.protectedBodies.body(foreign.fn)).toBeDefined(); + + const refused = yield* refusal(runRoot([answering(foreign)])); + expect(refused).toContain("canonical core owns"); + + // The positive control, in the same shape: a handler that delegates and + // answers with what came back runs the canonical component. + const output = yield* runRoot([ + { + *install() { + yield* Component.around( + { + *importComponent([name, position], next) { + return yield* next(name, position); + }, + }, + { at: "max" }, + ); + }, + }, + ]); + expect(String(output)).toContain("### ``"); + expect(String(output)).not.toContain("a foreign catalog"); + }); +}); diff --git a/packages/runtime/launcher.ts b/packages/runtime/launcher.ts index dc875fad0..2a11e374d 100644 --- a/packages/runtime/launcher.ts +++ b/packages/runtime/launcher.ts @@ -28,7 +28,8 @@ */ import { type Api, createApi } from "@effectionx/context-api"; -import { ensure, race, resource, scoped, until, withResolvers } from "effection"; +import { ensure, race, resource, scoped, until } from "effection"; +import { once } from "@effectionx/node/events"; import type { Operation } from "effection"; import { spawn as spawnChild } from "node:child_process"; import type { ChildProcess } from "node:child_process"; @@ -219,8 +220,6 @@ function runForeground(request: NativeLaunchRequest): Operation(); - const failed = withResolvers(); let child: ChildProcess | undefined; // Interrupt, then insist. A cancelled document may not continue — or @@ -233,31 +232,44 @@ function runForeground(request: NativeLaunchRequest): Operation failed.reject(error)); - child.once("exit", (code: number | null, signal: string | null) => { - const outcome: NativeLaunchOutcome = {}; - if (code !== null) { - outcome.exitCode = code; - } - if (signal !== null) { - outcome.signal = signal; - } - settled.resolve(outcome); - }); - - return yield* race([settled.operation, failed.operation]); + child = started; + + // Raced inline, in the same synchronous run as the spawn, so both arms are + // attached before the child can report anything — a spawned race attaches + // a turn later. Whichever loses is halted, which is what detaches it. + return yield* race([ + (function* (): Operation { + const [code, signal] = yield* once<[number | null, string | null]>(started, "exit"); + const outcome: NativeLaunchOutcome = {}; + if (code !== null) { + outcome.exitCode = code; + } + if (signal !== null) { + outcome.signal = signal; + } + return outcome; + })(), + (function* (): Operation { + const [error] = yield* once<[Error]>(started, "error"); + throw error; + })(), + ]); }); } /** * End one foreground child and wait for it to be gone. * + * Exported for `packages/runtime/tests/native-launcher.test.ts` and not from + * `mod.ts`: the listener this installs belongs to a bounded Promise, and the + * only way to observe that it is released on every settlement path is to hold + * the child. + * * Deliberately one promise rather than an Effection race: this runs while the * scope is already being dismantled, and the cheapest correct thing to do * there is to wait on the process's own events instead of starting more @@ -267,7 +279,7 @@ function runForeground(request: NativeLaunchRequest): Operation { +export function reap(child: ChildProcess): Promise { const pid = child.pid; if (pid === undefined || child.exitCode !== null || child.signalCode !== null) { return Promise.resolve(); @@ -290,6 +302,10 @@ function reap(child: ChildProcess): Promise { clearInterval(poll); clearTimeout(escalation); clearTimeout(deadline); + // The one funnel every settlement goes through — the exit event, the + // reachability poll, the escalation deadline, and the refusal that + // rejects — so the handler comes off however this ends. + child.off("exit", onExit); // Deno's `node:child_process` stops reporting a child's exit once a // signal that child ignored has been delivered, and holds the runtime // open on the handle it will now never settle. Dropping the reference is @@ -306,8 +322,9 @@ function reap(child: ChildProcess): Promise { } resolve(); }; + const onExit = (): void => done(); - child.once("exit", () => done()); + child.on("exit", onExit); // Reachability rather than the exit event, because that is the fact this // has to establish and the event is not dependable across runtimes here. const poll = setInterval(() => { diff --git a/packages/runtime/package.json b/packages/runtime/package.json index 97bf87c20..98d09a957 100644 --- a/packages/runtime/package.json +++ b/packages/runtime/package.json @@ -12,7 +12,7 @@ "@effectionx/context-api": "0.6.0", "@effectionx/fetch": "0.2.1", "@effectionx/fs": "0.3.0", - "@effectionx/node": "0.2.4", + "@effectionx/node": "0.2.5", "@effectionx/process": "0.8.1", "effection": "4.1.0" } diff --git a/packages/runtime/tests/fetch.test.ts b/packages/runtime/tests/fetch.test.ts index c0798f023..6227d4985 100644 --- a/packages/runtime/tests/fetch.test.ts +++ b/packages/runtime/tests/fetch.test.ts @@ -19,7 +19,7 @@ import { ensure, resource, scoped, sleep, spawn, withResolvers } from "effection import type { Operation } from "effection"; import { when } from "@effectionx/converge"; import { createServer } from "node:http"; -import type { IncomingMessage, ServerResponse } from "node:http"; +import type { IncomingMessage, Server, ServerResponse } from "node:http"; import { API, fetch } from "../apis.ts"; import type { RuntimeFetchResponse } from "../apis.ts"; import { Config } from "../config.ts"; @@ -27,6 +27,8 @@ import { Config } from "../config.ts"; interface Loopback { /** Where the server is listening. */ readonly origin: string; + /** The listener itself, so a case can say what it is still observing. */ + readonly server: Server; /** One entry per request the server accepted, in order. */ readonly requests: Array<{ method: string; path: string; headers: Record }>; } @@ -48,7 +50,23 @@ function useLoopback( }); const listening = withResolvers(); - server.on("error", (error: Error) => listening.reject(error)); + // Removed with the resource rather than after the first error: a listening + // server outlives its bind, and a handler left behind would still be + // holding a rejected resolver when the next test binds its own. + // + // Established before the handler exists, because `yield* ensure(...)` is + // itself a suspension: an owner halted while it registers unwinds with no + // cleanup at all, so nothing may be attached until it has completed. + let onError: ((error: Error) => void) | undefined; + + yield* ensure(() => { + if (onError) { + server.off("error", onError); + } + }); + + onError = (error: Error) => listening.reject(error); + server.on("error", onError); server.listen(0, "127.0.0.1", () => listening.resolve()); yield* listening.operation; @@ -66,7 +84,7 @@ function useLoopback( throw new Error("the loopback server reported no TCP address"); } - yield* provide({ origin: `http://127.0.0.1:${address.port}`, requests }); + yield* provide({ origin: `http://127.0.0.1:${address.port}`, requests, server }); }); } @@ -359,4 +377,47 @@ describe("Tier FR — the chainable calling shape", () => { expect(failure?.message).toContain("timed out after 60ms"); }); + /** + * The listener's error observer belongs to the loopback resource, not to the + * first error it sees. The count is read after the resource has been torn + * down and before the event is replayed, because an observer removed by its + * own event would leave the same count behind as one that was released. + */ + it("releases the loopback's error observer with the resource", function* () { + let observed: Server | undefined; + let live = 0; + let baseline = 0; + + yield* scoped(function* () { + const loopback = yield* useLoopback((_request, response) => response.end("{}")); + observed = loopback.server; + baseline = 0; + live = loopback.server.listenerCount("error"); + }); + + if (!observed) { + throw new Error("the loopback never came up"); + } + + expect(live).toBeGreaterThanOrEqual(baseline + 1); + expect(observed.listenerCount("error")).toBe(live - 1); + + // Delivered through a sentinel of this case's own, because an `error` an + // emitter has no listener for is thrown rather than dropped — so the + // replay needs one observer, and exactly one is what it must find. + let seen = 0; + const sentinel = (): void => { + seen += 1; + }; + + observed.on("error", sentinel); + try { + observed.emit("error", new Error("after the loopback was torn down")); + } finally { + observed.off("error", sentinel); + } + + expect(seen).toBe(1); + expect(observed.listenerCount("error")).toBe(live - 1); + }); }); diff --git a/packages/runtime/tests/native-launcher.test.ts b/packages/runtime/tests/native-launcher.test.ts index cd6bc1b53..c39546b73 100644 --- a/packages/runtime/tests/native-launcher.test.ts +++ b/packages/runtime/tests/native-launcher.test.ts @@ -22,11 +22,13 @@ import { randomUUID } from "node:crypto"; import * as path from "node:path"; import * as os from "node:os"; import process from "node:process"; +import { spawn as spawnChild } from "node:child_process"; import { flushOutput, installForegroundLauncher, nativeLaunch, NO_TERMINAL, + reap, reserveTerminal, } from "../launcher.ts"; @@ -239,6 +241,60 @@ describe("Tier FL — the foreground native launcher", () => { }); }); +describe("native launcher — the reaper's own listener", () => { + /** + * The reaper waits on the child's `exit` from inside a bounded Promise, so + * its handler is not covered by an Effection scope: `done()` is the only + * funnel out — the event itself, the reachability poll, the escalation + * deadline, and the refusal that rejects — and it is where the handler comes + * off. The count is read after the reap has settled and before the event is + * replayed, because a handler that removed itself on `exit` would leave the + * same count behind as one that was released. + */ + it("NLR1: releases the exit handler when the reap settles, and a later exit changes nothing", function* () { + const dir = yield* useTempDir(); + // Deliberately deaf to the interrupt, so the reap is still in flight while + // its handler is counted, and settles through the escalation rather than + // through the event — which is the path a self-removing listener would not + // have been released by. + const fake = yield* useFake(dir, "stubborn", { ignoreInterrupt: true, hang: true }); + const child = spawnChild(fake.command, [], { stdio: "ignore" }); + const before = child.listenerCount("exit"); + + // Not spawned: a Promise executor runs synchronously, so the handler is + // attached by the time `reap` has returned, and the count below is read + // with the reap unambiguously in flight rather than a turn after it. + const reaping = reap(child); + + // At least one more, not exactly one: a runtime may hold handlers of its + // own on this source, so the release below is measured against what was + // live rather than against the baseline. + const live = child.listenerCount("exit"); + expect(live).toBeGreaterThanOrEqual(before + 1); + + yield* until(reaping); + + expect(child.listenerCount("exit")).toBe(live - 1); + + child.emit("exit", 0, null); + + expect(child.listenerCount("exit")).toBe(live - 1); + }); + + /** A child already gone is answered without observing anything at all. */ + it("NLR2: installs nothing for a child that has already exited", function* () { + const dir = yield* useTempDir(); + const fake = yield* useFake(dir, "brief", { exitCode: 0 }); + const child = spawnChild(fake.command, [], { stdio: "ignore" }); + const before = child.listenerCount("exit"); + + yield* until(reap(child)); + yield* until(reap(child)); + + expect(child.listenerCount("exit")).toBe(before); + }); +}); + /** How many times the fake has beaten, or zero before its first beat. */ function* beats(file: string): Operation { try { diff --git a/packages/test-agent/deno.json b/packages/test-agent/deno.json index 9d9927717..4bd84d35d 100644 --- a/packages/test-agent/deno.json +++ b/packages/test-agent/deno.json @@ -6,7 +6,7 @@ }, "imports": { "@agentclientprotocol/sdk": "npm:@agentclientprotocol/sdk@1.3.0", - "@effectionx/node": "npm:@effectionx/node@0.2.4", + "@effectionx/node": "npm:@effectionx/node@0.2.5", "@effectionx/scope-eval": "npm:@effectionx/scope-eval@0.1.3", "@effectionx/stream-helpers": "npm:@effectionx/stream-helpers@0.8.3", "acorn": "npm:acorn@^8.16.0", diff --git a/packages/test-agent/package.json b/packages/test-agent/package.json index 8a5368c52..856665edf 100644 --- a/packages/test-agent/package.json +++ b/packages/test-agent/package.json @@ -8,7 +8,7 @@ }, "dependencies": { "@agentclientprotocol/sdk": "1.3.0", - "@effectionx/node": "0.2.4", + "@effectionx/node": "0.2.5", "@effectionx/scope-eval": "0.1.3", "@effectionx/stream-helpers": "0.8.3", "@executablemd/acp": "workspace:*", diff --git a/packages/test-agent/src/net.ts b/packages/test-agent/src/net.ts index 149f9ff6b..7fbb3aef9 100644 --- a/packages/test-agent/src/net.ts +++ b/packages/test-agent/src/net.ts @@ -9,6 +9,7 @@ import { createChannel, each, ensure, race, resource, spawn, withResolvers } from "effection"; import type { Operation, Stream, Task } from "effection"; import { fromReadable, on } from "@effectionx/node"; +import { once } from "@effectionx/node/events"; import { lines } from "@effectionx/stream-helpers"; import { connect, createServer } from "node:net"; import type { Socket } from "node:net"; @@ -34,21 +35,29 @@ export interface LineSocket { export function useLineSocket(socket: Socket): Operation { return resource(function* (provide) { const closed = withResolvers(); - socket.once("close", () => closed.resolve()); - yield* ensure(() => { + const onClose = (): void => closed.resolve(); + + // A lexical finalizer rather than `ensure()`: entering the `try` is + // synchronous, so there is no instant at which this socket is observed and + // the release is not yet armed. Detached first, because the destroy below + // emits the event this was observing. + try { + socket.on("close", onClose); + yield* provide({ + lines: lines()(fromReadable(socket)), + send(line) { + socket.write(line); + }, + end() { + socket.end(); + }, + closed: closed.operation, + }); + } finally { + socket.off("close", onClose); socket.destroy(); closed.resolve(); // settle on cancellation even if no 'close' follows - }); - yield* provide({ - lines: lines()(fromReadable(socket)), - send(line) { - socket.write(line); - }, - end() { - socket.end(); - }, - closed: closed.operation, - }); + } }); } @@ -81,24 +90,35 @@ export function useLineServer( } // Attach the close listener before close() (no missed-event race), and // only when the server actually came up, so a never-started or - // already-closed server leaves no pending operation. + // already-closed server leaves no pending operation. It stays attached + // through the wait — it is what the wait is for — and comes off + // synchronously afterwards, whether that wait settled or was halted. if (server.listening) { const closed = withResolvers(); - server.once("close", () => closed.resolve()); - server.close(); - yield* closed.operation; + const onClose = (): void => closed.resolve(); + + server.on("close", onClose); + try { + server.close(); + yield* closed.operation; + } finally { + server.off("close", onClose); + } } }); - // Attach the readiness listeners before listen, so the event is never - // missed by a later subscription. - const listening = withResolvers(); - server.once("listening", () => listening.resolve()); - server.once("error", (error) => { - listening.reject(error instanceof Error ? error : new Error(String(error))); - }); + // Raced inline, in the same synchronous run as `listen`, so both arms are + // attached before either event can be delivered — `listen` never emits in + // the turn it was called in, and a spawned race would attach a turn late. + // The loser is halted, which is what detaches it. server.listen(0, host); - yield* listening.operation; + yield* race([ + once(server, "listening"), + (function* (): Operation { + const [error] = yield* once(server, "error"); + throw error instanceof Error ? error : new Error(String(error)); + })(), + ]); const address = server.address(); if (!address || typeof address !== "object") { @@ -140,14 +160,24 @@ export function useLineClient( ): Operation> { return resource(function* (provide) { const socket = connect(port, host); - // Attach connect/error listeners before yielding, so neither is missed. - const connected = withResolvers(); - socket.once("connect", () => connected.resolve()); - socket.once("error", (error) => { - connected.reject(error instanceof Error ? error : new Error(String(error))); + // Owned before the handshake is awaited: a connect that fails has to leave + // no socket behind, and `useLineSocket` cannot take ownership of one until + // the handshake has settled. + yield* ensure(() => { + socket.destroy(); }); + + // Raced inline, in the same synchronous run as `connect`, for the reason + // given in `useLineServer`. + yield* race([ + once(socket, "connect"), + (function* (): Operation { + const [error] = yield* once(socket, "error"); + throw error instanceof Error ? error : new Error(String(error)); + })(), + ]); + const connection = yield* useLineSocket(socket); - yield* connected.operation; const inbound = createChannel(); const subscription = yield* inbound; // subscribe before the pump: no loss diff --git a/packages/test-agent/src/worker/acp-server.ts b/packages/test-agent/src/worker/acp-server.ts index 0b2a14681..e8dddf16e 100644 --- a/packages/test-agent/src/worker/acp-server.ts +++ b/packages/test-agent/src/worker/acp-server.ts @@ -222,17 +222,21 @@ export function useProcessStdio(): Operation { // already settled } }; - process.stdin.on("data", onData); - process.stdin.on("end", onEnd); - process.stdin.on("close", onClose); - process.stdin.on("error", onError); - yield* ensure(() => { + // A lexical finalizer rather than `ensure()`: entering the `try` is + // synchronous, so there is no instant at which standard input is observed + // and the release is not yet armed. + try { + process.stdin.on("data", onData); + process.stdin.on("end", onEnd); + process.stdin.on("close", onClose); + process.stdin.on("error", onError); + + yield* provide({ input, output }); + } finally { process.stdin.off("data", onData); process.stdin.off("end", onEnd); process.stdin.off("close", onClose); process.stdin.off("error", onError); - }); - - yield* provide({ input, output }); + } }); } diff --git a/packages/test-agent/tests/cross-package-resolution.test.ts b/packages/test-agent/tests/cross-package-resolution.test.ts index dca00fa0a..235ca324d 100644 --- a/packages/test-agent/tests/cross-package-resolution.test.ts +++ b/packages/test-agent/tests/cross-package-resolution.test.ts @@ -200,8 +200,12 @@ function describeOrigin(info: ComponentInfo): string { return `structural:${info.origin.construct}`; case "registered": return `registered:${info.origin.origin}${info.origin.reserved ? " (reserved)" : ""}`; + case "protected": + return `protected:${info.origin.origin}`; case "repository": return `repository:${info.origin.path}`; + case "workflow": + return `workflow:${info.origin.path}@${info.origin.sourceHash}`; case "declared-markdown": return `declared-markdown:${info.origin.origin}`; } diff --git a/packages/test-agent/tests/net.test.ts b/packages/test-agent/tests/net.test.ts index bcff1c324..97078f4fc 100644 --- a/packages/test-agent/tests/net.test.ts +++ b/packages/test-agent/tests/net.test.ts @@ -6,11 +6,11 @@ */ import { describe, it } from "@executablemd/test-support/bdd"; import { expect } from "@executablemd/test-support/expect"; -import { each, ensure, race, scoped, spawn, suspend, withResolvers } from "effection"; +import { each, ensure, race, scoped, sleep, spawn, suspend, withResolvers } from "effection"; import type { Operation } from "effection"; import { once } from "@effectionx/node"; import { connect } from "node:net"; -import { useLineClient, useLineServer } from "../src/net.ts"; +import { useLineClient, useLineServer, useLineSocket } from "../src/net.ts"; import type { LineSocket } from "../src/net.ts"; describe("Tier NR — line-socket adapter", () => { @@ -93,4 +93,52 @@ describe("Tier NR — line-socket adapter", () => { ]); expect(outcome).toBe("refused"); }); + + /** + * The socket adapter's own listener, rather than what it provides. + * + * A cancelled acquisition is the case the event never comes for, so the + * handler cannot be waiting for it to leave. What proves that is the count + * on the socket itself: one while the resource is live, back to where it + * started once the owner is halted, and unmoved by a `close` afterwards — + * a listener that removed itself when the event finally arrived would leave + * the same count behind as one that was never there. + */ + it("NR4: halting a line socket detaches it, and a later close changes nothing", function* () { + const server = yield* useLineServer("127.0.0.1", function* () { + yield* suspend(); + }); + + const socket = connect(server.port, "127.0.0.1"); + const connected = withResolvers(); + const onConnect = (): void => connected.resolve(); + + socket.on("connect", onConnect); + try { + yield* connected.operation; + } finally { + socket.off("connect", onConnect); + } + + const before = socket.listenerCount("close"); + const owner = yield* spawn(function* () { + yield* useLineSocket(socket); + yield* suspend(); + }); + yield* sleep(0); + + // At least one more, not exactly one: a runtime may hold handlers of its + // own on this source, so the release below is measured against what was + // live rather than against the baseline. + const live = socket.listenerCount("close"); + expect(live).toBeGreaterThanOrEqual(before + 1); + + yield* owner.halt(); + + expect(socket.listenerCount("close")).toBe(live - 1); + + socket.emit("close"); + + expect(socket.listenerCount("close")).toBe(live - 1); + }); }); diff --git a/packages/test-agent/tests/public-api.test.ts b/packages/test-agent/tests/public-api.test.ts index 082bd9832..57f4dcdeb 100644 --- a/packages/test-agent/tests/public-api.test.ts +++ b/packages/test-agent/tests/public-api.test.ts @@ -7,10 +7,10 @@ */ import { describe, it } from "@executablemd/test-support/bdd"; import { expect } from "@executablemd/test-support/expect"; -import { ensure, scoped, withResolvers } from "effection"; +import { ensure, race, scoped } from "effection"; import type { Operation } from "effection"; import { connect } from "node:net"; -import { once } from "@effectionx/node"; +import { once } from "@effectionx/node/events"; import * as os from "node:os"; import { useTestAgentController } from "../mod.ts"; import { parseRoute } from "../src/protocol.ts"; @@ -26,16 +26,24 @@ function* reachable(route: string): Operation { // The socket is closed, and its close observed, before the probe answers, // so a later probe never races the previous connection's teardown. return yield* scoped(function* () { - const settled = withResolvers(); const socket = connect({ host: parsed.value.host, port: parsed.value.port }); yield* ensure(function* () { socket.destroy(); yield* once(socket, "close"); }); - socket.once("connect", () => settled.resolve(true)); - socket.once("error", () => settled.resolve(false)); - return yield* settled.operation; + // Raced inline, in the same synchronous run as `connect`: a spawned race + // attaches its arms a turn later, and the socket can settle in that turn. + return yield* race([ + (function* (): Operation { + yield* once(socket, "connect"); + return true; + })(), + (function* (): Operation { + yield* once(socket, "error"); + return false; + })(), + ]); }); } diff --git a/packages/test-support/launch.ts b/packages/test-support/launch.ts index 49bc4f01f..c7cfc37ab 100644 --- a/packages/test-support/launch.ts +++ b/packages/test-support/launch.ts @@ -92,6 +92,16 @@ export interface CliRunOptions { * of file rather than nothing at all. */ stdin?: string; + /** + * Handed the child this launcher owns, before anything is attached to it, + * together with a reader for what has been captured from it so far. + * + * Package-private, for the cancellation regression: the listeners are on a + * value this operation owns and does not otherwise hand out, and what a + * cancelled run must prove is that they are gone *and* that a later chunk + * reaches nothing that is still accumulating. + */ + observeChild?: (child: ChildProcess, captured: () => { stdout: string; stderr: string }) => void; } /** A bounded run of `xmd`, synchronized like any `@effectionx/process` exec. */ @@ -222,44 +232,78 @@ function* withInput( input: string, ): Operation { const settled = withResolvers>(); - const closed = withResolvers(); - const child = spawnChild(launch.command, launch.arguments ?? [], { - detached: true, - shell: launch.shell, - cwd: options.cwd, - env: cliEnv(options), - stdio: "pipe", - }); - - child.stdout?.on("data", (chunk: Uint8Array) => { + // `close`, and nothing else, is what says this child and its pipes are done. + let closed = false; + // Declared before the cleanup below and assigned after it: a child that + // exists before its release is registered can be stranded, because `yield* + // ensure(...)` is itself a suspension and an owner halted while it registers + // unwinds with nothing on it. + let child: ChildProcess | undefined; + + const onStdout = (chunk: Uint8Array): void => { partial.stdout += text(chunk); - }); - child.stderr?.on("data", (chunk: Uint8Array) => { + }; + const onStderr = (chunk: Uint8Array): void => { partial.stderr += text(chunk); - }); + }; // A child that never started closes through this rather than through `close`, // so teardown has an end either way. - child.on("error", (error: Error) => { + const onError = (error: Error): void => { settled.resolve(Err(error)); - closed.resolve(); - }); + }; // A pipe the child could not use is the launcher's problem and not the run's: // the close below is what this whole path exists for, and a broken one would // otherwise raise on a process that is already reporting why. - child.stdin?.on("error", () => {}); - child.on("close", (code: number | null, signal: string | null) => { + const onStdinError = (): void => {}; + const onClose = (code: number | null, signal: string | null): void => { + closed = true; settled.resolve( Ok({ ...(code === null ? {} : { code }), ...(signal === null ? {} : { signal }), }), ); - closed.resolve(); + }; + + // Established before the child exists, so a run cancelled anywhere below + // still ends with its process group gone — and so that no instant exists in + // which a child is running with no cleanup registered for it. + // + // Teardown keeps every handler attached through the reap: `close` is what + // says the process is finished, and the capture must still be reading what + // the child writes on its way out. They come off synchronously once that + // wait has settled, whichever way it did. + yield* ensure(function* () { + if (child === undefined) { + return; + } + + try { + yield* reap(child, () => closed); + } finally { + child.stdout?.off("data", onStdout); + child.stderr?.off("data", onStderr); + child.off("error", onError); + child.stdin?.off("error", onStdinError); + child.off("close", onClose); + } + }); + + child = spawnChild(launch.command, launch.arguments ?? [], { + detached: true, + shell: launch.shell, + cwd: options.cwd, + env: cliEnv(options), + stdio: "pipe", }); - // Registered before the first suspension point, so a run cancelled anywhere - // below still ends with its process group gone. - yield* ensure(() => reap(child, closed.operation)); + options.observeChild?.(child, () => ({ stdout: partial.stdout, stderr: partial.stderr })); + + child.stdout?.on("data", onStdout); + child.stderr?.on("data", onStderr); + child.on("error", onError); + child.stdin?.on("error", onStdinError); + child.on("close", onClose); child.stdin?.end(input); @@ -289,24 +333,38 @@ function* withInput( * could be delivered to while the child is still reachable ends the run with * that fact rather than waiting on a `close` that is never coming. */ -function* reap(child: ChildProcess, closed: Operation): Operation { - const pid = child.pid; - if (pid === undefined || child.exitCode !== null || child.signalCode !== null) { - yield* closed; - return; - } +function* reap(child: ChildProcess, hasClosed: () => boolean): Operation { + const gone = withResolvers(); + const onGone = (): void => gone.resolve(); + + // `close`, and nothing else. An assigned `exitCode` or `signalCode` says the + // process ended; it does not say the pipes this run inherited have, and the + // capture is still reading them. + child.on("close", onGone); + try { + if (hasClosed()) { + return; + } - end(pid, "SIGTERM"); - const graceful = yield* timebox(TERMINATION_GRACE, () => closed); - if (!graceful.timeout) { - return; - } + const pid = child.pid; + + if (pid !== undefined) { + end(pid, "SIGTERM"); + const graceful = yield* timebox(TERMINATION_GRACE, () => gone.operation); + if (!graceful.timeout) { + return; + } + + const killed = end(pid, "SIGKILL"); + if (killed === "refused" && isReachable(pid)) { + throw new Error(`the launched process ${pid} could not be stopped: SIGKILL was refused`); + } + } - const killed = end(pid, "SIGKILL"); - if (killed === "refused" && isReachable(pid)) { - throw new Error(`the launched process ${pid} could not be stopped: SIGKILL was refused`); + yield* gone.operation; + } finally { + child.off("close", onGone); } - yield* closed; } /** What one signal delivery established about what it was aimed at. */ diff --git a/packages/testing/mod.ts b/packages/testing/mod.ts index b858f7203..09fb77e81 100644 --- a/packages/testing/mod.ts +++ b/packages/testing/mod.ts @@ -60,7 +60,12 @@ export { Test, testing, record, results, TestFailureError } from "./src/test-api.ts"; export type { TestApi, TestResult, BoundaryOutcome } from "./src/test-api.ts"; -export { installTestingComponents, TESTING_REGISTRATIONS } from "./src/components.ts"; +export { + installTestingComponents, + TESTING_REGISTRATIONS, + testingDocumentation, + useTestingComponents, +} from "./src/components.ts"; export { useTesting } from "./src/use-testing.ts"; // The nested-execution harness. This package owns the authored components and // the request-only host-profile surface; the trusted answer is attached to the diff --git a/packages/testing/src/components.md b/packages/testing/src/components.md new file mode 100644 index 000000000..dd2feb1f6 --- /dev/null +++ b/packages/testing/src/components.md @@ -0,0 +1,284 @@ +Long-form documentation for the components the testing package registers. + +Two components, and they work together: `` turns testing on for a +region, and `` states that a piece of work is *supposed* to fail. +The cases themselves are written with core's ``, which is inert until one +of these — or `xmd test` — asks for them. + +## Testing + +Turns on testing for its content. + +```mdx + + +… + + +``` + +Runs the `` elements inside and reports them. Two things fail the +document: a failing test, and finding **no tests at all** inside the region. +The second is deliberate — a testing region that silently matched nothing is +indistinguishable from one that passed, and a document whose tests stopped being +discovered should say so rather than go green. + +`` outside a testing region is skipped, so a document can carry its tests +and still be run as an ordinary document. + +## AssertThrows + +Asserts that its content fails. + +```mdx + + + +``` + +Passes only if the content fails *and* the failure message matches. `message` is +required and takes a string or a regular expression — required rather than +optional because an assertion that any failure will do passes for the wrong +reason as readily as the right one, and the message is what tells the two apart. + +Use it to pin a refusal you rely on: that a bad path fails, that an invalid +schema is rejected, that a guard actually guards. + +## Assert + +Asserts that a value is truthy. + +```mdx + +``` + +The base assertion. `msg` replaces the reported failure message, which is worth +writing whenever the expression alone would not tell a reader what went wrong. + +A passing assertion renders a diagnostic report only while testing or verbose +output is on, so an assertion inside an ordinary run is silent when it holds. A +failing one does not return. + +## AssertFalse + +Asserts that a value is falsy. + +```mdx + +``` + +The complement of ``. Prefer it over asserting `!value`, which reads as +a double negative at the point where a reader is trying to work out what should +be true. + +## AssertExists + +Asserts that a value is neither null nor undefined. + +```mdx + +``` + +Narrower than `` on purpose: `0`, `""` and `false` are all falsy and all +perfectly present, so an existence check written as a truthiness check fails on +legitimate values. + +## AssertEquals + +Asserts that two values are deeply equal. + +```mdx + +``` + +Deep equality, so objects and arrays compare by content rather than identity. +The expected value may also be written as content, which is easier to read when +it is large: + +```mdx + +{"name": "release", "steps": 3} + +``` + +## AssertNotEquals + +Asserts that two values are not deeply equal. + +```mdx + +``` + +Use it to pin that something actually changed — a step that is supposed to +rewrite a file, or a retry that should not return the first answer again. + +## AssertStrictEquals + +Asserts that two values are the same, compared with `===`. + +```mdx + +``` + +Identity rather than content. This is the assertion for "the same object", where +`` would pass for a copy. + +## AssertNotStrictEquals + +Asserts that two values are not the same, compared with `===`. + +```mdx + +``` + +The counterpart: pins that something was copied rather than shared, which is +what you want when a later step is going to mutate one of them. + +## AssertMatch + +Asserts that a string matches a pattern. + +```mdx + +``` + +For output whose exact text is not the contract — a message that carries a path, +a timestamp, or a count. Match the part that is the contract and leave the rest +free. + +## AssertNotMatch + +Asserts that a string does not match a pattern. + +```mdx + +``` + +Useful for the absence of something: that a rendering leaked no token, that a +diagnostic did not reach a user-facing surface. + +## AssertStringIncludes + +Asserts that a string contains a substring. + +```mdx + +``` + +The plainer form of `` when what you are looking for is literal +text rather than a shape. + +## AssertGreater + +Asserts that one number is greater than another. + +```mdx + +``` + +## AssertGreaterOrEqual + +Asserts that one number is greater than or equal to another. + +```mdx + +``` + +The inclusive form. Reach for it when the boundary value is acceptable — a +threshold that is met exactly is usually met. + +## AssertLess + +Asserts that one number is less than another. + +```mdx + +``` + +## AssertLessOrEqual + +Asserts that one number is less than or equal to another. + +```mdx + +``` + +## Execution + +Runs another document from inside a test, and asserts on how it finished. + +```mdx + + + +``` + +The child is real: its own journal, its own output, its own lifecycle. Pass +`source` instead of `target` to supply the Markdown directly, and `props` for +the child's properties. + +`as` binds the child's outcome — a settled result, or a suspension. Bind it: +**without `as`, a settled failure fails the owning test rather than passing +vacuously**, which is the safe direction but not usually what you meant to +write. The content is the child's declarations, then the assertions about it. + +## WorkflowRun + +Scopes a workflow-hosted execution. + +```mdx + + +… + + +``` + +A region that owns the workflow-hosted executions inside it, so a test that +drives a workflow has somewhere for that run's resources to belong and be torn +down. + +## DiagnosticJournal + +Gives a child execution a journal of its own. + +```mdx + + + + +``` + +Goes inside ``, before the assertions, and is **invalid anywhere +else**. Pair it with ``: this creates the journal, that reads +it. + +## CollectOutput + +Captures a child execution's output so a test can assert on it. + +```mdx + + + + +``` + +Goes inside ``, before the assertions, and is invalid anywhere else. +`as` is required. It changes nothing about the run, and a child that fails +partway still leaves what it printed — which is often exactly what the test +needs to see. + +## CollectJournal + +Captures a child execution's journal so a test can assert on it. + +```mdx + + + + +``` + +Placed like ``, and `as` is required. It reads a journal the run +already has — pair it with `` to create one. diff --git a/packages/testing/src/components.ts b/packages/testing/src/components.ts index 056ff628f..2b676695d 100644 --- a/packages/testing/src/components.ts +++ b/packages/testing/src/components.ts @@ -33,7 +33,9 @@ import { Err } from "effection"; import type { Operation } from "effection"; import { Component, + contributeDocumentation, documented, + packageDocumentation, registerComponents, Execution, TestActivation, @@ -42,6 +44,8 @@ import { import type { ComponentFailure, ComponentRegistration, + DocumentationContribution, + DocumentationReader, DocumentExecution, } from "@executablemd/core"; import { boundary, record, Test, testing, TestFailureError } from "./test-api.ts"; @@ -98,6 +102,37 @@ const TEST_TIMEOUT_MS = 20_000; * `` is deliberately absent: that construct is core's, and what this * package installs is what a test *does* (#441). */ +/** + * This package's long-form documentation, and the components it must cover. + * + * The set is derived from `TESTING_REGISTRATIONS` below, so adding a component + * to that array demands a section for it rather than quietly shipping one + * without. + */ +export function* testingDocumentation( + read?: DocumentationReader, +): Operation { + return yield* packageDocumentation( + new URL("./components.md", import.meta.url), + { owner: TESTING_ORIGIN, asset: "packages/testing/src/components.md" }, + TESTING_REGISTRATIONS.map((registration) => registration.name), + read, + ); +} + +/** + * This package's vocabulary, as declarations and nothing else. + * + * Registrations and the documentation that describes them, installed together + * so a scope that has one has the other. `xmd syntax` enters exactly this and + * stops: describing an environment installs no behavior chain, no activation + * guard and no execution middleware. + */ +export function* useTestingComponents(): Operation { + yield* registerComponents(TESTING_REGISTRATIONS); + yield* contributeDocumentation(testingDocumentation); +} + export const TESTING_REGISTRATIONS: readonly ComponentRegistration[] = [ // Non-reserved defaults: a repository component of any of these names is // chosen ahead of them, as it would be ahead of any other package's. @@ -224,7 +259,7 @@ export function* installHandlers( }, }); - yield* registerComponents(TESTING_REGISTRATIONS); + yield* useTestingComponents(); yield* Execution.around({ *execute([request], next) { // Fresh boundary collection per execution: outcomes reported by diff --git a/packages/web/mod.ts b/packages/web/mod.ts index 6ef87cd60..92277e524 100644 --- a/packages/web/mod.ts +++ b/packages/web/mod.ts @@ -14,7 +14,12 @@ * printed first and the form keeps waiting either way. */ -export { installWebComponents, WEB_REGISTRATIONS } from "./src/components.ts"; +export { + installWebComponents, + useWebComponents, + WEB_REGISTRATIONS, + webDocumentation, +} from "./src/components.ts"; export { installWebElicitation } from "./src/elicitation.ts"; export { liveForm } from "./src/live-form.ts"; export type { LiveFormInput } from "./src/live-form.ts"; diff --git a/packages/web/package.json b/packages/web/package.json index 54e333f6a..15caf8b37 100644 --- a/packages/web/package.json +++ b/packages/web/package.json @@ -7,7 +7,7 @@ ".": "./mod.ts" }, "dependencies": { - "@effectionx/node": "0.2.4", + "@effectionx/node": "0.2.5", "@executablemd/core": "workspace:*", "@executablemd/durable-streams": "workspace:*", "@executablemd/runtime": "workspace:*", diff --git a/packages/web/src/components.md b/packages/web/src/components.md new file mode 100644 index 000000000..852e23358 --- /dev/null +++ b/packages/web/src/components.md @@ -0,0 +1,26 @@ +Long-form documentation for the components the web package registers. + +One component. It answers the same question core's `` does — ask a +person something and validate the answer — and differs only in *where* the +asking happens. + +## WebForm + +Asks a person a question in a browser form. + +```mdx + +Choose how the release notes should be grouped. + +``` + +Builds the form from `schema` and shows its content above it. `uiSchema` sets +presentation options — ordering, widgets, labels — without changing what the +answer has to be. The validated response binds through `as`, which is required: +the answer is the point. + +Reach for `` instead when the document should not choose the browser. It +asks the same question and lets the host decide how — a terminal prompt, or +whatever else that host arranges. `` is for when the question genuinely +needs a form: several fields, a choice among many, anything awkward to type at a +prompt. diff --git a/packages/web/src/components.ts b/packages/web/src/components.ts index c7c90e85c..292e9db2c 100644 --- a/packages/web/src/components.ts +++ b/packages/web/src/components.ts @@ -11,14 +11,35 @@ * act — `installWebElicitation()` — and nothing about it is component metadata. */ -import { documented, registerComponents } from "@executablemd/core"; -import type { ComponentRegistration } from "@executablemd/core"; +import { + contributeDocumentation, + documented, + packageDocumentation, + registerComponents, +} from "@executablemd/core"; +import type { + ComponentRegistration, + DocumentationContribution, + DocumentationReader, +} from "@executablemd/core"; import type { Operation } from "effection"; import { WEB_FORM_PROPS, WEB_FORM_RETURNS, WebForm } from "./WebForm.ts"; export const WEB_ORIGIN = "@executablemd/web"; +/** This package's long-form documentation, derived from its registrations. */ +export function* webDocumentation( + read?: DocumentationReader, +): Operation { + return yield* packageDocumentation( + new URL("./components.md", import.meta.url), + { owner: WEB_ORIGIN, asset: "packages/web/src/components.md" }, + WEB_REGISTRATIONS.map((registration) => registration.name), + read, + ); +} + export const WEB_REGISTRATIONS: readonly ComponentRegistration[] = [ { name: "WebForm", @@ -38,6 +59,18 @@ export const WEB_REGISTRATIONS: readonly ComponentRegistration[] = [ }, ]; -export function* installWebComponents(): Operation { +/** + * This package's vocabulary, as declarations and nothing else. + * + * Registrations and the documentation that describes them, installed together + * so a scope that has one has the other. `xmd syntax` enters exactly this; + * installing the elicitation provider stays a separate, operational act. + */ +export function* useWebComponents(): Operation { yield* registerComponents(WEB_REGISTRATIONS); + yield* contributeDocumentation(webDocumentation); +} + +export function* installWebComponents(): Operation { + yield* useWebComponents(); } diff --git a/packages/web/src/responder.ts b/packages/web/src/responder.ts index da28282c1..b961228c1 100644 --- a/packages/web/src/responder.ts +++ b/packages/web/src/responder.ts @@ -15,9 +15,10 @@ */ import { type Api, createApi, type Operations } from "@effectionx/context-api"; -import { action } from "effection"; +import { ensure, scoped, withResolvers } from "effection"; import type { Operation } from "effection"; import { request } from "node:http"; +import type { ClientRequest, IncomingMessage } from "node:http"; import type { Json } from "./json.ts"; @@ -53,15 +54,75 @@ export interface FormResponse { * sends, and the server still checks it. `Host` is left to `node:http` to derive * from the URL, so it names where the request is actually going. */ -export function submitForm(url: string, data: Json): Operation { - return postJson(new URL("submit", url), JSON.stringify(data)); +export function submitForm( + url: string, + data: Json, + observe?: (request: ClientRequest) => void, +): Operation { + return postJson(new URL("submit", url), JSON.stringify(data), observe); } -function postJson(url: URL, body: string): Operation { - return action((resolve, reject) => { +function postJson( + url: URL, + body: string, + observe?: (request: ClientRequest) => void, +): Operation { + return scoped(function* () { const payload = new TextEncoder().encode(body); + const settled = withResolvers(); + const closed = withResolvers(); + // `close`, and nothing else. A destroyed request has not necessarily + // finished emitting: the peer's hang-up arrives afterwards, and this is + // what says there is nothing left to observe. + let finished = false; - const outgoing = request( + let received = ""; + let incoming: IncomingMessage | undefined; + let outgoing: ClientRequest | undefined; + + const onData = (chunk: string): void => { + received += chunk; + }; + const onEnd = (): void => { + settled.resolve({ status: incoming?.statusCode ?? 0, body: received }); + }; + const onFailure = (error: Error): void => settled.reject(error); + const onClose = (): void => { + finished = true; + closed.resolve(); + }; + + // Established before the request exists, because `yield* ensure(...)` is + // itself a suspension: an owner halted while it registers unwinds with no + // cleanup at all. + // + // The error observer stays attached across the destroy. `destroy()` on a + // live request makes the peer's end arrive as an asynchronous `error` — + // "socket hang up" — and an `error` an emitter has no listener for is + // thrown, not dropped. So the wait for `close` is what says the request can + // no longer emit, and only then is anything detached. + yield* ensure(function* () { + try { + if (outgoing && !finished) { + if (!outgoing.destroyed) { + outgoing.destroy(); + } + yield* closed.operation; + } + } finally { + if (incoming) { + incoming.off("data", onData); + incoming.off("end", onEnd); + incoming.off("error", onFailure); + } + if (outgoing) { + outgoing.off("error", onFailure); + outgoing.off("close", onClose); + } + } + }); + + outgoing = request( { protocol: url.protocol, hostname: url.hostname, @@ -74,25 +135,20 @@ function postJson(url: URL, body: string): Operation { Origin: url.origin, }, }, - (incoming) => { - let received = ""; - incoming.setEncoding("utf8"); - incoming.on("data", (chunk: string) => { - received += chunk; - }); - incoming.on("end", () => { - resolve({ status: incoming.statusCode ?? 0, body: received }); - }); - incoming.on("error", (error: Error) => reject(error)); + (response) => { + incoming = response; + response.setEncoding("utf8"); + response.on("data", onData); + response.on("end", onEnd); + response.on("error", onFailure); }, ); - outgoing.on("error", (error: Error) => reject(error)); + observe?.(outgoing); + outgoing.on("error", onFailure); + outgoing.on("close", onClose); outgoing.end(payload); - // Runs however the action ends, so a halted responder leaves no socket. - return () => { - outgoing.destroy(); - }; + return yield* settled.operation; }); } diff --git a/packages/web/src/response-channel.ts b/packages/web/src/response-channel.ts index 84ef5b9ad..5ffcd5b34 100644 --- a/packages/web/src/response-channel.ts +++ b/packages/web/src/response-channel.ts @@ -14,7 +14,7 @@ * when it chooses. */ -import { withResolvers } from "effection"; +import { ensure, resource, withResolvers } from "effection"; import type { Operation } from "effection"; import type { ServerResponse } from "node:http"; @@ -41,37 +41,47 @@ export class ResponseClosedError extends Error { } /** - * Wrap a Node response. + * Wrap a Node response, for as long as the request task that acquires it runs. * - * The listeners are armed here, when the channel is built, rather than when - * `finished` is first awaited. `finish` can fire between `end()` and the - * `yield*` that observes it, and a listener attached after the fact would wait - * for an event that already happened. Arming once, before anything is written, - * removes that window without depending on `writableFinished` being reported the - * same way by every runtime. + * The listeners are armed on acquisition, before anything is written, rather + * than when `finished` is first awaited. `finish` can fire between `end()` and + * the `yield*` that observes it, and a listener attached after the fact would + * wait for an event that already happened. Arming once removes that window + * without depending on `writableFinished` being reported the same way by every + * runtime. + * + * Both handlers come off when the task ends, however it ends. The first + * settlement is the answer — a `close` after a `finish` changes nothing — so + * the two no longer need to remove each other, and a request abandoned before + * either event leaves nothing attached to a response the server is about to + * destroy. */ -export function nodeResponseChannel(res: ServerResponse): ResponseChannel { - const settled = withResolvers(); +export function nodeResponseChannel(res: ServerResponse): Operation { + return resource(function* (provide) { + const settled = withResolvers(); - const onFinish = (): void => { - res.removeListener("close", onClose); - settled.resolve(); - }; - const onClose = (): void => { - res.removeListener("finish", onFinish); - settled.reject(new ResponseClosedError()); - }; + const onFinish = (): void => settled.resolve(); + const onClose = (): void => settled.reject(new ResponseClosedError()); - res.once("finish", onFinish); - res.once("close", onClose); + // A lexical finalizer rather than `ensure()`: entering the `try` is + // synchronous, so there is no instant at which this response is observed + // and the release is not yet armed. + try { + res.on("finish", onFinish); + res.on("close", onClose); - return { - head(status: number, headers: Record): void { - res.writeHead(status, headers); - }, - end(body?: string): void { - res.end(body); - }, - finished: settled.operation, - }; + yield* provide({ + head(status: number, headers: Record): void { + res.writeHead(status, headers); + }, + end(body?: string): void { + res.end(body); + }, + finished: settled.operation, + }); + } finally { + res.off("finish", onFinish); + res.off("close", onClose); + } + }); } diff --git a/packages/web/src/server.ts b/packages/web/src/server.ts index b90ca33e1..1ad8b44b6 100644 --- a/packages/web/src/server.ts +++ b/packages/web/src/server.ts @@ -88,7 +88,15 @@ export interface FormServer { * always run for real. */ export interface FormServerSeams { - responseChannel?(res: ServerResponse): ResponseChannel; + responseChannel?(res: ServerResponse): Operation; + /** + * Handed every accepted socket as it arrives. + * + * Package-private, for the cancellation regression: the listeners are on a + * value this operation owns and does not otherwise hand out, and their + * release is the thing under test. + */ + observeSocket?(socket: Socket): void; /** * After the listener is up, before `provide()`. * @@ -150,9 +158,20 @@ export function useFormServer( const server = createServer(); const channelFor = seams.responseChannel ?? nodeResponseChannel; + // One close handler per accepted socket, kept so teardown can detach them + // all before it destroys the connections they are attached to. + const closers = new Map void>(); const onConnection = (socket: Socket): void => { + // Before this server attaches anything, so a case can measure what the + // runtime was already holding on the connection. + seams.observeSocket?.(socket); sockets.add(socket); - socket.once("close", () => sockets.delete(socket)); + const onClose = (): void => { + sockets.delete(socket); + closers.delete(socket); + }; + closers.set(socket, onClose); + socket.on("close", onClose); }; // One long-lived observer, not a readiness-only one: an error handler that // stopped mattering once the server came up would leave a caller waiting on @@ -168,18 +187,26 @@ export function useFormServer( const onListening = (): void => listening.resolve(); - // Attached before `listen`, not after: an event emitted before its listener - // exists is simply lost, and readiness is emitted immediately. - server.once("listening", onListening); - server.on("connection", onConnection); - server.on("error", onError); - - // Registered before `listen`, so a server that fails to bind is still torn - // down by the same path as one that served for an hour. + // Registered before anything is attached and before `listen`, so a server + // that fails to bind is torn down by the same path as one that served for + // an hour — and so that no instant exists in which this server is observed + // and the release is not yet armed. `yield* ensure(...)` is itself a + // suspension, and an owner halted while it registers unwinds with nothing + // on it. yield* ensure(function* () { - server.removeListener("listening", onListening); - server.removeListener("connection", onConnection); - server.removeListener("error", onError); + server.off("listening", onListening); + server.off("connection", onConnection); + server.off("error", onError); + // Every accepted socket's close handler comes off in the same + // uninterrupted run as the listener's own, before this teardown + // suspends: `connection` is no longer observed, so nothing can be + // accepted and left untracked after this point. + for (const [socket, onClose] of closers) { + socket.off("close", onClose); + } + closers.clear(); + // Request tasks end before their connections do, so a response in flight + // is abandoned by its own scope rather than by a destroyed socket. if (acceptor) { yield* acceptor.halt(); } @@ -191,12 +218,23 @@ export function useFormServer( sockets.clear(); if (server.listening) { const closed = withResolvers(); - server.once("close", () => closed.resolve()); - server.close(); - yield* closed.operation; + const onClose = (): void => closed.resolve(); + server.on("close", onClose); + try { + server.close(); + yield* closed.operation; + } finally { + server.off("close", onClose); + } } }); + // Attached before `listen`, not after: an event emitted before its listener + // exists is simply lost, and readiness is emitted immediately. + server.on("listening", onListening); + server.on("connection", onConnection); + server.on("error", onError); + server.listen(0, HOST); yield* listening.operation; ready = true; @@ -210,7 +248,7 @@ export function useFormServer( const prefix = `/f/${token}/`; function* handle(req: IncomingMessage, res: ServerResponse): Operation { - const channel = channelFor(res); + const channel = yield* channelFor(res); const route = routeFor(req, prefix); if (route === undefined || req.headers.host !== `${HOST}:${port}`) { diff --git a/packages/web/tests/http-client.ts b/packages/web/tests/http-client.ts index 583ace9a6..a25664d9f 100644 --- a/packages/web/tests/http-client.ts +++ b/packages/web/tests/http-client.ts @@ -27,6 +27,15 @@ export interface HttpResponse { } export interface HttpConnection { + /** The connection itself, so a case can say what it is still observing. */ + socket: Socket; + /** + * How many times this client's own handlers have run. + * + * A cancelled connection must stop counting: a chunk delivered after + * teardown that still moved this would be a handler nobody released. + */ + callbacks(): number; /** Send text, encoded as UTF-8. */ write(text: string): void; /** @@ -53,9 +62,13 @@ export interface HttpConnection { * provokes one would fail for the wrong reason. A reset and a clean close settle * `ended` alike, because after a mid-stream refusal the peer may present either. */ -export function useConnection(port: number): Operation { +export function useConnection( + port: number, + observe?: (socket: Socket) => void, +): Operation { return resource(function* (provide) { const socket = connect(port, "127.0.0.1"); + observe?.(socket); yield* ensure(() => { socket.destroy(); }); @@ -63,11 +76,11 @@ export function useConnection(port: number): Operation { yield* action((resolve, reject) => { const onConnect = (): void => resolve(); const onError = (error: Error): void => reject(error); - socket.once("connect", onConnect); - socket.once("error", onError); + socket.on("connect", onConnect); + socket.on("error", onError); return () => { - socket.removeListener("connect", onConnect); - socket.removeListener("error", onError); + socket.off("connect", onConnect); + socket.off("error", onError); }; }); @@ -87,19 +100,32 @@ export function useConnection(port: number): Operation { notify?.(); }; - socket.on("data", (chunk: Buffer) => { + const onData = (chunk: Buffer): void => { buffer += chunk.toString("utf8"); advance(); - }); - socket.on("error", (error: Error) => { + }; + const onError = (error: Error): void => { ended.resolve(`error:${error.message}`); advance(); - }); - socket.on("close", () => { + }; + const onClose = (): void => { ended.resolve("close"); advance(); + }; + + // Established before the handlers exist, because `yield* ensure(...)` is + // itself a suspension: an owner halted while it registers unwinds with no + // cleanup at all. + yield* ensure(() => { + socket.off("data", onData); + socket.off("error", onError); + socket.off("close", onClose); }); + socket.on("data", onData); + socket.on("error", onError); + socket.on("close", onClose); + function* untilAfter(seen: number): Operation { if (version !== seen) { return; @@ -113,6 +139,10 @@ export function useConnection(port: number): Operation { } yield* provide({ + socket, + callbacks(): number { + return version; + }, write(text: string): void { socket.write(new Uint8Array(new TextEncoder().encode(text))); }, diff --git a/packages/web/tests/responder.test.ts b/packages/web/tests/responder.test.ts index 0b2609a9b..251ddbc8a 100644 --- a/packages/web/tests/responder.test.ts +++ b/packages/web/tests/responder.test.ts @@ -1,8 +1,10 @@ import { describe, it } from "@executablemd/test-support/bdd"; import { expect } from "@executablemd/test-support/expect"; -import { scoped, spawn, suspend, withResolvers } from "effection"; +import { ensure, scoped, sleep, spawn, suspend, withResolvers } from "effection"; import type { Operation } from "effection"; +import { createServer } from "node:http"; +import type { ClientRequest } from "node:http"; import { FormResponder, respond, submitForm } from "../src/responder.ts"; import type { FormResponse } from "../src/responder.ts"; import { useFormServer } from "../src/server.ts"; @@ -258,3 +260,128 @@ describe("responder: subject to the protocol, not exempt from it", () => { expect(later.body).toBe(""); }); }); + +describe("responder: the request it opens is its own", () => { + /** + * A submission cancelled before its response arrives is the case the events + * never come for. The server here accepts and answers nothing, so the halt + * lands while the request is live: the counts are read before the halt, again + * after it and before the event is replayed, and the replay must reach + * neither the abandoned operation nor anything still accumulating. + */ + it("releases the outgoing request's handlers when the submission is cancelled", function* () { + const server = createServer(() => {}); + const listening = withResolvers(); + const onError = (error: Error): void => listening.reject(error); + + yield* ensure(() => { + server.off("error", onError); + }); + server.on("error", onError); + + server.listen(0, "127.0.0.1", () => listening.resolve()); + yield* listening.operation; + yield* ensure(() => { + server.closeAllConnections(); + server.close(); + }); + + const address = server.address(); + if (address === null || typeof address === "string") { + throw new Error("the silent server did not listen on a TCP port"); + } + + let outgoing: ClientRequest | undefined; + let before = 0; + let settled = false; + + const owner = yield* spawn(function* () { + yield* submitForm( + `http://127.0.0.1:${address.port}/f/x/`, + { decision: "approve" }, + (request) => { + outgoing = request; + before = request.listenerCount("error"); + }, + ); + settled = true; + }); + + // A turn after the executor ran, so its handler is attached and its + // cleanup — the one `action()` returns — is the executor's own return + // value rather than something still being registered. + yield* sleep(0); + if (!outgoing) { + throw new Error("the responder opened no request"); + } + + // At least one more, not exactly one: a runtime may hold handlers of its + // own on this source, so the release below is measured against what was + // live rather than against the baseline. + const live = outgoing.listenerCount("error"); + expect(live).toBeGreaterThanOrEqual(before + 1); + expect(settled).toBe(false); + + yield* owner.halt(); + + expect(outgoing.listenerCount("error")).toBe(live - 1); + + let seen = 0; + const sentinel = (): void => { + seen += 1; + }; + + outgoing.on("error", sentinel); + try { + outgoing.emit("error", new Error("after the submission was cancelled")); + } finally { + outgoing.off("error", sentinel); + } + + expect(seen).toBe(1); + expect(outgoing.listenerCount("error")).toBe(live - 1); + expect(settled).toBe(false); + }); + + /** + * `postJson` observes its outgoing request and the response it brings back + * for exactly as long as the `action` runs. The count is read after the + * submission has settled and before the event is replayed, because a handler + * removed by its own event would leave the same count behind as one the + * action released. + */ + it("releases the outgoing request's error handler when the submission settles", function* () { + const server = yield* useFormServer(formInput()); + let outgoing: ClientRequest | undefined; + let before = 0; + + const response = yield* submitForm(server.url, { decision: "approve" }, (request) => { + outgoing = request; + before = request.listenerCount("error"); + }); + + expect(response.status).toBe(204); + if (!outgoing) { + throw new Error("the responder opened no request"); + } + + // The case above reads the count while it is live; this one is the + // completion path, where what matters is that settling released it. + expect(outgoing.listenerCount("error")).toBe(before); + + let seen = 0; + const sentinel = (): void => { + seen += 1; + }; + + outgoing.on("error", sentinel); + try { + outgoing.emit("error", new Error("after the submission settled")); + } finally { + outgoing.off("error", sentinel); + } + + expect(seen).toBe(1); + expect(outgoing.listenerCount("error")).toBe(before); + }); +}); diff --git a/packages/web/tests/server-lifecycle.test.ts b/packages/web/tests/server-lifecycle.test.ts index 2758fe828..4292def65 100644 --- a/packages/web/tests/server-lifecycle.test.ts +++ b/packages/web/tests/server-lifecycle.test.ts @@ -1,10 +1,16 @@ import { describe, it } from "@executablemd/test-support/bdd"; import { expect } from "@executablemd/test-support/expect"; -import { scoped, spawn, withResolvers } from "effection"; +import { ensure, resource, scoped, sleep, spawn, suspend, withResolvers } from "effection"; import type { Operation } from "effection"; +import { once } from "@effectionx/node/events"; +import { when } from "@effectionx/converge"; +import { Buffer } from "node:buffer"; +import { createServer } from "node:http"; +import type { ServerResponse } from "node:http"; import { connect } from "node:net"; import type { Socket } from "node:net"; +import { nodeResponseChannel } from "../src/response-channel.ts"; import type { ResponseChannel } from "../src/response-channel.ts"; import { useFormServer } from "../src/server.ts"; import type { FormServer } from "../src/server.ts"; @@ -20,30 +26,31 @@ interface HeldChannel { closeWithoutFinish: () => void; } -function heldChannel(): { seam: () => ResponseChannel; held: HeldChannel } { +function heldChannel(): { seam: () => Operation; held: HeldChannel } { const ends: { status: number; body: string | undefined }[] = []; const releases: { finish: () => void; fail: (error: Error) => void }[] = []; const firstEnd = withResolvers(); return { - seam: () => { - const settled = withResolvers(); - let status = 0; - releases.push({ - finish: () => settled.resolve(), - fail: (error: Error) => settled.reject(error), - }); - return { - head(next: number): void { - status = next; - }, - end(body?: string): void { - ends.push({ status, body }); - firstEnd.resolve(); - }, - finished: settled.operation, - }; - }, + seam: () => + resource(function* (provide) { + const settled = withResolvers(); + let status = 0; + releases.push({ + finish: () => settled.resolve(), + fail: (error: Error) => settled.reject(error), + }); + yield* provide({ + head(next: number): void { + status = next; + }, + end(body?: string): void { + ends.push({ status, body }); + firstEnd.resolve(); + }, + finished: settled.operation, + }); + }), held: { ends: () => ends, ended: firstEnd.operation, @@ -106,6 +113,7 @@ describe("form server: the submission resolves only after its response is sent", it("fails rather than resolving when the response closes before finishing", function* () { let refusedPort = 0; let leftover: KeepAlive | undefined; + const keepAlives = yield* useKeepAlives(); yield* scoped(function* () { const { seam, held } = heldChannel(); @@ -114,7 +122,7 @@ describe("form server: the submission resolves only after its response is sent", refusedPort = addressOf(server.url).port; // A keep-alive connection that must not survive teardown. - leftover = yield* openKeepAlive(refusedPort); + leftover = yield* keepAlives.open(refusedPort); yield* submitValid(server); yield* held.ended; @@ -145,13 +153,14 @@ describe("form server: failure reaches the caller", () => { let recordedPort = 0; let keepAlive: KeepAlive | undefined; let acquired = false; + const keepAlives = yield* useKeepAlives(); yield* scoped(function* () { try { yield* useFormServer(formInput(), { *afterListen(address) { recordedPort = address.port; - keepAlive = yield* openKeepAlive(address.port); + keepAlive = yield* keepAlives.open(address.port); throw new Error("setup failed after the listener came up"); }, }); @@ -205,11 +214,12 @@ describe("form server: teardown", () => { it("closes the port and its connections after a successful submission", function* () { let port = 0; let keepAlive: KeepAlive | undefined; + const keepAlives = yield* useKeepAlives(); yield* scoped(function* () { const server = yield* useFormServer(formInput()); port = addressOf(server.url).port; - keepAlive = yield* openKeepAlive(port); + keepAlive = yield* keepAlives.open(port); yield* submitValid(server); yield* server.submission; }); @@ -233,11 +243,12 @@ describe("form server: teardown", () => { */ it("releases the listener when the owning task is halted mid-wait", function* () { const ready = withResolvers<{ port: number; keepAlive: KeepAlive }>(); + const keepAlives = yield* useKeepAlives(); const owner = yield* spawn(function* () { const server = yield* useFormServer(formInput()); const port = addressOf(server.url).port; - ready.resolve({ port, keepAlive: yield* openKeepAlive(port) }); + ready.resolve({ port, keepAlive: yield* keepAlives.open(port) }); // Still waiting for a submission that never comes when the halt lands. yield* server.submission; }); @@ -256,6 +267,94 @@ describe("form server: teardown", () => { }); }); +/** + * A loopback that hands the test the live `ServerResponse` of one request. + * + * The response channel's listeners are what this measures, and they can only + * be counted on a response Node itself made. + */ +function useResponseUnderTest(): Operation { + return resource(function* (provide) { + const arrived = withResolvers(); + const server = createServer((incoming, outgoing) => { + incoming.resume(); + arrived.resolve(outgoing); + }); + + const listening = withResolvers(); + const onError = (error: Error): void => listening.reject(error); + + yield* ensure(() => { + server.off("error", onError); + }); + server.on("error", onError); + + server.listen(0, "127.0.0.1", () => listening.resolve()); + yield* listening.operation; + + yield* ensure(function* () { + server.closeAllConnections(); + const closed = withResolvers(); + const onClose = (): void => closed.resolve(); + + server.on("close", onClose); + try { + server.close(); + yield* closed.operation; + } finally { + server.off("close", onClose); + } + }); + + const address = server.address(); + if (address === null || typeof address === "string") { + throw new Error("the response fixture did not listen on a TCP port"); + } + + const connection = yield* useConnection(address.port); + connection.write(requestText({ method: "GET", path: "/", host: `127.0.0.1:${address.port}` })); + + yield* provide(yield* arrived.operation); + }); +} + +describe("form server: a response channel belongs to its request task", () => { + /** + * A request abandoned before it answered is the case `finish` never comes + * for, so the channel cannot be relying on it to let go. The counts are read + * on the response itself, and the one after the halt is read before the + * event is emitted: a handler that removed itself when `finish` finally + * arrived would leave the same count behind as one that was never there. + */ + it("detaches from the response when the task is halted, and a later finish changes nothing", function* () { + const res = yield* useResponseUnderTest(); + const counts = (): number[] => [res.listenerCount("finish"), res.listenerCount("close")]; + const before = counts(); + + const owner = yield* spawn(function* () { + yield* nodeResponseChannel(res); + yield* suspend(); + }); + yield* sleep(0); + + // Anchored to what was live, not to a delta from the baseline: how many + // handlers a runtime keeps on its own `ServerResponse`, and when it adds + // them, is the runtime's business — Bun attaches one to `finish` that the + // others do not. What this case owns is the pair the channel added. + const live = counts(); + expect(live[0]).toBeGreaterThan(before[0]); + expect(live[1]).toBeGreaterThan(before[1]); + + yield* owner.halt(); + + expect(counts()).toEqual([live[0] - 1, live[1] - 1]); + + res.emit("finish"); + + expect(counts()).toEqual([live[0] - 1, live[1] - 1]); + }); +}); + /** The message `submission` fails with; throws if it succeeds instead. */ function* failureOf(server: FormServer): Operation { try { @@ -291,16 +390,154 @@ export interface KeepAlive { * the server failed to destroy would hang teardown outright. The test returning * is the evidence, and `portRefuses` confirms the port went with it. */ -function* openKeepAlive(port: number): Operation { - const socket = connect(port, "127.0.0.1"); - socket.on("error", () => {}); - // Flowing rather than paused, so the connection behaves like a real client's - // and never holds unread bytes against teardown. - socket.resume(); - - const opened = withResolvers(); - socket.once("connect", () => opened.resolve()); - yield* opened.operation; - - return { socket, establishedWith: socket.remoteAddress }; +/** + * Somewhere to open keep-alive connections whose error observers outlive the + * server that destroys them. + * + * The reset a destroyed connection delivers must still reach a listener, and + * it arrives *during* the server's teardown. A connection acquired inside the + * server's own scope would have its observer removed first — destructors run + * in reverse order of registration — and the reset would surface as an + * uncaught error. So the holder is acquired before the server, which is what + * puts its cleanup after the server's. + */ +function useKeepAlives(): Operation<{ + open(port: number): Operation; + count(): number; + listeners(socket: Socket): number; +}> { + return resource(function* (provide) { + const observers = new Map void>(); + + yield* ensure(() => { + for (const [socket, onError] of observers) { + socket.off("error", onError); + } + observers.clear(); + }); + + // Declared here rather than inside `open` below, so the subscription and + // the teardown that walks it belong to the same owner: a nested generator + // is a scope of its own, and it ends long before this resource does. + const observe = (socket: Socket): void => { + const onError = (): void => {}; + + socket.on("error", onError); + observers.set(socket, onError); + }; + + yield* provide({ + *open(port: number): Operation { + const socket = connect(port, "127.0.0.1"); + + observe(socket); + // Flowing rather than paused, so the connection behaves like a real + // client's and never holds unread bytes against teardown. + socket.resume(); + + // Interpreted in the same synchronous run as `connect`, so the event + // cannot land before the wait is attached. + yield* once(socket, "connect"); + + return { socket, establishedWith: socket.remoteAddress }; + }, + count: () => observers.size, + listeners: (socket: Socket) => socket.listenerCount("error"), + }); + }); } + +/** What a raw client's own socket is carrying. */ +function clientCounts(socket: Socket): number[] { + return [ + socket.listenerCount("data"), + socket.listenerCount("error"), + socket.listenerCount("close"), + ]; +} + +describe("form server: accepted sockets and raw clients", () => { + /** + * One close handler per accepted socket, and a raw client's own three, both + * cancelled while they are live. Each count is measured against what the + * runtime was already holding, read once while the owner runs, again after + * it is torn down and before the events are replayed, and once more + * afterwards — together with what the client had received, so a late chunk + * reaching a still-accumulating buffer would show. + */ + it("releases the accepted socket's close handler and the client's own on cancellation", function* () { + const accepted: { socket: Socket; before: number }[] = []; + let client: Socket | undefined; + let clientBefore: number[] = []; + let acceptedLive = 0; + let clientLive: number[] = []; + let received: () => string = () => ""; + let callbacks: () => number = () => 0; + + const owner = yield* spawn(function* () { + const server = yield* useFormServer(formInput(), { + observeSocket: (socket) => accepted.push({ socket, before: socket.listenerCount("close") }), + }); + const { port } = addressOf(server.url); + const connection = yield* useConnection(port, (socket) => { + client = socket; + clientBefore = clientCounts(socket); + }); + + received = () => connection.receivedSoFar(); + callbacks = () => connection.callbacks(); + connection.write(requestText({ method: "GET", path: "/", host: `127.0.0.1:${port}` })); + yield* connection.response(); + + acceptedLive = accepted[0]?.socket.listenerCount("close") ?? 0; + clientLive = client ? clientCounts(client) : []; + yield* suspend(); + }); + + // Synchronized on the exchange having happened, so both owners are live + // with everything attached when the halt lands. + yield* when(function* () { + expect(clientLive.length).toBeGreaterThan(0); + }); + + const first = accepted[0]; + if (!first || !client) { + throw new Error("no connection was accepted"); + } + + // At least what each owner attached, not exactly it: how many handlers a + // runtime keeps on its own sockets, and when it adds them, is the + // runtime's business — Bun on Linux holds one the others do not. What + // these cases own is the pair each added, so the release below is measured + // against what was live rather than against the baseline. + expect(acceptedLive).toBeGreaterThanOrEqual(first.before + 1); + clientLive.forEach((count, index) => { + expect(count).toBeGreaterThanOrEqual(clientBefore[index] + 1); + }); + + // Empty, because the exchange above consumed it — which is what makes the + // replayed chunk below visible if anything is still appending. The + // callback count is not: the exchange ran the client's handlers, and a + // replayed event reaching one would move it again. + const seen = received(); + const ran = callbacks(); + + expect(ran).toBeGreaterThan(0); + + yield* owner.halt(); + + const released = clientLive.map((count) => count - 1); + + expect(first.socket.listenerCount("close")).toBe(acceptedLive - 1); + expect(clientCounts(client)).toEqual(released); + + first.socket.emit("close"); + client.emit("data", Buffer.from("after the client was cancelled")); + client.emit("close"); + + expect(first.socket.listenerCount("close")).toBe(acceptedLive - 1); + expect(clientCounts(client)).toEqual(released); + expect(received()).toBe(seen); + expect(callbacks()).toBe(ran); + }); +}); diff --git a/packages/web/tests/server-support.ts b/packages/web/tests/server-support.ts index 0724eabdc..e8cd9cf2b 100644 --- a/packages/web/tests/server-support.ts +++ b/packages/web/tests/server-support.ts @@ -8,8 +8,9 @@ * point of the refusal paths. */ -import { spawn, withResolvers } from "effection"; +import { race, spawn, withResolvers } from "effection"; import type { Operation } from "effection"; +import { once } from "@effectionx/node/events"; import { connect } from "node:net"; import { compileForm } from "../src/compile.ts"; @@ -90,12 +91,22 @@ export function* watchSubmission(server: FormServer): Operation<() => Submission /** Whether a fresh connection to this port is refused. */ export function* portRefuses(port: number): Operation { const socket = connect(port, "127.0.0.1"); - const settled = withResolvers(); - socket.once("connect", () => settled.resolve(false)); - socket.once("error", () => settled.resolve(true)); - const refused = yield* settled.operation; - socket.destroy(); - return refused; + try { + // Raced inline, in the same synchronous run as `connect`: a spawned race + // attaches its arms a turn later, and the socket can settle in that turn. + return yield* race([ + (function* (): Operation { + yield* once(socket, "connect"); + return false; + })(), + (function* (): Operation { + yield* once(socket, "error"); + return true; + })(), + ]); + } finally { + socket.destroy(); + } } /** A JSON body whose UTF-8 encoding is exactly `bytes` long. */ diff --git a/packages/workflow/deno.ts b/packages/workflow/deno.ts index 3ab0e1195..65b3948f7 100644 --- a/packages/workflow/deno.ts +++ b/packages/workflow/deno.ts @@ -83,7 +83,7 @@ export { withWorkflowWorkspace } from "./src/deno/workspace/published.ts"; * ``: its implementation names durable work after its own invocation, * so canonical execution builds it from the claimant it minted. */ -export { evaluationComponents } from "./src/deno/workspace/evaluate.ts"; +export { evaluationProfile } from "./src/deno/workspace/evaluate.ts"; export type { GeneratedEvaluationOptions } from "./src/deno/workspace/evaluate.ts"; export type { WorkflowWorkspaceOptions } from "./src/deno/workspace/published.ts"; export type { WorkflowAgentAttachment, WorkflowAgentInstaller } from "./src/deno/workspace/host.ts"; diff --git a/packages/workflow/mod.ts b/packages/workflow/mod.ts index af9fb96a7..21a18f2d9 100644 --- a/packages/workflow/mod.ts +++ b/packages/workflow/mod.ts @@ -213,6 +213,7 @@ export type { export { admitPushEvidence } from "./src/composition/push-evidence.ts"; export { COMPOSITION_REGISTRATIONS, + compositionDocumentation, useCompositionComponents, } from "./src/composition/installation.ts"; diff --git a/packages/workflow/src/composition/components.md b/packages/workflow/src/composition/components.md new file mode 100644 index 000000000..030999d78 --- /dev/null +++ b/packages/workflow/src/composition/components.md @@ -0,0 +1,190 @@ +Long-form documentation for the repository-composition components. + +Thirteen components for working with repositories, branches, pull requests and +issues from inside a document. They are mostly *regions*: ``, +`` and `` establish where the work happens, and the `Git.*`, +`PullRequest.*` and issue components act inside whatever those established. + +The pattern to hold onto is that a document says *what* it wants — this +repository, this branch, this commit — and the components decide whether that +means creating something or using what is already there. Running one twice does +not make two. + +## Repository + +Clones a repository, or uses the clone already there. + +```mdx + +… + +``` + +A region: its content runs with that repository as the working directory. +`name` identifies the checkout across runs, so a second run reuses the first's +rather than cloning again, and `url` says where it comes from. + +The checkout survives the run. Nothing here deletes, resets, cleans or repairs +one — a document that wants a clean tree asks for it explicitly. + +## Worktree + +Creates a linked checkout, or uses the one already there. + +```mdx + +… + +``` + +A region, expanding its content in the worktree. Use it to work on a branch +without disturbing the main checkout — two worktrees of one repository can be on +two branches at once, which is what makes a document that reviews one branch +while building another possible. + +`as` captures the path of the linked checkout, for a step that needs to name it. + +## Dir + +Changes the working directory for its content. + +```mdx + +… + +``` + +The plainest of the three regions: no repository, no branch, just a directory. +It pairs naturally with ``, which hands you a path this can +then work inside. + +## Git.Switch + +Switches to a branch, creating it if it does not exist. + +```mdx + +``` + +`base` says what a *new* branch starts from; it is ignored when the branch +already exists, so the same element is correct on the first run and on the +tenth. Self-closing, and it renders nothing. + +## Git.Add + +Stages paths for commit. + +```mdx + +``` + +`paths` is a list, so one element stages everything a step produced. Staging is +explicit rather than implied by `` because a document that commits +everything it happens to have touched is a document that commits surprises. + +## Git.Commit + +Commits what is staged. + +```mdx +Prepare 1.4 +``` + +The content is the commit message, so a message can be as long as it needs to be +and can interpolate what the document learned. Commits only what `` +staged. + +## Git.Push + +Publishes the current branch. + +```mdx + +``` + +Self-closing. Pushes the branch the working tree is on, to the remote the +repository was cloned from — so which branch is decided by `` above +it rather than repeated here. + +## PullRequest + +Opens a pull request, or updates the one already open. + +```mdx + +What changed, and why. + +``` + +The content is the body. Opening and updating are one element for the same +reason cloning and reusing are: a document that runs twice should end in the +state it describes, not with two pull requests. + +Push the branch first — `` — since there is nothing to open a pull +request against until the branch exists on the remote. + +## PullRequest.Reviews + +Reads the reviews on a pull request. + +```mdx + +``` + +`as` is required: this is a read, and the reviews are the result. Use it to let +a document act on what people said — hold a release until an approval lands, or +collect the changes a reviewer asked for. + +## PullRequest.Comments + +Reads the comments on a pull request. + +```mdx + +``` + +Comments rather than reviews: the discussion, including comments that carry no +verdict. `as` is required. + +## PullRequest.Checks + +Reads the check results on a pull request. + +```mdx + +``` + +`as` is required. This is what a document branches on when it should only +proceed once CI is green — read the checks, then decide, rather than merging and +hoping. + +## IssueTracker + +Selects the issue tracker its content works with. + +```mdx + +… + +``` + +A region, like `` above it: the issue components inside it resolve +against this tracker. A document that touches two trackers writes two regions +rather than repeating the URL on every element. + +## Issue + +Reads an issue, or files one. + +```mdx + + + +What failed, and what to try. + +``` + +The self-closing form with `url` reads an existing issue and binds it. The +paired form with `title` files a new one, with the content as the body — which +is how a document that finds something wrong can record it where the next person +will look. diff --git a/packages/workflow/src/composition/installation.ts b/packages/workflow/src/composition/installation.ts index ba4a68728..4d33dc6d5 100644 --- a/packages/workflow/src/composition/installation.ts +++ b/packages/workflow/src/composition/installation.ts @@ -21,8 +21,18 @@ */ import type { Operation } from "effection"; -import { documented, formDispatcher, registerComponents } from "@executablemd/core"; -import type { ComponentRegistration } from "@executablemd/core"; +import { + contributeDocumentation, + documented, + formDispatcher, + packageDocumentation, + registerComponents, +} from "@executablemd/core"; +import type { + ComponentRegistration, + DocumentationContribution, + DocumentationReader, +} from "@executablemd/core"; import { COMPOSITION_ORIGIN, dirDefinition } from "./definitions.ts"; import Repository, { props as repositoryProps } from "./components/Repository.ts"; import Worktree, { props as worktreeProps } from "./components/Worktree.ts"; @@ -54,6 +64,27 @@ import IssueTracker, { props as issueTrackerProps } from "./components/IssueTrac const dir = dirDefinition(); /** The one vocabulary every consumer of these components describes. */ +/** + * This boundary's long-form documentation, and the components it must cover. + * + * Derived from `COMPOSITION_REGISTRATIONS` below — including ``, which is + * registered from a definition rather than spelled inline, and would be the + * easiest one to leave undocumented if this list were maintained by hand. + */ +export function* compositionDocumentation( + read?: DocumentationReader, +): Operation { + return yield* packageDocumentation( + new URL("./components.md", import.meta.url), + { + owner: COMPOSITION_ORIGIN, + asset: "packages/workflow/src/composition/components.md", + }, + COMPOSITION_REGISTRATIONS.map((registration) => registration.name), + read, + ); +} + export const COMPOSITION_REGISTRATIONS: readonly ComponentRegistration[] = [ { name: "Repository", @@ -206,7 +237,16 @@ export const COMPOSITION_REGISTRATIONS: readonly ComponentRegistration[] = [ }, ]; -/** Register the composition vocabulary as ordinary defaults for this scope. */ -export function useCompositionComponents(): Operation { - return registerComponents(COMPOSITION_REGISTRATIONS); +/** + * Register the composition vocabulary as ordinary defaults for this scope. + * + * The registrations and the documentation that describes them, installed + * together so a scope that has one has the other. Both are declarations: this + * installs no provider, discovers no ambient repository, acquires no lock, + * spawns no Git and reads no credential, which is what lets `xmd syntax` enter + * it to describe the profile. + */ +export function* useCompositionComponents(): Operation { + yield* registerComponents(COMPOSITION_REGISTRATIONS); + yield* contributeDocumentation(compositionDocumentation); } diff --git a/packages/workflow/src/deno/composition/subprocess.ts b/packages/workflow/src/deno/composition/subprocess.ts index 050e584af..7895602b4 100644 --- a/packages/workflow/src/deno/composition/subprocess.ts +++ b/packages/workflow/src/deno/composition/subprocess.ts @@ -11,6 +11,8 @@ import { ensure, type Operation, withResolvers } from "effection"; import { spawn as spawnChild } from "node:child_process"; +import type { ChildProcessByStdio } from "node:child_process"; +import type { Readable, Writable } from "node:stream"; import process from "node:process"; /** What one invocation reported. A nonzero exit is an answer, not a throw. */ @@ -34,6 +36,18 @@ export interface ProcessInvocation { * to put either one nor a boundary that carries one unchanged. */ readonly input?: string; + /** + * Handed the child as it is spawned, before anything is attached to it, + * together with a reader for what has been captured from it so far. + * + * Package-private, for the cancellation regression: this is the only place a + * baseline can be measured, and the cleanup is already established by the + * time it is called. + */ + readonly observe?: ( + child: ChildProcessByStdio, + captured: () => { stdout: string; stderr: string }, + ) => void; } export function* runProcess({ @@ -42,6 +56,7 @@ export function* runProcess({ cwd, env, input, + observe, }: ProcessInvocation): Operation { // `node:child_process` rather than the runtime's own global: this adapter is // selected by the host, not written against one, and `spawn` replaces the @@ -55,14 +70,56 @@ export function* runProcess({ // waited for below would never arrive and a cancelled operation would hang // instead of tearing down. const options = { cwd, env: { ...env }, detached: true }; - const child = - input === undefined - ? spawnChild(command, [...args], { ...options, stdio: ["ignore", "pipe", "pipe"] }) - : spawnChild(command, [...args], { ...options, stdio: ["pipe", "pipe", "pipe"] }); + // One type for both spawns, so the child's own listeners can be removed + // through it: the two `stdio` shapes differ only in whether standard input is + // a pipe, and TypeScript's overloads make the union's `off` unresolvable. + // + // Declared before the cleanup below and assigned after it, because a child + // that exists before its release is registered can be stranded: `yield* + // ensure(...)` is itself a suspension, and an owner halted while it registers + // unwinds with nothing on it. + let child: ChildProcessByStdio | undefined; + + let settled = false; + // `close`, and nothing else. An assigned `exitCode` or `signalCode` says the + // process ended; it does not say the pipes this operation inherited have. + let closed = false; + let stdout = ""; + let stderr = ""; + // Declared after what it reads, so an observer may call it at once rather + // than only from a later turn. + const captured = () => ({ stdout, stderr }); + + const outcome = withResolvers(); - // Registered with no suspension point between spawning and registering, so a - // halt cannot land between the two and leave a process running after the - // scope that started it is gone. + // A pipe the command stops reading is the command's answer, and its exit + // status is what says so — but the write still fails, and an unhandled + // stream error would take the process down rather than this operation. + // Reporting it here lets `close` settle first when there is an exit to + // report. + const onStdinError = (error: Error): void => { + outcome.reject(error); + }; + const onStdout = (chunk: string): void => { + stdout += chunk; + }; + const onStderr = (chunk: string): void => { + stderr += chunk; + }; + // `close` rather than `exit`: it is the event that fires once both pipes have + // ended, so what is read here is everything the command wrote rather than + // whatever had arrived when it stopped. + const onExit = (code: number | null): void => { + closed = true; + outcome.resolve({ code: code ?? -1, stdout, stderr }); + }; + const onFailure = (error: Error): void => { + outcome.reject(error); + }; + + // Established before anything is attached, because `yield* ensure(...)` is + // itself a suspension: an owner halted while it registers unwinds with no + // cleanup on it, leaving the process running and every handler in place. // // Teardown waits for the child to close rather than only signalling it. // `kill` returns once the signal is queued, not once it has been delivered, @@ -70,58 +127,74 @@ export function* runProcess({ // finish while the process it started is still alive — and the disposable // directory it is working in is removed moments later. `close` is the event // that fires once the process is gone and both pipes have ended, which is - // what makes cancellation complete rather than merely started. - let settled = false; - const closed = withResolvers(); - child.on("close", () => closed.resolve()); + // what makes cancellation complete rather than merely started. It observes + // that with a listener of its own, so this cleanup depends on nothing the + // body below may not have reached. yield* ensure(function* () { - if (settled) { + if (child === undefined) { return; } - // The group, so the transport helper goes with the command that started it. - // A group that is already gone raises, and that is the same answer as a - // group that was killed — the wait below is what decides either way. + try { - if (child.pid !== undefined) { - process.kill(-child.pid, "SIGKILL"); + if (!settled) { + // The group, so the transport helper goes with the command that + // started it. A group that is already gone raises, and that is the + // same answer as a group that was killed — the wait below is what + // decides either way. + try { + if (child.pid !== undefined) { + process.kill(-child.pid, "SIGKILL"); + } + } catch { + child.kill("SIGKILL"); + } + + // Only `close` ends this wait. The handlers above stay attached + // through it, so whatever the command writes on its way out is still + // captured and a failure still reports. + if (!closed) { + const gone = withResolvers(); + const onGone = (): void => gone.resolve(); + + child.on("close", onGone); + try { + if (!closed) { + yield* gone.operation; + } + } finally { + child.off("close", onGone); + } + } } - } catch { - child.kill("SIGKILL"); + } finally { + // After the wait, and synchronously. Removing a handler that was never + // attached is a no-op, which is what lets this be armed before the child + // exists. + child.off("close", onExit); + child.off("error", onFailure); + child.stdout.off("data", onStdout); + child.stderr.off("data", onStderr); + child.stdin?.off("error", onStdinError); } - yield* closed.operation; }); - const outcome = withResolvers(); + child = + input === undefined + ? spawnChild(command, [...args], { ...options, stdio: ["ignore", "pipe", "pipe"] }) + : spawnChild(command, [...args], { ...options, stdio: ["pipe", "pipe", "pipe"] }); + + observe?.(child, captured); + + child.stdout.setEncoding("utf8"); + child.stderr.setEncoding("utf8"); + child.on("close", onExit); + child.on("error", onFailure); + child.stdout.on("data", onStdout); + child.stderr.on("data", onStderr); if (input !== undefined && child.stdin !== null) { - // A pipe the command stops reading is the command's answer, and its exit - // status is what says so — but the write still fails, and an unhandled - // stream error would take the process down rather than this operation. - // Reporting it here lets `close` settle first when there is an exit to - // report. - child.stdin.on("error", (error: Error) => { - outcome.reject(error); - }); + child.stdin.on("error", onStdinError); child.stdin.end(input, "utf8"); } - let stdout = ""; - let stderr = ""; - child.stdout.setEncoding("utf8"); - child.stderr.setEncoding("utf8"); - child.stdout.on("data", (chunk: string) => { - stdout += chunk; - }); - child.stderr.on("data", (chunk: string) => { - stderr += chunk; - }); - // `close` rather than `exit`: it is the event that fires once both pipes have - // ended, so what is read here is everything the command wrote rather than - // whatever had arrived when it stopped. - child.on("close", (code: number | null) => { - outcome.resolve({ code: code ?? -1, stdout, stderr }); - }); - child.on("error", (error: Error) => { - outcome.reject(error); - }); const result = yield* outcome.operation; settled = true; diff --git a/packages/workflow/src/deno/workspace/evaluate.ts b/packages/workflow/src/deno/workspace/evaluate.ts index 51d76d8a7..e6eb2eb4c 100644 --- a/packages/workflow/src/deno/workspace/evaluate.ts +++ b/packages/workflow/src/deno/workspace/evaluate.ts @@ -1,351 +1,240 @@ /** - * `` — the workflow host's generated-XMD boundary + * What a workflow run lets a generated fragment do * (specs/workflow-workspace-spec.md §8.4). * * An Agent proposes an observation by returning a fragment of Executable - * Markdown. This is the operation an authored workflow uses to have that - * fragment admitted and performed. It is an ordinary registered function - * component, not a new execution primitive: the document writes it where it - * wants the observation to happen, and everything else about the loop — - * ``, ``, ``, `` — is Markdown the reader can audit. + * Markdown, and an authored workflow writes `` where it wants that + * fragment admitted and performed. The component is canonical core's — public, + * protected, and the same in every execution. What this file states is the + * *ceiling*, which is the only part a host owns. * - * ## Registration is availability, not authority + * ## Every ceiling is captured before a document exists * - * Being registered is what makes this operation *reachable* from a trusted - * workflow document. It carries none of the authority the operation exercises. - * Every ceiling comes from values captured here, when the host installs the - * component and before any document exists: + * This is built once, when the workflow attachment assembles the execution, and + * canonical execution captures it by value there. Nothing a document, a prop, a + * binding, a context, a contextual API answer, middleware or a generated name + * does reaches back into it. * - * - the retained Workspace roots and the run's authoritative current root come - * from the captured run storage, read at invocation. They are as-of-admission - * provenance rather than a frozen future ceiling: a continuation holds the - * admission's roots by membership, so the run's own later publications and an - * advanced retained current root change nothing the admission was granted - * under; - * - the read-only `` observation comes from core's pinned constructor; - * - `` is admitted only when the captured request ceiling is non-empty; + * - the read table is core's read-only ``; `` joins it only + * where this host also states the exact requests it may perform; * - the write table is core's paired ``, workflow's own lexical `` * built from the same definition the ordinary registration owns, and core's - * self-closing ``; and - * - any further read or write comes from the captured host option. - * - * No prop, binding, context, contextual API answer, middleware return value or - * generated name supplies or widens any of them. The schema is closed on one - * required string and one optional closed selection, and paired content is - * refused — a `source` this element rendered rather than received would be a - * fragment nobody handed it. + * self-closing ``; + * - any further read or write comes from the captured host option; and + * - the Workspace basis is answered per invocation by the private operation + * below, because a run's own progress legitimately advances it: every + * committed mutation retains another immutable root. A continuation holds the + * admission's roots by membership, so later publications and an advanced + * current root change nothing the admission was granted under. * * ## `allow` narrows; it never grants * - * `allow` names an effect *class*, and the class resolves to the table this - * host installed before any document existed. Omitting it asks for `read`, - * which is what this component has always done. Asking for `write` reaches - * core's paired ``, workflow's `` and core's self-closing - * `` and nothing else — no Git, no Git host, no Issue, no process, - * no credential — and a host that installed no write table refuses the - * selection before the candidate is parsed. + * `allow` names an effect *class*, and the class resolves to the table stated + * here. Omitting it asks for `read`. Asking for `write` reaches core's paired + * ``, workflow's `` and core's self-closing `` and + * nothing else — no Git, no Git host, no Issue, no process, no credential — and + * a host that installed no write table refuses the selection before the + * candidate is parsed. * * A deletion is admitted on the same terms as the other two, and accounted for * the same way: it invokes the ordinary component, crosses the run's existing * effect transaction, and is retained by the `workspace_file` effect that - * transaction publishes. Nothing about it reaches the value below. - * - * `allow` and `as` are independent. `as` grants nothing: it is the ordinary - * caller-owned binding for the value this component returns, and that value has - * the same shape for every selection. An admitted write puts nothing in it. - * - * A repository component named `Evaluate` shadows this default, exactly as it - * shadows any other. That can change what a trusted workflow document does; it - * cannot recover the closure this registration holds. Agent-generated source - * never resolves through here at all: the evaluator resolves only its own closed - * table of pinned identities. + * transaction publishes. Nothing about it reaches the value `` + * returns. * - * ## It answers with a value, and declares no `returns` + * ## The `source` spelling * - * Each admitted observation's own value, in invocation order, with whatever the - * fragment rendered beside them under `output` — not the rendered output alone, - * which for an uncaptured `` is empty. It binds by reference under `as`, - * unchecked, so nothing about the value is rewritten on the way to the document; - * a document turns it into text where it wants text, with ``. `as="observation"` captures and suppresses it exactly as it does for - * any other component, which is how a document renders the result into the next - * `` in the same session. - * - * It is deliberately **not** wrapped in `printErrors`. A refused or failed - * observation must stop the authored loop unless the document puts a recovery - * boundary around it: returning a refusal as observation text would leave the - * Agent reasoning from a read that never happened. + * Workflow documents were written against `source` before `text` existed, so + * this profile admits it and canonical `` accepts it here without + * complaint. The ordinary run profile does not: it never shipped that spelling, + * and there is no document written against it to keep working. */ +import { API, timeoutFetch } from "@executablemd/runtime"; import type { Operation } from "effection"; -import { getExpansion, hasContent } from "@executablemd/core"; -import type { SourcePosition } from "@executablemd/core"; import { - pinnedFileDelete, - pinnedFileRead, - pinnedFileWrite, - pinnedMutation, + detachHeaders, + detachStatus, + directoryEntry, + fetchEntry, + fileDeleteEntry, + fileReadEntry, + fileWriteEntry, } from "@executablemd/core/host"; import type { - GeneratedEffectClass, - GeneratedMutation, - GeneratedObservation, - GeneratedObservationResult, + FetchResponseRecord, + FragmentEntry, + FragmentEvaluationInput, + FragmentFetchAccess, + FragmentFileAccess, + FragmentWorkspaceAccess, GeneratedRequest, } from "@executablemd/core/host"; -import type { ComponentInvocation } from "@executablemd/core"; -import type { IdentityClaimant, IdentityComponent } from "@executablemd/core/host"; -import type { Json } from "@executablemd/durable-streams"; -import type { FunctionComponent } from "@executablemd/core"; import type { WorkflowRunDatabase } from "../../storage/api.ts"; -import { - evaluateGeneratedFragment, - type GeneratedEvaluationPolicy, -} from "../../generated-observations.ts"; -import { COMPOSITION_ORIGIN, dirDefinition } from "../../composition/definitions.ts"; +import { COMPOSITION_ORIGIN } from "../../composition/definitions.ts"; +import { workflowFilesHandler } from "./files.ts"; +import { WORKSPACE_ROOT } from "./logical-path.ts"; import { workspaceRootSelection } from "./effect.ts"; -const ORIGIN = "@executablemd/workflow/generated"; - -/** The classes this component accepts, and the order a selection canonicalizes to. */ -const EFFECT_CLASSES: readonly GeneratedEffectClass[] = ["read", "write"]; - -/** - * The whole schema: one required string, one optional closed selection, and - * nothing else accepted. - * - * `allow` narrows the host's already-installed policy and can do nothing else. - * It names no identity, no root, no destination and no request — it says which - * of the two tables this host captured before the document existed this - * fragment may draw on, and omitting it asks for `read`. - */ -export const props = { - type: "object", - properties: { - source: { type: "string" }, - allow: { - type: "array", - minItems: 1, - uniqueItems: true, - items: { enum: [...EFFECT_CLASSES] }, - }, - }, - required: ["source"], - additionalProperties: false, -}; - -/** - * What the document reads back: a detached value, not text. - * - * The observations' own values, in the order the fragment invoked them, with - * whatever the fragment rendered kept beside them under `output`. An admitted - * `` written without a binding renders nothing — a component returning a - * non-string has nowhere to render — so a result taken from the rendered - * fragment would answer the Agent's question with an empty string. - * - * A value rather than a serialization, because deciding how a value becomes text - * is the document's to make and it has `` to make it with. The pinned - * identity each observation ran under stays on the host-facing result and out of - * this one: it says which implementation the host admitted, which is a fact about - * the run rather than something the next prompt is answering. - */ -function observationValue(result: GeneratedObservationResult): Json { - return { - observations: result.observations.map((observation) => ({ - name: observation.name, - value: observation.value, - })), - output: result.output, - }; -} - -/** A generated fragment offered to this host in a form it does not take. */ -export class GeneratedEvaluationError extends Error { - override name = "GeneratedEvaluationError"; -} - /** * What a host may configure about generated evaluation. * * Adapter-private values, supplied before the document runs. Every one of them - * is additive: production may supply none, which is the standard profile below + * is additive: production may supply none, which is the standard profile above * and nothing else. A document prop supplies none of them. */ export interface GeneratedEvaluationOptions { /** * The exact HTTP reads an admitted fragment may perform. * - * Empty admits `` not at all — the pinned identity is simply not on the - * allowlist, which is a different thing from admitting it and refusing every - * request. + * Empty admits `` not at all — the pinned identity is simply not on + * the allowlist, which is a different thing from admitting it and refusing + * every request. */ readonly requests?: readonly GeneratedRequest[]; /** Further read components this host admits beside core's pinned ones. */ - readonly reads?: readonly GeneratedObservation[]; + readonly reads?: readonly FragmentEntry[]; /** Further mutation components this host admits beside the standard profile's. */ - readonly writes?: readonly GeneratedMutation[]; + readonly writes?: readonly FragmentEntry[]; } /** - * The classes this element asked for, canonicalized. + * The run's retained Workspace basis, answered per invocation. * - * Parsed rather than read. The schema above already refuses everything but a - * non-empty duplicate-free subset, and what this adds is the canonical order - * the admission retains: two documents asking for the same two classes are - * asking for the same thing, so authored order is not part of the policy a - * continuation is held to. + * A private operation closed over this run's own storage rather than the + * storage itself: the profile canonical execution captures carries the answer + * an invocation needs and no way to reach the database that produced it. */ -function requestedClasses(value: Json | undefined): readonly GeneratedEffectClass[] | undefined { - if (value === undefined) { - return undefined; - } - if (!Array.isArray(value) || value.length === 0) { - throw new GeneratedEvaluationError( - " takes `allow` as a non-empty array of effect classes.", - ); - } - const selected = new Set(); - for (const entry of value) { - const effect = EFFECT_CLASSES.find((known) => known === entry); - if (effect === undefined) { - throw new GeneratedEvaluationError( - " admits the effect classes `read` and `write`, and nothing else.", - ); - } - if (selected.has(effect)) { - throw new GeneratedEvaluationError(" takes each effect class in `allow` once."); - } - selected.add(effect); - } - return EFFECT_CLASSES.filter((effect) => selected.has(effect)); +function workspaceAccess(database: WorkflowRunDatabase): FragmentWorkspaceAccess { + return { + *snapshot() { + const selection = yield* workspaceRootSelection(database); + return { roots: selection.roots, current: selection.current }; + }, + }; } /** - * The implementation, built from the claimant the execution delivered. + * The exact filesystem operations an admitted fragment performs in this run. + * + * This run's own transaction-bound handler, built here and handed to the + * profile, so an admitted effect still crosses the run's effect transaction and + * is still retained by the `workspace_file` effect that transaction publishes. + * What changed is where the handler comes from: a fragment no longer resolves + * `API.Files` when it runs, so the provider a document, a repository component + * or middleware installed nearer is not between a fragment and the Workspace. * - * Called once per attachment, by canonical execution, with the claimant it - * minted for this component — and by nothing else. The claimant is the argument - * of that call: it is closed over here and reachable from nowhere, so an - * implementation kept from this attachment names nothing anywhere else. + * Five operations, not seven. The handler also globs and makes temporary + * directories; an admitted fragment does neither. */ -function createEvaluate( - database: WorkflowRunDatabase, - options: GeneratedEvaluationOptions, - claim: IdentityClaimant, -): FunctionComponent { - // Copied at construction. What the host stated then is what every later - // invocation is bounded by, whatever happens to the object it passed. - const requests: GeneratedRequest[] = [...(options.requests ?? [])]; - // The standard profile, captured with them. Core's read-only `` is the - // read table; core's paired ``, workflow's own lexical `` and - // core's self-closing `` are the write table, and the Dir - // identity is built from the same definition the ordinary registration owns so - // the two cannot drift. Both tables are stated whether or not a document asks - // for the class — `allow` selects from what this host already installed, and - // can add nothing to it. - // - // The order is the retained policy, so it is stated once and kept: a - // continuation compares what it was granted under against what this profile - // holds now, and a host extension arriving last is what keeps a profile that - // took none comparable to itself. - const dir = dirDefinition(); - const reads: GeneratedObservation[] = [pinnedFileRead(), ...(options.reads ?? [])]; - const writes: GeneratedMutation[] = [ - pinnedFileWrite(), - // Versioned, because what this entry authorizes changed. The former - // `@executablemd/workflow/composition#Dir` authorized placement that created - // nothing; `` now recursively creates the directory it names, and that - // is persistent mutation of the run's Workspace. A continuation granted - // under the old identity was granted something narrower than this, so it - // must not silently receive the wider authority — the retained comparison - // sees a different identity and refuses before generated execution. - pinnedMutation(dir.name, `${COMPOSITION_ORIGIN}/dir-v2#Dir`, dir, "paired"), - pinnedFileDelete(), - ...(options.writes ?? []), - ]; - - return function* Evaluate( - elementProps: Record, - invocation: ComponentInvocation, - ): Operation { - if (yield* hasContent()) { - throw new GeneratedEvaluationError( - " takes the generated source as its `source` prop and renders no content of " + - "its own. Write it self-closing.", - ); - } - const source = elementProps.source; - if (typeof source !== "string") { - throw new GeneratedEvaluationError(" requires a `source` string to evaluate."); - } - // Before the durable name is claimed and before the candidate is read: a - // selection this host cannot state is not a fragment being refused. - const allow = requestedClasses(elementProps.allow); - - // The durable identity of this observation is this element's own invocation, - // so a replay restores the fragment this position admitted rather than - // whichever one a later turn happens to be holding. - // - // Claimed here, in the frame the engine invoked this in and on the exact - // invocation it was handed: the claimant answers only for an invocation this - // execution minted where resolution selected this implementation, and for - // nothing a handler substituted. It is read from no Context, no contextual - // Api answer, no definition and no registry answer — every one of those is - // composable, and any of them would let two `` sites share one - // durable name and each replay the other's admitted fragment. - const id = yield* claim(invocation); - - // Read where the work is not journaled: the element's shape and the run's - // current roots are both ordinary operations, and only the admission below - // belongs in the run's history. - const selection = yield* workspaceRootSelection(database); - - const policy: GeneratedEvaluationPolicy = { - workspaceRoots: selection.roots, - selectedRoot: selection.current, - requests, - reads, - writes, - ...(allow === undefined ? {} : { allow }), - }; - - // Where this authored `` was written, as diagnostic journal data - // beside the admission. A generated fragment's own elements are scanned - // from a dynamic string and carry no authored position of their own. - const expansion = yield* getExpansion(); - - return yield* admit(id, source, policy, expansion.position); +function workspaceFiles(database: WorkflowRunDatabase): FragmentFileAccess { + const handler = workflowFilesHandler(database); + return { + checkFilePath: (input) => handler.checkFilePath(input), + readTextFile: (input) => handler.readTextFile(input), + writeTextFile: (input) => handler.writeTextFile(input), + deleteFile: (input) => handler.deleteFile(input), + ensureDirectory: (input) => handler.ensureDirectory(input), + // The Workspace root, and a logical path rather than a host one — the same + // root the run's documents resolve against. Nothing an admitted fragment + // writes reaches the directory the caller invoked `xmd` from. + // deno-lint-ignore require-yield + *workingDirectory(): Operation { + return WORKSPACE_ROOT; + }, }; } -/** The admission itself, and the only part of this that is journaled. */ -function* admit( - id: string, - source: string, - policy: GeneratedEvaluationPolicy, - position: Readonly | undefined, -): Operation { - const result = yield* evaluateGeneratedFragment(id, source, policy, position); - return observationValue(result); -} - /** - * What a workflow host declares to the execution for one attachment. + * The ceiling a workflow run's generated fragments are admitted under. * - * `` is not registered with the rest of the workflow's components: - * its implementation names durable work after its own invocation, so canonical - * execution builds it from the claimant it minted (executable-mdx-spec §5.6). A - * run whose host declares none has no `` at all. + * An operation, because the effective Fetch timeout is resolved here — once, + * by the host, at assembly. A ceiling compared across a suspension cannot + * depend on where the comparison happened, so preflight and execution normalize + * against this value rather than reading the context again. */ -export function evaluationComponents( +export function* evaluationProfile( database: WorkflowRunDatabase, options: GeneratedEvaluationOptions = {}, -): readonly IdentityComponent[] { - return [ - { - name: "Evaluate", - origin: ORIGIN, - props, - factory: (claim: IdentityClaimant) => createEvaluate(database, options, claim), +): Operation { + const requests = [...(options.requests ?? [])]; + // The transport, read once here rather than by a fragment when it runs. It is + // the run's own `API.Fetch` as this assembly sees it — which is the provider + // the workflow host installed, not whichever one a document later composes + // around itself. + const transport = yield* fetchAccess(); + // Built from the same definition the ordinary registration owns, so the two + // cannot drift. Versioned in its revision because what the entry authorizes + // changed: the former `Dir` authorized placement that created nothing, and + // `` now recursively creates the directory it names. A continuation + // granted under the earlier revision must not silently receive the wider + // authority, and the retained comparison refuses it before generated + // execution. + const timeout = yield* timeoutFetch; + return { + read: [ + fileReadEntry(), + ...(requests.length === 0 ? [] : [fetchEntry(requests)]), + ...(options.reads ?? []), + ], + write: [ + fileWriteEntry(), + // Revision 3: the grant is the workflow's, so the identity names this + // package. What changed from revision 2 is the authority behind it — the + // body is now closed over the `ensureDirectory` this profile handed over + // rather than resolving a Files provider when it runs — so a continuation + // granted under the older, composable one is refused rather than + // re-granted. + // + // The version-1 alias is the exact string released builds retained for + // this entry, written out rather than assembled: that is what those + // journals hold, and nothing derives it. The pre-`dir-v2` spelling is + // deliberately absent — it named the placement-only ``, which + // created nothing, so answering for it here would hand a narrower grant + // the wider one. + directoryEntry({ origin: COMPOSITION_ORIGIN, key: "Dir", revision: "3" }, "Dir", [ + "@executablemd/workflow/composition/dir-v2#Dir", + ]), + fileDeleteEntry(), + ...(options.writes ?? []), + ], + files: workspaceFiles(database), + ...(requests.length === 0 ? {} : { fetch: transport }), + workspace: workspaceAccess(database), + deprecatedSourceAlias: true, + ...(timeout === undefined ? {} : { fetchTimeout: timeout }), + }; +} + +/** + * The transport an admitted `` performs its request through. + * + * Resolved at assembly, so the operation the profile holds is the one this host + * installed. A fragment reaches this bound function and never `API.Fetch`, so a + * handler composed around the contextual chain while the document runs neither + * sees the request nor answers it. + */ +// deno-lint-ignore require-yield +function* fetchAccess(): Operation { + // Read here, at assembly, and closed over. `API.Fetch.operations` resolves + // against whatever chain is current when it is *called*, so reading it inside + // the fragment's own body would be the dynamic lookup this exists to remove. + const perform = API.Fetch.operations.fetch; + return { + *fetch(request): Operation { + const response = yield* perform(request.url, { + method: request.method, + headers: { ...request.headers }, + ...(request.timeout === undefined ? {} : { timeout: request.timeout }), + }); + // Detached before the body is read, because a provider may invalidate its + // own header collection once the body has been consumed. A `HEAD` never + // asks for a body: there is none, and asking would fail against a + // provider that says so. + const status = detachStatus(response.status); + const headers = detachHeaders(response.headers); + const body = request.method === "HEAD" ? "" : yield* response.text(); + return { status, headers, body }; }, - ]; + }; } diff --git a/packages/workflow/tests/ambient-authentication.test.ts b/packages/workflow/tests/ambient-authentication.test.ts index 75d601486..70a2dc3aa 100644 --- a/packages/workflow/tests/ambient-authentication.test.ts +++ b/packages/workflow/tests/ambient-authentication.test.ts @@ -61,6 +61,8 @@ import { remoteBranch, remoteRefs, useBareRemote } from "./support/git-remotes.t import { useGitHttpRemote } from "./support/git-http.ts"; import type { GitHttpRemote } from "./support/git-http.ts"; import { useHomeWithoutAuthentication, useInvokingHome } from "./support/credential-home.ts"; +import { once } from "@effectionx/node/events"; +import { connect } from "node:net"; import { credential, useIssueTrackerServer } from "./support/issue-tracker-server.ts"; import { fixture as pullRequestFixture, @@ -1394,6 +1396,57 @@ describe("workflow GitHub source sessions", () => { // the request carried this process's credential rather than printing it. expect(server.requests[0]?.authorization === `Bearer ${credential()}`).toBe(true); }); + + /** + * A request the fixture is still answering is a task of the server's own + * scope, so tearing the server down ends it. Held on the server side rather + * than timed, and observed through the task's own cleanup, so what is proved + * is that the task was halted rather than that a socket closed. + */ + it("halts an in-flight request task when the server is torn down", function* () { + const reached = withResolvers(); + let cancelled = false; + let answered = false; + + yield* scoped(function* () { + const server = yield* useIssueTrackerServer({ + *hold() { + yield* ensure(() => { + cancelled = true; + }); + reached.resolve(); + yield* suspend(); + }, + }); + const { port } = new URL(server.url); + + const socket = connect(Number(port), "127.0.0.1"); + const onError = (): void => {}; + const onData = (): void => { + answered = true; + }; + + yield* ensure(() => { + socket.off("error", onError); + socket.off("data", onData); + socket.destroy(); + }); + + socket.on("error", onError); + socket.on("data", onData); + + yield* once(socket, "connect"); + socket.write( + `GET /repos/octo/project/issues/7 HTTP/1.1\r\nHost: 127.0.0.1:${port}\r\n` + + `Authorization: Bearer ${credential()}\r\nConnection: close\r\n\r\n`, + ); + + yield* reached.operation; + }); + + expect(cancelled).toBe(true); + expect(answered).toBe(false); + }); }); describe("workflow GitHub ambient credentials", () => { diff --git a/packages/workflow/tests/credential-helper.test.ts b/packages/workflow/tests/credential-helper.test.ts index 33d97f1e8..3c40530c5 100644 --- a/packages/workflow/tests/credential-helper.test.ts +++ b/packages/workflow/tests/credential-helper.test.ts @@ -17,8 +17,11 @@ */ import { describe, it } from "@executablemd/test-support/bdd"; +import type { ChildProcessByStdio } from "node:child_process"; +import type { Readable, Writable } from "node:stream"; +import { runProcess } from "../src/deno/composition/subprocess.ts"; import { expect } from "@executablemd/test-support/expect"; -import { scoped, until, type Operation } from "effection"; +import { scoped, spawn, until, withResolvers, type Operation } from "effection"; import { chmod, readFile, stat, writeFile } from "node:fs/promises"; import { spawnSync } from "node:child_process"; import { join } from "node:path"; @@ -482,4 +485,101 @@ describe("workflow credential injected infrastructure faults", () => { }); expect(String(raised)).toContain("injected marker-read failure"); }); + /** + * The child a command owns carries its handlers for as long as the scope + * that acquired it runs — `runProcess` registers with `ensure()`, and + * delegation gives a generator no frame of its own. Cancelling that scope + * while the child is still alive is the case the event never comes for, so + * the counts are read while they are live, again after the halt, and once + * more after the events are replayed. + */ + it("releases every handler when the command's owner is cancelled", function* () { + let child: ChildProcessByStdio | undefined; + let before: number[] = []; + let settled = false; + let captured: () => { stdout: string; stderr: string } = () => ({ stdout: "", stderr: "" }); + const spawned = withResolvers(); + + const owner = yield* spawn(function* () { + yield* runProcess({ + command: "sh", + args: ["-c", "sleep 30"], + cwd: ".", + env: {}, + observe: (started, reader) => { + child = started; + before = attachedToChild(started); + captured = reader; + spawned.resolve(); + }, + }); + settled = true; + }); + + // Resumed from inside the operation's own synchronous prefix, so the halt + // below lands at its very first suspension — the boundary at which an + // `ensure()` yielded after the subscriptions would not yet have been + // established. The cleanup here is armed before the child is reachable, so + // this is precisely where it has to hold. + yield* spawned.operation; + if (!child) { + throw new Error("the command never owned a child"); + } + + // One `close`, one `error`, and one on each output pipe — the reap keeps no + // standing listener of its own. Nothing on stdin, which this invocation + // left closed. + const mine = [1, 1, 1, 1, 0]; + const live = attachedToChild(child); + + live.forEach((count, index) => { + expect(count).toBeGreaterThanOrEqual(before[index] + mine[index]); + }); + + const seen = captured(); + + yield* owner.halt(); + + const released = live.map((count, index) => count - mine[index]); + + expect(attachedToChild(child)).toEqual(released); + + // Delivered through a sentinel, because an `error` an emitter has no + // listener for is thrown rather than dropped — so the replay needs one + // observer, and exactly one is what it must find. + let reached = 0; + const sentinel = (): void => { + reached += 1; + }; + + child.emit("close", 0, null); + child.stdout.emit("data", "after the owner was cancelled"); + child.stderr.emit("data", "after the owner was cancelled"); + child.on("error", sentinel); + try { + child.emit("error", new Error("after the owner was cancelled")); + } finally { + child.off("error", sentinel); + } + + expect(reached).toBe(1); + expect(attachedToChild(child)).toEqual(released); + // Nothing is still accumulating, and the cancelled operation produced no + // result that a later event could have resumed. + expect(captured()).toEqual(seen); + expect(settled).toBe(false); + }); }); + +/** What one owned child is carrying, across itself and its pipes. */ +function attachedToChild( + child: ChildProcessByStdio, +): number[] { + return [ + child.listenerCount("error"), + child.listenerCount("close"), + child.stdout.listenerCount("data"), + child.stderr.listenerCount("data"), + child.stdin?.listenerCount("error") ?? 0, + ]; +} diff --git a/packages/workflow/tests/generated-agent-component.test.ts b/packages/workflow/tests/generated-agent-component.test.ts index 980ff9ba6..8901a06f2 100644 --- a/packages/workflow/tests/generated-agent-component.test.ts +++ b/packages/workflow/tests/generated-agent-component.test.ts @@ -31,7 +31,7 @@ import { fileURLToPath } from "node:url"; import type { FetchInit, RuntimeFetchResponse } from "@executablemd/runtime"; import type { WorkflowRunDatabase } from "../mod.ts"; import { withWorkflowWorkspace } from "../src/deno/workspace/host.ts"; -import { evaluationComponents } from "../src/deno/workspace/evaluate.ts"; +import { evaluationProfile } from "../src/deno/workspace/evaluate.ts"; import type { GeneratedEvaluationOptions } from "../src/deno/workspace/evaluate.ts"; import { transactWorkspaceRoots } from "../src/deno/workspace/private.ts"; import { createRun, useStorageRoot, withStorage } from "./support/storage.ts"; @@ -123,10 +123,10 @@ function runDocument( return yield* collect( yield* executeInstalled( { ...inlineSource(source), stream: database.journal }, - // `` names durable work after its own invocation, so - // this run declares it to the execution and canonical execution - // builds it from the claimant it minted for this attachment. - [{ components: evaluationComponents(database, evaluation) }], + // `` is canonical core's. What this run states is the + // ceiling, captured by canonical execution before any document + // code exists. + [{ evaluation: yield* evaluationProfile(database, evaluation) }], ), ); }), @@ -216,6 +216,20 @@ function policyOf(event: DurableEvent): Record | undefined { return input; } +/** + * The Workspace basis one admission was recorded under. + * + * A version-2 policy carries it as one member, because a host that evaluates + * against no Workspace states none at all. + */ +function basisOf(event: DurableEvent): Record | undefined { + const workspace = policyOf(event)?.workspace; + if (typeof workspace !== "object" || workspace === null || Array.isArray(workspace)) { + return undefined; + } + return workspace; +} + /** * What the run reported, whether it rendered it or failed with it. * @@ -335,19 +349,30 @@ function* plant(database: WorkflowRunDatabase, path: string, content: string): O } } +/** + * One capability identity, in the closed structural shape a run retains. + * + * Written out here rather than imported from the profile, so a row states what + * the journal must hold rather than agreeing with whatever the profile computed. + */ +function capability(origin: string, key: string, revision: string): Record { + return { kind: "capability", origin, key, revision }; +} + describe("Tier WGAC — the registered Evaluate component", () => { - it("WGAC3: it takes only `source`, and refuses content", function* () { + it("WGAC3: the program is stated once, and the schema is closed", function* () { const root = yield* useStorageRoot(); yield* withStorage(root, function* () { const database = yield* createRun(); - // Paired content: a `source` this element rendered is not a fragment - // anybody handed it. + // The two input forms are disjoint. Content produces the program, so an + // element that also states one is stating it twice — and which of them + // ran would be a matter of precedence rather than of what was written. const paired = yield* runDocument( database, `\n\n\n`, ); - expect(reported(paired)).toContain("renders no content of its own"); + expect(reported(paired)).toContain("does not also carry `text`"); expect(admissions(paired.events)).toHaveLength(0); }); @@ -368,7 +393,7 @@ describe("Tier WGAC — the registered Evaluate component", () => { yield* withStorage(third, function* () { const database = yield* createRun(); const missing = yield* runDocument(database, `\n`); - expect(reported(missing)).toMatch(/source/i); + expect(reported(missing)).toMatch(/requires the program/i); expect(admissions(missing.events)).toHaveLength(0); }); }); @@ -401,8 +426,8 @@ describe("Tier WGAC — the registered Evaluate component", () => { const recorded = admissions(attempt.events); expect(recorded).toHaveLength(2); - const first = policyOf(recorded[0]!); - const second = policyOf(recorded[1]!); + const first = basisOf(recorded[0]!); + const second = basisOf(recorded[1]!); // The run moved between them, and the stated ceiling moved with it. expect(typeof first?.selectedRoot).toBe("string"); @@ -505,7 +530,7 @@ describe("Tier WGAC — the registered Evaluate component", () => { }); describe("WGAC7: the durable name survives the import boundary", () => { - it("refuses an invocation the wrapper built", function* () { + it("refuses any definition a wrapper answers the import with", function* () { const root = yield* useStorageRoot(); const source = yield* fixture(); yield* withStorage(root, function* () { @@ -513,62 +538,48 @@ describe("Tier WGAC — the registered Evaluate component", () => { yield* plant(database, "alpha.md", ADMITTED_NOTE); const attempt = yield* scoped(function* () { - // A structural stand-in — which is what the identity used to be, and - // what a wrapper would mint to give both sites one durable name. It - // answers the authored form too: implementing the whole public shape - // is exactly what a forger would do, and identity is the private - // field rather than the shape. - yield* interpose(() => ({ hasContent: () => false })); + // A wrapper that forwards the genuine issuance and changes nothing + // else. It is refused anyway, and that is the point: `` is + // a name canonical core owns, so the only definition that may reach + // the call site is the one canonical execution produced. A handler + // that answered an import here — however honestly — would be + // deciding what an unaudited fragment is allowed to do. + yield* interpose((invocation) => invocation); return yield* runDocument(database, source, CEILING); }); - expect(reported(attempt)).toContain("not an invocation the engine issued"); - // Refused before any admission: nothing was named, so nothing collapsed. + expect(reported(attempt)).toContain("canonical core owns"); + expect(reported(attempt)).toContain("only canonical execution answers one"); + // Refused before any admission: nothing was named, so nothing ran. expect(admissions(attempt.events)).toHaveLength(0); + expect(attempt.performed).toEqual([]); }); }); - it("refuses the first site's invocation routed at the second", function* () { + it("admits each site under its own name across a handler that only observes", function* () { const root = yield* useStorageRoot(); const source = yield* fixture(); yield* withStorage(root, function* () { const database = yield* createRun(); yield* plant(database, "alpha.md", ADMITTED_NOTE); + const observed: string[] = []; const attempt = yield* scoped(function* () { - // Ordinary delegation at the first site, then the first site's own - // invocation handed to the second — the substitution that would make - // both sites replay one admitted fragment. - let kept: ComponentInvocation | undefined; - yield* interpose((invocation) => { - kept ??= invocation; - return kept; + // Observing the import and handing back exactly what came out of the + // chain: the one thing a handler may still do at a protected name, + // and it stays supported. + yield* Component.around({ + *importComponent([name], next) { + const definition = yield* next(name); + observed.push(name); + return definition; + }, }); return yield* runDocument(database, source, CEILING); }); - expect(reported(attempt)).toMatch(/already been taken|has finished/); - // The first site was admitted under its own name. The second was - // refused rather than admitted under the first site's. - expect(admissions(attempt.events)).toHaveLength(1); - }); - }); - - it("admits each site under its own name when the wrapper delegates", function* () { - const root = yield* useStorageRoot(); - const source = yield* fixture(); - yield* withStorage(root, function* () { - const database = yield* createRun(); - yield* plant(database, "alpha.md", ADMITTED_NOTE); - - const attempt = yield* scoped(function* () { - // Forwarding the genuine issuance: ordinary delegation, and it stays - // supported. - yield* interpose((invocation) => invocation); - return yield* runDocument(database, source, CEILING); - }); - expect(attempt.failure).toBe(undefined); + expect(observed).toContain("Evaluate"); const recorded = admissions(attempt.events); expect(recorded).toHaveLength(2); expect(new Set(recorded.map(nameOf)).size).toBe(2); @@ -580,44 +591,6 @@ describe("Tier WGAC — the registered Evaluate component", () => { }); }); - it("refuses a live parent's invocation routed into the sites inside it", function* () { - const root = yield* useStorageRoot(); - const source = yield* fixture("nested-observations"); - yield* withStorage(root, function* () { - const database = yield* createRun(); - yield* plant(database, "alpha.md", ADMITTED_NOTE); - - const attempt = yield* scoped(function* () { - yield* useFrame(); - // `` is still running — it has not returned, and it never - // claimed anything — so its issuance is genuine, live and unspent - // while the sites in its content run. Routing it there is the - // substitution a spent or finished sibling's does not reach. Two - // things refuse it, and the first reached is that `` is not - // ``; Tier CIV nests one component inside itself to hold - // the projection on its own. - let parent: ComponentInvocation | undefined; - yield* interpose( - (invocation, name) => { - if (name === "Frame") { - parent = invocation; - return invocation; - } - return parent ?? invocation; - }, - ["Frame", "Evaluate"], - ); - return yield* runDocument(database, source, CEILING); - }); - - expect(reported(attempt)).toContain("invocation of "); - // Refused before admission: no record was written under the parent's - // identity, so nothing can replay under its retained history. - expect(admissions(attempt.events)).toHaveLength(0); - expect(attempt.performed).toEqual([]); - }); - }); - it("keeps each site's own identity under the same live parent", function* () { const root = yield* useStorageRoot(); const source = yield* fixture("nested-observations"); @@ -627,9 +600,13 @@ describe("Tier WGAC — the registered Evaluate component", () => { const attempt = yield* scoped(function* () { yield* useFrame(); - // The same document and the same parent, forwarding honestly. Being - // nested changes nothing about what each site is named. - yield* interpose((invocation) => invocation, ["Frame", "Evaluate"]); + // `` is still running while the sites in its content run, so + // its issuance is genuine, live and unspent — the substitution a + // spent or finished sibling's does not reach. It is routed at the + // ordinary component it wraps, and the `` sites inside it + // are left to canonical execution. Being nested changes nothing about + // what each site is named. + yield* interpose((invocation) => invocation, ["Frame"]); return yield* runDocument(database, source, CEILING); }); @@ -694,7 +671,13 @@ describe("Tier WGAC — the registered Evaluate component", () => { return yield* runDocument(database, source, CEILING); }); - expect(reported(attempt)).toContain("invocation of "); + // The claimant, not the import tier: the definition handed out at the + // real `` site was passed through untouched, so the import + // was answered by canonical execution and the refusal happens where the + // borrowed implementation tries to name durable work at somebody else's + // element. + expect(reported(attempt)).toContain("invoked by canonical core"); + expect(reported(attempt)).toContain("observes nothing"); // The real site was admitted, and nothing else was: no second record, // and no request under an identity the author wrote no observation at. const recorded = admissions(attempt.events); @@ -784,7 +767,12 @@ describe("Tier WGAC — the registered Evaluate component", () => { // committed, and the request it is holding not yet performed. const heldRecords = admissions(yield* owner.journal.readAll()); expect(heldRecords).toHaveLength(2); - expect(record).not.toBe(undefined); + // There is no registration to carry across. A protected name is the + // resolver's own table rather than an answer a registry gives, so the + // record a handler would have kept from the first attachment does not + // exist — and the implementation it did keep came out of the first + // attachment's own execution. + expect(record).toBe(undefined); expect(kept).not.toBe(undefined); const second = yield* scoped(function* () { @@ -794,10 +782,13 @@ describe("Tier WGAC — the registered Evaluate component", () => { // Refused because this invocation belongs to another installation — // not because the first attachment had gone. It is still running: its - // claimant is active, and the refusal says so by naming the domain - // rather than the execution. - expect(reported(second)).toContain("as this execution installed it"); - expect(reported(second)).not.toContain("is not running this"); + // Refused one layer earlier than the claimant would have refused it: + // the second attachment's handler answered a protected import with a + // definition that attachment's execution did not produce, and the first + // attachment's implementation never reaches the call site to be asked + // whose invocation this is. + expect(reported(second)).toContain("canonical core owns"); + expect(reported(second)).toContain("did not produce"); // Neither attachment admitted anything the other's expansion named, and // the borrowed implementation performed no request. expect(admissions(second.events)).toHaveLength(0); @@ -1024,15 +1015,19 @@ describe("Tier WGAC — the standard write table", () => { // resumed run is held to. const policy = policyOf(admissions(attempt.events)[0]!); expect(policy?.allowed).toEqual([ - { name: "File", identity: "@executablemd/core#File:write", forms: ["paired"] }, + { + name: "File", + identity: capability("@executablemd/core", "File:write", "2"), + forms: ["paired"], + }, { name: "Dir", - identity: "@executablemd/workflow/composition/dir-v2#Dir", + identity: capability("@executablemd/workflow/composition", "Dir", "3"), forms: ["paired"], }, { name: "File.Delete", - identity: "@executablemd/core#File.Delete", + identity: capability("@executablemd/core", "File.Delete", "2"), forms: ["self-closing"], }, ]); @@ -1083,13 +1078,13 @@ describe("Tier WGAC — the standard write table", () => { const admission = admissions(attempt.events)[0]!; expect(policyOf(admission)?.allowed).toContainEqual({ name: "File.Delete", - identity: "@executablemd/core#File.Delete", + identity: capability("@executablemd/core", "File.Delete", "2"), forms: ["self-closing"], }); expect(recordedNames(admission)).toEqual([ { name: "File.Delete", - identity: "@executablemd/core#File.Delete", + identity: capability("@executablemd/core", "File.Delete", "2"), form: "self-closing", }, ]); diff --git a/packages/workflow/tests/generated-observations.test.ts b/packages/workflow/tests/generated-observations.test.ts index 40be06c52..99ac2da82 100644 --- a/packages/workflow/tests/generated-observations.test.ts +++ b/packages/workflow/tests/generated-observations.test.ts @@ -180,9 +180,11 @@ describe("Tier WGX — the roots a run selects", () => { if (admission?.type !== "yield") { throw new Error("the run recorded no generated-XMD admission"); } + // The version-2 shape: a workflow host states a Workspace basis, and it + // travels as one member because an ordinary host states none at all. expect(admission.description.input).toMatchObject({ - roots: [PRIMARY, SECONDARY], - selectedRoot: PRIMARY, + version: 2, + workspace: { roots: [PRIMARY, SECONDARY], selectedRoot: PRIMARY }, }); }); diff --git a/packages/workflow/tests/git-push-crash.test.ts b/packages/workflow/tests/git-push-crash.test.ts index 75e8d7459..8f0e5b05e 100644 --- a/packages/workflow/tests/git-push-crash.test.ts +++ b/packages/workflow/tests/git-push-crash.test.ts @@ -18,11 +18,24 @@ import { describe, it } from "@executablemd/test-support/bdd"; import { expect } from "@executablemd/test-support/expect"; +import { once } from "@effectionx/node/events"; +import { Buffer } from "node:buffer"; +import { connect } from "node:net"; +import type { ChildProcess } from "node:child_process"; import process from "node:process"; import { DatabaseSync } from "node:sqlite"; import { fileURLToPath } from "node:url"; import { exec as execProcess } from "@effectionx/process"; -import { call, type Operation, race, scoped, spawn, withResolvers } from "effection"; +import { + call, + ensure, + type Operation, + race, + scoped, + spawn, + suspend, + withResolvers, +} from "effection"; import { WorkflowRunStorage } from "../mod.ts"; import { GIT_HOST_EFFECT } from "../src/git-host/effect.ts"; import { parseGitHostReconciliationRecord } from "../src/git-host/records.ts"; @@ -45,6 +58,17 @@ import { denoRepositoryHost } from "../src/deno/composition/host.ts"; import { denoGitAuthentication } from "../src/deno/composition/authentication.ts"; import { TEST_HELPER } from "./support/composition.ts"; +/** What one backend child is carrying on the streams the fixture observes. */ +function attached(child: ChildProcess): number[] { + return [ + child.listenerCount("error"), + child.listenerCount("close"), + child.stdout?.listenerCount("data") ?? 0, + child.stdout?.listenerCount("end") ?? 0, + child.stdin?.listenerCount("error") ?? 0, + ]; +} + const REPOSITORY = fileURLToPath(new URL("../../..", import.meta.url)); const CRASH_CHILD = fileURLToPath(new URL("./support/git-crash-child.ts", import.meta.url)); @@ -187,11 +211,17 @@ describe("workflow Git.Push across a process boundary", () => { it("adopts a protected publication under authentication it acquired afresh", function* () { const root = yield* useStorageRoot(); const bare = yield* useBareRemote(REMOTE); + // Every backend the fixture spawns, with what its streams were carrying + // before the request task attached anything — the runtime keeps listeners + // of its own on a child's stdout, so the baseline is measured rather than + // assumed to be nothing. + const backends: { child: ChildProcess; before: number[] }[] = []; const served = yield* useGitHttpRemote({ remote: bare, label: "protected", username: "crash-user", password: "crash-secret", + observeBackend: (child) => backends.push({ child, before: attached(child) }), }); const crashHome = yield* useInvokingHome([ { host: served.host, path: "remote.git", username: "crash-user", password: "crash-secret" }, @@ -345,5 +375,113 @@ describe("workflow Git.Push across a process boundary", () => { expect(remoteBranch(bare, PUSH_BRANCH)).toBe(pushedCommit); }); + + // Each backend belonged to the request task that spawned it, and that task + // has ended. The counts are read before the events are replayed, because a + // handler removed by its own event would leave the same counts behind as + // one the task released. + expect(backends.length).toBeGreaterThan(0); + for (const { child, before } of backends) { + expect(attached(child)).toEqual(before); + } + + // And nothing the fixture kept can still be reached through them. + const answered = served.requests.length; + for (const { child } of backends) { + child.emit("close", 0, null); + child.stdout?.emit("data", Buffer.from("late")); + child.stdout?.emit("end"); + } + + expect(served.requests.length).toBe(answered); + for (const { child, before } of backends) { + expect(attached(child)).toEqual(before); + } + }); + /** + * A backend the fixture is still talking to belongs to the request task that + * spawned it, and that task belongs to the server. Held open on the server + * side rather than timed, so the teardown below lands while the child is + * alive and every handler is attached: what it must prove is that the task + * killed and reaped the child and released all five before the events are + * replayed. + */ + it("kills and releases a backend the server was still talking to", function* () { + const bare = yield* useBareRemote(REMOTE); + const held = withResolvers(); + const backends: { child: ChildProcess; before: number[] }[] = []; + let live: number[] = []; + let answered = false; + + yield* scoped(function* () { + const served = yield* useGitHttpRemote({ + remote: bare, + label: "held", + username: "held-user", + password: "held-secret", + observeBackend: (child) => backends.push({ child, before: attached(child) }), + *holdBackend() { + held.resolve(); + yield* suspend(); + }, + }); + + const [host, port] = served.host.split(":"); + const name = served.locator.slice(served.locator.lastIndexOf("/") + 1); + const socket = connect(Number(port), host ?? "127.0.0.1"); + const onError = (): void => {}; + const onData = (): void => { + answered = true; + }; + + yield* ensure(() => { + socket.off("error", onError); + socket.off("data", onData); + socket.destroy(); + }); + + socket.on("error", onError); + socket.on("data", onData); + + yield* once(socket, "connect"); + const authorization = Buffer.from("held-user:held-secret").toString("base64"); + socket.write( + `GET /${name}/info/refs?service=git-upload-pack HTTP/1.1\r\n` + + `Host: ${served.host}\r\nAuthorization: Basic ${authorization}\r\n` + + `Connection: close\r\n\r\n`, + ); + + yield* held.operation; + + const observed = backends[0]; + if (!observed) { + throw new Error("the fixture spawned no backend"); + } + live = attached(observed.child); + }); + + const first = backends[0]; + if (!first) { + throw new Error("the fixture spawned no backend"); + } + + // Five: `error` and `close` on the child, `data` and `end` on its stdout, + // and `error` on its stdin — each on top of what the runtime already held. + live.forEach((count, index) => { + expect(count).toBeGreaterThanOrEqual(first.before[index] + 1); + }); + + // The task killed it and waited for it to be gone before letting go. + expect(first.child.exitCode !== null || first.child.signalCode !== null).toBe(true); + const released = live.map((count) => count - 1); + + expect(attached(first.child)).toEqual(released); + + first.child.emit("close", 0, null); + first.child.stdout?.emit("data", Buffer.from("after the server was torn down")); + + expect(attached(first.child)).toEqual(released); + // And the client was never answered, because the request never finished. + expect(answered).toBe(false); }); }); diff --git a/packages/workflow/tests/support/git-http.ts b/packages/workflow/tests/support/git-http.ts index a68fc8ede..28f81f862 100644 --- a/packages/workflow/tests/support/git-http.ts +++ b/packages/workflow/tests/support/git-http.ts @@ -14,11 +14,22 @@ */ import { spawn, spawnSync } from "node:child_process"; +import type { ChildProcess, ChildProcessByStdio } from "node:child_process"; +import type { Readable, Writable } from "node:stream"; import { Buffer } from "node:buffer"; import { createServer, type IncomingMessage, type ServerResponse } from "node:http"; +import type { Socket } from "node:net"; import { dirname } from "node:path"; import process from "node:process"; -import { ensure, type Operation, resource, until } from "effection"; +import { + ensure, + type Operation, + resource, + scoped, + until, + useScope, + withResolvers, +} from "effection"; import { git, type BareRemote } from "./git-remotes.ts"; /** One request this remote received, as a suite asserts on it. */ @@ -105,6 +116,23 @@ export interface GitHttpOptions { * authentication cleanup that must follow it. */ readonly closed?: () => void; + /** + * Handed every backend child as it is spawned. + * + * Package-private, for the cancellation regression: the listeners this + * fixture installs are on a process it owns and does not otherwise hand out, + * and their release is the thing under test. + */ + readonly observeBackend?: (child: ChildProcess) => void; + /** + * Run inside the request task, after the backend's cleanup is registered and + * before its input is forwarded. + * + * Package-private, for the cancellation regression: a backend is only + * observable while its request is still running, and this is what holds one + * there. + */ + readonly holdBackend?: () => Operation; } /** Where Git keeps `git-http-backend`, asked of Git rather than guessed. */ @@ -167,7 +195,16 @@ function headerEnd(buffered: Buffer): { at: number; width: number } | undefined return plain < 0 ? undefined : { at: plain, width: 2 }; } -/** Hand one request to `git-http-backend` and its answer back to the client. */ +/** + * Hand one request to `git-http-backend` and its answer back to the client, as + * a task of the server's own scope. + * + * The backend is a child process with four listeners on it, and both belong to + * this request rather than to the fixture: cancelling the server ends the + * request, which kills the backend, waits for it to be gone, and detaches + * every handler synchronously afterwards. A request that simply finishes takes + * the same path. + */ function serve( incoming: IncomingMessage, outgoing: ServerResponse, @@ -176,63 +213,129 @@ function serve( user: string, segment: string, directory: string, -): void { - const child = spawn(backend, [], { - env: cgiEnvironment(incoming, root, user, segment, directory), - stdio: ["pipe", "pipe", "pipe"], - }); - // A client that hung up mid-body closes this pipe under the backend. It is - // the client's answer rather than a fault of this fixture's, and an unhandled - // stream error here would take the whole suite process down. - child.stdin.on("error", () => {}); - incoming.pipe(child.stdin); + observe?: (child: ChildProcess) => void, + hold?: () => Operation, +): Operation { + return scoped(function* () { + // Declared before the cleanup below and assigned after it: a backend that + // exists before its release is registered can be stranded, because `yield* + // ensure(...)` is itself a suspension and an owner halted while it + // registers unwinds with nothing on it. + let child: ChildProcessByStdio | undefined; - let buffered = Buffer.alloc(0); - let started = false; - child.stdout.on("data", (chunk: Buffer) => { - if (started) { - outgoing.write(chunk); - return; - } - buffered = Buffer.concat([buffered, chunk]); - const end = headerEnd(buffered); - if (end === undefined) { - return; - } - let status = 200; - const headers: Record = {}; - for (const line of buffered.subarray(0, end.at).toString("utf8").split(/\r?\n/)) { - const separator = line.indexOf(":"); - if (separator < 0) { - continue; + let buffered = Buffer.alloc(0); + let started = false; + const answered = withResolvers(); + const closed = withResolvers(); + // `close`, and nothing else, is what says the backend and its pipes are + // done; an assigned exit status says only that the process ended. + let finished = false; + + // A client that hung up mid-body closes this pipe under the backend. It is + // the client's answer rather than a fault of this fixture's, and an + // unhandled stream error here would take the whole suite process down. + const onStdinError = (): void => {}; + const onStdout = (chunk: Buffer): void => { + if (started) { + outgoing.write(chunk); + return; } - const name = line.slice(0, separator).trim(); - const value = line.slice(separator + 1).trim(); - if (name.toLowerCase() === "status") { - status = Number.parseInt(value, 10) || 200; - } else { - headers[name] = value; + buffered = Buffer.concat([buffered, chunk]); + const end = headerEnd(buffered); + if (end === undefined) { + return; } - } - outgoing.writeHead(status, headers); - started = true; - const rest = buffered.subarray(end.at + end.width); - if (rest.length > 0) { - outgoing.write(rest); - } - }); - child.stdout.on("end", () => { - if (!started) { - outgoing.writeHead(500); - } - outgoing.end(); - }); - child.on("error", () => { - if (!started) { - outgoing.writeHead(500); + let status = 200; + const headers: Record = {}; + for (const line of buffered.subarray(0, end.at).toString("utf8").split(/\r?\n/)) { + const separator = line.indexOf(":"); + if (separator < 0) { + continue; + } + const name = line.slice(0, separator).trim(); + const value = line.slice(separator + 1).trim(); + if (name.toLowerCase() === "status") { + status = Number.parseInt(value, 10) || 200; + } else { + headers[name] = value; + } + } + outgoing.writeHead(status, headers); started = true; + const rest = buffered.subarray(end.at + end.width); + if (rest.length > 0) { + outgoing.write(rest); + } + }; + const onStdoutEnd = (): void => { + if (!started) { + outgoing.writeHead(500); + } + outgoing.end(); + answered.resolve(); + }; + const onChildError = (): void => { + if (!started) { + outgoing.writeHead(500); + started = true; + } + outgoing.end(); + answered.resolve(); + }; + const onClose = (): void => { + finished = true; + closed.resolve(); + }; + + // Established before the backend exists, so a request cancelled anywhere + // below still ends the process it started — and so that no instant exists + // in which a backend is running with no cleanup registered for it. + // + // Teardown keeps every handler attached through the close wait, so the + // answer this fixture was streaming is still being written while the + // backend ends. They come off, and the request is unpiped, synchronously + // once that wait has settled. + yield* ensure(function* () { + if (child === undefined) { + return; + } + + try { + child.kill("SIGKILL"); + + if (!finished) { + yield* closed.operation; + } + } finally { + incoming.unpipe(child.stdin); + child.stdin.off("error", onStdinError); + child.stdout.off("data", onStdout); + child.stdout.off("end", onStdoutEnd); + child.off("error", onChildError); + child.off("close", onClose); + } + }); + + child = spawn(backend, [], { + env: cgiEnvironment(incoming, root, user, segment, directory), + stdio: ["pipe", "pipe", "pipe"], + }); + + observe?.(child); + + child.stdin.on("error", onStdinError); + child.stdout.on("data", onStdout); + child.stdout.on("end", onStdoutEnd); + child.on("error", onChildError); + child.on("close", onClose); + + if (hold) { + yield* hold(); } - outgoing.end(); + + incoming.pipe(child.stdin); + + yield* answered.operation; }); } @@ -283,6 +386,10 @@ export function useGitHttpRemote(options: GitHttpOptions): Operation void>(); const server = createServer((incoming: IncomingMessage, outgoing: ServerResponse) => { const url = new URL(incoming.url ?? "/", "http://127.0.0.1"); const header = incoming.headers.authorization; @@ -310,8 +417,15 @@ export function useGitHttpRemote(options: GitHttpOptions): Operation options.closed?.()); + // is where that is observed. The handler is remembered so teardown can + // take it off before it destroys the connection it is attached to. + const held = incoming.socket; + const onHeldClose = (): void => { + holds.delete(held); + options.closed?.(); + }; + holds.set(held, onHeldClose); + held.on("close", onHeldClose); incoming.resume(); return; } @@ -323,11 +437,29 @@ export function useGitHttpRemote(options: GitHttpOptions): Operation + serve( + incoming, + outgoing, + backend, + entry.root, + entry.user, + segment, + entry.directory, + options.observeBackend, + options.holdBackend, + ), + ); }); yield* until(new Promise((resolve) => server.listen(0, "127.0.0.1", () => resolve()))); yield* ensure(function* () { + // Before the first suspension below, so a teardown halted part-way + // leaves no handler on a connection this server is finished with. + for (const [socket, onHeldClose] of holds) { + socket.off("close", onHeldClose); + } + holds.clear(); server.closeAllConnections(); yield* until(new Promise((resolve) => server.close(() => resolve()))); }); diff --git a/packages/workflow/tests/support/github.ts b/packages/workflow/tests/support/github.ts index 469325040..8ecb41251 100644 --- a/packages/workflow/tests/support/github.ts +++ b/packages/workflow/tests/support/github.ts @@ -18,7 +18,8 @@ import { createServer, type IncomingMessage, type ServerResponse } from "node:http"; import type { AddressInfo } from "node:net"; -import { ensure, type Operation, resource, until } from "effection"; +import { each, ensure, type Operation, resource, until, useScope } from "effection"; +import { fromReadable } from "@effectionx/node"; import type { GitHubAccess, GitHubHttpRequest, @@ -651,10 +652,20 @@ export function mutations(store: GitHubStore): string[] { */ export function useGitHubServer(store: GitHubStore): Operation { return resource(function* (provide) { + // Each request body is read by a task of this server's own scope, so + // tearing the server down ends the reads still in progress rather than + // leaving their listeners on sockets it is about to destroy. `fromReadable` + // is the scope-bound adapter for that: it attaches and detaches the + // stream's own handlers with the task. + const scope = yield* useScope(); const server = createServer((incoming: IncomingMessage, outgoing: ServerResponse) => { - const chunks: Buffer[] = []; - incoming.on("data", (chunk: Buffer) => chunks.push(chunk)); - incoming.on("end", () => { + scope.run(function* () { + const chunks: Uint8Array[] = []; + for (const chunk of yield* each(fromReadable(incoming))) { + chunks.push(chunk); + yield* each.next(); + } + const headers: Record = {}; for (const [name, value] of Object.entries(incoming.headers)) { headers[name === "authorization" ? "Authorization" : name] = String(value); diff --git a/packages/workflow/tests/support/issue-tracker-server.ts b/packages/workflow/tests/support/issue-tracker-server.ts index 54cabb51d..720182fd2 100644 --- a/packages/workflow/tests/support/issue-tracker-server.ts +++ b/packages/workflow/tests/support/issue-tracker-server.ts @@ -13,7 +13,8 @@ */ import { createServer, type IncomingMessage, type ServerResponse } from "node:http"; -import { ensure, type Operation, resource, until } from "effection"; +import { each, ensure, type Operation, resource, until, useScope } from "effection"; +import { fromReadable } from "@effectionx/node"; /** * The credential this tracker requires, held here and never in a document. @@ -93,6 +94,14 @@ export interface ServerOptions { readonly issues?: readonly ServedIssue[]; /** The credential every request must carry, so a scenario can prove one was. */ readonly token?: string; + /** + * Run inside the request task, after the body is read and before the answer + * is written. + * + * Package-private, for the cancellation regression: a request task is only + * observable while it is still running, and this is what holds one there. + */ + readonly hold?: () => Operation; } /** @@ -112,10 +121,24 @@ export function useIssueTrackerServer(options: ServerOptions = {}): Operation(); let origin = ""; + // Each request body is read by a task of this server's own scope, so + // tearing the server down ends the reads still in progress rather than + // leaving their listeners on sockets it is about to destroy. `fromReadable` + // is the scope-bound adapter for that: it attaches and detaches the + // stream's own handlers with the task. + const scope = yield* useScope(); const server = createServer((incoming: IncomingMessage, outgoing: ServerResponse) => { - const chunks: Buffer[] = []; - incoming.on("data", (chunk: Buffer) => chunks.push(chunk)); - incoming.on("end", () => { + scope.run(function* () { + const chunks: Uint8Array[] = []; + for (const chunk of yield* each(fromReadable(incoming))) { + chunks.push(chunk); + yield* each.next(); + } + + if (options.hold) { + yield* options.hold(); + } + const raw = Buffer.concat(chunks).toString("utf8"); const url = new URL(incoming.url ?? "/", origin); let body: unknown; diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml index 81786c4b8..5ea97ff75 100644 --- a/pnpm-lock.yaml +++ b/pnpm-lock.yaml @@ -27,8 +27,8 @@ importers: specifier: 0.1.1 version: 0.1.1 '@effectionx/node': - specifier: 0.2.4 - version: 0.2.4(effection@4.1.0) + specifier: 0.2.5 + version: 0.2.5(effection@4.1.0) '@effectionx/process': specifier: 0.8.1 version: 0.8.1(effection@4.1.0) @@ -222,8 +222,8 @@ importers: specifier: 0.1.1 version: 0.1.1 '@effectionx/node': - specifier: 0.2.4 - version: 0.2.4(effection@4.1.0) + specifier: 0.2.5 + version: 0.2.5(effection@4.1.0) '@effectionx/process': specifier: 0.8.1 version: 0.8.1(effection@4.1.0) @@ -306,8 +306,8 @@ importers: specifier: 0.3.0 version: 0.3.0(effection@4.1.0) '@effectionx/node': - specifier: 0.2.4 - version: 0.2.4(effection@4.1.0) + specifier: 0.2.5 + version: 0.2.5(effection@4.1.0) '@effectionx/process': specifier: 0.8.1 version: 0.8.1(effection@4.1.0) @@ -321,8 +321,8 @@ importers: specifier: 1.3.0 version: 1.3.0(zod@4.3.6) '@effectionx/node': - specifier: 0.2.4 - version: 0.2.4(effection@4.1.0) + specifier: 0.2.5 + version: 0.2.5(effection@4.1.0) '@effectionx/scope-eval': specifier: 0.1.3 version: 0.1.3(effection@4.1.0) @@ -408,8 +408,8 @@ importers: packages/web: dependencies: '@effectionx/node': - specifier: 0.2.4 - version: 0.2.4(effection@4.1.0) + specifier: 0.2.5 + version: 0.2.5(effection@4.1.0) '@executablemd/core': specifier: workspace:* version: link:../core @@ -566,6 +566,11 @@ packages: peerDependencies: effection: ^3 || ^4 + '@effectionx/node@0.2.5': + resolution: {integrity: sha512-hL8mROda8Lx375MVS+Ubu86+yMht/I0wOZG5VR6Pel0XUA5ReObQDYvNS6ocW0cNFKnEmvlVLWGWbcjJ+VkVhA==} + peerDependencies: + effection: ^3 || ^4 + '@effectionx/process@0.8.1': resolution: {integrity: sha512-xyXlFja0Ill80lQ3IYfksXtJkqVmWuUOogRn/qlHWCAGlZj+MGGF8gOFbyzk/3Kx4pj14riVGgF/cyT5XCzqDw==} peerDependencies: @@ -2733,6 +2738,10 @@ snapshots: dependencies: effection: 4.1.0 + '@effectionx/node@0.2.5(effection@4.1.0)': + dependencies: + effection: 4.1.0 + '@effectionx/process@0.8.1(effection@4.1.0)': dependencies: '@effectionx/context-api': 0.6.0(effection@4.1.0) diff --git a/scripts/build-npm.ts b/scripts/build-npm.ts index dfa434084..a018abe2c 100644 --- a/scripts/build-npm.ts +++ b/scripts/build-npm.ts @@ -22,6 +22,7 @@ */ import { ensure, exit, main, scoped, until } from "effection"; +import { validateDocumentation } from "./validate-documentation.ts"; import type { Operation } from "effection"; import { build } from "jsr:@deno/dnt@0.42.3"; import { @@ -50,12 +51,29 @@ import { z } from "npm:zod@^4"; * a document part of the product. */ function* packagedDocuments(pkgDir: URL): Operation { + const shipped: string[] = []; + // Component documentation lives beside the registration boundary it + // documents rather than in `src/documents/`, because that is where the + // components are and moving it would separate the two things that have to + // stay in step. One entry per boundary, named rather than swept for: `src/` + // also holds test documents and scenario fixtures, so being listed here is + // what declares an asset shipped. + for (const relative of [ + "src/components/components.md", + "src/agent/components.md", + "src/components.md", + "src/composition/components.md", + ]) { + if (yield* exists(new URL(relative, pkgDir))) { + shipped.push(relative); + } + } const documents = new URL("src/documents/", pkgDir); if (!(yield* exists(documents))) { - return []; + return shipped; } const names = yield* until(readdir(fromFileUrl(documents), { recursive: true })); - return names.map((name) => `src/documents/${name.split(sep).join("/")}`); + return [...shipped, ...names.map((name) => `src/documents/${name.split(sep).join("/")}`)]; } const ExportsSchema = z.union([z.string(), z.record(z.string(), z.string())]); @@ -113,6 +131,14 @@ await main(function* (args) { return; } + // Before anything is emitted. Copying the documentation assets is not the + // same as validating them: a package built from a set that has drifted from + // the components it documents would install cleanly and refuse the first time + // somebody asked it for documentation. The same assembly the run profile uses + // runs here, so a missing, unknown or duplicated section fails the build for + // exactly the reason it would fail a run. + yield* validateDocumentation(); + const repoRoot = new URL("../", import.meta.url); const rootDeno = RootDenoSchema.parse( diff --git a/scripts/oxlint-plugin.js b/scripts/oxlint-plugin.js index c034db388..1ee11cd75 100644 --- a/scripts/oxlint-plugin.js +++ b/scripts/oxlint-plugin.js @@ -5,6 +5,7 @@ import { noSyncFilesystem } from "./oxlint-rules/no-sync-filesystem.js"; import { noYieldInFinally } from "./oxlint-rules/no-yield-in-finally.js"; import { preferEffectionOperation } from "./oxlint-rules/prefer-effection-operation.js"; import { preferEffectionResult } from "./oxlint-rules/prefer-effection-result.js"; +import { requireScopeBoundEventRegistration } from "./oxlint-rules/require-scope-bound-event-registration.js"; export default { meta: { name: "executablemd" }, @@ -16,5 +17,6 @@ export default { "no-yield-in-finally": noYieldInFinally, "prefer-effection-operation": preferEffectionOperation, "prefer-effection-result": preferEffectionResult, + "require-scope-bound-event-registration": requireScopeBoundEventRegistration, }, }; diff --git a/scripts/oxlint-rules/require-scope-bound-event-registration.js b/scripts/oxlint-rules/require-scope-bound-event-registration.js new file mode 100644 index 000000000..1c16909b2 --- /dev/null +++ b/scripts/oxlint-rules/require-scope-bound-event-registration.js @@ -0,0 +1,1279 @@ +/** + * `local/require-scope-bound-event-registration` — a listener outlives nothing. + * + * An event listener an Effection operation installs is state that operation + * owns, and the architecture's ownership contract says owned state is released + * when the owner is. Nothing about an event source enforces that. A handler + * attached to a socket, a child process or a DOM target stays attached after + * the operation that attached it has returned, failed, been halted, or lost a + * `race()`; the source keeps calling it, and it keeps writing to state the + * abandoned owner left behind. + * + * Cleanup that runs when the event fires is not cleanup. A cancelled wait is + * precisely the case where the event never arrives, so `emitter.once()` and + * `addEventListener(..., { once: true })` are reported wherever an Effection + * owner reaches them. A one-event wait uses the scope-bound helper instead: + * + * const [event] = yield* once(socket, "close"); + * + * A subscription that outlives one event binds a stable handler and removes + * that same handler, from that same receiver and event, in the owner's own + * teardown. Removal is synchronous, so it belongs in one of four places: the + * `finally` of a `try` around the subscription, an `ensure()` established + * before the owner can suspend, a synchronous `finally` inside an `ensure()` + * whose wait needs the listener still attached, or the cleanup an `action()` + * executor returns. Node removes with `.off()`; the DOM removes with + * `.removeEventListener()`, matching capture mode included. + * + * Recognition is by binding rather than by spelling, because `on` and `once` + * are ordinary method names. What the rule resolves is: + * + * - values and types imported from `node:events`, `node:stream`, + * `node:child_process`, `node:net`, `node:http`, `node:https` and + * `node:process`; + * - a binding holding what one of those constructors or factories returned, + * and the `stdin`, `stdout`, `stderr` and `socket` members of such a binding; + * - a parameter or property annotated with one of those imported types; + * - a class extending a recognized emitter, and `this` inside it; + * - a local interface that declares a paired `on`/`off` or + * `addEventListener`/`removeEventListener` surface; + * - `process` and its streams while they still name the host global; and + * - DOM targets reached through `XMLHttpRequest`, `Worker`, `EventTarget`, + * `AbortController`/`AbortSignal`, `document`, `window`, `self` and + * `globalThis`. + * + * An owner is a generator function, or the executor of `action()` imported + * from `effection`. A callback declared inside one inherits it, because it + * closes over that scope. Nothing else is an owner: a standalone fixture + * process, a browser-lifetime callback and a plain Promise helper install + * listeners for a lifetime Effection does not manage, and are accepted. A + * helper that ought to be scope-bound is refactored into an operation rather + * than left as a plain function to escape this rule. + * + * There is no autofix. Choosing the owner, the handler binding and the order + * of teardown changes what the program does when it is cancelled, which is the + * decision this rule exists to make visible. + */ + +/** Node modules whose event sources the rule recognizes, by specifier. */ +const NODE_MODULES = new Map([ + ["events", { values: ["EventEmitter"], types: ["EventEmitter"] }], + [ + "stream", + { + values: ["Readable", "Writable", "Duplex", "Transform", "PassThrough", "Stream"], + types: ["Readable", "Writable", "Duplex", "Transform", "PassThrough", "Stream"], + }, + ], + [ + "child_process", + { + values: ["spawn", "exec", "execFile", "fork", "ChildProcess"], + types: ["ChildProcess", "ChildProcessWithoutNullStreams", "ChildProcessByStdio"], + }, + ], + [ + "net", + { + values: ["connect", "createConnection", "createServer", "Socket", "Server"], + types: ["Socket", "Server"], + }, + ], + [ + "http", + { + values: ["createServer", "request", "get", "Server", "ClientRequest"], + types: ["Server", "IncomingMessage", "ServerResponse", "ClientRequest"], + }, + ], + [ + "https", + { + values: ["createServer", "request", "get", "Server"], + types: ["Server", "IncomingMessage", "ServerResponse"], + }, + ], + ["process", { values: [], types: [] }], + ["worker_threads", { values: ["Worker", "MessagePort"], types: ["Worker", "MessagePort"] }], +]); + +/** DOM constructors whose instances are event targets. */ +const DOM_CONSTRUCTORS = new Set([ + "AbortController", + "EventSource", + "EventTarget", + "MessageChannel", + "WebSocket", + "Worker", + "XMLHttpRequest", +]); + +/** DOM globals that are event targets in their own right. */ +const DOM_GLOBALS = new Set(["document", "globalThis", "self", "window"]); + +/** DOM types a parameter or property may be annotated with. */ +const DOM_TYPES = new Set([ + "AbortSignal", + "EventSource", + "EventTarget", + "MessagePort", + "WebSocket", + "Worker", + "XMLHttpRequest", +]); + +/** Members of a recognized source that are themselves event sources. */ +const SOURCE_MEMBERS = new Set(["stdin", "stdout", "stderr", "socket", "signal", "port1", "port2"]); + +const REGISTER = new Set(["on", "once", "addEventListener"]); +const REMOVE = new Set(["off", "removeEventListener", "removeListener"]); + +/** Nodes that stop a search for suspensions belonging to one owner. */ +const FUNCTIONS = new Set(["FunctionDeclaration", "FunctionExpression", "ArrowFunctionExpression"]); + +/** The module a specifier names, with any `node:` prefix removed. */ +function moduleOf(source) { + return typeof source === "string" ? source.replace(/^node:/u, "") : ""; +} + +/** The property a member expression reads, whether named or spelled out. */ +function memberName(node) { + if (node.type !== "MemberExpression") { + return null; + } + + if (!node.computed) { + return node.property.type === "Identifier" ? node.property.name : null; + } + + return typeof node.property.value === "string" ? node.property.value : null; +} + +/** The names a type annotation refers to, for the simple forms the rule reads. */ +function typeNames(annotation) { + const reference = annotation?.typeAnnotation ?? annotation; + + if (reference?.type === "TSUnionType") { + return reference.types.flatMap((member) => typeNames(member)); + } + + const name = typeName(reference); + + return name === null ? [] : [name]; +} + +/** The name a single type reference refers to. */ +function typeName(annotation) { + const reference = annotation?.typeAnnotation ?? annotation; + + if (!reference) { + return null; + } + + if (reference.type === "TSTypeReference") { + const name = reference.typeName; + if (name?.type === "Identifier") { + return name.name; + } + if (name?.type === "TSQualifiedName" && name.right?.type === "Identifier") { + return name.right.name; + } + } + + return null; +} + +function isStringLiteral(node) { + return node?.type === "Literal" && typeof node.value === "string"; +} + +/** The name a property declares, whether written plainly or as a string. */ +function propertyName(property) { + if (property.type !== "Property") { + return null; + } + + if (!property.computed && property.key?.type === "Identifier") { + return property.key.name; + } + + return isStringLiteral(property.key) ? property.key.value : null; +} + +/** + * How a listener options object spells a boolean member, when it says so + * statically. `{ "once": true }` and `{ once: ONCE }` for a `const ONCE = true` + * are the same registration as `{ once: true }`. + */ +function staticFlag(node, name, resolve) { + if (node?.type !== "ObjectExpression") { + return undefined; + } + + const property = node.properties.find((entry) => propertyName(entry) === name); + + if (property === undefined) { + return undefined; + } + + const value = property.value; + + if (value?.type === "Literal" && typeof value.value === "boolean") { + return value.value; + } + + return resolve(value); +} + +/** + * How a listener argument list spells its capture mode. `{ capture: true }` + * and a bare `true` are the same registration, so both normalize to the same + * value and a removal has to agree with it. + */ +function captureOf(node, resolve) { + if (node === undefined) { + return false; + } + + if (node.type === "Literal" && typeof node.value === "boolean") { + return node.value; + } + + return staticFlag(node, "capture", resolve) === true; +} + +export const requireScopeBoundEventRegistration = { + meta: { + type: "problem", + messages: { + rawOnce: + '{{receiver}}.once("{{event}}") removes its listener only when the event arrives, which a cancelled wait never gets. Wait with the scope-bound once() from @effectionx/node/events.', + onceOption: + 'addEventListener("{{event}}", …, { once: true }) removes its listener only when the event arrives, which a cancelled wait never gets. Wait with the scope-bound once() from @effectionx/node/events.', + anonymous: + 'This listener on "{{event}}" has no name, so nothing can remove it. Bind the handler and remove that binding in the owner\'s teardown.', + dynamicEvent: + "This listener's event name is computed, so the pairing with its removal cannot be established. Register each event under a literal name.", + missing: + 'Nothing removes {{handler}} from {{receiver}} "{{event}}" when this operation ends. Remove it in a finally around the subscription, an ensure() established before the owner suspends, a synchronous finally inside that ensure(), or the cleanup returned by action().', + mismatched: + 'The cleanup for {{receiver}} "{{event}}" does not name the same receiver, event, handler and capture mode this registration used, so the handler stays attached. Remove {{handler}} from {{receiver}} "{{event}}".', + selfRemoving: + '{{handler}} removes itself only when "{{event}}" arrives, which a cancelled owner never gets. Remove it in the owner\'s teardown as well.', + late: "This cleanup is registered after the owner can already suspend, so a failure in between leaves {{handler}} attached to {{receiver}}. Establish it in the same synchronous prefix as the subscription.", + unarmed: + "This ensure() is yielded after {{handler}} was attached, and entering it is itself a suspension: an owner halted there unwinds with no cleanup registered at all, leaving {{handler}} on {{receiver}}. Establish the ensure() before the subscription, or put both inside a try whose finally removes it.", + suspended: + "Teardown suspends before it removes {{handler}} from {{receiver}}, so a halt during that wait leaves the handler attached. Remove it in a synchronous finally around the wait.", + removeListener: + "removeListener() is not the removal this rule pairs with. Use .off() so the subscription and its cleanup read as one pair.", + }, + }, + + create(context) { + const source = context.sourceCode; + const text = source.text; + + /** Imported bindings, by local name, that name a recognized module value. */ + const values = new Map(); + /** Imported bindings, by local name, that name a recognized module type. */ + const types = new Map(); + /** Local interfaces whose declared surface is a paired event source. */ + const structural = new Set(); + /** Classes extending a recognized emitter, by declaration range. */ + const emitterClasses = []; + const registrations = []; + const removals = []; + /** `collection.set(receiver, handler)` calls, which pair a source with its handler. */ + const records = []; + const reports = []; + + function variableOf(node) { + for (let scope = source.getScope(node); scope; scope = scope.upper) { + const found = scope.variables.find((variable) => variable.name === node.name); + + if (found) { + return found; + } + } + + return null; + } + + /** Whether an identifier still names what the host put there. */ + function isGlobal(node) { + if (node.type !== "Identifier") { + return false; + } + + const variable = variableOf(node); + return variable === null || variable.defs.length === 0; + } + + /** The declaration an identifier resolves to, or null. */ + function definitionOf(node) { + if (node.type !== "Identifier") { + return null; + } + + const variable = variableOf(node); + return variable?.defs?.[0] ?? null; + } + + /** Whether an identifier resolves to `name` imported from `module`. */ + function importsFrom(node, module, name) { + const definition = definitionOf(node); + + if (definition?.type !== "ImportBinding") { + return false; + } + + const declaration = definition.parent; + + if (declaration.importKind === "type" || definition.node.importKind === "type") { + return false; + } + + return ( + moduleOf(declaration.source.value) === module && + definition.node.type === "ImportSpecifier" && + definition.node.imported.name === name + ); + } + + function isProcessGlobal(node) { + if (isGlobal(node) && node.name === "process") { + return true; + } + + const definition = definitionOf(node); + + return ( + definition?.type === "ImportBinding" && + moduleOf(definition.parent.source.value) === "process" && + definition.node.type === "ImportDefaultSpecifier" + ); + } + + /** Whether a callee names a constructor or factory of a Node event source. */ + function isNodeFactory(node) { + if (node.type === "MemberExpression") { + const name = memberName(node); + const object = node.object; + + if (object.type !== "Identifier" || name === null) { + return false; + } + + const definition = definitionOf(object); + + if (definition?.type !== "ImportBinding") { + return false; + } + + const module = moduleOf(definition.parent.source.value); + const recognized = NODE_MODULES.get(module); + + return Boolean(recognized?.values.includes(name)); + } + + if (node.type !== "Identifier") { + return false; + } + + const definition = definitionOf(node); + + if (definition?.type !== "ImportBinding") { + return false; + } + + const module = moduleOf(definition.parent.source.value); + const recognized = NODE_MODULES.get(module); + + return Boolean( + recognized && + definition.node.type === "ImportSpecifier" && + recognized.values.includes(definition.node.imported.name), + ); + } + + /** Whether a callee names a DOM constructor whose instances are targets. */ + function isDomConstructor(node) { + return node.type === "Identifier" && DOM_CONSTRUCTORS.has(node.name) && isGlobal(node); + } + + /** Whether an expression produces an event source. */ + function producesSource(node) { + if (!node) { + return false; + } + + if (node.type === "NewExpression") { + return ( + isNodeFactory(node.callee) || + isDomConstructor(node.callee) || + isEmitterSubclass(node.callee) + ); + } + + if (node.type === "CallExpression") { + return isNodeFactory(node.callee); + } + + if (node.type === "AwaitExpression" || node.type === "TSNonNullExpression") { + return producesSource(node.expression); + } + + // A binding chosen between two sources is still a source; one whose + // alternative is something else is not. + if (node.type === "ConditionalExpression") { + return producesSource(node.consequent) && producesSource(node.alternate); + } + + return isSource(node); + } + + /** Whether a callee names a class the file declares as an emitter. */ + function isEmitterSubclass(node) { + if (node.type !== "Identifier") { + return false; + } + + const definition = definitionOf(node); + const declaration = definition?.node; + + return ( + declaration?.type === "ClassDeclaration" && + emitterClasses.some((range) => range[0] === declaration.range[0]) + ); + } + + /** Whether an annotation names an event source, in any member of a union. */ + function isSourceType(annotation, at) { + return typeNames(annotation).some((name) => { + if (DOM_TYPES.has(name) || structural.has(name)) { + return true; + } + + const binding = types.get(name); + return binding !== undefined && binding <= at; + }); + } + + /** + * Whether an expression names an event source. Aliases and members are + * followed, so `child.stdout` and a binding that merely holds a socket both + * resolve to what produced them. + */ + function isSource(node) { + if (!node) { + return false; + } + + if (node.type === "MemberExpression") { + const name = memberName(node); + + if (name === null) { + return false; + } + + if (SOURCE_MEMBERS.has(name)) { + return isSource(node.object) || isProcessGlobal(node.object); + } + + if (isGlobal(node.object) && node.object.name === "globalThis") { + return DOM_GLOBALS.has(name) || name === "process"; + } + + return false; + } + + if (node.type === "ThisExpression") { + return inEmitterClass(node); + } + + if (node.type !== "Identifier") { + return false; + } + + if (isProcessGlobal(node)) { + return true; + } + + if (isGlobal(node) && DOM_GLOBALS.has(node.name)) { + return true; + } + + const definition = definitionOf(node); + + if (!definition) { + return false; + } + + if (definition.type === "Parameter") { + const parameter = definition.name ?? definition.node; + + if (isSourceType(parameter?.typeAnnotation, node.range[0])) { + return true; + } + + // A default value is the only thing a parameter says about itself when + // it carries no annotation. + const pattern = parameter?.parent; + + return ( + pattern?.type === "AssignmentPattern" && + pattern.left === parameter && + producesSource(pattern.right) + ); + } + + if (definition.type === "Variable" || definition.type === "VariableDeclarator") { + const declarator = definition.node; + + if (isSourceType(declarator.id?.typeAnnotation, node.range[0])) { + return true; + } + + // Only a binding that holds the source itself resolves: reassignment + // and conditional initialization say nothing about what it now holds. + return producesSource(declarator.init); + } + + return false; + } + + /** Whether a node sits inside a class that extends a recognized emitter. */ + function inEmitterClass(node) { + return emitterClasses.some((range) => range[0] <= node.range[0] && node.range[1] <= range[1]); + } + + /** The normalized text of a receiver expression, used as its identity. */ + function receiverKey(node) { + return text.slice(node.range[0], node.range[1]).replace(/\s+/gu, ""); + } + + /** A `const`-bound boolean, when the binding says so and nothing reassigns it. */ + function constantBoolean(node) { + if (node?.type !== "Identifier") { + return undefined; + } + + const variable = variableOf(node); + const definition = variable?.defs?.[0]; + + if (definition?.type !== "Variable" && definition?.type !== "VariableDeclarator") { + return undefined; + } + + const initializer = definition.node.init; + + return definition.parent?.kind === "const" && + initializer?.type === "Literal" && + typeof initializer.value === "boolean" + ? initializer.value + : undefined; + } + + /** Constructs that can decide not to run what is nested inside them. */ + const BRANCHES = new Set([ + "IfStatement", + "SwitchStatement", + "SwitchCase", + "ForStatement", + "ForOfStatement", + "ForInStatement", + "WhileStatement", + "DoWhileStatement", + "CatchClause", + "ConditionalExpression", + "LogicalExpression", + ]); + + /** + * Whether cleanup at `to` is reached whenever the registration at `from` + * has happened, with nothing suspending in between. + * + * Cleanup nested in a branch the registration is not nested in may simply + * not run, which is a listener left attached rather than a pair. + */ + function unconditionallyReached(from, to) { + const path = []; + + for (let parent = to.parent; parent; parent = parent.parent) { + if (contains(parent.range, from)) { + break; + } + path.push(parent); + } + + if (path.some((node) => BRANCHES.has(node.type))) { + return false; + } + + return adjacent(from, to); + } + + /** The identifier a member chain is rooted in, or null. */ + function rootIdentifier(node) { + let current = node; + + while (current?.type === "MemberExpression") { + current = current.object; + } + + return current?.type === "Identifier" ? current : null; + } + + /** + * Whether two expressions name the same value, rather than merely being + * spelled the same. Two connections both called `socket` are two sockets, + * and removing a handler from one says nothing about the other. + */ + function sameValue(left, right) { + if (receiverKey(left) !== receiverKey(right)) { + return false; + } + + const first = rootIdentifier(left); + const second = rootIdentifier(right); + + if (!first || !second) { + return true; + } + + return variableOf(first) === variableOf(second); + } + + /** + * The nearest enclosing Effection owner. + * + * Nearest, and nothing wider. A nested generator or `action()` executor is + * a scope of its own and ends before the one around it, so cleanup an + * outer owner performs runs too late to be this registration's pair — + * whether it names the handler directly or walks a collection the handler + * was recorded in. A plain callback declared inside an owner is not a + * scope, so it inherits that owner and may register on its behalf. + */ + function ownerOf(node) { + for (let parent = node.parent; parent; parent = parent.parent) { + if ( + (parent.type === "FunctionDeclaration" || parent.type === "FunctionExpression") && + parent.generator + ) { + return { node: parent, kind: "generator" }; + } + + if (FUNCTIONS.has(parent.type) && isActionExecutor(parent)) { + return { node: parent, kind: "action" }; + } + } + + return null; + } + + /** Whether a function is the executor `action()` was given. */ + function isActionExecutor(node) { + const call = node.parent; + + return ( + call?.type === "CallExpression" && + call.arguments[0] === node && + importsFrom(call.callee, "effection", "action") + ); + } + + function contains(range, node) { + return range[0] <= node.range[0] && node.range[1] <= range[1]; + } + + /** Suspensions performed between two positions of one block. */ + function suspendsBetween(block, from, to) { + let found = false; + + walk(block, (node) => { + if (node.type === "YieldExpression" && node.range[0] >= from && node.range[1] <= to) { + found = true; + } + }); + + return found; + } + + /** The statement of its own block that contains `node`. */ + function statementOf(node) { + let current = node; + + for (let parent = node.parent; parent; current = parent, parent = parent.parent) { + if (parent.type === "BlockStatement" || parent.type === "Program") { + return current; + } + } + + return null; + } + + /** The innermost block that holds both nodes. */ + function commonBlock(first, second) { + for (let parent = first.parent; parent; parent = parent.parent) { + if ( + (parent.type === "BlockStatement" || parent.type === "Program") && + contains(parent.range, second) + ) { + return parent; + } + } + + return null; + } + + /** The statement of `block` that holds `node`. */ + function statementIn(block, node) { + let current = node; + + for (let parent = node.parent; parent; current = parent, parent = parent.parent) { + if (parent === block) { + return current; + } + } + + return null; + } + + /** + * Whether two nodes are reached in one uninterrupted synchronous run: the + * statements holding them, in the innermost block holding both, with + * nothing that suspends in between. + */ + function adjacent(first, second) { + const block = commonBlock(first, second); + + if (!block) { + return false; + } + + const earlier = statementIn(block, first); + const later = statementIn(block, second); + + if (!earlier || !later || earlier === later) { + return true; + } + + const [start, end] = earlier.range[0] < later.range[0] ? [earlier, later] : [later, earlier]; + + return !suspendsBetween(block, start.range[1], end.range[0]); + } + + /** Visit every node under `root` that the same function evaluates. */ + function walk(root, visit) { + if (!root || typeof root.type !== "string") { + return; + } + + visit(root); + + for (const key of Object.keys(root)) { + if (key === "parent" || key === "loc" || key === "range") { + continue; + } + + const value = root[key]; + + for (const child of Array.isArray(value) ? value : [value]) { + if (child && typeof child === "object" && typeof child.type === "string") { + if (FUNCTIONS.has(child.type)) { + continue; + } + walk(child, visit); + } + } + } + } + + return { + ImportDeclaration(node) { + const module = moduleOf(node.source.value); + const recognized = NODE_MODULES.get(module); + + if (!recognized) { + return; + } + + for (const specifier of node.specifiers) { + if (specifier.type !== "ImportSpecifier") { + continue; + } + + const imported = specifier.imported.name; + + if (recognized.values.includes(imported)) { + values.set(specifier.local.name, node.range[0]); + } + + if (recognized.types.includes(imported)) { + types.set(specifier.local.name, node.range[0]); + } + } + }, + + TSInterfaceDeclaration(node) { + const members = node.body.body + .filter((member) => member.type === "TSMethodSignature" && !member.computed) + .map((member) => member.key?.name); + + if ( + (members.includes("on") && members.includes("off")) || + (members.includes("addEventListener") && members.includes("removeEventListener")) + ) { + structural.add(node.id.name); + } + }, + + ClassDeclaration(node) { + const parent = node.superClass; + + if (parent && (isNodeFactory(parent) || isDomConstructor(parent))) { + emitterClasses.push(node.range); + } + }, + + CallExpression(node) { + const callee = node.callee; + + if (callee.type !== "MemberExpression") { + return; + } + + const method = memberName(callee); + + if (method === null) { + return; + } + + if (REMOVE.has(method)) { + removals.push({ node, method, receiver: callee.object }); + return; + } + + if (method === "set" && node.arguments.length === 2) { + records.push({ node, collection: callee.object }); + return; + } + + if (!REGISTER.has(method)) { + return; + } + + registrations.push({ node, method, receiver: callee.object }); + }, + + "Program:exit"() { + for (const entry of registrations) { + classify(entry); + } + + reports.sort((left, right) => left.node.range[0] - right.node.range[0]); + + for (const entry of reports) { + context.report(entry); + } + }, + }; + + /** Report what one registration is missing, if anything. */ + function classify(entry) { + const { node, method, receiver } = entry; + + if (!isSource(receiver)) { + return; + } + + const owner = ownerOf(node); + + if (!owner) { + return; + } + + const [event, handler, options] = node.arguments; + const receiverText = receiverKey(receiver); + + if (!isStringLiteral(event)) { + reports.push({ node, messageId: "dynamicEvent", data: {} }); + return; + } + + if (method === "once") { + reports.push({ + node, + messageId: "rawOnce", + data: { receiver: receiverText, event: event.value }, + }); + return; + } + + if (method === "addEventListener" && staticFlag(options, "once", constantBoolean) === true) { + reports.push({ node, messageId: "onceOption", data: { event: event.value } }); + return; + } + + if (handler?.type !== "Identifier") { + reports.push({ node, messageId: "anonymous", data: { event: event.value } }); + return; + } + + const capture = method === "addEventListener" ? captureOf(options, constantBoolean) : false; + const remover = method === "addEventListener" ? "removeEventListener" : "off"; + + // Only the owner's own cleanup counts. Two operations in one file often + // name their socket `socket`, and a removal in one of them says nothing + // about what the other leaves attached. + const owned = removals.filter((removal) => contains(owner.node.range, removal.node)); + + const pairs = owned.filter( + (removal) => + sameValue(removal.receiver, receiver) && + isStringLiteral(removal.node.arguments[0]) && + removal.node.arguments[0].value === event.value && + removal.node.arguments[1]?.type === "Identifier" && + sameValue(removal.node.arguments[1], handler) && + (method !== "addEventListener" || + captureOf(removal.node.arguments[2], constantBoolean) === capture), + ); + + const data = { receiver: receiverText, event: event.value, handler: handler.name }; + const candidates = pairs.filter((removal) => removal.method === remover); + + if (candidates.length === 0) { + if (pairs.length > 0) { + reports.push({ node, messageId: "removeListener", data }); + return; + } + + if (detachedInBulk(entry, receiverText, handler, event.value, owner)) { + return; + } + + const near = owned.some( + (removal) => + receiverKey(removal.receiver) === receiverText || + (removal.node.arguments[1]?.type === "Identifier" && + removal.node.arguments[1].name === handler.name), + ); + + reports.push({ node, messageId: near ? "mismatched" : "missing", data }); + return; + } + + const verdicts = candidates.map((removal) => placement(removal, node, owner, handler)); + + if (verdicts.includes("accepted")) { + return; + } + + const verdict = verdicts.find((value) => value !== "unrelated") ?? "missing"; + + reports.push({ + node, + messageId: verdict === "unrelated" ? "missing" : verdict, + data, + }); + } + + /** + * Whether the owner remembers this pair in a collection and detaches that + * whole collection in its teardown. + * + * One handler per accepted connection cannot be named individually, so the + * collection is the pairing: the registration records the receiver and its + * handler together, and teardown walks the same collection removing each + * handler from the receiver it was recorded against. + */ + function detachedInBulk(entry, receiverText, handler, event, owner) { + const collections = records + .filter( + (record) => + contains(owner.node.range, record.node) && + sameValue(record.node.arguments[0], entry.receiver) && + record.node.arguments[1]?.type === "Identifier" && + sameValue(record.node.arguments[1], handler) && + // Recorded in the same synchronous run as the subscription: a pair + // written down after the owner could suspend is one the teardown + // in between would not have found. + unconditionallyReached(entry.node, record.node), + ) + .map((record) => record.collection); + + if (collections.length === 0) { + return false; + } + + return removals.some((removal) => { + if ( + removal.method !== + (entry.method === "addEventListener" ? "removeEventListener" : "off") || + !contains(owner.node.range, removal.node) || + !isStringLiteral(removal.node.arguments[0]) || + removal.node.arguments[0].value !== event + ) { + return false; + } + + const loop = iterationOf(removal.node); + + if (!loop || !collections.some((collection) => sameValue(collection, loop.right))) { + return false; + } + + const pattern = loop.left.declarations?.[0]?.id ?? loop.left; + + return ( + pattern?.type === "ArrayPattern" && + pattern.elements[0]?.name === receiverKey(removal.receiver) && + pattern.elements[1]?.name === removal.node.arguments[1]?.name && + releasedBeforeSuspending(removal.node, owner) + ); + }); + } + + /** The `for…of` whose body contains this node, within one function. */ + function iterationOf(node) { + for (let parent = node.parent; parent; parent = parent.parent) { + if (parent.type === "ForOfStatement") { + return parent; + } + + if (FUNCTIONS.has(parent.type)) { + return null; + } + } + + return null; + } + + /** Whether a bulk detach runs in teardown, before that teardown suspends. */ + function releasedBeforeSuspending(removal, owner) { + const cleanup = ensureCleanup(removal, owner); + + if (cleanup) { + return !suspendsBefore(cleanup, removal, protectedBy(removal)?.range); + } + + return protectedBy(removal) !== null; + } + + /** + * Where a matching removal sits relative to the registration, and whether + * that place runs when the owner ends. + */ + function placement(removal, registration, owner, handler) { + // Only a handler this file declares can remove itself. A handler that + // arrived as a parameter is somebody else's function, and its + // declaration is the enclosing signature rather than a body. + const definition = definitionOf(handler); + const handlerNode = + definition?.type === "Variable" || definition?.type === "VariableDeclarator" + ? definition.node.init + : definition?.type === "FunctionName" + ? definition.node + : null; + + if ( + handlerNode && + FUNCTIONS.has(handlerNode.type) && + contains(handlerNode.range, removal.node) + ) { + return "selfRemoving"; + } + + for (let parent = removal.node.parent; parent; parent = parent.parent) { + if ( + parent.type === "TryStatement" && + parent.finalizer && + contains(parent.finalizer.range, removal.node) + ) { + // The `finally` of a `try` that covers the subscription runs on every + // way out of it, which is the whole guarantee this rule is after. A + // subscription made just before that `try` is covered too, as long + // as nothing between the two can suspend. + if (contains(parent.block.range, registration)) { + return "accepted"; + } + + if (guards(parent, registration)) { + return "accepted"; + } + + // Otherwise the `finally` still protects whatever its `try` suspends + // on, which is what an ensure() with a listener-dependent wait needs. + const cleanup = ensureCleanup(removal.node, owner); + + if (cleanup) { + return suspendsBefore(cleanup, removal.node, parent.range) + ? "suspended" + : armed(cleanup, registration); + } + + if (statementOf(parent)?.parent === statementOf(registration)?.parent) { + return "late"; + } + } + + if (parent === owner.node) { + break; + } + } + + const cleanup = ensureCleanup(removal.node, owner); + + if (cleanup) { + if (suspendsBefore(cleanup, removal.node, protectedBy(removal.node)?.range)) { + return "suspended"; + } + + return armed(cleanup, registration); + } + + if (owner.kind === "action" && returnedCleanup(removal.node, owner.node)) { + return "accepted"; + } + + return "unrelated"; + } + + /** + * Whether a `try` statement covers a subscription made just before it, + * with no suspension in between to fail through. + */ + function guards(statement, registration) { + const subscription = statementOf(registration); + const guard = statementOf(statement); + + if (!subscription || !guard || subscription === guard) { + return false; + } + + // A lexical finalizer only covers what runs in its own frame. A `try` + // inside an `ensure()` cleanup is a different function, reached only once + // that `ensure()` has been established — which is the very thing the + // ordering rule in `armed()` is about. + if (functionOf(statement) !== functionOf(registration)) { + return false; + } + + return ( + subscription.range[1] <= guard.range[0] && unconditionallyReached(registration, statement) + ); + } + + /** The nearest function a node runs in, or null at module level. */ + function functionOf(node) { + for (let parent = node.parent; parent; parent = parent.parent) { + if (FUNCTIONS.has(parent.type)) { + return parent; + } + } + + return null; + } + + /** The `ensure()` cleanup function a node sits inside, within this owner. */ + function ensureCleanup(node, owner) { + for (let parent = node.parent; parent; parent = parent.parent) { + if ( + FUNCTIONS.has(parent.type) && + parent.parent?.type === "CallExpression" && + parent.parent.arguments[0] === parent && + importsFrom(parent.parent.callee, "effection", "ensure") + ) { + return parent; + } + + if (parent === owner.node) { + return null; + } + } + + return null; + } + + /** + * Whether the cleanup suspends before it reaches the removal. A suspension + * inside a `try` whose `finally` performs that removal does not count: it + * is the wait the listener is being kept alive for. + */ + function suspendsBefore(cleanup, removal, protectedRange) { + let found = false; + + walk(cleanup.body, (node) => { + if (node.type !== "YieldExpression" || node.range[1] > removal.range[0]) { + return; + } + + if (protectedRange && contains(protectedRange, node)) { + return; + } + + found = true; + }); + + return found; + } + + /** The `try` whose `finally` performs this removal, if there is one. */ + function protectedBy(removal) { + for (let parent = removal.parent; parent; parent = parent.parent) { + if ( + parent.type === "TryStatement" && + parent.finalizer && + contains(parent.finalizer.range, removal) + ) { + return parent; + } + } + + return null; + } + + /** + * Whether the `ensure()` holding this cleanup was established in the same + * uninterrupted synchronous run as the subscription. + */ + function armed(cleanup, registration) { + const call = cleanup.parent; + + if (!unconditionallyReached(registration, call)) { + return "late"; + } + + // Established means finished. `yield* ensure(...)` is itself a + // suspension, so an owner halted while it is registering unwinds with no + // cleanup on it at all — measured: the listener stays attached and the + // cleanup never runs. Only an `ensure()` that completed before the + // subscription existed has covered it. + return call.range[1] <= registration.range[0] ? "accepted" : "unarmed"; + } + + /** Whether a node sits in a function the action executor returns. */ + function returnedCleanup(node, executor) { + for (let parent = node.parent; parent; parent = parent.parent) { + if (FUNCTIONS.has(parent.type)) { + const owner = parent.parent; + + if (owner?.type === "ReturnStatement" && contains(executor.range, owner)) { + return true; + } + + if (executor.body === parent.parent || parent === executor) { + return false; + } + } + + if (parent === executor) { + return executor.body === node || contains(executor.range, node); + } + } + + return false; + } + }, +}; diff --git a/scripts/runtime-test-exclusions.ts b/scripts/runtime-test-exclusions.ts index 227651421..c254f2067 100644 --- a/scripts/runtime-test-exclusions.ts +++ b/scripts/runtime-test-exclusions.ts @@ -47,6 +47,18 @@ const DENO_ONLY_TOOLING: RuntimeExclusion[] = [ "builds the npm package with dnt, which only runs under Deno; the test calls Deno.readTextFileSync", issue: DERIVED_SCOPE, }, + { + path: "scripts/tests/documentation-validation.test.ts", + reason: + "runs the repository's build gate as a subprocess — `deno run scripts/validate-documentation.ts` — and the Node and Bun shards have no `deno` on PATH, so every case fails with `Executable not found in $PATH`. The subject is the gate, which is a Deno entrypoint; a local Bun run passes only because a developer machine happens to have Deno installed", + issue: DERIVED_SCOPE, + }, + { + path: "scripts/tests/jsr-consumer-documentation.test.ts", + reason: + "its subject is the JSR distribution: it runs `deno publish --dry-run` to learn what the publish filter selects, stages exactly those files, and runs a consumer under `deno run` against them. Every step is Deno's own packaging, so there is nothing here a Node or Bun run would be exercising", + issue: DERIVED_SCOPE, + }, { path: "scripts/tests/adapter-distribution.test.ts", reason: @@ -155,6 +167,12 @@ const DENO_ONLY_TOOLING: RuntimeExclusion[] = [ "exercises Deno-private node:sqlite transaction identities and real SQLite savepoint failure behavior; node:sqlite remains behind --experimental-sqlite on Node 22", issue: "https://github.com/taras/executable.md/issues/365", }, + { + path: "packages/core/tests/syntax-loaded-copy.test.ts", + reason: + "builds the second copy of the protected-component module with `deno bundle`, which is Deno's; the witness comparison and the private body table it proves are runtime-neutral and are also covered by syntax-component.test.ts under all three", + issue: DERIVED_SCOPE, + }, { path: "packages/core/tests/loaded-copy-files.test.ts", reason: @@ -413,6 +431,12 @@ const DENO_ONLY_TOOLING: RuntimeExclusion[] = [ "runs the representative observation-loop document against a real node:sqlite WorkflowRun database with the run's Workspace attached, substituting only the agent process; the store and the DOFS Workspace under it are both the Deno adapter's", issue: "https://github.com/taras/executable.md/issues/367", }, + { + path: "packages/cli/tests/evaluate-workflow.test.ts", + reason: + "proves the workflow evaluation profile — its Workspace-bound ceiling, its advancing basis and its released `source` spelling — against a real node:sqlite WorkflowRun database with the Deno DOFS Workspace attached. The claim is that this host's profile differs from the ordinary one and nothing else does, so a substituted store would be a substitute for the subject; Bun has no node:sqlite and Node 22 keeps it behind --experimental-sqlite", + issue: "https://github.com/taras/executable.md/issues/713", + }, { path: "packages/workflow/tests/workflow-suspension.test.ts", reason: diff --git a/scripts/smoke-fetch.ts b/scripts/smoke-fetch.ts index b981babe1..4db587632 100644 --- a/scripts/smoke-fetch.ts +++ b/scripts/smoke-fetch.ts @@ -42,7 +42,23 @@ function useLoopback(): Operation { }); const listening = withResolvers(); - server.on("error", (error: Error) => listening.reject(error)); + // Removed with the resource rather than after the first error: a listening + // server outlives its bind, and a handler left behind would still be + // holding a rejected resolver when the next test binds its own. + // + // Established before the handler exists, because `yield* ensure(...)` is + // itself a suspension: an owner halted while it registers unwinds with no + // cleanup at all, so nothing may be attached until it has completed. + let onError: ((error: Error) => void) | undefined; + + yield* ensure(() => { + if (onError) { + server.off("error", onError); + } + }); + + onError = (error: Error) => listening.reject(error); + server.on("error", onError); server.listen(0, "127.0.0.1", () => listening.resolve()); yield* listening.operation; diff --git a/scripts/tests/cli-npm-bin.test.ts b/scripts/tests/cli-npm-bin.test.ts index af7caefc3..44b049ce8 100644 --- a/scripts/tests/cli-npm-bin.test.ts +++ b/scripts/tests/cli-npm-bin.test.ts @@ -230,6 +230,56 @@ describe("npm CLI package", { sanitizeOps: false, sanitizeResources: false }, () expect(entries.map((entry: { name?: string }) => entry?.name)).not.toContain(name); } + // `` is public in every build, and this one describes it exactly + // once, from the canonical origin, with the approved description. A package + // that lost the protected tier would either omit it or list it twice. + const syntax = entries.filter((entry: { name?: string }) => entry?.name === "Syntax"); + expect(syntax).toHaveLength(1); + expect(syntax[0].origin).toEqual({ kind: "protected", origin: "@executablemd/core" }); + expect(syntax[0].sourceKind).toBe("protected"); + expect(syntax[0].forms).toEqual(["self-closing"]); + expect(syntax[0].returnMode).toBe("text"); + expect(syntax[0].description).toBe( + "Inspect available components and control-flow constructs. `` lists the " + + 'symbols available here; `` renders selected documentation.', + ); + + // FE28. `` is the tier's second member and is public in every + // build for the same reason: a package that shipped the protected tier + // without it would leave a document able to write the name and no + // implementation able to answer it. This boundary is asserted explicitly + // because `--changed` cannot discover it: nothing in the emitted package + // shares a path with the core sources this name is defined in. + const evaluate = entries.filter((entry: { name?: string }) => entry?.name === "Evaluate"); + expect(evaluate).toHaveLength(1); + expect(evaluate[0].origin).toEqual({ kind: "protected", origin: "@executablemd/core" }); + expect(evaluate[0].sourceKind).toBe("protected"); + // Both spellings, because the two input forms are two ways of stating one + // argument. + expect(evaluate[0].forms).toEqual(["self-closing", "paired"]); + expect(evaluate[0].description).toBe( + 'Evaluate program text. `` runs it.', + ); + + // The documentation assets travel with the package, and the emitted binary + // resolves them from its own tree rather than from a checkout. `Prompt` + // lives in core's *agent* boundary rather than in its own `components.md`, + // so this exercises a second copied asset path: a build that copied only + // the first would still answer for core's own components and fail here. + const documented = yield* runEmittedBinIn(elsewhere, ["syntax", "Prompt"]); + expect(documented.code).toBe(0); + expect(documented.stdout).toContain("### ``"); + expect(documented.stdout).toContain("Sends a prompt and renders the reply"); + expect(documented.stdout).toContain("**Available in this evaluation:** yes"); + + // And it is the same answer the source tree gives, whole. + const fromSource = yield* exec(Deno.execPath(), { + arguments: ["run", "-A", path.join(ROOT, "packages/cli/src/deno.ts"), "syntax", "Prompt"], + cwd: elsewhere, + env: Deno.env.toObject(), + }).join(); + expect(documented.stdout).toBe(fromSource.stdout); + // The command's public grammar travels with those bytes. `--run` is gone, // and this directory has no agent to reach and no `DEFAULT_AGENT_NAME` that // resolves here — so a build that still accepted the switch would fail on diff --git a/scripts/tests/documentation-validation.test.ts b/scripts/tests/documentation-validation.test.ts new file mode 100644 index 000000000..4a411d140 --- /dev/null +++ b/scripts/tests/documentation-validation.test.ts @@ -0,0 +1,85 @@ +/** + * Tier SYN — the build gate for first-party documentation. + * + * `scripts/validate-documentation.ts` is what stands between a drifted + * documentation set and a published distribution. A gate nobody has watched + * fail is a gate nobody knows is connected, so each case here plants one class + * of drift in a real shipped asset, runs the real entrypoint, and puts the file + * back. + */ + +import { describe, it } from "@executablemd/test-support/bdd"; +import { expect } from "@executablemd/test-support/expect"; +import { ensure } from "effection"; +import type { Operation } from "effection"; +import { readTextFile, writeTextFile } from "@effectionx/fs"; +import { exec } from "@effectionx/process"; + +/** A shipped asset, restored however the case ends. */ +function* planted(relative: string, change: (text: string) => string): Operation { + const url = new URL(`../../${relative}`, import.meta.url); + const original = yield* readTextFile(url); + yield* ensure(() => writeTextFile(url, original)); + yield* writeTextFile(url, change(original)); +} + +/** The real gate, as the build runs it. */ +function* validate(): Operation<{ ok: boolean; stderr: string }> { + const run = yield* exec("deno", { + arguments: ["run", "--allow-all", "scripts/validate-documentation.ts"], + }).join(); + return { ok: run.code === 0, stderr: run.stderr }; +} + +const CORE = "packages/core/src/components/components.md"; +const COMPOSITION = "packages/workflow/src/composition/components.md"; + +describe("Tier SYN — the documentation build gate", () => { + it("SYN41: passes on the shipped set", function* () { + const clean = yield* validate(); + expect(clean.ok).toBe(true); + expect(clean.stderr).toContain("complete"); + }); + + it("SYN42: refuses a deleted section", function* () { + // `## Fetch` and its body, gone — the drift that happens when a component + // is documented and the section is later lost to a bad merge. + yield* planted(CORE, (text) => { + const start = text.indexOf("## Fetch"); + const end = text.indexOf("## Glob"); + return text.slice(0, start) + text.slice(end); + }); + const refused = yield* validate(); + expect(refused.ok).toBe(false); + expect(refused.stderr).toContain("Fetch"); + expect(refused.stderr).toContain("no documentation"); + }); + + it("SYN43: refuses an unknown section", function* () { + // Documentation for something the package does not supply — a rename that + // updated the code and not the file. + yield* planted(CORE, (text) => `${text}\n## Nonexistent\n\nAbout nothing.\n`); + const refused = yield* validate(); + expect(refused.ok).toBe(false); + expect(refused.stderr).toContain("Nonexistent"); + }); + + it("SYN44: refuses a duplicated section", function* () { + yield* planted(CORE, (text) => `${text}\n## Fetch\n\nA second Fetch.\n`); + const refused = yield* validate(); + expect(refused.ok).toBe(false); + expect(refused.stderr).toContain("Fetch"); + }); + + it("SYN45: refuses drift in a package outside core", function* () { + // The same gate covers every boundary, not only the first one wired. + yield* planted(COMPOSITION, (text) => { + const start = text.indexOf("## Git.Push"); + const end = text.indexOf("## PullRequest\n"); + return text.slice(0, start) + text.slice(end); + }); + const refused = yield* validate(); + expect(refused.ok).toBe(false); + expect(refused.stderr).toContain("Git.Push"); + }); +}); diff --git a/scripts/tests/fixtures/event-registration-bindings.ts b/scripts/tests/fixtures/event-registration-bindings.ts new file mode 100644 index 000000000..dba98ba82 --- /dev/null +++ b/scripts/tests/fixtures/event-registration-bindings.ts @@ -0,0 +1,92 @@ +/** Values whose `on`, `once` and `addEventListener` are somebody else's. */ +import { action as act, ensure as guard } from "effection"; +import type { Operation } from "effection"; +import { EventEmitter as NodeEmitter } from "node:events"; +import { connect } from "./event-registration-not-net.ts"; + +const handle = () => {}; + +/** A router of this application's own, not an event source. */ +const routes = { + on(_path: string, _handler: () => void) {}, + once(_path: string, _handler: () => void) {}, +}; + +export function* anApplicationRouter(): Operation { + routes.on("/health", handle); + routes.once("/ready", handle); +} + +export function* aBindingFromAnotherModule(): Operation { + const client = connect("postgres://localhost"); + client.on("notice", handle); + client.once("end", handle); +} + +/** Declared with `on` alone, which pairs with nothing. */ +interface Unsubscribable { + on(event: string, listener: () => void): unknown; +} + +export function* anUnpairedInterface(stream: Unsubscribable): Operation { + stream.on("data", handle); +} + +/** A class of the same name as the Node export, extending nothing. */ +class EventEmitter { + on(_event: string, _handler: () => void) {} + once(_event: string, _handler: () => void) {} +} + +export function* aShadowedConstructor(): Operation { + const emitter = new EventEmitter(); + emitter.on("ready", handle); + emitter.once("ready", handle); +} + +export function* aShadowedProcess(): Operation { + const process = { on: (_event: string, _handler: () => void) => {} }; + process.on("SIGINT", handle); +} + +export function* aShadowedDomGlobal(): Operation { + const document = { addEventListener: (_event: string, _handler: () => void) => {} }; + document.addEventListener("click", handle); +} + +export function* aParameterWithNoType(source: unknown): Operation { + (source as { on(event: string, handler: () => void): void }).on("data", handle); +} + +export function* aReassignedBinding(): Operation { + let socket; + socket = routes; + socket.on("data", handle); +} + +/** + * `ensure` and `action` are resolved through their import, not their spelling, + * so a renamed one still establishes the owner and its cleanup. + */ +export function* anAliasedEnsurePairs(): Operation { + const emitter = new NodeEmitter(); + let onReady: (() => void) | undefined; + + yield* guard(() => { + if (onReady) { + emitter.off("ready", onReady); + } + }); + + onReady = () => {}; + emitter.on("ready", onReady); +} + +export function nextReadyUnderAnAliasedAction(emitter: NodeEmitter): Operation { + return act((resolve) => { + const onReady = (): void => resolve(); + + emitter.on("ready", onReady); + return () => emitter.off("ready", onReady); + }); +} diff --git a/scripts/tests/fixtures/event-registration-exempted.ts b/scripts/tests/fixtures/event-registration-exempted.ts new file mode 100644 index 000000000..d5f9ffbf4 --- /dev/null +++ b/scripts/tests/fixtures/event-registration-exempted.ts @@ -0,0 +1,13 @@ +/* oxlint-disable local/require-scope-bound-event-registration */ +/** A file-wide directive states an invariant for nothing, and is rejected. */ +import type { Operation } from "effection"; +import { EventEmitter } from "node:events"; + +const handle = () => {}; + +export function* twoSubscriptions(): Operation { + const emitter = new EventEmitter(); + + emitter.on("ready", handle); + emitter.on("failed", handle); +} diff --git a/scripts/tests/fixtures/event-registration-not-net.ts b/scripts/tests/fixtures/event-registration-not-net.ts new file mode 100644 index 000000000..d7fecc33d --- /dev/null +++ b/scripts/tests/fixtures/event-registration-not-net.ts @@ -0,0 +1,7 @@ +/** A `connect` that is not the one `node:net` exports. */ +export function connect(_url: string) { + return { + on(_event: string, _handler: () => void) {}, + once(_event: string, _handler: () => void) {}, + }; +} diff --git a/scripts/tests/fixtures/event-registration-paired.ts b/scripts/tests/fixtures/event-registration-paired.ts new file mode 100644 index 000000000..cc52bbfad --- /dev/null +++ b/scripts/tests/fixtures/event-registration-paired.ts @@ -0,0 +1,143 @@ +/** The four teardown shapes that release a listener with its owner. */ +import { action, ensure, resource, withResolvers } from "effection"; +import type { Operation } from "effection"; +import { once } from "@effectionx/node/events"; +import { spawn as spawnChild } from "node:child_process"; +import { createServer } from "node:net"; +import type { Socket } from "node:net"; + +export function* finallyAroundTheSubscription(child = spawnChild("cat", [])): Operation { + const chunks: string[] = []; + const onData = (chunk: string) => chunks.push(chunk); + + try { + child.stdout.on("data", onData); + yield* action((resolve) => { + const onExit = () => resolve(); + child.on("exit", onExit); + return () => child.off("exit", onExit); + }); + } finally { + child.stdout.off("data", onData); + } + + return chunks.join(""); +} + +export function* ensureEstablishedBeforeTheSubscription(): Operation { + const server = createServer(); + const accepted: Socket[] = []; + let onConnection: ((socket: Socket) => void) | undefined; + + yield* ensure(() => { + if (onConnection) { + server.off("connection", onConnection); + } + }); + + onConnection = (socket: Socket) => accepted.push(socket); + server.on("connection", onConnection); + + yield* once(server, "listening"); +} + +export function* listenerDependentTeardown(): Operation { + const server = createServer(); + + // The wait needs a listener, so the cleanup attaches its own rather than + // depending on one the body established: registering it here and pairing it + // in this same frame's `finally` is what makes the whole thing atomic. + yield* ensure(function* () { + const closed = withResolvers(); + const onClose = () => closed.resolve(); + + server.on("close", onClose); + try { + server.close(); + yield* closed.operation; + } finally { + server.off("close", onClose); + } + }); + + yield* once(server, "listening"); +} + +export function useSocketErrors(socket: Socket): Operation { + return resource(function* (provide) { + const errors: Error[] = []; + let onError: ((error: Error) => void) | undefined; + + yield* ensure(() => { + if (onError) { + socket.off("error", onError); + } + }); + + onError = (error: Error) => errors.push(error); + socket.on("error", onError); + + yield* provide(errors); + }); +} + +export function nextMessage(port: MessagePort): Operation { + return action((resolve) => { + const onMessage = (event: MessageEvent) => resolve(event.data); + + port.addEventListener("message", onMessage); + return () => port.removeEventListener("message", onMessage); + }); +} + +export function nextCapturedClick(target: EventTarget): Operation { + return action((resolve) => { + const onClick = () => resolve(); + + target.addEventListener("click", onClick, true); + return () => target.removeEventListener("click", onClick, true); + }); +} + +export function* theTryCoversASubscriptionJustBeforeIt(socket: Socket): Operation { + const errors: Error[] = []; + const onError = (error: Error) => errors.push(error); + + socket.on("error", onError); + try { + yield* once(socket, "close"); + } finally { + socket.off("error", onError); + } +} + +export function* aCollectionIsThePairingForOnePerConnection(): Operation { + const server = createServer(); + const closers = new Map void>(); + + let onConnection: ((accepted: Socket) => void) | undefined; + + yield* ensure(function* () { + if (onConnection) { + server.off("connection", onConnection); + } + for (const [socket, onClose] of closers) { + socket.off("close", onClose); + } + closers.clear(); + yield* once(server, "close"); + }); + + onConnection = (accepted: Socket): void => { + const onClose = (): void => closers.delete(accepted); + closers.set(accepted, onClose); + accepted.on("close", onClose); + }; + server.on("connection", onConnection); + + yield* once(server, "listening"); +} + +export function* theCorrectedHelperIsTheOneEventWait(socket: Socket): Operation { + yield* once(socket, "close"); +} diff --git a/scripts/tests/fixtures/event-registration-raw-once.ts b/scripts/tests/fixtures/event-registration-raw-once.ts new file mode 100644 index 000000000..2692fd331 --- /dev/null +++ b/scripts/tests/fixtures/event-registration-raw-once.ts @@ -0,0 +1,45 @@ +/** One-event APIs whose cleanup depends on the event arriving. */ +import { action, ensure } from "effection"; +import type { Operation } from "effection"; +import { EventEmitter } from "node:events"; +import { connect } from "node:net"; + +export function* waitForClose(): Operation { + const socket = connect(1234, "localhost"); + + yield* action((resolve) => { + socket.once("close", () => resolve()); + return () => {}; + }); +} + +export function* waitForReady(): Operation { + const emitter = new EventEmitter(); + + emitter.once("ready", handle); + + function handle() {} + + yield* ensure(() => { + emitter.off("ready", handle); + }); +} + +export function* watchTheDocument(): Operation { + const target = new EventTarget(); + + target.addEventListener("ready", handle, { once: true }); + + function handle() {} + + yield* ensure(() => { + target.removeEventListener("ready", handle); + }); +} + +export function* alsoOnceThroughAnAlias(): Operation { + const socket = connect(1234, "localhost"); + const alias = socket; + + alias.once("error", () => {}); +} diff --git a/scripts/tests/fixtures/event-registration-sources.ts b/scripts/tests/fixtures/event-registration-sources.ts new file mode 100644 index 000000000..21bfb758d --- /dev/null +++ b/scripts/tests/fixtures/event-registration-sources.ts @@ -0,0 +1,106 @@ +/** One unpaired subscription per source family the rule recognizes. */ +import type { Operation } from "effection"; +import { EventEmitter } from "node:events"; +import { spawn as spawnChild } from "node:child_process"; +import { createServer as createNetServer, connect } from "node:net"; +import { createServer as createHttpServer } from "node:http"; +import type { IncomingMessage } from "node:http"; +import { PassThrough } from "node:stream"; +import type { Readable } from "node:stream"; +import process from "node:process"; + +const handle = () => {}; + +export function* fromAConstructor(): Operation { + const emitter = new EventEmitter(); + emitter.on("ready", handle); +} + +export function* fromAFactory(): Operation { + const server = createNetServer(); + server.on("connection", handle); +} + +export function* fromAConnection(): Operation { + const socket = connect(1234, "localhost"); + socket.on("data", handle); +} + +export function* fromAnHttpServer(): Operation { + const server = createHttpServer(); + server.on("request", handle); +} + +export function* fromAChildStream(): Operation { + const child = spawnChild("cat", []); + child.stdout.on("data", handle); +} + +export function* fromAStream(): Operation { + const stream = new PassThrough(); + stream.on("data", handle); +} + +export function* fromAnAlias(): Operation { + const emitter = new EventEmitter(); + const alias = emitter; + alias.on("ready", handle); +} + +export function* fromAnAnnotatedParameter(request: IncomingMessage): Operation { + request.on("data", handle); +} + +export function* fromAnAnnotatedReadable(stream: Readable): Operation { + stream.on("data", handle); +} + +export function* fromTheProcessGlobal(): Operation { + process.on("SIGINT", handle); +} + +export function* fromAProcessStream(): Operation { + process.stdin.on("data", handle); +} + +class Bus extends EventEmitter { + *listen(): Operation { + this.on("ready", handle); + } +} + +export const bus = new Bus(); + +export function* fromAnEmitterSubclass(): Operation { + const own = new Bus(); + own.on("ready", handle); +} + +/** A structural surface the file declares, paired the way the policy requires. */ +export interface InputStream { + on(event: "data", listener: () => void): unknown; + off(event: "data", listener: () => void): unknown; +} + +export function* fromAStructuralInterface(stream: InputStream): Operation { + stream.on("data", handle); +} + +export function* fromAnXhr(): Operation { + const request = new XMLHttpRequest(); + request.addEventListener("load", handle); +} + +export function* fromAnAbortSignal(): Operation { + const controller = new AbortController(); + controller.signal.addEventListener("abort", handle); +} + +export function* fromADomGlobal(): Operation { + globalThis.addEventListener("unhandledrejection", handle); +} + +export function* fromAWorker(): Operation { + const worker = new Worker("./worker.js"); + worker.addEventListener("message", handle); +} diff --git a/scripts/tests/fixtures/event-registration-suppressed.ts b/scripts/tests/fixtures/event-registration-suppressed.ts new file mode 100644 index 000000000..d0ced695d --- /dev/null +++ b/scripts/tests/fixtures/event-registration-suppressed.ts @@ -0,0 +1,14 @@ +/** A narrow directive covers one line, and one only. */ +import type { Operation } from "effection"; +import { EventEmitter } from "node:events"; + +const handle = () => {}; + +export function* twoSubscriptions(): Operation { + const emitter = new EventEmitter(); + + // The stated invariant this directive stands on would go here. + // oxlint-disable-next-line local/require-scope-bound-event-registration + emitter.on("ready", handle); + emitter.on("failed", handle); +} diff --git a/scripts/tests/fixtures/event-registration-unowned.ts b/scripts/tests/fixtures/event-registration-unowned.ts new file mode 100644 index 000000000..2b51c6db2 --- /dev/null +++ b/scripts/tests/fixtures/event-registration-unowned.ts @@ -0,0 +1,30 @@ +/** + * Listeners whose lifetime is the process or the page, installed where no + * Effection scope owns them. A standalone fixture program lives exactly as + * long as its listeners do; there is nothing to release them from. + */ +import { EventEmitter } from "node:events"; +import { createServer } from "node:net"; +import process from "node:process"; + +const handle = () => {}; + +process.on("SIGINT", handle); +globalThis.addEventListener("unhandledrejection", handle); + +export function serveForTheProcessLifetime() { + const server = createServer(); + server.on("connection", handle); + return server; +} + +export function waitForReady(emitter: EventEmitter): Promise { + return new Promise((resolve) => { + emitter.on("ready", () => resolve()); + }); +} + +export function subscribeForThePageLifetime() { + const target = new EventTarget(); + target.addEventListener("ready", handle); +} diff --git a/scripts/tests/fixtures/event-registration-unpaired.ts b/scripts/tests/fixtures/event-registration-unpaired.ts new file mode 100644 index 000000000..dbb4cef18 --- /dev/null +++ b/scripts/tests/fixtures/event-registration-unpaired.ts @@ -0,0 +1,294 @@ +/** Subscriptions whose cleanup does not release what they attached. */ +import { ensure, resource, sleep } from "effection"; +import type { Operation } from "effection"; +import { spawn as spawnChild } from "node:child_process"; +import { connect, createServer } from "node:net"; +import type { Socket } from "node:net"; + +export function* anonymousHandler(): Operation { + const server = createServer(); + + server.on("connection", () => {}); +} + +export function* noCleanupAtAll(): Operation { + const server = createServer(); + const onConnection = () => {}; + + server.on("connection", onConnection); +} + +export function* differentEvent(): Operation { + const server = createServer(); + const onConnection = () => {}; + + server.on("connection", onConnection); + + yield* ensure(() => { + server.off("close", onConnection); + }); +} + +export function* differentHandler(): Operation { + const server = createServer(); + const onConnection = () => {}; + const onClose = () => {}; + + server.on("connection", onConnection); + + yield* ensure(() => { + server.off("connection", onClose); + }); +} + +export function* differentReceiver(): Operation { + const server = createServer(); + const other = createServer(); + const onConnection = () => {}; + + server.on("connection", onConnection); + + yield* ensure(() => { + other.off("connection", onConnection); + }); +} + +export function* armedTooLate(): Operation { + const child = spawnChild("cat", []); + const onExit = () => {}; + + child.on("exit", onExit); + + yield* sleep(1); + + yield* ensure(() => { + child.off("exit", onExit); + }); +} + +export function* removesItselfOnly(): Operation { + const child = spawnChild("cat", []); + const onExit = () => { + child.off("exit", onExit); + }; + + child.on("exit", onExit); +} + +export function* removesAfterASuspension(): Operation { + const child = spawnChild("cat", []); + const onExit = () => {}; + + child.on("exit", onExit); + + yield* ensure(function* () { + yield* sleep(1); + child.off("exit", onExit); + }); +} + +export function* removesWithRemoveListener(): Operation { + const child = spawnChild("cat", []); + const onExit = () => {}; + + child.on("exit", onExit); + + yield* ensure(() => { + child.removeListener("exit", onExit); + }); +} + +export function* dynamicEventName(event: string): Operation { + const server = createServer(); + const onEvent = () => {}; + + server.on(event, onEvent); + + yield* ensure(() => { + server.off(event, onEvent); + }); +} + +export function* aTryOpenedAfterASuspension(socket: Socket): Operation { + const errors: Error[] = []; + const onError = (error: Error) => errors.push(error); + + socket.on("error", onError); + + yield* sleep(1); + + try { + yield* sleep(1); + } finally { + socket.off("error", onError); + } +} + +export function* recordedButNeverDetached(): Operation { + const server = createServer(); + const closers = new Map void>(); + + const onConnection = (accepted: Socket): void => { + const onClose = (): void => closers.delete(accepted); + closers.set(accepted, onClose); + accepted.on("close", onClose); + }; + + server.on("connection", onConnection); + yield* ensure(() => { + server.off("connection", onConnection); + closers.clear(); + }); +} + +export function* detachedFromAnotherCollection(): Operation { + const server = createServer(); + const closers = new Map void>(); + const others = new Map void>(); + + const onConnection = (accepted: Socket): void => { + const onClose = (): void => closers.delete(accepted); + closers.set(accepted, onClose); + accepted.on("close", onClose); + }; + + server.on("connection", onConnection); + yield* ensure(() => { + server.off("connection", onConnection); + for (const [socket, onClose] of others) { + socket.off("close", onClose); + } + }); +} + +export function* mismatchedCapture(): Operation { + const target = new EventTarget(); + const onReady = () => {}; + + target.addEventListener("ready", onReady, true); + + yield* ensure(() => { + target.removeEventListener("ready", onReady); + }); +} + +export function* cleanupBehindACondition(flag: boolean): Operation { + const server = createServer(); + const onConnection = () => {}; + + server.on("connection", onConnection); + + if (flag) { + yield* ensure(() => { + server.off("connection", onConnection); + }); + } +} + +export function* cleanupInAnOuterOwner(): Operation { + const server = createServer(); + const onConnection = () => {}; + + const subscribe = function* (): Operation { + server.on("connection", onConnection); + }; + + yield* subscribe(); + yield* ensure(() => { + server.off("connection", onConnection); + }); +} + +export function* recordedAfterASuspension(): Operation { + const closers = new Map void>(); + const accepted = connect(1234, "localhost"); + const onClose = (): void => closers.delete(accepted); + + accepted.on("close", onClose); + + yield* sleep(1); + + closers.set(accepted, onClose); + yield* ensure(() => { + for (const [socket, handler] of closers) { + socket.off("close", handler); + } + }); +} + +export function* recordedInAShadowedCollection(): Operation { + const server = createServer(); + const onConnection = (accepted: Socket): void => { + const closers = new Map void>(); + const onClose = (): void => closers.delete(accepted); + accepted.on("close", onClose); + closers.set(accepted, onClose); + }; + const closers = new Map void>(); + + server.on("connection", onConnection); + yield* ensure(() => { + server.off("connection", onConnection); + for (const [socket, onClose] of closers) { + socket.off("close", onClose); + } + }); +} + +const ALWAYS = true; + +export function* onceSpelledAsAString(): Operation { + const target = new EventTarget(); + const onReady = () => {}; + + target.addEventListener("ready", onReady, { "once": true }); + + yield* ensure(() => { + target.removeEventListener("ready", onReady); + }); +} + +export function* onceBoundToAConstant(): Operation { + const target = new EventTarget(); + const onReady = () => {}; + + target.addEventListener("ready", onReady, { once: ALWAYS }); + + yield* ensure(() => { + target.removeEventListener("ready", onReady); + }); +} + +export function useSocketObservers(): Operation<{ watch(port: number): Operation }> { + const observers = new Map void>(); + + return resource(function* (provide) { + yield* ensure(() => { + for (const [socket, onError] of observers) { + socket.off("error", onError); + } + observers.clear(); + }); + + yield* provide({ + *watch(port: number): Operation { + const socket = connect(port, "localhost"); + const onError = (): void => {}; + + socket.on("error", onError); + observers.set(socket, onError); + yield* sleep(1); + }, + }); + }); +} + +export function* ensureYieldedAfterTheSubscription(): Operation { + const server = createServer(); + const onConnection = () => {}; + + server.on("connection", onConnection); + yield* ensure(() => { + server.off("connection", onConnection); + }); +} diff --git a/scripts/tests/jsr-consumer-documentation.test.ts b/scripts/tests/jsr-consumer-documentation.test.ts new file mode 100644 index 000000000..a10628792 --- /dev/null +++ b/scripts/tests/jsr-consumer-documentation.test.ts @@ -0,0 +1,249 @@ +/** + * Tier SYN — a JSR consumer's named lookup. + * + * Two claims, and they are different. That `components.md` appears in + * `deno publish --dry-run` says the asset is *selected into the payload* — the + * publish filter keeps it, rather than dropping it as tooling. That a consumer + * can render it says the asset is *reachable from the published layout*, which + * is a fact about how the module resolves it and not about the file list. + * + * So this proves both, in that order, and stages exactly what publish selected + * rather than copying a source directory: a recursive copy would pass even if + * the publish filter excluded every asset. The consumer then writes ordinary + * XMD and invokes the public `` surface, because that is the + * thing an author actually reaches — calling the index directly would skip + * selection, the renderer, availability and the whole component. + */ + +import { describe, it } from "@executablemd/test-support/bdd"; +import { expect } from "@executablemd/test-support/expect"; +import { ensure, until } from "effection"; +import type { Operation } from "effection"; +import { ensureDir, writeTextFile } from "@effectionx/fs"; +import { exec } from "@effectionx/process"; +import { timebox } from "@effectionx/timebox"; +import type { ProcessResult } from "@effectionx/process"; +import { cp, mkdir, mkdtemp, rm } from "node:fs/promises"; +import { tmpdir } from "node:os"; +import path from "node:path"; +import { fileURLToPath } from "node:url"; +import { z } from "npm:zod@^4"; + +const ROOT = fileURLToPath(new URL("../../", import.meta.url)); +const TIMEOUT = 180_000; + +/** The workspace members a consumer of core has to resolve. */ +const MEMBERS = ["core", "runtime", "durable-streams", "acp"] as const; + +/** Every documentation asset the product ships, by package-relative path. */ +const ASSETS: Record = { + core: ["src/components/components.md", "src/agent/components.md"], +}; + +/** A package manifest, only as far as this needs it. */ +const Manifest = z.object({ + exports: z.union([z.string(), z.record(z.string(), z.string())]), +}); + +/** The document the consumer runs: the public surface, nothing else. */ +const CONSUMER_DOCUMENT = '\n'; + +/** Which files `deno publish` would actually send, for one package. */ +function* publishedFiles(pkg: string): Operation { + const dry = yield* timebox(TIMEOUT, function* () { + return yield* exec(Deno.execPath(), { + // Not `--quiet`: the file listing *is* the evidence, and quiet suppresses + // exactly the lines this reads. + arguments: ["publish", "--dry-run", "--allow-dirty"], + cwd: path.join(ROOT, "packages", pkg), + env: Deno.env.toObject(), + }).join(); + }); + if (dry.timeout) { + throw new Error(`deno publish --dry-run timed out for ${pkg}`); + } + const listed: string[] = []; + for (const line of `${dry.value.stdout}\n${dry.value.stderr}`.split("\n")) { + const trimmed = line.trim(); + const marker = trimmed.indexOf("file:///"); + if (marker === -1) { + continue; + } + const url = trimmed.slice(marker).split(" ")[0] ?? ""; + listed.push(fileURLToPath(url)); + } + return listed; +} + +describe("Tier SYN — a staged JSR consumer", () => { + it("SYN47: publishes the documentation assets and renders them for a consumer", function* () { + // 1. The publish filter selects the assets. This is the payload claim, and + // it is checked against the real `deno publish` selection rather than + // against a directory listing. + const selected = yield* publishedFiles("core"); + expect(selected.length).toBeGreaterThan(0); + for (const asset of ASSETS.core ?? []) { + const expected = path.join(ROOT, "packages/core", asset); + expect([asset, selected.includes(expected)]).toEqual([asset, true]); + } + + const staged = yield* until(mkdtemp(path.join(tmpdir(), "xmd-jsr-consumer-"))); + yield* ensure(function* () { + yield* until(rm(staged, { recursive: true, force: true })); + }); + + // 2. Stage exactly what publish selected, file by file. A recursive copy + // would pass even if the filter dropped every asset, which is the thing + // this case exists to catch. + const staging: Record = {}; + for (const member of MEMBERS) { + const from = path.join(ROOT, "packages", member); + const to = path.join(staged, member); + const files = member === "core" ? selected : yield* publishedFiles(member); + for (const file of files) { + const relative = path.relative(from, file); + if (relative.startsWith("..")) { + continue; + } + const target = path.join(to, relative); + yield* until(mkdir(path.dirname(target), { recursive: true })); + yield* until(cp(file, target)); + } + staging[member] = to; + } + + // Every asset is in the staged tree because publish selected it, not + // because a copy swept the directory. + for (const asset of ASSETS.core ?? []) { + const stagedAsset = path.join(staging.core ?? "", asset); + expect([asset, yield* exists(stagedAsset)]).toEqual([asset, true]); + } + + // 3. A consumer outside the repository, resolving the staged packages + // through an import map of its own that names no path in this checkout. + const consumer = path.join(staged, "consumer"); + yield* ensureDir(consumer); + yield* writeTextFile(path.join(consumer, "document.md"), CONSUMER_DOCUMENT); + const imports = yield* consumerImports(staging); + yield* writeTextFile(path.join(consumer, "deno.json"), JSON.stringify({ imports }, null, 2)); + yield* writeTextFile( + path.join(consumer, "main.ts"), + [ + "// The public surface, assembled the way a consumer would: core's own", + "// registrations plus its Agent boundary, so the document can name a", + "// component from each of the two documentation assets. One call brings", + "// the Agent registrations and the documentation that describes them,", + "// which is the whole point of the bootstrap being one thing.", + 'import { collect, useAgentComponents } from "@executablemd/core";', + "// The host boundary is its own entrypoint, and a consumer reaches it", + "// the same way: `@executablemd/core/host`.", + 'import { executeInstalled } from "@executablemd/core/host";', + 'import { InMemoryStream } from "@executablemd/durable-streams";', + 'import { main, scoped, until } from "effection";', + 'import { readFile } from "node:fs/promises";', + "", + "await main(function* () {", + ' const content = yield* until(readFile("document.md", "utf8"));', + " const rendered = yield* scoped(function* () {", + " yield* useAgentComponents();", + " return yield* collect(", + " yield* executeInstalled(", + " {", + ' path: "document.md",', + " content,", + " stream: new InMemoryStream(),", + " includes: [],", + " },", + " // Nothing to install: the bootstrap above already put both the", + " // registrations and their documentation in this scope.", + " [],", + " ),", + " );", + " });", + " console.log(String(rendered));", + "});", + ].join("\n"), + ); + + const run = yield* timebox(TIMEOUT, function* () { + return yield* exec(Deno.execPath(), { + arguments: ["run", "--allow-all", "main.ts"], + cwd: consumer, + env: Deno.env.toObject(), + }).join(); + }); + if (run.timeout) { + throw new Error("the staged JSR consumer timed out"); + } + if (run.value.code !== 0) { + throw new Error(`the staged JSR consumer exited ${run.value.code}\n${run.value.stderr}`); + } + + // 4. The same program, resolved against the workspace source instead of the + // staged packages. One document, one profile, two resolutions — so the + // only thing the comparison can differ on is the distribution, which is + // exactly what is under test. Comparing against a different profile + // would compare two catalogs and prove nothing about packaging. + const fromSource = yield* timebox(TIMEOUT, function* () { + return yield* exec(Deno.execPath(), { + arguments: ["run", "--allow-all", "--config", path.join(ROOT, "deno.json"), "main.ts"], + cwd: consumer, + env: Deno.env.toObject(), + }).join(); + }); + if (fromSource.timeout) { + throw new Error("the source surface timed out"); + } + + expect(run.value.stdout.trimEnd()).toBe(fromSource.value.stdout.trimEnd()); + // And it is a real answer: both components, from two different asset + // files, with their metadata and availability. + expect(run.value.stdout).toContain("### ``"); + expect(run.value.stdout).toContain("### ``"); + expect(run.value.stdout).toContain("Asks a person a structured question"); + expect(run.value.stdout).toContain("Sends a prompt and renders the reply"); + expect(run.value.stdout).toContain("`@executablemd/core` (registered default)"); + expect(run.value.stdout).toContain("**Available in this evaluation:** yes"); + }); +}); + +/** The import map a consumer of the staged packages writes. */ +function* consumerImports(staging: Record): Operation> { + const rootManifest = z + .object({ imports: z.record(z.string(), z.string()) }) + .parse(JSON.parse(yield* until(Deno.readTextFile(path.join(ROOT, "deno.json"))))); + + const imports: Record = {}; + // External dependencies resolve as they would for any consumer; workspace + // paths do not travel, which is the point of staging. + for (const [name, target] of Object.entries(rootManifest.imports)) { + if (target.startsWith("npm:") || target.startsWith("jsr:") || target.startsWith("http")) { + imports[name] = target; + } + } + for (const [name, dir] of Object.entries(staging)) { + const manifest = Manifest.parse( + JSON.parse(yield* until(Deno.readTextFile(path.join(dir, "deno.json")))), + ); + const exported = + typeof manifest.exports === "string" ? { ".": manifest.exports } : manifest.exports; + for (const [subpath, target] of Object.entries(exported)) { + const specifier = + subpath === "." + ? `@executablemd/${name}` + : `@executablemd/${name}/${subpath.replace(/^\.\//, "")}`; + imports[specifier] = path.join(dir, target); + } + } + return imports; +} + +/** Whether a staged path is present. */ +function* exists(target: string): Operation { + try { + yield* until(Deno.stat(target)); + return true; + } catch { + return false; + } +} diff --git a/scripts/tests/oxlint-policy.test.ts b/scripts/tests/oxlint-policy.test.ts index 275679d0d..82c43e63a 100644 --- a/scripts/tests/oxlint-policy.test.ts +++ b/scripts/tests/oxlint-policy.test.ts @@ -57,6 +57,7 @@ const GATE_RULES = [ "local/no-yield-in-finally", "local/prefer-effection-operation", "local/prefer-effection-result", + "local/require-scope-bound-event-registration", ]; interface Override { diff --git a/scripts/tests/plan-component-compiled.test.ts b/scripts/tests/plan-component-compiled.test.ts index 67a222919..4c1a1b0f5 100644 --- a/scripts/tests/plan-component-compiled.test.ts +++ b/scripts/tests/plan-component-compiled.test.ts @@ -26,6 +26,7 @@ import { exec } from "@effectionx/process"; import { timebox } from "@effectionx/timebox"; import type { ProcessResult } from "@effectionx/process"; import { createHash } from "node:crypto"; +import { fileURLToPath as fromFileUrl } from "node:url"; import { mkdtemp } from "node:fs/promises"; import { tmpdir } from "node:os"; import path from "node:path"; @@ -100,6 +101,98 @@ describe("compiled xmd", { sanitizeOps: false, sanitizeResources: false }, () => expect(names).not.toContain(name); } + // `` is public, and the compiled binary describes it exactly once + // from the canonical origin. The protected tier ships inside the binary + // rather than being assembled by whoever installs the profile, so a build + // that lost it would describe no catalog component at all. + const syntax = entries.filter((entry: { name?: string }) => entry?.name === "Syntax"); + expect(syntax).toHaveLength(1); + expect(syntax[0].origin).toEqual({ kind: "protected", origin: "@executablemd/core" }); + expect(syntax[0].sourceKind).toBe("protected"); + expect(syntax[0].forms).toEqual(["self-closing"]); + expect(syntax[0].returnMode).toBe("text"); + expect(syntax[0].description).toBe( + "Inspect available components and control-flow constructs. `` lists the " + + 'symbols available here; `` renders selected documentation.', + ); + + // FE28, at the compiled boundary. `` is the tier's second member + // and ships inside the binary for the same reason: a build that carried the + // protected tier without it would let a document write the name with no + // implementation able to answer it, at a person's first `` rather + // than here. This boundary is asserted explicitly because `--changed` + // cannot see it — nothing in `dist/xmd` shares a path with the core sources + // the name is defined in, and no test shard builds the binary at all. + const evaluate = entries.filter((entry: { name?: string }) => entry?.name === "Evaluate"); + expect(evaluate).toHaveLength(1); + expect(evaluate[0].origin).toEqual({ kind: "protected", origin: "@executablemd/core" }); + expect(evaluate[0].sourceKind).toBe("protected"); + // Both spellings, because the two input forms are two ways of stating one + // argument. + expect(evaluate[0].forms).toEqual(["self-closing", "paired"]); + expect(evaluate[0].description).toBe( + 'Evaluate program text. `` runs it.', + ); + // And the closed schema travels with it: a build that shipped a widened one + // would let a fragment-bearing prop nobody validated reach the body. + expect(evaluate[0].props.additionalProperties).toBe(false); + expect(Object.keys(evaluate[0].props.properties)).toEqual(["text", "source", "allow"]); + + // The documentation asset travels with the binary, not with a checkout. A + // build that forgot `--include` would still list the component and still + // print its metadata, and would silently have no prose to attach — so the + // probe is the documentation itself, asked for from a directory that is not + // the checkout. + const lookup = yield* timebox(TIMEOUT, function* () { + return yield* exec(BINARY, { + arguments: ["syntax", "Elicit", "--include", elsewhere], + cwd: elsewhere, + }).join(); + }); + if (lookup.timeout) { + throw new Error("the compiled binary timed out documenting one component"); + } + expect(lookup.value.code).toBe(0); + expect(lookup.value.stdout).toContain("### ``"); + expect(lookup.value.stdout).toContain("Asks a person a structured question"); + expect(lookup.value.stdout).toContain("**Available in this evaluation:** yes"); + + // A component from a boundary *outside* core's own documentation file, so + // the probe exercises a second copied asset path rather than proving only + // that the first one shipped. + const outside = yield* timebox(TIMEOUT, function* () { + return yield* exec(BINARY, { + arguments: ["syntax", "Git.Commit", "--include", elsewhere], + cwd: elsewhere, + }).join(); + }); + if (outside.timeout) { + throw new Error("the compiled binary timed out documenting a composition component"); + } + expect(outside.value.code).toBe(0); + expect(outside.value.stdout).toContain("### ``"); + expect(outside.value.stdout).toContain("Commits what is staged"); + + // And the compiled answer is the source answer, byte for byte. + const fromSource = yield* timebox(TIMEOUT, function* () { + return yield* exec("deno", { + arguments: [ + "run", + "--allow-all", + fromFileUrl(new URL("../../packages/cli/src/deno.ts", import.meta.url)), + "syntax", + "Git.Commit", + "--include", + elsewhere, + ], + cwd: elsewhere, + }).join(); + }); + if (fromSource.timeout) { + throw new Error("the source CLI timed out documenting a composition component"); + } + expect(fromSource.value.stdout).toBe(outside.value.stdout); + // The command surface those bytes belong to is source-only in this build // too: help describes both explicit compositions and names no option that // would run the approved program. diff --git a/scripts/tests/scope-bound-event-registration.test.ts b/scripts/tests/scope-bound-event-registration.test.ts new file mode 100644 index 000000000..68df0b148 --- /dev/null +++ b/scripts/tests/scope-bound-event-registration.test.ts @@ -0,0 +1,493 @@ +/** + * The lifecycle contract this repository requires of the one-event helper it + * standardises on, `once()` from `@effectionx/node/events`. + * + * A listener is scoped state. The helper therefore has to behave like every + * other Effection resource: register nothing until somebody interprets it, + * register exactly once when they do, and detach on every way the interpreting + * scope can end — the event arriving, a halt, or losing a `race()`. Cleanup + * that runs only when the event fires is not cleanup, because the event is + * exactly what a cancelled wait never gets. + * + * Node and the DOM count listeners differently, so both families are proved + * here: `EventEmitter` answers `listenerCount()` directly, and `EventTarget` + * has no equivalent, so the target below counts its own registrations. + * + * Tracked upstream as thefrontside/effectionx#251. Until a release that fixes + * it is published, this file fails on construction being eager, which is the + * defect. + */ +import { describe, it } from "@executablemd/test-support/bdd"; +import { expect } from "@executablemd/test-support/expect"; +import { once } from "@effectionx/node/events"; +import { readTextFile, walk } from "@effectionx/fs"; +import { each, race, sleep, spawn } from "effection"; +import type { Operation, Task } from "effection"; +import { EventEmitter } from "node:events"; +import path from "node:path"; + +import { oxlint, ROOT, violations } from "./oxlint.ts"; + +/** + * An `EventTarget` that reports how many listeners it is holding. The platform + * exposes no count, so the registrations are tallied as they are made. + */ +class CountingTarget extends EventTarget { + #live = new Map(); + + override addEventListener( + type: string, + listener: EventListenerOrEventListenerObject | null, + options?: AddEventListenerOptions | boolean, + ): void { + this.#live.set(type, (this.#live.get(type) ?? 0) + 1); + super.addEventListener(type, listener, options); + } + + override removeEventListener( + type: string, + listener: EventListenerOrEventListenerObject | null, + options?: EventListenerOptions | boolean, + ): void { + this.#live.set(type, (this.#live.get(type) ?? 0) - 1); + super.removeEventListener(type, listener, options); + } + + listenerCount(type: string): number { + return this.#live.get(type) ?? 0; + } +} + +/** Give a spawned child its first turn, so its registrations have happened. */ +function started(): Operation { + return sleep(0); +} + +/** Let a settled child's teardown finish before its listeners are counted. */ +function settled(): Operation { + return sleep(0); +} + +describe("@effectionx/node once() is scope-bound", () => { + describe("with a Node EventEmitter", () => { + it("registers nothing until the operation is interpreted", function* () { + const emitter = new EventEmitter(); + + once(emitter, "ready"); + + expect(emitter.listenerCount("ready")).toBe(0); + }); + + it("registers exactly one listener when interpreted", function* () { + const emitter = new EventEmitter(); + + yield* spawn(function* () { + yield* once(emitter, "ready"); + }); + yield* started(); + + expect(emitter.listenerCount("ready")).toBe(1); + }); + + it("settles once with the emitted arguments and detaches", function* () { + const emitter = new EventEmitter(); + const waiter: Task = yield* spawn(function* () { + return yield* once(emitter, "ready"); + }); + yield* started(); + + emitter.emit("ready", "first", 2); + const args = yield* waiter; + + expect(args).toEqual(["first", 2]); + expect(emitter.listenerCount("ready")).toBe(0); + }); + + it("detaches when the interpreting scope halts before the event", function* () { + const emitter = new EventEmitter(); + const waiter = yield* spawn(function* () { + yield* once(emitter, "ready"); + }); + yield* started(); + + yield* waiter.halt(); + + expect(emitter.listenerCount("ready")).toBe(0); + }); + + it("detaches the arm that loses a race", function* () { + const emitter = new EventEmitter(); + const winner: Task = yield* spawn(function* () { + return yield* race([once(emitter, "ready"), once(emitter, "failed")]); + }); + yield* started(); + + expect(emitter.listenerCount("failed")).toBe(1); + + emitter.emit("ready", "won"); + yield* winner; + yield* settled(); + + expect(emitter.listenerCount("failed")).toBe(0); + }); + + /** + * A leaked listener is not inert: it still holds a resolver for work + * nobody is waiting on, and the next interpretation would see two. An + * event after the halt therefore has to reach nothing at all, and a fresh + * wait has to observe the event after it rather than the one it missed. + */ + it("ignores an event emitted after the wait was abandoned", function* () { + const emitter = new EventEmitter(); + const abandoned = yield* spawn(function* () { + yield* once(emitter, "ready"); + }); + yield* started(); + yield* abandoned.halt(); + + // Asserted before the emit, not after: a leaked listener that removes + // itself when the event finally arrives leaves the same count behind as + // one that was never there. + expect(emitter.listenerCount("ready")).toBe(0); + + emitter.emit("ready", "ignored"); + + const waiter: Task = yield* spawn(function* () { + return yield* once(emitter, "ready"); + }); + yield* started(); + + expect(emitter.listenerCount("ready")).toBe(1); + + emitter.emit("ready", "observed"); + + expect(yield* waiter).toEqual(["observed"]); + }); + }); + + describe("with a DOM EventTarget", () => { + it("registers nothing until the operation is interpreted", function* () { + const target = new CountingTarget(); + + once(target, "ready"); + + expect(target.listenerCount("ready")).toBe(0); + }); + + it("registers exactly one listener when interpreted", function* () { + const target = new CountingTarget(); + + yield* spawn(function* () { + yield* once(target, "ready"); + }); + yield* started(); + + expect(target.listenerCount("ready")).toBe(1); + }); + + it("settles once with the dispatched event and detaches", function* () { + const target = new CountingTarget(); + const waiter: Task = yield* spawn(function* () { + return yield* once(target, "ready"); + }); + yield* started(); + + target.dispatchEvent(new Event("ready")); + const [event] = yield* waiter; + + expect(event.type).toBe("ready"); + expect(target.listenerCount("ready")).toBe(0); + }); + + it("detaches when the interpreting scope halts before the event", function* () { + const target = new CountingTarget(); + const waiter = yield* spawn(function* () { + yield* once(target, "ready"); + }); + yield* started(); + + yield* waiter.halt(); + + expect(target.listenerCount("ready")).toBe(0); + }); + + it("detaches the arm that loses a race", function* () { + const target = new CountingTarget(); + const winner: Task = yield* spawn(function* () { + return yield* race([once(target, "ready"), once(target, "failed")]); + }); + yield* started(); + + expect(target.listenerCount("failed")).toBe(1); + + target.dispatchEvent(new Event("ready")); + yield* winner; + yield* settled(); + + expect(target.listenerCount("failed")).toBe(0); + }); + + it("ignores an event dispatched after the wait was abandoned", function* () { + const target = new CountingTarget(); + const abandoned = yield* spawn(function* () { + yield* once(target, "ready"); + }); + yield* started(); + yield* abandoned.halt(); + + // Asserted before the dispatch, for the reason given above. + expect(target.listenerCount("ready")).toBe(0); + + target.dispatchEvent(new Event("ready")); + + const waiter: Task = yield* spawn(function* () { + return yield* once(target, "ready"); + }); + yield* started(); + + expect(target.listenerCount("ready")).toBe(1); + + const dispatched = new Event("ready"); + target.dispatchEvent(dispatched); + + expect((yield* waiter)[0]).toBe(dispatched); + }); + }); +}); + +const RULE = "require-scope-bound-event-registration"; + +/** The directories `deno task lint` passes to oxlint. */ +const LINTED = ["packages", "scripts", ".reviews/components"]; + +/** What the lint task's `--ignore-pattern` arguments keep out. */ +const UNLINTED = [ + `${path.sep}npm${path.sep}`, + path.join("scripts", "tests", "fixtures"), + path.join("packages", "workflow", "vendor", "cloudflare-computer-dofs"), + `${path.sep}node_modules${path.sep}`, +]; + +/** A whole-file directive, as opposed to `-next-line` or `-line`. */ +const FILE_WIDE = new RegExp(`(?:oxlint|eslint)-disable(?!-next-line|-line)[^\\n]*${RULE}`, "u"); + +const NARROW = new RegExp(`oxlint-disable-next-line[^\\n]*local/${RULE}`, "u"); + +function reported(fixture: string): Operation { + return violations(`scripts/tests/fixtures/${fixture}`, RULE); +} + +/** The rule's own diagnostics for a fixture, in source order. */ +function* diagnostics(fixture: string): Operation<{ line: number; message: string }[]> { + const output = yield* oxlint(["--format=json", `scripts/tests/fixtures/${fixture}`]); + const report: { + diagnostics: { code: string; message: string; labels: { span: { line: number } }[] }[]; + } = JSON.parse(output); + + return report.diagnostics + .filter((entry) => entry.code === `local(${RULE})`) + .map((entry) => ({ line: entry.labels[0].span.line, message: entry.message })) + .sort((left, right) => left.line - right.line); +} + +/** Every file `deno task lint` actually reads. */ +function* linted(): Operation { + const files: string[] = []; + for (const directory of LINTED) { + const entries = walk(path.join(ROOT, directory), { + includeDirs: false, + skip: [/node_modules/u, /[/\\]npm[/\\]/u], + }); + for (const entry of yield* each(entries)) { + if ( + /\.(?:ts|tsx|js|mjs|cjs)$/u.test(entry.path) && + !UNLINTED.some((fragment) => entry.path.includes(fragment)) + ) { + files.push(entry.path); + } + yield* each.next(); + } + } + return files; +} + +describe("local/require-scope-bound-event-registration", () => { + /** + * In fixture order: `.once()` on a socket inside an `action()`, on an + * emitter whose `ensure()` would otherwise pair it, `{ once: true }` on a + * DOM target, and `.once()` reached through an alias of the socket. + */ + it("reports one-event APIs whose cleanup waits for the event", function* () { + expect(yield* reported("event-registration-raw-once.ts")).toEqual([11, 19, 31, 44]); + }); + + it("names the scope-bound helper as the replacement for a one-event wait", function* () { + const report = yield* diagnostics("event-registration-raw-once.ts"); + + expect(report[0].message).toContain("@effectionx/node/events"); + expect(report[0].message).toContain("cancelled wait never gets"); + }); + + /** + * In fixture order: an inline handler, no cleanup at all, a cleanup naming a + * different event, a different handler and a different receiver, an + * `ensure()` armed after the owner has already suspended, a handler that + * only removes itself, a teardown that suspends before removing, a + * `removeListener()` where `.off()` belongs, a computed event name, a + * `try` opened only after the owner has suspended, a per-connection handler + * recorded in a collection nothing walks and one walked from a different + * collection, a DOM removal whose capture mode disagrees with the + * registration, cleanup behind a condition that may not run, cleanup in an + * outer owner rather than the one that subscribed, a pair recorded only + * after the owner suspended, a pair recorded into a shadowed collection of + * the same name, and a one-shot option spelled as a string key or bound to + * a constant, a pair a nested generator records that only the resource + * around it walks, and an `ensure()` yielded after the subscription — whose + * own registration is a suspension an owner can be halted at. + */ + it("reports every way a subscription outlives its owner", function* () { + expect(yield* reported("event-registration-unpaired.ts")).toEqual([ + 11, 18, 25, 37, 49, 60, 75, 82, 94, 105, 116, 134, 137, 152, 155, 168, 179, 193, 207, 224, + 229, 244, 255, 278, 290, + ]); + }); + + it("distinguishes a missing cleanup from one that names the wrong pair", function* () { + const report = yield* diagnostics("event-registration-unpaired.ts"); + const at = (line: number) => report.find((entry) => entry.line === line)?.message ?? ""; + + expect(at(18)).toContain("Nothing removes onConnection"); + expect(at(25)).toContain("does not name the same receiver, event, handler and capture mode"); + expect(at(60)).toContain("registered after the owner can already suspend"); + expect(at(75)).toContain("removes itself only when"); + expect(at(82)).toContain("Teardown suspends before it removes"); + expect(at(94)).toContain("removeListener()"); + expect(at(105)).toContain("event name is computed"); + expect(at(116)).toContain("registered after the owner can already suspend"); + expect(at(134)).toContain("Nothing removes onClose"); + expect(at(152)).toContain("does not name the same receiver, event, handler and capture mode"); + expect(at(179)).toContain("registered after the owner can already suspend"); + expect(at(193)).toContain("Nothing removes onConnection"); + expect(at(207)).toContain("Nothing removes onClose"); + expect(at(224)).toContain("does not name the same receiver, event, handler and capture mode"); + expect(at(244)).toContain("{ once: true }"); + expect(at(255)).toContain("{ once: true }"); + expect(at(278)).toContain("Nothing removes onError"); + expect(at(229)).toContain("yielded after onConnection was attached"); + expect(at(290)).toContain("Establish the ensure() before the subscription"); + }); + + /** + * A `finally` around the subscription, an `ensure()` armed in the same + * synchronous prefix, an `ensure()` whose listener-dependent wait is closed + * by a synchronous inner `finally`, the cleanup an `action()` returns, and + * the corrected helper itself. + */ + it("accepts the four teardown shapes and the scope-bound helper", function* () { + expect(yield* reported("event-registration-paired.ts")).toEqual([]); + }); + + /** + * The acceptance above is only worth something if the rule was looking. One + * registration per source family, each with no cleanup at all, is reported: + * constructors, factories, connections, child streams, aliases, annotated + * parameters, the `process` global and its streams, an emitter subclass and + * its `this`, a paired structural interface, and the DOM families. + */ + it("recognizes every source family it claims to", function* () { + expect(yield* reported("event-registration-sources.ts")).toEqual([ + 16, 21, 26, 31, 36, 41, 47, 51, 55, 59, 63, 68, 76, 86, 91, 96, 100, 105, + ]); + }); + + /** + * The near misses of that list: a router with an `on()` of its own, a + * `connect` from another module, an interface declaring `on` and no `off`, a + * class named `EventEmitter` that extends nothing, a local `process` and a + * local `document`, an unannotated parameter, and a binding that was + * reassigned after it was declared. `ensure` and `action` imported under + * other names still pair and still own, because both are resolved through + * their import rather than their spelling. + */ + it("accepts values whose on/once belong to somebody else", function* () { + expect(yield* reported("event-registration-bindings.ts")).toEqual([]); + }); + + it("accepts listeners installed where no Effection scope owns them", function* () { + expect(yield* reported("event-registration-unowned.ts")).toEqual([]); + }); + + it("suppresses the line a narrow directive covers, and no other", function* () { + expect(yield* reported("event-registration-suppressed.ts")).toEqual([13]); + }); + + /** + * A file-wide directive silences every subscription below it while stating + * an invariant for none of them. Nothing is reported here, which is exactly + * why the form is forbidden and why the repository is checked for it below. + */ + it("is silenced entirely by a file-wide exemption", function* () { + expect(yield* reported("event-registration-exempted.ts")).toEqual([]); + }); + + it("finds no broad exemption anywhere on the lint surface", function* () { + const exempted: string[] = []; + for (const file of yield* linted()) { + if (FILE_WIDE.test(yield* readTextFile(file))) { + exempted.push(path.relative(ROOT, file)); + } + } + + expect(exempted).toEqual([]); + }); + + it("finds no configuration entry turning the rule off for a path", function* () { + for (const config of [".oxlintrc.json", "oxlint.shared.json", ".reviews/.oxlintrc.json"]) { + const source = yield* readTextFile(path.join(ROOT, config)); + const off = new RegExp(`"local/${RULE}":\\s*(?:"off"|\\["off")`, "u").test(source); + expect([config, off]).toEqual([config, false]); + } + }); + + it("keeps every suppression narrow and explained", function* () { + const unexplained: string[] = []; + for (const file of yield* linted()) { + const lines = (yield* readTextFile(file)).split("\n"); + for (const [index, line] of lines.entries()) { + if (!NARROW.test(line)) { + continue; + } + const preceding = lines + .slice(Math.max(0, index - 8), index) + .filter((entry) => /^\s*(?:\/\/|\*|\/\*)/u.test(entry)) + .filter((entry) => !NARROW.test(entry)); + if (preceding.length === 0) { + unexplained.push(`${path.relative(ROOT, file)}:${index + 1}`); + } + } + } + + expect(unexplained).toEqual([]); + }); + + /** + * Non-vacuous on both sweeps: the surface has files to read, and the pattern + * that finds no broad exemption above does find the one the fixture carries. + */ + it("sweeps a populated lint surface with a pattern that matches", function* () { + const files = yield* linted(); + + expect(files.length).toBeGreaterThan(100); + + const exempted = yield* readTextFile( + path.join(ROOT, "scripts/tests/fixtures/event-registration-exempted.ts"), + ); + expect(FILE_WIDE.test(exempted)).toBe(true); + expect(NARROW.test(exempted)).toBe(false); + + const suppressed = yield* readTextFile( + path.join(ROOT, "scripts/tests/fixtures/event-registration-suppressed.ts"), + ); + expect(NARROW.test(suppressed)).toBe(true); + }); +}); diff --git a/scripts/validate-documentation.ts b/scripts/validate-documentation.ts new file mode 100644 index 000000000..350444bf9 --- /dev/null +++ b/scripts/validate-documentation.ts @@ -0,0 +1,61 @@ +/** + * Build the complete first-party documentation index, or refuse. + * + * The one entrypoint every distribution passes through before it is produced. + * Copying the Markdown assets is not the same as validating them: a build that + * only copied would happily ship a package whose documentation had drifted from + * the components it documents, and the first person to notice would be an + * author whose `` refused at run time. + * + * So the check is the real assembly. It enters the same bootstraps the `run` + * profile enters, collects through the same Api, and builds the same index — + * which means a missing section, an unknown heading and a component documented + * twice each fail the build for exactly the reason they would fail a run. + */ + +import { main, scoped } from "effection"; +import type { Operation } from "effection"; +import { capturedDocumentation, documentationIndexFor } from "@executablemd/core"; +import type { ComponentOrigin, DocumentationIndex } from "@executablemd/core"; +import { useRunProfileRegistry } from "../packages/cli/src/syntax.ts"; + +/** Assemble the complete index, throwing whatever it refuses with. */ +export function* validateDocumentation(): Operation { + // Inside the bootstrap scope, because a contribution belongs to the scope + // that installed it: collecting outside would find core's terminal alone and + // pass every rule vacuously. + const index = yield* scoped(function* () { + yield* useRunProfileRegistry(); + return documentationIndexFor(yield* capturedDocumentation()); + }); + // Read two entries back, one from each side of the terminal, so a build + // cannot pass by assembling an index that holds nothing: an empty set + // satisfies every rule above vacuously, and an index holding core's + // contribution alone would satisfy them for a profile that bootstrapped no + // package at all. + read(index, "Syntax", { kind: "protected", origin: "@executablemd/core" }); + read(index, "WebForm", { + kind: "registered", + origin: "@executablemd/web", + reserved: false, + }); + return 1; +} + +/** One entry the index must actually hold, read the way a document reads it. */ +function read(index: DocumentationIndex, name: string, origin: ComponentOrigin): void { + const sample = index.documentationFor(name, origin); + if (sample === undefined || sample.length === 0) { + throw new Error( + `the documentation index built without <${name}>'s documentation, so it is not the ` + + "index this product ships", + ); + } +} + +if (import.meta.main) { + await main(function* () { + const boundaries = yield* validateDocumentation(); + console.error(`component documentation: complete across ${boundaries} profile`); + }); +} diff --git a/specs/executable-mdx-spec.md b/specs/executable-mdx-spec.md index 64eca1729..35d134826 100644 --- a/specs/executable-mdx-spec.md +++ b/specs/executable-mdx-spec.md @@ -2657,16 +2657,20 @@ A component name is resolved in tiers, and the first tier that answers wins: repository file named after one never stands in for it. A structural name written where its construct gives it no meaning is a printed error, not a missing component. -2. **a host claiming the name** — a reserved registration protecting a language +2. **a component canonical core protects** — the engine's own claim rather than + a host's, so the name means the same thing in every execution. `` + (§5.3.1) is the one member. The table is the resolver's own and is consulted + unconditionally, so no option a caller passes puts anything in front of it. +3. **a host claiming the name** — a reserved registration protecting a language or security invariant, or a *declared Markdown component*: exact first-party Markdown a trusted host handed this execution. Both claim the name rather than offering a default for it, so two claims on one name are refused where they are installed and this tier never chooses between them. -3. **the workflow component bundle** this execution is closed over, when a +4. **the workflow component bundle** this execution is closed over, when a trusted host installed one. -4. **a repository-local file**, by the candidate order below. -5. **a registered default**, including the components core supplies. -6. **nothing**, which is the unresolved printed error. +5. **a repository-local file**, by the candidate order below. +6. **a registered default**, including the components core supplies. +7. **nothing**, which is the unresolved printed error. So a repository component overrides any ordinary package default, core's included, and a reserved registration overrides the repository. Only genuine @@ -2674,6 +2678,194 @@ absence falls through to a default: a candidate that exists but cannot be read, imported, parsed, or compiled fails where it is loaded, so a broken local component is never quietly replaced. +#### 5.3.1 ``, the protected symbols component + +`` outputs the components and control-flow constructs a document may +write at the site it is written at, as the Markdown `xmd syntax` prints. One +symbol construction and one Markdown renderer serve both, so an operator +printing a profile and an agent being told what to write are never given +different accounts of one environment. + +```mdx + +``` + +renders the symbols available where it is written. It is a **text component**: the ordinary +engine-owned `as` captures the same text and emits nothing. + +```mdx + +``` + +It is **self-closing only** and declares one optional prop, `names`: a non-empty +array of unique component-name strings. A paired spelling, any other prop, an +empty list, a duplicate, a non-string member and a name this site has no +component for are each refused before the symbols are read, so a refusal +produces no partial text and no successful retained result. + +**The named form renders documentation.** `` +renders each selected component's symbol metadata followed by the long-form +documentation its owning package ships, once each, **in symbol order** whatever +order they were asked for in. `as` captures the same text in either form. + +Documentation joins to metadata by component name **and owning package**. Only a +registration and a protected component come from a package; a repository file, a +bundled blob and declared Markdown are this run's, so a repository `Elicit.md` +receives none of the built-in `Elicit`'s prose. **A first-party package documents every component it supplies.** The index is +built from one `components.md` per registration boundary — core's own, its Agent +registrations, the CLI, testing, web and the repository-composition set — each +contributed by that boundary's own bootstrap from the same declarations it +registers, so a component added to a package demands documentation without +anyone maintaining a second list. What `xmd syntax NAME` and a document's own +`` each read is the one snapshot canonical execution captured +at collection, which is what makes them one index rather than two. + +A boundary that supplies a component with no section refuses the whole index — as does an unknown heading, a duplicate, and a +component documented twice for one package. A partially documented first-party +package is not a valid build: a reader cannot tell an undocumented component +from one that has nothing to say, and the product's own reference would be +silently incomplete. + +The sentence *No long-form documentation is available for this component.* is +therefore for **custom** components only, which no package governs. A Markdown +component — a repository file, a bundle member, a host's declared Markdown — +takes its long-form documentation from its own document's body, read through the +canonical frontmatter split; an empty body uses the sentence. + +Named lookup selects **component entries**. A structural construct is not a +component, so `` refuses as an unknown component while +the bare form continues to list structural syntax alongside components. + +The documentation is read from the owning package through the direct filesystem, +not through `API.Fs` or the document-facing `Files` authority. Both of those are +middleware a running document can compose around, and a document that could +answer the read would decide what the product says about itself. + +**Documentation composes with the components it describes.** A package's +bootstrap installs its registrations and its documentation together — the +registrations, and additive middleware on one stable, namespaced contextual Api +— so a host that bootstraps a package gets both from one call. Canonical core is +the middleware **terminal**: every chain ends there, so core's own components are +documented in an execution that bootstrapped no other package at all, and every +wrapper delegates before appending its own. Composition order therefore decides +how the list reads and nothing else. + +Canonical execution collects the composed contributions **once**, after the +trusted host's bootstrap and before the root import or any document code, and +snapshots the answer field by field. That captured snapshot — not a field on an +execution installation — is what `xmd syntax` and an authored `` both +consume. So middleware a running document or component installs afterwards +composes into a chain nothing reads, and two executions assembled in separate +scopes each read their own. + +**One captured contribution is four values**: its owning package, its asset +identity, its exact documentation text, and its component-name set. Equality +covers all four. Set insertion order and JavaScript object identity do not +matter, so two bootstraps that build fresh objects and list the same names in +different orders have contributed the same value. + +**A repetition of the same value adds nothing and succeeds.** One package's +declarative vocabulary is deliberately entered at more than one layer — the +repository-composition set by an ordinary run's bootstrap and again inside a +workflow attachment, because either may be the only one, and a nested run or +evaluation host the same way — and the inner scope descends from the outer, so +both wrappers sit in one chain. One bootstrap and two identical bootstraps +produce exactly the same captured documentation, and the repeated bootstrap +keeps both its registrations and one documentation value. + +**Collection classifies duplicates, and a convenience helper does not.** The +middleware namespace is public: a package may compose it directly and hand back +two value-identical contributions rather than going through the helper most +bootstraps use, and that assembly is exactly as valid. So canonical collection +snapshots the composed chain by value first, folds every completely identical +contribution to one, and only then classifies what is left. A helper may +suppress a duplicate of its own as an optimization, but only where bypassing it +produces exactly the same accepted snapshot and the same conflict behavior. + +**Two non-identical contributions overlapping one owning-package and +component-name pair refuse before root execution**, whichever middleware or +bootstrap order supplied them — no order chooses a winner, and there is no +last-write-wins. A changed asset, changed text, or a different-but-overlapping +component set each refuse. Comparing the asset alone would coalesce two +bootstraps that genuinely disagree and silently keep whichever ran first. + +**A different owner is unequal but is not by itself a conflict.** Documentation +joins by component name *and* origin, so two packages may each document a +same-spelled component and neither answers for the other's. Disjoint component +sets from one owner coexist for the same reason. Collection is where a genuine +conflict is caught, because it is the only boundary every execution passes +through: a document that writes bare `` builds no documentation index, +and one that writes no `` at all builds no reference either, so deferring +the check to named lookup would let both run to completion on an assembly nobody +validated. + +Bootstrap and execution scopes are isolated, and collection is already +execution-scoped — so "refuse only within one scope" is not a separate duplicate +policy; it only describes which contribution list either policy would inspect. + +A host-maintained list of every package's documentation, kept beside a +host-maintained list of every package's registrations, is two lists that drift; +this is one. + +**Reference and availability are separate.** The reference carries two inputs. +Bare `` reports what may **execute** at this site. The named form +selects from the **enclosing authoring symbols** and states, per entry, +`**Available in this evaluation:** yes` or `no`. At a root the two are one set +of symbols and everything selected is available; a trusted evaluation boundary that +narrows execution keeps the enclosing reference, so a nested author can be told +how a component works where they may not run one — and is told which it is. +Neither input carries definitions, import witnesses, invocation capabilities, +providers, registrations or any other execution authority. + +`xmd syntax Elicit` is the same lookup — one selection, one index, one renderer — +so the command and the component cannot describe one component two ways. The +compact `xmd syntax` and its version-2 `--json` are unchanged: documentation is +prose rather than a symbol entry, and putting it in routine output would make +every default Plan prompt unnecessarily large. + +**Canonical core owns the name.** A repository `Syntax.md`, `Syntax.ts` or +directory candidate never wins selection; an ordinary or reserved registration +under the name is refused atomically at registration; a workflow bundle member +and a host's declared Markdown under the name are each refused at admission, +before the root import; and `Component.importComponent` middleware may observe +the import, delegate it and refuse it by throwing, but cannot answer it, replace +what came back, mutate it, or hand back a definition kept from another import or +built by another loaded copy. + +**The symbols are the execution's own.** Canonical core builds them at the root from +the selection inputs that execution captured before any installation, middleware +or document code ran — its includes, the registry it started with, the identity +components and exact Markdown its host declared, and the component bundle it is +closed over — and carries them lexically on canonical core's own expansion +authority. A trusted host may state the symbols its profile describes instead, +captured on the same terms; one execution accepts one, and two are refused rather +than ordered. Nothing is built until an occurrence asks. + +Seeing a component in the symbols grants nothing. It neither registers, resolves +nor authorizes that component: what a name means is still this section's +decision, and what may run is still the execution's. + +**The symbols say where a component came from.** A protected component is reported under +the symbol origin kind `protected`, carrying the canonical core origin — never +as a reserved registration, which is a *host's* claim under a name and can be +absent, replaced or refused where this cannot. A workflow-bundle member is +reported under the kind `workflow`, carrying its canonical repository-relative +path **and** the blob's own object id, so it stays distinguishable from a +repository candidate, which is whatever that path holds now. Both kinds are +additions, so the symbols are **version 2**: a version-1 reader was promised a +closed set of origins, and neither emitting an unknown kind nor reusing a +neighbouring one would keep that promise. Nothing else about the shape changed. + +**Each occurrence reads once.** It claims the durable identity the execution +minted for it, performs one `syntax_symbols` durable read, and retains +exactly `{ symbols: string }`. On continuation that record is parsed as a closed +protocol and returned without consulting the filesystem, the registry, the +bundle, the host or the lexical reference again; a missing, additional or +mistyped member is stale input and refuses before output or binding. Two authored +occurrences are two identities and two reads, repeated reads of one +binding read nothing again, and a failed or cancelled read completes +its teardown and commits no record. + #### The run profile's repository declarations Thirteen names — `Repository`, `Worktree`, `Dir`, `Git.Switch`, `Git.Add`, @@ -2846,7 +3038,12 @@ expands no progress body at all. Its five private capabilities — ``, ``, ``, `` and `` — are the closure those exact bytes carry, and are syntax no -document may write. [The plan command](./plan-command-spec.md) is the contract. +document may write. The vocabulary the Agent is shown is not among them: the +packaged bytes write the public `` (§5.3.1), whose own +`syntax_symbols` read retains exactly `{ symbols }`, so a continuation +restores the symbols the run actually showed rather than rebuilding them, and +`` retains exactly `{ instruction }` beside it. +[The plan command](./plan-command-spec.md) is the contract. **Which Agent a Plan is written with is the host's to say, not the Component's.** The declaration carries a trusted-host capability: the agent a Plan conversation @@ -2929,7 +3126,7 @@ Its spelling is canonical and is checked where the declaration is made. Omitting it means both forms — what every registration meant before it existed — and the only arrays accepted are `["self-closing"]`, `["paired"]` and `["self-closing", "paired"]`. An empty array, a reversed pair, a repeated member -and a form no invocation has are each refused there, so a catalog can be +and a form no invocation has are each refused there, so two entries can be compared without being normalized first and a reader never has to wonder whether a different order meant something else. The identity components a host declares to an execution (§5.6) are held to the same rule, whether they are registered by @@ -3053,7 +3250,7 @@ directory — every structural construct, and the one implementation selection chooses for every other name — and returns it as a versioned value: ```typescript -interface SyntaxCatalog { +interface SyntaxSymbols { readonly version: 1; readonly categories: readonly [ { readonly kind: "structural"; readonly entries: readonly StructuralSyntaxEntry[] }, @@ -3131,7 +3328,7 @@ skipped without being read. The default includes are `["components", "."]`, and what this decides is whether describing a repository reads its `node_modules`, its `.git` and its build output before discarding every path in them. -**A partial catalog is never presented as a complete one.** A missing include +**A partial set of symbols is never presented as a complete one.** A missing include contributes nothing, exactly as it does during execution. An include that exists but cannot be enumerated fails the whole request: one that is not a directory, one that cannot be read, and one that is itself a symbolic link. A directory @@ -3170,7 +3367,7 @@ digest as its origin. Its private closure contributes nothing: those names are not syntax a document may write, so listing them would describe an environment that does not exist. -**Inspection is observation, never authority.** Building a catalog installs only +**Inspection is observation, never authority.** Building symbols installs only the declarative registration layer selection needs. It enters no execution, constructs no durable stream, installs no Files, Service, Agent or elicitation provider, starts no testing session, reserves no terminal, mints no invocation @@ -3180,15 +3377,15 @@ from that plain declaration, and its authority-bearing factory is reached only by a real execution. **`xmd syntax` is the command.** It writes deterministic Markdown by default and -the catalog above as JSON with `--json`, both from one inspection. Its only +the same symbols as JSON with `--json`, both from one inspection. Its only options are the ordered, repeatable `--include`, which defaults to `["components", "."]` and is replaced rather than extended by explicit values, and `--json`. It takes no document and no run option, because it runs nothing. An inspection failure is reported on stderr with exit status 1 and no partial -catalog on stdout. +output on stdout. -**The command has not succeeded until stdout has taken the whole catalog.** A -pipe holds far less than a catalog, so backpressure changes how long the command +**The command has not succeeded until stdout has taken every byte.** A +pipe holds far less than the whole rendering, so backpressure changes how long the command takes and never which bytes arrive: a reader that consumes slowly receives exactly what a regular-file redirect receives, in both forms. A sink that closes or refuses the write is reported on stderr with exit status 1, rather than @@ -3512,6 +3709,26 @@ A component the workflow definition is closed over records its own shape: "content": "discovered.\n" } } ``` +A component canonical core claims the name of (§5.3.1) records the fact and +nothing else: + +```json +{ "type": "import_component", "name": "Syntax" } +{ "status": "ok", "value": { "kind": "protected" } } +``` + +The record is closed on that single member because there is nothing else that +would be true to write. A protected component has no path to record, no origin +to look up, and no implementation to serialize: it is core's own declaration, +present in every execution rather than supplied by anything this one installed. +So the record says only *which tier answered*, and replay asks the running +execution for the implementation it built for that name. An execution that built +none refuses rather than resolving the name again — a replay that fell back to +the ordinary tiers would run whatever a repository, a registry or a host is +offering under that name today, which is the substitution the tier exists to +prevent. A record carrying any member beside `kind`, or a `kind` this protocol +does not define, is stale input. + One journal entry per component, whatever it resolved to. A repository entry captures both *which file was found* (path) and *what was in it* (content); a registration entry captures the origin that named it, because a function cannot @@ -3662,6 +3879,72 @@ function isTypedDefinition(value: unknown): boolean { } ``` +#### 5.3.2 ``, the protected evaluation component + +`` runs program text the document did not author. It is **public** — +any author may write it — and canonical core owns what it means, on the same +terms `` is owned: no registration, repository file, bundle member, +declared Markdown component, import-handler answer or second loaded copy +replaces it, and claiming the name is refused where the claim is made. + +**Protection settles which implementation runs, and grants nothing.** Everything +the component can reach was stated by a trusted host at the installation +boundary, as one `ExecutionInstallation.evaluation` profile captured by value +before any installation runs. An execution accepts one profile and refuses two — +even two identical ones, because a host that stated it twice has an assembly +nobody validated. An execution offered none has no evaluation at all, and says +so when the element is written rather than at startup: a document that never +writes `` is not asking for a ceiling. + +**Two disjoint input forms.** + +```mdx + + + + Read the changelog and report the version. + +``` + +Self-closing takes the program as `text`. Paired makes the content the producer, +and what it renders is the program. An element stating both is stating the +program twice and is **refused** rather than resolved by precedence. The +workflow host additionally accepts the released `source` spelling, silently; the +ordinary profile refuses it, and no profile has ever accepted `program`. + +The producer renders through an **execution-owned, one-shot projection** +delivered directly to the protected body. It bypasses the public +`Component.content()` and `Component.tryContent()` chain, so a handler cannot +decide what program ran; it narrows only the `SyntaxReference`, so the producer +keeps its own imports, declarations, bindings, providers, working directory and +error mode; and it is not published through `ActiveProjection`, so nothing else +in the execution can obtain or influence it. + +**`allow` narrows; it never grants.** It names an effect *class* — `read` or +`write` — and the class resolves to a table the host already installed. Omitting +it asks for `read`. A class the host installed nothing for is refused before the +program is read. + +**Entries name capabilities, not definitions.** A host states which operation an +admitted name runs; canonical capture reads that operation off the host's object +exactly once, binds it behind a revocation the execution owns, and closes core's +own body over the bound result. An admitted element therefore reaches those +operations and never `API.Files`, `API.Fetch` or `API.Env` — so what a fragment +may do is not a property of what the document, a repository component or +middleware installed. A directory an admitted fragment creates scopes its +content through the evaluation's own cursor rather than the contextual +environment. + +**What it answers with** is each admitted observation's own value in invocation +order, with whatever the fragment rendered kept beside them under `output`. It +declares no `returns`, so the value binds by reference under `as`. An admitted +mutation contributes nothing to it. It is deliberately not a printing boundary. + +**One occurrence is one durable decision.** A continuation restores the +admission rather than making it again, and refuses before any effect if the run +now offers different text, or states ceilings — effect classes, Workspace roots, +pinned identities, forms or requests — the admission was not granted under. + ### 5.4 The root document is a component The entry point treats the root document through the same import @@ -3819,7 +4102,7 @@ of the first chunk and the end of the last are trimmed. Matching compares Unicode code points and completes in time bounded by the product of the pattern and label sizes. -A selector must resolve to exactly one catalog entry. Zero matches and several +A selector must resolve to exactly one symbol entry. Zero matches and several matches both fail. Diagnostics report canonical encoded references, so a duplicate canonical path is reported as an ambiguity rather than resolved. @@ -4102,7 +4385,7 @@ Because the record carries the content it was taken from, the selection is verified against it rather than merely parsed: a recorded exact target must still resolve to itself in the recorded content, and a recorded failure must be exactly the failure the recorded selector produces against that content. A -catalog, a match list, or a kind that the recorded document contradicts is +a catalog, a match list, or a kind that the recorded document contradicts is therefore malformed too. Reading a record is **total**, and identification is separate from reading. An @@ -4356,7 +4639,7 @@ Matched targets: `multiple-matches` lists the matches; an invalid selector and a no-match list the whole catalog under `Available targets:`, or say `The document has no -targets.` when the catalog is empty. Every other failure keeps the printed-error +targets.` when the catalog are empty. Every other failure keeps the printed-error behavior it already had. A filename containing `#` is written `%23`, and every literal `%` is written @@ -8506,7 +8789,7 @@ escaping rule or a trailing newline. The component is **self-closing only**, and that is declared to canonical dispatch rather than decided in its body (§5.6), so what runs, what the syntax -catalog advertises and what a refusal says all come from one value. Any paired +the symbols advertise and what a refusal says all come from one value. Any paired spelling — including one whose content is empty — is refused as a printed component failure naming the `` spelling, and the content written between the tags never expands. @@ -8865,7 +9148,7 @@ and what every refusal says are Markdown; the four phases it may reach — ``, ``, `` and `` — are declared to canonical execution by an eligible compiled macOS or Linux host, appear in no `xmd run` profile, repository lookup -or public catalog, and carry no contextual authority. An installation that +or public syntax symbols, and carry no contextual authority. An installation that cannot replace itself therefore has no component to reach rather than a check it could forget. The command's complete contract is [`xmd upgrade`](./upgrade-command-spec.md). @@ -9703,6 +9986,7 @@ trusted-host events may have no authored source. | Resolve components (glob) | `glob` | `resolve:{dir}` | Only when `useDurableGlobResolver` middleware is installed | | Read over HTTP | `fetch` | `fetch:{expansion id}` | Normalized request in `description.input`; status, detached headers and text body in the result (§6.18) | | Admit generated XMD | `generated_xmd` | `generated:{fragment id}` | The canonical class selection, retained roots, selected root, every selected entry as a name, identity and admitted forms, and the exact request policy in `description.input`; the admitted source, that same policy, and the identity and form of each element the fragment named in the result (workflow-workspace-spec §8.4) | +| Read the symbols | `syntax_symbols` | `syntax_symbols:{expansion id}` | One per authored `` occurrence. The success payload is closed on exactly `{ symbols: string }` — the rendered Markdown the component returned — so a continuation restores the symbols the run actually showed without consulting the filesystem, registry, bundle, host or lexical reference again. A missing, additional or mistyped member is stale input and refuses before output or binding; a cancelled read completes teardown and commits nothing (§5.3.1) | ### 10.2 Example journal for a multi-component document @@ -10547,7 +10831,7 @@ platform's. Driven through `execute()` against real files, ordinary core selection and real expansion. A sibling that must not run records its own mark, so a stopped document is proved by what did not happen rather than by absent output. The -recovery boundaries themselves are Tier OM's and Tier RF's, the catalog row is +recovery boundaries themselves are Tier OM's and Tier RF's, the symbols row is Tier SY's, the loop's own bound is Tier LOOP's, and missing-`` settlement is Tier RV's; these rows cross-reference them rather than restating them. @@ -10650,7 +10934,7 @@ Each row names the derivation it kills. | CR32 | Registration replay | A reserved registration records its origin and replays | | CR33/CR34 | Origin mismatch | A recorded origin that is missing or replaced fails explicitly rather than invoking another component | -### Tier SY — The syntax catalog +### Tier SY — The syntax symbols Provider-neutral: the filesystem is stubbed at the contextual `API.Fs` boundary, so the include-boundary rows are the same on every host. Defined in §5.3. @@ -10664,10 +10948,10 @@ so the include-boundary rows are the same on every host. Defined in §5.3. | SY7c | Pruning | A lower-case, hidden or dotted directory is never read — at the top level or deeper — while the direct, nested and index candidates beside it stay discoverable; every skipped directory throws if it is read, and the recorded reads name only the ones a name reaches | | SY8–SY11 | Selection decides | Include order, `.md` before `.ts`, direct before index, registered fallback, and a repository override appearing once as user-provided | | SY12–SY18c | Include boundaries | An absent include contributes nothing; a non-directory root, a symbolic-link root, and a selection-relevant link to a directory or to nothing each fail the whole request; a relevant link to a file is selected; a link behind a lower-case, dotted or hidden prefix is ignored even beside one that is refused; and the diagnostic names the configured include and the logical entry rather than the resolved target | -| SY13b | An unreadable reachable directory | An include root that refuses, and a valid-name directory beneath it that refuses, each fail the whole request rather than shortening the catalog | +| SY13b | An unreadable reachable directory | An include root that refuses, and a valid-name directory beneath it that refuses, each fail the whole request rather than shortening the symbols | | SY13c | Include spellings | `.`, `./` and `.//` read the same directories and `./Ns` and `.//Ns` read the same directories, none of them absolute, and each spelling still selects exactly what it selects today | | SY13d | An absolute root | An include with two leading separators reads only beneath that exact prefix; the directory one separator away throws if it is read | -| SY19–SY22 | Markdown documentation | String `description`/`as`/`context` reach the catalog; a non-string value documents nothing; an undocumented component stays complete; a declared `returns` reports `value` mode with its schema | +| SY19–SY22 | Markdown documentation | String `description`/`as`/`context` reach the symbols; a non-string value documents nothing; an undocumented component stays complete; a declared `returns` reports `value` mode with its schema | | SY23 | Opaque TypeScript | A repository `.ts` entry is origin-only and carries no contract field | | SY24/SY25 | Complete contracts | ``'s two forms, ``'s one, ``'s one and its capture, and text and value return modes; declaration order of captures and forms survives, each canonical forms spelling is accepted, and every other one is refused | | SY26–SY27 | Declared components | One is described without its factory being called and a repository file still overrides it; two declarations of one name are refused before either factory; and a declaration inspection refuses is refused identically by registration | @@ -10733,18 +11017,108 @@ and renders that file's marker; `xmd run -#Section` executes `Section` of that same file, rendering its marker and not its sibling section's; and `xmd test -` keeps the test command's own path behavior. +### Tier SYN — The public `` component + +Named `SYN` rather than `SY` or `SL`, which already name the syntax symbols and +own-scope context updates. The symbols *value* is Tier SY's; this tier is the +component that observes one at an authored site. + +| # | Test | Verify | +|---|------|--------| +| SYN1–SYN4 | One occurrence | The bare form renders the symbols once and `as` binds the same text emitting nothing; a paired spelling and an unknown prop refuse before any read; two occurrences read independently and a reused binding reads nothing again | +| SYN5–SYN10 | The name canonical core owns | A repository `Syntax.md`, `Syntax.ts` and directory candidate never win selection, with an ordinary nearby component as the positive control that repository discovery is live; ordinary and reserved registrations are refused atomically; a workflow bundle member and a host's declared Markdown are each refused at admission before the root import | +| SYN6 | The origin selection reports | Selection answers `{ kind: "protected", origin: "@executablemd/core" }` — its own kind, not a reserved registration | +| SYN11–SYN15 | The import chain | Middleware that answers, substitutes, mutates, redirects, delegates twice or reuses another import's definition cannot run a replacement; ordinary delegation reaches canonical ``; a deliberate middleware refusal stays a refusal; document-authored context and a look-alike reference change nothing | +| SYN16–SYN18 | The site described | An ordinary run reports its own includes and registry; a workflow root reports its bundle without importing or running a member; a declared Markdown component's body reports the site it inherited | +| SYN20–SYN22 | The record kept | Continuation restores the retained symbols after the environment moves and rediscovers nothing; missing, additional and wrong-typed payloads refuse before output or binding; a cancelled read completes teardown and commits nothing | +| SYN23, SYN25b | Never authority | Symbols naming a component neither register, resolve nor authorize it; a fixed narrower reference answers with exactly the symbols it was handed and adds nothing — the seam `` installs through | +| SYN24 | One description | Inspection and validation describe the component identically, from one declaration | +| SYN26 | Another loaded copy | A protected implementation built by a second loaded copy answers for nothing in the active execution | +| SYN27 | Protected provenance | The symbols report the component under the `protected` origin kind in both the structured entry and the rendered Markdown, and never as a reserved registration; `inspectComponent` agrees | +| SYN28 | Pinned provenance | A workflow-bundle component is reported at its path *and* blob object id, under the `workflow` origin kind, and stays in the user-provided category | +| SYN29 | The named form | Selected entries render their metadata and documentation once each in symbol order, not request order, with nothing else of the symbols; `as` binds identical text | +| SYN30 | Availability and absence | Each entry states whether it is available in the current evaluation; a selected entry with no authored documentation renders its metadata and says so | +| SYN31 | Atomic refusal | An unknown name, an empty list, a duplicate, a non-string member, a non-array value and an undeclared prop each refuse with no successful retained result | +| SYN39 | Named retention | The occurrence retains its final rendered text, a continuation restores it without rereading documentation or rebuilding the symbols, and a corrupted record refuses | +| SYN25c | The narrowing seam | A narrowed reference reports the narrowed vocabulary bare, documents the enclosing symbols by name, and marks each entry's availability truthfully in both directions | +| SYN32–SYN34 | Parsing one file | Bundle prose, a section per level-two heading with deeper headings kept inside it, a fenced heading read as the example it is, and a refusal for a duplicate section or a heading that is not a component name | +| SYN35–SYN37 | Building the index | A heading naming something the package does not supply refuses; one component documented twice refuses; documentation attaches by name and owning package, never to a repository replacement | +| SYN38 | Exact coverage | A package supplying a component it does not document refuses the whole index — deleting any one built-in's section fails — with a fully covered package as the positive control | +| SYN40 | The protection boundary | A repository component wrapping the named form with planted `API.Fs` middleware cannot change what the documentation says, and the read never reaches that Api | +| SYN41–SYN45 | The build gate | `scripts/validate-documentation.ts` assembles the complete first-party index before any distribution is produced: the shipped set passes, and a deleted section, an unknown heading, a duplicated section, and drift in a package outside core each fail it | +| SYN46 | Cancelling a named read | Teardown completes and no successful `syntax_symbols` record is committed | +| SYN25g | Collection captures by value | Rewriting a contribution's source object, its text, its owner and its name set after the collector returned changes neither the snapshot nor what a reference built from it renders | +| SYN25h | One call, both halves | A profile whose package bootstrap was not entered describes the component and says it is undocumented; entering the bootstrap supplies the prose *and* keeps canonical core's own, so a wrapper that replaced rather than appended fails here | +| SYN25i | Duplicates refuse either way | Two contributions naming one component of one package refuse whichever order they were bootstrapped in, and each refusal names the pair it actually saw | +| SYN25l.1 | A value-identical repetition adds nothing | One package entered twice — fresh contribution objects, the same four values, the names listed in the opposite order — leaves a named ``, a bare `` and a document containing no `` each working, and one bootstrap captures exactly what two capture | +| SYN25l.2 | A changed value refuses, in either order | For one owner and an overlapping component, a changed asset, changed text and a different-but-overlapping component set each refuse before root execution, whichever order the bootstraps supplied them | +| SYN25l.3 | Distinct owner and disjoint set are controls | Two owners documenting a same-spelled component do not conflict and neither takes the other's prose; one owner accounting for disjoint sets from two files does not conflict either | +| SYN25l.4 | Overlapping executions are isolated | An assembly carrying a genuine conflict refuses while a second execution live at the same time renders its own documentation, unaffected | +| SYN25l.5 | Layered trusted bootstraps keep both halves | An inner trusted layer entering the same package bootstrap as the outer keeps every layer's registrations resolvable *and* the package's documentation, rendered once from one coalesced value | +| SYN25l.6 | Collection owns duplicate classification | Two direct middleware wrappers that never call the convenience helper, contributing fresh objects with the same four values and differently ordered name sets, leave the named form, the bare form and a no-`` root each working with one contribution captured; the same three disagreements reached that way still refuse before the root in either order | +| SYN25j | Scopes are isolated | A sibling scope that bootstrapped nothing reads none of the first scope's contributions, and the first scope's contribution does not outlive it | +| SYN25k | Document-time middleware reaches nothing | A component that composes around the `Documentation` Api and renders `` inside its own scope is shown what the host bootstrapped, not what it installed | +| SX17 | One index, two surfaces | `xmd syntax NAME` and `` return the same text for a component outside core's own file | + +### Tier FE — The public `` component + +Defined in §5.3.2. The rows drive the real component through a real execution +against a real captured profile; the filesystem operations are a recorder that +is never installed as a provider, so an operation appearing in its log went +through the captured capability because there is no other way to reach it. + +| ID | Evidence | +| --- | --- | +| FE1 | `text` evaluates a fragment in an ordinary run and returns `{ observations, output }`. | +| FE2 | Paired content privately renders to the same exact text and result as the `text` form. | +| FE3 | `text` plus children, `source` plus either new form, missing input, and invalid `allow` refuse before producer or fragment effects. | +| FE4 | Omitted `allow` is `read`; read permits self-closing File and refuses every write form before effects. | +| FE5 | `write` permits only the named host-profile write forms and never creates authority from text. | +| FE6 | Root frontmatter, root props, `returns`, and independent `` selection refuse before effects. | +| FE7 | With `as`, the result object is captured; without `as`, it is discarded; neither form emits fragment output. | +| FE8 | Public `` and a directly nested Plan see the enclosing Evaluate vocabulary for the selected `allow`; generated text is validated against that same vocabulary. No private Syntax implementation is involved. **Receives #758's SY19**, which #759 cannot prove: it installs no narrower syntax reference, and `` is not in the generated-XMD pinned identity table until this issue admits it. | +| FE9 | A deferred Plan's public `` sees its own ordinary vocabulary; a later narrower Evaluate rejects incompatible text before effects. **Receives #758's SY20**, for the same reason. | +| FE10 | Exact text and policy survive journal/continuation; changed text or policy refuses before effects. | +| FE11 | Completed Plan work and completed fragment effects replay without repetition. | +| FE12 | Hostile or malformed durable records fail closed before effects. | +| FE13 | Declared-component private closures and producer-private authority remain unavailable. | +| FE14 | Middleware answer A is retained by stable provider identity; continuation with B refuses before A or B runs; unchanged A resumes. | +| FE15 | Provider registrations disappear at teardown; losing or cancelled resolution claims cannot later execute. | +| FE16 | Repeated Evaluate occurrences cannot consume one another's records. | +| FE17 | Actual cancellation stops active work, waits for cleanup, and records no false success. | +| FE18 | Existing untagged #369 records remain readable; new records use the closed version-2 shape. | +| FE19 | Workflow `source` and canonical `text` have identical behavior without a warning; the ordinary profile rejects `source`, and every profile rejects `program`. | +| FE20 | Exact trusted Fetch limits are retained and narrowed; missing or changed limits refuse before requests. | +| FE21 | Ordinary and workflow hosts exercise the same fragment evaluator rather than separate semantic implementations. | +| FE22 | `xmd plan` output remains ordinary text usable at the command boundary and acquires no special complete-root type. | +| FE23 | An evaluation lacking write authority refuses the write before effects, including when the write component appears in trusted symbols. **Receives the Evaluate clause of #758's SY21**; the half that stays in #759 is that symbols text alone registers, resolves and authorizes nothing. | +| FE24 | Repository files, workflow bundle members, declared Markdown, ordinary and reserved registrations, middleware answers or mutations, document context, and another loaded package copy each fail to replace ``; honest middleware delegation reaches canonical core and a deliberate middleware refusal remains a refusal. | +| FE25 | A replacement whose body deliberately ignores `allow={["read"]}` and writes through wider ambient authority is never invoked; canonical `` refuses the write before any provider call or mutation. | +| FE26 | Protection itself grants nothing: a profile with no write table stays unable to write, and adding `` to the protected table does not add a class or component identity to `allow`. | +| FE27 | Structural constructs remain selected by structural dispatch, while repository replacements for ordinary ``, ``, and `` still win at ordinary authored sites. Their replaceability does not let them enter a generated fragment unless the trusted profile admitted that exact identity and form. | +| FE28 | Source, npm, and compiled symbols report `` with protected origin and the approved description, and no host bootstrap is needed to make the name available. | +| FE29 | Each execution accepts one private fragment-evaluation profile; a missing or duplicate profile refuses before paired-content production or fragment effects, and document-controlled state cannot read, replace, or widen it. | +| FE30 | A trusted nested-run or evaluation-host layering control enters the same declarative package bootstrap through inherited and local layers. The child keeps the package registration and renders the same named documentation as the single-bootstrap control; Evaluate still admits only the identity selected by `allow`. A non-identical owner/component overlap refuses before child root or fragment effects. This receives #765's SYN25l.5 rather than creating a second collector in Evaluate. | + +Each refusal case needs a negative control proving no producer, middleware +answer, request, file mutation, or other program effect occurred. The +implementation tiers carrying the elaborated evidence are `GX` for the durable +protocol, the ceiling table and the profile's entry rules, `CIV` for what a +provider's stated identity is bound to, and `FT` for `` itself. + ### Tier SX — The `xmd syntax` command | # | Test | Verify | |---|------|--------| | SX1–SX3 | The run profile | Core, Agent, testing and web defaults and `` are described from the declarations the runtime installers register, with no execution claimant minted | | SX2 | Documentation completeness | Every complete built-in in the profile states a description, and the testing contracts read as they are: `` requires `message` and binds the caught error segment, an ordinary assertion binds its diagnostic report, and an assertion that refuses expected children reports one form | -| SX4–SX6 | Renderers take a value | Both formats render from a supplied catalog with the filesystem refusing every call, twice with identical bytes, under the fixed category headings; every table cell is escaped, a prop name holding a pipe included | +| SX4–SX6 | Renderers take a value | Both formats render from a supplied symbols with the filesystem refusing every call, twice with identical bytes, under the fixed category headings; every table cell is escaped, a prop name holding a pipe included | | SX7/SX8 | Includes | Repeated values select in caller order and replace the defaults; absent, the defaults apply | -| SX9 | Failure | An unusable include exits 1, reports on stderr and prints no catalog | -| SX10/SX11 | Formats | Markdown by default, version-1 JSON with `--json`; the catalog is inspection, and `xmd plan` is the command that writes with the same structured value | +| SX9 | Failure | An unusable include exits 1, reports on stderr and prints no symbols | +| SX10/SX11 | Formats | Markdown by default, version-2 JSON with `--json`; the symbols are inspection, and `xmd plan` is the command that writes with the same structured value | +| SX16 | Named lookup | `xmd syntax Elicit` renders that component's metadata and long-form documentation through the same selection, index and renderer `` uses; the compact symbols are unchanged and an unknown name refuses whole | | SX12 | A package tree | Bare `xmd syntax` succeeds with the default includes in a repository whose `node_modules` holds directory links | -| SX13–SX15 | Delivery | A real pipeline reading a catalog larger than one pipe buffer receives the bytes a regular-file redirect receives, in both forms; a consumer that closes early leaves the command reporting on stderr with exit 1 rather than an unhandled write failure | +| SX13–SX15 | Delivery | A real pipeline reading symbols larger than one pipe buffer receives the bytes a regular-file redirect receives, in both forms; a consumer that closes early leaves the command reporting on stderr with exit 1 rather than an unhandled write failure | ### Tier SDL — Delivering a rendered result @@ -10787,7 +11161,7 @@ every refusal is proven by the phase tripwires that stayed at zero. | # | Test | Verify | |---|------|--------| -| PS1–PS3 | Fixed grammar | One request preserved byte for byte and a second positional refused; every retained option accepted before and after it; every `--run` spelling answered with the migration, and every other removed option — both short aliases and the aggregate and generated property names included — answered with the one refusal that names `xmd run`, before any catalog, Agent, session, review or filesystem activity, and before `--help` can short-circuit the dispatch in either order; a name that merely begins like a property option keeps the generic unknown-option refusal; a first token of `prompt` refused in preflight rather than read as a document path, with `xmd run ./prompt` still executing a document of that name | +| PS1–PS3 | Fixed grammar | One request preserved byte for byte and a second positional refused; every retained option accepted before and after it; every `--run` spelling answered with the migration, and every other removed option — both short aliases and the aggregate and generated property names included — answered with the one refusal that names `xmd run`, before any symbols, Agent, session, review or filesystem activity, and before `--help` can short-circuit the dispatch in either order; a name that merely begins like a property option keeps the generic unknown-option refusal; a first token of `prompt` refused in preflight rather than read as a document path, with `xmd run ./prompt` still executing a document of that name | | PS4/PS5 | Help | The complete `xmd plan --help` output and the program summary carry only the retained grammar, both explicit compositions and the journal warning, and no removed option appears in either; `xmd run --help` still exposes every option it configures | | C2–C3 | The packaged adapter and Component | The command executes the checked-in Markdown value root under ``, which invokes the packaged `` Component, and the turn text is that Component's own words; generation, repair, review, revision, approval, stopping, exhaustion and the final explanation are Markdown under visible headings, every Plan-producing turn states the complete Plan requirements for itself, `` stays one turn, and what a person reads says each thing once however many rounds it took | | C4–C6 | Session and ceiling | One enclosing Session carries every turn, defaults differ per invocation and `--session` supplies the exact override; the authorship profile gives the assistant an empty host-owned directory, no MCP servers, no native tools and a private strict denial no command line reaches; a draft is data throughout, and no draft effect ever happens | @@ -10811,9 +11185,9 @@ rather than restating. | PO1–PO3 | Phases and counters | Each phase precedes the work it names, and one reaches the operator while a turn is still blocked; the repair and attempt ordinals come from the loop bounds and stop at them; Stop and exhaustion announce themselves and keep their exact endings | | PO4/PO5 | Surface and disclosure | Neither ordinary `` form announces anything or expands a progress body; default progress holds no request, draft, diagnostic, feedback or approved source, and verbose adds exactly the two blocks, in phase order | | PO6/PO7 | Channels and grammar | A non-terminal stderr receives normalized Markdown and a stated terminal receives it rendered, while stdout and `--output` stay byte-identical; `--verbose` and `--journal` work on either side of the request, help carries them and the journal warning, and the short aliases, every removed spelling and a retained option that reaches this grammar written where the journal path goes all refuse before any work, while `--help` keeps its ordinary precedence | -| PO8/PO9/PO16 | The journal file | No `--journal` writes no file; one creates the path before the catalog and the first turn, parses as the existing JSONL in commit order, ends terminally and holds no program execution; an existing path and an uncreatable one each report their exact refusal and reach nothing; and an ordinary failure — where no append failed — leaves a wholly parseable file with no partial trailing record | +| PO8/PO9/PO16 | The journal file | No `--journal` writes no file; one creates the path before the symbols and the first turn, parses as the existing JSONL in commit order, ends terminally and holds no program execution; an existing path and an uncreatable one each report their exact refusal and reach nothing; and an ordinary failure — where no append failed — leaves a wholly parseable file with no partial trailing record | | PO10–PO12 | The secret and persistence boundaries | A secret in a draft or in a failed check's findings reaches neither the progress nor the file while the earlier prefix stays readable, and the same values without it are shown and recorded; a refused entry reports the exact journal-write diagnostic and preserves what committed | -| PO13–PO15 | Failure and ordering | A progress destination that fails cancels the live turn, waits for every owned teardown and delivers nothing; every existing ending keeps its order and no phase claims delivery; the packaged adapter says nothing of its own and the catalog is built once, from `` | +| PO13–PO15 | Failure and ordering | A progress destination that fails cancels the live turn, waits for every owned teardown and delivers nothing; every existing ending keeps its order and no phase claims delivery; the packaged adapter says nothing of its own and the symbols are observed once, through public ``, while continuation restores that observation without rebuilding it | ### Tier UG — The `xmd upgrade` command @@ -11100,11 +11474,11 @@ Defined in [Workflow runs](./workflow-spec.md) §9.4 and §9.6–§9.7. | WRR10/WRR10b | Outer rollback cache coherence | Failure and cancellation after an uncommitted removal and negative lookup roll back and invalidate both authoritative DOFS caches | | WRR11 | Historical file size | Every historical file entry's declared size agrees with its retained DOFS manifest during read-only recognition | -### Tier DT — Document target catalog, selectors, and projection +### Tier DT — Document target symbols, selectors, and projection | # | Test | Verify | |---|------|--------| -| DT1–DT5 | Outline | ATX and Setext headings catalog in source order; a skipped depth still nests; the outermost depth is the smallest present; a sole outermost heading is the title and several are path levels | +| DT1–DT5 | Outline | ATX and Setext headings symbols in source order; a skipped depth still nests; the outermost depth is the smallest present; a sole outermost heading is the title and several are path levels | | DT6 | Case | Matching is case-sensitive | | DT7–DT9 | Labels | Formatting, link destinations, inline code, image alt text and passive HTML tags reduce to statically rendered text; a heading rendering no text is unaddressable | | DT8 | Normalization | NFC-equivalent spellings are one label and Unicode whitespace collapses | @@ -11113,9 +11487,9 @@ Defined in [Workflow runs](./workflow-spec.md) §9.4 and §9.6–§9.7. | DT12 | Nested flow | Headings in block quotes, lists, fences, exec fences and raw HTML are not targets | | DT13 | Component children | A component child holding blank lines and `#` lines contributes no target — the regression that kills raw Remark discovery | | DT14–DT17 | Addressability | A heading overlapping component syntax or carrying an interpolation is unaddressable and blocks its subtree; escaped interpolation stays static; a computed sole title still leaves its sections addressable | -| DT18/DT19 | Empty catalog | A document with no heading addresses nothing, and a sole title is no target | +| DT18/DT19 | Empty symbols | A document with no heading addresses nothing, and a sole title is no target | | DT20–DT22 | Matching | Literal levels, embedded `*`, and `**` across zero or more levels | -| DT23 | Exactly one | Zero matches and several matches both fail, reporting matches and the catalog | +| DT23 | Exactly one | Zero matches and several matches both fail, reporting matches and the symbols | | DT24–DT26 | Selector syntax | Empty, leading/trailing slash, empty level, malformed escape, NUL and non-UTF-8 are refused; `+` is a plus | | DT27 | Termination | A wildcard-dense selector against a long label completes without exponential search | | DT28 | Wildcard whitespace | Whitespace beside a wildcard is matched; only the level's outer edges trim | @@ -11125,15 +11499,15 @@ Defined in [Workflow runs](./workflow-spec.md) §9.4 and §9.6–§9.7. | DT50/DT51 | Formatter totality | A path that cannot encode losslessly — NUL, an unpaired surrogate — is refused, and every formatted reference parses back to what it named | | DT34–DT39 | Projection | Preamble, ancestor direct content and the selected subtree are retained; siblings are absent; a non-leaf keeps its descendants; a sole title stays | | DT40–DT43 | Positions | A retained element keeps its authored offset and line, CRLF included; frontmatter, props and return mode survive; the untargeted parse still scans the whole body | -| DT44–DT47 | Inspection | The catalog is reported without selecting; a glob resolves to the exact target; an unresolvable target fails inspection; the failure's data is frozen and rebuilt | +| DT44–DT47 | Inspection | The symbols report without selecting; a glob resolves to the exact target; an unresolvable target fails inspection; the failure's data is frozen and rebuilt | | DT67–DT78 | Descriptions | The first direct static paragraph describes its section, past blank lines and comment-only HTML, kept whole with formatting reduced and whitespace collapsed; a fence, component, list, quote or child heading first leaves no description and later prose is not reached; an interpolation or an inline component omits the paragraph whole rather than as a static prefix | -| DT79/DT80 | Structured catalog | `targetInfo` carries the same targets, order and duplicates as `targets`, each with its own description, on the unselected path and beside an unchanged exact `target` on the selected one | +| DT79/DT80 | Structured symbols | `targetInfo` carries the same targets, order and duplicates as `targets`, each with its own description, on the unselected path and beside an unchanged exact `target` on the selected one | | DT52/DT53 | Recognition | A failure from a separately loaded copy, and one built here, are read on the same terms | | DT54–DT56 | Reconstruction | The result is a fresh local error, never the candidate; a mutable nested list is copied and later mutation changes nothing; a revoked Proxy cannot reach through a result already built | | DT57 | Closed data | Enumerable, non-enumerable and symbol-keyed extras are refused | | DT58 | Canonical lists | Raw spaces, tabs, no-break spaces, edge whitespace, lowercase escapes, NUL, non-string entries and sparse lists are refused — asserted against the data parser, so the derived message cannot mask the check | | DT59 | Dotted heading paths | `.` and `..` are legal heading labels, so `../../etc/passwd` and `Alpha/../Beta` are canonical heading paths, never filesystem authority; structural parsing accepts them when the rest of the failure is consistent | -| DT60 | Semantic outcome | Fields no selection could have produced — a `no-match` whose selector matches, a single-match ambiguity, a match outside the catalog, an `invalid-selector` that parses — are refused | +| DT60 | Semantic outcome | Fields no selection could have produced — a `no-match` whose selector matches, a single-match ambiguity, a match outside the symbols, an `invalid-selector` that parses — are refused | | DT61/DT62 | Closed shell | A cause, an enumerable payload, and a message that does not derive from its data are refused; no planted payload survives stringification, spreading, symbol enumeration, or a journal round trip | ### Tier TX — Targeted execution and replay @@ -11152,7 +11526,7 @@ Defined in [Workflow runs](./workflow-spec.md) §9.4 and §9.6–§9.7. | TX22/TX23 | Recorded content | An untargeted journal replays untargeted; replay projects the recorded content, not the file on disk | | TX24 | Failed selection | A journal from a selector that matched nothing never answers a later valid one | | TX25–TX27 | Failed replay | The same failing selector replays its own recorded failure with no authored effect; a different failure kind or a different selector is stale; live and replayed failures are the same structural error | -| TX28–TX32 | Malformed records | Starting from a valid failed-selection journal and corrupting only the record: a missing or non-array catalog, an unknown kind, extra record or failure data, a catalog or target the recorded document does not derive, and inconsistent kind/matches data are each refused before completed-Close reuse, resumed with the failing selector and with a valid one, expanding nothing and appending nothing | +| TX28–TX32 | Malformed records | Starting from a valid failed-selection journal and corrupting only the record: a missing or non-array symbols, an unknown kind, extra record or failure data, symbols or target the recorded document does not derive, and inconsistent kind/matches data are each refused before completed-Close reuse, resumed with the failing selector and with a valid one, expanding nothing and appending nothing | | TX33 | Not vacuous | An uncorrupted record still replays its recorded failure | | TX34–TX37 | Totality, inside the value | Recorded markdown whose frontmatter no parser accepts, an unreadable member, a record refusing key enumeration, and a value that is not a record at all each become the one fixed diagnostic — cause-free, carrying no planted value, expanding nothing and appending nothing | | DT63–DT66 | Preamble boundary | A section before the title is addressable and is not preamble; selecting a later section neither renders nor executes it; it stays independently addressable; real preamble text before the first heading is still retained | @@ -11472,6 +11846,10 @@ Defined in [Workflow workspaces](./workflow-workspace-spec.md) §§8.4 and 9. | GX18c–GX18d | Controls | An empty and a rendered-only fragment preserve output across partial replay and invent no nested durable effect | | GX22, GX22b | Progression | The run's own root addition with the added root current, and standing on another root it already retained, each resume and reach the generated component | | GX22c–GX22d | Root loss | Losing a non-selected admission root and losing the admission's selected root each refuse with the fixed diagnostic before the component runs, the live component count unchanged | +| GX14b–GX14c | One ceiling per identity | Two admitted identities whose four terms differ only in where a space falls each perform exactly their own admitted request, and neither entry's limit admits the other's: the run's ceiling table is keyed by the terms encoded rather than joined, because nothing makes a separator illegal in an origin or a key | +| FE18/GX21z | Version-1 reconciliation is stated, not inferred | The standard core admission a released build wrote — `@executablemd/core#File:read` under the untagged policy shape, with no revision in it — resumes because `pinnedFileRead()` states that exact string as the version-1 identity it succeeds; an arbitrary host string a released build retained verbatim resumes the same way against its entry's stated alias. An entry that states no alias refuses the record that named it, with the admitted read never reaching the provider; a string no entry lists refuses; and a string never reconciles to a component answer, whose arm did not exist when version-1 records were written | +| FE18/GX21x, FE18/GX21y | The closed version-2 shape | A literal version-1 refusal replays as the refusal it recorded. A version-2 identity written as a string, with no kind, an unknown kind, a missing or extra member, a non-string member, as an array or as null each refuse, as do a tagged result holding an untagged policy, a mixed or unknown version, a duplicate or empty class or form list, and a malformed request field. Each version has a positive control resuming on the unmodified record, so no row passes because a forged record never resumes at all | +| EP18–EP20 | One name, one implementation | A provider-backed name held by two entries across disjoint forms and both tables is resolved by one lookup and sealed as one shared definition and schema; a second identity for that name — within one table, across the two, or under another origin — refuses at capture naming both spellings rather than letting assembly order decide; and one name held as both a capability and a component answer refuses, because core supplying the body and the chain answering for it are different grants. In `evaluation-profile.test.ts` | ### Tier WGAC — Generated `` effects and the `` boundary @@ -11780,8 +12158,8 @@ Identifiers match `packages/core/tests/switch.test.ts` one to one. | SW58 | Partial replay | From a journal prefix without the root Close, the selected effect restores, selection is rebuilt, the run continues at the next live effect and the output is reproduced | | SW59 | Completed replay | The retained result is returned with no selector, matcher, import or block | -The shared entrypoints carry the rest: `syntax-catalog.test.ts` SY4b and SY5b -freeze both catalog entries and prove no repository file or registration can +The shared entrypoints carry the rest: `syntax-symbols.test.ts` SY4b and SY5b +freeze both symbol entries and prove no repository file or registration can supply either name; `document-validation.test.ts` Tier DV `` covers static and dynamic operands, a definite failure beside a dynamic one, the diagnostic a case shares with its switch, nested ownership, a stray case and the @@ -12047,28 +12425,33 @@ Runs against a loopback server, so what is asserted is what a real host reports. | FR11 | One transport | Every form of the call crosses `API.Fetch`, so a host refusal covers all of them | | FR12 | The contextual default | A call that names no timeout resolves `Config.timeoutFetch` | -### Tier FE — `` (§6.18) +### Tier FT — `` (§6.18) Every case substitutes a provider at `API.Fetch` and counts what it was asked to perform, separately from the binding, the rendered output, and the journal. +Named `FT` rather than `FE`, which names the public `` component. One +prefix cannot mean two components: a reader following `FE14` has to reach one +row, and a fragment's admitted identity and a Fetch body read are not the same +subject. + | # | Test | Verify | |---|------|--------| -| FE1 | Refusal before transport | Mutating and unknown methods, a lowercase `get`, a relative or non-HTTP URL, a body, an unknown prop, a missing URL, a non-string header value, and two spellings of one name each refuse with request count zero | -| FE2–FE4 | The normalized request | GET by default, HEAD verbatim, lowercase header names in lexicographic order with values untouched | -| FE5–FE9 | The bound | Accepted spellings reach the provider as milliseconds; every rejected class costs no request; an explicit prop outranks `Config.timeoutFetch`; no bound at all is an absent field | -| FE10–FE13 | The retained response | One canonical JSON value, identical in the binding and the journal; a provider that rewrites its headers afterwards changes neither; a provider that cannot enumerate them is refused | -| FE14 | Bodies | GET reads once, HEAD not at all — including when a body read would fail | -| FE15–FE17 | What a status means | A captured non-2xx is data and the document carries on; an uncaptured 2xx renders nothing; an uncaptured non-2xx records the response and stops later executable work | -| FE18–FE20 | The binding seam | `hasBinding()` answers for the invocation that asked — siblings, a nested invocation inside its caller, and two invocations live at once | -| FE21–FE24 | Failures | Transport, body read, timeout and cancellation bind nothing and commit no response; a halt tears the provider down in both phases with no late work | -| FE25–FE28 | Authority | Middleware may observe and delegate but cannot widen; a synthetic answer performs no request; eval's own `fetch` and a same-name repository component cross the same ceiling | +| FT1 | Refusal before transport | Mutating and unknown methods, a lowercase `get`, a relative or non-HTTP URL, a body, an unknown prop, a missing URL, a non-string header value, and two spellings of one name each refuse with request count zero | +| FT2–FT4 | The normalized request | GET by default, HEAD verbatim, lowercase header names in lexicographic order with values untouched | +| FT5–FT9 | The bound | Accepted spellings reach the provider as milliseconds; every rejected class costs no request; an explicit prop outranks `Config.timeoutFetch`; no bound at all is an absent field | +| FT10–FT13 | The retained response | One canonical JSON value, identical in the binding and the journal; a provider that rewrites its headers afterwards changes neither; a provider that cannot enumerate them is refused | +| FT14 | Bodies | GET reads once, HEAD not at all — including when a body read would fail | +| FT15–FT17 | What a status means | A captured non-2xx is data and the document carries on; an uncaptured 2xx renders nothing; an uncaptured non-2xx records the response and stops later executable work | +| FT18–FT20 | The binding seam | `hasBinding()` answers for the invocation that asked — siblings, a nested invocation inside its caller, and two invocations live at once | +| FT21–FT24 | Failures | Transport, body read, timeout and cancellation bind nothing and commit no response; a halt tears the provider down in both phases with no late work | +| FT25–FT28 | Authority | Middleware may observe and delegate but cannot widen; a synthetic answer performs no request; eval's own `fetch` and a same-name repository component cross the same ceiling | | GX11–GX14 | Generated admission | The pinned identity in the host's `read` table performs the exact admitted request once; a scheme, host, path, method, header or timeout mismatch performs none; admitting `` with no stated request is refused outright | -| FE29–FE31 | History | A partial replay restores the response with no second request; the event names the expansion and its source position; an interruption before the commit leaves no record and one continuation commits one | -| FE32–FE34 | The secret gate | The scanner sees URL, request headers, status, response headers and body in one event; a canary in the request or the response refuses the append, binds nothing, and stops the document | -| FE35 | Independence | Cancelling one invocation tears down only its own request | -| FEC1–FEC2 | Diagnostic retention | A run without `--journal` performs the request and writes nothing; with one, exactly one Fetch Yield holds the normalized request and the complete response | -| FEW1 | Workflow retention | A killed run holds one committed response, and a resume restores it without asking the server again | +| FT29–FT31 | History | A partial replay restores the response with no second request; the event names the expansion and its source position; an interruption before the commit leaves no record and one continuation commits one | +| FT32–FT34 | The secret gate | The scanner sees URL, request headers, status, response headers and body in one event; a canary in the request or the response refuses the append, binds nothing, and stops the document | +| FT35 | Independence | Cancelling one invocation tears down only its own request | +| FTC1–FTC2 | Diagnostic retention | A run without `--journal` performs the request and writes nothing; with one, exactly one Fetch Yield holds the normalized request and the complete response | +| FTW1 | Workflow retention | A killed run holds one committed response, and a resume restores it without asking the server again | ### Tier CIV — The identity a host's component names its work after @@ -12098,6 +12481,9 @@ Defined in §5.6, with the selection rule in §5.3. | CIV20 | Reading costs nothing | Reading the form, twice, leaves the durable identity unspent — the claim after it still succeeds | | CIV21 | Owner-kept | A repository component that imports nothing reads the canonical form from the object it received, for both authored forms; the fact appears on no definition a handler holding one can read | | CIV22 | Only the engine's own invocation enters a body | The dispatcher enters a form-specific body for a genuine, live, selected invocation of the form it answers, and refuses every other call before the body: a structural look-alike implementing `hasContent()`, a descriptor-for-descriptor clone, an object built on the prototype, an issuance canonical resolution selected nothing for, one selected for another dispatcher, and a closed one. Each look-alike's own `hasContent()` answers plausibly, which is why none of them is asked | +| CIV23 | The identity a provider states for its answer | A request states a claim on the exact object its handler returns, under the name and provider origin canonical execution fixed when it minted that request. A different object, a copy, an object edited after the claim, a competing provider installation, another key and another revision each identify nothing, and the first statement stands after every one of them. The provider states only key and revision, and a partial identity is refused rather than half-recorded. Backs FE14 | +| CIV24 | Identification is one atomic answer | The claim and core's own claim-time copy of what was claimed come back from one call, so nothing downstream reads the chain's object again: an answer whose member alternates between the claimed value and a substitution is sealed as the claimed one, with the plant proven live by the next read. At execution scale, an answer whose props schema alternates validates the fragment against the claimed contract. Backs FE14 | +| CIV25 | A request belongs to one resolution occurrence | Provider installation and occurrence authority are separate. An installation receives a registrar and installs import middleware once; every middleware invocation receives a fresh request with readonly name and position, plus `next`. The request captures the exact resolution-window object, provider-installation token, origin and name, and `claim` accepts only the answer plus key and revision. Its handler closes the request synchronously in `finally` on return, failure or cancellation, while an outer request stays live across `yield* next()` and may claim its replacement after the delegated handler returns. The enclosing resolution closes in its existing `finally`. A claim succeeds only during active execution, through an open request whose captured window is the exact current object and whose fixed name matches it, with no different statement by that provider in the window. Identification takes the expected window explicitly and accepts only that exact window and name. Thus a stale request from resolution N refuses during N+1 of the same name, a stale `Open` request cannot retag while `Other` is live, and a losing inner request refuses after its handler returns while the still-live outer request may claim after delegation. One installation answers several names and repeated same-name resolutions through distinct requests; unidentified replacements remain ordinary valid middleware answers outside fragment evaluation. Two owners live at once each answer only for what they recorded, and tearing one down leaves the other working. No Context, shared symbol, public brand or module-global registry participates. Backs FE15 | ### Tier NEX — Nested document executions (`specs/testing-spec.md`) @@ -12122,7 +12508,7 @@ timed. The behavioral rows run the real `xmd run` command against a document on disk, so what they observe is what a reader of that command sees. VB5 is a TypeScript -row: it inspects the catalog without running anything. +row: it inspects the symbols without running anything. | Criterion | Evidence | | --- | --- | @@ -12165,8 +12551,8 @@ user's own `~/.xmd/repositories`. | # | Test | Verify | |---|------|--------| -| ORC1 | One declaration surface | All thirteen names appear in the catalog with complete contracts; a repository file of one of those names shadows the default; catalog construction performs no ambient discovery, lock, Git, credential or network operation | -| ORC2 | Runtime declaration parity | The same catalog assertion holds under Deno, Node and Bun; on a runtime that installs no operational repository provider, representative Repository, Worktree, Git, Issue and PullRequest forms each report an absent provider with zero mutation, while `` remains operational through that runtime's host `API.Files` provider | +| ORC1 | One declaration surface | All thirteen names appear in the symbols with complete contracts; a repository file of one of those names shadows the default; symbol construction performs no ambient discovery, lock, Git, credential or network operation | +| ORC2 | Runtime declaration parity | The same symbols assertion holds under Deno, Node and Bun; on a runtime that installs no operational repository provider, representative Repository, Worktree, Git, Issue and PullRequest forms each report an absent provider with zero mutation, while `` remains operational through that runtime's host `API.Files` provider | | ORC3 | Ambient primary checkout | From a normal repository, root Switch/Add/Commit select the ambient Repository and the contextual checkout; outside Git, a root Worktree, Git operation or PullRequest refusal names how to run inside one | | ORC4 | Ambient linked worktree | Invoked from a linked worktree, Repository identity follows the canonical common directory, Git acts on that worktree's root, and the primary checkout is untouched | | ORC5 | Origin is not local authority | A repository with no `origin` creates a Worktree and performs local Git; Push and PullRequest refuse before a credential, session or transport exists | diff --git a/specs/plan-command-spec.md b/specs/plan-command-spec.md index 2f380cc54..6f9b00c9d 100644 --- a/specs/plan-command-spec.md +++ b/specs/plan-command-spec.md @@ -87,7 +87,7 @@ fixed command preflight -> --journal: exclusively create the named path -> execute the exact packaged plan command document, which is an adapter -> , the packaged Component, with the request as its Prompt - -> announce Preparing, then build the run-profile syntax catalog + -> announce Preparing, then build the run-profile syntax symbols -> the authorship frame, and one Session inside it -> generate, check, repair, review, revise, approve, explain or fail, announcing each phase on stderr before it happens @@ -100,7 +100,7 @@ fixed command preflight ``` Each phase hands the next one a value. No phase after the first failure begins, -so a refused command line reaches no catalog, a failed turn reaches no review, +so a refused command line reaches no symbols, a failed turn reaches no review, and a review that stopped reaches no stdout and no file. Writing a Plan is a conversation, and a conversation is not a run. The durable @@ -143,7 +143,7 @@ result. | Option | What it configures | | --- | --- | -| `--include …` | the ordered component search path the syntax catalog and the structural checks resolve through | +| `--include …` | the ordered component search path the syntax symbols and the structural checks resolve through | | `--agent-provider ` | which provider writes the Plan | | `--default-agent ` | which agent that provider defaults to, overriding `DEFAULT_AGENT_NAME` | | `--session ` | the logical assistant session the planning conversation belongs to | @@ -258,12 +258,12 @@ and the refusal says where an approved Plan goes now. `-e`/`--eval` stays exclusive to `xmd run`. A plan supplies a request, not a document. Supplying one anyway is refused in the command's own preflight, with `unrecognized option for xmd plan: --eval — inline documents are exclusive to -xmd run`, before the catalog, the command document, the review or the file +xmd run`, before the symbols, the command document, the review or the file exists. `--agent-provider` and `--default-agent` are resolved into one authorship configuration once per invocation, and an unknown provider fails there, before -the catalog is built. No permission mode is settled: this command starts no +the symbols are built. No permission mode is settled: this command starts no program, and the ceiling its authorship runs under is the host's rather than the command line's. @@ -288,7 +288,7 @@ The message answers both readings, because the token is ambiguous by construction. Nothing else changes: only the exact first token is recognized, so `xmd run ./prompt`, `xmd run prompt` and `xmd ./prompt` still execute a document that is legitimately called that. The refusal exits nonzero and establishes -nothing — no catalog, no Agent, no Session, no authorship directory and no +nothing — no symbols, no Agent, no Session, no authorship directory and no output. ### `--session ` @@ -300,7 +300,7 @@ never falls back to the generated one by accident. Ordinary provider session continuation applies when the configured provider already holds that name. The plan command document still supplies the current -request and the current catalog in this invocation's initial turn. A continued +request and the current symbols in this invocation's initial turn. A continued conversation still produces source and starts no program. ### Help @@ -310,7 +310,7 @@ xmd plan --help ``` Help needs no request. It describes the request, `--output`, `--session`, -`--verbose`, `--journal`, the authorship and catalog options and the deadline; +`--verbose`, `--journal`, the authorship and symbol options and the deadline; it states that the approved Plan is the only result, that stdout carries its exact bytes when `--output` is absent, and that planning never runs the approved program — and it writes out both explicit compositions. No removed option @@ -322,7 +322,7 @@ It ends with what a journal costs, separated from everything above it: Secret detection checks journal entries before they are recorded, but it may not catch every sensitive detail. The journal can contain prompts, drafts, and review answers. ``` -Help reads no catalog, contacts no provider, places no session, asks nobody +Help reads no symbols, contacts no provider, places no session, asks nobody anything, creates no file and runs nothing. ## The packaged plan command document @@ -341,10 +341,18 @@ The host supplies two fixed internal inputs as that root's props: - `session` — the resolved logical assistant-session name. They are the adapter's own, and nothing a Plan declares is bound here: the -properties a Plan's root declares are resolved by whoever runs it. The catalog -is not among them: it is built inside ``, from a closure the host -captured, so an authored phase can say that the preparation is starting before -it happens. +properties a Plan's root declares are resolved by whoever runs it. The symbols +are not among them either. The command states the vocabulary its profile +describes at the execution boundary, captured before any installed code runs, +and the packaged Component reaches them by writing the public `` any +document may write — so an authored phase can say that preparation is starting +before the read happens, and the symbols the Agent is shown are the ones an +operator can print. + +The profile it states is the ordinary `run` one, in the caller's includes. A +Plan is a program a later `xmd run` executes, and this authorship execution +searches no repository and refuses almost every capability, so symbols derived +from it would describe a vocabulary the approved program would not have. **The root is an adapter, not the workflow.** Its whole body is two elements: it projects `props.request` into `` without adding whitespace, supplies @@ -371,7 +379,7 @@ placements stay distinct even when their authored names match. The host owns the provider instruction layer and the Agent ceiling. The Markdown owns the text of each generation, repair and revision request: the initial prompt -preserves the Prompt, includes the host's catalog, and asks for one complete +preserves the Prompt, includes the host's symbols, and asks for one complete replacement root as source only — written as a Plan, with every requested outcome kept as reader-facing prose and each component placed immediately after the sentences describing the action it performs. That authorship rule is repeated in @@ -388,20 +396,30 @@ surfaces' endings, each written once. The command's wording is unchanged; the component's says that no Plan was returned rather than that nothing was output or run. TypeScript supplies neither the words nor the choice between them. -**The five private capabilities.** The Component's phases are components only these -exact bytes may write, declared by the host with the definition and revoked with -the execution: `` builds and freezes the catalog, the instruction -identity, the -session placement, the surface and whether that placement outlives the -invocation, and refuses a continuation whose instructions render differently — -as stale input, before a directory, a provider, a turn or a review exists; paired -`` installs the constrained frame and does not return until every -part of it has torn down; paired `` says which phase is running; -`` answers about one draft without -executing it; and `` structurally admits the approved bytes after that -teardown and retains them as one Plan artifact — the invocation identity, the -instruction identity, the approved source, its digest and that successful -admission — before the Component renders them. +**The five private capabilities.** They are components only these exact bytes may +write, declared by the host with the definition and revoked with the execution. +`` freezes the instruction identity, session placement, surface and +whether that placement outlives the invocation, and refuses a continuation whose +instructions render differently — as stale input, before a directory, a provider, +a turn or a review exists. Paired `` installs the constrained +frame and does not return until every part of it has torn down; paired +`` says which phase is running; `` answers about one +draft without executing it; and `` structurally admits the approved +bytes after that teardown and retains them as one Plan artifact — the invocation +identity, the instruction identity, the approved source, its digest and that +successful admission — before the Component renders them. + +**The symbols are not one of them.** What a document may write is a public +question with a public answer, and canonical core owns both, so `Plan.md` writes +the same `` any document writes and binds the vocabulary +directly into every authorship prompt. Its retention is core's: one +`syntax_symbols` read per occurrence, retaining exactly +`{ symbols: string }`, hostile-parsed on continuation so a resumed authorship is +shown the vocabulary the run actually showed it rather than one rebuilt from a +tree that has moved. `` retains exactly `{ instruction }` beside it, +so the symbols and the question are two records that can be read and reconciled +independently, and a missing, additional or mistyped member in either refuses +before authorship begins. Whether the placement is durable is carried across that boundary rather than re-derived, because `` is the last thing that sees the public @@ -573,8 +591,8 @@ the complete versioned `DocumentValidation` core produced. The component performs no candidate execution. It asks the invocation's one structural check — `validateDocumentStructure()` under the ordinary run-profile registry, the `` identity, the caller's ordered includes and the run -profile's declarations, which include `` itself because the catalog the -agent was shown says the profile has it. The admission that follows teardown and +profile's declarations, which include `` itself because the symbols the +agent was shown say the profile has it. The admission that follows teardown and the command's own gate ask that same check, so the three cannot come to differ about what a program is for a reason nobody chose. @@ -643,7 +661,7 @@ budget. **The explanation turn.** A tenth draft that still has problems after its repairs leaves nothing to approve and nothing left to revise into, so no review opens for it: there is no decision to offer. The workflow instead makes exactly -one more `` in the same enclosing Session, automatically. The Session already holds the original Prompt, the catalog, +one more `` in the same enclosing Session, automatically. The Session already holds the original Prompt, the symbols, every draft, every earlier diagnostic and every revision request, so nothing is resent: the turn carries only the final diagnostics, which were produced after the agent's last draft and have not appeared in the conversation. It asks for a @@ -694,7 +712,7 @@ is happening rather than an account of what already finished. The phases are: | Phase | Announced before | | --- | --- | -| Preparing the Plan | the syntax catalog is built and the session is set up | +| Preparing the Plan | the syntax symbols are built and the session is set up | | Drafting the Plan | the first Agent turn, naming which of the ten attempts this is | | Checking the draft | every structural check, including the ones after a repair | | Repairing the draft | each repair turn, naming which of the three repairs this is | @@ -832,7 +850,7 @@ Help says plainly that the gate may not catch every sensitive detail and that the journal can contain prompts, drafts and review answers. -An existing path is refused before catalog preparation, session placement, Agent +An existing path is refused before symbol preparation, session placement, Agent startup, review or artifact creation, and is left byte-identical: ```text @@ -909,7 +927,7 @@ writes. ## Timeouts -`--timeout` bounds the whole command: preflight, catalog construction, the +`--timeout` bounds the whole command: preflight, symbol construction, the command document's execution, Elicitation, its teardown, the structural validation and the artifact. It covers no later program, because this command starts none. Expiry is Effection cancellation, so structured teardown completes @@ -928,9 +946,9 @@ ending of this command does. | Failure | Reaches | | --- | --- | | a malformed command line, a removed option, an unknown option, or `--save` | nothing | -| a `--journal` path that exists, or one this command cannot create | no catalog, session, turn, review, stdout or file | +| a `--journal` path that exists, or one this command cannot create | no symbols, session, turn, review, stdout or file | | an unknown `--agent-provider` | nothing | -| a catalog an include makes unreadable | no turn, review, stdout or file | +| symbols an include makes unreadable | no turn, review, stdout or file | | a `--journal` entry the file will not take | no stdout or file; the committed prefix stays | | a progress destination that stops accepting bytes | no stdout or file; accepted bytes stay | | a host that supplies no Agent context, or a provider that cannot establish the authorship profile's ceiling | no session, no turn | @@ -963,7 +981,7 @@ neither observation never interpreted what it wrote. | # | Criterion | Required observation | | --- | --- | --- | | PS1 | Fixed grammar | Every retained option is accepted before and after the request, one request is preserved byte for byte, and a second positional is refused with the approved sentence | -| PS2 | The removed switch | Bare, valued, repeated, before-request, after-request and value-position `--run` forms all return the exact migration text, before any catalog, Agent, session, review, filesystem or document activity | +| PS2 | The removed switch | Bare, valued, repeated, before-request, after-request and value-position `--run` forms all return the exact migration text, before any symbols, Agent, session, review, filesystem or document activity | | PS3 | The removed options | One representative of every other removed class, both short aliases, and the aggregate and generated property names return the exact generic refusal before authorship, ahead of the shared timeout and secret-detection grammar checks; a name that merely begins like a property option keeps the generic unknown-option refusal | | PS4 | Help | The complete `xmd plan --help` output and the program summary contain only the retained grammar and both explicit compositions; no removed option appears anywhere in either. Help beside every retained option is still help; help beside a removed one, in either order, is that option's refusal | | PS5 | Run is unchanged | `xmd run --help` still exposes its execution, prop, permission, timeout, presentation, journal and secret-detection options | @@ -975,19 +993,19 @@ neither observation never interpreted what it wrote. | PS11 | Adapter and Component | The command document remains the exact thin adapter, and `` remains a bare-or-captured exact text component | | PS12 | Product copy | Architecture, specifications, README and the homepage state that Plan produces source, Run executes source, and composition decides when it runs | | C2–C5, C8, C9, C13, C14 | Authorship | The packaged adapter and Component, one Session, the profile ceiling, the repair and review bounds, safe presentation, the authored endings, directory lifetime and narrative preservation are unchanged by this command producing source only, and keep their evidence | -| PO1 | Progress precedes the work | Preparing arrives before the catalog is built, Drafting before the first turn, Checking before validation, Waiting before review and Finalizing before authorship teardown; an early phase reaches the operator while a turn is still blocked | +| PO1 | Progress precedes the work | Preparing arrives before the symbols are built, Drafting before the first turn, Checking before validation, Waiting before review and Finalizing before authorship teardown; an early phase reaches the operator while a turn is still blocked | | PO2 | Counters come from the bounds | One invalid attempt uses repair ordinals 1st–3rd with a check before each result; a requested change announces the 2nd attempt; the counters reach the 10th and there is no 11th | | PO3 | Terminal phases | Stop announces itself before teardown and keeps its exact final diagnostic; a tenth-attempt exhaustion announces itself before the automatic explanation, opens no review and produces no Plan | | PO4 | The ordinary surface is silent | A bare `` emits only exact approved source, a captured `` binds the same bytes and emits nothing, and neither expands a progress body whatever verbosity the declaration carries | | PO5 | Disclosure | Default progress excludes the request, the drafts, the diagnostics, the feedback and every Agent, provider and tool output; verbose adds every cleared draft and each invalid check's exact structured JSON, in phase order, and nothing else | | PO6 | Channels | A non-terminal stderr receives normalized Markdown, a stated terminal receives it rendered, and stdout and `--output` stay byte-identical exact source in both | -| PO7 | The two options | `--verbose` and `--journal` are accepted on either side of the request, help contains them and the journal warning, and `-V`, `-j`, `--trace` and every removed spelling refuse before the catalog or a session exists; a retained option that reaches this command's grammar, written where the journal path goes, is that option rather than a filename and refuses before any catalog, session, provider, filesystem or artifact work — while `--help` and `-h` keep their ordinary precedence and answer with help, creating no journal and beginning no authorship | -| PO8 | The journal file | With no `--journal` no file appears; with one, the path exists before the catalog and the first turn, a successful trace parses as the existing JSONL events in commit order and ends terminally, and it holds no program-execution event | -| PO9 | Journal refusals | A pre-existing journal is byte-identical and refuses with the exact copy before any catalog, session, turn, review or artifact work; a path that cannot be created reports the other exact copy | +| PO7 | The two options | `--verbose` and `--journal` are accepted on either side of the request, help contains them and the journal warning, and `-V`, `-j`, `--trace` and every removed spelling refuse before the symbols or a session exists; a retained option that reaches this command's grammar, written where the journal path goes, is that option rather than a filename and refuses before any symbols, session, provider, filesystem or artifact work — while `--help` and `-h` keep their ordinary precedence and answer with help, creating no journal and beginning no authorship | +| PO8 | The journal file | With no `--journal` no file appears; with one, the path exists before the symbols and the first turn, a successful trace parses as the existing JSONL events in commit order and ends terminally, and it holds no program-execution event | +| PO9 | Journal refusals | A pre-existing journal is byte-identical and refuses with the exact copy before any symbols, session, turn, review or artifact work; a path that cannot be created reports the other exact copy | | PO10 | A secret in a draft | It reaches neither the progress nor the journal, the earlier prefix stays readable, teardown completes, and no source or artifact is delivered — while the same draft without it is displayed and recorded | | PO11 | A secret in a diagnostic | The same, for a failed check's structured findings | | PO12 | A refused entry | An append failure after a committed entry reports the exact journal-write diagnostic, preserves the records committed before it, completes teardown and delivers no Plan | | PO16 | An ordinary failure | A journal-backed invocation that fails for its own reason — a failed turn, with neither a secret rejection nor a write failure — exits non-zero, delivers no source and no artifact, completes teardown, and leaves a file whose every entry parses and whose bytes are exactly those entries re-serialized: no append failed, so there is no partial or unterminated trailing record | | PO13 | A failed destination | A consumer that fails while a turn is live cancels that turn, waits for every owned teardown, attempts no artifact sink, keeps the bytes stderr accepted, and uses the exact progress-failure diagnostic | | PO14 | Ordering is unchanged | Cancellation, teardown failure, final validation refusal, the `--output` refusal and a successful delivery all keep their order, and no phase claims an artifact was delivered | -| PO15 | The adapter and the catalog | The packaged adapter emits no prose of its own, and the catalog is built exactly once, from ``, after Preparing | +| PO15 | The adapter and the symbols | The packaged adapter emits no prose of its own, and the symbols are observed exactly once, through public ``, after Preparing; continuation restores that observation without rebuilding it | diff --git a/specs/workflow-workspace-spec.md b/specs/workflow-workspace-spec.md index 349c66f28..23fc01262 100644 --- a/specs/workflow-workspace-spec.md +++ b/specs/workflow-workspace-spec.md @@ -1994,15 +1994,22 @@ about it is rewritten on the way to the document. Turning it into text is the document's decision, made where the text is wanted: the representative flow binds it and renders it into the next `` with ``. -**How it is supplied.** `` is not registered by the attachment. It -names durable work after its own invocation, so the host **declares** it to the -execution — one entry in the installation, captured before anything else is -installed — and canonical execution calls the host's factory once for this -attachment with the claimant it minted, then registers what comes back -(executable-mdx-spec §5.6). The claimant is the argument of that one call: the -implementation closes over it, and it is published nowhere, so a document, a -component and middleware reach the implementation and never the claimant. A run -whose host declares none has no `` at all. +**How it is supplied.** `` is canonical core's own protected +component, and this host supplies none of it. Core claims the name ahead of +every host and author tier, so it means the same thing in every execution +(executable-mdx-spec §5.3.2). What the attachment supplies is the *ceiling*: +one `ExecutionInstallation.evaluation` profile, captured by value before +anything else is installed. An execution accepts one profile and refuses two, +and a run whose host states none has no evaluation at all — which `` +says when it is written rather than at startup. + +Each entry in that profile names a **capability** rather than a definition. +Canonical capture reads the host's own operation off its object exactly once, +binds it behind a revocation the execution owns, and closes core's own body over +the bound result. An admitted `` therefore reaches the `readTextFile` +this attachment handed over — still the run's transaction-bound one — and never +resolves `API.Files` while it runs. That is what keeps a fragment's authority +out of reach of the document, a repository component and middleware. **How its durable operation is named.** The implementation claims on the exact invocation the engine handed it, in that invocation's own frame. A durable name @@ -2039,14 +2046,15 @@ and takes the read-only `File` identity from core. No prop, binding, Context, contextual API answer, component registration, generated name or middleware return value supplies or widens any of them. -A repository component named `Evaluate` shadows this default exactly as it -shadows any other. That can change what a trusted document does; it cannot -recover the captured closure, and it cannot name durable work — canonical -resolution selected it rather than the implementation this execution built, so -an invocation of it is in no domain. Agent-generated source never resolves through the -registration at all — the evaluator consults only its own closed table of pinned -identities — so a fragment naming `` is refused like any other -unadmitted component, live registration or not. +A repository component named `Evaluate` does not shadow this one, and neither +does a registration, a bundle member, a declared Markdown component or a second +loaded copy: the name is canonical core's, and claiming it is refused where the +claim is made. `Component.importComponent` middleware composes around the import +exactly as it composes around any other — it may observe it, delegate it and +refuse it by throwing — and what it cannot do is answer one. Agent-generated +source resolves through none of that: the evaluator consults only its own closed +table of pinned identities, so a fragment naming `` is refused like any +other unadmitted component. #### Mutation-proposal admission @@ -3440,7 +3448,7 @@ fetch operation requires its own language and durability contract. | workflow Agent isolation | built by #302: no directory attachment, an empty host-owned working directory, no MCP servers, an empty requested tool set and deny-all with a failing permission path; the portable no-tool proof is tracked by #496 | | workflow Agent session retention | built by #302: a row in the run's own database, keyed by the engine-derived Session expansion identity alone — the authored name is descriptive — with provider, agent command and policy fingerprint beside it as compatibility attributes. A `` places one and creates nothing; the first subscribed Prompt constructs it, and the mapping commits after the backend accepted that turn and the provider made its canonical tagged assertion, before anything the turn produced is exposed. Occupancy of a provider key is never identity — including a record held for a first turn nobody accepted — and missing, mismatched, replaced or ambiguous assertions each refuse instead of starting a replacement session | | generated-XMD admission | built by #369, through `@executablemd/core/host`; the workflow policy wrapper is internal. Host policy is a read table and a write table of exact pinned identities, each entry carrying the authored forms it is admitted for, and an authored `allow` selects a canonical subset of the closed classes `read` and `write` — omitted means `read`. The complete fragment is preflighted inside one `generated_xmd` effect before its first generated effect | -| `` and the authored loop | built by #302 and #369: a workflow-host component with a closed schema of one required `source` and one optional `allow`, declared to the execution rather than registered by the attachment — canonical execution calls the host's factory with the claimant it minted and registers what comes back, which provides availability only. Its ceilings come from the run's own storage, core's pinned `` read and write identities and the versioned paired `@executablemd/workflow/composition/dir-v2#Dir`; iteration, branching, approval and exhaustion are ordinary Markdown | +| `` and the authored loop | built by #302 and #369 and made canonical by #713: core's own protected component, with a closed schema of `text`, this host's released `source` alias and an optional `allow`. The attachment supplies no component — it supplies one captured evaluation profile whose entries name capabilities rather than definitions, so an admitted element reaches the run's transaction-bound operations directly. Ceilings come from the run's own storage, core's `` read, write and delete identities at revision 2 and the paired `@executablemd/workflow/composition#Dir` at revision 3; iteration, branching, approval and exhaustion are ordinary Markdown | | generated-XMD mutation-proposal admission | built by #369 and #567, amended by #643: the standard Deno profile's write table is core's paired `File:write`, the paired versioned `@executablemd/workflow/composition/dir-v2#Dir` and core's self-closing `File.Delete`, in that retained order and followed by any host extension. `allow={["write"]}` intentionally authorizes the current Dir's persistent recursive creation; the former unversioned identity never authorizes it and a retained table naming that identity refuses before generated execution. Admitted mutations run as the ordinary components they are through the run's effect transactions, directory ensure and deletion each publishing the same `workspace_file` effect their authored forms do; the evaluator adds no receipt or result entry, so a write-only fragment still binds `{ observations: [], output: "" }`; and approval is authored control flow before the element. Local Git, Git-host, issue, process, execution, credential and external-write effects are outside the class | | Deno-local DOFS persistence | POC proven by #349 / PR #350 | | scoped Deno Worker Shell | containment proven by #351 / PR #353 and transactions by #357 / PR #362; production integration unbuilt |