Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -134,7 +134,7 @@ Background maintenance (V2: per-task cron scheduling). A process-wide 15-min tim
Both types of drops deduplicate with age-based reclaims and freeze their drop mode (`full`, `truncated`, or `edit_marker`) in the `tags.drop_mode` column for deterministic replay on later passes. Truncated-mode tool shells replace all original argument keys with an inert marker `{ dropped: "[dropped §N§]" }` across OpenCode, Pi, and Rust; tool dispatch verifies arguments via `assertExecutableToolInput` (`src/hooks/magic-context/dropped-input-guard.ts` / `packages/pi-plugin/src/dropped-input-guard-pi.ts`), throwing an actionable recovery error (`ctx_expand` original arguments) if a dropped placeholder is submitted. Two-pass age reclaim advances `tool_reclaim_watermark` only on an actual application opportunity, freezing during plain execute residency. Open tool arcs (pending/running tool calls without completed outputs or errored status) are excluded from reclaim, drop, truncate, or `edit_marker` selection via `partHasCompletedResult` (`src/hooks/magic-context/tool-drop-target.ts`), reclaiming closed arcs (completed string outputs or errored tool parts with `status === "error"`) while clamping deep clones of message parts so OpenCode's live in-memory execution objects stay byte-identical.
- **Tiered emergency drop and reclaim episodes (derived force band: ≥85% at the default threshold, up to 92% at raised thresholds)**: target-headroom eviction down to `fixedFloor + 0.30 × (ceiling − fixedFloor)`, tools oldest-first across tiers (T3 misc → T2 edit/search → T1 navigation), newest-20% recency reserve on T1/T2. `floorTags` (full active set, for floor accounting) vs `tags` (droppable candidates). Reclaim uses a continuous pressure-episode contract across OpenCode and Pi: `last_emergency_input_sample` acts as an episode latch admitting one non-empty emergency batch per force-pressure episode; zero-removal evaluations leave it armed. Fresh usage readings within the episode do not release the latch; dropping below the force band rearms it for future pressure, and independent provider-visible mutations rearm it so candidates ride already-priced busts. Heuristic cleanup, caveman compression, duplicate cleanup, and reasoning clearing adhere to the same episode discipline, admitting one originating application per continuous primary execute episode. Below 95%, the newest-20 dropped tool calls keep a `[dropped §N§]` skeleton (the `tool_use` survives, output replaced); older drops are fully removed. At provider-proven or estimated ≥95% pressure, TypeScript emergency selection yields both `protected_tags` and the T1/T2 recency reserve, fully removing selected completed arcs while retaining open arcs and the newest K=3 `ctx_reduce` exemplars; reasoning-adjacent tool arcs retain a paired skeleton (`requiresToolArcSkeleton` in `heuristic-cleanup.ts` and `packages/pi-plugin/src/heuristic-cleanup-pi.ts`) to prevent Anthropic from merging signed assistant turns. The target and non-empty-batch episode discipline do not change. At force pressure, a head cap landing inside the first completed arc admits the whole atomic component if it fits before the protected tail, rather than fencing back to an empty head.
- **Compaction markers**: inject an OpenCode-compatible compaction boundary so `filterCompacted` stops at the historian's last compartment, shrinking the transform-input array. The marker move is **deferred** from historian publish into the next materializing pass (one bust covers both the `<session-history>` rebuild and the boundary advance); CAS-guarded, restart-safe.
- **Content stripping** (`strip-content.ts`, `caveman.ts`, `sentinel.ts`): stateless strip functions + deterministic in-place sentinel replacement + persisted watermarks. Provider-aware: empty-content sentinels only stay empty for providers that accept them (`modelAcceptsEmptyContent`); others get a `[dropped]` placeholder (e.g. Copilot/Bedrock break tool adjacency on empty parts — #135). Merged-assistant reasoning neutralization tracks applied message IDs via `merged_reasoning_stripped_ids` in `session_meta` (with CoreState units twin in Rust), freezing the applied set on cache-busting passes and replaying on defer passes while keeping whitespace-only text parts sentinel-invisible, with frozen sentinel IDs surviving temporary transform absence across compaction marker advances until `message.deleted`. For Anthropic Fable 5.1 thinking-prefix binding mismatches (classified in `overflow-detection.ts`), the recovery arm strips bound reasoning from the targeted assistant (`stripReasoningFromAssistantIds` in `strip-content.ts`), recording the frozen ID (`binding_mismatch:<id>`) in `merged_reasoning_stripped_ids` for replay stability before clearing the armed target upon live turn completion. Trailing assistant blank decisions (`TrailingBlankDecision`) freeze while a message is newest (on every pass, capturing the last-live shape), becoming immutable once historical (`findTrailingBlankDecisionCandidates` / `applyFrozenTrailingBlankDecisions`), normalizing late-visible empty text blocks on non-newest assistant messages across race directions and cloning message part arrays before length-changing splices to avoid mutating live execution object graphs. Trailing blank classifications snapshot source shapes before sentinel insertion so `keep` decisions never manufacture absent suffixes; poisoned keeps demote to `strip` via CAS on cache-busting passes. The transform wrapper (`src/plugin/messages-transform.ts`) preserves user-terminated prompt tails (`preserveUserTerminatedTail`) when OpenCode concurrently appends a pending assistant shell mid-transform, re-anchoring the user message at the wire tail so the shell cannot turn into an illegal assistant prefill.
- **Content stripping** (`strip-content.ts`, `caveman.ts`, `sentinel.ts`): stateless strip functions + deterministic in-place sentinel replacement + persisted watermarks. Adapter-aware: empty-content sentinels only stay empty for models whose wire accepts them (`modelAcceptsEmptyContent`, resolved once per pass from `api.npm === "@ai-sdk/anthropic"` via `anthropic-wire.ts` so a custom provider fronting Claude is covered); others get a `[dropped]` placeholder (e.g. Copilot/Bedrock break tool adjacency on empty parts — #135). Every consumer takes that one resolved boolean, so no two phases of a pass can disagree. An unresolved model or registry carries the session's last materialized answer forward rather than narrowing, and any real change is priced through the m[0] upgrade identity so it folds exactly once; Rust mode stays canonical-only (`docs/AUDIT-KNOWN-ISSUES.md` A55 has the full state table). Merged-assistant reasoning neutralization tracks applied message IDs via `merged_reasoning_stripped_ids` in `session_meta` (with CoreState units twin in Rust), freezing the applied set on cache-busting passes and replaying on defer passes while keeping whitespace-only text parts sentinel-invisible, with frozen sentinel IDs surviving temporary transform absence across compaction marker advances until `message.deleted`. For Anthropic Fable 5.1 thinking-prefix binding mismatches (classified in `overflow-detection.ts`), the recovery arm strips bound reasoning from the targeted assistant (`stripReasoningFromAssistantIds` in `strip-content.ts`), recording the frozen ID (`binding_mismatch:<id>`) in `merged_reasoning_stripped_ids` for replay stability before clearing the armed target upon live turn completion. Trailing assistant blank decisions (`TrailingBlankDecision`) freeze while a message is newest (on every pass, capturing the last-live shape), becoming immutable once historical (`findTrailingBlankDecisionCandidates` / `applyFrozenTrailingBlankDecisions`), normalizing late-visible empty text blocks on non-newest assistant messages across race directions and cloning message part arrays before length-changing splices to avoid mutating live execution object graphs. Trailing blank classifications snapshot source shapes before sentinel insertion so `keep` decisions never manufacture absent suffixes; poisoned keeps demote to `strip` via CAS on cache-busting passes. The transform wrapper (`src/plugin/messages-transform.ts`) preserves user-terminated prompt tails (`preserveUserTerminatedTail`) when OpenCode concurrently appends a pending assistant shell mid-transform, re-anchoring the user message at the wire tail so the shell cannot turn into an illegal assistant prefill.
- **Message / git-commit indexes**: Maintain an FTS5 raw-message index outside the search hot path (via async reconciliation + live `message.updated` events) and a HEAD-only non-merge git-commit corpus populated by the dream timer. Out-of-band orphan session sweeps (`src/features/magic-context/message-index.ts`) discover candidate sessions by unioning across all harness-scoped `SESSION_SCOPED_TABLES` against OpenCode's authoritative session table. Session deletions record pending cleanups in `pending_session_cleanup` (distinguishing Rust module deletions via `:rust` harness tags), retried on dream timer ticks (`retryPendingSessionCleanups` / `retryPendingRustSessionCleanupsForProject`); `deleteSessionScopedRows` protects session coordinates until module acknowledgement. For directories that are not git repositories or are empty (no commits yet), the sweep coordinator future-dates the last sweep time to park them on a 24-hour re-probe cooldown to avoid error log flooding.
- **Commit-detection utility**: A unified detection helper (`src/shared/commit-detection.ts`) parses git commit hashes (7-12 hex chars) paired with commit-action verbs (commit, cherry-pick, merge, rebase) to detect commit boundaries consistently across the historian trigger, OpenCode note-nudge, and Pi note-nudge.
- **System-prompt injection** (`system-prompt-hash.ts`): injects only the Magic Context guidance text + a frozen `Today's date:` line (per-session sticky, updated only on cache-busting passes). Guidance is concatenated into `output.system[0]` with a blank line separator rather than appended as a second array entry so wire serializers emit a single `system` role message compatible with strict chat templates (Qwen/vLLM/LiteLLM). Guidance is gated by `ctx_reduce` availability (resolved from the tools map of the session's first user message). While the verdict is provisional (before the first user message has been processed/persisted), the guidance block still renders using a fail-open default, but the computed system prompt hash is not written to the database to prevent cache-busting flips when the first user message freezes the verdict. Prompt surfaces resolve presets ("full" vs "light", using `src/shared/prompt-surface.ts` and `src/tools/light-descriptions.ts` for light guidance and tool descriptions under the ratified 1825-token budget ceiling). Adjunct blocks (`<project-docs>` / `<user-profile>`) are NOT here — they moved into m[0]/m[1] so the system prompt stays maximally cache-stable. Skipped entirely for OpenCode's internal `title`/`summary`/`compaction` agents and for hidden child sessions (detected by the `magic-context-` title prefix).
Expand Down
1 change: 1 addition & 0 deletions STRUCTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -149,6 +149,7 @@ Unless specified otherwise, TypeScript paths are relative to `packages/plugin/`
- `src/hooks/magic-context/supersession-reclaim.ts`: Select superseded spent control-plane tool outputs (oldest todowrite, ctx_reduce, zero-value meta calls) and older edit/write calls for the same file under the `smart_drops` configuration flag, derivation anchored in a newest-20 owner floor from persisted tag chronology.
- `src/hooks/magic-context/tool-drop-target.ts`: Candidate selection for tool output reduction, protecting open tool arcs via `partHasCompletedResult` while replacing dropped invocation arguments with inert markers and reclaiming completed or errored arcs.
- `src/hooks/magic-context/dropped-input-guard.ts`: Intercept tool execution before dispatch if arguments contain dropped placeholder strings, replacing dropped inputs with inert non-executable markers (`{ dropped: "[dropped §N§]" }`) and instructing the model to recover original arguments via `ctx_expand`.
- `src/hooks/magic-context/anthropic-wire.ts`: Resolve, once per process from `client.provider.list()`, which `(providerID, modelID)` pairs OpenCode serializes with the `@ai-sdk/anthropic` adapter, so empty-part sentinels are gated on the wire format rather than the provider name.
- `src/hooks/magic-context/sentinel.ts`: Decide provider predicates (such as `modelAcceptsEmptyContent` and `variantChangeBustsProviderCache`, with cache preservation for Anthropic Fable 5.1 and OpenAI GPT-6 Astra) controlling strip/flush behavior.
- `src/hooks/magic-context/tail-hygiene-walk.ts`: Single-walk tail hygiene measurement instrument (`{U, T}` baseline), tracking active vs protected mass, computing baseline deltas, and calculating low-overhead structural size proxy signatures of served messages.
- `src/hooks/magic-context/protected-tail-boundary.ts`: Calculate the eligible head and protected tail boundary, enforce completed tool-arc fencing, admit oversize leading completed components under the head cap, and emit self-describing boundary diagnostics.
Expand Down
48 changes: 48 additions & 0 deletions docs/AUDIT-KNOWN-ISSUES.md
Original file line number Diff line number Diff line change
Expand Up @@ -906,3 +906,51 @@ it. Results carry `status=pending` so the caller can see the condition has not
fired. The same reasoning covers restricted children that carry `ctx_search`
(retrospective): notes are the same sensitivity class as project memories,
which those children already search.

## A55: the Anthropic-wire empty-sentinel capability is TS-mode only (Rust parity deferred)

`modelAcceptsEmptyContent` resolves the empty-sentinel capability from the
adapter OpenCode actually uses (`api.npm === "@ai-sdk/anthropic"`), so a custom
provider fronting Claude — a Bedrock or Vertex gateway under any provider id —
now gets the merged-reasoning strip that its wire requires. See
`hooks/magic-context/anthropic-wire.ts`.

The `mc-module` Rust transform decides the same fact from the provider id, in one
shared predicate — `request_accepts_empty_content`
(`crates/mc-module/src/transform.rs:13753`) — plus about eight inline
`provider_id != Some("anthropic")` comparisons in the neighbouring trailing-blank,
user-terminated-tail, and reasoning-cutoff lanes. Rust mode
(`transform_mode: "rust"`) therefore stays canonical-Anthropic only, and
`rust-mode-transform.ts` passes `modelAcceptsEmptyContent(providerID)` without a
model id on purpose: widening the host lane alone would let the two lanes
disagree about the served bytes, which is worse than the narrow gate.

The parity work is smaller than the comparison count suggests. The module already
folds this capability into its pass digest (`transform.rs:12229`), so it would
handle a change correctly once the resolved boolean rides `TransformRequest`
instead of being re-derived from `provider_id`.

Pi is unaffected. Pi resolves its own capability from `ctx.model.provider` and
never calls the OpenCode strip functions, so there is nothing to widen there
until Pi exposes a comparable adapter fact.

`resolveEmptySentinelCapability` resolves five states per pass. "Folds?" is whether
a change of this component alone can materialize m[0]:

| Pass state | Capability | Recorded as widened | Folds? |
|---|---|---|---|
| Registry resolved, canonical `anthropic` | on | no | never — component absent both sides |
| Registry resolved, custom provider on `@ai-sdk/anthropic` | on | yes | once, when it first differs |
| Registry resolved, any other adapter | off | no | never |
| Registry unresolved, live model matches the cached m[0] model | carried forward | carried forward | never — nothing changed |
| Registry unresolved, live model differs or is unobservable | off | no | never — `model_change` owns that fold |

Two operational consequences. A gateway added to `opencode.json` mid-session needs
an OpenCode restart, because the adapter snapshot is read once per process. And a
failed lookup does not narrow a session that already served widened bytes: it keeps
the carried answer until a retry succeeds, because narrowing would stop replaying
persisted merged-reasoning strips and reinstate the 400.

Closing the gap means carrying one resolved boolean in the transform request and
replacing the module's provider comparisons with it, plus Rust-side coverage. Do
not re-flag the TS/Rust asymmetry as a bug until that work is scheduled.
4 changes: 4 additions & 0 deletions packages/plugin/src/config/latch-permanence-guard.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -166,6 +166,10 @@ const KNOWN_SLOTS: Record<string, KnownSlot> = {
classification: "PUBLICATION",
reason: "Publishes last-known-good model metadata, not a failure verdict; refresh writes a later successful value.",
},
"packages/plugin/src/hooks/magic-context/anthropic-wire.ts:loadPromise": {
classification: "VERDICT",
reason: "Saved: only a successful load is memoized. A failure clears the slot and retries after a 60-second cooldown. resolveEmptySentinelCapability() carries the session's prior answer forward while the registry is unresolved, so it does not narrow.",
},
};

function sourceFiles(directory: string): string[] {
Expand Down
Loading